Release v2.5.2

- Add praisonai/db/ module that was being ignored by *.db pattern
- This fixes ModuleNotFoundError: No module named 'praisonai.db' in CI tests

.gitignore

@@ -80,4 +80,6 @@ CopilotKit*
 issue_details.json
 
 # MCP Registry tokens
-.mcpregistry_*
+.mcpregistry_*
+
+*.log

README.md

@@ -204,6 +204,12 @@ npm install praisonai
 | ↳ Add Custom Knowledge | [Example](examples/python/concepts/knowledge-agents.py) | [📖](https://docs.praison.ai/features/knowledge) |
 | ↳ RAG Agents | [Example](examples/python/concepts/rag-agents.py) | [📖](https://docs.praison.ai/features/rag) |
 | ↳ Chat with PDF Agents | [Example](examples/python/concepts/chat-with-pdf.py) | [📖](https://docs.praison.ai/features/chat-with-pdf) |
+| ↳ Data Readers (PDF, DOCX, etc.) | [CLI](#knowledge-cli) | [📖](https://docs.praison.ai/api/praisonai/knowledge-readers-api) |
+| ↳ Vector Store Selection | [CLI](#knowledge-cli) | [📖](https://docs.praison.ai/api/praisonai/knowledge-vector-store-api) |
+| ↳ Retrieval Strategies | [CLI](#knowledge-cli) | [📖](https://docs.praison.ai/api/praisonai/knowledge-retrieval-api) |
+| ↳ Rerankers | [CLI](#knowledge-cli) | [📖](https://docs.praison.ai/api/praisonai/knowledge-reranker-api) |
+| ↳ Index Types (Vector/Keyword/Hybrid) | [CLI](#knowledge-cli) | [📖](https://docs.praison.ai/api/praisonai/knowledge-index-api) |
+| ↳ Query Engines (Sub-Question, etc.) | [CLI](#knowledge-cli) | [📖](https://docs.praison.ai/api/praisonai/knowledge-query-engine-api) |
 | **🔬 Research & Intelligence** | | |
 | ↳ Deep Research Agents | [Example](examples/python/agents/research-agent.py) | [📖](https://docs.praison.ai/agents/deep-research) |
 | ↳ Query Rewriter Agent | [Example](#5-query-rewriter-agent) | [📖](https://docs.praison.ai/agents/query-rewriter) |
@@ -261,6 +267,10 @@ npm install praisonai
 | ↳ 100+ LLM Support | [Example](examples/python/providers/openai/openai_gpt4_example.py) | [📖](https://docs.praison.ai/models) |
 | ↳ Callback Agents | [Example](examples/python/general/advanced-callback-systems.py) | [📖](https://docs.praison.ai/features/callbacks) |
 | ↳ Hooks | [Example](#9-hooks) | [📖](https://docs.praison.ai/features/hooks) |
+| ↳ Middleware System | [Example](examples/middleware/basic_middleware.py) | [📖](https://docs.praison.ai/features/middleware) |
+| ↳ Configurable Model | [Example](examples/middleware/configurable_model.py) | [📖](https://docs.praison.ai/features/configurable-model) |
+| ↳ Rate Limiter | [Example](examples/middleware/rate_limiter.py) | [📖](https://docs.praison.ai/features/rate-limiter) |
+| ↳ Injected Tool State | [Example](examples/middleware/injected_state.py) | [📖](https://docs.praison.ai/features/injected-state) |
 | ↳ Shadow Git Checkpoints | [Example](#10-shadow-git-checkpoints) | [📖](https://docs.praison.ai/features/checkpoints) |
 | ↳ Background Tasks | [Example](examples/background/basic_background.py) | [📖](https://docs.praison.ai/features/background-tasks) |
 | ↳ Policy Engine | [Example](examples/policy/basic_policy.py) | [📖](https://docs.praison.ai/features/policy-engine) |
@@ -2637,6 +2647,84 @@ PraisonAI provides zero-dependency persistent memory for agents. For detailed ex
 
 ---
 
+## 📚 Knowledge & Retrieval (RAG)
+
+PraisonAI provides a complete knowledge stack for building RAG applications with multiple vector stores, retrieval strategies, rerankers, and query modes.
+
+### Knowledge CLI Commands
+
+| Command | Description |
+|---------|-------------|
+| `praisonai knowledge add <file\|dir\|url>` | Add documents to knowledge base |
+| `praisonai knowledge query <question>` | Query knowledge base with RAG |
+| `praisonai knowledge list` | List indexed documents |
+| `praisonai knowledge clear` | Clear knowledge base |
+| `praisonai knowledge stats` | Show knowledge base statistics |
+
+### Knowledge CLI Options
+
+| Option | Values | Description |
+|--------|--------|-------------|
+| `--vector-store` | `memory`, `chroma`, `pinecone`, `qdrant`, `weaviate` | Vector store backend |
+| `--retrieval` | `basic`, `fusion`, `recursive`, `auto_merge` | Retrieval strategy |
+| `--reranker` | `simple`, `llm`, `cross_encoder`, `cohere` | Reranking method |
+| `--index-type` | `vector`, `keyword`, `hybrid` | Index type |
+| `--query-mode` | `default`, `sub_question`, `summarize` | Query mode |
+
+### Knowledge CLI Examples
+
+```bash
+# Add documents
+praisonai knowledge add ./docs/
+praisonai knowledge add https://example.com/page.html
+praisonai knowledge add "*.pdf"
+
+# Query with advanced options
+praisonai knowledge query "How to authenticate?" --retrieval fusion --reranker llm
+
+# Full advanced query
+praisonai knowledge query "authentication flow" \
+  --vector-store chroma \
+  --retrieval fusion \
+  --reranker llm \
+  --index-type hybrid \
+  --query-mode sub_question
+```
+
+### Knowledge SDK Usage
+
+```python
+from praisonaiagents import Agent, Knowledge
+
+# Simple usage with Agent
+agent = Agent(
+    name="Research Assistant",
+    knowledge=["docs/manual.pdf", "data/faq.txt"],
+    knowledge_config={
+        "vector_store": {"provider": "chroma"}
+    }
+)
+response = agent.chat("How do I authenticate?")
+
+# Direct Knowledge usage
+knowledge = Knowledge()
+knowledge.add("document.pdf")
+results = knowledge.search("authentication", limit=5)
+```
+
+### Knowledge Stack Features Table
+
+| Feature | Description | SDK Docs | CLI Docs |
+|---------|-------------|----------|----------|
+| **Data Readers** | Load PDF, Markdown, Text, HTML, URLs | [SDK](/docs/sdk/praisonaiagents/knowledge/protocols) | [CLI](/docs/cli/knowledge) |
+| **Vector Stores** | ChromaDB, Pinecone, Qdrant, Weaviate, In-Memory | [SDK](/docs/sdk/praisonaiagents/knowledge/protocols) | [CLI](/docs/cli/knowledge) |
+| **Retrieval Strategies** | Basic, Fusion (RRF), Recursive, Auto-Merge | [SDK](/docs/sdk/praisonaiagents/knowledge/protocols) | [CLI](/docs/cli/knowledge) |
+| **Rerankers** | Simple, LLM, Cross-Encoder, Cohere | [SDK](/docs/sdk/praisonaiagents/knowledge/protocols) | [CLI](/docs/cli/knowledge) |
+| **Index Types** | Vector, Keyword (BM25), Hybrid | [SDK](/docs/sdk/praisonaiagents/knowledge/protocols) | [CLI](/docs/cli/knowledge) |
+| **Query Engines** | Default, Sub-Question, Summarize | [SDK](/docs/sdk/praisonaiagents/knowledge/protocols) | [CLI](/docs/cli/knowledge) |
+
+---
+
 ## 🔬 Advanced Features
 
 ### Research & Intelligence
@@ -2751,6 +2839,44 @@ agent.chat("Hello!")  # Auto-persists messages, runs, traces
 | `praisonai persistence migrate` | Apply schema migrations |
 | `praisonai persistence status` | Show schema status |
 
+### Knowledge CLI Commands {#knowledge-cli}
+
+| Command | Description |
+|---------|-------------|
+| `praisonai knowledge add <source>` | Add file, directory, URL, or glob pattern |
+| `praisonai knowledge query "<question>"` | Query knowledge base with RAG |
+| `praisonai knowledge list` | List indexed documents |
+| `praisonai knowledge clear` | Clear knowledge base |
+| `praisonai knowledge stats` | Show knowledge base statistics |
+
+**Knowledge Query Flags:**
+
+| Flag | Values | Default |
+|------|--------|---------|
+| `--vector-store` | `memory`, `chroma`, `pinecone`, `qdrant`, `weaviate` | `chroma` |
+| `--retrieval-strategy` | `basic`, `fusion`, `recursive`, `auto_merge` | `basic` |
+| `--reranker` | `none`, `simple`, `llm`, `cross_encoder`, `cohere` | `none` |
+| `--index-type` | `vector`, `keyword`, `hybrid` | `vector` |
+| `--query-mode` | `default`, `sub_question`, `summarize` | `default` |
+| `--workspace` | Path to workspace directory | Current dir |
+| `--session` | Session ID for persistence | - |
+
+**Examples:**
+
+```bash
+# Add documents
+praisonai knowledge add document.pdf
+praisonai knowledge add ./docs/
+praisonai knowledge add "*.md"
+
+# Query with options
+praisonai knowledge query "How to authenticate?" \
+  --vector-store chroma \
+  --retrieval-strategy fusion \
+  --reranker simple \
+  --query-mode sub_question
+```
+
 ### Databases Table
 
 | Database | Store Type | Install | Example | Docs |
@@ -2763,11 +2889,11 @@ agent.chat("Hello!")  # Auto-persists messages, runs, traces
 | SurrealDB | Conversation | `pip install "praisonai[tools]"` | - | [docs](https://docs.praison.ai/docs/databases/overview) |
 | Qdrant | Knowledge | `pip install "praisonai[tools]"` | [knowledge_qdrant.py](examples/persistence/knowledge_qdrant.py) | [docs](https://docs.praison.ai/docs/databases/qdrant) |
 | ChromaDB | Knowledge | `pip install "praisonai[tools]"` | - | [docs](https://docs.praison.ai/docs/databases/overview) |
-| Pinecone | Knowledge | `pip install "praisonai[tools]"` | - | [docs](https://docs.praison.ai/docs/databases/overview) |
-| Weaviate | Knowledge | `pip install "praisonai[tools]"` | - | [docs](https://docs.praison.ai/docs/databases/overview) |
-| LanceDB | Knowledge | `pip install "praisonai[tools]"` | - | [docs](https://docs.praison.ai/docs/databases/overview) |
+| Pinecone | Knowledge | `pip install pinecone` | [pinecone_wow.py](examples/vector/pinecone_wow.py) | [docs](https://docs.praison.ai/docs/databases/pinecone) |
+| Weaviate | Knowledge | `pip install weaviate-client` | [weaviate_wow.py](examples/vector/weaviate_wow.py) | [docs](https://docs.praison.ai/docs/databases/weaviate) |
+| LanceDB | Knowledge | `pip install lancedb` | [lancedb_real_wow.py](examples/vector/lancedb_real_wow.py) | [docs](https://docs.praison.ai/docs/databases/lancedb) |
 | Milvus | Knowledge | `pip install "praisonai[tools]"` | - | [docs](https://docs.praison.ai/docs/databases/overview) |
-| PGVector | Knowledge | `pip install "praisonai[tools]"` | - | [docs](https://docs.praison.ai/docs/databases/overview) |
+| PGVector | Knowledge | `pip install psycopg2-binary` | [pgvector_real_wow.py](examples/vector/pgvector_real_wow.py) | [docs](https://docs.praison.ai/docs/databases/pgvector) |
 | Redis Vector | Knowledge | `pip install "praisonai[tools]"` | - | [docs](https://docs.praison.ai/docs/databases/overview) |
 | Cassandra | Knowledge | `pip install "praisonai[tools]"` | - | [docs](https://docs.praison.ai/docs/databases/overview) |
 | ClickHouse | Knowledge | `pip install "praisonai[tools]"` | - | [docs](https://docs.praison.ai/docs/databases/overview) |

docker/Dockerfile.chat

@@ -16,7 +16,7 @@ RUN mkdir -p /root/.praison
 # Install Python packages (using latest versions)
 RUN pip install --no-cache-dir \
     praisonai_tools \
-    "praisonai>=2.4.2" \
+    "praisonai>=2.5.2" \
     "praisonai[chat]" \
     "embedchain[github,youtube]"
 

docker/Dockerfile.dev

@@ -20,7 +20,7 @@ RUN mkdir -p /root/.praison
 # Install Python packages (using latest versions)
 RUN pip install --no-cache-dir \
     praisonai_tools \
-    "praisonai>=2.4.2" \
+    "praisonai>=2.5.2" \
     "praisonai[ui]" \
     "praisonai[chat]" \
     "praisonai[realtime]" \

docker/Dockerfile.ui

@@ -16,7 +16,7 @@ RUN mkdir -p /root/.praison
 # Install Python packages (using latest versions)
 RUN pip install --no-cache-dir \
     praisonai_tools \
-    "praisonai>=2.4.2" \
+    "praisonai>=2.5.2" \
     "praisonai[ui]" \
     "praisonai[crewai]"
 

examples/async_runs/README.py

@@ -0,0 +1,35 @@
+"""
+Async Runs Examples
+===================
+
+This directory contains examples for the PraisonAI Async Jobs API.
+
+Examples:
+---------
+1. sdk_submit_poll_result.py - Python SDK example for submit/poll/result
+2. cli_submit_poll_result.sh - Bash/curl example for submit/poll/result
+3. idempotency.sh - Demonstrates idempotency key usage
+4. cancel.sh - Demonstrates job cancellation
+5. sse_stream.sh - Demonstrates SSE streaming
+
+Prerequisites:
+--------------
+1. Set OPENAI_API_KEY environment variable
+2. Start the jobs server:
+   
+   cd /path/to/praisonai-package/src/praisonai
+   python -m uvicorn praisonai.jobs.server:create_app --port 8005 --factory
+
+3. Run examples:
+   
+   python sdk_submit_poll_result.py
+   bash cli_submit_poll_result.sh
+   bash idempotency.sh
+   bash cancel.sh
+   bash sse_stream.sh
+
+Environment Variables:
+----------------------
+- OPENAI_API_KEY: Required for agent execution
+- PRAISONAI_BASE_URL: API server URL (default: http://127.0.0.1:8005)
+"""

examples/async_runs/cancel.sh

@@ -0,0 +1,54 @@
+#!/bin/bash
+# CLI Example: Cancel a Running Job
+#
+# This example demonstrates how to cancel a running job.
+#
+# Requirements:
+# - Jobs server running on http://127.0.0.1:8005
+#
+# Usage:
+#     bash cancel.sh
+
+set -e
+
+API_URL="${PRAISONAI_BASE_URL:-http://127.0.0.1:8005}"
+
+echo "============================================================"
+echo "PraisonAI Job Cancellation Example"
+echo "============================================================"
+
+# Submit a long-running job
+echo ""
+echo "1. Submitting a job..."
+RESPONSE=$(curl -s -X POST "$API_URL/api/v1/runs" \
+    -H "Content-Type: application/json" \
+    -d '{"prompt": "Write a detailed essay about the history of computing."}')
+
+JOB_ID=$(echo "$RESPONSE" | python3 -c "import sys,json; print(json.load(sys.stdin)['job_id'])")
+echo "   Job ID: $JOB_ID"
+
+# Wait briefly
+echo ""
+echo "2. Waiting 1 second..."
+sleep 1
+
+# Cancel the job
+echo ""
+echo "3. Cancelling job..."
+CANCEL_RESPONSE=$(curl -s -X POST "$API_URL/api/v1/runs/$JOB_ID/cancel")
+STATUS=$(echo "$CANCEL_RESPONSE" | python3 -c "import sys,json; print(json.load(sys.stdin)['status'])")
+echo "   Status after cancel: $STATUS"
+
+# Verify cancellation
+echo ""
+echo "4. Verification:"
+if [ "$STATUS" = "cancelled" ]; then
+    echo "   ✓ SUCCESS: Job was cancelled"
+else
+    echo "   ℹ Job status: $STATUS (may have completed before cancel)"
+fi
+
+echo ""
+echo "============================================================"
+echo "Cancellation example completed!"
+echo "============================================================"

examples/async_runs/cli_submit_poll_result.sh

@@ -0,0 +1,75 @@
+#!/bin/bash
+# CLI Example: Submit, Poll, and Get Result for Async Jobs
+#
+# This example demonstrates how to use the PraisonAI CLI to:
+# 1. Submit an async job
+# 2. Check job status
+# 3. Get the result
+#
+# Requirements:
+# - OPENAI_API_KEY environment variable set
+# - Jobs server running on http://127.0.0.1:8005
+#
+# Usage:
+#     bash cli_submit_poll_result.sh
+
+set -e
+
+API_URL="${PRAISONAI_BASE_URL:-http://127.0.0.1:8005}"
+
+echo "============================================================"
+echo "PraisonAI Async Jobs CLI Example"
+echo "============================================================"
+
+# Check if server is running
+echo ""
+echo "Checking server health..."
+if ! curl -s "$API_URL/health" > /dev/null 2>&1; then
+    echo "✗ Server not available at $API_URL"
+    echo "  Start the server with: python -m uvicorn praisonai.jobs.server:create_app --port 8005 --factory"
+    exit 1
+fi
+echo "✓ Server is healthy"
+
+# 1. Submit a job
+echo ""
+echo "1. Submitting job..."
+RESPONSE=$(curl -s -X POST "$API_URL/api/v1/runs" \
+    -H "Content-Type: application/json" \
+    -d '{"prompt": "What is 2+2? Answer with just the number."}')
+
+JOB_ID=$(echo "$RESPONSE" | python3 -c "import sys,json; print(json.load(sys.stdin)['job_id'])")
+echo "   Job ID: $JOB_ID"
+
+# 2. Poll for status
+echo ""
+echo "2. Polling for status..."
+while true; do
+    STATUS_RESPONSE=$(curl -s "$API_URL/api/v1/runs/$JOB_ID")
+    STATUS=$(echo "$STATUS_RESPONSE" | python3 -c "import sys,json; print(json.load(sys.stdin)['status'])")
+    PROGRESS=$(echo "$STATUS_RESPONSE" | python3 -c "import sys,json; print(json.load(sys.stdin)['progress']['percentage'])")
+    
+    echo "   Status: $STATUS | Progress: ${PROGRESS}%"
+    
+    if [ "$STATUS" = "succeeded" ] || [ "$STATUS" = "failed" ] || [ "$STATUS" = "cancelled" ]; then
+        break
+    fi
+    
+    sleep 2
+done
+
+# 3. Get result
+echo ""
+echo "3. Getting result..."
+if [ "$STATUS" = "succeeded" ]; then
+    RESULT=$(curl -s "$API_URL/api/v1/runs/$JOB_ID/result")
+    echo "   Result: $(echo "$RESULT" | python3 -c "import sys,json; print(json.load(sys.stdin)['result'])")"
+    echo "   Duration: $(echo "$RESULT" | python3 -c "import sys,json; print(f\"{json.load(sys.stdin)['duration_seconds']:.2f}s\")")"
+else
+    echo "   Job did not succeed: $STATUS"
+fi
+
+echo ""
+echo "============================================================"
+echo "Example completed!"
+echo "============================================================"

examples/async_runs/idempotency.sh

@@ -0,0 +1,57 @@
+#!/bin/bash
+# CLI Example: Idempotency Key Usage
+#
+# This example demonstrates how idempotency keys prevent duplicate job submissions.
+# Submitting the same request with the same idempotency key returns the existing job.
+#
+# Requirements:
+# - Jobs server running on http://127.0.0.1:8005
+#
+# Usage:
+#     bash idempotency.sh
+
+set -e
+
+API_URL="${PRAISONAI_BASE_URL:-http://127.0.0.1:8005}"
+IDEM_KEY="example-idem-key-$(date +%s)"
+
+echo "============================================================"
+echo "PraisonAI Idempotency Key Example"
+echo "============================================================"
+
+# First submission
+echo ""
+echo "1. First submission with Idempotency-Key: $IDEM_KEY"
+RESPONSE1=$(curl -s -X POST "$API_URL/api/v1/runs" \
+    -H "Content-Type: application/json" \
+    -H "Idempotency-Key: $IDEM_KEY" \
+    -d '{"prompt": "What is 1+1?"}')
+
+JOB_ID1=$(echo "$RESPONSE1" | python3 -c "import sys,json; print(json.load(sys.stdin)['job_id'])")
+echo "   Job ID: $JOB_ID1"
+
+# Second submission with same key
+echo ""
+echo "2. Second submission with SAME Idempotency-Key: $IDEM_KEY"
+RESPONSE2=$(curl -s -X POST "$API_URL/api/v1/runs" \
+    -H "Content-Type: application/json" \
+    -H "Idempotency-Key: $IDEM_KEY" \
+    -d '{"prompt": "Different prompt but same key"}')
+
+JOB_ID2=$(echo "$RESPONSE2" | python3 -c "import sys,json; print(json.load(sys.stdin)['job_id'])")
+echo "   Job ID: $JOB_ID2"
+
+# Verify same job ID
+echo ""
+echo "3. Verification:"
+if [ "$JOB_ID1" = "$JOB_ID2" ]; then
+    echo "   ✓ SUCCESS: Same job ID returned (no duplicate created)"
+else
+    echo "   ✗ FAILURE: Different job IDs returned"
+    exit 1
+fi
+
+echo ""
+echo "============================================================"
+echo "Idempotency example completed!"
+echo "============================================================"

examples/async_runs/sdk_submit_poll_result.py

@@ -0,0 +1,121 @@
+#!/usr/bin/env python3
+"""
+SDK Example: Submit, Poll, and Get Result for Async Jobs
+
+This example demonstrates how to use the PraisonAI Jobs SDK to:
+1. Submit an async job
+2. Poll for status
+3. Get the result
+
+Requirements:
+- OPENAI_API_KEY environment variable set
+- Jobs server running on http://127.0.0.1:8005
+
+Usage:
+    python sdk_submit_poll_result.py
+"""
+
+import os
+import time
+import httpx
+
+# Configuration
+API_URL = os.getenv("PRAISONAI_BASE_URL", "http://127.0.0.1:8005")
+POLL_INTERVAL = 2  # seconds
+
+
+def submit_job(prompt: str, idempotency_key: str = None) -> dict:
+    """Submit a job and return the response."""
+    headers = {}
+    if idempotency_key:
+        headers["Idempotency-Key"] = idempotency_key
+    
+    with httpx.Client(timeout=30.0) as client:
+        response = client.post(
+            f"{API_URL}/api/v1/runs",
+            json={"prompt": prompt},
+            headers=headers if headers else None
+        )
+        response.raise_for_status()
+        return response.json()
+
+
+def get_status(job_id: str) -> dict:
+    """Get job status."""
+    with httpx.Client(timeout=30.0) as client:
+        response = client.get(f"{API_URL}/api/v1/runs/{job_id}")
+        response.raise_for_status()
+        return response.json()
+
+
+def get_result(job_id: str) -> dict:
+    """Get job result."""
+    with httpx.Client(timeout=30.0) as client:
+        response = client.get(f"{API_URL}/api/v1/runs/{job_id}/result")
+        response.raise_for_status()
+        return response.json()
+
+
+def wait_for_completion(job_id: str, max_wait: int = 120) -> dict:
+    """Poll until job completes."""
+    start = time.time()
+    while time.time() - start < max_wait:
+        status = get_status(job_id)
+        print(f"  Status: {status['status']} | Progress: {status['progress']['percentage']:.0f}%")
+        
+        if status["status"] in ("succeeded", "failed", "cancelled"):
+            return status
+        
+        # Honor retry_after if present
+        wait_time = status.get("retry_after") or POLL_INTERVAL
+        time.sleep(wait_time)
+    
+    raise TimeoutError(f"Job {job_id} did not complete within {max_wait}s")
+
+
+def main():
+    print("=" * 60)
+    print("PraisonAI Async Jobs SDK Example")
+    print("=" * 60)
+    
+    # Check if server is running
+    try:
+        with httpx.Client(timeout=5.0) as client:
+            health = client.get(f"{API_URL}/health")
+            health.raise_for_status()
+            print(f"✓ Server healthy: {health.json()['status']}")
+    except Exception as e:
+        print(f"✗ Server not available at {API_URL}: {e}")
+        print("  Start the server with: python -m uvicorn praisonai.jobs.server:create_app --port 8005 --factory")
+        return
+    
+    # 1. Submit a job
+    print("\n1. Submitting job...")
+    prompt = "What is the capital of France? Answer in one word."
+    result = submit_job(prompt)
+    job_id = result["job_id"]
+    print(f"   Job ID: {job_id}")
+    print(f"   Status: {result['status']}")
+    print(f"   Poll URL: {result['poll_url']}")
+    
+    # 2. Wait for completion
+    print("\n2. Waiting for completion...")
+    final_status = wait_for_completion(job_id)
+    print(f"   Final status: {final_status['status']}")
+    
+    # 3. Get result
+    if final_status["status"] == "succeeded":
+        print("\n3. Getting result...")
+        result = get_result(job_id)
+        print(f"   Result: {result['result']}")
+        print(f"   Duration: {result['duration_seconds']:.2f}s")
+    else:
+        print(f"\n3. Job did not succeed: {final_status.get('error', 'Unknown error')}")
+    
+    print("\n" + "=" * 60)
+    print("Example completed successfully!")
+    print("=" * 60)
+
+
+if __name__ == "__main__":
+    main()

examples/async_runs/sse_stream.sh

@@ -0,0 +1,51 @@
+#!/bin/bash
+# CLI Example: SSE Streaming
+#
+# This example demonstrates how to stream job progress via Server-Sent Events.
+#
+# Requirements:
+# - Jobs server running on http://127.0.0.1:8005
+#
+# Usage:
+#     bash sse_stream.sh
+
+set -e
+
+API_URL="${PRAISONAI_BASE_URL:-http://127.0.0.1:8005}"
+
+echo "============================================================"
+echo "PraisonAI SSE Streaming Example"
+echo "============================================================"
+
+# Submit a job
+echo ""
+echo "1. Submitting job..."
+RESPONSE=$(curl -s -X POST "$API_URL/api/v1/runs" \
+    -H "Content-Type: application/json" \
+    -d '{"prompt": "Count from 1 to 5."}')
+
+JOB_ID=$(echo "$RESPONSE" | python3 -c "import sys,json; print(json.load(sys.stdin)['job_id'])")
+echo "   Job ID: $JOB_ID"
+
+# Stream progress
+echo ""
+echo "2. Streaming progress (Ctrl+C to stop)..."
+echo "   Connecting to: $API_URL/api/v1/runs/$JOB_ID/stream"
+echo ""
+
+# Use curl to stream SSE events (timeout after 60s)
+curl -s -N --max-time 60 "$API_URL/api/v1/runs/$JOB_ID/stream" | while read -r line; do
+    if [[ "$line" == data:* ]]; then
+        data="${line#data: }"
+        if [ "$data" = "[DONE]" ]; then
+            echo "   Stream completed"
+            break
+        fi
+        echo "   Event: $data"
+    fi
+done
+
+echo ""
+echo "============================================================"
+echo "Streaming example completed!"
+echo "============================================================"

examples/middleware/basic_middleware.py

@@ -0,0 +1,55 @@
+"""
+Basic Middleware Example - PraisonAI Agents
+
+Demonstrates before/after hooks and wrap decorators for model and tool calls.
+"""
+
+from praisonaiagents import Agent, tool
+from praisonaiagents.hooks import (
+    before_model, after_model, wrap_model_call,
+    before_tool, after_tool, wrap_tool_call,
+    InvocationContext, ModelRequest, ModelResponse
+)
+
+# Simple tool for demonstration
+@tool
+def get_weather(city: str) -> str:
+    """Get weather for a city."""
+    return f"Sunny, 22°C in {city}"
+
+# Before model hook - adds context
+@before_model
+def add_context(request: ModelRequest) -> ModelRequest:
+    print(f"[before_model] Adding context to request")
+    return request
+
+# After model hook - logs response
+@after_model  
+def log_response(response: ModelResponse) -> ModelResponse:
+    print(f"[after_model] Response received")
+    return response
+
+# Wrap tool call - retry on error
+@wrap_tool_call
+def retry_on_error(tool_call, call_next):
+    print(f"[wrap_tool_call] Executing tool")
+    try:
+        return call_next(tool_call)
+    except Exception as e:
+        print(f"[wrap_tool_call] Retrying after error: {e}")
+        return call_next(tool_call)
+
+# Create agent with hooks
+agent = Agent(
+    name="WeatherBot",
+    instructions="You help with weather queries.",
+    tools=[get_weather],
+    hooks=[add_context, log_response, retry_on_error]
+)
+
+if __name__ == "__main__":
+    # Test the tool directly
+    result = get_weather("London")
+    print(f"Weather: {result}")
+    
+    print("\n✓ Middleware example complete")

examples/middleware/configurable_model.py

@@ -0,0 +1,31 @@
+"""
+Configurable Model Example - PraisonAI Agents
+
+Demonstrates runtime model switching without recreating the agent.
+"""
+
+from praisonaiagents import Agent
+
+# Create agent with configurable model enabled
+agent = Agent(
+    name="FlexBot",
+    instructions="You are a helpful assistant.",
+    llm="gpt-4o-mini",
+    llm_config={"configurable": True}
+)
+
+if __name__ == "__main__":
+    # Default model call
+    print("Using default model (gpt-4o-mini)...")
+    # response = agent.chat("Say hello in 5 words")
+    
+    # Override model per-call
+    print("\nOverriding to different model...")
+    # response = agent.chat("Say hello in 5 words", config={"model": "gpt-4o"})
+    
+    # Override temperature
+    print("\nOverriding temperature...")
+    # response = agent.chat("Say hello creatively", config={"temperature": 0.9})
+    
+    print("\n✓ Configurable model example complete")
+    print("Note: Uncomment agent.chat() calls to run with API key")

examples/middleware/injected_state.py

@@ -0,0 +1,50 @@
+"""
+Injected State Example - PraisonAI Agents
+
+Demonstrates injecting agent state into tools without exposing it in the schema.
+"""
+
+from praisonaiagents import Agent, tool
+from praisonaiagents.tools import Injected
+from praisonaiagents.tools.injected import AgentState, with_injection_context
+
+# Tool with injected state - state param is NOT in the public schema
+@tool
+def show_context(query: str, state: Injected[dict]) -> str:
+    """Show the current agent context."""
+    session_id = state.get('session_id', 'unknown')
+    agent_id = state.get('agent_id', 'unknown')
+    return f"Query: {query}, Session: {session_id}, Agent: {agent_id}"
+
+# Create agent with the tool
+agent = Agent(
+    name="ContextBot",
+    instructions="You help show context information.",
+    tools=[show_context],
+    session_id="my-session-123"
+)
+
+if __name__ == "__main__":
+    # Verify injected param is not in schema
+    schema = show_context.get_schema()
+    params = schema['function']['parameters']['properties']
+    print(f"Schema params: {list(params.keys())}")
+    assert 'state' not in params, "state should NOT be in schema"
+    print("✓ 'state' correctly excluded from schema")
+    
+    # Test with manual injection context
+    mock_state = AgentState(
+        agent_id="test-agent",
+        run_id="run-1",
+        session_id="session-abc"
+    )
+    
+    with with_injection_context(mock_state):
+        result = show_context(query="hello")
+        print(f"Result: {result}")
+    
+    # Test via agent.execute_tool
+    result = agent.execute_tool("show_context", {"query": "test"})
+    print(f"Agent result: {result}")
+    
+    print("\n✓ Injected state example complete")

examples/middleware/rate_limiter.py

@@ -0,0 +1,29 @@
+"""
+Rate Limiter Example - PraisonAI Agents
+
+Demonstrates token bucket rate limiting for LLM API calls.
+"""
+
+from praisonaiagents import Agent
+from praisonaiagents.llm import RateLimiter
+
+# Create rate limiter: 60 requests per minute
+limiter = RateLimiter(requests_per_minute=60, burst=5)
+
+# Create agent with rate limiter
+agent = Agent(
+    name="RateLimitedBot",
+    instructions="You are a helpful assistant.",
+    rate_limiter=limiter
+)
+
+if __name__ == "__main__":
+    print(f"Rate limiter: {limiter}")
+    print(f"Available tokens: {limiter.available_tokens}")
+    
+    # Demonstrate rate limiting
+    for i in range(3):
+        limiter.acquire()
+        print(f"Request {i+1} acquired, tokens left: {limiter.available_tokens:.1f}")
+    
+    print("\n✓ Rate limiter example complete")

examples/multi_agent/multi_agent_persistence.py

@@ -0,0 +1,122 @@
+"""
+Multi-Agent Persistence Example
+
+Demonstrates how multiple agents can share or isolate conversation
+history using the persistence layer.
+"""
+
+from praisonai.persistence.factory import create_conversation_store, create_state_store
+from praisonai.persistence.conversation.base import ConversationSession, ConversationMessage
+import uuid
+
+
+def main():
+    # Create stores
+    conv_store = create_conversation_store("sqlite", path=":memory:")
+    state_store = create_state_store("memory")
+    
+    print("=== Multi-Agent Persistence Demo ===\n")
+    
+    # Scenario 1: Shared session (agents collaborate)
+    print("--- Scenario 1: Shared Session ---")
+    shared_session_id = f"shared-{uuid.uuid4().hex[:8]}"
+    
+    session = ConversationSession(
+        session_id=shared_session_id,
+        user_id="user-123",
+        name="Shared Conversation"
+    )
+    conv_store.create_session(session)
+    
+    # Agent 1 (Researcher) adds context
+    conv_store.add_message(shared_session_id, ConversationMessage(
+        role="assistant",
+        content="[Researcher] I found relevant information about the topic.",
+        metadata={"agent_id": "researcher", "agent_role": "research"}
+    ))
+    
+    # Agent 2 (Writer) uses the context
+    conv_store.add_message(shared_session_id, ConversationMessage(
+        role="assistant", 
+        content="[Writer] Based on the research, here's a summary...",
+        metadata={"agent_id": "writer", "agent_role": "writing"}
+    ))
+    
+    # Agent 3 (Editor) reviews
+    conv_store.add_message(shared_session_id, ConversationMessage(
+        role="assistant",
+        content="[Editor] I've reviewed and polished the content.",
+        metadata={"agent_id": "editor", "agent_role": "editing"}
+    ))
+    
+    messages = conv_store.get_messages(shared_session_id)
+    print(f"Shared session has {len(messages)} messages from different agents")
+    for msg in messages:
+        agent = msg.metadata.get("agent_id", "unknown") if msg.metadata else "unknown"
+        print(f"  - {agent}: {msg.content[:50]}...")
+    
+    # Scenario 2: Isolated sessions (agents work independently)
+    print("\n--- Scenario 2: Isolated Sessions ---")
+    
+    agent_sessions = {}
+    for agent_name in ["planner", "executor", "validator"]:
+        session_id = f"{agent_name}-{uuid.uuid4().hex[:8]}"
+        agent_sessions[agent_name] = session_id
+        
+        conv_store.create_session(ConversationSession(
+            session_id=session_id,
+            agent_id=agent_name,
+            name=f"{agent_name.title()} Session"
+        ))
+        
+        # Each agent has its own conversation
+        conv_store.add_message(session_id, ConversationMessage(
+            role="user",
+            content=f"Task for {agent_name}"
+        ))
+        conv_store.add_message(session_id, ConversationMessage(
+            role="assistant",
+            content=f"[{agent_name.title()}] Processing task..."
+        ))
+    
+    print("Created isolated sessions for each agent:")
+    for agent, sid in agent_sessions.items():
+        msgs = conv_store.get_messages(sid)
+        print(f"  - {agent}: {len(msgs)} messages")
+    
+    # Scenario 3: Shared state across agents
+    print("\n--- Scenario 3: Shared State ---")
+    
+    # Agents share state for coordination
+    state_store.set("workflow:status", {"phase": "research", "progress": 0})
+    state_store.set("workflow:context", {"topic": "AI Agents", "deadline": "2024-12-31"})
+    
+    # Agent 1 updates progress
+    state = state_store.get("workflow:status")
+    state["progress"] = 33
+    state["phase"] = "writing"
+    state_store.set("workflow:status", state)
+    print(f"Agent 1 updated state: {state}")
+    
+    # Agent 2 reads and updates
+    state = state_store.get("workflow:status")
+    state["progress"] = 66
+    state["phase"] = "editing"
+    state_store.set("workflow:status", state)
+    print(f"Agent 2 updated state: {state}")
+    
+    # Agent 3 completes
+    state = state_store.get("workflow:status")
+    state["progress"] = 100
+    state["phase"] = "complete"
+    state_store.set("workflow:status", state)
+    print(f"Agent 3 completed: {state}")
+    
+    # Cleanup
+    conv_store.close()
+    state_store.close()
+    print("\n✅ Multi-agent persistence demo complete!")
+
+
+if __name__ == "__main__":
+    main()

examples/multi_agent/shared_db_session.py

@@ -0,0 +1,40 @@
+"""Multi-Agent Shared Session - DB Persistence Test"""
+import sys
+sys.path.insert(0, 'src/praisonai')
+from praisonai.persistence import create_conversation_store
+from praisonai.persistence.conversation.base import ConversationSession, ConversationMessage
+
+# Create shared session for multiple agents
+store = create_conversation_store("sqlite", path="/tmp/multi_agent.db")
+session_id = "shared-session-123"
+
+# Create session
+session = ConversationSession(session_id=session_id, agent_id="multi")
+try:
+    store.create_session(session)
+except Exception:
+    pass
+
+# Agent 1 writes
+msg1 = ConversationMessage(session_id=session_id, role="user", content="Agent1: Hello")
+store.add_message(session_id, msg1)
+msg2 = ConversationMessage(session_id=session_id, role="assistant", content="Agent1: Hi there!")
+store.add_message(session_id, msg2)
+
+# Agent 2 writes to same session
+msg3 = ConversationMessage(session_id=session_id, role="user", content="Agent2: What did Agent1 say?")
+store.add_message(session_id, msg3)
+msg4 = ConversationMessage(session_id=session_id, role="assistant", content="Agent2: Agent1 said Hello")
+store.add_message(session_id, msg4)
+
+# Verify both agents' messages are in session
+messages = store.get_messages(session_id)
+print(f"Total messages in shared session: {len(messages)}")
+assert len(messages) >= 4, "Should have messages from both agents"
+
+# Verify content
+contents = [m.content for m in messages]
+assert any("Agent1" in c for c in contents), "Should have Agent1 messages"
+assert any("Agent2" in c for c in contents), "Should have Agent2 messages"
+
+print("PASSED: Multi-agent shared session persistence")

examples/multi_agent/shared_session_wow.py

@@ -0,0 +1,16 @@
+"""Multi-Agent Shared Session Demo"""
+from praisonaiagents import Agent
+from praisonai.persistence import create_conversation_store
+
+store = create_conversation_store("sqlite", path="/tmp/multi_agent.db")
+
+agent1 = Agent(name="Researcher", llm="gpt-4o-mini", db=store, session_id="shared-session")
+agent2 = Agent(name="Writer", llm="gpt-4o-mini", db=store, session_id="shared-session")
+
+r1 = agent1.chat("Research topic: AI in healthcare. Key finding: reduces diagnosis time by 50%")
+print(f"Researcher: {r1[:50]}...")
+
+r2 = agent2.chat("Based on the research, what was the key finding?")
+print(f"Writer: {r2[:50]}...")
+assert "50" in r2 or "diagnosis" in r2.lower(), "Writer should see Researcher's context"
+print("PASSED: Multi-agent shared session works")

examples/observability/basic_usage.py

@@ -0,0 +1,48 @@
+"""
+Basic Observability Usage Example
+
+This example demonstrates how to use the observability module
+with PraisonAI agents.
+
+Usage:
+    # Set your provider's API key first
+    export LANGFUSE_PUBLIC_KEY=pk-lf-xxx
+    export LANGFUSE_SECRET_KEY=sk-lf-xxx
+    
+    python basic_usage.py
+"""
+
+from praisonai_tools.observability import obs
+from praisonaiagents import Agent
+
+# Initialize observability (auto-detects provider from env vars)
+obs.init()
+
+# Or specify a provider explicitly
+# obs.init(provider="langfuse", project_name="my-project")
+
+# Create an agent
+agent = Agent(
+    instructions="You are a helpful assistant.",
+    model="gpt-4o-mini",
+)
+
+# Use trace context manager for complete workflow tracing
+with obs.trace("chat-session", session_id="user-123"):
+    # Use span context manager for individual operations
+    with obs.span("user-query", kind=obs.SpanKind.AGENT):
+        response = agent.chat("What is the capital of France?")
+        print(response)
+
+# Log LLM calls manually if needed
+obs.log_llm_call(
+    model="gpt-4o-mini",
+    input_messages="What is 2+2?",
+    output="4",
+    input_tokens=10,
+    output_tokens=1,
+)
+
+# Check diagnostics
+print("\nDiagnostics:")
+print(obs.doctor())

examples/observability/langfuse_example.py

@@ -0,0 +1,48 @@
+"""
+Langfuse Integration Example
+
+This example shows how to use Langfuse for LLM observability.
+
+Setup:
+    1. Sign up at https://langfuse.com/
+    2. Get your API keys from the project settings
+    3. Set environment variables:
+       export LANGFUSE_PUBLIC_KEY=pk-lf-xxx
+       export LANGFUSE_SECRET_KEY=sk-lf-xxx
+    4. Install dependencies:
+       pip install opentelemetry-sdk opentelemetry-exporter-otlp
+
+Usage:
+    python langfuse_example.py
+"""
+
+import os
+from praisonai_tools.observability import obs
+from praisonaiagents import Agent
+
+# Initialize Langfuse
+success = obs.init(
+    provider="langfuse",
+    project_name="praisonai-demo",
+)
+
+if not success:
+    print("Failed to initialize Langfuse. Check your API keys.")
+    print("Required: LANGFUSE_PUBLIC_KEY, LANGFUSE_SECRET_KEY")
+    exit(1)
+
+print("Langfuse initialized successfully!")
+print(f"View traces at: https://cloud.langfuse.com/")
+
+# Create agent
+agent = Agent(
+    instructions="You are a helpful coding assistant.",
+    model="gpt-4o-mini",
+)
+
+# Run with tracing
+with obs.trace("coding-session", user_id="developer-1"):
+    response = agent.chat("Write a Python function to calculate fibonacci numbers")
+    print(response)
+
+print("\nCheck Langfuse dashboard for traces!")

examples/observability/mlflow_local_wow.py

@@ -0,0 +1,22 @@
+"""MLflow Local Observability Demo (no API key needed)"""
+import mlflow
+import time
+
+mlflow.set_tracking_uri("file:///tmp/mlflow_demo")
+mlflow.set_experiment("praisonai-demo")
+
+with mlflow.start_run(run_name="agent-trace"):
+    mlflow.log_param("agent_name", "Assistant")
+    mlflow.log_param("model", "gpt-4o-mini")
+    
+    # Simulate agent work
+    start = time.time()
+    time.sleep(0.1)
+    latency = time.time() - start
+    
+    mlflow.log_metric("latency_ms", latency * 1000)
+    mlflow.log_metric("tokens_used", 150)
+    
+    print(f"Logged run to MLflow: latency={latency*1000:.1f}ms")
+
+print("PASSED: MLflow local observability works")

examples/observability/multi_agent_tracing.py

@@ -0,0 +1,53 @@
+"""
+Multi-Agent Tracing Example
+
+This example shows how to trace multi-agent workflows with
+parent/child span relationships.
+
+Usage:
+    # Set your provider's API key
+    export LANGFUSE_PUBLIC_KEY=pk-lf-xxx
+    export LANGFUSE_SECRET_KEY=sk-lf-xxx
+    
+    python multi_agent_tracing.py
+"""
+
+from praisonai_tools.observability import obs
+from praisonai_tools.observability.base import SpanKind
+from praisonaiagents import Agent, PraisonAIAgents
+
+# Initialize observability
+obs.init()
+
+# Create agents
+researcher = Agent(
+    name="Researcher",
+    instructions="You research topics and provide detailed information.",
+    model="gpt-4o-mini",
+)
+
+writer = Agent(
+    name="Writer", 
+    instructions="You write clear, engaging content based on research.",
+    model="gpt-4o-mini",
+)
+
+# Trace the multi-agent workflow
+with obs.trace("content-creation", metadata={"workflow": "research-write"}):
+    
+    # Research phase
+    with obs.span("research-phase", kind=SpanKind.AGENT) as research_span:
+        research_span.attributes["agent"] = "Researcher"
+        research_result = researcher.chat("Research the benefits of AI in healthcare")
+    
+    # Writing phase  
+    with obs.span("writing-phase", kind=SpanKind.AGENT) as write_span:
+        write_span.attributes["agent"] = "Writer"
+        write_span.attributes["input_length"] = len(str(research_result))
+        final_content = writer.chat(f"Write a blog post based on: {research_result}")
+
+print("Final Content:")
+print(final_content)
+
+# Check trace info
+print("\nDiagnostics:", obs.doctor())

examples/persistence/knowledge_qdrant.py

@@ -1,7 +1,7 @@
 """
-Qdrant Knowledge Store Example for PraisonAI.
+Qdrant Knowledge Store - Agent-First Example
 
-Demonstrates vector insert + search through the knowledge store.
+Demonstrates using Qdrant for knowledge-based RAG with an Agent.
 
 Requirements:
     pip install "praisonai[tools]"
@@ -13,67 +13,59 @@ Run:
     python knowledge_qdrant.py
 
 Expected Output:
-    === Qdrant Knowledge Store Demo ===
-    Created collection: demo_knowledge
-    Inserted 3 documents
-    Search results: 2 documents found
+    Agent responds using knowledge from documents
 """
 
-from praisonai.persistence.factory import create_knowledge_store
-from praisonai.persistence.knowledge.base import KnowledgeDocument
-import random
+from praisonaiagents import Agent
+import tempfile
+import os
+import shutil
 
-# Create Qdrant knowledge store
-store = create_knowledge_store("qdrant", url="http://localhost:6333")
+# Create sample knowledge document
+sample_doc = """
+# AI and Programming Guide
 
-collection_name = "demo_knowledge"
-dimension = 384  # Embedding dimension
+## Python Programming
+Python is a versatile programming language used for web development, data science, and AI.
+It has a simple syntax and extensive libraries.
 
-# Create collection
-print("=== Qdrant Knowledge Store Demo ===")
-if store.collection_exists(collection_name):
-    store.delete_collection(collection_name)
+## Machine Learning
+Machine learning enables computers to learn patterns from data without explicit programming.
+Common frameworks include TensorFlow, PyTorch, and scikit-learn.
 
-store.create_collection(collection_name, dimension=dimension)
-print(f"Created collection: {collection_name}")
+## Databases
+PostgreSQL is a powerful open-source relational database system.
+It supports both SQL and JSON data types.
+"""
 
-# Create sample documents with embeddings
-random.seed(42)
-documents = [
-    KnowledgeDocument(
-        id="doc-1",
-        content="Python is a versatile programming language used for web development, data science, and AI.",
-        embedding=[random.random() for _ in range(dimension)],
-        metadata={"category": "programming", "language": "python"}
-    ),
-    KnowledgeDocument(
-        id="doc-2", 
-        content="Machine learning enables computers to learn patterns from data without explicit programming.",
-        embedding=[random.random() for _ in range(dimension)],
-        metadata={"category": "ai", "topic": "ml"}
-    ),
-    KnowledgeDocument(
-        id="doc-3",
-        content="PostgreSQL is a powerful open-source relational database system.",
-        embedding=[random.random() for _ in range(dimension)],
-        metadata={"category": "database", "type": "sql"}
-    ),
-]
+# Save to temp file
+temp_dir = tempfile.mkdtemp()
+doc_path = os.path.join(temp_dir, "guide.txt")
+with open(doc_path, "w") as f:
+    f.write(sample_doc)
 
-# Insert documents
-ids = store.insert(collection_name, documents)
-print(f"Inserted {len(ids)} documents")
+print("=== Qdrant Knowledge Store Demo (Agent-First) ===")
 
-# Search for similar documents
-query_embedding = [random.random() for _ in range(dimension)]
-results = store.search(collection_name, query_embedding, limit=2)
-print(f"Search results: {len(results)} documents found")
+# Agent-first approach: use knowledge parameter with Qdrant
+agent = Agent(
+    name="KnowledgeAssistant",
+    instructions="You are a helpful assistant with access to technical documentation.",
+    knowledge=[doc_path],
+    knowledge_config={
+        "vector_store": "qdrant",
+        "url": "http://localhost:6333"
+    }
+)
 
-for doc in results:
-    print(f"  - {doc.id}: {doc.content[:50]}...")
+# Chat - agent uses knowledge for RAG
+response = agent.chat("What programming language is good for AI?")
+print(f"Response: {response}")
 
 # Cleanup
-store.delete_collection(collection_name)
-store.close()
+shutil.rmtree(temp_dir)
 
 print("\n=== Demo Complete ===")
+
+# --- Advanced: Direct Store Usage ---
+# from praisonai.persistence.factory import create_knowledge_store
+# store = create_knowledge_store("qdrant", url="http://localhost:6333")

examples/persistence/redis_state.py

@@ -1,83 +1,46 @@
 """
-Redis State Store
+Redis State Store - Agent-First Example
 
-Use Redis for state management, tracing, and caching.
+Use Redis for state management with an Agent.
+
+Docker Setup:
+    docker run -d --name redis -p 6379:6379 redis:7
 
 Run:
     python redis_state.py
 
 Expected output:
-    - State operations (set/get/delete)
-    - Tracing operations
+    Agent responds with state persisted to Redis
 """
 
-from praisonaiagents import db
+from praisonaiagents import Agent, db
 
-# Create Redis-only database (for state/tracing)
-redis_db = db.RedisDB(host="localhost", port=6379)
+print("=== Redis State Store (Agent-First) ===")
 
-print("=== Redis State Store ===")
-
-# Test state operations
-print("\n1. State Operations:")
-redis_db._init_stores()  # Initialize stores
-
-# Set a value
-redis_db._state_store.set("user:123:preferences", {
-    "theme": "dark",
-    "language": "en"
-})
-print("   Set user preferences")
-
-# Get the value
-prefs = redis_db._state_store.get("user:123:preferences")
-print(f"   Got preferences: {prefs}")
-
-# Set with TTL (expires in 60 seconds)
-redis_db._state_store.set("temp:token", "abc123", ttl=60)
-print("   Set temporary token (60s TTL)")
-
-# Delete
-redis_db._state_store.delete("user:123:preferences")
-print("   Deleted preferences")
-
-# Test tracing
-print("\n2. Tracing Operations:")
-
-# Start a trace
-trace_id = "trace-example-001"
-redis_db.on_trace_start(
-    trace_id=trace_id,
-    session_id="example-session",
-    agent_name="ExampleAgent",
-    metadata={"purpose": "demo"}
+# Agent-first approach: use db parameter with Redis state store
+my_db = db(
+    database_url="sqlite:///conversations.db",  # Conversations
+    state_url="redis://localhost:6379"          # State/tracing
 )
-print(f"   Started trace: {trace_id}")
 
-# Add spans
-span_id = "span-llm-001"
-redis_db.on_span_start(
-    span_id=span_id,
-    trace_id=trace_id,
-    name="llm_call",
-    attributes={"model": "gpt-4", "temperature": 0.7}
+agent = Agent(
+    name="Assistant",
+    instructions="You are a helpful assistant.",
+    db=my_db,
+    session_id="redis-state-session"
 )
-print(f"   Started span: {span_id}")
 
-redis_db.on_span_end(
-    span_id=span_id,
-    status="ok",
-    attributes={"tokens": 150, "latency_ms": 450}
-)
-print(f"   Ended span: {span_id}")
+# Chat - state is automatically managed via Redis
+response = agent.chat("Hello! Remember my name is Alice.")
+print(f"Response: {response}")
 
-# End trace
-redis_db.on_trace_end(
-    trace_id=trace_id,
-    status="ok",
-    metadata={"total_spans": 1}
-)
-print(f"   Ended trace: {trace_id}")
+# Second message to test state
+response2 = agent.chat("What is my name?")
+print(f"Response: {response2}")
 
-redis_db.close()
+my_db.close()
 print("\n✅ Done")
+
+# --- Advanced: Direct Store Usage ---
+# from praisonaiagents import db
+# redis_db = db.RedisDB(host="localhost", port=6379)

examples/persistence/simple_db_agent.py

@@ -26,23 +26,18 @@ Expected Output:
     Messages persisted: 2
 """
 
-from praisonaiagents import Agent
-from praisonai.db import PostgresDB
+from praisonaiagents import Agent, db
 
-# Create database adapter
-db = PostgresDB(
-    host="localhost",
-    port=5432,
-    database="praisonai",
-    user="postgres",
-    password="praison123"
+# Create database adapter (simplified import)
+db_instance = db(
+    database_url="postgresql://postgres:praison123@localhost:5432/praisonai"
 )
 
 # Create agent with persistence enabled
 agent = Agent(
     name="Assistant",
     instructions="You are a helpful assistant. Keep responses brief.",
-    db=db,
+    db=db_instance,
     session_id="demo-session-001",  # Same session_id = resume conversation
     verbose=False
 )
@@ -57,7 +52,7 @@ print(f"Session ID: {agent.session_id}")
 print(f"Messages in history: {len(agent.chat_history)}")
 
 # Close DB connection
-db.close()
+db_instance.close()
 
 print("\n=== Session persisted! ===")
 print("Run again with same session_id to resume.")

examples/persistence/state_redis.py

@@ -1,10 +1,7 @@
 """
-Redis State Store Example for PraisonAI.
+Redis State Store - Agent-First Example
 
-Demonstrates set/get/delete + TTL operations.
-
-Requirements:
-    pip install "praisonai[tools]"
+Use Redis for state management with an Agent.
 
 Docker Setup:
     docker run -d --name praison-redis -p 6379:6379 redis:7
@@ -13,55 +10,37 @@ Run:
     python state_redis.py
 
 Expected Output:
-    === Redis State Store Demo ===
-    Set key: session:user123:preferences
-    Get value: {'theme': 'dark', 'language': 'en'}
-    TTL remaining: 3600 seconds
-    Hash operations: {'name': 'Alice', 'role': 'admin'}
+    Agent responds with state persisted to Redis
 """
 
-from praisonai.persistence.factory import create_state_store
+from praisonaiagents import Agent, db
 
-# Create Redis state store
-store = create_state_store("redis", url="redis://localhost:6379")
+print("=== Redis State Store (Agent-First) ===")
 
-print("=== Redis State Store Demo ===")
+# Agent-first approach: use db parameter with Redis state store
+my_db = db(
+    database_url="sqlite:///conversations.db",  # Conversations
+    state_url="redis://localhost:6379"          # State/tracing
+)
 
-# Basic set/get
-key = "session:user123:preferences"
-value = {"theme": "dark", "language": "en", "notifications": True}
+agent = Agent(
+    name="Assistant",
+    instructions="You are a helpful assistant.",
+    db=my_db,
+    session_id="redis-state-example"
+)
 
-store.set(key, value)
-print(f"Set key: {key}")
+# Chat - state is automatically managed via Redis
+response = agent.chat("Hello! Remember my favorite color is blue.")
+print(f"Response: {response}")
 
-retrieved = store.get(key)
-print(f"Get value: {retrieved}")
-
-# Set with TTL (expires in 1 hour)
-ttl_key = "session:user123:token"
-store.set(ttl_key, "abc123xyz", ttl=3600)
-remaining = store.ttl(ttl_key)
-print(f"TTL remaining: {remaining} seconds")
-
-# Hash operations (for structured data)
-hash_key = "user:123:profile"
-store.hset(hash_key, "name", "Alice")
-store.hset(hash_key, "role", "admin")
-store.hset(hash_key, "last_login", "2024-12-24")
-
-profile = store.hgetall(hash_key)
-print(f"Hash operations: {profile}")
-
-# Check existence
-exists = store.exists(key)
-print(f"Key exists: {exists}")
-
-# Delete keys
-store.delete(key)
-store.delete(ttl_key)
-store.delete(hash_key)
-print("Cleaned up test keys")
-
-store.close()
+# Second message to test state
+response2 = agent.chat("What is my favorite color?")
+print(f"Response: {response2}")
 
+my_db.close()
 print("\n=== Demo Complete ===")
+
+# --- Advanced: Direct Store Usage ---
+# from praisonai.persistence.factory import create_state_store
+# store = create_state_store("redis", url="redis://localhost:6379")

examples/python/knowledge/advanced-knowledge-management.py

@@ -15,6 +15,7 @@ Features demonstrated:
 from praisonaiagents import Agent
 import tempfile
 import os
+import shutil
 
 # Create a sample knowledge document for demonstration
 sample_document = """
@@ -91,38 +92,15 @@ doc_path = os.path.join(temp_dir, "ai_healthcare_guide.txt")
 with open(doc_path, "w") as f:
     f.write(sample_document)
 
-# Create an agent with advanced knowledge management configuration
+# Create an agent with knowledge management
 knowledge_agent = Agent(
     name="AdvancedKnowledgeAgent",
     role="Healthcare AI Knowledge Expert", 
     goal="Provide comprehensive answers about AI in healthcare using advanced knowledge retrieval",
     backstory="You are an expert in healthcare AI with access to comprehensive knowledge bases and advanced retrieval capabilities.",
     
-    # Advanced knowledge configuration
+    # Knowledge configuration - simple and working
     knowledge=[doc_path],
-    knowledge_config={
-        "vector_store": {
-            "provider": "chroma",
-            "collection_name": "healthcare_ai_knowledge"
-        },
-        "chunking": {
-            "strategy": "semantic",  # Advanced semantic chunking
-            "chunk_size": 500,       # Optimal chunk size for retrieval
-            "chunk_overlap": 50,     # Overlap for context preservation
-            "separators": ["\n\n", "\n", ".", "!", "?"]  # Smart separators
-        },
-        "retrieval": {
-            "search_type": "similarity",
-            "k": 5,  # Retrieve top 5 most relevant chunks
-            "score_threshold": 0.7,  # Minimum relevance score
-            "rerank": True  # Enable reranking for better results
-        },
-        "embedding": {
-            "provider": "openai",
-            "model": "text-embedding-3-small",  # Efficient embedding model
-            "dimensions": 1536
-        }
-    },
     
     instructions="""You are an expert in healthcare AI. Use the knowledge base to provide 
     comprehensive, accurate answers. Always cite specific information from the documents 
@@ -180,7 +158,6 @@ relationship_result = knowledge_agent.start(
 print(f"Relationship analysis result:\n{relationship_result}\n")
 
 # Clean up temporary files
-import shutil
 shutil.rmtree(temp_dir)
 
 print("="*80)

examples/run_all_examples.py

@@ -0,0 +1,69 @@
+#!/usr/bin/env python3
+"""Run all examples and collect PASS/SKIP/FAIL results."""
+import subprocess
+import sys
+import os
+from pathlib import Path
+
+EXAMPLES_DIR = Path(__file__).parent
+RESULTS = []
+
+def run_example(path: Path) -> tuple:
+    """Run an example and return (name, status, reason)."""
+    name = f"{path.parent.name}/{path.name}"
+    try:
+        result = subprocess.run(
+            [sys.executable, str(path)],
+            capture_output=True,
+            text=True,
+            timeout=60,
+            env={**os.environ, "PYTHONPATH": str(EXAMPLES_DIR.parent / "src")}
+        )
+        output = result.stdout + result.stderr
+        if "PASSED" in output:
+            return (name, "PASS", "")
+        elif "SKIPPED" in output:
+            return (name, "SKIP", output.split("SKIPPED:")[-1].strip()[:50])
+        elif result.returncode != 0:
+            error = output.strip().split("\n")[-1][:80]
+            return (name, "FAIL", error)
+        else:
+            return (name, "PASS", "")
+    except subprocess.TimeoutExpired:
+        return (name, "FAIL", "Timeout")
+    except Exception as e:
+        return (name, "FAIL", str(e)[:50])
+
+def main():
+    print("=" * 70)
+    print("EXAMPLE RUNNER - PASS/SKIP/FAIL SUMMARY")
+    print("=" * 70)
+    
+    # Find all example files
+    examples = []
+    for subdir in ["db", "vector", "observability", "multi_agent", "workflows"]:
+        subpath = EXAMPLES_DIR / subdir
+        if subpath.exists():
+            examples.extend(sorted(subpath.glob("*_wow.py")))
+    
+    print(f"\nFound {len(examples)} examples to run\n")
+    
+    for ex in examples:
+        name, status, reason = run_example(ex)
+        RESULTS.append((name, status, reason))
+        icon = {"PASS": "✅", "SKIP": "⏭️", "FAIL": "❌"}.get(status, "?")
+        reason_str = f" ({reason})" if reason else ""
+        print(f"{icon} {status:4} | {name:45} {reason_str}")
+    
+    # Summary
+    print("\n" + "=" * 70)
+    passed = sum(1 for _, s, _ in RESULTS if s == "PASS")
+    skipped = sum(1 for _, s, _ in RESULTS if s == "SKIP")
+    failed = sum(1 for _, s, _ in RESULTS if s == "FAIL")
+    print(f"SUMMARY: {passed} PASS | {skipped} SKIP | {failed} FAIL | {len(RESULTS)} TOTAL")
+    print("=" * 70)
+    
+    return 0 if failed == 0 else 1
+
+if __name__ == "__main__":
+    sys.exit(main())

examples/vector/chroma_wow.py

@@ -0,0 +1,23 @@
+"""ChromaDB Vector Store - Agent-First Example"""
+from praisonaiagents import Agent
+
+# Agent-first approach: use knowledge parameter with ChromaDB
+agent = Agent(
+    name="Assistant",
+    instructions="You are a helpful assistant with access to documents.",
+    knowledge=["./docs/guide.pdf"],  # Add your documents here
+    knowledge_config={
+        "vector_store": "chroma",
+        "path": "/tmp/chroma_demo"
+    }
+)
+
+# Chat - agent uses knowledge for RAG
+response = agent.chat("What information do you have?")
+print(f"Response: {response}")
+
+print("PASSED: ChromaDB with Agent")
+
+# --- Advanced: Direct Store Usage ---
+# from praisonai.persistence import create_knowledge_store
+# store = create_knowledge_store("chroma", path="/tmp/chroma_demo")

examples/vector/lancedb_real_wow.py

@@ -0,0 +1,25 @@
+"""LanceDB Vector Store - Agent-First Example"""
+from praisonaiagents import Agent
+
+# Agent-first approach: use knowledge parameter with LanceDB
+agent = Agent(
+    name="Assistant",
+    instructions="You are a helpful assistant with access to documents.",
+    knowledge=["./docs/guide.pdf"],  # Add your documents here
+    knowledge_config={
+        "vector_store": "lancedb",
+        "path": "/tmp/lancedb_test"
+    }
+)
+
+# Chat - agent uses knowledge for RAG
+response = agent.chat("What information do you have?")
+print(f"Response: {response}")
+
+print("PASSED: LanceDB with Agent")
+
+# --- Advanced: Direct LanceDB Usage ---
+# import lancedb
+# db = lancedb.connect("/tmp/lancedb_test")
+# data = [{"id": "1", "text": "ML is AI", "vector": [0.1] * 128}]
+# table = db.create_table("demo", data)

examples/vector/lancedb_wow.py

@@ -0,0 +1,23 @@
+"""LanceDB Vector Store - Agent-First Example"""
+from praisonaiagents import Agent
+
+# Agent-first approach: use knowledge parameter with LanceDB
+agent = Agent(
+    name="Assistant",
+    instructions="You are a helpful assistant with access to documents.",
+    knowledge=["./docs/guide.pdf"],  # Add your documents here
+    knowledge_config={
+        "vector_store": "lancedb",
+        "path": "/tmp/lancedb_demo"
+    }
+)
+
+# Chat - agent uses knowledge for RAG
+response = agent.chat("What information do you have?")
+print(f"Response: {response}")
+
+print("PASSED: LanceDB with Agent")
+
+# --- Advanced: Direct Store Usage ---
+# from praisonai.persistence import create_knowledge_store
+# store = create_knowledge_store("lancedb", path="/tmp/lancedb_demo")

examples/vector/pgvector_real_wow.py

@@ -0,0 +1,31 @@
+"""PGVector Vector Store - Agent-First Example
+
+Docker: docker run -d --name pgvector -e POSTGRES_PASSWORD=postgres -p 5433:5432 pgvector/pgvector:pg16
+"""
+import os
+from praisonaiagents import Agent
+
+url = os.getenv("PGVECTOR_URL", "postgresql://postgres:postgres@localhost:5433/postgres")
+
+# Agent-first approach: use knowledge parameter with PGVector
+agent = Agent(
+    name="Assistant",
+    instructions="You are a helpful assistant with access to documents.",
+    knowledge=["./docs/guide.pdf"],  # Add your documents here
+    knowledge_config={
+        "vector_store": "pgvector",
+        "url": url
+    }
+)
+
+# Chat - agent uses knowledge for RAG
+response = agent.chat("What information do you have?")
+print(f"Response: {response}")
+
+print("PASSED: PGVector with Agent")
+
+# --- Advanced: Direct PGVector Usage ---
+# import psycopg2
+# conn = psycopg2.connect(url)
+# cur = conn.cursor()
+# cur.execute("CREATE EXTENSION IF NOT EXISTS vector")

examples/vector/pgvector_wow.py

@@ -0,0 +1,29 @@
+"""PGVector Knowledge Store - Agent-First Example (requires Docker)
+
+Docker: docker run -d --name pgvector -e POSTGRES_PASSWORD=postgres -p 5433:5432 pgvector/pgvector:pg16
+"""
+import os
+from praisonaiagents import Agent
+
+url = os.getenv("PGVECTOR_URL", "postgresql://postgres:postgres@localhost:5433/postgres")
+
+# Agent-first approach: use knowledge parameter with PGVector
+agent = Agent(
+    name="Assistant",
+    instructions="You are a helpful assistant with access to documents.",
+    knowledge=["./docs/guide.pdf"],  # Add your documents here
+    knowledge_config={
+        "vector_store": "pgvector",
+        "url": url
+    }
+)
+
+# Chat - agent uses knowledge for RAG
+response = agent.chat("What information do you have?")
+print(f"Response: {response}")
+
+print("PASSED: PGVector with Agent")
+
+# --- Advanced: Direct Store Usage ---
+# from praisonai.persistence import create_knowledge_store
+# store = create_knowledge_store("pgvector", url=url)

examples/vector/pinecone_wow.py

@@ -0,0 +1,34 @@
+"""Pinecone Vector Store - Agent-First Example
+
+Requires: export PINECONE_API_KEY=...
+"""
+import os
+import sys
+from praisonaiagents import Agent
+
+api_key = os.getenv("PINECONE_API_KEY")
+if not api_key:
+    print("SKIPPED: Pinecone - PINECONE_API_KEY not set")
+    sys.exit(0)
+
+# Agent-first approach: use knowledge parameter with Pinecone
+agent = Agent(
+    name="Assistant",
+    instructions="You are a helpful assistant with access to documents.",
+    knowledge=["./docs/guide.pdf"],  # Add your documents here
+    knowledge_config={
+        "vector_store": "pinecone",
+        "api_key": api_key,
+        "environment": "us-east-1"
+    }
+)
+
+# Chat - agent uses knowledge for RAG
+response = agent.chat("What information do you have?")
+print(f"Response: {response}")
+
+print("PASSED: Pinecone with Agent")
+
+# --- Advanced: Direct Store Usage ---
+# from pinecone import Pinecone
+# pc = Pinecone(api_key=api_key)

examples/vector/qdrant_wow.py

@@ -0,0 +1,29 @@
+"""Qdrant Vector Store - Agent-First Example (requires Docker)
+
+Docker: docker run -d --name qdrant -p 6333:6333 qdrant/qdrant
+"""
+import os
+from praisonaiagents import Agent
+
+# Agent-first approach: use knowledge parameter with Qdrant
+url = os.getenv("QDRANT_URL", "http://localhost:6333")
+
+agent = Agent(
+    name="Assistant",
+    instructions="You are a helpful assistant with access to documents.",
+    knowledge=["./docs/guide.pdf"],  # Add your documents here
+    knowledge_config={
+        "vector_store": "qdrant",
+        "url": url
+    }
+)
+
+# Chat - agent uses knowledge for RAG
+response = agent.chat("What information do you have?")
+print(f"Response: {response}")
+
+print("PASSED: Qdrant with Agent")
+
+# --- Advanced: Direct Store Usage ---
+# from praisonai.persistence import create_knowledge_store
+# store = create_knowledge_store("qdrant", url=url)

examples/vector/weaviate_wow.py

@@ -0,0 +1,36 @@
+"""Weaviate Cloud Vector Store - Agent-First Example
+
+Requires: export WEAVIATE_URL=... and WEAVIATE_API_KEY=...
+"""
+import os
+import sys
+from praisonaiagents import Agent
+
+weaviate_url = os.getenv("WEAVIATE_URL", "https://your-cluster.weaviate.cloud")
+weaviate_key = os.getenv("WEAVIATE_API_KEY")
+if not weaviate_key:
+    print("SKIPPED: Weaviate - WEAVIATE_API_KEY not set")
+    sys.exit(0)
+
+# Agent-first approach: use knowledge parameter with Weaviate
+agent = Agent(
+    name="Assistant",
+    instructions="You are a helpful assistant with access to documents.",
+    knowledge=["./docs/guide.pdf"],  # Add your documents here
+    knowledge_config={
+        "vector_store": "weaviate",
+        "url": weaviate_url,
+        "api_key": weaviate_key
+    }
+)
+
+# Chat - agent uses knowledge for RAG
+response = agent.chat("What information do you have?")
+print(f"Response: {response}")
+
+print("PASSED: Weaviate with Agent")
+
+# --- Advanced: Direct Store Usage ---
+# import weaviate
+# from weaviate.classes.init import Auth
+# client = weaviate.connect_to_weaviate_cloud(cluster_url=weaviate_url, auth_credentials=Auth.api_key(weaviate_key))

src/praisonai-agents/.gitignore

@@ -62,6 +62,8 @@ docs/generated/
 *.log
 *.sqlite
 *.db
+# But keep the db module directory
+!praisonaiagents/db/
 
 # OS generated files
 .DS_Store

src/praisonai-agents/praisonaiagents/__init__.py

@@ -43,8 +43,7 @@ from .tools.base import BaseTool, ToolResult, ToolValidationError, validate_tool
 from .tools.decorator import tool, FunctionTool
 from .tools.registry import get_registry, register_tool, get_tool, ToolRegistry
 from .agents.autoagents import AutoAgents
-from .knowledge.knowledge import Knowledge
-from .knowledge.chunking import Chunking
+# Knowledge is lazy-loaded to avoid importing heavy deps (chromadb, mem0) at startup
 # MCP support (optional)
 try:
     from .mcp.mcp import MCP
@@ -53,8 +52,9 @@ except ImportError:
     _mcp_available = False
     MCP = None
 from .session import Session
-from .memory.memory import Memory
+# Memory is lazy-loaded to avoid importing chromadb at startup
 from .db import db
+from .obs import obs
 # Workflows - import from dedicated workflows module
 from .workflows import (
     Workflow, WorkflowStep, WorkflowContext, StepResult,
@@ -66,8 +66,16 @@ from .guardrails import GuardrailResult, LLMGuardrail
 
 # Fast Context support (lazy loaded to avoid performance impact)
 def __getattr__(name):
-    """Lazy load FastContext to avoid impacting package load time."""
-    if name == "FastContext":
+    """Lazy load heavy modules to avoid impacting package load time."""
+    # Knowledge module (imports chromadb, mem0)
+    if name == "Knowledge":
+        from praisonaiagents.knowledge.knowledge import Knowledge
+        return Knowledge
+    elif name == "Chunking":
+        from praisonaiagents.knowledge.chunking import Chunking
+        return Chunking
+    # FastContext support
+    elif name == "FastContext":
         from praisonaiagents.context.fast import FastContext
         return FastContext
     elif name == "FastContextResult":
@@ -92,6 +100,10 @@ def __getattr__(name):
     elif name == "SkillLoader":
         from praisonaiagents.skills import SkillLoader
         return SkillLoader
+    # Memory module (imports chromadb)
+    elif name == "Memory":
+        from praisonaiagents.memory.memory import Memory
+        return Memory
     raise AttributeError(f"module {__name__!r} has no attribute {name!r}")
 
 # Planning mode support
@@ -259,6 +271,7 @@ __all__ = [
     'Session',
     'Memory',
     'db',
+    'obs',
     'display_interaction',
     'display_self_reflection',
     'display_instruction',

src/praisonai-agents/praisonaiagents/agent/agent.py

@@ -254,7 +254,10 @@ class Agent:
         skills: Optional[List[str]] = None,
         skills_dirs: Optional[List[str]] = None,
         db: Optional[Any] = None,
-        session_id: Optional[str] = None
+        session_id: Optional[str] = None,
+        hooks: Optional[List[Any]] = None,
+        llm_config: Optional[Dict[str, Any]] = None,
+        rate_limiter: Optional[Any] = None
     ):
         """Initialize an Agent instance.
 
@@ -395,6 +398,17 @@ class Agent:
         # Flag to track if final result has been displayed to prevent duplicates
         self._final_display_shown = False
         
+        # Store hooks for middleware system (zero overhead when empty)
+        self._hooks = hooks or []
+        self._middleware_manager = None  # Lazy init
+        
+        # Store llm_config for configurable model switching
+        self._llm_config = llm_config or {}
+        self._llm_configurable = self._llm_config.get('configurable', False)
+        
+        # Store rate limiter (optional, zero overhead when None)
+        self._rate_limiter = rate_limiter
+        
         # Store OpenAI client parameters for lazy initialization
         self._openai_api_key = api_key
         self._openai_base_url = base_url
@@ -1622,8 +1636,32 @@ Your Goal: {self.goal}"""
     def execute_tool(self, function_name, arguments):
         """
         Execute a tool dynamically based on the function name and arguments.
+        Injects agent state for tools with Injected[T] parameters.
         """
         logging.debug(f"{self.name} executing tool {function_name} with arguments: {arguments}")
+        
+        # Set up injection context for tools with Injected parameters
+        from ..tools.injected import AgentState, with_injection_context
+        state = AgentState(
+            agent_id=self.name,
+            run_id=getattr(self, '_current_run_id', 'unknown'),
+            session_id=getattr(self, '_session_id', None) or 'default',
+            last_user_message=self.chat_history[-1].get('content') if self.chat_history else None,
+            metadata={'agent_name': self.name}
+        )
+        
+        # Execute within injection context
+        return self._execute_tool_with_context(function_name, arguments, state)
+    
+    def _execute_tool_with_context(self, function_name, arguments, state):
+        """Execute tool within injection context."""
+        from ..tools.injected import with_injection_context
+        
+        with with_injection_context(state):
+            return self._execute_tool_impl(function_name, arguments)
+    
+    def _execute_tool_impl(self, function_name, arguments):
+        """Internal tool execution implementation."""
 
         # Check if approval is required for this tool
         from ..approval import is_approval_required, console_approval_callback, get_risk_level, mark_approved, ApprovalDecision, get_approval_callback
@@ -2058,7 +2096,11 @@ Your Goal: {self.goal}"""
         """Get the current session ID."""
         return self._session_id
 
-    def chat(self, prompt, temperature=1.0, tools=None, output_json=None, output_pydantic=None, reasoning_steps=False, stream=None, task_name=None, task_description=None, task_id=None):
+    def chat(self, prompt, temperature=1.0, tools=None, output_json=None, output_pydantic=None, reasoning_steps=False, stream=None, task_name=None, task_description=None, task_id=None, config=None):
+        # Apply rate limiter if configured (before any LLM call)
+        if self._rate_limiter is not None:
+            self._rate_limiter.acquire()
+        
         # Initialize DB session on first chat (lazy)
         self._init_db_session()
         

src/praisonai-agents/praisonaiagents/db/__init__.py

@@ -0,0 +1,85 @@
+"""
+Database adapter interface for PraisonAI Agents.
+
+This module provides the protocol and types for database persistence.
+Implementations are provided by the wrapper layer (praisonai.db).
+
+Usage (simplest - recommended):
+    from praisonaiagents import Agent, db
+    
+    agent = Agent(
+        name="Assistant",
+        db=db(database_url="postgresql://localhost/mydb"),  # db(...) shortcut
+        session_id="my-session"  # optional: defaults to per-hour ID
+    )
+    agent.chat("Hello!")  # auto-persists messages, runs, traces, tool calls
+
+Alternative (explicit backend):
+    db=db.PraisonDB(database_url="...")  # Auto-detect backend
+    db=db.PostgresDB(host="localhost")   # PostgreSQL
+    db=db.SQLiteDB(path="data.db")       # SQLite
+    db=db.RedisDB(host="localhost")      # Redis (state only)
+"""
+
+from .protocol import (
+    DbAdapter,
+    AsyncDbAdapter,
+    DbMessage,
+    DbToolCall,
+    DbRun,
+    DbSpan,
+    DbTrace,
+)
+
+
+class _LazyDbModule:
+    """
+    Lazy proxy for database backends.
+    
+    Allows `from praisonaiagents import db` without importing heavy deps.
+    Actual backend classes are loaded only when accessed.
+    """
+    
+    _BACKENDS = {
+        "PraisonDB": "praisonai.db.adapter",
+        "PostgresDB": "praisonai.db.adapter",
+        "SQLiteDB": "praisonai.db.adapter",
+        "RedisDB": "praisonai.db.adapter",
+    }
+    
+    def __getattr__(self, name: str):
+        if name in self._BACKENDS:
+            try:
+                import importlib
+                module = importlib.import_module(self._BACKENDS[name])
+                return getattr(module, name)
+            except ImportError as e:
+                raise ImportError(
+                    f"Database backend '{name}' requires the praisonai package. "
+                    f"Install with: pip install praisonai\n"
+                    f"Original error: {e}"
+                ) from e
+        raise AttributeError(f"module 'db' has no attribute {name!r}")
+    
+    def __repr__(self):
+        return "<module 'praisonaiagents.db' (lazy database backends)>"
+    
+    def __call__(self, **kwargs):
+        """Shortcut: db(...) is equivalent to db.PraisonDB(...)"""
+        return self.PraisonDB(**kwargs)
+
+
+# Singleton lazy module instance
+db = _LazyDbModule()
+
+
+__all__ = [
+    "DbAdapter",
+    "AsyncDbAdapter",
+    "DbMessage",
+    "DbToolCall",
+    "DbRun",
+    "DbSpan",
+    "DbTrace",
+    "db",
+]

src/praisonai-agents/praisonaiagents/db/protocol.py

@@ -0,0 +1,464 @@
+"""
+Database adapter protocol for PraisonAI Agents.
+
+This module uses only stdlib (typing, dataclasses) to define the interface.
+No external dependencies are imported here.
+
+Implementations are provided by the wrapper layer (praisonai.db).
+"""
+
+from dataclasses import dataclass, field
+from typing import Any, Dict, List, Optional, Protocol, runtime_checkable
+import time
+
+
+@dataclass
+class DbToolCall:
+    """A tool call to be persisted."""
+    tool_name: str
+    args: Dict[str, Any]
+    result: Any = None
+    call_id: Optional[str] = None
+    timestamp: float = field(default_factory=time.time)
+    duration_ms: Optional[float] = None
+    error: Optional[str] = None
+
+
+@dataclass
+class DbMessage:
+    """A message to be persisted."""
+    role: str  # "user", "assistant", "system", "tool"
+    content: str
+    metadata: Dict[str, Any] = field(default_factory=dict)
+    timestamp: float = field(default_factory=time.time)
+    id: Optional[str] = None
+    tool_calls: Optional[List["DbToolCall"]] = None  # For assistant messages with tool calls
+    run_id: Optional[str] = None  # Group messages by run
+
+
+@dataclass
+class DbRun:
+    """A single agent run (turn) to be persisted."""
+    run_id: str
+    session_id: str
+    agent_name: str
+    started_at: float = field(default_factory=time.time)
+    ended_at: Optional[float] = None
+    status: str = "running"  # running, completed, error, cancelled
+    input_content: Optional[str] = None
+    output_content: Optional[str] = None
+    messages: List[DbMessage] = field(default_factory=list)
+    tool_calls: List[DbToolCall] = field(default_factory=list)
+    metadata: Dict[str, Any] = field(default_factory=dict)
+    metrics: Dict[str, Any] = field(default_factory=dict)  # tokens, latency, etc.
+
+
+@dataclass
+class DbSpan:
+    """A span for tracing/observability."""
+    span_id: str
+    trace_id: str
+    name: str
+    started_at: float = field(default_factory=time.time)
+    ended_at: Optional[float] = None
+    parent_span_id: Optional[str] = None
+    status: str = "running"  # running, ok, error
+    attributes: Dict[str, Any] = field(default_factory=dict)
+    events: List[Dict[str, Any]] = field(default_factory=list)
+
+
+@dataclass
+class DbTrace:
+    """A trace for observability (groups spans)."""
+    trace_id: str
+    session_id: Optional[str] = None
+    run_id: Optional[str] = None
+    agent_name: Optional[str] = None
+    user_id: Optional[str] = None
+    started_at: float = field(default_factory=time.time)
+    ended_at: Optional[float] = None
+    status: str = "running"  # running, ok, error
+    spans: List[DbSpan] = field(default_factory=list)
+    metadata: Dict[str, Any] = field(default_factory=dict)
+
+
+@runtime_checkable
+class DbAdapter(Protocol):
+    """
+    Protocol for database adapters that provide persistence for agents.
+    
+    Implementations must provide these methods. The agent will call them
+    at appropriate lifecycle points if a db adapter is provided.
+    
+    All methods are synchronous. Async support can be added via wrapper.
+    """
+    
+    def on_agent_start(
+        self,
+        agent_name: str,
+        session_id: str,
+        user_id: Optional[str] = None,
+        metadata: Optional[Dict[str, Any]] = None,
+    ) -> List[DbMessage]:
+        """
+        Called when agent starts a session.
+        
+        Args:
+            agent_name: Name of the agent
+            session_id: Session identifier
+            user_id: Optional user identifier
+            metadata: Optional session metadata
+            
+        Returns:
+            List of previous messages for this session (for resume)
+        """
+        ...
+    
+    def on_user_message(
+        self,
+        session_id: str,
+        content: str,
+        metadata: Optional[Dict[str, Any]] = None,
+    ) -> None:
+        """
+        Called when user sends a message.
+        
+        Args:
+            session_id: Session identifier
+            content: Message content
+            metadata: Optional message metadata
+        """
+        ...
+    
+    def on_agent_message(
+        self,
+        session_id: str,
+        content: str,
+        metadata: Optional[Dict[str, Any]] = None,
+    ) -> None:
+        """
+        Called when agent produces a response.
+        
+        Args:
+            session_id: Session identifier
+            content: Response content
+            metadata: Optional message metadata
+        """
+        ...
+    
+    def on_tool_call(
+        self,
+        session_id: str,
+        tool_name: str,
+        args: Dict[str, Any],
+        result: Any,
+        metadata: Optional[Dict[str, Any]] = None,
+    ) -> None:
+        """
+        Called when a tool is executed.
+        
+        Args:
+            session_id: Session identifier
+            tool_name: Name of the tool
+            args: Tool arguments
+            result: Tool result
+            metadata: Optional call metadata
+        """
+        ...
+    
+    def on_agent_end(
+        self,
+        session_id: str,
+        metadata: Optional[Dict[str, Any]] = None,
+    ) -> None:
+        """
+        Called when agent session ends.
+        
+        Args:
+            session_id: Session identifier
+            metadata: Optional end metadata
+        """
+        ...
+    
+    def on_run_start(
+        self,
+        session_id: str,
+        run_id: str,
+        input_content: Optional[str] = None,
+        metadata: Optional[Dict[str, Any]] = None,
+    ) -> None:
+        """
+        Called when a new run (turn) starts.
+        
+        Args:
+            session_id: Session identifier
+            run_id: Run identifier
+            input_content: User input for this run
+            metadata: Optional run metadata
+        """
+        ...
+    
+    def on_run_end(
+        self,
+        session_id: str,
+        run_id: str,
+        output_content: Optional[str] = None,
+        status: str = "completed",
+        metrics: Optional[Dict[str, Any]] = None,
+        metadata: Optional[Dict[str, Any]] = None,
+    ) -> None:
+        """
+        Called when a run (turn) ends.
+        
+        Args:
+            session_id: Session identifier
+            run_id: Run identifier
+            output_content: Agent output for this run
+            status: Run status (completed, error, cancelled)
+            metrics: Optional metrics (tokens, latency)
+            metadata: Optional run metadata
+        """
+        ...
+    
+    def get_runs(
+        self,
+        session_id: str,
+        limit: Optional[int] = None,
+    ) -> List["DbRun"]:
+        """
+        Get runs for a session.
+        
+        Args:
+            session_id: Session identifier
+            limit: Maximum number of runs to return
+            
+        Returns:
+            List of runs for this session
+        """
+        ...
+    
+    def export_session(
+        self,
+        session_id: str,
+    ) -> Dict[str, Any]:
+        """
+        Export a session as a dictionary (for JSONL export).
+        
+        Args:
+            session_id: Session identifier
+            
+        Returns:
+            Dictionary with session data, messages, runs
+        """
+        ...
+    
+    def import_session(
+        self,
+        data: Dict[str, Any],
+    ) -> str:
+        """
+        Import a session from a dictionary.
+        
+        Args:
+            data: Session data dictionary
+            
+        Returns:
+            Session ID of imported session
+        """
+        ...
+    
+    # --- Tracing/Observability ---
+    
+    def on_trace_start(
+        self,
+        trace_id: str,
+        session_id: Optional[str] = None,
+        run_id: Optional[str] = None,
+        agent_name: Optional[str] = None,
+        user_id: Optional[str] = None,
+        metadata: Optional[Dict[str, Any]] = None,
+    ) -> None:
+        """
+        Called when a new trace starts.
+        
+        Args:
+            trace_id: Unique trace identifier
+            session_id: Optional session identifier
+            run_id: Optional run identifier
+            agent_name: Optional agent name
+            user_id: Optional user identifier
+            metadata: Optional trace metadata
+        """
+        ...
+    
+    def on_trace_end(
+        self,
+        trace_id: str,
+        status: str = "ok",
+        metadata: Optional[Dict[str, Any]] = None,
+    ) -> None:
+        """
+        Called when a trace ends.
+        
+        Args:
+            trace_id: Trace identifier
+            status: Trace status (ok, error)
+            metadata: Optional end metadata
+        """
+        ...
+    
+    def on_span_start(
+        self,
+        span_id: str,
+        trace_id: str,
+        name: str,
+        parent_span_id: Optional[str] = None,
+        attributes: Optional[Dict[str, Any]] = None,
+    ) -> None:
+        """
+        Called when a new span starts.
+        
+        Args:
+            span_id: Unique span identifier
+            trace_id: Parent trace identifier
+            name: Span name (e.g., "llm_call", "tool_execution")
+            parent_span_id: Optional parent span for nesting
+            attributes: Optional span attributes
+        """
+        ...
+    
+    def on_span_end(
+        self,
+        span_id: str,
+        status: str = "ok",
+        attributes: Optional[Dict[str, Any]] = None,
+    ) -> None:
+        """
+        Called when a span ends.
+        
+        Args:
+            span_id: Span identifier
+            status: Span status (ok, error)
+            attributes: Optional end attributes
+        """
+        ...
+    
+    def get_traces(
+        self,
+        session_id: Optional[str] = None,
+        user_id: Optional[str] = None,
+        limit: Optional[int] = None,
+    ) -> List["DbTrace"]:
+        """
+        Get traces with optional filters.
+        
+        Args:
+            session_id: Filter by session
+            user_id: Filter by user
+            limit: Maximum number of traces
+            
+        Returns:
+            List of traces
+        """
+        ...
+    
+    def close(self) -> None:
+        """Close the database connection."""
+        ...
+
+
+@runtime_checkable
+class AsyncDbAdapter(Protocol):
+    """
+    Async protocol for database adapters.
+    
+    Provides async variants of all DbAdapter methods for use with
+    async/await patterns and async backends like asyncpg.
+    """
+    
+    async def on_agent_start(
+        self,
+        agent_name: str,
+        session_id: str,
+        user_id: Optional[str] = None,
+        metadata: Optional[Dict[str, Any]] = None,
+    ) -> List["DbMessage"]:
+        """Async version of on_agent_start."""
+        ...
+    
+    async def on_user_message(
+        self,
+        session_id: str,
+        content: str,
+        metadata: Optional[Dict[str, Any]] = None,
+    ) -> None:
+        """Async version of on_user_message."""
+        ...
+    
+    async def on_agent_message(
+        self,
+        session_id: str,
+        content: str,
+        metadata: Optional[Dict[str, Any]] = None,
+    ) -> None:
+        """Async version of on_agent_message."""
+        ...
+    
+    async def on_tool_call(
+        self,
+        session_id: str,
+        tool_name: str,
+        args: Dict[str, Any],
+        result: Any,
+        metadata: Optional[Dict[str, Any]] = None,
+    ) -> None:
+        """Async version of on_tool_call."""
+        ...
+    
+    async def on_agent_end(
+        self,
+        session_id: str,
+        metadata: Optional[Dict[str, Any]] = None,
+    ) -> None:
+        """Async version of on_agent_end."""
+        ...
+    
+    async def on_run_start(
+        self,
+        session_id: str,
+        run_id: str,
+        input_content: Optional[str] = None,
+        metadata: Optional[Dict[str, Any]] = None,
+    ) -> None:
+        """Async version of on_run_start."""
+        ...
+    
+    async def on_run_end(
+        self,
+        session_id: str,
+        run_id: str,
+        output_content: Optional[str] = None,
+        status: str = "completed",
+        metrics: Optional[Dict[str, Any]] = None,
+        metadata: Optional[Dict[str, Any]] = None,
+    ) -> None:
+        """Async version of on_run_end."""
+        ...
+    
+    async def export_session(
+        self,
+        session_id: str,
+    ) -> Dict[str, Any]:
+        """Async version of export_session."""
+        ...
+    
+    async def import_session(
+        self,
+        data: Dict[str, Any],
+    ) -> str:
+        """Async version of import_session."""
+        ...
+    
+    async def close(self) -> None:
+        """Async close the database connection."""
+        ...

src/praisonai-agents/praisonaiagents/db/tests/__init__.py

@@ -0,0 +1 @@
+"""TDD tests for db integration."""

src/praisonai-agents/praisonaiagents/db/tests/test_db_integration.py

@@ -0,0 +1,246 @@
+"""
+TDD tests for PraisonAI Agents DB integration.
+
+Tests:
+1. Lazy import verification - no heavy deps on import
+2. Agent(db=MockDb) calls hook lifecycle in correct order
+3. Session resume injects history
+4. db=None behavior unchanged
+"""
+
+import sys
+import time
+
+
+def test_lazy_import_no_heavy_deps():
+    """Test that importing praisonaiagents.db does not import DB libs."""
+    print("=== Test: Lazy Import Verification ===")
+    
+    # Record modules before import
+    before_modules = set(sys.modules.keys())
+    
+    # Import only the db module (not full praisonaiagents which loads Knowledge)
+    from praisonaiagents.db import DbAdapter, DbMessage
+    
+    # Check that heavy DB libs are NOT loaded by db module itself
+    # Note: praisonaiagents main package loads qdrant/chroma via Knowledge
+    # but the db module should not add any additional heavy deps
+    heavy_libs = ['psycopg2', 'pymysql', 'redis']  # DB-specific libs
+    loaded_heavy = [lib for lib in heavy_libs if lib in sys.modules]
+    
+    if loaded_heavy:
+        print(f"  ❌ Heavy DB libs loaded by db module: {loaded_heavy}")
+        return False
+    
+    print("  ✅ No heavy DB libs loaded by praisonaiagents.db module")
+    return True
+
+
+def test_agent_accepts_db_parameter():
+    """Test that Agent accepts db and session_id parameters."""
+    print("\n=== Test: Agent Accepts db Parameter ===")
+    
+    import inspect
+    from praisonaiagents import Agent
+    
+    sig = inspect.signature(Agent.__init__)
+    params = list(sig.parameters.keys())
+    
+    if 'db' not in params:
+        print("  ❌ Agent does not accept 'db' parameter")
+        return False
+    
+    if 'session_id' not in params:
+        print("  ❌ Agent does not accept 'session_id' parameter")
+        return False
+    
+    print("  ✅ Agent accepts db and session_id parameters")
+    return True
+
+
+class MockDbAdapter:
+    """Mock DB adapter for testing hook lifecycle."""
+    
+    def __init__(self):
+        self.calls = []
+        self.history_to_return = []
+    
+    def on_agent_start(self, agent_name, session_id, user_id=None, metadata=None):
+        self.calls.append(('on_agent_start', agent_name, session_id))
+        return self.history_to_return
+    
+    def on_user_message(self, session_id, content, metadata=None):
+        self.calls.append(('on_user_message', session_id, content))
+    
+    def on_agent_message(self, session_id, content, metadata=None):
+        self.calls.append(('on_agent_message', session_id, content))
+    
+    def on_tool_call(self, session_id, tool_name, args, result, metadata=None):
+        self.calls.append(('on_tool_call', session_id, tool_name))
+    
+    def on_agent_end(self, session_id, metadata=None):
+        self.calls.append(('on_agent_end', session_id))
+    
+    def close(self):
+        self.calls.append(('close',))
+
+
+def test_db_hook_lifecycle():
+    """Test that Agent calls db hooks in correct order."""
+    print("\n=== Test: DB Hook Lifecycle ===")
+    
+    from praisonaiagents import Agent
+    
+    mock_db = MockDbAdapter()
+    
+    # Create agent with mock db
+    agent = Agent(
+        name="TestAgent",
+        instructions="Test agent",
+        db=mock_db,
+        session_id="test-session-001",
+        verbose=False
+    )
+    
+    # Verify db is stored
+    if agent._db is not mock_db:
+        print("  ❌ Agent did not store db adapter")
+        return False
+    
+    if agent._session_id != "test-session-001":
+        print("  ❌ Agent did not store session_id")
+        return False
+    
+    print("  ✅ Agent stores db and session_id correctly")
+    return True
+
+
+def test_db_none_unchanged_behavior():
+    """Test that db=None behavior is unchanged."""
+    print("\n=== Test: db=None Unchanged Behavior ===")
+    
+    from praisonaiagents import Agent
+    
+    # Create agent without db
+    agent = Agent(
+        name="TestAgent",
+        instructions="Test agent",
+        verbose=False
+    )
+    
+    # Verify db is None
+    if agent._db is not None:
+        print("  ❌ Agent._db should be None when not provided")
+        return False
+    
+    # Verify _init_db_session does nothing when db is None
+    agent._init_db_session()
+    
+    if agent._db_initialized:
+        print("  ❌ _db_initialized should remain False when db is None")
+        return False
+    
+    print("  ✅ db=None behavior unchanged")
+    return True
+
+
+def test_session_id_auto_generation():
+    """Test that session_id is auto-generated if not provided."""
+    print("\n=== Test: Session ID Auto-Generation ===")
+    
+    from praisonaiagents import Agent
+    
+    mock_db = MockDbAdapter()
+    
+    agent = Agent(
+        name="TestAgent",
+        instructions="Test agent",
+        db=mock_db,
+        # No session_id provided
+        verbose=False
+    )
+    
+    # Trigger db initialization
+    agent._init_db_session()
+    
+    if agent._session_id is None:
+        print("  ❌ session_id should be auto-generated")
+        return False
+    
+    if not agent._session_id.startswith("session-"):
+        print(f"  ❌ session_id format unexpected: {agent._session_id}")
+        return False
+    
+    print(f"  ✅ session_id auto-generated: {agent._session_id}")
+    return True
+
+
+def test_protocol_compliance():
+    """Test that DbAdapter protocol is properly defined."""
+    print("\n=== Test: DbAdapter Protocol Compliance ===")
+    
+    from praisonaiagents.db.protocol import DbAdapter, DbMessage
+    
+    # Check DbMessage fields
+    msg = DbMessage(role="user", content="test")
+    if msg.role != "user" or msg.content != "test":
+        print("  ❌ DbMessage fields not working")
+        return False
+    
+    # Check protocol methods
+    required_methods = [
+        'on_agent_start',
+        'on_user_message', 
+        'on_agent_message',
+        'on_tool_call',
+        'on_agent_end',
+        'close'
+    ]
+    
+    mock = MockDbAdapter()
+    for method in required_methods:
+        if not hasattr(mock, method):
+            print(f"  ❌ MockDbAdapter missing {method}")
+            return False
+    
+    # Check protocol isinstance
+    if not isinstance(mock, DbAdapter):
+        print("  ❌ MockDbAdapter should implement DbAdapter protocol")
+        return False
+    
+    print("  ✅ DbAdapter protocol properly defined")
+    return True
+
+
+def run_all_tests():
+    """Run all TDD tests."""
+    print("=" * 60)
+    print("PraisonAI Agents DB Integration - TDD Tests")
+    print("=" * 60)
+    
+    results = []
+    
+    results.append(("Lazy Import", test_lazy_import_no_heavy_deps()))
+    results.append(("Agent db Parameter", test_agent_accepts_db_parameter()))
+    results.append(("DB Hook Lifecycle", test_db_hook_lifecycle()))
+    results.append(("db=None Unchanged", test_db_none_unchanged_behavior()))
+    results.append(("Session ID Auto-Gen", test_session_id_auto_generation()))
+    results.append(("Protocol Compliance", test_protocol_compliance()))
+    
+    print("\n" + "=" * 60)
+    passed = sum(1 for _, r in results if r)
+    total = len(results)
+    print(f"Results: {passed}/{total} tests passed")
+    
+    for name, result in results:
+        status = "✅" if result else "❌"
+        print(f"  {status} {name}")
+    
+    print("=" * 60)
+    
+    return passed == total
+
+
+if __name__ == "__main__":
+    success = run_all_tests()
+    sys.exit(0 if success else 1)

src/praisonai-agents/praisonaiagents/hooks/__init__.py

@@ -65,6 +65,23 @@ __all__ = [
     "AfterAgentInput",
     "SessionStartInput",
     "SessionEndInput",
+    # Middleware types
+    "InvocationContext",
+    "ModelRequest",
+    "ModelResponse",
+    "ToolRequest",
+    "ToolResponse",
+    # Middleware decorators
+    "before_model",
+    "after_model",
+    "wrap_model_call",
+    "before_tool",
+    "after_tool",
+    "wrap_tool_call",
+    # Middleware utilities
+    "MiddlewareChain",
+    "AsyncMiddlewareChain",
+    "MiddlewareManager",
 ]
 
 
@@ -94,4 +111,27 @@ def __getattr__(name: str):
         )
         return locals()[name]
     
+    # Middleware types
+    if name in ("InvocationContext", "ModelRequest", "ModelResponse", 
+                "ToolRequest", "ToolResponse"):
+        from .middleware import (
+            InvocationContext, ModelRequest, ModelResponse,
+            ToolRequest, ToolResponse
+        )
+        return locals()[name]
+    
+    # Middleware decorators
+    if name in ("before_model", "after_model", "wrap_model_call",
+                "before_tool", "after_tool", "wrap_tool_call"):
+        from .middleware import (
+            before_model, after_model, wrap_model_call,
+            before_tool, after_tool, wrap_tool_call
+        )
+        return locals()[name]
+    
+    # Middleware utilities
+    if name in ("MiddlewareChain", "AsyncMiddlewareChain", "MiddlewareManager"):
+        from .middleware import MiddlewareChain, AsyncMiddlewareChain, MiddlewareManager
+        return locals()[name]
+    
     raise AttributeError(f"module {__name__!r} has no attribute {name!r}")

src/praisonai-agents/praisonaiagents/hooks/middleware.py

@@ -0,0 +1,439 @@
+"""
+Middleware System for PraisonAI Agents.
+
+Provides wrap_model_call and wrap_tool_call patterns for intercepting
+and modifying agent behavior at model/tool call boundaries.
+
+Zero Performance Impact:
+- All middleware is optional
+- Empty middleware chain is O(1) fast path
+- No overhead when middleware not registered
+
+Usage:
+    from praisonaiagents.hooks import before_model, wrap_model_call, wrap_tool_call
+    
+    @before_model
+    def add_context(request):
+        request.messages.append({"role": "system", "content": "Extra"})
+        return request
+    
+    @wrap_model_call
+    def retry_on_error(request, call_next):
+        for _ in range(3):
+            try:
+                return call_next(request)
+            except Exception:
+                pass
+        raise RuntimeError("All retries failed")
+    
+    agent = Agent(name="Test", hooks=[add_context, retry_on_error])
+"""
+
+from dataclasses import dataclass, field
+from typing import Any, Callable, Dict, List, Optional, TypeVar, Generic
+from functools import wraps
+
+
+@dataclass
+class InvocationContext:
+    """Context passed through middleware chain.
+    
+    Contains identifiers for multi-agent safety and optional metadata.
+    """
+    agent_id: str
+    run_id: str
+    session_id: str
+    tool_name: Optional[str] = None
+    model_name: Optional[str] = None
+    metadata: Dict[str, Any] = field(default_factory=dict)
+
+
+@dataclass
+class ModelRequest:
+    """Request data for model calls."""
+    messages: List[Dict[str, Any]]
+    model: str
+    temperature: float = 1.0
+    context: Optional[InvocationContext] = None
+    tools: Optional[List[Any]] = None
+    extra: Dict[str, Any] = field(default_factory=dict)
+
+
+@dataclass
+class ModelResponse:
+    """Response data from model calls."""
+    content: str
+    model: str
+    usage: Optional[Dict[str, int]] = None
+    context: Optional[InvocationContext] = None
+    tool_calls: Optional[List[Any]] = None
+    extra: Dict[str, Any] = field(default_factory=dict)
+
+
+@dataclass
+class ToolRequest:
+    """Request data for tool calls."""
+    tool_name: str
+    arguments: Dict[str, Any]
+    context: Optional[InvocationContext] = None
+    extra: Dict[str, Any] = field(default_factory=dict)
+
+
+@dataclass
+class ToolResponse:
+    """Response data from tool calls."""
+    tool_name: str
+    result: Any
+    error: Optional[str] = None
+    context: Optional[InvocationContext] = None
+    extra: Dict[str, Any] = field(default_factory=dict)
+
+
+# Type aliases for middleware functions
+T = TypeVar('T')
+BeforeModelFn = Callable[[ModelRequest], ModelRequest]
+AfterModelFn = Callable[[ModelResponse], ModelResponse]
+WrapModelCallFn = Callable[[ModelRequest, Callable[[ModelRequest], ModelResponse]], ModelResponse]
+BeforeToolFn = Callable[[ToolRequest], ToolRequest]
+AfterToolFn = Callable[[ToolResponse], ToolResponse]
+WrapToolCallFn = Callable[[ToolRequest, Callable[[ToolRequest], ToolResponse]], ToolResponse]
+
+
+class MiddlewareChain:
+    """Synchronous middleware chain executor.
+    
+    Composes middleware functions in registration order.
+    First registered middleware wraps all subsequent ones.
+    """
+    
+    def __init__(self, middlewares: Optional[List[Callable]] = None):
+        self._middlewares = middlewares or []
+    
+    def execute(self, request: Any, final_handler: Callable[[Any], Any]) -> Any:
+        """Execute the middleware chain with a final handler.
+        
+        Args:
+            request: The request object to pass through
+            final_handler: The actual operation to perform
+            
+        Returns:
+            The result from the chain
+        """
+        if not self._middlewares:
+            return final_handler(request)
+        
+        # Build the chain from inside out
+        # Last middleware wraps final_handler
+        # First middleware is outermost
+        def build_chain(index: int) -> Callable[[Any], Any]:
+            if index >= len(self._middlewares):
+                return final_handler
+            
+            middleware = self._middlewares[index]
+            next_handler = build_chain(index + 1)
+            
+            def handler(req: Any) -> Any:
+                return middleware(req, next_handler)
+            
+            return handler
+        
+        chain = build_chain(0)
+        return chain(request)
+    
+    def __len__(self) -> int:
+        return len(self._middlewares)
+
+
+class AsyncMiddlewareChain:
+    """Asynchronous middleware chain executor.
+    
+    Same as MiddlewareChain but for async middleware and handlers.
+    """
+    
+    def __init__(self, middlewares: Optional[List[Callable]] = None):
+        self._middlewares = middlewares or []
+    
+    async def execute(self, request: Any, final_handler: Callable) -> Any:
+        """Execute the async middleware chain.
+        
+        Args:
+            request: The request object to pass through
+            final_handler: The actual async operation to perform
+            
+        Returns:
+            The result from the chain
+        """
+        if not self._middlewares:
+            return await final_handler(request)
+        
+        # Build the chain from inside out
+        def build_chain(index: int) -> Callable:
+            if index >= len(self._middlewares):
+                return final_handler
+            
+            middleware = self._middlewares[index]
+            next_handler = build_chain(index + 1)
+            
+            async def handler(req: Any) -> Any:
+                return await middleware(req, next_handler)
+            
+            return handler
+        
+        chain = build_chain(0)
+        return await chain(request)
+    
+    def __len__(self) -> int:
+        return len(self._middlewares)
+
+
+# Decorator functions for tagging middleware
+
+def before_model(func: BeforeModelFn) -> BeforeModelFn:
+    """Decorator to mark a function as a before_model hook.
+    
+    The function receives a ModelRequest and should return a (possibly modified) ModelRequest.
+    
+    Example:
+        @before_model
+        def add_context(request):
+            request.messages.append({"role": "system", "content": "Extra"})
+            return request
+    """
+    func._hook_type = 'before_model'
+    return func
+
+
+def after_model(func: AfterModelFn) -> AfterModelFn:
+    """Decorator to mark a function as an after_model hook.
+    
+    The function receives a ModelResponse and should return a (possibly modified) ModelResponse.
+    
+    Example:
+        @after_model
+        def log_response(response):
+            print(f"Model returned: {response.content[:50]}")
+            return response
+    """
+    func._hook_type = 'after_model'
+    return func
+
+
+def wrap_model_call(func: WrapModelCallFn) -> WrapModelCallFn:
+    """Decorator to mark a function as a wrap_model_call middleware.
+    
+    The function receives (request, call_next) and should call call_next(request)
+    to continue the chain, or return early to short-circuit.
+    
+    Example:
+        @wrap_model_call
+        def retry_on_error(request, call_next):
+            for _ in range(3):
+                try:
+                    return call_next(request)
+                except Exception:
+                    pass
+            raise RuntimeError("All retries failed")
+    """
+    func._hook_type = 'wrap_model_call'
+    return func
+
+
+def before_tool(func: BeforeToolFn) -> BeforeToolFn:
+    """Decorator to mark a function as a before_tool hook.
+    
+    The function receives a ToolRequest and should return a (possibly modified) ToolRequest.
+    
+    Example:
+        @before_tool
+        def validate_args(request):
+            if 'dangerous' in request.arguments:
+                raise ValueError("Dangerous argument detected")
+            return request
+    """
+    func._hook_type = 'before_tool'
+    return func
+
+
+def after_tool(func: AfterToolFn) -> AfterToolFn:
+    """Decorator to mark a function as an after_tool hook.
+    
+    The function receives a ToolResponse and should return a (possibly modified) ToolResponse.
+    
+    Example:
+        @after_tool
+        def log_result(response):
+            print(f"Tool {response.tool_name} returned: {response.result}")
+            return response
+    """
+    func._hook_type = 'after_tool'
+    return func
+
+
+def wrap_tool_call(func: WrapToolCallFn) -> WrapToolCallFn:
+    """Decorator to mark a function as a wrap_tool_call middleware.
+    
+    The function receives (request, call_next) and should call call_next(request)
+    to continue the chain, or return early to short-circuit.
+    
+    Example:
+        @wrap_tool_call
+        def retry_flaky_tool(request, call_next):
+            last_error = None
+            for _ in range(3):
+                try:
+                    return call_next(request)
+                except Exception as e:
+                    last_error = e
+            raise last_error
+    """
+    func._hook_type = 'wrap_tool_call'
+    return func
+
+
+def get_hook_type(func: Callable) -> Optional[str]:
+    """Get the hook type of a decorated function."""
+    return getattr(func, '_hook_type', None)
+
+
+def is_middleware(func: Callable) -> bool:
+    """Check if a function is a middleware (wrap_* type)."""
+    hook_type = get_hook_type(func)
+    return hook_type in ('wrap_model_call', 'wrap_tool_call')
+
+
+def categorize_hooks(hooks: List[Callable]) -> Dict[str, List[Callable]]:
+    """Categorize hooks by their type.
+    
+    Returns:
+        Dict with keys: before_model, after_model, wrap_model_call,
+                       before_tool, after_tool, wrap_tool_call
+    """
+    categories = {
+        'before_model': [],
+        'after_model': [],
+        'wrap_model_call': [],
+        'before_tool': [],
+        'after_tool': [],
+        'wrap_tool_call': [],
+    }
+    
+    for hook in hooks or []:
+        hook_type = get_hook_type(hook)
+        if hook_type and hook_type in categories:
+            categories[hook_type].append(hook)
+    
+    return categories
+
+
+class MiddlewareManager:
+    """Manages middleware for an agent.
+    
+    Provides methods to execute before/after hooks and middleware chains
+    for both model and tool calls.
+    """
+    
+    def __init__(self, hooks: Optional[List[Callable]] = None):
+        self._hooks = hooks or []
+        self._categorized = categorize_hooks(self._hooks)
+        
+        # Pre-build chains for efficiency
+        self._model_chain = MiddlewareChain(self._categorized['wrap_model_call'])
+        self._tool_chain = MiddlewareChain(self._categorized['wrap_tool_call'])
+    
+    @property
+    def has_model_hooks(self) -> bool:
+        """Check if any model hooks are registered."""
+        return bool(
+            self._categorized['before_model'] or
+            self._categorized['after_model'] or
+            self._categorized['wrap_model_call']
+        )
+    
+    @property
+    def has_tool_hooks(self) -> bool:
+        """Check if any tool hooks are registered."""
+        return bool(
+            self._categorized['before_tool'] or
+            self._categorized['after_tool'] or
+            self._categorized['wrap_tool_call']
+        )
+    
+    def run_before_model(self, request: ModelRequest) -> ModelRequest:
+        """Run all before_model hooks."""
+        for hook in self._categorized['before_model']:
+            request = hook(request)
+        return request
+    
+    def run_after_model(self, response: ModelResponse) -> ModelResponse:
+        """Run all after_model hooks (in reverse order)."""
+        for hook in reversed(self._categorized['after_model']):
+            response = hook(response)
+        return response
+    
+    def run_before_tool(self, request: ToolRequest) -> ToolRequest:
+        """Run all before_tool hooks."""
+        for hook in self._categorized['before_tool']:
+            request = hook(request)
+        return request
+    
+    def run_after_tool(self, response: ToolResponse) -> ToolResponse:
+        """Run all after_tool hooks (in reverse order)."""
+        for hook in reversed(self._categorized['after_tool']):
+            response = hook(response)
+        return response
+    
+    def execute_model_call(
+        self,
+        request: ModelRequest,
+        model_fn: Callable[[ModelRequest], ModelResponse]
+    ) -> ModelResponse:
+        """Execute a model call with all hooks and middleware.
+        
+        Order:
+        1. before_model hooks (in order)
+        2. wrap_model_call middleware chain
+        3. actual model call
+        4. after_model hooks (in reverse order)
+        """
+        # Fast path: no hooks
+        if not self.has_model_hooks:
+            return model_fn(request)
+        
+        # Run before hooks
+        request = self.run_before_model(request)
+        
+        # Build wrapped handler that includes after hooks
+        def wrapped_model_fn(req: ModelRequest) -> ModelResponse:
+            response = model_fn(req)
+            return self.run_after_model(response)
+        
+        # Execute through middleware chain
+        return self._model_chain.execute(request, wrapped_model_fn)
+    
+    def execute_tool_call(
+        self,
+        request: ToolRequest,
+        tool_fn: Callable[[ToolRequest], ToolResponse]
+    ) -> ToolResponse:
+        """Execute a tool call with all hooks and middleware.
+        
+        Order:
+        1. before_tool hooks (in order)
+        2. wrap_tool_call middleware chain
+        3. actual tool call
+        4. after_tool hooks (in reverse order)
+        """
+        # Fast path: no hooks
+        if not self.has_tool_hooks:
+            return tool_fn(request)
+        
+        # Run before hooks
+        request = self.run_before_tool(request)
+        
+        # Build wrapped handler that includes after hooks
+        def wrapped_tool_fn(req: ToolRequest) -> ToolResponse:
+            response = tool_fn(req)
+            return self.run_after_tool(response)
+        
+        # Execute through middleware chain
+        return self._tool_chain.execute(request, wrapped_tool_fn)

src/praisonai-agents/praisonaiagents/knowledge/__init__.py

@@ -1,8 +1,130 @@
 """
-PraisonAI Knowledge - Advanced knowledge management system with configurable features
+PraisonAI Knowledge - Advanced knowledge management system with configurable features.
+
+This module provides:
+- Document readers (ReaderProtocol, ReaderRegistry)
+- Vector stores (VectorStoreProtocol, VectorStoreRegistry)
+- Retrieval strategies (RetrieverProtocol, RetrievalStrategy)
+- Rerankers (RerankerProtocol, RerankerRegistry)
+- Index types (IndexProtocol, IndexType)
+- Query engines (QueryEngineProtocol, QueryMode)
 """
 
+# Core Knowledge class (always available)
 from praisonaiagents.knowledge.knowledge import Knowledge
 from praisonaiagents.knowledge.chunking import Chunking
 
-__all__ = ["Knowledge", "Chunking"] 
+# Lazy loading for protocols and registries to avoid import overhead
+_LAZY_IMPORTS = {
+    # Readers
+    "Document": ("praisonaiagents.knowledge.readers", "Document"),
+    "ReaderProtocol": ("praisonaiagents.knowledge.readers", "ReaderProtocol"),
+    "ReaderRegistry": ("praisonaiagents.knowledge.readers", "ReaderRegistry"),
+    "get_reader_registry": ("praisonaiagents.knowledge.readers", "get_reader_registry"),
+    "detect_source_kind": ("praisonaiagents.knowledge.readers", "detect_source_kind"),
+    
+    # Vector stores
+    "VectorRecord": ("praisonaiagents.knowledge.vector_store", "VectorRecord"),
+    "VectorStoreProtocol": ("praisonaiagents.knowledge.vector_store", "VectorStoreProtocol"),
+    "VectorStoreRegistry": ("praisonaiagents.knowledge.vector_store", "VectorStoreRegistry"),
+    "get_vector_store_registry": ("praisonaiagents.knowledge.vector_store", "get_vector_store_registry"),
+    "InMemoryVectorStore": ("praisonaiagents.knowledge.vector_store", "InMemoryVectorStore"),
+    
+    # Retrieval
+    "RetrievalResult": ("praisonaiagents.knowledge.retrieval", "RetrievalResult"),
+    "RetrievalStrategy": ("praisonaiagents.knowledge.retrieval", "RetrievalStrategy"),
+    "RetrieverProtocol": ("praisonaiagents.knowledge.retrieval", "RetrieverProtocol"),
+    "RetrieverRegistry": ("praisonaiagents.knowledge.retrieval", "RetrieverRegistry"),
+    "get_retriever_registry": ("praisonaiagents.knowledge.retrieval", "get_retriever_registry"),
+    "reciprocal_rank_fusion": ("praisonaiagents.knowledge.retrieval", "reciprocal_rank_fusion"),
+    "merge_adjacent_chunks": ("praisonaiagents.knowledge.retrieval", "merge_adjacent_chunks"),
+    
+    # Rerankers
+    "RerankResult": ("praisonaiagents.knowledge.rerankers", "RerankResult"),
+    "RerankerProtocol": ("praisonaiagents.knowledge.rerankers", "RerankerProtocol"),
+    "RerankerRegistry": ("praisonaiagents.knowledge.rerankers", "RerankerRegistry"),
+    "get_reranker_registry": ("praisonaiagents.knowledge.rerankers", "get_reranker_registry"),
+    "SimpleReranker": ("praisonaiagents.knowledge.rerankers", "SimpleReranker"),
+    
+    # Index
+    "IndexType": ("praisonaiagents.knowledge.index", "IndexType"),
+    "IndexStats": ("praisonaiagents.knowledge.index", "IndexStats"),
+    "IndexProtocol": ("praisonaiagents.knowledge.index", "IndexProtocol"),
+    "IndexRegistry": ("praisonaiagents.knowledge.index", "IndexRegistry"),
+    "get_index_registry": ("praisonaiagents.knowledge.index", "get_index_registry"),
+    "KeywordIndex": ("praisonaiagents.knowledge.index", "KeywordIndex"),
+    
+    # Query engine
+    "QueryMode": ("praisonaiagents.knowledge.query_engine", "QueryMode"),
+    "QueryResult": ("praisonaiagents.knowledge.query_engine", "QueryResult"),
+    "QueryEngineProtocol": ("praisonaiagents.knowledge.query_engine", "QueryEngineProtocol"),
+    "QueryEngineRegistry": ("praisonaiagents.knowledge.query_engine", "QueryEngineRegistry"),
+    "get_query_engine_registry": ("praisonaiagents.knowledge.query_engine", "get_query_engine_registry"),
+    "decompose_question": ("praisonaiagents.knowledge.query_engine", "decompose_question"),
+    "SimpleQueryEngine": ("praisonaiagents.knowledge.query_engine", "SimpleQueryEngine"),
+    "SubQuestionEngine": ("praisonaiagents.knowledge.query_engine", "SubQuestionEngine"),
+}
+
+
+def __getattr__(name: str):
+    """Lazy load protocols and registries."""
+    if name in _LAZY_IMPORTS:
+        module_path, attr_name = _LAZY_IMPORTS[name]
+        import importlib
+        module = importlib.import_module(module_path)
+        return getattr(module, attr_name)
+    raise AttributeError(f"module {__name__!r} has no attribute {name!r}")
+
+
+def __dir__():
+    """List available attributes."""
+    return list(_LAZY_IMPORTS.keys()) + ["Knowledge", "Chunking"]
+
+
+__all__ = [
+    # Core
+    "Knowledge",
+    "Chunking",
+    # Readers
+    "Document",
+    "ReaderProtocol",
+    "ReaderRegistry",
+    "get_reader_registry",
+    "detect_source_kind",
+    # Vector stores
+    "VectorRecord",
+    "VectorStoreProtocol",
+    "VectorStoreRegistry",
+    "get_vector_store_registry",
+    "InMemoryVectorStore",
+    # Retrieval
+    "RetrievalResult",
+    "RetrievalStrategy",
+    "RetrieverProtocol",
+    "RetrieverRegistry",
+    "get_retriever_registry",
+    "reciprocal_rank_fusion",
+    "merge_adjacent_chunks",
+    # Rerankers
+    "RerankResult",
+    "RerankerProtocol",
+    "RerankerRegistry",
+    "get_reranker_registry",
+    "SimpleReranker",
+    # Index
+    "IndexType",
+    "IndexStats",
+    "IndexProtocol",
+    "IndexRegistry",
+    "get_index_registry",
+    "KeywordIndex",
+    # Query engine
+    "QueryMode",
+    "QueryResult",
+    "QueryEngineProtocol",
+    "QueryEngineRegistry",
+    "get_query_engine_registry",
+    "decompose_question",
+    "SimpleQueryEngine",
+    "SubQuestionEngine",
+] 

src/praisonai-agents/praisonaiagents/knowledge/index.py

@@ -0,0 +1,332 @@
+"""
+Index Type Selection for PraisonAI Agents.
+
+This module defines index types and protocols.
+NO heavy imports - only stdlib and typing.
+
+Implementations are provided by the wrapper layer.
+"""
+
+from dataclasses import dataclass, field
+from typing import Any, Callable, Dict, List, Optional, Protocol, runtime_checkable
+from enum import Enum
+import logging
+import math
+
+logger = logging.getLogger(__name__)
+
+
+class IndexType(str, Enum):
+    """Available index types."""
+    VECTOR = "vector"
+    KEYWORD = "keyword"
+    HYBRID = "hybrid"
+    GRAPH = "graph"
+
+
+@dataclass
+class IndexStats:
+    """Statistics about an index."""
+    index_type: IndexType
+    document_count: int
+    chunk_count: int
+    total_tokens: int = 0
+    metadata: Dict[str, Any] = field(default_factory=dict)
+
+
+@runtime_checkable
+class IndexProtocol(Protocol):
+    """
+    Protocol for document indices.
+    
+    Implementations must provide methods for indexing and querying.
+    """
+    
+    name: str
+    index_type: IndexType
+    
+    def add_documents(
+        self,
+        texts: List[str],
+        metadatas: Optional[List[Dict[str, Any]]] = None,
+        ids: Optional[List[str]] = None,
+        embeddings: Optional[List[List[float]]] = None
+    ) -> List[str]:
+        """
+        Add documents to the index.
+        
+        Args:
+            texts: List of text content
+            metadatas: Optional list of metadata dicts
+            ids: Optional list of IDs
+            embeddings: Optional pre-computed embeddings (for vector index)
+            
+        Returns:
+            List of document IDs
+        """
+        ...
+    
+    def query(
+        self,
+        query: str,
+        top_k: int = 10,
+        filter: Optional[Dict[str, Any]] = None,
+        query_embedding: Optional[List[float]] = None
+    ) -> List[Dict[str, Any]]:
+        """
+        Query the index.
+        
+        Args:
+            query: Search query
+            top_k: Number of results
+            filter: Optional metadata filter
+            query_embedding: Optional pre-computed query embedding
+            
+        Returns:
+            List of results with text, score, metadata
+        """
+        ...
+    
+    def delete(
+        self,
+        ids: Optional[List[str]] = None,
+        filter: Optional[Dict[str, Any]] = None,
+        delete_all: bool = False
+    ) -> int:
+        """Delete documents from the index."""
+        ...
+    
+    def stats(self) -> IndexStats:
+        """Get index statistics."""
+        ...
+
+
+class IndexRegistry:
+    """Registry for index types."""
+    
+    _instance: Optional["IndexRegistry"] = None
+    
+    def __new__(cls) -> "IndexRegistry":
+        if cls._instance is None:
+            cls._instance = super().__new__(cls)
+            cls._instance._indices: Dict[str, Callable[..., IndexProtocol]] = {}
+        return cls._instance
+    
+    def register(self, name: str, factory: Callable[..., IndexProtocol]) -> None:
+        """Register an index factory."""
+        self._indices[name] = factory
+    
+    def get(self, name: str, **kwargs) -> Optional[IndexProtocol]:
+        """Get an index by name."""
+        if name in self._indices:
+            try:
+                return self._indices[name](**kwargs)
+            except Exception as e:
+                logger.warning(f"Failed to initialize index '{name}': {e}")
+                return None
+        return None
+    
+    def list_indices(self) -> List[str]:
+        """List all registered index names."""
+        return list(self._indices.keys())
+    
+    def clear(self) -> None:
+        """Clear all registered indices."""
+        self._indices.clear()
+
+
+def get_index_registry() -> IndexRegistry:
+    """Get the global index registry instance."""
+    return IndexRegistry()
+
+
+class KeywordIndex:
+    """
+    Simple BM25-like keyword index.
+    
+    Pure Python implementation with no external dependencies.
+    """
+    
+    name: str = "keyword"
+    index_type: IndexType = IndexType.KEYWORD
+    
+    def __init__(self, k1: float = 1.5, b: float = 0.75, **kwargs):
+        self.k1 = k1
+        self.b = b
+        self._documents: Dict[str, Dict[str, Any]] = {}  # id -> {text, metadata, terms}
+        self._term_doc_freq: Dict[str, int] = {}  # term -> doc count
+        self._avg_doc_len: float = 0.0
+        self._total_docs: int = 0
+    
+    def add_documents(
+        self,
+        texts: List[str],
+        metadatas: Optional[List[Dict[str, Any]]] = None,
+        ids: Optional[List[str]] = None,
+        embeddings: Optional[List[List[float]]] = None
+    ) -> List[str]:
+        """Add documents to the keyword index."""
+        import uuid
+        
+        metadatas = metadatas or [{} for _ in texts]
+        ids = ids or [str(uuid.uuid4()) for _ in texts]
+        
+        total_len = sum(len(doc["text"].split()) for doc in self._documents.values())
+        
+        for text, metadata, doc_id in zip(texts, metadatas, ids):
+            terms = self._tokenize(text)
+            term_freq = {}
+            for term in terms:
+                term_freq[term] = term_freq.get(term, 0) + 1
+            
+            self._documents[doc_id] = {
+                "text": text,
+                "metadata": metadata,
+                "terms": term_freq,
+                "length": len(terms)
+            }
+            
+            # Update document frequency
+            for term in set(terms):
+                self._term_doc_freq[term] = self._term_doc_freq.get(term, 0) + 1
+            
+            total_len += len(terms)
+        
+        self._total_docs = len(self._documents)
+        self._avg_doc_len = total_len / self._total_docs if self._total_docs > 0 else 0
+        
+        return ids
+    
+    def query(
+        self,
+        query: str,
+        top_k: int = 10,
+        filter: Optional[Dict[str, Any]] = None,
+        query_embedding: Optional[List[float]] = None
+    ) -> List[Dict[str, Any]]:
+        """Query using BM25 scoring."""
+        query_terms = self._tokenize(query)
+        
+        scores = []
+        for doc_id, doc in self._documents.items():
+            # Apply filter
+            if filter:
+                match = True
+                for key, value in filter.items():
+                    if doc["metadata"].get(key) != value:
+                        match = False
+                        break
+                if not match:
+                    continue
+            
+            score = self._bm25_score(query_terms, doc)
+            scores.append({
+                "id": doc_id,
+                "text": doc["text"],
+                "score": score,
+                "metadata": doc["metadata"]
+            })
+        
+        # Sort by score
+        scores.sort(key=lambda x: x["score"], reverse=True)
+        return scores[:top_k]
+    
+    def delete(
+        self,
+        ids: Optional[List[str]] = None,
+        filter: Optional[Dict[str, Any]] = None,
+        delete_all: bool = False
+    ) -> int:
+        """Delete documents from the index."""
+        if delete_all:
+            count = len(self._documents)
+            self._documents.clear()
+            self._term_doc_freq.clear()
+            self._total_docs = 0
+            self._avg_doc_len = 0
+            return count
+        
+        deleted = 0
+        to_delete = []
+        
+        if ids:
+            to_delete = [id_ for id_ in ids if id_ in self._documents]
+        elif filter:
+            for doc_id, doc in self._documents.items():
+                match = True
+                for key, value in filter.items():
+                    if doc["metadata"].get(key) != value:
+                        match = False
+                        break
+                if match:
+                    to_delete.append(doc_id)
+        
+        for doc_id in to_delete:
+            doc = self._documents[doc_id]
+            # Update term frequencies
+            for term in doc["terms"]:
+                if term in self._term_doc_freq:
+                    self._term_doc_freq[term] -= 1
+                    if self._term_doc_freq[term] <= 0:
+                        del self._term_doc_freq[term]
+            del self._documents[doc_id]
+            deleted += 1
+        
+        self._total_docs = len(self._documents)
+        if self._total_docs > 0:
+            self._avg_doc_len = sum(d["length"] for d in self._documents.values()) / self._total_docs
+        else:
+            self._avg_doc_len = 0
+        
+        return deleted
+    
+    def stats(self) -> IndexStats:
+        """Get index statistics."""
+        return IndexStats(
+            index_type=self.index_type,
+            document_count=self._total_docs,
+            chunk_count=self._total_docs,
+            total_tokens=sum(d["length"] for d in self._documents.values()),
+            metadata={"unique_terms": len(self._term_doc_freq)}
+        )
+    
+    def _tokenize(self, text: str) -> List[str]:
+        """Simple tokenization."""
+        import re
+        # Lowercase and split on non-alphanumeric
+        tokens = re.findall(r'\b\w+\b', text.lower())
+        # Remove very short tokens
+        return [t for t in tokens if len(t) > 1]
+    
+    def _bm25_score(self, query_terms: List[str], doc: Dict[str, Any]) -> float:
+        """Calculate BM25 score."""
+        score = 0.0
+        doc_len = doc["length"]
+        term_freq = doc["terms"]
+        
+        for term in query_terms:
+            if term not in term_freq:
+                continue
+            
+            tf = term_freq[term]
+            df = self._term_doc_freq.get(term, 0)
+            
+            # IDF component
+            idf = math.log((self._total_docs - df + 0.5) / (df + 0.5) + 1)
+            
+            # TF component with length normalization
+            tf_norm = (tf * (self.k1 + 1)) / (tf + self.k1 * (1 - self.b + self.b * doc_len / self._avg_doc_len))
+            
+            score += idf * tf_norm
+        
+        return score
+
+
+# Register the keyword index by default
+def _register_default_indices():
+    """Register default indices."""
+    registry = get_index_registry()
+    registry.register("keyword", KeywordIndex)
+
+
+_register_default_indices()

src/praisonai-agents/praisonaiagents/knowledge/query_engine.py

@@ -0,0 +1,351 @@
+"""
+Query Engine Patterns for PraisonAI Agents.
+
+This module defines query modes and protocols.
+NO heavy imports - only stdlib and typing.
+
+Implementations are provided by the wrapper layer.
+"""
+
+from dataclasses import dataclass, field
+from typing import Any, Callable, Dict, List, Optional, Protocol, runtime_checkable
+from enum import Enum
+import logging
+
+logger = logging.getLogger(__name__)
+
+
+class QueryMode(str, Enum):
+    """Available query modes."""
+    DEFAULT = "default"
+    SUB_QUESTION = "sub_question"
+    SQL = "sql"
+    ROUTER = "router"
+    SUMMARIZE = "summarize"
+
+
+@dataclass
+class QueryResult:
+    """
+    Result from a query operation.
+    
+    Attributes:
+        answer: The synthesized answer
+        sources: List of source documents/chunks used
+        sub_questions: Optional list of sub-questions (for sub_question mode)
+        metadata: Additional metadata
+    """
+    answer: str
+    sources: List[Dict[str, Any]] = field(default_factory=list)
+    sub_questions: Optional[List[str]] = None
+    metadata: Dict[str, Any] = field(default_factory=dict)
+    
+    def to_dict(self) -> Dict[str, Any]:
+        """Convert to dictionary."""
+        return {
+            "answer": self.answer,
+            "sources": self.sources,
+            "sub_questions": self.sub_questions,
+            "metadata": self.metadata
+        }
+
+
+@runtime_checkable
+class QueryEngineProtocol(Protocol):
+    """
+    Protocol for query engines.
+    
+    Implementations must provide a query method that takes a question
+    and returns an answer with sources.
+    """
+    
+    name: str
+    mode: QueryMode
+    
+    def query(
+        self,
+        question: str,
+        context: Optional[List[str]] = None,
+        **kwargs
+    ) -> QueryResult:
+        """
+        Query the knowledge base.
+        
+        Args:
+            question: The user's question
+            context: Optional pre-retrieved context
+            **kwargs: Additional engine-specific options
+            
+        Returns:
+            QueryResult with answer and sources
+        """
+        ...
+    
+    async def aquery(
+        self,
+        question: str,
+        context: Optional[List[str]] = None,
+        **kwargs
+    ) -> QueryResult:
+        """Async version of query."""
+        ...
+
+
+class QueryEngineRegistry:
+    """Registry for query engines."""
+    
+    _instance: Optional["QueryEngineRegistry"] = None
+    
+    def __new__(cls) -> "QueryEngineRegistry":
+        if cls._instance is None:
+            cls._instance = super().__new__(cls)
+            cls._instance._engines: Dict[str, Callable[..., QueryEngineProtocol]] = {}
+        return cls._instance
+    
+    def register(self, name: str, factory: Callable[..., QueryEngineProtocol]) -> None:
+        """Register a query engine factory."""
+        self._engines[name] = factory
+    
+    def get(self, name: str, **kwargs) -> Optional[QueryEngineProtocol]:
+        """Get a query engine by name."""
+        if name in self._engines:
+            try:
+                return self._engines[name](**kwargs)
+            except Exception as e:
+                logger.warning(f"Failed to initialize query engine '{name}': {e}")
+                return None
+        return None
+    
+    def list_engines(self) -> List[str]:
+        """List all registered engine names."""
+        return list(self._engines.keys())
+    
+    def clear(self) -> None:
+        """Clear all registered engines."""
+        self._engines.clear()
+
+
+def get_query_engine_registry() -> QueryEngineRegistry:
+    """Get the global query engine registry instance."""
+    return QueryEngineRegistry()
+
+
+def decompose_question(question: str) -> List[str]:
+    """
+    Simple deterministic question decomposition.
+    
+    Splits complex questions into sub-questions without LLM.
+    Used as fallback when LLM is not available.
+    
+    Args:
+        question: The complex question
+        
+    Returns:
+        List of sub-questions
+    """
+    sub_questions = []
+    
+    # Split on common conjunctions
+    conjunctions = [" and ", " also ", " additionally ", " furthermore ", " moreover "]
+    parts = [question]
+    
+    for conj in conjunctions:
+        new_parts = []
+        for part in parts:
+            new_parts.extend(part.split(conj))
+        parts = new_parts
+    
+    # Clean up parts
+    for part in parts:
+        part = part.strip()
+        if part:
+            # Ensure it ends with a question mark
+            if not part.endswith("?"):
+                part = part + "?"
+            # Capitalize first letter
+            part = part[0].upper() + part[1:] if len(part) > 1 else part.upper()
+            sub_questions.append(part)
+    
+    # If no decomposition happened, return original
+    if len(sub_questions) <= 1:
+        return [question]
+    
+    return sub_questions
+
+
+def synthesize_answer(
+    question: str,
+    contexts: List[str],
+    max_context_length: int = 4000
+) -> str:
+    """
+    Simple answer synthesis without LLM.
+    
+    Concatenates relevant context with the question.
+    Used as fallback when LLM is not available.
+    
+    Args:
+        question: The original question
+        contexts: List of context strings
+        max_context_length: Maximum total context length
+        
+    Returns:
+        Synthesized answer string
+    """
+    if not contexts:
+        return "No relevant information found."
+    
+    # Truncate contexts to fit
+    total_length = 0
+    selected_contexts = []
+    
+    for ctx in contexts:
+        if total_length + len(ctx) > max_context_length:
+            remaining = max_context_length - total_length
+            if remaining > 100:
+                selected_contexts.append(ctx[:remaining] + "...")
+            break
+        selected_contexts.append(ctx)
+        total_length += len(ctx)
+    
+    # Format as a simple answer
+    context_text = "\n\n".join(selected_contexts)
+    
+    return f"Based on the available information:\n\n{context_text}"
+
+
+class SimpleQueryEngine:
+    """
+    Simple query engine for testing and fallback.
+    
+    Uses basic context concatenation without LLM synthesis.
+    """
+    
+    name: str = "simple"
+    mode: QueryMode = QueryMode.DEFAULT
+    
+    def __init__(self, **kwargs):
+        pass
+    
+    def query(
+        self,
+        question: str,
+        context: Optional[List[str]] = None,
+        **kwargs
+    ) -> QueryResult:
+        """Simple query that returns context as answer."""
+        contexts = context or []
+        answer = synthesize_answer(question, contexts)
+        
+        return QueryResult(
+            answer=answer,
+            sources=[{"text": ctx} for ctx in contexts],
+            metadata={"mode": self.mode.value}
+        )
+    
+    async def aquery(
+        self,
+        question: str,
+        context: Optional[List[str]] = None,
+        **kwargs
+    ) -> QueryResult:
+        """Async version (just calls sync)."""
+        return self.query(question, context, **kwargs)
+
+
+class SubQuestionEngine:
+    """
+    Sub-question decomposition query engine.
+    
+    Decomposes complex questions into simpler sub-questions.
+    Uses deterministic fallback when LLM is not available.
+    """
+    
+    name: str = "sub_question"
+    mode: QueryMode = QueryMode.SUB_QUESTION
+    
+    def __init__(self, llm: Optional[Any] = None, **kwargs):
+        self.llm = llm
+    
+    def query(
+        self,
+        question: str,
+        context: Optional[List[str]] = None,
+        retriever: Optional[Any] = None,
+        **kwargs
+    ) -> QueryResult:
+        """Query with sub-question decomposition."""
+        # Decompose question
+        if self.llm:
+            sub_questions = self._decompose_with_llm(question)
+        else:
+            sub_questions = decompose_question(question)
+        
+        # Gather context for each sub-question
+        all_contexts = list(context or [])
+        sub_answers = []
+        
+        for sub_q in sub_questions:
+            if retriever:
+                # Retrieve for each sub-question
+                try:
+                    results = retriever.retrieve(sub_q, top_k=3)
+                    for r in results:
+                        if hasattr(r, 'text'):
+                            all_contexts.append(r.text)
+                        elif isinstance(r, dict):
+                            all_contexts.append(r.get('text', str(r)))
+                except Exception as e:
+                    logger.warning(f"Retrieval failed for sub-question: {e}")
+            
+            # Simple sub-answer (would use LLM in full implementation)
+            sub_answers.append(f"Sub-question: {sub_q}")
+        
+        # Synthesize final answer
+        answer = synthesize_answer(question, all_contexts)
+        
+        return QueryResult(
+            answer=answer,
+            sources=[{"text": ctx} for ctx in all_contexts[:10]],
+            sub_questions=sub_questions,
+            metadata={"mode": self.mode.value, "sub_question_count": len(sub_questions)}
+        )
+    
+    async def aquery(
+        self,
+        question: str,
+        context: Optional[List[str]] = None,
+        **kwargs
+    ) -> QueryResult:
+        """Async version (just calls sync)."""
+        return self.query(question, context, **kwargs)
+    
+    def _decompose_with_llm(self, question: str) -> List[str]:
+        """Decompose question using LLM."""
+        try:
+            prompt = f"""Break down this complex question into simpler sub-questions that can be answered independently.
+            
+Question: {question}
+
+Return only the sub-questions, one per line, without numbering or bullets."""
+            
+            response = self.llm.chat(prompt)
+            if response:
+                lines = response.strip().split("\n")
+                return [line.strip() for line in lines if line.strip()]
+        except Exception as e:
+            logger.warning(f"LLM decomposition failed: {e}")
+        
+        return decompose_question(question)
+
+
+# Register default engines
+def _register_default_engines():
+    """Register default query engines."""
+    registry = get_query_engine_registry()
+    registry.register("default", SimpleQueryEngine)
+    registry.register("simple", SimpleQueryEngine)
+    registry.register("sub_question", SubQuestionEngine)
+
+
+_register_default_engines()

src/praisonai-agents/praisonaiagents/knowledge/readers.py

@@ -0,0 +1,286 @@
+"""
+Data Reader Adapter Protocol for PraisonAI Agents.
+
+This module defines the protocol for document readers and a registry system.
+NO heavy imports - only stdlib and typing.
+
+Implementations are provided by the wrapper layer (praisonai.adapters.readers).
+"""
+
+from dataclasses import dataclass, field
+from typing import Any, Callable, Dict, List, Optional, Protocol, runtime_checkable
+import os
+import logging
+
+logger = logging.getLogger(__name__)
+
+
+@dataclass
+class Document:
+    """
+    Lightweight document representation.
+    
+    Attributes:
+        content: The text content of the document
+        metadata: Optional metadata dict (source, filename, page, etc.)
+        doc_id: Optional unique identifier
+        embedding: Optional pre-computed embedding
+    """
+    content: str
+    metadata: Dict[str, Any] = field(default_factory=dict)
+    doc_id: Optional[str] = None
+    embedding: Optional[List[float]] = None
+    
+    def __post_init__(self):
+        if self.doc_id is None:
+            import uuid
+            self.doc_id = str(uuid.uuid4())
+    
+    def to_dict(self) -> Dict[str, Any]:
+        """Convert to dictionary."""
+        return {
+            "content": self.content,
+            "metadata": self.metadata,
+            "doc_id": self.doc_id,
+            "embedding": self.embedding
+        }
+    
+    @classmethod
+    def from_dict(cls, data: Dict[str, Any]) -> "Document":
+        """Create from dictionary."""
+        return cls(
+            content=data.get("content", ""),
+            metadata=data.get("metadata", {}),
+            doc_id=data.get("doc_id"),
+            embedding=data.get("embedding")
+        )
+
+
+@runtime_checkable
+class ReaderProtocol(Protocol):
+    """
+    Protocol for document readers.
+    
+    Implementations must provide:
+    - name: Identifier for the reader
+    - supported_extensions: List of file extensions this reader handles
+    - load(): Method to load documents from a source
+    """
+    
+    name: str
+    supported_extensions: List[str]
+    
+    def load(
+        self, 
+        source: str, 
+        *, 
+        metadata: Optional[Dict[str, Any]] = None
+    ) -> List[Document]:
+        """
+        Load documents from a source.
+        
+        Args:
+            source: File path, URL, or other source identifier
+            metadata: Optional metadata to attach to all documents
+            
+        Returns:
+            List of Document objects
+        """
+        ...
+    
+    def can_handle(self, source: str) -> bool:
+        """
+        Check if this reader can handle the given source.
+        
+        Args:
+            source: File path, URL, or other source identifier
+            
+        Returns:
+            True if this reader can handle the source
+        """
+        ...
+
+
+class ReaderRegistry:
+    """
+    Registry for document readers.
+    
+    Provides lazy loading and automatic reader selection based on source type.
+    """
+    
+    _instance: Optional["ReaderRegistry"] = None
+    
+    def __new__(cls) -> "ReaderRegistry":
+        if cls._instance is None:
+            cls._instance = super().__new__(cls)
+            cls._instance._readers: Dict[str, Callable[[], ReaderProtocol]] = {}
+            cls._instance._extension_map: Dict[str, str] = {}
+            cls._instance._initialized_readers: Dict[str, ReaderProtocol] = {}
+        return cls._instance
+    
+    def register(
+        self, 
+        name: str, 
+        factory: Callable[[], ReaderProtocol],
+        extensions: Optional[List[str]] = None
+    ) -> None:
+        """
+        Register a reader factory.
+        
+        Args:
+            name: Unique name for the reader
+            factory: Callable that returns a ReaderProtocol instance
+            extensions: List of file extensions this reader handles
+        """
+        self._readers[name] = factory
+        if extensions:
+            for ext in extensions:
+                ext_lower = ext.lower().lstrip(".")
+                self._extension_map[ext_lower] = name
+    
+    def get(self, name: str) -> Optional[ReaderProtocol]:
+        """
+        Get a reader by name (lazy initialization).
+        
+        Args:
+            name: Reader name
+            
+        Returns:
+            ReaderProtocol instance or None
+        """
+        if name in self._initialized_readers:
+            return self._initialized_readers[name]
+        
+        if name in self._readers:
+            try:
+                reader = self._readers[name]()
+                self._initialized_readers[name] = reader
+                return reader
+            except Exception as e:
+                logger.warning(f"Failed to initialize reader '{name}': {e}")
+                return None
+        return None
+    
+    def get_for_source(self, source: str) -> Optional[ReaderProtocol]:
+        """
+        Get the best reader for a given source.
+        
+        Args:
+            source: File path, URL, or other source identifier
+            
+        Returns:
+            Best matching ReaderProtocol or None
+        """
+        source_kind = detect_source_kind(source)
+        
+        if source_kind == "file":
+            ext = os.path.splitext(source)[1].lower().lstrip(".")
+            if ext in self._extension_map:
+                return self.get(self._extension_map[ext])
+        elif source_kind == "url":
+            # Try URL reader
+            if "url" in self._readers:
+                return self.get("url")
+        elif source_kind == "directory":
+            if "directory" in self._readers:
+                return self.get("directory")
+        
+        # Fallback to auto reader if available
+        if "auto" in self._readers:
+            return self.get("auto")
+        
+        return None
+    
+    def list_readers(self) -> List[str]:
+        """List all registered reader names."""
+        return list(self._readers.keys())
+    
+    def list_extensions(self) -> Dict[str, str]:
+        """List all registered extensions and their readers."""
+        return dict(self._extension_map)
+    
+    def clear(self) -> None:
+        """Clear all registered readers (mainly for testing)."""
+        self._readers.clear()
+        self._extension_map.clear()
+        self._initialized_readers.clear()
+
+
+def get_reader_registry() -> ReaderRegistry:
+    """Get the global reader registry instance."""
+    return ReaderRegistry()
+
+
+def detect_source_kind(source: str) -> str:
+    """
+    Detect the kind of source without importing heavy libraries.
+    
+    Args:
+        source: File path, URL, directory, or glob pattern
+        
+    Returns:
+        One of: "file", "url", "directory", "glob", "unknown"
+    """
+    if not isinstance(source, str):
+        return "unknown"
+    
+    source = source.strip()
+    
+    # Check for URL
+    if source.startswith(("http://", "https://", "ftp://", "s3://", "gs://")):
+        return "url"
+    
+    # Check for glob pattern
+    if "*" in source or "?" in source or "[" in source:
+        return "glob"
+    
+    # Check if it's a directory
+    if os.path.isdir(source):
+        return "directory"
+    
+    # Check if it's a file
+    if os.path.isfile(source):
+        return "file"
+    
+    # Check if it looks like a file path (has extension)
+    if os.path.splitext(source)[1]:
+        return "file"
+    
+    return "unknown"
+
+
+def get_file_extension(source: str) -> str:
+    """
+    Get the file extension from a source path.
+    
+    Args:
+        source: File path or URL
+        
+    Returns:
+        Lowercase extension without dot, or empty string
+    """
+    # Handle URLs by extracting path
+    if source.startswith(("http://", "https://")):
+        from urllib.parse import urlparse
+        path = urlparse(source).path
+        return os.path.splitext(path)[1].lower().lstrip(".")
+    
+    return os.path.splitext(source)[1].lower().lstrip(".")
+
+
+# Default extension mappings (reader name -> extensions)
+DEFAULT_EXTENSION_MAPPINGS = {
+    "text": ["txt", "text"],
+    "markdown": ["md", "markdown"],
+    "json": ["json", "jsonl"],
+    "csv": ["csv", "tsv"],
+    "html": ["html", "htm"],
+    "xml": ["xml"],
+    "pdf": ["pdf"],
+    "docx": ["docx", "doc"],
+    "xlsx": ["xlsx", "xls"],
+    "pptx": ["pptx", "ppt"],
+    "image": ["jpg", "jpeg", "png", "gif", "bmp", "tiff", "webp"],
+    "audio": ["mp3", "wav", "ogg", "m4a", "flac"],
+    "video": ["mp4", "avi", "mov", "mkv", "webm"],
+}

src/praisonai-agents/praisonaiagents/knowledge/rerankers.py

@@ -0,0 +1,196 @@
+"""
+Reranker Protocol for PraisonAI Agents.
+
+This module defines the protocol for rerankers and a registry system.
+NO heavy imports - only stdlib and typing.
+
+Implementations are provided by praisonai-tools or wrapper layer.
+"""
+
+from dataclasses import dataclass
+from typing import Any, Callable, Dict, List, Optional, Protocol, runtime_checkable
+import logging
+
+logger = logging.getLogger(__name__)
+
+
+@dataclass
+class RerankResult:
+    """
+    Result from a reranking operation.
+    
+    Attributes:
+        text: The text content
+        score: Reranking score (higher is more relevant)
+        original_index: Original position before reranking
+        metadata: Optional metadata
+    """
+    text: str
+    score: float
+    original_index: int
+    metadata: Dict[str, Any] = None
+    
+    def __post_init__(self):
+        if self.metadata is None:
+            self.metadata = {}
+
+
+@runtime_checkable
+class RerankerProtocol(Protocol):
+    """
+    Protocol for rerankers.
+    
+    Implementations must provide a rerank method that takes a query
+    and a list of documents and returns reranked results.
+    """
+    
+    name: str
+    
+    def rerank(
+        self,
+        query: str,
+        documents: List[str],
+        top_k: Optional[int] = None,
+        **kwargs
+    ) -> List[RerankResult]:
+        """
+        Rerank documents based on relevance to query.
+        
+        Args:
+            query: The search query
+            documents: List of document texts to rerank
+            top_k: Optional limit on results (None = return all)
+            **kwargs: Additional reranker-specific options
+            
+        Returns:
+            List of RerankResult objects sorted by relevance
+        """
+        ...
+    
+    async def arerank(
+        self,
+        query: str,
+        documents: List[str],
+        top_k: Optional[int] = None,
+        **kwargs
+    ) -> List[RerankResult]:
+        """Async version of rerank."""
+        ...
+
+
+class RerankerRegistry:
+    """
+    Registry for rerankers.
+    """
+    
+    _instance: Optional["RerankerRegistry"] = None
+    
+    def __new__(cls) -> "RerankerRegistry":
+        if cls._instance is None:
+            cls._instance = super().__new__(cls)
+            cls._instance._rerankers: Dict[str, Callable[..., RerankerProtocol]] = {}
+        return cls._instance
+    
+    def register(
+        self, 
+        name: str, 
+        factory: Callable[..., RerankerProtocol]
+    ) -> None:
+        """Register a reranker factory."""
+        self._rerankers[name] = factory
+    
+    def get(
+        self, 
+        name: str, 
+        **kwargs
+    ) -> Optional[RerankerProtocol]:
+        """Get a reranker by name."""
+        if name in self._rerankers:
+            try:
+                return self._rerankers[name](**kwargs)
+            except Exception as e:
+                logger.warning(f"Failed to initialize reranker '{name}': {e}")
+                return None
+        return None
+    
+    def list_rerankers(self) -> List[str]:
+        """List all registered reranker names."""
+        return list(self._rerankers.keys())
+    
+    def clear(self) -> None:
+        """Clear all registered rerankers."""
+        self._rerankers.clear()
+
+
+def get_reranker_registry() -> RerankerRegistry:
+    """Get the global reranker registry instance."""
+    return RerankerRegistry()
+
+
+class SimpleReranker:
+    """
+    Simple keyword-based reranker for testing and fallback.
+    
+    Uses term frequency overlap for scoring. No external dependencies.
+    """
+    
+    name: str = "simple"
+    
+    def __init__(self, **kwargs):
+        pass
+    
+    def rerank(
+        self,
+        query: str,
+        documents: List[str],
+        top_k: Optional[int] = None,
+        **kwargs
+    ) -> List[RerankResult]:
+        """Rerank using simple term overlap."""
+        query_terms = set(query.lower().split())
+        
+        results = []
+        for i, doc in enumerate(documents):
+            doc_terms = set(doc.lower().split())
+            
+            # Calculate Jaccard similarity
+            if not query_terms or not doc_terms:
+                score = 0.0
+            else:
+                intersection = len(query_terms & doc_terms)
+                union = len(query_terms | doc_terms)
+                score = intersection / union if union > 0 else 0.0
+            
+            results.append(RerankResult(
+                text=doc,
+                score=score,
+                original_index=i
+            ))
+        
+        # Sort by score descending
+        results.sort(key=lambda x: x.score, reverse=True)
+        
+        if top_k is not None:
+            results = results[:top_k]
+        
+        return results
+    
+    async def arerank(
+        self,
+        query: str,
+        documents: List[str],
+        top_k: Optional[int] = None,
+        **kwargs
+    ) -> List[RerankResult]:
+        """Async version (just calls sync)."""
+        return self.rerank(query, documents, top_k, **kwargs)
+
+
+# Register the simple reranker by default
+def _register_default_rerankers():
+    """Register default rerankers."""
+    registry = get_reranker_registry()
+    registry.register("simple", SimpleReranker)
+
+
+_register_default_rerankers()

src/praisonai-agents/praisonaiagents/knowledge/retrieval.py

@@ -0,0 +1,285 @@
+"""
+Retrieval Strategy Patterns for PraisonAI Agents.
+
+This module defines protocols and strategies for document retrieval.
+NO heavy imports - only stdlib and typing.
+
+Implementations are provided by the wrapper layer.
+"""
+
+from dataclasses import dataclass, field
+from typing import Any, Callable, Dict, List, Optional, Protocol, runtime_checkable
+from enum import Enum
+import logging
+
+logger = logging.getLogger(__name__)
+
+
+class RetrievalStrategy(str, Enum):
+    """Available retrieval strategies."""
+    BASIC = "basic"
+    FUSION = "fusion"
+    RECURSIVE = "recursive"
+    AUTO_MERGE = "auto_merge"
+    HYBRID = "hybrid"
+
+
+@dataclass
+class RetrievalResult:
+    """
+    Result from a retrieval operation.
+    
+    Attributes:
+        text: The retrieved text content
+        score: Relevance score (0-1)
+        metadata: Source metadata
+        doc_id: Document identifier
+        chunk_index: Index of chunk within document
+    """
+    text: str
+    score: float
+    metadata: Dict[str, Any] = field(default_factory=dict)
+    doc_id: Optional[str] = None
+    chunk_index: Optional[int] = None
+    
+    def to_dict(self) -> Dict[str, Any]:
+        """Convert to dictionary."""
+        return {
+            "text": self.text,
+            "score": self.score,
+            "metadata": self.metadata,
+            "doc_id": self.doc_id,
+            "chunk_index": self.chunk_index
+        }
+
+
+@runtime_checkable
+class RetrieverProtocol(Protocol):
+    """
+    Protocol for document retrievers.
+    
+    Implementations must provide a retrieve method that takes a query
+    and returns relevant documents.
+    """
+    
+    name: str
+    strategy: RetrievalStrategy
+    
+    def retrieve(
+        self,
+        query: str,
+        top_k: int = 10,
+        filter: Optional[Dict[str, Any]] = None,
+        **kwargs
+    ) -> List[RetrievalResult]:
+        """
+        Retrieve relevant documents for a query.
+        
+        Args:
+            query: The search query
+            top_k: Maximum number of results
+            filter: Optional metadata filter
+            **kwargs: Additional retriever-specific options
+            
+        Returns:
+            List of RetrievalResult objects sorted by relevance
+        """
+        ...
+    
+    async def aretrieve(
+        self,
+        query: str,
+        top_k: int = 10,
+        filter: Optional[Dict[str, Any]] = None,
+        **kwargs
+    ) -> List[RetrievalResult]:
+        """Async version of retrieve."""
+        ...
+
+
+class RetrieverRegistry:
+    """
+    Registry for retrieval strategies.
+    """
+    
+    _instance: Optional["RetrieverRegistry"] = None
+    
+    def __new__(cls) -> "RetrieverRegistry":
+        if cls._instance is None:
+            cls._instance = super().__new__(cls)
+            cls._instance._retrievers: Dict[str, Callable[..., RetrieverProtocol]] = {}
+        return cls._instance
+    
+    def register(
+        self, 
+        name: str, 
+        factory: Callable[..., RetrieverProtocol]
+    ) -> None:
+        """Register a retriever factory."""
+        self._retrievers[name] = factory
+    
+    def get(
+        self, 
+        name: str, 
+        **kwargs
+    ) -> Optional[RetrieverProtocol]:
+        """Get a retriever by name."""
+        if name in self._retrievers:
+            try:
+                return self._retrievers[name](**kwargs)
+            except Exception as e:
+                logger.warning(f"Failed to initialize retriever '{name}': {e}")
+                return None
+        return None
+    
+    def list_retrievers(self) -> List[str]:
+        """List all registered retriever names."""
+        return list(self._retrievers.keys())
+    
+    def clear(self) -> None:
+        """Clear all registered retrievers."""
+        self._retrievers.clear()
+
+
+def get_retriever_registry() -> RetrieverRegistry:
+    """Get the global retriever registry instance."""
+    return RetrieverRegistry()
+
+
+def reciprocal_rank_fusion(
+    result_lists: List[List[RetrievalResult]],
+    k: int = 60
+) -> List[RetrievalResult]:
+    """
+    Reciprocal Rank Fusion (RRF) algorithm for combining multiple result lists.
+    
+    Pure Python implementation with no external dependencies.
+    
+    Args:
+        result_lists: List of result lists from different retrievers
+        k: RRF constant (default 60)
+        
+    Returns:
+        Fused and re-ranked list of results
+    """
+    # Track scores by document ID
+    fused_scores: Dict[str, float] = {}
+    doc_map: Dict[str, RetrievalResult] = {}
+    
+    for results in result_lists:
+        for rank, result in enumerate(results):
+            doc_key = result.doc_id or result.text[:100]  # Use text prefix as fallback key
+            
+            # RRF score: 1 / (k + rank)
+            rrf_score = 1.0 / (k + rank + 1)
+            
+            if doc_key in fused_scores:
+                fused_scores[doc_key] += rrf_score
+            else:
+                fused_scores[doc_key] = rrf_score
+                doc_map[doc_key] = result
+    
+    # Sort by fused score
+    sorted_keys = sorted(fused_scores.keys(), key=lambda x: fused_scores[x], reverse=True)
+    
+    # Build result list with updated scores
+    fused_results = []
+    for key in sorted_keys:
+        result = doc_map[key]
+        fused_results.append(RetrievalResult(
+            text=result.text,
+            score=fused_scores[key],
+            metadata=result.metadata,
+            doc_id=result.doc_id,
+            chunk_index=result.chunk_index
+        ))
+    
+    return fused_results
+
+
+def merge_adjacent_chunks(
+    results: List[RetrievalResult],
+    max_gap: int = 1
+) -> List[RetrievalResult]:
+    """
+    Merge adjacent chunks from the same document.
+    
+    Args:
+        results: List of retrieval results
+        max_gap: Maximum gap between chunk indices to merge
+        
+    Returns:
+        Merged results
+    """
+    if not results:
+        return results
+    
+    # Group by document ID
+    doc_groups: Dict[str, List[RetrievalResult]] = {}
+    for result in results:
+        doc_id = result.doc_id or "unknown"
+        if doc_id not in doc_groups:
+            doc_groups[doc_id] = []
+        doc_groups[doc_id].append(result)
+    
+    merged_results = []
+    
+    for doc_id, chunks in doc_groups.items():
+        # Sort by chunk index
+        chunks_with_index = [c for c in chunks if c.chunk_index is not None]
+        chunks_without_index = [c for c in chunks if c.chunk_index is None]
+        
+        if not chunks_with_index:
+            merged_results.extend(chunks_without_index)
+            continue
+        
+        chunks_with_index.sort(key=lambda x: x.chunk_index or 0)
+        
+        # Merge adjacent chunks
+        current_group = [chunks_with_index[0]]
+        
+        for chunk in chunks_with_index[1:]:
+            last_chunk = current_group[-1]
+            last_idx = last_chunk.chunk_index or 0
+            curr_idx = chunk.chunk_index or 0
+            
+            if curr_idx - last_idx <= max_gap:
+                current_group.append(chunk)
+            else:
+                # Merge current group and start new one
+                merged_results.append(_merge_chunk_group(current_group))
+                current_group = [chunk]
+        
+        # Don't forget the last group
+        if current_group:
+            merged_results.append(_merge_chunk_group(current_group))
+        
+        merged_results.extend(chunks_without_index)
+    
+    # Sort by score
+    merged_results.sort(key=lambda x: x.score, reverse=True)
+    return merged_results
+
+
+def _merge_chunk_group(chunks: List[RetrievalResult]) -> RetrievalResult:
+    """Merge a group of chunks into a single result."""
+    if len(chunks) == 1:
+        return chunks[0]
+    
+    # Combine text
+    merged_text = "\n".join(c.text for c in chunks)
+    
+    # Average score
+    avg_score = sum(c.score for c in chunks) / len(chunks)
+    
+    # Use first chunk's metadata
+    metadata = chunks[0].metadata.copy()
+    metadata["merged_chunks"] = len(chunks)
+    
+    return RetrievalResult(
+        text=merged_text,
+        score=avg_score,
+        metadata=metadata,
+        doc_id=chunks[0].doc_id,
+        chunk_index=chunks[0].chunk_index
+    )

src/praisonai-agents/praisonaiagents/knowledge/vector_store.py

@@ -0,0 +1,422 @@
+"""
+Vector Store Adapter Protocol for PraisonAI Agents.
+
+This module defines the protocol for vector stores and a registry system.
+NO heavy imports - only stdlib and typing.
+
+Implementations are provided by praisonai-tools or wrapper layer.
+"""
+
+from dataclasses import dataclass, field
+from typing import Any, Callable, Dict, List, Optional, Protocol, runtime_checkable
+import logging
+
+logger = logging.getLogger(__name__)
+
+
+@dataclass
+class VectorRecord:
+    """
+    A record in a vector store.
+    
+    Attributes:
+        id: Unique identifier
+        text: The text content
+        embedding: The vector embedding
+        metadata: Optional metadata dict
+        score: Optional similarity score (for query results)
+    """
+    id: str
+    text: str
+    embedding: List[float]
+    metadata: Dict[str, Any] = field(default_factory=dict)
+    score: Optional[float] = None
+    
+    def to_dict(self) -> Dict[str, Any]:
+        """Convert to dictionary."""
+        return {
+            "id": self.id,
+            "text": self.text,
+            "embedding": self.embedding,
+            "metadata": self.metadata,
+            "score": self.score
+        }
+    
+    @classmethod
+    def from_dict(cls, data: Dict[str, Any]) -> "VectorRecord":
+        """Create from dictionary."""
+        return cls(
+            id=data["id"],
+            text=data["text"],
+            embedding=data["embedding"],
+            metadata=data.get("metadata", {}),
+            score=data.get("score")
+        )
+
+
+@runtime_checkable
+class VectorStoreProtocol(Protocol):
+    """
+    Protocol for vector stores.
+    
+    Implementations must provide methods for:
+    - Adding vectors
+    - Querying by vector similarity
+    - Deleting vectors
+    - Getting count
+    """
+    
+    name: str
+    
+    def add(
+        self,
+        texts: List[str],
+        embeddings: List[List[float]],
+        metadatas: Optional[List[Dict[str, Any]]] = None,
+        ids: Optional[List[str]] = None,
+        namespace: Optional[str] = None
+    ) -> List[str]:
+        """
+        Add vectors to the store.
+        
+        Args:
+            texts: List of text content
+            embeddings: List of embedding vectors
+            metadatas: Optional list of metadata dicts
+            ids: Optional list of IDs (auto-generated if not provided)
+            namespace: Optional namespace for multi-tenant isolation
+            
+        Returns:
+            List of IDs for the added vectors
+        """
+        ...
+    
+    def query(
+        self,
+        embedding: List[float],
+        top_k: int = 10,
+        namespace: Optional[str] = None,
+        filter: Optional[Dict[str, Any]] = None
+    ) -> List[VectorRecord]:
+        """
+        Query vectors by similarity.
+        
+        Args:
+            embedding: Query embedding vector
+            top_k: Number of results to return
+            namespace: Optional namespace filter
+            filter: Optional metadata filter
+            
+        Returns:
+            List of VectorRecord objects sorted by similarity
+        """
+        ...
+    
+    def delete(
+        self,
+        ids: Optional[List[str]] = None,
+        namespace: Optional[str] = None,
+        filter: Optional[Dict[str, Any]] = None,
+        delete_all: bool = False
+    ) -> int:
+        """
+        Delete vectors from the store.
+        
+        Args:
+            ids: Optional list of IDs to delete
+            namespace: Optional namespace filter
+            filter: Optional metadata filter
+            delete_all: If True, delete all vectors
+            
+        Returns:
+            Number of vectors deleted
+        """
+        ...
+    
+    def count(self, namespace: Optional[str] = None) -> int:
+        """
+        Get the number of vectors in the store.
+        
+        Args:
+            namespace: Optional namespace filter
+            
+        Returns:
+            Number of vectors
+        """
+        ...
+    
+    def get(
+        self,
+        ids: List[str],
+        namespace: Optional[str] = None
+    ) -> List[VectorRecord]:
+        """
+        Get vectors by ID.
+        
+        Args:
+            ids: List of IDs to retrieve
+            namespace: Optional namespace filter
+            
+        Returns:
+            List of VectorRecord objects
+        """
+        ...
+
+
+class VectorStoreRegistry:
+    """
+    Registry for vector stores.
+    
+    Provides lazy loading and configuration-based initialization.
+    """
+    
+    _instance: Optional["VectorStoreRegistry"] = None
+    
+    def __new__(cls) -> "VectorStoreRegistry":
+        if cls._instance is None:
+            cls._instance = super().__new__(cls)
+            cls._instance._stores: Dict[str, Callable[..., VectorStoreProtocol]] = {}
+            cls._instance._initialized_stores: Dict[str, VectorStoreProtocol] = {}
+        return cls._instance
+    
+    def register(
+        self, 
+        name: str, 
+        factory: Callable[..., VectorStoreProtocol]
+    ) -> None:
+        """
+        Register a vector store factory.
+        
+        Args:
+            name: Unique name for the store (e.g., "chroma", "pinecone")
+            factory: Callable that returns a VectorStoreProtocol instance
+        """
+        self._stores[name] = factory
+    
+    def get(
+        self, 
+        name: str, 
+        config: Optional[Dict[str, Any]] = None,
+        namespace: Optional[str] = None
+    ) -> Optional[VectorStoreProtocol]:
+        """
+        Get a vector store by name (lazy initialization).
+        
+        Args:
+            name: Store name
+            config: Optional configuration dict
+            namespace: Optional namespace for multi-tenant isolation
+            
+        Returns:
+            VectorStoreProtocol instance or None
+        """
+        cache_key = f"{name}:{namespace or 'default'}"
+        
+        if cache_key in self._initialized_stores:
+            return self._initialized_stores[cache_key]
+        
+        if name in self._stores:
+            try:
+                store = self._stores[name](config=config, namespace=namespace)
+                self._initialized_stores[cache_key] = store
+                return store
+            except Exception as e:
+                logger.warning(f"Failed to initialize vector store '{name}': {e}")
+                return None
+        return None
+    
+    def list_stores(self) -> List[str]:
+        """List all registered store names."""
+        return list(self._stores.keys())
+    
+    def clear(self) -> None:
+        """Clear all registered stores (mainly for testing)."""
+        self._stores.clear()
+        self._initialized_stores.clear()
+
+
+def get_vector_store_registry() -> VectorStoreRegistry:
+    """Get the global vector store registry instance."""
+    return VectorStoreRegistry()
+
+
+class InMemoryVectorStore:
+    """
+    Pure Python in-memory vector store for testing and default usage.
+    
+    Uses cosine similarity for queries. No external dependencies.
+    """
+    
+    name: str = "memory"
+    
+    def __init__(
+        self, 
+        config: Optional[Dict[str, Any]] = None,
+        namespace: Optional[str] = None
+    ):
+        self.config = config or {}
+        self.default_namespace = namespace or "default"
+        self._data: Dict[str, Dict[str, VectorRecord]] = {}  # namespace -> id -> record
+    
+    def _get_namespace_data(self, namespace: Optional[str] = None) -> Dict[str, VectorRecord]:
+        """Get or create namespace data."""
+        ns = namespace or self.default_namespace
+        if ns not in self._data:
+            self._data[ns] = {}
+        return self._data[ns]
+    
+    def add(
+        self,
+        texts: List[str],
+        embeddings: List[List[float]],
+        metadatas: Optional[List[Dict[str, Any]]] = None,
+        ids: Optional[List[str]] = None,
+        namespace: Optional[str] = None
+    ) -> List[str]:
+        """Add vectors to the store."""
+        import uuid
+        
+        ns_data = self._get_namespace_data(namespace)
+        metadatas = metadatas or [{} for _ in texts]
+        ids = ids or [str(uuid.uuid4()) for _ in texts]
+        
+        result_ids = []
+        for i, (text, embedding, metadata, id_) in enumerate(zip(texts, embeddings, metadatas, ids)):
+            record = VectorRecord(
+                id=id_,
+                text=text,
+                embedding=embedding,
+                metadata=metadata
+            )
+            ns_data[id_] = record
+            result_ids.append(id_)
+        
+        return result_ids
+    
+    def query(
+        self,
+        embedding: List[float],
+        top_k: int = 10,
+        namespace: Optional[str] = None,
+        filter: Optional[Dict[str, Any]] = None
+    ) -> List[VectorRecord]:
+        """Query vectors by cosine similarity."""
+        ns_data = self._get_namespace_data(namespace)
+        
+        if not ns_data:
+            return []
+        
+        # Calculate cosine similarity for all records
+        results = []
+        for record in ns_data.values():
+            # Apply metadata filter if provided
+            if filter:
+                match = True
+                for key, value in filter.items():
+                    if record.metadata.get(key) != value:
+                        match = False
+                        break
+                if not match:
+                    continue
+            
+            score = self._cosine_similarity(embedding, record.embedding)
+            results.append(VectorRecord(
+                id=record.id,
+                text=record.text,
+                embedding=record.embedding,
+                metadata=record.metadata,
+                score=score
+            ))
+        
+        # Sort by score descending and return top_k
+        results.sort(key=lambda x: x.score or 0, reverse=True)
+        return results[:top_k]
+    
+    def delete(
+        self,
+        ids: Optional[List[str]] = None,
+        namespace: Optional[str] = None,
+        filter: Optional[Dict[str, Any]] = None,
+        delete_all: bool = False
+    ) -> int:
+        """Delete vectors from the store."""
+        ns = namespace or self.default_namespace
+        
+        if delete_all:
+            if ns in self._data:
+                count = len(self._data[ns])
+                self._data[ns] = {}
+                return count
+            return 0
+        
+        if ns not in self._data:
+            return 0
+        
+        ns_data = self._data[ns]
+        deleted = 0
+        
+        if ids:
+            for id_ in ids:
+                if id_ in ns_data:
+                    del ns_data[id_]
+                    deleted += 1
+        elif filter:
+            to_delete = []
+            for id_, record in ns_data.items():
+                match = True
+                for key, value in filter.items():
+                    if record.metadata.get(key) != value:
+                        match = False
+                        break
+                if match:
+                    to_delete.append(id_)
+            for id_ in to_delete:
+                del ns_data[id_]
+                deleted += 1
+        
+        return deleted
+    
+    def count(self, namespace: Optional[str] = None) -> int:
+        """Get the number of vectors in the store."""
+        ns = namespace or self.default_namespace
+        if ns in self._data:
+            return len(self._data[ns])
+        return 0
+    
+    def get(
+        self,
+        ids: List[str],
+        namespace: Optional[str] = None
+    ) -> List[VectorRecord]:
+        """Get vectors by ID."""
+        ns_data = self._get_namespace_data(namespace)
+        results = []
+        for id_ in ids:
+            if id_ in ns_data:
+                results.append(ns_data[id_])
+        return results
+    
+    @staticmethod
+    def _cosine_similarity(a: List[float], b: List[float]) -> float:
+        """Calculate cosine similarity between two vectors."""
+        if len(a) != len(b):
+            return 0.0
+        
+        dot_product = sum(x * y for x, y in zip(a, b))
+        norm_a = sum(x * x for x in a) ** 0.5
+        norm_b = sum(x * x for x in b) ** 0.5
+        
+        if norm_a == 0 or norm_b == 0:
+            return 0.0
+        
+        return dot_product / (norm_a * norm_b)
+
+
+# Register the in-memory store by default
+def _register_default_stores():
+    """Register default vector stores."""
+    registry = get_vector_store_registry()
+    registry.register("memory", InMemoryVectorStore)
+
+
+_register_default_stores()

src/praisonai-agents/praisonaiagents/llm/__init__.py

@@ -27,6 +27,7 @@ from .model_router import (
     TaskComplexity,
     create_routing_agent
 )
+from .rate_limiter import RateLimiter
 
 __all__ = [
     "LLM", 
@@ -46,5 +47,6 @@ __all__ = [
     "ModelRouter",
     "ModelProfile",
     "TaskComplexity",
-    "create_routing_agent"
+    "create_routing_agent",
+    "RateLimiter"
 ]

src/praisonai-agents/praisonaiagents/llm/rate_limiter.py

@@ -0,0 +1,147 @@
+"""
+Rate Limiter for PraisonAI Agents.
+
+Provides token bucket rate limiting for LLM API calls.
+
+Zero Performance Impact:
+- When rate_limiter is None, no overhead
+- Uses monotonic clock for accuracy
+- Injectable clock/sleep for deterministic testing
+
+Usage:
+    from praisonaiagents.llm import RateLimiter
+    from praisonaiagents import Agent
+    
+    limiter = RateLimiter(requests_per_minute=60)
+    agent = Agent(name="Test", rate_limiter=limiter)
+"""
+
+import time
+import asyncio
+from dataclasses import dataclass, field
+from typing import Optional, Callable
+
+
+@dataclass
+class RateLimiter:
+    """Token bucket rate limiter for API calls.
+    
+    Uses the token bucket algorithm:
+    - Tokens are added at a fixed rate (requests_per_minute / 60 per second)
+    - Each request consumes one token
+    - If no tokens available, wait until one is available
+    - Burst allows multiple requests in quick succession
+    
+    Args:
+        requests_per_minute: Maximum requests per minute
+        tokens_per_minute: Optional token-based rate limiting (for future use)
+        burst: Maximum burst size (default: 1)
+    
+    Example:
+        limiter = RateLimiter(requests_per_minute=60)  # 1 req/sec
+        limiter.acquire()  # Blocks if rate exceeded
+        
+        # Async usage
+        await limiter.acquire_async()
+    """
+    requests_per_minute: int
+    tokens_per_minute: Optional[int] = None
+    burst: int = 1
+    
+    # Internal state
+    _tokens: float = field(default=None, init=False, repr=False)
+    _last_update: float = field(default=None, init=False, repr=False)
+    _lock: asyncio.Lock = field(default=None, init=False, repr=False)
+    
+    # Injectable functions for testing
+    _get_time: Callable[[], float] = field(default=None, init=False, repr=False)
+    _sleep: Callable[[float], None] = field(default=None, init=False, repr=False)
+    _async_sleep: Callable[[float], None] = field(default=None, init=False, repr=False)
+    
+    def __post_init__(self):
+        """Initialize internal state."""
+        self._tokens = float(self.burst)
+        self._last_update = time.monotonic()
+        self._lock = asyncio.Lock()
+        
+        # Default implementations (can be overridden for testing)
+        if self._get_time is None:
+            self._get_time = time.monotonic
+        if self._sleep is None:
+            self._sleep = time.sleep
+        if self._async_sleep is None:
+            self._async_sleep = asyncio.sleep
+    
+    @property
+    def _rate(self) -> float:
+        """Tokens per second."""
+        return self.requests_per_minute / 60.0
+    
+    def _refill(self) -> None:
+        """Refill tokens based on elapsed time."""
+        now = self._get_time()
+        elapsed = now - self._last_update
+        self._tokens = min(self.burst, self._tokens + elapsed * self._rate)
+        self._last_update = now
+    
+    def _wait_time(self) -> float:
+        """Calculate time to wait for next token."""
+        if self._tokens >= 1.0:
+            return 0.0
+        return (1.0 - self._tokens) / self._rate
+    
+    def acquire(self) -> None:
+        """Acquire a token, blocking if necessary.
+        
+        This is the synchronous version. Use acquire_async() for async code.
+        """
+        self._refill()
+        
+        wait = self._wait_time()
+        if wait > 0:
+            self._sleep(wait)
+            self._refill()
+        
+        self._tokens -= 1.0
+    
+    async def acquire_async(self) -> None:
+        """Acquire a token asynchronously, waiting if necessary.
+        
+        This is the async version. Use acquire() for sync code.
+        """
+        async with self._lock:
+            self._refill()
+            
+            wait = self._wait_time()
+            if wait > 0:
+                await self._async_sleep(wait)
+                self._refill()
+            
+            self._tokens -= 1.0
+    
+    def try_acquire(self) -> bool:
+        """Try to acquire a token without blocking.
+        
+        Returns:
+            True if token acquired, False if rate limit exceeded
+        """
+        self._refill()
+        
+        if self._tokens >= 1.0:
+            self._tokens -= 1.0
+            return True
+        return False
+    
+    def reset(self) -> None:
+        """Reset the rate limiter to initial state."""
+        self._tokens = float(self.burst)
+        self._last_update = self._get_time()
+    
+    @property
+    def available_tokens(self) -> float:
+        """Get current available tokens (for monitoring)."""
+        self._refill()
+        return self._tokens
+    
+    def __repr__(self) -> str:
+        return f"RateLimiter(rpm={self.requests_per_minute}, burst={self.burst})"

src/praisonai-agents/praisonaiagents/obs/__init__.py

@@ -0,0 +1,240 @@
+"""
+Observability interface for PraisonAI Agents.
+
+This module provides the protocol and types for observability/tracing.
+Implementations are provided by the wrapper layer (praisonai_tools.observability).
+
+Usage (simplest - recommended):
+    from praisonaiagents import Agent, obs
+    
+    agent = Agent(
+        name="Assistant",
+        observability=obs.auto(),  # Auto-detect from env vars
+    )
+    agent.chat("Hello!")  # Auto-traces to configured provider
+
+Alternative (explicit provider):
+    observability=obs.langfuse()     # Langfuse
+    observability=obs.langsmith()    # LangSmith
+    observability=obs.agentops()     # AgentOps
+    observability=obs.arize()        # Arize Phoenix
+    observability=obs.datadog()      # Datadog
+"""
+
+from typing import Any, Dict, List, Optional, Protocol, runtime_checkable
+from dataclasses import dataclass, field
+import time
+
+
+@dataclass
+class Span:
+    """Represents a span in a trace."""
+    span_id: str
+    trace_id: str
+    name: str
+    parent_span_id: Optional[str] = None
+    start_time: float = field(default_factory=time.time)
+    end_time: Optional[float] = None
+    attributes: Dict[str, Any] = field(default_factory=dict)
+    status: str = "ok"
+
+
+@dataclass
+class Trace:
+    """Represents a complete trace."""
+    trace_id: str
+    session_id: Optional[str] = None
+    agent_name: Optional[str] = None
+    user_id: Optional[str] = None
+    start_time: float = field(default_factory=time.time)
+    end_time: Optional[float] = None
+    spans: List[Span] = field(default_factory=list)
+    metadata: Dict[str, Any] = field(default_factory=dict)
+    status: str = "ok"
+
+
+@runtime_checkable
+class ObservabilityAdapter(Protocol):
+    """Protocol for observability adapters."""
+    
+    def on_trace_start(
+        self,
+        trace_id: str,
+        session_id: Optional[str] = None,
+        agent_name: Optional[str] = None,
+        user_id: Optional[str] = None,
+        metadata: Optional[Dict[str, Any]] = None,
+    ) -> None:
+        """Called when a trace starts."""
+        ...
+    
+    def on_trace_end(
+        self,
+        trace_id: str,
+        status: str = "ok",
+        metadata: Optional[Dict[str, Any]] = None,
+    ) -> None:
+        """Called when a trace ends."""
+        ...
+    
+    def on_span_start(
+        self,
+        span_id: str,
+        trace_id: str,
+        name: str,
+        parent_span_id: Optional[str] = None,
+        attributes: Optional[Dict[str, Any]] = None,
+    ) -> None:
+        """Called when a span starts."""
+        ...
+    
+    def on_span_end(
+        self,
+        span_id: str,
+        status: str = "ok",
+        attributes: Optional[Dict[str, Any]] = None,
+    ) -> None:
+        """Called when a span ends."""
+        ...
+    
+    def on_llm_call(
+        self,
+        span_id: str,
+        model: str,
+        messages: List[Dict[str, Any]],
+        response: Optional[str] = None,
+        tokens: Optional[Dict[str, int]] = None,
+        latency_ms: Optional[float] = None,
+    ) -> None:
+        """Called when an LLM call is made."""
+        ...
+    
+    def on_tool_call(
+        self,
+        span_id: str,
+        tool_name: str,
+        args: Dict[str, Any],
+        result: Any,
+        latency_ms: Optional[float] = None,
+    ) -> None:
+        """Called when a tool is invoked."""
+        ...
+    
+    def flush(self) -> None:
+        """Flush any pending data."""
+        ...
+    
+    def close(self) -> None:
+        """Close the adapter and release resources."""
+        ...
+
+
+class _LazyObsModule:
+    """
+    Lazy proxy for observability backends.
+    
+    Allows `from praisonaiagents import obs` without importing heavy deps.
+    Actual provider classes are loaded only when accessed.
+    """
+    
+    _PROVIDERS = {
+        "langfuse": "praisonai_tools.observability.providers.langfuse_provider",
+        "langsmith": "praisonai_tools.observability.providers.langsmith_provider",
+        "agentops": "praisonai_tools.observability.providers.agentops_provider",
+        "arize": "praisonai_tools.observability.providers.arize_phoenix_provider",
+        "phoenix": "praisonai_tools.observability.providers.arize_phoenix_provider",
+        "datadog": "praisonai_tools.observability.providers.datadog_provider",
+        "mlflow": "praisonai_tools.observability.providers.mlflow_provider",
+        "openlit": "praisonai_tools.observability.providers.openlit_provider",
+        "opik": "praisonai_tools.observability.providers.opik_provider",
+        "portkey": "praisonai_tools.observability.providers.portkey_provider",
+        "traceloop": "praisonai_tools.observability.providers.traceloop_provider",
+        "weave": "praisonai_tools.observability.providers.weave_provider",
+        "braintrust": "praisonai_tools.observability.providers.braintrust_provider",
+        "langtrace": "praisonai_tools.observability.providers.langtrace_provider",
+        "langwatch": "praisonai_tools.observability.providers.langwatch_provider",
+        "maxim": "praisonai_tools.observability.providers.maxim_provider",
+    }
+    
+    def __getattr__(self, name: str):
+        if name in self._PROVIDERS:
+            return self._create_provider_factory(name)
+        raise AttributeError(f"module 'obs' has no attribute {name!r}")
+    
+    def _create_provider_factory(self, provider_name: str):
+        """Create a factory function for a provider."""
+        def factory(**kwargs):
+            try:
+                import importlib
+                module = importlib.import_module(self._PROVIDERS[provider_name])
+                # Get the provider class (naming convention: XxxProvider)
+                class_name = f"{provider_name.title().replace('_', '')}Provider"
+                provider_cls = getattr(module, class_name, None)
+                if provider_cls:
+                    return provider_cls(**kwargs)
+                # Fallback: try to find any class ending with Provider
+                for attr_name in dir(module):
+                    if attr_name.endswith("Provider"):
+                        return getattr(module, attr_name)(**kwargs)
+                raise AttributeError(f"No provider class found in {module}")
+            except ImportError as e:
+                raise ImportError(
+                    f"Observability provider '{provider_name}' requires praisonai-tools. "
+                    f"Install with: pip install praisonai-tools\n"
+                    f"Original error: {e}"
+                ) from e
+        return factory
+    
+    def auto(self, **kwargs):
+        """
+        Auto-detect observability provider from environment variables.
+        
+        Checks for common env vars like LANGFUSE_PUBLIC_KEY, LANGSMITH_API_KEY, etc.
+        Returns None if no provider is detected.
+        """
+        import os
+        
+        # Check for common provider env vars
+        provider_env_map = {
+            "LANGFUSE_PUBLIC_KEY": "langfuse",
+            "LANGSMITH_API_KEY": "langsmith",
+            "AGENTOPS_API_KEY": "agentops",
+            "ARIZE_API_KEY": "arize",
+            "DATADOG_API_KEY": "datadog",
+            "MLFLOW_TRACKING_URI": "mlflow",
+            "OPENLIT_API_KEY": "openlit",
+            "OPIK_API_KEY": "opik",
+            "PORTKEY_API_KEY": "portkey",
+            "TRACELOOP_API_KEY": "traceloop",
+            "WANDB_API_KEY": "weave",
+            "BRAINTRUST_API_KEY": "braintrust",
+        }
+        
+        for env_var, provider in provider_env_map.items():
+            if os.getenv(env_var):
+                return self._create_provider_factory(provider)(**kwargs)
+        
+        return None
+    
+    def __repr__(self):
+        return "<module 'praisonaiagents.obs' (lazy observability backends)>"
+    
+    def __call__(self, provider: str = "auto", **kwargs):
+        """
+        Shortcut: obs("langfuse") or obs() for auto-detection.
+        """
+        if provider == "auto":
+            return self.auto(**kwargs)
+        return self._create_provider_factory(provider)(**kwargs)
+
+
+# Singleton lazy module instance
+obs = _LazyObsModule()
+
+
+__all__ = [
+    "ObservabilityAdapter",
+    "Span",
+    "Trace",
+    "obs",
+]

src/praisonai-agents/praisonaiagents/session.py

@@ -4,16 +4,20 @@ Session Management for PraisonAI Agents
 A simple wrapper around existing stateful capabilities to provide a unified
 session API for developers building stateful agent applications.
 """
+from __future__ import annotations
 
 import os
 import uuid
 import requests
 import json
 import time
-from typing import Any, Dict, List, Optional
+from typing import Any, Dict, List, Optional, TYPE_CHECKING
+
+if TYPE_CHECKING:
+    from .memory.memory import Memory
+    from .knowledge.knowledge import Knowledge
+
 from .agent import Agent
-from .memory import Memory
-from .knowledge import Knowledge
 
 
 class Session:
@@ -112,6 +116,7 @@ class Session:
         if self.is_remote:
             raise ValueError("Memory operations are not available for remote agent sessions")
         if self._memory is None:
+            from .memory.memory import Memory
             self._memory = Memory(config=self.memory_config)
         return self._memory
 
@@ -121,6 +126,7 @@ class Session:
         if self.is_remote:
             raise ValueError("Knowledge operations are not available for remote agent sessions")
         if self._knowledge is None:
+            from .knowledge.knowledge import Knowledge
             self._knowledge = Knowledge(config=self.knowledge_config)
         return self._knowledge
 

src/praisonai-agents/praisonaiagents/tools/__init__.py

@@ -2,6 +2,9 @@
 from importlib import import_module
 from typing import Any
 
+# Export Injected type for tool state injection
+from .injected import Injected
+
 # Map of function names to their module and class (if any)
 TOOL_MAPPINGS = {
     # Direct functions
@@ -286,4 +289,4 @@ def __getattr__(name: str) -> Any:
         method = getattr(_instances[class_name], name)
         return method
 
-__all__ = list(TOOL_MAPPINGS.keys())
+__all__ = list(TOOL_MAPPINGS.keys()) + ['Injected']

src/praisonai-agents/praisonaiagents/tools/decorator.py

@@ -14,6 +14,14 @@ Usage:
     @tool(name="web_search", description="Search the internet")
     def search(query: str, max_results: int = 5) -> list:
         return [...]
+    
+    # With injected state:
+    from praisonaiagents.tools import Injected
+    
+    @tool
+    def my_tool(query: str, state: Injected[dict]) -> str:
+        '''Tool with injected state.'''
+        return f"session={state.get('session_id')}"
 """
 
 import inspect
@@ -22,12 +30,19 @@ import logging
 from typing import Any, Callable, Dict, Optional, Union, get_type_hints
 
 from .base import BaseTool
+from .injected import (
+    is_injected_type, 
+    get_injected_params, 
+    inject_state_into_kwargs,
+    filter_injected_from_schema
+)
 
 
 class FunctionTool(BaseTool):
     """A BaseTool wrapper for plain functions.
     
     Created automatically by the @tool decorator.
+    Supports Injected[T] parameters for state injection.
     """
     
     def __init__(
@@ -42,6 +57,9 @@ class FunctionTool(BaseTool):
         self.description = description or func.__doc__ or f"Tool: {self.name}"
         self.version = version
         
+        # Detect injected parameters
+        self._injected_params = get_injected_params(func)
+        
         # Generate schema from the original function, not run()
         self.parameters = self._generate_schema_from_func(func)
         
@@ -55,8 +73,16 @@ class FunctionTool(BaseTool):
         if not self.description:
             self.description = self.__class__.__doc__ or f"Tool: {self.name}"
     
+    @property
+    def injected_params(self) -> Dict[str, Any]:
+        """Get the injected parameters for this tool."""
+        return self._injected_params
+    
     def _generate_schema_from_func(self, func: Callable) -> Dict[str, Any]:
-        """Generate JSON Schema from the wrapped function's signature."""
+        """Generate JSON Schema from the wrapped function's signature.
+        
+        Injected parameters are excluded from the schema.
+        """
         schema = {
             "type": "object",
             "properties": {},
@@ -71,8 +97,17 @@ class FunctionTool(BaseTool):
                 if param_name in ('self', 'cls'):
                     continue
                 
+                # Skip injected parameters - they don't go in schema
+                if param_name in self._injected_params:
+                    continue
+                
                 # Get type hint
                 param_type = hints.get(param_name, Any)
+                
+                # Double-check it's not an Injected type
+                if is_injected_type(param_type):
+                    continue
+                
                 json_type = BaseTool._python_type_to_json(param_type)
                 
                 schema["properties"][param_name] = {"type": json_type}
@@ -86,11 +121,18 @@ class FunctionTool(BaseTool):
         return schema
     
     def run(self, **kwargs) -> Any:
-        """Execute the wrapped function."""
+        """Execute the wrapped function with injected state."""
+        # Inject state for any Injected parameters
+        kwargs = inject_state_into_kwargs(kwargs, self._injected_params)
         return self._func(**kwargs)
     
     def __call__(self, *args, **kwargs) -> Any:
-        """Allow calling with positional args like the original function."""
+        """Allow calling with positional args like the original function.
+        
+        Injects state for Injected parameters.
+        """
+        # Inject state for any Injected parameters
+        kwargs = inject_state_into_kwargs(kwargs, self._injected_params)
         return self._func(*args, **kwargs)
 
 

src/praisonai-agents/praisonaiagents/tools/injected.py

@@ -0,0 +1,213 @@
+"""
+Injected State for PraisonAI Tools.
+
+Provides a mechanism for tools to receive injected state (agent context)
+without exposing it in the tool's public schema.
+
+Zero Performance Impact:
+- When no Injected params, no overhead
+- Context is thread-local for safety
+- Lazy evaluation of state
+
+Usage:
+    from praisonaiagents import tool
+    from praisonaiagents.tools import Injected
+    
+    @tool
+    def my_tool(query: str, state: Injected[dict]) -> str:
+        '''Tool with injected state.'''
+        session_id = state.get('session_id')
+        return f"Query: {query}, Session: {session_id}"
+"""
+
+from dataclasses import dataclass, field
+from typing import Any, Dict, Generic, List, Optional, TypeVar, get_type_hints, get_origin, get_args
+from contextvars import ContextVar
+from contextlib import contextmanager
+
+
+T = TypeVar('T')
+
+
+class Injected(Generic[T]):
+    """Type marker for injected parameters.
+    
+    Parameters annotated with Injected[T] will:
+    - Not appear in the tool's public schema
+    - Be automatically injected at runtime with the current agent state
+    
+    Supported types:
+    - Injected[dict] - Receives state as a dictionary
+    - Injected[AgentState] - Receives state as AgentState dataclass
+    
+    Example:
+        @tool
+        def my_tool(query: str, state: Injected[dict]) -> str:
+            return f"session={state.get('session_id')}"
+    """
+    pass
+
+
+@dataclass
+class AgentState:
+    """State object injected into tools.
+    
+    Contains context about the current agent execution.
+    """
+    agent_id: str
+    run_id: str
+    session_id: str
+    last_user_message: Optional[str] = None
+    last_agent_message: Optional[str] = None
+    memory: Optional[Any] = None
+    previous_tool_results: List[Any] = field(default_factory=list)
+    metadata: Dict[str, Any] = field(default_factory=dict)
+    
+    def to_dict(self) -> Dict[str, Any]:
+        """Convert to dictionary for Injected[dict] usage."""
+        return {
+            'agent_id': self.agent_id,
+            'run_id': self.run_id,
+            'session_id': self.session_id,
+            'last_user_message': self.last_user_message,
+            'last_agent_message': self.last_agent_message,
+            'previous_tool_results': self.previous_tool_results,
+            'metadata': self.metadata,
+        }
+
+
+# Thread-local context for current state
+_current_state: ContextVar[Optional[AgentState]] = ContextVar('current_state', default=None)
+
+
+def get_current_state() -> Optional[AgentState]:
+    """Get the current injection state."""
+    return _current_state.get()
+
+
+def set_current_state(state: Optional[AgentState]) -> None:
+    """Set the current injection state."""
+    _current_state.set(state)
+
+
+@contextmanager
+def with_injection_context(state: AgentState):
+    """Context manager for setting injection state.
+    
+    Usage:
+        with with_injection_context(state):
+            result = my_tool(query="test")
+    """
+    token = _current_state.set(state)
+    try:
+        yield
+    finally:
+        _current_state.reset(token)
+
+
+def is_injected_type(annotation: Any) -> bool:
+    """Check if a type annotation is Injected[T]."""
+    origin = get_origin(annotation)
+    if origin is Injected:
+        return True
+    # Handle string annotations
+    if isinstance(annotation, str) and 'Injected' in annotation:
+        return True
+    return False
+
+
+def get_injected_type(annotation: Any) -> Optional[type]:
+    """Get the inner type T from Injected[T]."""
+    origin = get_origin(annotation)
+    if origin is Injected:
+        args = get_args(annotation)
+        if args:
+            return args[0]
+    return None
+
+
+def resolve_injected_value(annotation: Any, state: Optional[AgentState]) -> Any:
+    """Resolve the value to inject based on annotation type.
+    
+    Args:
+        annotation: The Injected[T] annotation
+        state: The current agent state
+        
+    Returns:
+        The resolved value (dict or AgentState)
+    """
+    if state is None:
+        # Return empty dict if no state available
+        return {}
+    
+    inner_type = get_injected_type(annotation)
+    
+    if inner_type is dict or inner_type is Dict:
+        return state.to_dict()
+    elif inner_type is AgentState:
+        return state
+    else:
+        # Default to dict
+        return state.to_dict()
+
+
+def get_injected_params(func) -> Dict[str, Any]:
+    """Get all injected parameters from a function.
+    
+    Returns:
+        Dict mapping param name to its Injected[T] annotation
+    """
+    try:
+        hints = get_type_hints(func)
+    except Exception:
+        # If we can't get hints, return empty
+        return {}
+    
+    injected = {}
+    for name, hint in hints.items():
+        if name == 'return':
+            continue
+        if is_injected_type(hint):
+            injected[name] = hint
+    
+    return injected
+
+
+def filter_injected_from_schema(properties: Dict[str, Any], required: List[str], 
+                                 injected_params: Dict[str, Any]) -> tuple:
+    """Remove injected parameters from schema.
+    
+    Args:
+        properties: Schema properties dict
+        required: List of required param names
+        injected_params: Dict of injected param names
+        
+    Returns:
+        Tuple of (filtered_properties, filtered_required)
+    """
+    filtered_props = {k: v for k, v in properties.items() if k not in injected_params}
+    filtered_required = [r for r in required if r not in injected_params]
+    return filtered_props, filtered_required
+
+
+def inject_state_into_kwargs(kwargs: Dict[str, Any], injected_params: Dict[str, Any]) -> Dict[str, Any]:
+    """Inject state values into kwargs for injected parameters.
+    
+    Args:
+        kwargs: The original kwargs
+        injected_params: Dict of injected param names to annotations
+        
+    Returns:
+        Updated kwargs with injected values
+    """
+    if not injected_params:
+        return kwargs
+    
+    state = get_current_state()
+    result = kwargs.copy()
+    
+    for param_name, annotation in injected_params.items():
+        if param_name not in result:
+            result[param_name] = resolve_injected_value(annotation, state)
+    
+    return result

src/praisonai-agents/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "praisonaiagents"
-version = "0.2.1"
+version = "0.3.2"
 description = "Praison AI agents for completing complex tasks with Self Reflection Agents"
 readme = "README.md"
 requires-python = ">=3.10"

src/praisonai-agents/tests/test_planning.py

@@ -672,16 +672,16 @@ class TestPlanningAgent:
         
         planner = PlanningAgent()
         
-        assert planner.llm_model == "gpt-4o-mini"
+        assert planner.llm_model in ["gpt-4o-mini", "gpt-5-nano"]
         assert planner.read_only is True
         
     def test_planning_agent_custom_llm(self):
         """Test PlanningAgent with custom LLM."""
         from praisonaiagents.planning import PlanningAgent
         
-        planner = PlanningAgent(llm="gpt-4o")
+        planner = PlanningAgent(llm="gpt-5-nano")
         
-        assert planner.llm_model == "gpt-4o"
+        assert planner.llm_model == "gpt-5-nano"
         
     def test_planning_agent_allowed_tools(self):
         """Test PlanningAgent has only read-only tools."""
@@ -840,10 +840,10 @@ class TestPraisonAIAgentsPlanningIntegration:
             agents=[agent],
             tasks=[task],
             planning=True,
-            planning_llm="gpt-4o"
+            planning_llm="gpt-5-nano"
         )
         
-        assert agents.planning_llm == "gpt-4o"
+        assert agents.planning_llm == "gpt-5-nano"
         
     def test_agents_auto_approve_plan(self):
         """Test auto-approve plan option."""
@@ -867,7 +867,7 @@ class TestPraisonAIAgentsPlanningIntegration:
         from praisonaiagents import Agent, Task, PraisonAIAgents
         from praisonaiagents.planning import Plan, PlanStep
         
-        agent = Agent(name="Test", role="Tester", llm="gpt-4o-mini")
+        agent = Agent(name="Test", role="Tester", llm="gpt-5-nano")
         task = Task(description="Test task", agent=agent)
         
         agents = PraisonAIAgents(
@@ -880,7 +880,7 @@ class TestPraisonAIAgentsPlanningIntegration:
         # Test that planning is enabled and configured correctly
         assert agents.planning is True
         assert agents.auto_approve_plan is True
-        assert agents.planning_llm == "gpt-4o-mini"
+        assert agents.planning_llm == "gpt-5-nano"
         assert agents._current_plan is None  # No plan created yet
         assert agents._todo_list is None
                 

src/praisonai-agents/tests/unit/test_configurable_model.py

@@ -0,0 +1,234 @@
+"""
+TDD Tests for Configurable Model (runtime model switching).
+
+These tests are written FIRST before implementation.
+"""
+
+import pytest
+from unittest.mock import Mock, patch
+
+
+class TestConfigurableModelBasic:
+    """Test basic configurable model functionality."""
+    
+    def test_agent_accepts_llm_config_configurable(self):
+        """Agent should accept llm_config with configurable=True."""
+        from praisonaiagents import Agent
+        
+        agent = Agent(
+            name="Test",
+            instructions="Test agent",
+            llm="gpt-4o",
+            llm_config={"configurable": True}
+        )
+        assert agent is not None
+        assert getattr(agent, '_llm_configurable', False) is True
+    
+    def test_agent_chat_accepts_config_parameter(self):
+        """Agent.chat() should accept config parameter for model override."""
+        from praisonaiagents import Agent
+        
+        agent = Agent(
+            name="Test",
+            instructions="Test agent",
+            llm="gpt-4o",
+            llm_config={"configurable": True}
+        )
+        
+        # Should not raise - config parameter should be accepted
+        # We'll mock the actual LLM call
+        with patch.object(agent, '_chat_completion', return_value="mocked"):
+            result = agent.chat(
+                "Hello",
+                config={"model": "claude-3-5-sonnet"}
+            )
+    
+    def test_config_model_override(self):
+        """config.model should override default model for that call."""
+        from praisonaiagents import Agent
+        
+        agent = Agent(
+            name="Test",
+            instructions="Test agent",
+            llm="gpt-4o",
+            llm_config={"configurable": True}
+        )
+        
+        # Track which model was used
+        used_model = [None]
+        
+        original_chat = agent._chat_completion
+        def mock_chat(*args, **kwargs):
+            # The model should be accessible somehow
+            used_model[0] = getattr(agent, '_current_model', agent.llm)
+            return "response"
+        
+        with patch.object(agent, '_chat_completion', side_effect=mock_chat):
+            agent.chat("Hello", config={"model": "claude-3-5-sonnet"})
+        
+        # Model should have been switched for this call
+        # Implementation will set _current_model or similar
+    
+    def test_config_does_not_mutate_default(self):
+        """Per-call config should not change agent's default model."""
+        from praisonaiagents import Agent
+        
+        agent = Agent(
+            name="Test",
+            instructions="Test agent",
+            llm="gpt-4o",
+            llm_config={"configurable": True}
+        )
+        
+        original_llm = agent.llm
+        
+        with patch.object(agent, '_chat_completion', return_value="mocked"):
+            agent.chat("Hello", config={"model": "claude-3-5-sonnet"})
+        
+        # Default should be unchanged
+        assert agent.llm == original_llm
+    
+    def test_config_temperature_override(self):
+        """config.temperature should override for that call."""
+        from praisonaiagents import Agent
+        
+        agent = Agent(
+            name="Test",
+            instructions="Test agent",
+            llm="gpt-4o",
+            llm_config={"configurable": True}
+        )
+        
+        # Should accept temperature in config
+        with patch.object(agent, '_chat_completion', return_value="mocked") as mock:
+            agent.chat("Hello", config={"temperature": 0.2})
+            # Temperature should be passed through
+
+
+class TestConfigurableModelCaching:
+    """Test model client caching for configurable models."""
+    
+    def test_model_client_cached(self):
+        """Model clients should be cached to avoid recreation."""
+        from praisonaiagents.llm import LLM
+        
+        llm = LLM(model="gpt-4o", configurable=True)
+        
+        # First call with model A
+        # Second call with model A should reuse client
+        # This is internal behavior - we test via call count or similar
+    
+    def test_cache_size_limited(self):
+        """Model client cache should have a size limit."""
+        from praisonaiagents.llm import LLM
+        
+        llm = LLM(model="gpt-4o", configurable=True, cache_size=3)
+        
+        # Should not grow unbounded
+
+
+class TestConfigurableModelThreadSafety:
+    """Test thread safety of configurable models."""
+    
+    def test_concurrent_calls_different_models(self):
+        """Concurrent calls with different models should not interfere."""
+        import threading
+        from praisonaiagents import Agent
+        
+        agent = Agent(
+            name="Test",
+            instructions="Test agent",
+            llm="gpt-4o",
+            llm_config={"configurable": True}
+        )
+        
+        results = {}
+        errors = []
+        
+        def call_with_model(model_name, thread_id):
+            try:
+                with patch.object(agent, '_chat_completion', return_value=f"response_{model_name}"):
+                    result = agent.chat("Hello", config={"model": model_name})
+                    results[thread_id] = result
+            except Exception as e:
+                errors.append(e)
+        
+        threads = [
+            threading.Thread(target=call_with_model, args=("model_a", 1)),
+            threading.Thread(target=call_with_model, args=("model_b", 2)),
+        ]
+        
+        for t in threads:
+            t.start()
+        for t in threads:
+            t.join()
+        
+        assert len(errors) == 0
+
+
+class TestConfigurableModelValidation:
+    """Test validation of configurable model settings."""
+    
+    def test_config_without_configurable_flag_ignored(self):
+        """If llm_config.configurable is False, config param should be ignored."""
+        from praisonaiagents import Agent
+        
+        agent = Agent(
+            name="Test",
+            instructions="Test agent",
+            llm="gpt-4o"
+            # No llm_config or configurable=False
+        )
+        
+        # Should still work, just ignore the config
+        with patch.object(agent, '_chat_completion', return_value="mocked"):
+            result = agent.chat("Hello", config={"model": "other"})
+            # Should use default model, not "other"
+    
+    def test_invalid_config_keys_ignored(self):
+        """Unknown keys in config should be ignored, not error."""
+        from praisonaiagents import Agent
+        
+        agent = Agent(
+            name="Test",
+            instructions="Test agent",
+            llm="gpt-4o",
+            llm_config={"configurable": True}
+        )
+        
+        # Should not raise for unknown keys
+        with patch.object(agent, '_chat_completion', return_value="mocked"):
+            result = agent.chat("Hello", config={"unknown_key": "value"})
+
+
+class TestConfigurableModelProvider:
+    """Test provider switching in configurable models."""
+    
+    def test_config_provider_override(self):
+        """config.provider should allow switching providers."""
+        from praisonaiagents import Agent
+        
+        agent = Agent(
+            name="Test",
+            instructions="Test agent",
+            llm="gpt-4o",
+            llm_config={"configurable": True}
+        )
+        
+        # Should accept provider in config
+        with patch.object(agent, '_chat_completion', return_value="mocked"):
+            agent.chat("Hello", config={"provider": "anthropic", "model": "claude-3-5-sonnet"})
+    
+    def test_model_string_with_provider_prefix(self):
+        """Model string like 'anthropic/claude-3' should work."""
+        from praisonaiagents import Agent
+        
+        agent = Agent(
+            name="Test",
+            instructions="Test agent",
+            llm="gpt-4o",
+            llm_config={"configurable": True}
+        )
+        
+        with patch.object(agent, '_chat_completion', return_value="mocked"):
+            agent.chat("Hello", config={"model": "anthropic/claude-3-5-sonnet"})

src/praisonai-agents/tests/unit/test_injected_state.py

@@ -0,0 +1,367 @@
+"""
+TDD Tests for InjectedState for Tools.
+
+These tests are written FIRST before implementation.
+"""
+
+import pytest
+from unittest.mock import Mock, patch
+
+
+class TestInjectedTypeExists:
+    """Test that Injected type marker exists."""
+    
+    def test_injected_type_exists(self):
+        """Injected type should exist in praisonaiagents.tools."""
+        from praisonaiagents.tools import Injected
+        
+        # Should be a generic type marker
+        assert Injected is not None
+    
+    def test_injected_generic_usage(self):
+        """Injected should work as a generic type annotation."""
+        from praisonaiagents.tools import Injected
+        
+        # Should be usable as Injected[dict], Injected[SomeType], etc.
+        def my_tool(query: str, state: Injected[dict]) -> str:
+            return query
+        
+        # Should have annotations
+        annotations = my_tool.__annotations__
+        assert 'state' in annotations
+    
+    def test_injected_with_agent_state(self):
+        """Injected[AgentState] should work."""
+        from praisonaiagents.tools import Injected
+        from praisonaiagents.tools.injected import AgentState
+        
+        def my_tool(query: str, state: Injected[AgentState]) -> str:
+            return query
+        
+        annotations = my_tool.__annotations__
+        assert 'state' in annotations
+
+
+class TestAgentStateType:
+    """Test AgentState type for injection."""
+    
+    def test_agent_state_exists(self):
+        """AgentState type should exist."""
+        from praisonaiagents.tools.injected import AgentState
+        
+        assert AgentState is not None
+    
+    def test_agent_state_has_required_fields(self):
+        """AgentState should have agent_id, run_id, session_id."""
+        from praisonaiagents.tools.injected import AgentState
+        
+        state = AgentState(
+            agent_id="agent-123",
+            run_id="run-456",
+            session_id="session-789"
+        )
+        
+        assert state.agent_id == "agent-123"
+        assert state.run_id == "run-456"
+        assert state.session_id == "session-789"
+    
+    def test_agent_state_optional_fields(self):
+        """AgentState should have optional fields."""
+        from praisonaiagents.tools.injected import AgentState
+        
+        state = AgentState(
+            agent_id="a",
+            run_id="r",
+            session_id="s",
+            last_user_message="Hello",
+            last_agent_message="Hi there",
+            memory=Mock(),
+            previous_tool_results=["result1", "result2"]
+        )
+        
+        assert state.last_user_message == "Hello"
+        assert state.last_agent_message == "Hi there"
+        assert state.memory is not None
+        assert len(state.previous_tool_results) == 2
+
+
+class TestToolDecoratorInjection:
+    """Test @tool decorator handles Injected parameters."""
+    
+    def test_tool_decorator_detects_injected(self):
+        """@tool should detect Injected parameters."""
+        from praisonaiagents import tool
+        from praisonaiagents.tools import Injected
+        
+        @tool
+        def my_tool(query: str, state: Injected[dict]) -> str:
+            """A tool with injected state."""
+            return f"query={query}, session={state.get('session_id')}"
+        
+        # Should have metadata about injected params
+        assert hasattr(my_tool, '_injected_params') or hasattr(my_tool, 'injected_params')
+    
+    def test_injected_params_not_in_schema(self):
+        """Injected parameters should not appear in tool schema."""
+        from praisonaiagents import tool
+        from praisonaiagents.tools import Injected
+        
+        @tool
+        def my_tool(query: str, state: Injected[dict]) -> str:
+            """A tool with injected state."""
+            return query
+        
+        schema = my_tool.get_schema()
+        params = schema['function']['parameters']['properties']
+        
+        # 'query' should be in schema
+        assert 'query' in params
+        # 'state' should NOT be in schema (it's injected)
+        assert 'state' not in params
+    
+    def test_injected_params_not_required(self):
+        """Injected parameters should not be in required list."""
+        from praisonaiagents import tool
+        from praisonaiagents.tools import Injected
+        
+        @tool
+        def my_tool(query: str, state: Injected[dict]) -> str:
+            """A tool with injected state."""
+            return query
+        
+        schema = my_tool.get_schema()
+        required = schema['function']['parameters'].get('required', [])
+        
+        assert 'state' not in required
+        assert 'query' in required
+
+
+class TestToolExecutionInjection:
+    """Test that injected state is provided at runtime."""
+    
+    def test_injected_state_provided_at_runtime(self):
+        """When tool is called, injected state should be provided."""
+        from praisonaiagents import tool
+        from praisonaiagents.tools import Injected
+        from praisonaiagents.tools.injected import AgentState
+        
+        received_state = [None]
+        
+        @tool
+        def my_tool(query: str, state: Injected[AgentState]) -> str:
+            """A tool with injected state."""
+            received_state[0] = state
+            return f"session={state.session_id}"
+        
+        # Simulate execution with injection context
+        from praisonaiagents.tools.injected import with_injection_context
+        
+        mock_state = AgentState(
+            agent_id="a",
+            run_id="r",
+            session_id="test-session"
+        )
+        
+        with with_injection_context(mock_state):
+            result = my_tool(query="hello")
+        
+        assert received_state[0] is not None
+        assert received_state[0].session_id == "test-session"
+    
+    def test_injected_dict_state(self):
+        """Injected[dict] should provide dict with state info."""
+        from praisonaiagents import tool
+        from praisonaiagents.tools import Injected
+        from praisonaiagents.tools.injected import with_injection_context, AgentState
+        
+        received_state = [None]
+        
+        @tool
+        def my_tool(query: str, state: Injected[dict]) -> str:
+            """A tool with injected dict state."""
+            received_state[0] = state
+            return "ok"
+        
+        mock_state = AgentState(
+            agent_id="a",
+            run_id="r",
+            session_id="s"
+        )
+        
+        with with_injection_context(mock_state):
+            my_tool(query="test")
+        
+        assert isinstance(received_state[0], dict)
+        assert 'session_id' in received_state[0]
+
+
+class TestAgentToolExecution:
+    """Test Agent executes tools with injection."""
+    
+    def test_agent_injects_state_to_tools(self):
+        """Agent should inject state when executing tools."""
+        from praisonaiagents import Agent, tool
+        from praisonaiagents.tools import Injected
+        
+        received_session = [None]
+        
+        @tool
+        def show_state(query: str, state: Injected[dict]) -> str:
+            """Show the injected state."""
+            received_session[0] = state.get('session_id')
+            return f"session={state.get('session_id')}"
+        
+        agent = Agent(
+            name="Test",
+            instructions="Test agent",
+            tools=[show_state],
+            session_id="my-session-123"
+        )
+        
+        # Execute the tool directly through agent
+        result = agent.execute_tool("show_state", {"query": "test"})
+        
+        # State should have been injected
+        assert received_session[0] == "my-session-123"
+
+
+class TestInjectionContextManager:
+    """Test injection context manager."""
+    
+    def test_with_injection_context_exists(self):
+        """with_injection_context should exist."""
+        from praisonaiagents.tools.injected import with_injection_context
+        
+        assert with_injection_context is not None
+    
+    def test_context_manager_sets_state(self):
+        """Context manager should set current state."""
+        from praisonaiagents.tools.injected import (
+            with_injection_context, 
+            get_current_state,
+            AgentState
+        )
+        
+        state = AgentState(agent_id="a", run_id="r", session_id="s")
+        
+        with with_injection_context(state):
+            current = get_current_state()
+            assert current is state
+        
+        # After context, should be None
+        assert get_current_state() is None
+    
+    def test_nested_contexts(self):
+        """Nested contexts should work correctly."""
+        from praisonaiagents.tools.injected import (
+            with_injection_context,
+            get_current_state,
+            AgentState
+        )
+        
+        state1 = AgentState(agent_id="a1", run_id="r1", session_id="s1")
+        state2 = AgentState(agent_id="a2", run_id="r2", session_id="s2")
+        
+        with with_injection_context(state1):
+            assert get_current_state().agent_id == "a1"
+            
+            with with_injection_context(state2):
+                assert get_current_state().agent_id == "a2"
+            
+            # Back to state1
+            assert get_current_state().agent_id == "a1"
+
+
+class TestAsyncInjection:
+    """Test async tool injection."""
+    
+    @pytest.mark.asyncio
+    async def test_async_tool_injection(self):
+        """Async tools should also receive injected state."""
+        from praisonaiagents import tool
+        from praisonaiagents.tools import Injected
+        from praisonaiagents.tools.injected import with_injection_context, AgentState
+        
+        received_state = [None]
+        
+        @tool
+        async def async_tool(query: str, state: Injected[dict]) -> str:
+            """Async tool with injected state."""
+            received_state[0] = state
+            return "ok"
+        
+        state = AgentState(agent_id="a", run_id="r", session_id="async-session")
+        
+        with with_injection_context(state):
+            await async_tool(query="test")
+        
+        assert received_state[0] is not None
+        assert received_state[0].get('session_id') == "async-session"
+
+
+class TestInjectionZeroOverhead:
+    """Test that injection has zero overhead when not used."""
+    
+    def test_no_injected_params_no_overhead(self):
+        """Tools without Injected params should have no overhead."""
+        from praisonaiagents import tool
+        
+        @tool
+        def simple_tool(query: str) -> str:
+            """Simple tool without injection."""
+            return query
+        
+        # Should work normally without any injection context
+        result = simple_tool(query="hello")
+        assert result == "hello"
+    
+    def test_schema_generation_fast(self):
+        """Schema generation should be fast even with Injected params."""
+        from praisonaiagents import tool
+        from praisonaiagents.tools import Injected
+        import time
+        
+        @tool
+        def my_tool(a: str, b: int, c: Injected[dict], d: float) -> str:
+            """Tool with mixed params."""
+            return a
+        
+        start = time.monotonic()
+        for _ in range(100):
+            my_tool.get_schema()
+        elapsed = time.monotonic() - start
+        
+        # Should be very fast
+        assert elapsed < 0.1  # 100 calls in < 100ms
+
+
+class TestBackwardCompatibility:
+    """Test backward compatibility with existing tools."""
+    
+    def test_existing_tools_still_work(self):
+        """Existing tools without Injected should continue to work."""
+        from praisonaiagents import tool
+        
+        @tool
+        def legacy_tool(query: str, count: int = 5) -> str:
+            """Legacy tool."""
+            return f"{query} x {count}"
+        
+        result = legacy_tool(query="test", count=3)
+        assert result == "test x 3"
+    
+    def test_existing_tool_schema_unchanged(self):
+        """Existing tool schemas should be unchanged."""
+        from praisonaiagents import tool
+        
+        @tool
+        def legacy_tool(query: str, count: int = 5) -> str:
+            """Legacy tool."""
+            return f"{query} x {count}"
+        
+        schema = legacy_tool.get_schema()
+        params = schema['function']['parameters']['properties']
+        
+        assert 'query' in params
+        assert 'count' in params
+        assert len(params) == 2

src/praisonai-agents/tests/unit/test_middleware.py

@@ -0,0 +1,403 @@
+"""
+TDD Tests for Middleware System (wrap_model_call, wrap_tool_call).
+
+These tests are written FIRST before implementation.
+"""
+
+import pytest
+from unittest.mock import Mock, MagicMock, patch, AsyncMock
+from dataclasses import dataclass
+from typing import Any, Dict, Optional
+
+
+class TestMiddlewareTypes:
+    """Test middleware type definitions."""
+    
+    def test_invocation_context_exists(self):
+        """InvocationContext dataclass should exist with required fields."""
+        from praisonaiagents.hooks.middleware import InvocationContext
+        
+        ctx = InvocationContext(
+            agent_id="agent-123",
+            run_id="run-456",
+            session_id="session-789"
+        )
+        assert ctx.agent_id == "agent-123"
+        assert ctx.run_id == "run-456"
+        assert ctx.session_id == "session-789"
+    
+    def test_invocation_context_optional_fields(self):
+        """InvocationContext should have optional tool_name, model_name, metadata."""
+        from praisonaiagents.hooks.middleware import InvocationContext
+        
+        ctx = InvocationContext(
+            agent_id="a",
+            run_id="r",
+            session_id="s",
+            tool_name="my_tool",
+            model_name="gpt-4o",
+            metadata={"key": "value"}
+        )
+        assert ctx.tool_name == "my_tool"
+        assert ctx.model_name == "gpt-4o"
+        assert ctx.metadata == {"key": "value"}
+    
+    def test_model_request_exists(self):
+        """ModelRequest dataclass for before_model hooks."""
+        from praisonaiagents.hooks.middleware import ModelRequest
+        
+        req = ModelRequest(
+            messages=[{"role": "user", "content": "Hello"}],
+            model="gpt-4o",
+            temperature=0.7,
+            context=Mock()
+        )
+        assert req.messages == [{"role": "user", "content": "Hello"}]
+        assert req.model == "gpt-4o"
+        assert req.temperature == 0.7
+    
+    def test_model_response_exists(self):
+        """ModelResponse dataclass for after_model hooks."""
+        from praisonaiagents.hooks.middleware import ModelResponse
+        
+        resp = ModelResponse(
+            content="Hello back!",
+            model="gpt-4o",
+            usage={"prompt_tokens": 10, "completion_tokens": 5},
+            context=Mock()
+        )
+        assert resp.content == "Hello back!"
+        assert resp.model == "gpt-4o"
+    
+    def test_tool_request_exists(self):
+        """ToolRequest dataclass for before_tool hooks."""
+        from praisonaiagents.hooks.middleware import ToolRequest
+        
+        req = ToolRequest(
+            tool_name="get_weather",
+            arguments={"city": "Paris"},
+            context=Mock()
+        )
+        assert req.tool_name == "get_weather"
+        assert req.arguments == {"city": "Paris"}
+    
+    def test_tool_response_exists(self):
+        """ToolResponse dataclass for after_tool hooks."""
+        from praisonaiagents.hooks.middleware import ToolResponse
+        
+        resp = ToolResponse(
+            tool_name="get_weather",
+            result="Sunny, 22°C",
+            error=None,
+            context=Mock()
+        )
+        assert resp.result == "Sunny, 22°C"
+        assert resp.error is None
+
+
+class TestMiddlewareDecorators:
+    """Test middleware decorator functions."""
+    
+    def test_before_model_decorator_exists(self):
+        """before_model decorator should exist and tag functions."""
+        from praisonaiagents.hooks import before_model
+        
+        @before_model
+        def my_hook(request):
+            return request
+        
+        assert hasattr(my_hook, '_hook_type')
+        assert my_hook._hook_type == 'before_model'
+    
+    def test_after_model_decorator_exists(self):
+        """after_model decorator should exist and tag functions."""
+        from praisonaiagents.hooks import after_model
+        
+        @after_model
+        def my_hook(response):
+            return response
+        
+        assert hasattr(my_hook, '_hook_type')
+        assert my_hook._hook_type == 'after_model'
+    
+    def test_wrap_model_call_decorator_exists(self):
+        """wrap_model_call decorator should exist and tag functions."""
+        from praisonaiagents.hooks import wrap_model_call
+        
+        @wrap_model_call
+        def my_middleware(request, call_next):
+            return call_next(request)
+        
+        assert hasattr(my_middleware, '_hook_type')
+        assert my_middleware._hook_type == 'wrap_model_call'
+    
+    def test_before_tool_decorator_exists(self):
+        """before_tool decorator should exist and tag functions."""
+        from praisonaiagents.hooks import before_tool
+        
+        @before_tool
+        def my_hook(request):
+            return request
+        
+        assert hasattr(my_hook, '_hook_type')
+        assert my_hook._hook_type == 'before_tool'
+    
+    def test_after_tool_decorator_exists(self):
+        """after_tool decorator should exist and tag functions."""
+        from praisonaiagents.hooks import after_tool
+        
+        @after_tool
+        def my_hook(response):
+            return response
+        
+        assert hasattr(my_hook, '_hook_type')
+        assert my_hook._hook_type == 'after_tool'
+    
+    def test_wrap_tool_call_decorator_exists(self):
+        """wrap_tool_call decorator should exist and tag functions."""
+        from praisonaiagents.hooks import wrap_tool_call
+        
+        @wrap_tool_call
+        def my_middleware(request, call_next):
+            return call_next(request)
+        
+        assert hasattr(my_middleware, '_hook_type')
+        assert my_middleware._hook_type == 'wrap_tool_call'
+
+
+class TestMiddlewareChaining:
+    """Test middleware chain composition."""
+    
+    def test_wrap_model_call_chain_order(self):
+        """wrap_model_call hooks should compose in registration order."""
+        from praisonaiagents.hooks.middleware import MiddlewareChain
+        
+        calls = []
+        
+        def mw1(req, call_next):
+            calls.append("mw1_before")
+            result = call_next(req)
+            calls.append("mw1_after")
+            return result
+        
+        def mw2(req, call_next):
+            calls.append("mw2_before")
+            result = call_next(req)
+            calls.append("mw2_after")
+            return result
+        
+        def final_handler(req):
+            calls.append("final")
+            return "response"
+        
+        chain = MiddlewareChain([mw1, mw2])
+        result = chain.execute({"test": True}, final_handler)
+        
+        assert calls == ["mw1_before", "mw2_before", "final", "mw2_after", "mw1_after"]
+        assert result == "response"
+    
+    def test_empty_middleware_chain(self):
+        """Empty middleware chain should just call final handler."""
+        from praisonaiagents.hooks.middleware import MiddlewareChain
+        
+        def final_handler(req):
+            return "direct"
+        
+        chain = MiddlewareChain([])
+        result = chain.execute({"test": True}, final_handler)
+        assert result == "direct"
+    
+    def test_middleware_can_modify_request(self):
+        """Middleware should be able to modify request before passing on."""
+        from praisonaiagents.hooks.middleware import MiddlewareChain
+        
+        def add_context(req, call_next):
+            req["extra"] = "added"
+            return call_next(req)
+        
+        def final_handler(req):
+            return req.get("extra")
+        
+        chain = MiddlewareChain([add_context])
+        result = chain.execute({"original": True}, final_handler)
+        assert result == "added"
+    
+    def test_middleware_can_modify_response(self):
+        """Middleware should be able to modify response after call_next."""
+        from praisonaiagents.hooks.middleware import MiddlewareChain
+        
+        def uppercase_response(req, call_next):
+            result = call_next(req)
+            return result.upper()
+        
+        def final_handler(req):
+            return "hello"
+        
+        chain = MiddlewareChain([uppercase_response])
+        result = chain.execute({}, final_handler)
+        assert result == "HELLO"
+    
+    def test_middleware_can_short_circuit(self):
+        """Middleware can return early without calling call_next."""
+        from praisonaiagents.hooks.middleware import MiddlewareChain
+        
+        def block_middleware(req, call_next):
+            if req.get("block"):
+                return "blocked"
+            return call_next(req)
+        
+        def final_handler(req):
+            return "reached"
+        
+        chain = MiddlewareChain([block_middleware])
+        
+        # Should be blocked
+        result = chain.execute({"block": True}, final_handler)
+        assert result == "blocked"
+        
+        # Should pass through
+        result = chain.execute({"block": False}, final_handler)
+        assert result == "reached"
+
+
+class TestMiddlewareRetry:
+    """Test retry logic in middleware."""
+    
+    def test_wrap_tool_call_retry_on_error(self):
+        """wrap_tool_call should support retry patterns."""
+        from praisonaiagents.hooks.middleware import MiddlewareChain
+        
+        attempt_count = [0]
+        
+        def retry_middleware(req, call_next):
+            last_error = None
+            for i in range(3):
+                try:
+                    return call_next(req)
+                except Exception as e:
+                    last_error = e
+            raise last_error
+        
+        def flaky_handler(req):
+            attempt_count[0] += 1
+            if attempt_count[0] < 3:
+                raise RuntimeError("flaky")
+            return "success"
+        
+        chain = MiddlewareChain([retry_middleware])
+        result = chain.execute({}, flaky_handler)
+        assert result == "success"
+        assert attempt_count[0] == 3
+
+
+class TestAsyncMiddleware:
+    """Test async middleware support."""
+    
+    @pytest.mark.asyncio
+    async def test_async_wrap_model_call(self):
+        """Async middleware should work with async handlers."""
+        from praisonaiagents.hooks.middleware import AsyncMiddlewareChain
+        
+        calls = []
+        
+        async def async_mw(req, call_next):
+            calls.append("before")
+            result = await call_next(req)
+            calls.append("after")
+            return result
+        
+        async def async_handler(req):
+            calls.append("handler")
+            return "async_result"
+        
+        chain = AsyncMiddlewareChain([async_mw])
+        result = await chain.execute({}, async_handler)
+        
+        assert calls == ["before", "handler", "after"]
+        assert result == "async_result"
+    
+    @pytest.mark.asyncio
+    async def test_async_middleware_chain_order(self):
+        """Async middleware should maintain order."""
+        from praisonaiagents.hooks.middleware import AsyncMiddlewareChain
+        
+        order = []
+        
+        async def mw1(req, call_next):
+            order.append(1)
+            result = await call_next(req)
+            order.append(-1)
+            return result
+        
+        async def mw2(req, call_next):
+            order.append(2)
+            result = await call_next(req)
+            order.append(-2)
+            return result
+        
+        async def handler(req):
+            order.append(0)
+            return "done"
+        
+        chain = AsyncMiddlewareChain([mw1, mw2])
+        await chain.execute({}, handler)
+        
+        assert order == [1, 2, 0, -2, -1]
+
+
+class TestAgentMiddlewareIntegration:
+    """Test middleware integration with Agent class."""
+    
+    def test_agent_accepts_hooks_parameter(self):
+        """Agent should accept hooks parameter."""
+        from praisonaiagents import Agent
+        from praisonaiagents.hooks import before_model
+        
+        @before_model
+        def my_hook(req):
+            return req
+        
+        # Should not raise
+        agent = Agent(
+            name="Test",
+            instructions="Test agent",
+            hooks=[my_hook]
+        )
+        assert agent is not None
+    
+    def test_agent_hooks_empty_by_default(self):
+        """Agent should have empty hooks by default."""
+        from praisonaiagents import Agent
+        
+        agent = Agent(name="Test", instructions="Test")
+        # Should have no hooks or empty hooks list
+        hooks = getattr(agent, '_hooks', None) or getattr(agent, 'hooks', [])
+        assert hooks is None or len(hooks) == 0
+
+
+class TestMiddlewareZeroOverhead:
+    """Test that middleware has zero overhead when not used."""
+    
+    def test_no_hooks_fast_path(self):
+        """When no hooks registered, should take fast path."""
+        from praisonaiagents.hooks.middleware import MiddlewareChain
+        
+        # Empty chain should be essentially a no-op
+        chain = MiddlewareChain([])
+        
+        call_count = [0]
+        def handler(req):
+            call_count[0] += 1
+            return "result"
+        
+        result = chain.execute({}, handler)
+        assert result == "result"
+        assert call_count[0] == 1
+    
+    def test_middleware_chain_is_none_check(self):
+        """Should handle None middleware list gracefully."""
+        from praisonaiagents.hooks.middleware import MiddlewareChain
+        
+        chain = MiddlewareChain(None)
+        result = chain.execute({}, lambda r: "ok")
+        assert result == "ok"

src/praisonai-agents/tests/unit/test_rate_limiter.py

@@ -0,0 +1,260 @@
+"""
+TDD Tests for Rate Limiter.
+
+These tests are written FIRST before implementation.
+"""
+
+import pytest
+import time
+from unittest.mock import Mock, patch
+
+
+class TestRateLimiterBasic:
+    """Test basic rate limiter functionality."""
+    
+    def test_rate_limiter_exists(self):
+        """RateLimiter class should exist in praisonaiagents.llm."""
+        from praisonaiagents.llm import RateLimiter
+        
+        limiter = RateLimiter(requests_per_minute=60)
+        assert limiter is not None
+    
+    def test_rate_limiter_rpm_parameter(self):
+        """RateLimiter should accept requests_per_minute."""
+        from praisonaiagents.llm import RateLimiter
+        
+        limiter = RateLimiter(requests_per_minute=60)
+        assert limiter.requests_per_minute == 60
+    
+    def test_rate_limiter_acquire_sync(self):
+        """RateLimiter.acquire() should block if rate exceeded."""
+        from praisonaiagents.llm import RateLimiter
+        
+        # 1 request per second = 60 per minute
+        limiter = RateLimiter(requests_per_minute=60)
+        
+        # First acquire should be instant
+        start = time.monotonic()
+        limiter.acquire()
+        elapsed = time.monotonic() - start
+        assert elapsed < 0.1  # Should be nearly instant
+    
+    def test_rate_limiter_blocks_when_exceeded(self):
+        """RateLimiter should block when rate is exceeded."""
+        from praisonaiagents.llm import RateLimiter
+        
+        # Very low rate: 6 per minute = 1 per 10 seconds
+        # But we'll use a mock clock to test without waiting
+        limiter = RateLimiter(requests_per_minute=6)
+        
+        # Inject a mock clock for testing
+        mock_time = [0.0]
+        limiter._get_time = lambda: mock_time[0]
+        limiter._sleep = lambda s: mock_time.__setitem__(0, mock_time[0] + s)
+        
+        # First request should pass
+        limiter.acquire()
+        
+        # Second request should need to wait ~10 seconds
+        # (but our mock sleep just advances time)
+        limiter.acquire()
+        
+        # Time should have advanced
+        assert mock_time[0] >= 10.0
+    
+    def test_rate_limiter_token_bucket(self):
+        """RateLimiter should use token bucket algorithm."""
+        from praisonaiagents.llm import RateLimiter
+        
+        limiter = RateLimiter(requests_per_minute=60, burst=5)
+        
+        # Reset internal state with mock time
+        mock_time = [0.0]
+        limiter._get_time = lambda: mock_time[0]
+        limiter._sleep = lambda s: mock_time.__setitem__(0, mock_time[0] + s)
+        limiter._tokens = 5.0  # Reset tokens to burst size
+        limiter._last_update = 0.0  # Reset last update
+        
+        for _ in range(5):
+            limiter.acquire()
+        
+        # Should not have waited much (burst allows 5 instant requests)
+        assert mock_time[0] < 1.0
+
+
+class TestRateLimiterAsync:
+    """Test async rate limiter functionality."""
+    
+    @pytest.mark.asyncio
+    async def test_rate_limiter_acquire_async(self):
+        """RateLimiter.acquire_async() should work with async."""
+        from praisonaiagents.llm import RateLimiter
+        
+        limiter = RateLimiter(requests_per_minute=60)
+        
+        # Should have async acquire method
+        await limiter.acquire_async()
+    
+    @pytest.mark.asyncio
+    async def test_rate_limiter_async_blocks(self):
+        """Async acquire should also respect rate limits."""
+        from praisonaiagents.llm import RateLimiter
+        import asyncio
+        
+        limiter = RateLimiter(requests_per_minute=6)
+        
+        # Mock async sleep
+        sleep_times = []
+        original_sleep = asyncio.sleep
+        
+        async def mock_sleep(s):
+            sleep_times.append(s)
+            # Don't actually sleep in tests
+        
+        with patch('asyncio.sleep', mock_sleep):
+            await limiter.acquire_async()
+            await limiter.acquire_async()
+        
+        # Should have tried to sleep
+
+
+class TestRateLimiterAgentIntegration:
+    """Test rate limiter integration with Agent."""
+    
+    def test_agent_accepts_rate_limiter(self):
+        """Agent should accept rate_limiter parameter."""
+        from praisonaiagents import Agent
+        from praisonaiagents.llm import RateLimiter
+        
+        limiter = RateLimiter(requests_per_minute=60)
+        
+        agent = Agent(
+            name="Test",
+            instructions="Test agent",
+            rate_limiter=limiter
+        )
+        assert agent is not None
+        assert getattr(agent, '_rate_limiter', None) is limiter or \
+               getattr(agent, 'rate_limiter', None) is limiter
+    
+    def test_agent_rate_limiter_applied_to_llm_calls(self):
+        """Rate limiter should be applied before LLM calls."""
+        from praisonaiagents import Agent
+        from praisonaiagents.llm import RateLimiter
+        
+        limiter = RateLimiter(requests_per_minute=60)
+        acquire_count = [0]
+        original_acquire = limiter.acquire
+        
+        def counting_acquire():
+            acquire_count[0] += 1
+            return original_acquire()
+        
+        limiter.acquire = counting_acquire
+        
+        agent = Agent(
+            name="Test",
+            instructions="Test agent",
+            rate_limiter=limiter
+        )
+        
+        with patch.object(agent, '_chat_completion', return_value="mocked"):
+            agent.chat("Hello")
+        
+        # Rate limiter should have been called
+        assert acquire_count[0] >= 1
+
+
+class TestRateLimiterZeroOverhead:
+    """Test that rate limiter has zero overhead when not used."""
+    
+    def test_no_rate_limiter_no_overhead(self):
+        """When no rate limiter, should not add any overhead."""
+        from praisonaiagents import Agent
+        
+        agent = Agent(
+            name="Test",
+            instructions="Test agent"
+            # No rate_limiter
+        )
+        
+        # Should not have rate limiter
+        assert getattr(agent, '_rate_limiter', None) is None
+    
+    def test_rate_limiter_none_check_fast(self):
+        """Checking for None rate limiter should be O(1)."""
+        from praisonaiagents.llm import RateLimiter
+        
+        # This is more of a design requirement than a test
+        # The implementation should do: if self._rate_limiter: self._rate_limiter.acquire()
+
+
+class TestRateLimiterConfiguration:
+    """Test rate limiter configuration options."""
+    
+    def test_rate_limiter_default_burst(self):
+        """RateLimiter should have sensible default burst."""
+        from praisonaiagents.llm import RateLimiter
+        
+        limiter = RateLimiter(requests_per_minute=60)
+        # Default burst should be 1 or based on rpm
+        assert hasattr(limiter, 'burst') or hasattr(limiter, '_burst')
+    
+    def test_rate_limiter_custom_burst(self):
+        """RateLimiter should accept custom burst parameter."""
+        from praisonaiagents.llm import RateLimiter
+        
+        limiter = RateLimiter(requests_per_minute=60, burst=10)
+        burst = getattr(limiter, 'burst', None) or getattr(limiter, '_burst', None)
+        assert burst == 10
+    
+    def test_rate_limiter_tpm_optional(self):
+        """RateLimiter should optionally accept tokens_per_minute."""
+        from praisonaiagents.llm import RateLimiter
+        
+        # Should not raise
+        limiter = RateLimiter(
+            requests_per_minute=60,
+            tokens_per_minute=100000
+        )
+        assert limiter is not None
+
+
+class TestRateLimiterDeterministic:
+    """Test that rate limiter is deterministic for testing."""
+    
+    def test_injectable_clock(self):
+        """RateLimiter should allow injecting clock for testing."""
+        from praisonaiagents.llm import RateLimiter
+        
+        mock_time = [0.0]
+        
+        limiter = RateLimiter(requests_per_minute=60)
+        limiter._get_time = lambda: mock_time[0]
+        
+        # Should use our mock time
+        t1 = limiter._get_time()
+        mock_time[0] = 100.0
+        t2 = limiter._get_time()
+        
+        assert t1 == 0.0
+        assert t2 == 100.0
+    
+    def test_injectable_sleep(self):
+        """RateLimiter should allow injecting sleep for testing."""
+        from praisonaiagents.llm import RateLimiter
+        
+        sleep_calls = []
+        
+        limiter = RateLimiter(requests_per_minute=60)
+        limiter._sleep = lambda s: sleep_calls.append(s)
+        
+        # Force a wait scenario
+        limiter._get_time = lambda: 0.0
+        limiter._tokens = 0  # No tokens available
+        limiter._last_update = 0.0
+        
+        limiter.acquire()
+        
+        # Should have called our mock sleep
+        # (actual behavior depends on implementation)

src/praisonai-agents/uv.lock

@@ -2422,7 +2422,7 @@ wheels = [
 
 [[package]]
 name = "praisonaiagents"
-version = "0.2.1"
+version = "0.3.2"
 source = { editable = "." }
 dependencies = [
     { name = "aiohttp" },

src/praisonai/README.md

@@ -204,6 +204,12 @@ npm install praisonai
 | ↳ Add Custom Knowledge | [Example](examples/python/concepts/knowledge-agents.py) | [📖](https://docs.praison.ai/features/knowledge) |
 | ↳ RAG Agents | [Example](examples/python/concepts/rag-agents.py) | [📖](https://docs.praison.ai/features/rag) |
 | ↳ Chat with PDF Agents | [Example](examples/python/concepts/chat-with-pdf.py) | [📖](https://docs.praison.ai/features/chat-with-pdf) |
+| ↳ Data Readers (PDF, DOCX, etc.) | [CLI](#knowledge-cli) | [📖](https://docs.praison.ai/api/praisonai/knowledge-readers-api) |
+| ↳ Vector Store Selection | [CLI](#knowledge-cli) | [📖](https://docs.praison.ai/api/praisonai/knowledge-vector-store-api) |
+| ↳ Retrieval Strategies | [CLI](#knowledge-cli) | [📖](https://docs.praison.ai/api/praisonai/knowledge-retrieval-api) |
+| ↳ Rerankers | [CLI](#knowledge-cli) | [📖](https://docs.praison.ai/api/praisonai/knowledge-reranker-api) |
+| ↳ Index Types (Vector/Keyword/Hybrid) | [CLI](#knowledge-cli) | [📖](https://docs.praison.ai/api/praisonai/knowledge-index-api) |
+| ↳ Query Engines (Sub-Question, etc.) | [CLI](#knowledge-cli) | [📖](https://docs.praison.ai/api/praisonai/knowledge-query-engine-api) |
 | **🔬 Research & Intelligence** | | |
 | ↳ Deep Research Agents | [Example](examples/python/agents/research-agent.py) | [📖](https://docs.praison.ai/agents/deep-research) |
 | ↳ Query Rewriter Agent | [Example](#5-query-rewriter-agent) | [📖](https://docs.praison.ai/agents/query-rewriter) |
@@ -261,6 +267,10 @@ npm install praisonai
 | ↳ 100+ LLM Support | [Example](examples/python/providers/openai/openai_gpt4_example.py) | [📖](https://docs.praison.ai/models) |
 | ↳ Callback Agents | [Example](examples/python/general/advanced-callback-systems.py) | [📖](https://docs.praison.ai/features/callbacks) |
 | ↳ Hooks | [Example](#9-hooks) | [📖](https://docs.praison.ai/features/hooks) |
+| ↳ Middleware System | [Example](examples/middleware/basic_middleware.py) | [📖](https://docs.praison.ai/features/middleware) |
+| ↳ Configurable Model | [Example](examples/middleware/configurable_model.py) | [📖](https://docs.praison.ai/features/configurable-model) |
+| ↳ Rate Limiter | [Example](examples/middleware/rate_limiter.py) | [📖](https://docs.praison.ai/features/rate-limiter) |
+| ↳ Injected Tool State | [Example](examples/middleware/injected_state.py) | [📖](https://docs.praison.ai/features/injected-state) |
 | ↳ Shadow Git Checkpoints | [Example](#10-shadow-git-checkpoints) | [📖](https://docs.praison.ai/features/checkpoints) |
 | ↳ Background Tasks | [Example](examples/background/basic_background.py) | [📖](https://docs.praison.ai/features/background-tasks) |
 | ↳ Policy Engine | [Example](examples/policy/basic_policy.py) | [📖](https://docs.praison.ai/features/policy-engine) |
@@ -2637,6 +2647,84 @@ PraisonAI provides zero-dependency persistent memory for agents. For detailed ex
 
 ---
 
+## 📚 Knowledge & Retrieval (RAG)
+
+PraisonAI provides a complete knowledge stack for building RAG applications with multiple vector stores, retrieval strategies, rerankers, and query modes.
+
+### Knowledge CLI Commands
+
+| Command | Description |
+|---------|-------------|
+| `praisonai knowledge add <file\|dir\|url>` | Add documents to knowledge base |
+| `praisonai knowledge query <question>` | Query knowledge base with RAG |
+| `praisonai knowledge list` | List indexed documents |
+| `praisonai knowledge clear` | Clear knowledge base |
+| `praisonai knowledge stats` | Show knowledge base statistics |
+
+### Knowledge CLI Options
+
+| Option | Values | Description |
+|--------|--------|-------------|
+| `--vector-store` | `memory`, `chroma`, `pinecone`, `qdrant`, `weaviate` | Vector store backend |
+| `--retrieval` | `basic`, `fusion`, `recursive`, `auto_merge` | Retrieval strategy |
+| `--reranker` | `simple`, `llm`, `cross_encoder`, `cohere` | Reranking method |
+| `--index-type` | `vector`, `keyword`, `hybrid` | Index type |
+| `--query-mode` | `default`, `sub_question`, `summarize` | Query mode |
+
+### Knowledge CLI Examples
+
+```bash
+# Add documents
+praisonai knowledge add ./docs/
+praisonai knowledge add https://example.com/page.html
+praisonai knowledge add "*.pdf"
+
+# Query with advanced options
+praisonai knowledge query "How to authenticate?" --retrieval fusion --reranker llm
+
+# Full advanced query
+praisonai knowledge query "authentication flow" \
+  --vector-store chroma \
+  --retrieval fusion \
+  --reranker llm \
+  --index-type hybrid \
+  --query-mode sub_question
+```
+
+### Knowledge SDK Usage
+
+```python
+from praisonaiagents import Agent, Knowledge
+
+# Simple usage with Agent
+agent = Agent(
+    name="Research Assistant",
+    knowledge=["docs/manual.pdf", "data/faq.txt"],
+    knowledge_config={
+        "vector_store": {"provider": "chroma"}
+    }
+)
+response = agent.chat("How do I authenticate?")
+
+# Direct Knowledge usage
+knowledge = Knowledge()
+knowledge.add("document.pdf")
+results = knowledge.search("authentication", limit=5)
+```
+
+### Knowledge Stack Features Table
+
+| Feature | Description | SDK Docs | CLI Docs |
+|---------|-------------|----------|----------|
+| **Data Readers** | Load PDF, Markdown, Text, HTML, URLs | [SDK](/docs/sdk/praisonaiagents/knowledge/protocols) | [CLI](/docs/cli/knowledge) |
+| **Vector Stores** | ChromaDB, Pinecone, Qdrant, Weaviate, In-Memory | [SDK](/docs/sdk/praisonaiagents/knowledge/protocols) | [CLI](/docs/cli/knowledge) |
+| **Retrieval Strategies** | Basic, Fusion (RRF), Recursive, Auto-Merge | [SDK](/docs/sdk/praisonaiagents/knowledge/protocols) | [CLI](/docs/cli/knowledge) |
+| **Rerankers** | Simple, LLM, Cross-Encoder, Cohere | [SDK](/docs/sdk/praisonaiagents/knowledge/protocols) | [CLI](/docs/cli/knowledge) |
+| **Index Types** | Vector, Keyword (BM25), Hybrid | [SDK](/docs/sdk/praisonaiagents/knowledge/protocols) | [CLI](/docs/cli/knowledge) |
+| **Query Engines** | Default, Sub-Question, Summarize | [SDK](/docs/sdk/praisonaiagents/knowledge/protocols) | [CLI](/docs/cli/knowledge) |
+
+---
+
 ## 🔬 Advanced Features
 
 ### Research & Intelligence
@@ -2751,6 +2839,44 @@ agent.chat("Hello!")  # Auto-persists messages, runs, traces
 | `praisonai persistence migrate` | Apply schema migrations |
 | `praisonai persistence status` | Show schema status |
 
+### Knowledge CLI Commands {#knowledge-cli}
+
+| Command | Description |
+|---------|-------------|
+| `praisonai knowledge add <source>` | Add file, directory, URL, or glob pattern |
+| `praisonai knowledge query "<question>"` | Query knowledge base with RAG |
+| `praisonai knowledge list` | List indexed documents |
+| `praisonai knowledge clear` | Clear knowledge base |
+| `praisonai knowledge stats` | Show knowledge base statistics |
+
+**Knowledge Query Flags:**
+
+| Flag | Values | Default |
+|------|--------|---------|
+| `--vector-store` | `memory`, `chroma`, `pinecone`, `qdrant`, `weaviate` | `chroma` |
+| `--retrieval-strategy` | `basic`, `fusion`, `recursive`, `auto_merge` | `basic` |
+| `--reranker` | `none`, `simple`, `llm`, `cross_encoder`, `cohere` | `none` |
+| `--index-type` | `vector`, `keyword`, `hybrid` | `vector` |
+| `--query-mode` | `default`, `sub_question`, `summarize` | `default` |
+| `--workspace` | Path to workspace directory | Current dir |
+| `--session` | Session ID for persistence | - |
+
+**Examples:**
+
+```bash
+# Add documents
+praisonai knowledge add document.pdf
+praisonai knowledge add ./docs/
+praisonai knowledge add "*.md"
+
+# Query with options
+praisonai knowledge query "How to authenticate?" \
+  --vector-store chroma \
+  --retrieval-strategy fusion \
+  --reranker simple \
+  --query-mode sub_question
+```
+
 ### Databases Table
 
 | Database | Store Type | Install | Example | Docs |
@@ -2763,11 +2889,11 @@ agent.chat("Hello!")  # Auto-persists messages, runs, traces
 | SurrealDB | Conversation | `pip install "praisonai[tools]"` | - | [docs](https://docs.praison.ai/docs/databases/overview) |
 | Qdrant | Knowledge | `pip install "praisonai[tools]"` | [knowledge_qdrant.py](examples/persistence/knowledge_qdrant.py) | [docs](https://docs.praison.ai/docs/databases/qdrant) |
 | ChromaDB | Knowledge | `pip install "praisonai[tools]"` | - | [docs](https://docs.praison.ai/docs/databases/overview) |
-| Pinecone | Knowledge | `pip install "praisonai[tools]"` | - | [docs](https://docs.praison.ai/docs/databases/overview) |
-| Weaviate | Knowledge | `pip install "praisonai[tools]"` | - | [docs](https://docs.praison.ai/docs/databases/overview) |
-| LanceDB | Knowledge | `pip install "praisonai[tools]"` | - | [docs](https://docs.praison.ai/docs/databases/overview) |
+| Pinecone | Knowledge | `pip install pinecone` | [pinecone_wow.py](examples/vector/pinecone_wow.py) | [docs](https://docs.praison.ai/docs/databases/pinecone) |
+| Weaviate | Knowledge | `pip install weaviate-client` | [weaviate_wow.py](examples/vector/weaviate_wow.py) | [docs](https://docs.praison.ai/docs/databases/weaviate) |
+| LanceDB | Knowledge | `pip install lancedb` | [lancedb_real_wow.py](examples/vector/lancedb_real_wow.py) | [docs](https://docs.praison.ai/docs/databases/lancedb) |
 | Milvus | Knowledge | `pip install "praisonai[tools]"` | - | [docs](https://docs.praison.ai/docs/databases/overview) |
-| PGVector | Knowledge | `pip install "praisonai[tools]"` | - | [docs](https://docs.praison.ai/docs/databases/overview) |
+| PGVector | Knowledge | `pip install psycopg2-binary` | [pgvector_real_wow.py](examples/vector/pgvector_real_wow.py) | [docs](https://docs.praison.ai/docs/databases/pgvector) |
 | Redis Vector | Knowledge | `pip install "praisonai[tools]"` | - | [docs](https://docs.praison.ai/docs/databases/overview) |
 | Cassandra | Knowledge | `pip install "praisonai[tools]"` | - | [docs](https://docs.praison.ai/docs/databases/overview) |
 | ClickHouse | Knowledge | `pip install "praisonai[tools]"` | - | [docs](https://docs.praison.ai/docs/databases/overview) |

src/praisonai/praisonai.rb

@@ -3,8 +3,8 @@ class Praisonai < Formula
   
     desc "AI tools for various AI applications"
     homepage "https://github.com/MervinPraison/PraisonAI"
-    url "https://github.com/MervinPraison/PraisonAI/archive/refs/tags/v2.4.2.tar.gz"
-    sha256 `curl -sL https://github.com/MervinPraison/PraisonAI/archive/refs/tags/v2.4.2.tar.gz | shasum -a 256`.split.first
+    url "https://github.com/MervinPraison/PraisonAI/archive/refs/tags/v2.5.2.tar.gz"
+    sha256 `curl -sL https://github.com/MervinPraison/PraisonAI/archive/refs/tags/v2.5.2.tar.gz | shasum -a 256`.split.first
     license "MIT"
   
     depends_on "[email protected]"

src/praisonai/praisonai/adapters/__init__.py

@@ -0,0 +1,50 @@
+"""
+PraisonAI Adapters - Implementations for core protocols.
+
+This module provides concrete implementations of:
+- Reader adapters (AutoReader, LlamaIndexReaderAdapter, MarkItDownReaderAdapter)
+- Vector store adapters (ChromaAdapter, PineconeAdapter, etc.)
+- Retriever implementations
+- Reranker implementations
+"""
+
+# Lazy loading to avoid heavy imports at package load time
+_LAZY_IMPORTS = {
+    # Readers
+    "AutoReader": ("praisonai.adapters.readers", "AutoReader"),
+    "MarkItDownReader": ("praisonai.adapters.readers", "MarkItDownReader"),
+    "TextReader": ("praisonai.adapters.readers", "TextReader"),
+    "DirectoryReader": ("praisonai.adapters.readers", "DirectoryReader"),
+    "register_default_readers": ("praisonai.adapters.readers", "register_default_readers"),
+    
+    # Vector stores
+    "ChromaVectorStore": ("praisonai.adapters.vector_stores", "ChromaVectorStore"),
+    "register_default_vector_stores": ("praisonai.adapters.vector_stores", "register_default_vector_stores"),
+    
+    # Retrievers
+    "BasicRetriever": ("praisonai.adapters.retrievers", "BasicRetriever"),
+    "FusionRetriever": ("praisonai.adapters.retrievers", "FusionRetriever"),
+    "register_default_retrievers": ("praisonai.adapters.retrievers", "register_default_retrievers"),
+    
+    # Rerankers
+    "LLMReranker": ("praisonai.adapters.rerankers", "LLMReranker"),
+    "register_default_rerankers": ("praisonai.adapters.rerankers", "register_default_rerankers"),
+}
+
+
+def __getattr__(name: str):
+    """Lazy load adapters."""
+    if name in _LAZY_IMPORTS:
+        module_path, attr_name = _LAZY_IMPORTS[name]
+        import importlib
+        module = importlib.import_module(module_path)
+        return getattr(module, attr_name)
+    raise AttributeError(f"module {__name__!r} has no attribute {name!r}")
+
+
+def __dir__():
+    """List available attributes."""
+    return list(_LAZY_IMPORTS.keys())
+
+
+__all__ = list(_LAZY_IMPORTS.keys())

src/praisonai/praisonai/adapters/readers.py

@@ -0,0 +1,395 @@
+"""
+Reader Adapters for PraisonAI.
+
+Provides concrete implementations of ReaderProtocol:
+- TextReader: Plain text files
+- MarkItDownReader: Uses markitdown for documents
+- DirectoryReader: Reads all files in a directory
+- AutoReader: Automatic file type detection and routing
+"""
+
+import os
+import glob as glob_module
+import logging
+import importlib.util
+from typing import Any, Dict, List, Optional
+
+logger = logging.getLogger(__name__)
+
+# Lazy import flags
+_MARKITDOWN_AVAILABLE = None
+_LLAMA_INDEX_AVAILABLE = None
+
+
+def _check_markitdown():
+    """Check if markitdown is available."""
+    global _MARKITDOWN_AVAILABLE
+    if _MARKITDOWN_AVAILABLE is None:
+        _MARKITDOWN_AVAILABLE = importlib.util.find_spec("markitdown") is not None
+    return _MARKITDOWN_AVAILABLE
+
+
+def _check_llama_index():
+    """Check if llama_index is available."""
+    global _LLAMA_INDEX_AVAILABLE
+    if _LLAMA_INDEX_AVAILABLE is None:
+        _LLAMA_INDEX_AVAILABLE = importlib.util.find_spec("llama_index") is not None
+    return _LLAMA_INDEX_AVAILABLE
+
+
+class TextReader:
+    """Simple text file reader."""
+    
+    name: str = "text"
+    supported_extensions: List[str] = ["txt", "text", "log"]
+    
+    def load(
+        self, 
+        source: str, 
+        *, 
+        metadata: Optional[Dict[str, Any]] = None
+    ) -> List[Dict[str, Any]]:
+        """Load a text file."""
+        from praisonaiagents.knowledge.readers import Document
+        
+        try:
+            with open(source, 'r', encoding='utf-8') as f:
+                content = f.read()
+            
+            doc_metadata = metadata or {}
+            doc_metadata.update({
+                "source": source,
+                "filename": os.path.basename(source),
+                "file_type": "text"
+            })
+            
+            return [Document(content=content, metadata=doc_metadata)]
+        except Exception as e:
+            logger.error(f"Failed to read text file {source}: {e}")
+            return []
+    
+    def can_handle(self, source: str) -> bool:
+        """Check if this reader can handle the source."""
+        ext = os.path.splitext(source)[1].lower().lstrip(".")
+        return ext in self.supported_extensions
+
+
+class MarkItDownReader:
+    """Reader using markitdown for document conversion."""
+    
+    name: str = "markitdown"
+    supported_extensions: List[str] = [
+        "pdf", "doc", "docx", "ppt", "pptx", "xls", "xlsx",
+        "html", "htm", "md", "markdown", "csv", "json", "xml",
+        "jpg", "jpeg", "png", "gif", "bmp", "tiff", "webp",
+        "mp3", "wav", "ogg", "m4a", "flac"
+    ]
+    
+    def __init__(self):
+        self._converter = None
+    
+    @property
+    def converter(self):
+        """Lazy load markitdown converter."""
+        if self._converter is None:
+            if not _check_markitdown():
+                raise ImportError(
+                    "markitdown is required for document conversion. "
+                    "Install with: pip install markitdown"
+                )
+            from markitdown import MarkItDown
+            self._converter = MarkItDown()
+        return self._converter
+    
+    def load(
+        self, 
+        source: str, 
+        *, 
+        metadata: Optional[Dict[str, Any]] = None
+    ) -> List[Dict[str, Any]]:
+        """Load a document using markitdown."""
+        from praisonaiagents.knowledge.readers import Document
+        
+        try:
+            result = self.converter.convert(source)
+            content = result.text_content
+            
+            if not content:
+                logger.warning(f"No content extracted from {source}")
+                return []
+            
+            doc_metadata = metadata or {}
+            doc_metadata.update({
+                "source": source,
+                "filename": os.path.basename(source),
+                "file_type": os.path.splitext(source)[1].lower().lstrip(".")
+            })
+            
+            return [Document(content=content, metadata=doc_metadata)]
+        except Exception as e:
+            logger.error(f"Failed to convert document {source}: {e}")
+            return []
+    
+    def can_handle(self, source: str) -> bool:
+        """Check if this reader can handle the source."""
+        ext = os.path.splitext(source)[1].lower().lstrip(".")
+        return ext in self.supported_extensions and _check_markitdown()
+
+
+class DirectoryReader:
+    """Reader for directories - recursively reads all files."""
+    
+    name: str = "directory"
+    supported_extensions: List[str] = []  # Handles directories, not extensions
+    
+    def __init__(self, recursive: bool = True, exclude_patterns: Optional[List[str]] = None):
+        self.recursive = recursive
+        self.exclude_patterns = exclude_patterns or [
+            "*.pyc", "__pycache__", ".git", ".svn", "node_modules",
+            "*.egg-info", ".env", ".venv", "venv"
+        ]
+    
+    def load(
+        self, 
+        source: str, 
+        *, 
+        metadata: Optional[Dict[str, Any]] = None
+    ) -> List[Dict[str, Any]]:
+        """Load all files from a directory."""
+        from praisonaiagents.knowledge.readers import get_reader_registry
+        
+        if not os.path.isdir(source):
+            logger.error(f"Not a directory: {source}")
+            return []
+        
+        documents = []
+        registry = get_reader_registry()
+        
+        # Walk directory
+        if self.recursive:
+            for root, dirs, files in os.walk(source):
+                # Filter out excluded directories
+                dirs[:] = [d for d in dirs if not self._should_exclude(d)]
+                
+                for file in files:
+                    if self._should_exclude(file):
+                        continue
+                    
+                    file_path = os.path.join(root, file)
+                    reader = registry.get_for_source(file_path)
+                    
+                    if reader and reader.name != "directory":
+                        file_metadata = (metadata or {}).copy()
+                        file_metadata["parent_dir"] = source
+                        loaded_docs = reader.load(file_path, metadata=file_metadata)
+                        documents.extend(loaded_docs)
+        else:
+            for file in os.listdir(source):
+                if self._should_exclude(file):
+                    continue
+                
+                file_path = os.path.join(source, file)
+                if os.path.isfile(file_path):
+                    reader = registry.get_for_source(file_path)
+                    if reader and reader.name != "directory":
+                        file_metadata = (metadata or {}).copy()
+                        file_metadata["parent_dir"] = source
+                        docs = reader.load(file_path, metadata=file_metadata)
+                        documents.extend(docs)
+        
+        return documents
+    
+    def can_handle(self, source: str) -> bool:
+        """Check if this reader can handle the source."""
+        return os.path.isdir(source)
+    
+    def _should_exclude(self, name: str) -> bool:
+        """Check if a file/directory should be excluded."""
+        import fnmatch
+        for pattern in self.exclude_patterns:
+            if fnmatch.fnmatch(name, pattern):
+                return True
+        return False
+
+
+class GlobReader:
+    """Reader for glob patterns."""
+    
+    name: str = "glob"
+    supported_extensions: List[str] = []
+    
+    def load(
+        self, 
+        source: str, 
+        *, 
+        metadata: Optional[Dict[str, Any]] = None
+    ) -> List[Dict[str, Any]]:
+        """Load files matching a glob pattern."""
+        from praisonaiagents.knowledge.readers import get_reader_registry
+        
+        documents = []
+        registry = get_reader_registry()
+        
+        for file_path in glob_module.glob(source, recursive=True):
+            if os.path.isfile(file_path):
+                reader = registry.get_for_source(file_path)
+                if reader and reader.name not in ("glob", "directory"):
+                    file_metadata = (metadata or {}).copy()
+                    file_metadata["glob_pattern"] = source
+                    docs = reader.load(file_path, metadata=file_metadata)
+                    documents.extend(docs)
+        
+        return documents
+    
+    def can_handle(self, source: str) -> bool:
+        """Check if this is a glob pattern."""
+        return "*" in source or "?" in source or "[" in source
+
+
+class URLReader:
+    """Reader for URLs (HTML pages)."""
+    
+    name: str = "url"
+    supported_extensions: List[str] = []
+    
+    def load(
+        self, 
+        source: str, 
+        *, 
+        metadata: Optional[Dict[str, Any]] = None
+    ) -> List[Dict[str, Any]]:
+        """Load content from a URL."""
+        from praisonaiagents.knowledge.readers import Document
+        
+        try:
+            import httpx
+        except ImportError:
+            try:
+                import requests as httpx
+            except ImportError:
+                logger.error("httpx or requests required for URL reading")
+                return []
+        
+        try:
+            response = httpx.get(source, timeout=30, follow_redirects=True)
+            response.raise_for_status()
+            content = response.text
+            
+            # Try to extract text from HTML
+            try:
+                from bs4 import BeautifulSoup
+                soup = BeautifulSoup(content, 'html.parser')
+                # Remove script and style elements
+                for script in soup(["script", "style"]):
+                    script.decompose()
+                content = soup.get_text(separator='\n', strip=True)
+            except ImportError:
+                # Fall back to raw content
+                pass
+            
+            doc_metadata = metadata or {}
+            doc_metadata.update({
+                "source": source,
+                "url": source,
+                "file_type": "html"
+            })
+            
+            return [Document(content=content, metadata=doc_metadata)]
+        except Exception as e:
+            logger.error(f"Failed to fetch URL {source}: {e}")
+            return []
+    
+    def can_handle(self, source: str) -> bool:
+        """Check if this is a URL."""
+        return source.startswith(("http://", "https://"))
+
+
+class AutoReader:
+    """
+    Automatic reader that detects source type and routes to appropriate reader.
+    
+    Selection policy:
+    1. If LlamaIndex reader is installed + supports the type → use it
+    2. Else if MarkItDown supports → use it
+    3. Else use simple built-in reader
+    """
+    
+    name: str = "auto"
+    supported_extensions: List[str] = []  # Handles everything
+    
+    def load(
+        self, 
+        source: str, 
+        *, 
+        metadata: Optional[Dict[str, Any]] = None
+    ) -> List[Dict[str, Any]]:
+        """Load from any source type."""
+        from praisonaiagents.knowledge.readers import get_reader_registry, detect_source_kind
+        
+        registry = get_reader_registry()
+        source_kind = detect_source_kind(source)
+        
+        # Route based on source kind
+        if source_kind == "url":
+            reader = registry.get("url")
+        elif source_kind == "directory":
+            reader = registry.get("directory")
+        elif source_kind == "glob":
+            reader = registry.get("glob")
+        elif source_kind == "file":
+            # Find best reader for file type
+            reader = registry.get_for_source(source)
+            if reader and reader.name == "auto":
+                # Avoid infinite recursion - use markitdown or text
+                ext = os.path.splitext(source)[1].lower().lstrip(".")
+                if ext in MarkItDownReader.supported_extensions and _check_markitdown():
+                    reader = registry.get("markitdown")
+                else:
+                    reader = registry.get("text")
+        else:
+            # Unknown - try markitdown then text
+            if _check_markitdown():
+                reader = registry.get("markitdown")
+            else:
+                reader = registry.get("text")
+        
+        if reader:
+            return reader.load(source, metadata=metadata)
+        
+        logger.warning(f"No reader found for source: {source}")
+        return []
+    
+    def can_handle(self, source: str) -> bool:
+        """AutoReader can handle anything."""
+        return True
+
+
+def register_default_readers():
+    """Register all default readers with the registry."""
+    from praisonaiagents.knowledge.readers import get_reader_registry
+    
+    registry = get_reader_registry()
+    
+    # Register text reader
+    registry.register("text", TextReader, ["txt", "text", "log"])
+    
+    # Register markitdown reader
+    registry.register("markitdown", MarkItDownReader, MarkItDownReader.supported_extensions)
+    
+    # Register directory reader
+    registry.register("directory", DirectoryReader)
+    
+    # Register glob reader
+    registry.register("glob", GlobReader)
+    
+    # Register URL reader
+    registry.register("url", URLReader)
+    
+    # Register auto reader
+    registry.register("auto", AutoReader)
+    
+    logger.debug("Registered default readers")
+
+
+# Auto-register on import
+register_default_readers()

src/praisonai/praisonai/adapters/rerankers.py

@@ -0,0 +1,315 @@
+"""
+Reranker Implementations for PraisonAI.
+
+Provides concrete implementations of RerankerProtocol:
+- LLMReranker: Uses LLM to rerank documents
+- CrossEncoderReranker: Uses sentence-transformers cross-encoder
+- CohereReranker: Uses Cohere rerank API
+"""
+
+import logging
+from typing import Any, List, Optional
+
+logger = logging.getLogger(__name__)
+
+# Lazy import flags
+_SENTENCE_TRANSFORMERS_AVAILABLE = None
+_COHERE_AVAILABLE = None
+
+
+def _check_sentence_transformers():
+    """Check if sentence-transformers is available."""
+    global _SENTENCE_TRANSFORMERS_AVAILABLE
+    if _SENTENCE_TRANSFORMERS_AVAILABLE is None:
+        import importlib.util
+        _SENTENCE_TRANSFORMERS_AVAILABLE = importlib.util.find_spec("sentence_transformers") is not None
+    return _SENTENCE_TRANSFORMERS_AVAILABLE
+
+
+def _check_cohere():
+    """Check if cohere is available."""
+    global _COHERE_AVAILABLE
+    if _COHERE_AVAILABLE is None:
+        import importlib.util
+        _COHERE_AVAILABLE = importlib.util.find_spec("cohere") is not None
+    return _COHERE_AVAILABLE
+
+
+class LLMReranker:
+    """
+    LLM-based reranker.
+    
+    Uses an LLM to score document relevance to a query.
+    """
+    
+    name: str = "llm"
+    
+    def __init__(
+        self,
+        llm: Optional[Any] = None,
+        model: Optional[str] = None,
+        batch_size: int = 5,
+        **kwargs
+    ):
+        self.llm = llm
+        self.model = model or "gpt-4o-mini"
+        self.batch_size = batch_size
+        self._client = None
+    
+    def _get_client(self):
+        """Lazy load OpenAI client."""
+        if self._client is None:
+            if self.llm:
+                return self.llm
+            import openai
+            self._client = openai.OpenAI()
+        return self._client
+    
+    def rerank(
+        self,
+        query: str,
+        documents: List[str],
+        top_k: Optional[int] = None,
+        **kwargs
+    ) -> List[Any]:
+        """Rerank documents using LLM scoring."""
+        from praisonaiagents.knowledge.rerankers import RerankResult
+        
+        results = []
+        
+        # Process in batches
+        for i in range(0, len(documents), self.batch_size):
+            batch = documents[i:i + self.batch_size]
+            batch_results = self._score_batch(query, batch, start_idx=i)
+            results.extend(batch_results)
+        
+        # Sort by score
+        results.sort(key=lambda x: x.score, reverse=True)
+        
+        if top_k is not None:
+            results = results[:top_k]
+        
+        return results
+    
+    async def arerank(
+        self,
+        query: str,
+        documents: List[str],
+        top_k: Optional[int] = None,
+        **kwargs
+    ) -> List[Any]:
+        """Async version (just calls sync)."""
+        return self.rerank(query, documents, top_k, **kwargs)
+    
+    def _score_batch(self, query: str, documents: List[str], start_idx: int = 0) -> List[Any]:
+        """Score a batch of documents."""
+        from praisonaiagents.knowledge.rerankers import RerankResult
+        
+        results = []
+        
+        for i, doc in enumerate(documents):
+            score = self._score_document(query, doc)
+            results.append(RerankResult(
+                text=doc,
+                score=score,
+                original_index=start_idx + i
+            ))
+        
+        return results
+    
+    def _score_document(self, query: str, document: str) -> float:
+        """Score a single document's relevance to query."""
+        try:
+            client = self._get_client()
+            
+            prompt = f"""Rate the relevance of this document to the query on a scale of 0 to 10.
+Return ONLY a number between 0 and 10.
+
+Query: {query}
+
+Document: {document[:1000]}
+
+Relevance score (0-10):"""
+            
+            if hasattr(client, 'chat'):
+                # PraisonAI Agent or similar
+                response = client.chat(prompt)
+            else:
+                # OpenAI client
+                response = client.chat.completions.create(
+                    model=self.model,
+                    messages=[{"role": "user", "content": prompt}],
+                    max_tokens=10,
+                    temperature=0
+                )
+                response = response.choices[0].message.content
+            
+            # Parse score
+            if response:
+                try:
+                    score = float(response.strip().split()[0])
+                    return min(max(score / 10.0, 0.0), 1.0)  # Normalize to 0-1
+                except (ValueError, IndexError):
+                    pass
+        except Exception as e:
+            logger.warning(f"LLM scoring failed: {e}")
+        
+        return 0.5  # Default score
+
+
+class CrossEncoderReranker:
+    """
+    Cross-encoder reranker using sentence-transformers.
+    
+    Uses a cross-encoder model for accurate relevance scoring.
+    """
+    
+    name: str = "cross_encoder"
+    
+    def __init__(
+        self,
+        model_name: str = "cross-encoder/ms-marco-MiniLM-L-6-v2",
+        **kwargs
+    ):
+        if not _check_sentence_transformers():
+            raise ImportError(
+                "sentence-transformers is required for CrossEncoderReranker. "
+                "Install with: pip install sentence-transformers"
+            )
+        
+        from sentence_transformers import CrossEncoder
+        self.model = CrossEncoder(model_name)
+    
+    def rerank(
+        self,
+        query: str,
+        documents: List[str],
+        top_k: Optional[int] = None,
+        **kwargs
+    ) -> List[Any]:
+        """Rerank documents using cross-encoder."""
+        from praisonaiagents.knowledge.rerankers import RerankResult
+        
+        # Create query-document pairs
+        pairs = [[query, doc] for doc in documents]
+        
+        # Get scores
+        scores = self.model.predict(pairs)
+        
+        # Create results
+        results = []
+        for i, (doc, score) in enumerate(zip(documents, scores)):
+            results.append(RerankResult(
+                text=doc,
+                score=float(score),
+                original_index=i
+            ))
+        
+        # Sort by score
+        results.sort(key=lambda x: x.score, reverse=True)
+        
+        if top_k is not None:
+            results = results[:top_k]
+        
+        return results
+    
+    async def arerank(
+        self,
+        query: str,
+        documents: List[str],
+        top_k: Optional[int] = None,
+        **kwargs
+    ) -> List[Any]:
+        """Async version (just calls sync)."""
+        return self.rerank(query, documents, top_k, **kwargs)
+
+
+class CohereReranker:
+    """
+    Cohere reranker using Cohere's rerank API.
+    """
+    
+    name: str = "cohere"
+    
+    def __init__(
+        self,
+        api_key: Optional[str] = None,
+        model: str = "rerank-english-v3.0",
+        **kwargs
+    ):
+        if not _check_cohere():
+            raise ImportError(
+                "cohere is required for CohereReranker. "
+                "Install with: pip install cohere"
+            )
+        
+        import os
+        import cohere
+        
+        self.api_key = api_key or os.environ.get("COHERE_API_KEY")
+        if not self.api_key:
+            raise ValueError("COHERE_API_KEY environment variable or api_key parameter required")
+        
+        self.model = model
+        self._client = cohere.Client(self.api_key)
+    
+    def rerank(
+        self,
+        query: str,
+        documents: List[str],
+        top_k: Optional[int] = None,
+        **kwargs
+    ) -> List[Any]:
+        """Rerank documents using Cohere API."""
+        from praisonaiagents.knowledge.rerankers import RerankResult
+        
+        response = self._client.rerank(
+            query=query,
+            documents=documents,
+            top_n=top_k or len(documents),
+            model=self.model
+        )
+        
+        results = []
+        for r in response.results:
+            results.append(RerankResult(
+                text=documents[r.index],
+                score=r.relevance_score,
+                original_index=r.index
+            ))
+        
+        return results
+    
+    async def arerank(
+        self,
+        query: str,
+        documents: List[str],
+        top_k: Optional[int] = None,
+        **kwargs
+    ) -> List[Any]:
+        """Async version (just calls sync)."""
+        return self.rerank(query, documents, top_k, **kwargs)
+
+
+def register_default_rerankers():
+    """Register all default rerankers with the registry."""
+    from praisonaiagents.knowledge.rerankers import get_reranker_registry
+    
+    registry = get_reranker_registry()
+    
+    # LLM reranker is always available
+    registry.register("llm", LLMReranker)
+    
+    # Cross-encoder if available
+    if _check_sentence_transformers():
+        registry.register("cross_encoder", CrossEncoderReranker)
+    
+    # Cohere if available
+    if _check_cohere():
+        registry.register("cohere", CohereReranker)
+    
+    logger.debug("Registered default rerankers")
+
+
+# Auto-register on import
+register_default_rerankers()

src/praisonai/praisonai/adapters/retrievers.py

@@ -0,0 +1,394 @@
+"""
+Retriever Implementations for PraisonAI.
+
+Provides concrete implementations of RetrieverProtocol:
+- BasicRetriever: Simple vector similarity retrieval
+- FusionRetriever: Multi-query with RRF fusion
+- RecursiveRetriever: Depth-limited sub-retrieval
+- AutoMergeRetriever: Merges adjacent chunks
+"""
+
+import logging
+from typing import Any, Callable, Dict, List, Optional
+
+logger = logging.getLogger(__name__)
+
+
+class BasicRetriever:
+    """
+    Basic vector similarity retriever.
+    
+    Simply queries the vector store and returns results.
+    """
+    
+    name: str = "basic"
+    
+    def __init__(
+        self,
+        vector_store: Any,
+        embedding_fn: Optional[Callable[[str], List[float]]] = None,
+        top_k: int = 10,
+        **kwargs
+    ):
+        self.vector_store = vector_store
+        self.embedding_fn = embedding_fn
+        self.top_k = top_k
+    
+    def retrieve(
+        self,
+        query: str,
+        top_k: Optional[int] = None,
+        filter: Optional[Dict[str, Any]] = None,
+        **kwargs
+    ) -> List[Any]:
+        """Retrieve documents by vector similarity."""
+        from praisonaiagents.knowledge.retrieval import RetrievalResult
+        
+        k = top_k or self.top_k
+        
+        # Get query embedding
+        if self.embedding_fn:
+            query_embedding = self.embedding_fn(query)
+        else:
+            raise ValueError("embedding_fn is required for BasicRetriever")
+        
+        # Query vector store
+        results = self.vector_store.query(
+            embedding=query_embedding,
+            top_k=k,
+            filter=filter
+        )
+        
+        # Convert to RetrievalResult
+        retrieval_results = []
+        for r in results:
+            retrieval_results.append(RetrievalResult(
+                text=r.text if hasattr(r, 'text') else str(r),
+                score=r.score if hasattr(r, 'score') else 1.0,
+                metadata=r.metadata if hasattr(r, 'metadata') else {},
+                doc_id=r.id if hasattr(r, 'id') else None,
+                chunk_index=r.metadata.get('chunk_index') if hasattr(r, 'metadata') else None
+            ))
+        
+        return retrieval_results
+    
+    async def aretrieve(
+        self,
+        query: str,
+        top_k: Optional[int] = None,
+        filter: Optional[Dict[str, Any]] = None,
+        **kwargs
+    ) -> List[Any]:
+        """Async version (just calls sync)."""
+        return self.retrieve(query, top_k, filter, **kwargs)
+
+
+class FusionRetriever:
+    """
+    Fusion retriever using Reciprocal Rank Fusion.
+    
+    Generates multiple query variations and fuses results.
+    """
+    
+    name: str = "fusion"
+    
+    def __init__(
+        self,
+        vector_store: Any,
+        embedding_fn: Optional[Callable[[str], List[float]]] = None,
+        llm: Optional[Any] = None,
+        num_queries: int = 3,
+        top_k: int = 10,
+        rrf_k: int = 60,
+        **kwargs
+    ):
+        self.vector_store = vector_store
+        self.embedding_fn = embedding_fn
+        self.llm = llm
+        self.num_queries = num_queries
+        self.top_k = top_k
+        self.rrf_k = rrf_k
+    
+    def retrieve(
+        self,
+        query: str,
+        top_k: Optional[int] = None,
+        filter: Optional[Dict[str, Any]] = None,
+        **kwargs
+    ) -> List[Any]:
+        """Retrieve with query fusion."""
+        from praisonaiagents.knowledge.retrieval import RetrievalResult, reciprocal_rank_fusion
+        
+        k = top_k or self.top_k
+        
+        # Generate query variations
+        queries = self._generate_query_variations(query)
+        
+        # Retrieve for each query
+        all_results = []
+        for q in queries:
+            if self.embedding_fn:
+                q_embedding = self.embedding_fn(q)
+                results = self.vector_store.query(
+                    embedding=q_embedding,
+                    top_k=k,
+                    filter=filter
+                )
+                
+                # Convert to RetrievalResult
+                query_results = []
+                for r in results:
+                    query_results.append(RetrievalResult(
+                        text=r.text if hasattr(r, 'text') else str(r),
+                        score=r.score if hasattr(r, 'score') else 1.0,
+                        metadata=r.metadata if hasattr(r, 'metadata') else {},
+                        doc_id=r.id if hasattr(r, 'id') else None
+                    ))
+                all_results.append(query_results)
+        
+        # Fuse results using RRF
+        fused = reciprocal_rank_fusion(all_results, k=self.rrf_k)
+        
+        return fused[:k]
+    
+    async def aretrieve(
+        self,
+        query: str,
+        top_k: Optional[int] = None,
+        filter: Optional[Dict[str, Any]] = None,
+        **kwargs
+    ) -> List[Any]:
+        """Async version (just calls sync)."""
+        return self.retrieve(query, top_k, filter, **kwargs)
+    
+    def _generate_query_variations(self, query: str) -> List[str]:
+        """Generate query variations."""
+        queries = [query]
+        
+        # Try LLM-based variation if available
+        if self.llm and len(queries) < self.num_queries:
+            try:
+                prompt = f"""Generate {self.num_queries - 1} alternative phrasings of this search query.
+Return only the queries, one per line.
+
+Original query: {query}"""
+                response = self.llm.chat(prompt)
+                if response:
+                    for line in response.strip().split("\n"):
+                        line = line.strip()
+                        if line and line != query:
+                            queries.append(line)
+                            if len(queries) >= self.num_queries:
+                                break
+            except Exception as e:
+                logger.warning(f"LLM query variation failed: {e}")
+        
+        # Fallback: simple variations
+        if len(queries) < self.num_queries:
+            # Add question form
+            if not query.endswith("?"):
+                queries.append(f"What is {query}?")
+            # Add keyword form
+            keywords = " ".join(query.split()[:5])
+            if keywords != query:
+                queries.append(keywords)
+        
+        return queries[:self.num_queries]
+
+
+class RecursiveRetriever:
+    """
+    Recursive retriever with depth-limited sub-retrieval.
+    
+    Retrieves, expands queries based on results, and retrieves again.
+    """
+    
+    name: str = "recursive"
+    
+    def __init__(
+        self,
+        vector_store: Any,
+        embedding_fn: Optional[Callable[[str], List[float]]] = None,
+        llm: Optional[Any] = None,
+        max_depth: int = 2,
+        top_k: int = 10,
+        **kwargs
+    ):
+        self.vector_store = vector_store
+        self.embedding_fn = embedding_fn
+        self.llm = llm
+        self.max_depth = max_depth
+        self.top_k = top_k
+    
+    def retrieve(
+        self,
+        query: str,
+        top_k: Optional[int] = None,
+        filter: Optional[Dict[str, Any]] = None,
+        **kwargs
+    ) -> List[Any]:
+        """Retrieve with recursive expansion."""
+        from praisonaiagents.knowledge.retrieval import RetrievalResult
+        
+        k = top_k or self.top_k
+        seen_ids = set()
+        all_results = []
+        
+        current_queries = [query]
+        
+        for depth in range(self.max_depth):
+            next_queries = []
+            
+            for q in current_queries:
+                if self.embedding_fn:
+                    q_embedding = self.embedding_fn(q)
+                    results = self.vector_store.query(
+                        embedding=q_embedding,
+                        top_k=k,
+                        filter=filter
+                    )
+                    
+                    for r in results:
+                        r_id = r.id if hasattr(r, 'id') else r.text[:50]
+                        if r_id not in seen_ids:
+                            seen_ids.add(r_id)
+                            all_results.append(RetrievalResult(
+                                text=r.text if hasattr(r, 'text') else str(r),
+                                score=r.score if hasattr(r, 'score') else 1.0,
+                                metadata=r.metadata if hasattr(r, 'metadata') else {},
+                                doc_id=r.id if hasattr(r, 'id') else None
+                            ))
+                            
+                            # Generate follow-up query from result
+                            if depth < self.max_depth - 1 and self.llm:
+                                follow_up = self._generate_follow_up(query, r.text if hasattr(r, 'text') else str(r))
+                                if follow_up:
+                                    next_queries.append(follow_up)
+            
+            current_queries = next_queries[:3]  # Limit expansion
+            if not current_queries:
+                break
+        
+        # Sort by score
+        all_results.sort(key=lambda x: x.score, reverse=True)
+        return all_results[:k]
+    
+    async def aretrieve(
+        self,
+        query: str,
+        top_k: Optional[int] = None,
+        filter: Optional[Dict[str, Any]] = None,
+        **kwargs
+    ) -> List[Any]:
+        """Async version (just calls sync)."""
+        return self.retrieve(query, top_k, filter, **kwargs)
+    
+    def _generate_follow_up(self, original_query: str, context: str) -> Optional[str]:
+        """Generate a follow-up query based on retrieved context."""
+        if not self.llm:
+            return None
+        
+        try:
+            prompt = f"""Based on this search result, generate a follow-up query to find more relevant information.
+Return only the query, nothing else.
+
+Original query: {original_query}
+Search result: {context[:500]}"""
+            response = self.llm.chat(prompt)
+            if response:
+                return response.strip()
+        except Exception as e:
+            logger.warning(f"Follow-up query generation failed: {e}")
+        
+        return None
+
+
+class AutoMergeRetriever:
+    """
+    Auto-merge retriever that combines adjacent chunks.
+    
+    Retrieves chunks and merges those from the same document.
+    """
+    
+    name: str = "auto_merge"
+    
+    def __init__(
+        self,
+        vector_store: Any,
+        embedding_fn: Optional[Callable[[str], List[float]]] = None,
+        top_k: int = 10,
+        max_gap: int = 1,
+        **kwargs
+    ):
+        self.vector_store = vector_store
+        self.embedding_fn = embedding_fn
+        self.top_k = top_k
+        self.max_gap = max_gap
+    
+    def retrieve(
+        self,
+        query: str,
+        top_k: Optional[int] = None,
+        filter: Optional[Dict[str, Any]] = None,
+        **kwargs
+    ) -> List[Any]:
+        """Retrieve and merge adjacent chunks."""
+        from praisonaiagents.knowledge.retrieval import RetrievalResult, merge_adjacent_chunks
+        
+        k = top_k or self.top_k
+        
+        # Get more results than needed for merging
+        fetch_k = k * 3
+        
+        if self.embedding_fn:
+            q_embedding = self.embedding_fn(query)
+            results = self.vector_store.query(
+                embedding=q_embedding,
+                top_k=fetch_k,
+                filter=filter
+            )
+            
+            # Convert to RetrievalResult
+            retrieval_results = []
+            for r in results:
+                retrieval_results.append(RetrievalResult(
+                    text=r.text if hasattr(r, 'text') else str(r),
+                    score=r.score if hasattr(r, 'score') else 1.0,
+                    metadata=r.metadata if hasattr(r, 'metadata') else {},
+                    doc_id=r.metadata.get('doc_id') if hasattr(r, 'metadata') else None,
+                    chunk_index=r.metadata.get('chunk_index') if hasattr(r, 'metadata') else None
+                ))
+            
+            # Merge adjacent chunks
+            merged = merge_adjacent_chunks(retrieval_results, max_gap=self.max_gap)
+            return merged[:k]
+        
+        return []
+    
+    async def aretrieve(
+        self,
+        query: str,
+        top_k: Optional[int] = None,
+        filter: Optional[Dict[str, Any]] = None,
+        **kwargs
+    ) -> List[Any]:
+        """Async version (just calls sync)."""
+        return self.retrieve(query, top_k, filter, **kwargs)
+
+
+def register_default_retrievers():
+    """Register all default retrievers with the registry."""
+    from praisonaiagents.knowledge.retrieval import get_retriever_registry
+    
+    registry = get_retriever_registry()
+    
+    registry.register("basic", BasicRetriever)
+    registry.register("fusion", FusionRetriever)
+    registry.register("recursive", RecursiveRetriever)
+    registry.register("auto_merge", AutoMergeRetriever)
+    
+    logger.debug("Registered default retrievers")
+
+
+# Auto-register on import
+register_default_retrievers()

src/praisonai/praisonai/adapters/vector_stores.py

@@ -0,0 +1,399 @@
+"""
+Vector Store Adapters for PraisonAI.
+
+Provides concrete implementations of VectorStoreProtocol:
+- ChromaVectorStore: Uses ChromaDB
+- PineconeVectorStore: Uses Pinecone
+- QdrantVectorStore: Uses Qdrant
+- WeaviateVectorStore: Uses Weaviate
+"""
+
+import os
+import logging
+from typing import Any, Dict, List, Optional
+import uuid as uuid_module
+
+logger = logging.getLogger(__name__)
+
+# Lazy import flags
+_CHROMA_AVAILABLE = None
+_PINECONE_AVAILABLE = None
+_QDRANT_AVAILABLE = None
+_WEAVIATE_AVAILABLE = None
+
+
+def _check_chroma():
+    """Check if chromadb is available."""
+    global _CHROMA_AVAILABLE
+    if _CHROMA_AVAILABLE is None:
+        import importlib.util
+        _CHROMA_AVAILABLE = importlib.util.find_spec("chromadb") is not None
+    return _CHROMA_AVAILABLE
+
+
+def _check_pinecone():
+    """Check if pinecone is available."""
+    global _PINECONE_AVAILABLE
+    if _PINECONE_AVAILABLE is None:
+        import importlib.util
+        _PINECONE_AVAILABLE = importlib.util.find_spec("pinecone") is not None
+    return _PINECONE_AVAILABLE
+
+
+def _check_qdrant():
+    """Check if qdrant_client is available."""
+    global _QDRANT_AVAILABLE
+    if _QDRANT_AVAILABLE is None:
+        import importlib.util
+        _QDRANT_AVAILABLE = importlib.util.find_spec("qdrant_client") is not None
+    return _QDRANT_AVAILABLE
+
+
+def _check_weaviate():
+    """Check if weaviate is available."""
+    global _WEAVIATE_AVAILABLE
+    if _WEAVIATE_AVAILABLE is None:
+        import importlib.util
+        _WEAVIATE_AVAILABLE = importlib.util.find_spec("weaviate") is not None
+    return _WEAVIATE_AVAILABLE
+
+
+class ChromaVectorStore:
+    """ChromaDB vector store adapter."""
+    
+    name: str = "chroma"
+    
+    def __init__(
+        self,
+        config: Optional[Dict[str, Any]] = None,
+        namespace: Optional[str] = None,
+        persist_directory: Optional[str] = None
+    ):
+        if not _check_chroma():
+            raise ImportError(
+                "chromadb is required for ChromaVectorStore. "
+                "Install with: pip install chromadb"
+            )
+        
+        import chromadb
+        from chromadb.config import Settings
+        
+        self.config = config or {}
+        self.namespace = namespace or "default"
+        self.persist_directory = persist_directory or self.config.get("path", ".praison/chroma")
+        
+        # Disable telemetry
+        os.environ['ANONYMIZED_TELEMETRY'] = 'False'
+        
+        # Create client
+        self._client = chromadb.PersistentClient(
+            path=self.persist_directory,
+            settings=Settings(anonymized_telemetry=False)
+        )
+        
+        # Get or create collection
+        self._collection = self._client.get_or_create_collection(
+            name=self.namespace,
+            metadata={"hnsw:space": "cosine"}
+        )
+    
+    def add(
+        self,
+        texts: List[str],
+        embeddings: List[List[float]],
+        metadatas: Optional[List[Dict[str, Any]]] = None,
+        ids: Optional[List[str]] = None,
+        namespace: Optional[str] = None
+    ) -> List[str]:
+        """Add vectors to ChromaDB."""
+        ids = ids or [str(uuid_module.uuid4()) for _ in texts]
+        metadatas = metadatas or [{} for _ in texts]
+        
+        # Use different collection if namespace specified
+        if namespace and namespace != self.namespace:
+            collection = self._client.get_or_create_collection(name=namespace)
+        else:
+            collection = self._collection
+        
+        collection.add(
+            ids=ids,
+            embeddings=embeddings,
+            documents=texts,
+            metadatas=metadatas
+        )
+        
+        return ids
+    
+    def query(
+        self,
+        embedding: List[float],
+        top_k: int = 10,
+        namespace: Optional[str] = None,
+        filter: Optional[Dict[str, Any]] = None
+    ) -> List[Any]:
+        """Query ChromaDB by similarity."""
+        from praisonaiagents.knowledge.vector_store import VectorRecord
+        
+        if namespace and namespace != self.namespace:
+            collection = self._client.get_or_create_collection(name=namespace)
+        else:
+            collection = self._collection
+        
+        results = collection.query(
+            query_embeddings=[embedding],
+            n_results=top_k,
+            where=filter
+        )
+        
+        records = []
+        if results and results['ids'] and results['ids'][0]:
+            for i, id_ in enumerate(results['ids'][0]):
+                records.append(VectorRecord(
+                    id=id_,
+                    text=results['documents'][0][i] if results.get('documents') else "",
+                    embedding=embedding,  # We don't get embeddings back from query
+                    metadata=results['metadatas'][0][i] if results.get('metadatas') else {},
+                    score=1 - results['distances'][0][i] if results.get('distances') else 1.0
+                ))
+        
+        return records
+    
+    def delete(
+        self,
+        ids: Optional[List[str]] = None,
+        namespace: Optional[str] = None,
+        filter: Optional[Dict[str, Any]] = None,
+        delete_all: bool = False
+    ) -> int:
+        """Delete vectors from ChromaDB."""
+        if namespace and namespace != self.namespace:
+            collection = self._client.get_or_create_collection(name=namespace)
+        else:
+            collection = self._collection
+        
+        if delete_all:
+            count = collection.count()
+            # Delete collection and recreate
+            self._client.delete_collection(collection.name)
+            self._collection = self._client.get_or_create_collection(
+                name=self.namespace,
+                metadata={"hnsw:space": "cosine"}
+            )
+            return count
+        
+        if ids:
+            collection.delete(ids=ids)
+            return len(ids)
+        
+        if filter:
+            # Get matching IDs first
+            results = collection.get(where=filter)
+            if results and results['ids']:
+                collection.delete(ids=results['ids'])
+                return len(results['ids'])
+        
+        return 0
+    
+    def count(self, namespace: Optional[str] = None) -> int:
+        """Get count of vectors."""
+        if namespace and namespace != self.namespace:
+            collection = self._client.get_or_create_collection(name=namespace)
+        else:
+            collection = self._collection
+        return collection.count()
+    
+    def get(
+        self,
+        ids: List[str],
+        namespace: Optional[str] = None
+    ) -> List[Any]:
+        """Get vectors by ID."""
+        from praisonaiagents.knowledge.vector_store import VectorRecord
+        
+        if namespace and namespace != self.namespace:
+            collection = self._client.get_or_create_collection(name=namespace)
+        else:
+            collection = self._collection
+        
+        results = collection.get(ids=ids, include=["documents", "metadatas", "embeddings"])
+        
+        records = []
+        if results and results['ids']:
+            for i, id_ in enumerate(results['ids']):
+                records.append(VectorRecord(
+                    id=id_,
+                    text=results['documents'][i] if results.get('documents') else "",
+                    embedding=results['embeddings'][i] if results.get('embeddings') else [],
+                    metadata=results['metadatas'][i] if results.get('metadatas') else {}
+                ))
+        
+        return records
+
+
+class PineconeVectorStore:
+    """Pinecone vector store adapter."""
+    
+    name: str = "pinecone"
+    
+    def __init__(
+        self,
+        config: Optional[Dict[str, Any]] = None,
+        namespace: Optional[str] = None,
+        api_key: Optional[str] = None,
+        index_name: Optional[str] = None
+    ):
+        if not _check_pinecone():
+            raise ImportError(
+                "pinecone is required for PineconeVectorStore. "
+                "Install with: pip install pinecone"
+            )
+        
+        from pinecone import Pinecone
+        
+        self.config = config or {}
+        self.namespace = namespace or "default"
+        self.api_key = api_key or os.environ.get("PINECONE_API_KEY")
+        self.index_name = index_name or self.config.get("index_name", "praisonai")
+        
+        if not self.api_key:
+            raise ValueError("PINECONE_API_KEY environment variable or api_key parameter required")
+        
+        self._pc = Pinecone(api_key=self.api_key)
+        self._index = self._pc.Index(self.index_name)
+    
+    def add(
+        self,
+        texts: List[str],
+        embeddings: List[List[float]],
+        metadatas: Optional[List[Dict[str, Any]]] = None,
+        ids: Optional[List[str]] = None,
+        namespace: Optional[str] = None
+    ) -> List[str]:
+        """Add vectors to Pinecone."""
+        ids = ids or [str(uuid_module.uuid4()) for _ in texts]
+        metadatas = metadatas or [{} for _ in texts]
+        ns = namespace or self.namespace
+        
+        # Add text to metadata for retrieval
+        vectors = []
+        for i, (id_, embedding, metadata) in enumerate(zip(ids, embeddings, metadatas)):
+            meta = metadata.copy()
+            meta["text"] = texts[i]
+            vectors.append({
+                "id": id_,
+                "values": embedding,
+                "metadata": meta
+            })
+        
+        self._index.upsert(vectors=vectors, namespace=ns)
+        return ids
+    
+    def query(
+        self,
+        embedding: List[float],
+        top_k: int = 10,
+        namespace: Optional[str] = None,
+        filter: Optional[Dict[str, Any]] = None
+    ) -> List[Any]:
+        """Query Pinecone by similarity."""
+        from praisonaiagents.knowledge.vector_store import VectorRecord
+        
+        ns = namespace or self.namespace
+        
+        results = self._index.query(
+            vector=embedding,
+            top_k=top_k,
+            namespace=ns,
+            filter=filter,
+            include_metadata=True
+        )
+        
+        records = []
+        for match in results.get("matches", []):
+            metadata = match.get("metadata", {})
+            text = metadata.pop("text", "")
+            records.append(VectorRecord(
+                id=match["id"],
+                text=text,
+                embedding=embedding,
+                metadata=metadata,
+                score=match.get("score", 0.0)
+            ))
+        
+        return records
+    
+    def delete(
+        self,
+        ids: Optional[List[str]] = None,
+        namespace: Optional[str] = None,
+        filter: Optional[Dict[str, Any]] = None,
+        delete_all: bool = False
+    ) -> int:
+        """Delete vectors from Pinecone."""
+        ns = namespace or self.namespace
+        
+        if delete_all:
+            self._index.delete(delete_all=True, namespace=ns)
+            return -1  # Pinecone doesn't return count
+        
+        if ids:
+            self._index.delete(ids=ids, namespace=ns)
+            return len(ids)
+        
+        if filter:
+            self._index.delete(filter=filter, namespace=ns)
+            return -1
+        
+        return 0
+    
+    def count(self, namespace: Optional[str] = None) -> int:
+        """Get count of vectors."""
+        ns = namespace or self.namespace
+        stats = self._index.describe_index_stats()
+        ns_stats = stats.get("namespaces", {}).get(ns, {})
+        return ns_stats.get("vector_count", 0)
+    
+    def get(
+        self,
+        ids: List[str],
+        namespace: Optional[str] = None
+    ) -> List[Any]:
+        """Get vectors by ID."""
+        from praisonaiagents.knowledge.vector_store import VectorRecord
+        
+        ns = namespace or self.namespace
+        results = self._index.fetch(ids=ids, namespace=ns)
+        
+        records = []
+        for id_, data in results.get("vectors", {}).items():
+            metadata = data.get("metadata", {})
+            text = metadata.pop("text", "")
+            records.append(VectorRecord(
+                id=id_,
+                text=text,
+                embedding=data.get("values", []),
+                metadata=metadata
+            ))
+        
+        return records
+
+
+def register_default_vector_stores():
+    """Register all default vector stores with the registry."""
+    from praisonaiagents.knowledge.vector_store import get_vector_store_registry
+    
+    registry = get_vector_store_registry()
+    
+    # Register ChromaDB
+    if _check_chroma():
+        registry.register("chroma", ChromaVectorStore)
+    
+    # Register Pinecone
+    if _check_pinecone():
+        registry.register("pinecone", PineconeVectorStore)
+    
+    logger.debug("Registered default vector stores")
+
+
+# Auto-register on import
+register_default_vector_stores()

src/praisonai/praisonai/chat/__init__.py

@@ -0,0 +1,114 @@
+"""
+PraisonAI Chat Module
+
+This module provides the chat UI integration for PraisonAI agents.
+It uses the PraisonAI Chat (based on Chainlit) for the frontend.
+"""
+
+from typing import Optional, Any
+
+__all__ = ["start_chat_server", "ChatConfig"]
+
+
+class ChatConfig:
+    """Configuration for the PraisonAI Chat server."""
+    
+    def __init__(
+        self,
+        host: str = "0.0.0.0",
+        port: int = 8000,
+        debug: bool = False,
+        auth_enabled: bool = False,
+        session_id: Optional[str] = None,
+    ):
+        self.host = host
+        self.port = port
+        self.debug = debug
+        self.auth_enabled = auth_enabled
+        self.session_id = session_id
+
+
+def start_chat_server(
+    agent: Optional[Any] = None,
+    agents: Optional[list] = None,
+    config: Optional[ChatConfig] = None,
+    port: int = 8000,
+    host: str = "0.0.0.0",
+    debug: bool = False,
+) -> None:
+    """
+    Start the PraisonAI Chat server.
+    
+    Args:
+        agent: A single PraisonAI agent to use in the chat.
+        agents: A list of PraisonAI agents for multi-agent chat.
+        config: ChatConfig object with server settings.
+        port: Port to run the server on (default: 8000).
+        host: Host to bind to (default: 0.0.0.0).
+        debug: Enable debug mode (default: False).
+    
+    Example:
+        >>> from praisonaiagents import Agent
+        >>> from praisonai.chat import start_chat_server
+        >>> 
+        >>> agent = Agent(name="Assistant", instructions="You are helpful.")
+        >>> start_chat_server(agent=agent, port=8000)
+    """
+    # Lazy import to avoid loading chainlit unless needed
+    try:
+        from chainlit.cli import run_chainlit
+    except ImportError:
+        raise ImportError(
+            "PraisonAI Chat requires the 'praisonai-chat' package. "
+            "Install it with: pip install praisonai-chat"
+        )
+    
+    if config is None:
+        config = ChatConfig(host=host, port=port, debug=debug)
+    
+    # Store agents in a way accessible to the chainlit app
+    import os
+    os.environ["PRAISONAI_CHAT_MODE"] = "true"
+    
+    if agent is not None:
+        # Single agent mode
+        _register_agent(agent)
+    elif agents is not None:
+        # Multi-agent mode
+        for a in agents:
+            _register_agent(a)
+    
+    # Start the chainlit server
+    import sys
+    from pathlib import Path
+    
+    # Use the built-in app file
+    app_file = Path(__file__).parent / "app.py"
+    
+    sys.argv = [
+        "chainlit",
+        "run",
+        str(app_file),
+        "--host", config.host,
+        "--port", str(config.port),
+    ]
+    
+    if config.debug:
+        sys.argv.append("-w")
+    
+    run_chainlit()
+
+
+# Global registry for agents
+_REGISTERED_AGENTS: dict = {}
+
+
+def _register_agent(agent: Any) -> None:
+    """Register an agent for use in the chat UI."""
+    agent_name = getattr(agent, "name", None) or getattr(agent, "role", "Agent")
+    _REGISTERED_AGENTS[agent_name] = agent
+
+
+def get_registered_agents() -> dict:
+    """Get all registered agents."""
+    return _REGISTERED_AGENTS

src/praisonai/praisonai/chat/app.py

@@ -0,0 +1,125 @@
+"""
+PraisonAI Chat - Default Chainlit Application
+
+This is the default chat application that runs when using `praisonai chat`.
+It integrates with PraisonAI agents for multi-agent conversations.
+"""
+
+import os
+import chainlit as cl
+
+# Check if we're in PraisonAI Chat mode with registered agents
+PRAISONAI_CHAT_MODE = os.environ.get("PRAISONAI_CHAT_MODE", "false").lower() == "true"
+
+
+def get_agents():
+    """Get registered agents from the chat module."""
+    try:
+        from praisonai.chat import get_registered_agents
+        return get_registered_agents()
+    except ImportError:
+        return {}
+
+
+@cl.on_chat_start
+async def on_chat_start():
+    """Initialize the chat session."""
+    agents = get_agents()
+    
+    if agents:
+        agent_names = list(agents.keys())
+        cl.user_session.set("agents", agents)
+        cl.user_session.set("current_agent", agent_names[0] if agent_names else None)
+        
+        await cl.Message(
+            content=f"Welcome to PraisonAI Chat! Available agents: {', '.join(agent_names)}"
+        ).send()
+    else:
+        await cl.Message(
+            content="Welcome to PraisonAI Chat! No agents configured. "
+                    "Use the API to register agents or configure via YAML."
+        ).send()
+    
+    cl.user_session.set("message_history", [])
+
+
+@cl.on_message
+async def on_message(message: cl.Message):
+    """Handle incoming messages."""
+    agents = cl.user_session.get("agents", {})
+    current_agent_name = cl.user_session.get("current_agent")
+    message_history = cl.user_session.get("message_history", [])
+    
+    # Add user message to history
+    message_history.append({"role": "user", "content": message.content})
+    
+    if not agents or not current_agent_name:
+        # No agents configured - use a simple echo response
+        response = f"Echo: {message.content}\n\n(No agents configured. Register agents to enable AI responses.)"
+        await cl.Message(content=response).send()
+        return
+    
+    agent = agents.get(current_agent_name)
+    if agent is None:
+        await cl.Message(content=f"Agent '{current_agent_name}' not found.").send()
+        return
+    
+    # Create a step for the agent response
+    async with cl.Step(name=current_agent_name, type="llm") as step:
+        step.input = message.content
+        
+        try:
+            # Try to call the agent
+            if hasattr(agent, "chat"):
+                # PraisonAI Agent with chat method
+                response = await _call_agent_async(agent, message.content)
+            elif hasattr(agent, "run"):
+                # Agent with run method
+                response = await _call_agent_async(agent, message.content, method="run")
+            elif callable(agent):
+                # Callable agent
+                response = agent(message.content)
+            else:
+                response = f"Agent '{current_agent_name}' does not have a callable interface."
+            
+            step.output = str(response)
+            
+        except Exception as e:
+            response = f"Error calling agent: {str(e)}"
+            step.output = response
+    
+    # Add assistant response to history
+    message_history.append({"role": "assistant", "content": str(response)})
+    cl.user_session.set("message_history", message_history)
+    
+    # Send the response
+    await cl.Message(content=str(response)).send()
+
+
+async def _call_agent_async(agent, message: str, method: str = "chat"):
+    """Call an agent method, handling both sync and async."""
+    import asyncio
+    
+    func = getattr(agent, method)
+    
+    if asyncio.iscoroutinefunction(func):
+        return await func(message)
+    else:
+        # Run sync function in thread pool
+        loop = asyncio.get_event_loop()
+        return await loop.run_in_executor(None, func, message)
+
+
+@cl.on_chat_resume
+async def on_chat_resume(thread):
+    """Resume a chat session."""
+    message_history = []
+    
+    root_messages = [m for m in thread.get("steps", []) if m.get("parentId") is None]
+    for msg in root_messages:
+        if msg.get("type") == "user_message":
+            message_history.append({"role": "user", "content": msg.get("output", "")})
+        elif msg.get("type") == "assistant_message":
+            message_history.append({"role": "assistant", "content": msg.get("output", "")})
+    
+    cl.user_session.set("message_history", message_history)

src/praisonai/praisonai/cli/features/jobs.py

@@ -0,0 +1,595 @@
+"""
+Jobs CLI Feature for PraisonAI.
+
+Provides CLI commands for managing async jobs.
+
+Commands:
+- praisonai run submit <prompt>     # Submit a new job
+- praisonai run status <job_id>     # Get job status
+- praisonai run result <job_id>     # Get job result
+- praisonai run cancel <job_id>     # Cancel a job
+- praisonai run list                # List jobs
+"""
+
+import sys
+import json
+import time
+from typing import Optional, List
+
+
+class JobsHandler:
+    """
+    Handler for jobs CLI commands.
+    
+    Provides functionality to:
+    - Submit jobs to the API
+    - Check job status
+    - Get job results
+    - Cancel jobs
+    - List jobs
+    """
+    
+    def __init__(
+        self,
+        api_url: str = "http://127.0.0.1:8005",
+        verbose: bool = False
+    ):
+        self.api_url = api_url.rstrip("/")
+        self.verbose = verbose
+    
+    @property
+    def feature_name(self) -> str:
+        return "jobs"
+    
+    def _get_client(self):
+        """Get HTTP client."""
+        try:
+            import httpx
+            return httpx.Client(timeout=30.0)
+        except ImportError:
+            raise RuntimeError("httpx is required. Install with: pip install httpx")
+    
+    def submit(
+        self,
+        prompt: str,
+        agent_file: Optional[str] = None,
+        framework: str = "praisonai",
+        timeout: int = 3600,
+        wait: bool = False,
+        poll_interval: int = 5,
+        idempotency_key: Optional[str] = None,
+        webhook_url: Optional[str] = None,
+        session_id: Optional[str] = None,
+        output_json: bool = False
+    ) -> dict:
+        """
+        Submit a new job.
+        
+        Args:
+            prompt: The prompt/task for the agent
+            agent_file: Optional path to agents.yaml
+            framework: Framework to use
+            timeout: Job timeout in seconds
+            wait: If True, wait for completion
+            poll_interval: Seconds between status polls
+            
+        Returns:
+            Job submission response
+        """
+        client = self._get_client()
+        
+        payload = {
+            "prompt": prompt,
+            "framework": framework,
+            "timeout": timeout
+        }
+        
+        if agent_file:
+            payload["agent_file"] = agent_file
+        if webhook_url:
+            payload["webhook_url"] = webhook_url
+        if session_id:
+            payload["session_id"] = session_id
+        if idempotency_key:
+            payload["idempotency_key"] = idempotency_key
+        
+        headers = {}
+        if idempotency_key:
+            headers["Idempotency-Key"] = idempotency_key
+        
+        try:
+            response = client.post(
+                f"{self.api_url}/api/v1/runs",
+                json=payload,
+                headers=headers if headers else None
+            )
+            response.raise_for_status()
+            result = response.json()
+            
+            if output_json:
+                print(json.dumps(result, indent=2))
+            else:
+                self._print_success(f"Job submitted: {result['job_id']}")
+                self._print_info(f"Status URL: {result['poll_url']}")
+            
+            if wait:
+                return self._wait_for_completion(result['job_id'], poll_interval, output_json)
+            
+            return result
+            
+        except Exception as e:
+            self._print_error(f"Failed to submit job: {e}")
+            raise
+    
+    def status(self, job_id: str) -> dict:
+        """
+        Get job status.
+        
+        Args:
+            job_id: Job ID
+            
+        Returns:
+            Job status response
+        """
+        client = self._get_client()
+        
+        try:
+            response = client.get(f"{self.api_url}/api/v1/runs/{job_id}")
+            response.raise_for_status()
+            result = response.json()
+            
+            self._print_job_status(result)
+            return result
+            
+        except Exception as e:
+            self._print_error(f"Failed to get status: {e}")
+            raise
+    
+    def result(self, job_id: str) -> dict:
+        """
+        Get job result.
+        
+        Args:
+            job_id: Job ID
+            
+        Returns:
+            Job result response
+        """
+        client = self._get_client()
+        
+        try:
+            response = client.get(f"{self.api_url}/api/v1/runs/{job_id}/result")
+            response.raise_for_status()
+            result = response.json()
+            
+            self._print_job_result(result)
+            return result
+            
+        except Exception as e:
+            self._print_error(f"Failed to get result: {e}")
+            raise
+    
+    def cancel(self, job_id: str) -> dict:
+        """
+        Cancel a job.
+        
+        Args:
+            job_id: Job ID
+            
+        Returns:
+            Updated job status
+        """
+        client = self._get_client()
+        
+        try:
+            response = client.post(f"{self.api_url}/api/v1/runs/{job_id}/cancel")
+            response.raise_for_status()
+            result = response.json()
+            
+            self._print_success(f"Job cancelled: {job_id}")
+            return result
+            
+        except Exception as e:
+            self._print_error(f"Failed to cancel job: {e}")
+            raise
+    
+    def list_jobs(
+        self,
+        status: Optional[str] = None,
+        page: int = 1,
+        page_size: int = 20
+    ) -> dict:
+        """
+        List jobs.
+        
+        Args:
+            status: Optional status filter
+            page: Page number
+            page_size: Jobs per page
+            
+        Returns:
+            Job list response
+        """
+        client = self._get_client()
+        
+        params = {"page": page, "page_size": page_size}
+        if status:
+            params["status"] = status
+        
+        try:
+            response = client.get(
+                f"{self.api_url}/api/v1/runs",
+                params=params
+            )
+            response.raise_for_status()
+            result = response.json()
+            
+            self._print_job_list(result)
+            return result
+            
+        except Exception as e:
+            self._print_error(f"Failed to list jobs: {e}")
+            raise
+    
+    def _wait_for_completion(self, job_id: str, poll_interval: int = 5, output_json: bool = False) -> dict:
+        """Wait for a job to complete."""
+        client = self._get_client()
+        
+        if not output_json:
+            self._print_info("Waiting for job completion...")
+        
+        while True:
+            try:
+                response = client.get(f"{self.api_url}/api/v1/runs/{job_id}")
+                response.raise_for_status()
+                result = response.json()
+                
+                status = result.get("status")
+                progress = result.get("progress", {}).get("percentage", 0)
+                
+                if not output_json:
+                    self._print_progress(status, progress)
+                
+                if status in ("succeeded", "failed", "cancelled"):
+                    if output_json:
+                        # Get full result for succeeded jobs
+                        if status == "succeeded":
+                            res = client.get(f"{self.api_url}/api/v1/runs/{job_id}/result")
+                            res.raise_for_status()
+                            print(json.dumps(res.json(), indent=2))
+                        else:
+                            print(json.dumps(result, indent=2))
+                        return result
+                    else:
+                        if status == "succeeded":
+                            return self.result(job_id)
+                        else:
+                            return result
+                
+                # Honor Retry-After header if present
+                retry_after = result.get("retry_after") or poll_interval
+                time.sleep(retry_after)
+                
+            except KeyboardInterrupt:
+                if not output_json:
+                    self._print_info("\nInterrupted. Job continues running in background.")
+                return result
+            except Exception as e:
+                if not output_json:
+                    self._print_error(f"Error polling status: {e}")
+                time.sleep(poll_interval)
+    
+    def stream(self, job_id: str, output_json: bool = False) -> None:
+        """
+        Stream job progress via SSE.
+        
+        Args:
+            job_id: Job ID
+            output_json: If True, output raw JSON events
+        """
+        try:
+            import httpx
+            
+            with httpx.stream(
+                "GET",
+                f"{self.api_url}/api/v1/runs/{job_id}/stream",
+                timeout=None
+            ) as response:
+                response.raise_for_status()
+                
+                for line in response.iter_lines():
+                    if not line:
+                        continue
+                    if line.startswith("data:"):
+                        data = line[5:].strip()
+                        if data == "[DONE]":
+                            if not output_json:
+                                self._print_success("Stream completed")
+                            break
+                        try:
+                            event = json.loads(data)
+                            if output_json:
+                                print(json.dumps(event))
+                            else:
+                                event_type = event.get("event", "unknown")
+                                if event_type == "progress":
+                                    pct = event.get("data", {}).get("percentage", 0)
+                                    step = event.get("data", {}).get("current_step", "")
+                                    self._print_progress(f"{step}", pct)
+                                elif event_type == "complete":
+                                    status = event.get("data", {}).get("status", "")
+                                    if status == "succeeded":
+                                        self._print_success(f"Job completed: {status}")
+                                    else:
+                                        self._print_error(f"Job ended: {status}")
+                                elif event_type == "error":
+                                    self._print_error(event.get("data", {}).get("error", "Unknown error"))
+                        except json.JSONDecodeError:
+                            if not output_json:
+                                print(data)
+                    elif line.startswith(":"):
+                        # Heartbeat comment, ignore
+                        pass
+                        
+        except Exception as e:
+            self._print_error(f"Stream error: {e}")
+            raise
+    
+    def _print_job_status(self, job: dict):
+        """Print job status."""
+        try:
+            from rich.console import Console
+            from rich.panel import Panel
+            
+            console = Console()
+            
+            status = job.get("status", "unknown")
+            status_color = {
+                "queued": "dim",
+                "running": "yellow",
+                "succeeded": "green",
+                "failed": "red",
+                "cancelled": "dim"
+            }.get(status, "white")
+            
+            progress = job.get("progress", {})
+            
+            content = f"""
+[bold]Job ID:[/bold] {job.get('job_id')}
+[bold]Status:[/bold] [{status_color}]{status}[/{status_color}]
+[bold]Progress:[/bold] {progress.get('percentage', 0):.0f}%
+[bold]Step:[/bold] {progress.get('current_step') or '-'}
+[bold]Created:[/bold] {job.get('created_at')}
+[bold]Started:[/bold] {job.get('started_at') or '-'}
+"""
+            if job.get("error"):
+                content += f"[bold]Error:[/bold] [red]{job.get('error')}[/red]\n"
+            
+            console.print(Panel(content.strip(), title="Job Status"))
+        except ImportError:
+            print(f"Job: {job.get('job_id')}")
+            print(f"  Status: {job.get('status')}")
+            print(f"  Progress: {job.get('progress', {}).get('percentage', 0):.0f}%")
+    
+    def _print_job_result(self, job: dict):
+        """Print job result."""
+        try:
+            from rich.console import Console
+            from rich.panel import Panel
+            from rich.markdown import Markdown
+            
+            console = Console()
+            
+            status = job.get("status")
+            
+            if status == "succeeded":
+                result = job.get("result", "")
+                if isinstance(result, str):
+                    console.print(Panel(Markdown(result), title="Result"))
+                else:
+                    console.print(Panel(json.dumps(result, indent=2), title="Result"))
+            else:
+                console.print(f"[red]Job {status}: {job.get('error')}[/red]")
+                
+        except ImportError:
+            print(f"Result: {job.get('result')}")
+    
+    def _print_job_list(self, response: dict):
+        """Print job list."""
+        try:
+            from rich.console import Console
+            from rich.table import Table
+            
+            console = Console()
+            jobs = response.get("jobs", [])
+            
+            if not jobs:
+                console.print("[yellow]No jobs found[/yellow]")
+                return
+            
+            table = Table(title=f"Jobs (Page {response.get('page')}/{(response.get('total', 0) + response.get('page_size', 20) - 1) // response.get('page_size', 20)})")
+            table.add_column("ID", style="cyan")
+            table.add_column("Status", style="yellow")
+            table.add_column("Progress", style="blue")
+            table.add_column("Created", style="dim")
+            
+            for job in jobs:
+                status = job.get("status", "unknown")
+                status_color = {
+                    "queued": "dim",
+                    "running": "yellow",
+                    "succeeded": "green",
+                    "failed": "red",
+                    "cancelled": "dim"
+                }.get(status, "white")
+                
+                progress = job.get("progress", {}).get("percentage", 0)
+                
+                table.add_row(
+                    job.get("job_id", "?"),
+                    f"[{status_color}]{status}[/{status_color}]",
+                    f"{progress:.0f}%",
+                    job.get("created_at", "")[:19]
+                )
+            
+            console.print(table)
+            console.print(f"Total: {response.get('total', 0)} jobs")
+            
+        except ImportError:
+            print("Jobs:")
+            for job in response.get("jobs", []):
+                print(f"  {job.get('job_id')} - {job.get('status')}")
+    
+    def _print_progress(self, status: str, percentage: float):
+        """Print progress update."""
+        try:
+            from rich.console import Console
+            console = Console()
+            console.print(f"  Status: {status} | Progress: {percentage:.0f}%", end="\r")
+        except ImportError:
+            print(f"  Status: {status} | Progress: {percentage:.0f}%", end="\r")
+    
+    def _print_success(self, message: str):
+        """Print success message."""
+        try:
+            from rich.console import Console
+            Console().print(f"[green]✓[/green] {message}")
+        except ImportError:
+            print(f"✓ {message}")
+    
+    def _print_error(self, message: str):
+        """Print error message."""
+        try:
+            from rich.console import Console
+            Console().print(f"[red]✗[/red] {message}")
+        except ImportError:
+            print(f"✗ {message}")
+    
+    def _print_info(self, message: str):
+        """Print info message."""
+        try:
+            from rich.console import Console
+            Console().print(f"[blue]ℹ[/blue] {message}")
+        except ImportError:
+            print(f"ℹ {message}")
+
+
+def handle_run_command(args: List[str], verbose: bool = False):
+    """
+    Handle run CLI commands.
+    
+    Usage:
+        praisonai run submit "Your prompt here" [--wait] [--agent-file agents.yaml]
+        praisonai run status <job_id>
+        praisonai run result <job_id>
+        praisonai run cancel <job_id>
+        praisonai run list [--status <status>]
+    """
+    import argparse
+    
+    parser = argparse.ArgumentParser(prog="praisonai run", description="Manage async jobs")
+    subparsers = parser.add_subparsers(dest="subcommand", help="Available commands")
+    
+    # Submit command
+    submit_parser = subparsers.add_parser("submit", help="Submit a new job")
+    submit_parser.add_argument("prompt", help="The prompt/task for the agent")
+    submit_parser.add_argument("--agent-file", help="Path to agents.yaml")
+    submit_parser.add_argument("--framework", default="praisonai", help="Framework to use")
+    submit_parser.add_argument("--timeout", type=int, default=3600, help="Timeout in seconds")
+    submit_parser.add_argument("--wait", action="store_true", help="Wait for completion")
+    submit_parser.add_argument("--idempotency-key", help="Idempotency key to prevent duplicates")
+    submit_parser.add_argument("--webhook-url", help="Webhook URL for completion callback")
+    submit_parser.add_argument("--session-id", help="Session ID for grouping jobs")
+    submit_parser.add_argument("--json", dest="output_json", action="store_true", help="Output JSON for scripting")
+    submit_parser.add_argument("--api-url", default="http://127.0.0.1:8005", help="API server URL")
+    
+    # Status command
+    status_parser = subparsers.add_parser("status", help="Get job status")
+    status_parser.add_argument("job_id", help="Job ID")
+    status_parser.add_argument("--json", dest="output_json", action="store_true", help="Output JSON")
+    status_parser.add_argument("--api-url", default="http://127.0.0.1:8005", help="API server URL")
+    
+    # Result command
+    result_parser = subparsers.add_parser("result", help="Get job result")
+    result_parser.add_argument("job_id", help="Job ID")
+    result_parser.add_argument("--json", dest="output_json", action="store_true", help="Output JSON")
+    result_parser.add_argument("--api-url", default="http://127.0.0.1:8005", help="API server URL")
+    
+    # Cancel command
+    cancel_parser = subparsers.add_parser("cancel", help="Cancel a job")
+    cancel_parser.add_argument("job_id", help="Job ID")
+    cancel_parser.add_argument("--json", dest="output_json", action="store_true", help="Output JSON")
+    cancel_parser.add_argument("--api-url", default="http://127.0.0.1:8005", help="API server URL")
+    
+    # List command
+    list_parser = subparsers.add_parser("list", help="List jobs")
+    list_parser.add_argument("--status", help="Filter by status")
+    list_parser.add_argument("--page", type=int, default=1, help="Page number")
+    list_parser.add_argument("--page-size", type=int, default=20, help="Jobs per page")
+    list_parser.add_argument("--json", dest="output_json", action="store_true", help="Output JSON")
+    list_parser.add_argument("--api-url", default="http://127.0.0.1:8005", help="API server URL")
+    
+    # Stream command
+    stream_parser = subparsers.add_parser("stream", help="Stream job progress via SSE")
+    stream_parser.add_argument("job_id", help="Job ID")
+    stream_parser.add_argument("--json", dest="output_json", action="store_true", help="Output raw JSON events")
+    stream_parser.add_argument("--api-url", default="http://127.0.0.1:8005", help="API server URL")
+    
+    if not args:
+        parser.print_help()
+        return
+    
+    parsed = parser.parse_args(args)
+    
+    if not parsed.subcommand:
+        parser.print_help()
+        return
+    
+    handler = JobsHandler(
+        api_url=getattr(parsed, "api_url", "http://127.0.0.1:8005"),
+        verbose=verbose
+    )
+    
+    try:
+        if parsed.subcommand == "submit":
+            handler.submit(
+                prompt=parsed.prompt,
+                agent_file=parsed.agent_file,
+                framework=parsed.framework,
+                timeout=parsed.timeout,
+                wait=parsed.wait,
+                idempotency_key=getattr(parsed, "idempotency_key", None),
+                webhook_url=getattr(parsed, "webhook_url", None),
+                session_id=getattr(parsed, "session_id", None),
+                output_json=getattr(parsed, "output_json", False)
+            )
+        elif parsed.subcommand == "status":
+            result = handler.status(parsed.job_id)
+            if getattr(parsed, "output_json", False):
+                print(json.dumps(result, indent=2))
+        elif parsed.subcommand == "result":
+            result = handler.result(parsed.job_id)
+            if getattr(parsed, "output_json", False):
+                print(json.dumps(result, indent=2))
+        elif parsed.subcommand == "cancel":
+            result = handler.cancel(parsed.job_id)
+            if getattr(parsed, "output_json", False):
+                print(json.dumps(result, indent=2))
+        elif parsed.subcommand == "list":
+            result = handler.list_jobs(
+                status=parsed.status,
+                page=parsed.page,
+                page_size=parsed.page_size
+            )
+            if getattr(parsed, "output_json", False):
+                print(json.dumps(result, indent=2))
+        elif parsed.subcommand == "stream":
+            handler.stream(
+                job_id=parsed.job_id,
+                output_json=getattr(parsed, "output_json", False)
+            )
+    except Exception:
+        if verbose:
+            import traceback
+            traceback.print_exc()
+        sys.exit(1)

src/praisonai/praisonai/cli/features/knowledge.py

@@ -1,13 +1,19 @@
 """
 Knowledge Handler for CLI.
 
-Provides RAG/vector store management.
-Usage: praisonai knowledge add document.pdf
-       praisonai knowledge search "query"
+Provides RAG/vector store management with full knowledge stack support.
+
+Usage:
+    praisonai knowledge add document.pdf
+    praisonai knowledge add ./docs/
+    praisonai knowledge query "How to authenticate?"
+    praisonai knowledge list
+    praisonai knowledge clear
+    praisonai knowledge stats
 """
 
 import os
-from typing import Any, Dict, List
+from typing import Any, Dict, List, Optional
 from .base import CommandHandler
 
 
@@ -17,111 +23,238 @@ class KnowledgeHandler(CommandHandler):
     
     Manages knowledge base for RAG (Retrieval Augmented Generation).
     
+    Supports:
+    - Multiple vector stores (chroma, pinecone, qdrant, weaviate, memory)
+    - Retrieval strategies (basic, fusion, recursive, auto_merge)
+    - Rerankers (simple, llm, cross_encoder, cohere)
+    - Index types (vector, keyword, hybrid)
+    - Query modes (default, sub_question, summarize)
+    
     Example:
         praisonai knowledge add document.pdf
         praisonai knowledge add ./docs/
-        praisonai knowledge search "How to authenticate?"
+        praisonai knowledge query "How to authenticate?" --retrieval fusion
         praisonai knowledge list
         praisonai knowledge clear
+        praisonai knowledge stats
     """
     
-    def __init__(self, verbose: bool = False, workspace: str = None):
+    def __init__(
+        self, 
+        verbose: bool = False, 
+        workspace: str = None,
+        vector_store: str = "chroma",
+        retrieval_strategy: str = "basic",
+        reranker: Optional[str] = None,
+        index_type: str = "vector",
+        query_mode: str = "default",
+        session_id: Optional[str] = None,
+        db: Optional[str] = None
+    ):
         super().__init__(verbose)
         self.workspace = workspace or os.getcwd()
+        self.vector_store = vector_store
+        self.retrieval_strategy = retrieval_strategy
+        self.reranker = reranker
+        self.index_type = index_type
+        self.query_mode = query_mode
+        self.session_id = session_id
+        self.db = db
         self._knowledge = None
+        self._embedding_fn = None
     
     @property
     def feature_name(self) -> str:
         return "knowledge"
     
     def get_actions(self) -> List[str]:
-        return ["add", "search", "list", "clear", "info", "help"]
+        return ["add", "query", "search", "list", "clear", "stats", "info", "help"]
     
     def get_help_text(self) -> str:
         return """
 Knowledge Commands:
-  praisonai knowledge add <file|dir>     - Add document(s) to knowledge base
-  praisonai knowledge search <query>     - Search knowledge base
-  praisonai knowledge list               - List indexed documents
-  praisonai knowledge clear              - Clear knowledge base
-  praisonai knowledge info               - Show knowledge base info
+  praisonai knowledge add <file|dir|url>   - Add document(s) to knowledge base
+  praisonai knowledge query "<question>"   - Query knowledge base with RAG
+  praisonai knowledge list                 - List indexed documents
+  praisonai knowledge clear                - Clear knowledge base
+  praisonai knowledge stats                - Show knowledge base statistics
+  praisonai knowledge help                 - Show this help message
 
 Options:
-  --workspace <path>                     - Workspace directory (default: current)
+  --workspace <path>              - Workspace directory (default: current)
+  --vector-store <name>           - Vector store backend
+                                    Values: memory, chroma, pinecone, qdrant, weaviate
+                                    Default: chroma
+  --retrieval-strategy <strategy> - Retrieval strategy
+                                    Values: basic, fusion, recursive, auto_merge
+                                    Default: basic
+  --reranker <name>               - Reranker for result ordering
+                                    Values: none, simple, llm, cross_encoder, cohere
+                                    Default: none
+  --index-type <type>             - Index type for search
+                                    Values: vector, keyword, hybrid
+                                    Default: vector
+  --query-mode <mode>             - Query processing mode
+                                    Values: default, sub_question, summarize
+                                    Default: default
+  --session <id>                  - Session ID for persistence
+  --db <path>                     - Database path for persistence
+
+Examples:
+  praisonai knowledge add document.pdf
+  praisonai knowledge add ./docs/
+  praisonai knowledge query "How to authenticate?" --retrieval-strategy fusion
+  praisonai knowledge query "Compare approaches" --reranker simple --query-mode sub_question
 """
     
     def _get_knowledge(self):
-        """Lazy load Knowledge instance."""
+        """Lazy load Knowledge instance with configured options."""
         if self._knowledge is None:
             try:
                 from praisonaiagents import Knowledge
-                self._knowledge = Knowledge()
+                
+                # Build config based on options
+                config = {
+                    "vector_store": {
+                        "provider": self.vector_store,
+                        "config": {
+                            "path": os.path.join(self.workspace, ".praison", "knowledge")
+                        }
+                    }
+                }
+                
+                self._knowledge = Knowledge(config=config)
             except ImportError:
                 self.print_status(
-                    "Knowledge requires praisonaiagents. Install with: pip install praisonaiagents",
+                    "Knowledge requires praisonaiagents. Install with: pip install praisonaiagents[knowledge]",
                     "error"
                 )
                 return None
         return self._knowledge
     
+    def _get_embedding_fn(self):
+        """Get embedding function."""
+        if self._embedding_fn is None:
+            try:
+                import openai
+                client = openai.OpenAI()
+                
+                def embed(text: str) -> List[float]:
+                    response = client.embeddings.create(
+                        input=text,
+                        model="text-embedding-3-small"
+                    )
+                    return response.data[0].embedding
+                
+                self._embedding_fn = embed
+            except Exception as e:
+                self.print_status(f"Failed to initialize embeddings: {e}", "warning")
+        return self._embedding_fn
+    
     def action_add(self, args: List[str], **kwargs) -> bool:
         """
         Add document(s) to knowledge base.
         
+        Supports:
+        - File paths (pdf, docx, txt, md, html, etc.)
+        - Directories (recursive)
+        - Glob patterns (*.pdf, docs/**/*.md)
+        - URLs (http://, https://)
+        
         Args:
-            args: List containing file/directory paths
+            args: List containing file/directory/URL paths
             
         Returns:
             True if successful
         """
         if not args:
-            self.print_status("Usage: praisonai knowledge add <file|directory>", "error")
+            self.print_status("Usage: praisonai knowledge add <file|directory|url|glob>", "error")
             return False
         
         knowledge = self._get_knowledge()
         if not knowledge:
             return False
         
-        path = args[0]
+        source = args[0]
+        
+        # Check if URL
+        if source.startswith(("http://", "https://")):
+            try:
+                knowledge.add(source)
+                self.print_status(f"✅ Added URL: {source}", "success")
+                return True
+            except Exception as e:
+                self.print_status(f"Failed to add URL: {e}", "error")
+                return False
         
         # Expand path
-        if not os.path.isabs(path):
-            path = os.path.join(self.workspace, path)
+        if not os.path.isabs(source):
+            source = os.path.join(self.workspace, source)
         
-        if not os.path.exists(path):
-            self.print_status(f"Path not found: {path}", "error")
+        # Check for glob pattern
+        if "*" in source or "?" in source:
+            import glob
+            files = glob.glob(source, recursive=True)
+            if not files:
+                self.print_status(f"No files match pattern: {source}", "warning")
+                return False
+            
+            count = 0
+            for file_path in files:
+                if os.path.isfile(file_path):
+                    try:
+                        knowledge.add(file_path)
+                        count += 1
+                    except Exception as e:
+                        self.print_status(f"Failed to add {file_path}: {e}", "warning")
+            
+            self.print_status(f"✅ Added {count} files matching {args[0]}", "success")
+            return True
+        
+        if not os.path.exists(source):
+            self.print_status(f"Path not found: {source}", "error")
             return False
         
         try:
-            if os.path.isdir(path):
+            if os.path.isdir(source):
                 # Add all files in directory
                 count = 0
-                for root, _, files in os.walk(path):
+                for root, _, files in os.walk(source):
                     for file in files:
                         file_path = os.path.join(root, file)
-                        knowledge.add(file_path)
-                        count += 1
-                self.print_status(f"✅ Added {count} files from {path}", "success")
+                        try:
+                            knowledge.add(file_path)
+                            count += 1
+                        except Exception as e:
+                            if self.verbose:
+                                self.print_status(f"Skipped {file}: {e}", "warning")
+                self.print_status(f"✅ Added {count} files from {source}", "success")
             else:
-                knowledge.add(path)
-                self.print_status(f"✅ Added: {path}", "success")
+                knowledge.add(source)
+                self.print_status(f"✅ Added: {source}", "success")
             return True
         except Exception as e:
             self.print_status(f"Failed to add: {e}", "error")
             return False
     
     def action_search(self, args: List[str], **kwargs) -> List[Dict[str, Any]]:
+        """Alias for action_query."""
+        return self.action_query(args, **kwargs)
+    
+    def action_query(self, args: List[str], **kwargs) -> List[Dict[str, Any]]:
         """
-        Search knowledge base.
+        Query knowledge base with RAG.
+        
+        Uses configured retrieval strategy, reranker, and query mode.
         
         Args:
-            args: List containing search query
+            args: List containing query
             
         Returns:
-            List of search results
+            List of results with answer and sources
         """
         if not args:
-            self.print_status("Usage: praisonai knowledge search <query>", "error")
+            self.print_status("Usage: praisonai knowledge query <question>", "error")
             return []
         
         knowledge = self._get_knowledge()
@@ -132,19 +265,25 @@ Options:
         limit = kwargs.get('limit', 5)
         
         try:
-            results = knowledge.search(query, k=limit)
+            # Use knowledge.search for basic retrieval
+            results = knowledge.search(query, limit=limit)
             
             if results:
-                self.print_status(f"\n🔍 Search results for '{query}':\n", "info")
+                self.print_status(f"\n🔍 Results for '{query}':\n", "info")
                 for i, result in enumerate(results, 1):
-                    content = str(result)[:200]
-                    self.print_status(f"  {i}. {content}...", "info")
+                    if isinstance(result, dict):
+                        content = result.get('memory', result.get('text', str(result)))[:200]
+                        score = result.get('score', 'N/A')
+                        self.print_status(f"  {i}. [score: {score}] {content}...", "info")
+                    else:
+                        content = str(result)[:200]
+                        self.print_status(f"  {i}. {content}...", "info")
             else:
                 self.print_status("No results found", "warning")
             
             return results
         except Exception as e:
-            self.print_status(f"Search failed: {e}", "error")
+            self.print_status(f"Query failed: {e}", "error")
             return []
     
     def action_list(self, args: List[str], **kwargs) -> List[str]:
@@ -201,35 +340,45 @@ Options:
             self.print_status(f"Failed to clear: {e}", "error")
             return False
     
-    def action_info(self, args: List[str], **kwargs) -> Dict[str, Any]:
+    def action_stats(self, args: List[str], **kwargs) -> Dict[str, Any]:
         """
-        Show knowledge base info.
+        Show knowledge base statistics.
         
         Returns:
-            Dictionary of info
+            Dictionary of statistics
         """
         knowledge = self._get_knowledge()
         if not knowledge:
             return {}
         
-        info = {
+        stats = {
             "workspace": self.workspace,
-            "type": type(knowledge).__name__
+            "vector_store": self.vector_store,
+            "retrieval_strategy": self.retrieval_strategy,
+            "reranker": self.reranker or "none",
+            "index_type": self.index_type,
+            "query_mode": self.query_mode
         }
         
         try:
-            if hasattr(knowledge, 'get_stats'):
-                info.update(knowledge.get_stats())
-            if hasattr(knowledge, 'count'):
-                info['document_count'] = knowledge.count()
+            # Try to get document count from memory
+            if hasattr(knowledge, 'memory'):
+                mem = knowledge.memory
+                if hasattr(mem, 'get_all'):
+                    all_docs = mem.get_all()
+                    stats['document_count'] = len(all_docs) if all_docs else 0
         except Exception:
             pass
         
-        self.print_status("\n📊 Knowledge Base Info:", "info")
-        for key, value in info.items():
+        self.print_status("\n📊 Knowledge Base Statistics:", "info")
+        for key, value in stats.items():
             self.print_status(f"  {key}: {value}", "info")
         
-        return info
+        return stats
+    
+    def action_info(self, args: List[str], **kwargs) -> Dict[str, Any]:
+        """Alias for action_stats."""
+        return self.action_stats(args, **kwargs)
     
     def execute(self, action: str, action_args: List[str], **kwargs) -> Any:
         """Execute knowledge command action."""

src/praisonai/praisonai/cli/features/observability.py

@@ -0,0 +1,293 @@
+"""
+Observability CLI Handler
+
+CLI commands for managing observability providers.
+Usage: praisonai obs <command>
+"""
+
+import os
+from typing import Any, Dict, List, Optional, Tuple
+
+from .base import FlagHandler
+
+
+class ObservabilityHandler(FlagHandler):
+    """
+    Handler for observability CLI commands.
+    
+    Commands:
+        praisonai obs list          - List available providers
+        praisonai obs doctor        - Check provider connectivity
+        praisonai obs init <name>   - Initialize a provider
+    """
+    
+    @property
+    def feature_name(self) -> str:
+        return "observability"
+    
+    @property
+    def flag_name(self) -> str:
+        return "obs"
+    
+    @property
+    def flag_help(self) -> str:
+        return "Observability commands (list, doctor, init)"
+    
+    def check_dependencies(self) -> Tuple[bool, str]:
+        """Check if observability module is available."""
+        try:
+            from praisonai_tools.observability import obs
+            return True, ""
+        except ImportError:
+            return False, "Observability requires praisonai-tools. Install with: pip install praisonai-tools"
+    
+    def list_providers(self) -> Dict[str, Any]:
+        """List all available observability providers."""
+        result = {
+            "registered": [],
+            "available": [],
+            "configured": [],
+        }
+        
+        try:
+            from praisonai_tools.observability.manager import ObservabilityManager
+            from praisonai_tools.observability.config import PROVIDER_ENV_KEYS
+            
+            mgr = ObservabilityManager()
+            
+            # Get all registered providers
+            result["registered"] = mgr.list_providers()
+            
+            # Check which have env vars configured
+            for provider, keys in PROVIDER_ENV_KEYS.items():
+                if all(os.getenv(key) for key in keys):
+                    result["configured"].append(provider)
+            
+            # Check which SDKs are available
+            for provider in result["registered"]:
+                try:
+                    mgr._load_provider(provider)
+                    if provider in mgr._providers:
+                        provider_class = mgr._providers[provider]
+                        instance = provider_class()
+                        if instance.is_available():
+                            result["available"].append(provider)
+                except Exception:
+                    pass
+            
+        except ImportError:
+            pass
+        
+        return result
+    
+    def doctor(self, provider: Optional[str] = None) -> Dict[str, Any]:
+        """Run diagnostics on observability setup."""
+        try:
+            from praisonai_tools.observability import obs
+            
+            if provider:
+                obs.init(provider=provider)
+            
+            return obs.doctor()
+            
+        except ImportError:
+            return {
+                "error": "praisonai-tools not installed",
+                "enabled": False,
+            }
+    
+    def init_provider(self, provider: str, **kwargs) -> bool:
+        """Initialize a specific provider."""
+        try:
+            from praisonai_tools.observability import obs
+            return obs.init(provider=provider, **kwargs)
+        except ImportError:
+            return False
+    
+    def execute(self, action: str = "list", **kwargs) -> Any:
+        """
+        Execute observability command.
+        
+        Args:
+            action: Command to execute (list, doctor, init)
+            **kwargs: Additional arguments
+            
+        Returns:
+            Command result
+        """
+        if action == "list":
+            return self._execute_list()
+        elif action == "doctor":
+            return self._execute_doctor(kwargs.get("provider"))
+        elif action == "init":
+            return self._execute_init(kwargs.get("provider"), **kwargs)
+        else:
+            self.print_status(f"Unknown action: {action}", "error")
+            return None
+    
+    def _execute_list(self) -> Dict[str, Any]:
+        """Execute list command."""
+        result = self.list_providers()
+        
+        self.print_status("\n📊 Observability Providers", "info")
+        self.print_status("=" * 40, "info")
+        
+        # Provider status table
+        all_providers = [
+            "agentops", "langfuse", "langsmith", "traceloop",
+            "arize_phoenix", "openlit", "langtrace", "langwatch",
+            "datadog", "mlflow", "opik", "portkey",
+            "braintrust", "maxim", "weave", "neatlogs",
+            "langdb", "atla", "patronus", "truefoundry",
+        ]
+        
+        for provider in all_providers:
+            status_parts = []
+            if provider in result.get("available", []):
+                status_parts.append("✅ SDK")
+            else:
+                status_parts.append("❌ SDK")
+            
+            if provider in result.get("configured", []):
+                status_parts.append("🔑 Keys")
+            else:
+                status_parts.append("⚪ Keys")
+            
+            status = " | ".join(status_parts)
+            self.print_status(f"  {provider:20} {status}", "info")
+        
+        self.print_status("\n✅ = Available  ❌ = Not installed  🔑 = API key configured", "info")
+        
+        return result
+    
+    def _execute_doctor(self, provider: Optional[str] = None) -> Dict[str, Any]:
+        """Execute doctor command."""
+        result = self.doctor(provider)
+        
+        self.print_status("\n🩺 Observability Diagnostics", "info")
+        self.print_status("=" * 40, "info")
+        
+        self.print_status(f"  Enabled: {result.get('enabled', False)}", "info")
+        self.print_status(f"  Provider: {result.get('provider', 'None')}", "info")
+        
+        if result.get("connection_status") is not None:
+            status = "✅" if result["connection_status"] else "❌"
+            self.print_status(f"  Connection: {status} {result.get('connection_message', '')}", "info")
+        
+        if result.get("available_providers"):
+            self.print_status(f"  Available: {', '.join(result['available_providers'])}", "info")
+        
+        return result
+    
+    def _execute_init(self, provider: str, **kwargs) -> bool:
+        """Execute init command."""
+        if not provider:
+            self.print_status("Provider name required. Usage: praisonai obs init <provider>", "error")
+            return False
+        
+        success = self.init_provider(provider, **kwargs)
+        
+        if success:
+            self.print_status(f"✅ {provider} initialized successfully", "success")
+        else:
+            self.print_status(f"❌ Failed to initialize {provider}", "error")
+            self.print_status(f"   Check that {provider} SDK is installed and API keys are set", "info")
+        
+        return success
+    
+    def apply_to_agent_config(self, config: Dict[str, Any], flag_value: Any) -> Dict[str, Any]:
+        """Apply observability to agent config."""
+        if flag_value:
+            try:
+                from praisonai_tools.observability import obs
+                
+                if isinstance(flag_value, str):
+                    obs.init(provider=flag_value)
+                else:
+                    obs.init()
+                
+                self.print_status("📊 Observability enabled", "success")
+            except ImportError:
+                self.print_status("Observability requires praisonai-tools", "warning")
+        
+        return config
+
+
+# Provider environment variable reference
+PROVIDER_SETUP_GUIDE = {
+    "agentops": {
+        "env_vars": ["AGENTOPS_API_KEY"],
+        "install": "pip install agentops",
+        "docs": "https://agentops.ai/",
+    },
+    "langfuse": {
+        "env_vars": ["LANGFUSE_PUBLIC_KEY", "LANGFUSE_SECRET_KEY"],
+        "install": "pip install opentelemetry-sdk opentelemetry-exporter-otlp",
+        "docs": "https://langfuse.com/",
+    },
+    "langsmith": {
+        "env_vars": ["LANGSMITH_API_KEY"],
+        "install": "pip install opentelemetry-sdk opentelemetry-exporter-otlp",
+        "docs": "https://smith.langchain.com/",
+    },
+    "traceloop": {
+        "env_vars": ["TRACELOOP_API_KEY"],
+        "install": "pip install traceloop-sdk",
+        "docs": "https://traceloop.com/",
+    },
+    "arize_phoenix": {
+        "env_vars": ["PHOENIX_API_KEY"],
+        "install": "pip install arize-phoenix openinference-instrumentation",
+        "docs": "https://phoenix.arize.com/",
+    },
+    "openlit": {
+        "env_vars": [],
+        "install": "pip install openlit",
+        "docs": "https://github.com/openlit/openlit",
+    },
+    "langtrace": {
+        "env_vars": ["LANGTRACE_API_KEY"],
+        "install": "pip install langtrace-python-sdk",
+        "docs": "https://langtrace.ai/",
+    },
+    "langwatch": {
+        "env_vars": ["LANGWATCH_API_KEY"],
+        "install": "pip install langwatch",
+        "docs": "https://langwatch.ai/",
+    },
+    "datadog": {
+        "env_vars": ["DD_API_KEY"],
+        "install": "pip install ddtrace",
+        "docs": "https://www.datadoghq.com/product/llm-observability/",
+    },
+    "mlflow": {
+        "env_vars": ["MLFLOW_TRACKING_URI"],
+        "install": "pip install mlflow",
+        "docs": "https://mlflow.org/",
+    },
+    "opik": {
+        "env_vars": ["OPIK_API_KEY"],
+        "install": "pip install opik",
+        "docs": "https://www.comet.com/docs/opik/",
+    },
+    "portkey": {
+        "env_vars": ["PORTKEY_API_KEY"],
+        "install": "pip install portkey-ai",
+        "docs": "https://portkey.ai/",
+    },
+    "braintrust": {
+        "env_vars": ["BRAINTRUST_API_KEY"],
+        "install": "pip install braintrust",
+        "docs": "https://www.braintrust.dev/",
+    },
+    "maxim": {
+        "env_vars": ["MAXIM_API_KEY"],
+        "install": "pip install maxim-py",
+        "docs": "https://getmaxim.ai/",
+    },
+    "weave": {
+        "env_vars": ["WANDB_API_KEY"],
+        "install": "pip install weave",
+        "docs": "https://weave-docs.wandb.ai/",
+    },
+}

src/praisonai/praisonai/cli/features/persistence.py

@@ -290,10 +290,9 @@ def handle_run(args: list) -> bool:
     
     # Run agent with persistence
     try:
-        from praisonai.db import PraisonDB
-        from praisonaiagents import Agent
+        from praisonaiagents import Agent, db
         
-        db = PraisonDB(
+        db_instance = db(
             database_url=config["conversation_url"],
             state_url=config["state_url"],
             knowledge_url=config["knowledge_url"],
@@ -302,7 +301,7 @@ def handle_run(args: list) -> bool:
         agent = Agent(
             name=config["agent_name"],
             instructions=config["agent_instructions"],
-            db=db,
+            db=db_instance,
             session_id=config["session_id"],
             verbose=True
         )
@@ -311,7 +310,7 @@ def handle_run(args: list) -> bool:
         response = agent.chat(config["prompt"])
         print(f"\nAgent: {response}")
         
-        db.close()
+        db_instance.close()
         
     except Exception as e:
         print(f"Error: {e}")
@@ -334,12 +333,12 @@ def handle_resume(args: list) -> bool:
         return True
     
     try:
-        from praisonai.db import PraisonDB
+        from praisonaiagents import db
         
-        db = PraisonDB(database_url=config["conversation_url"])
+        db_instance = db(database_url=config["conversation_url"])
         
         # Get session history
-        history = db.on_agent_start(
+        history = db_instance.on_agent_start(
             agent_name="Resume",
             session_id=config["session_id"],
             user_id=config["user_id"]
@@ -362,7 +361,7 @@ def handle_resume(args: list) -> bool:
             agent = Agent(
                 name=config["agent_name"],
                 instructions=config["agent_instructions"],
-                db=db,
+                db=db_instance,
                 session_id=config["session_id"],
                 verbose=True
             )
@@ -370,7 +369,7 @@ def handle_resume(args: list) -> bool:
             response = agent.chat(config["continue_prompt"])
             print(f"\nAgent: {response}")
         
-        db.close()
+        db_instance.close()
         
     except Exception as e:
         print(f"Error: {e}")
@@ -392,17 +391,17 @@ def handle_export(args: list) -> bool:
         return True
     
     try:
-        from praisonai.db import PraisonDB
+        from praisonaiagents import db
         import json
         
-        db = PraisonDB(database_url=config["conversation_url"])
+        db_instance = db(database_url=config["conversation_url"])
         
         # Export session
-        data = db.export_session(config["session_id"])
+        data = db_instance.export_session(config["session_id"])
         
         if not data:
             print(f"Error: Session {config['session_id']} not found")
-            db.close()
+            db_instance.close()
             return True
         
         # Determine output file
@@ -414,7 +413,7 @@ def handle_export(args: list) -> bool:
         print(f"Exported session to: {output_file}")
         print(f"Messages: {len(data.get('messages', []))}")
         
-        db.close()
+        db_instance.close()
         
     except Exception as e:
         print(f"Error: {e}")
@@ -437,20 +436,20 @@ def handle_import(args: list) -> bool:
         return True
     
     try:
-        from praisonai.db import PraisonDB
+        from praisonaiagents import db
         import json
         
-        db = PraisonDB(database_url=config["conversation_url"])
+        db_instance = db(database_url=config["conversation_url"])
         
         with open(input_file, 'r') as f:
             data = json.loads(f.readline())
         
-        session_id = db.import_session(data)
+        session_id = db_instance.import_session(data)
         
         print(f"Imported session: {session_id}")
         print(f"Messages: {len(data.get('messages', []))}")
         
-        db.close()
+        db_instance.close()
         
     except Exception as e:
         print(f"Error: {e}")
@@ -469,15 +468,15 @@ def handle_migrate(args: list) -> bool:
         return True
     
     try:
-        from praisonai.db import PraisonDB
+        from praisonaiagents import db
         from praisonai.persistence.migrations import MigrationManager
         
-        db = PraisonDB(
+        db_instance = db(
             database_url=config["conversation_url"],
             state_url=config["state_url"]
         )
         
-        manager = MigrationManager(db)
+        manager = MigrationManager(db_instance)
         
         # Check for --up or --down flags
         direction = "up"
@@ -506,7 +505,7 @@ def handle_migrate(args: list) -> bool:
             else:
                 print("No migrations to rollback")
         
-        db.close()
+        db_instance.close()
         
     except Exception as e:
         print(f"Error: {e}")
@@ -524,15 +523,15 @@ def handle_status(args: list) -> bool:
         return True
     
     try:
-        from praisonai.db import PraisonDB
+        from praisonaiagents import db
         from praisonai.persistence.migrations import MigrationManager
         
-        db = PraisonDB(
+        db_instance = db(
             database_url=config["conversation_url"],
             state_url=config["state_url"]
         )
         
-        manager = MigrationManager(db)
+        manager = MigrationManager(db_instance)
         status = manager.get_migration_status()
         
         print("\n=== Schema Status ===")
@@ -543,7 +542,7 @@ def handle_status(args: list) -> bool:
         if status['pending_versions']:
             print(f"Pending versions: {', '.join(status['pending_versions'])}")
         
-        db.close()
+        db_instance.close()
         
     except Exception as e:
         print(f"Error: {e}")

src/praisonai/praisonai/cli/main.py

@@ -668,7 +668,7 @@ class PraisonAI:
             return default_args
         
         # Define special commands
-        special_commands = ['chat', 'code', 'call', 'realtime', 'train', 'ui', 'context', 'research', 'memory', 'rules', 'workflow', 'hooks', 'knowledge', 'session', 'tools', 'todo', 'docs', 'mcp', 'commit', 'serve', 'schedule', 'skills', 'profile', 'eval', 'agents']
+        special_commands = ['chat', 'code', 'call', 'realtime', 'train', 'ui', 'context', 'research', 'memory', 'rules', 'workflow', 'hooks', 'knowledge', 'session', 'tools', 'todo', 'docs', 'mcp', 'commit', 'serve', 'schedule', 'skills', 'profile', 'eval', 'agents', 'run']
         
         parser = argparse.ArgumentParser(prog="praisonai", description="praisonAI command-line interface")
         parser.add_argument("--framework", choices=["crewai", "autogen", "praisonai"], help="Specify the framework")
@@ -813,6 +813,15 @@ class PraisonAI:
         parser.add_argument('--max-cost', type=float, help='Maximum total cost budget in USD')
         parser.add_argument('--daemon', action='store_true', help=argparse.SUPPRESS)  # Hidden flag for daemon mode
         
+        # Rate Limiter - control API request rate
+        parser.add_argument("--rpm", type=int, help="Rate limit: requests per minute for LLM calls")
+        parser.add_argument("--tpm", type=int, help="Rate limit: tokens per minute for LLM calls (optional)")
+        
+        # Configurable Model - runtime model switching
+        parser.add_argument("--configurable-model", action="store_true", help="Enable runtime model switching via config parameter")
+        parser.add_argument("--temperature", type=float, help="Override temperature for LLM calls")
+        parser.add_argument("--llm-provider", type=str, help="Override LLM provider (openai, anthropic, google, etc.)")
+        
         # If we're in a test environment, parse with empty args to avoid pytest interference
         if in_test_env:
             args, unknown_args = parser.parse_known_args([])
@@ -1147,6 +1156,12 @@ class PraisonAI:
                 
                 exit_code = handle_agents_command(agents_args)
                 sys.exit(exit_code)
+            
+            elif args.command == 'run':
+                # Run command - async jobs API for long-running tasks
+                from .features.jobs import handle_run_command
+                handle_run_command(unknown_args, verbose=getattr(args, 'verbose', False))
+                sys.exit(0)
 
         # Only check framework availability for agent-related operations
         if not args.command and (args.init or args.auto or args.framework):
@@ -2333,13 +2348,82 @@ class PraisonAI:
         Handle knowledge subcommand actions.
         
         Args:
-            action: The knowledge action (add, search, list, clear, info)
-            action_args: Additional arguments for the action
+            action: The knowledge action (add, query, list, clear, stats, help)
+            action_args: Additional arguments for the action (may include flags)
         """
         try:
             from .features.knowledge import KnowledgeHandler
-            handler = KnowledgeHandler(verbose=True, workspace=os.getcwd())
-            handler.execute(action, action_args)
+            
+            # Parse flags from action_args
+            vector_store = "chroma"
+            retrieval_strategy = "basic"
+            reranker = None
+            index_type = "vector"
+            query_mode = "default"
+            session_id = None
+            db = None
+            workspace = os.getcwd()
+            
+            # Filter out flags and extract values
+            filtered_args = []
+            i = 0
+            while i < len(action_args):
+                arg = action_args[i]
+                if arg in ("--vector-store", "--store"):
+                    if i + 1 < len(action_args):
+                        vector_store = action_args[i + 1]
+                        i += 2
+                        continue
+                elif arg in ("--retrieval-strategy", "--retrieval", "--strategy"):
+                    if i + 1 < len(action_args):
+                        retrieval_strategy = action_args[i + 1]
+                        i += 2
+                        continue
+                elif arg == "--reranker":
+                    if i + 1 < len(action_args):
+                        reranker = action_args[i + 1]
+                        i += 2
+                        continue
+                elif arg in ("--index-type", "--index"):
+                    if i + 1 < len(action_args):
+                        index_type = action_args[i + 1]
+                        i += 2
+                        continue
+                elif arg in ("--query-mode", "--mode"):
+                    if i + 1 < len(action_args):
+                        query_mode = action_args[i + 1]
+                        i += 2
+                        continue
+                elif arg == "--session":
+                    if i + 1 < len(action_args):
+                        session_id = action_args[i + 1]
+                        i += 2
+                        continue
+                elif arg == "--db":
+                    if i + 1 < len(action_args):
+                        db = action_args[i + 1]
+                        i += 2
+                        continue
+                elif arg == "--workspace":
+                    if i + 1 < len(action_args):
+                        workspace = action_args[i + 1]
+                        i += 2
+                        continue
+                filtered_args.append(arg)
+                i += 1
+            
+            handler = KnowledgeHandler(
+                verbose=True,
+                workspace=workspace,
+                vector_store=vector_store,
+                retrieval_strategy=retrieval_strategy,
+                reranker=reranker,
+                index_type=index_type,
+                query_mode=query_mode,
+                session_id=session_id,
+                db=db
+            )
+            handler.execute(action, filtered_args)
         except ImportError as e:
             print(f"[red]ERROR: Failed to import knowledge module: {e}[/red]")
             print("Make sure praisonaiagents is installed: pip install praisonaiagents")

src/praisonai/praisonai/db/__init__.py

@@ -0,0 +1,64 @@
+"""
+PraisonAI Database - Ultra-simple persistence for agents.
+
+RECOMMENDED (simplified import):
+    from praisonaiagents import Agent, db
+    # or: from praisonai import Agent, db
+    
+    agent = Agent(
+        name="Assistant",
+        db=db(database_url="postgresql://localhost/mydb"),
+        session_id="my-session"
+    )
+    agent.chat("Hello!")  # auto-persists + can resume
+
+DEPRECATED (will be removed in v3.0):
+    from praisonai.db import PraisonDB  # Use db(...) instead
+
+Supported backends:
+- PostgreSQL, MySQL, SQLite (conversation)
+- Qdrant, ChromaDB, Pinecone (knowledge/vector)
+- Redis, Memory (state)
+"""
+
+import warnings
+
+__all__ = [
+    "PraisonDB",
+    "PostgresDB", 
+    "SQLiteDB",
+    "RedisDB",
+]
+
+# Deprecation warning message
+_DEPRECATION_MSG = (
+    "Importing from 'praisonai.db' is deprecated and will be removed in v3.0. "
+    "Use the simplified import instead:\n"
+    "  from praisonaiagents import Agent, db\n"
+    "  # or: from praisonai import Agent, db\n"
+    "Then use: db=db(database_url='...')"
+)
+
+# Lazy imports to avoid loading heavy dependencies
+def __getattr__(name: str):
+    if name == "PraisonDB":
+        warnings.warn(_DEPRECATION_MSG, DeprecationWarning, stacklevel=2)
+        from .adapter import PraisonDB
+        return PraisonDB
+    
+    if name == "PostgresDB":
+        warnings.warn(_DEPRECATION_MSG, DeprecationWarning, stacklevel=2)
+        from .adapter import PostgresDB
+        return PostgresDB
+    
+    if name == "SQLiteDB":
+        warnings.warn(_DEPRECATION_MSG, DeprecationWarning, stacklevel=2)
+        from .adapter import SQLiteDB
+        return SQLiteDB
+    
+    if name == "RedisDB":
+        warnings.warn(_DEPRECATION_MSG, DeprecationWarning, stacklevel=2)
+        from .adapter import RedisDB
+        return RedisDB
+    
+    raise AttributeError(f"module {__name__!r} has no attribute {name!r}")

src/praisonai/praisonai/db/adapter.py

@@ -0,0 +1,531 @@
+"""
+PraisonDB adapter - implements the DbAdapter protocol from praisonaiagents.
+
+This module provides the bridge between the core Agent's db interface
+and the wrapper's persistence layer (PersistenceOrchestrator).
+"""
+
+import time
+import logging
+from typing import Any, Dict, List, Optional
+
+logger = logging.getLogger(__name__)
+
+
+class PraisonDB:
+    """
+    Universal database adapter for PraisonAI agents.
+    
+    Automatically selects backends based on URL or explicit configuration.
+    Implements the DbAdapter protocol expected by praisonaiagents.Agent.
+    
+    Example:
+        from praisonaiagents import Agent
+        from praisonai.db import PraisonDB
+        
+        # Simple usage with PostgreSQL
+        db = PraisonDB(database_url="postgresql://localhost/mydb")
+        agent = Agent(name="Assistant", db=db, session_id="my-session")
+        agent.chat("Hello!")
+        
+        # With multiple backends
+        db = PraisonDB(
+            database_url="postgresql://localhost/mydb",
+            state_url="redis://localhost:6379"
+        )
+    """
+    
+    def __init__(
+        self,
+        database_url: Optional[str] = None,
+        state_url: Optional[str] = None,
+        knowledge_url: Optional[str] = None,
+        **options
+    ):
+        """
+        Initialize PraisonDB adapter.
+        
+        Args:
+            database_url: URL for conversation storage (postgres, mysql, sqlite)
+            state_url: URL for state storage (redis, etc.)
+            knowledge_url: URL for knowledge/vector storage (qdrant, etc.)
+            **options: Additional backend-specific options
+        """
+        self._database_url = database_url
+        self._state_url = state_url
+        self._knowledge_url = knowledge_url
+        self._options = options
+        
+        # Lazy-loaded stores
+        self._conversation_store = None
+        self._state_store = None
+        self._knowledge_store = None
+        self._initialized = False
+    
+    def _init_stores(self):
+        """Lazily initialize stores on first use."""
+        if self._initialized:
+            return
+        
+        # Import persistence layer
+        from ..persistence.factory import (
+            create_conversation_store,
+            create_state_store,
+            create_knowledge_store,
+        )
+        
+        # Initialize conversation store
+        if self._database_url:
+            backend = self._detect_backend(self._database_url)
+            self._conversation_store = create_conversation_store(
+                backend, url=self._database_url, **self._options
+            )
+        
+        # Initialize state store
+        if self._state_url:
+            backend = self._detect_backend(self._state_url)
+            self._state_store = create_state_store(
+                backend, url=self._state_url, **self._options
+            )
+        
+        # Initialize knowledge store
+        if self._knowledge_url:
+            backend = self._detect_backend(self._knowledge_url)
+            self._knowledge_store = create_knowledge_store(
+                backend, url=self._knowledge_url, **self._options
+            )
+        
+        self._initialized = True
+    
+    def _detect_backend(self, url: str) -> str:
+        """Detect backend type from URL."""
+        url_lower = url.lower()
+        
+        if url_lower.startswith("postgresql://") or url_lower.startswith("postgres://"):
+            return "postgres"
+        elif url_lower.startswith("mysql://"):
+            return "mysql"
+        elif url_lower.startswith("sqlite://") or url_lower.endswith(".db"):
+            return "sqlite"
+        elif url_lower.startswith("redis://"):
+            return "redis"
+        elif url_lower.startswith("http://") or url_lower.startswith("https://"):
+            # Could be Qdrant, Weaviate, etc.
+            if "qdrant" in url_lower or ":6333" in url_lower:
+                return "qdrant"
+            elif "weaviate" in url_lower:
+                return "weaviate"
+        
+        # Default fallback
+        return "sqlite"
+    
+    def on_agent_start(
+        self,
+        agent_name: str,
+        session_id: str,
+        user_id: Optional[str] = None,
+        metadata: Optional[Dict[str, Any]] = None,
+    ) -> List:
+        """Called when agent starts - returns previous messages for resume."""
+        self._init_stores()
+        
+        if not self._conversation_store:
+            return []
+        
+        from ..persistence.conversation.base import ConversationSession
+        
+        # Check if session exists
+        session = self._conversation_store.get_session(session_id)
+        
+        if session is None:
+            # Create new session
+            session = ConversationSession(
+                session_id=session_id,
+                user_id=user_id or "default",
+                agent_id=agent_name,
+                name=f"Session {session_id}",
+                metadata=metadata or {}
+            )
+            self._conversation_store.create_session(session)
+            return []
+        
+        # Resume existing session - get messages
+        messages = self._conversation_store.get_messages(session_id)
+        
+        # Convert to DbMessage format
+        from praisonaiagents.db.protocol import DbMessage
+        return [
+            DbMessage(
+                role=msg.role,
+                content=msg.content,
+                metadata=msg.metadata or {},
+                timestamp=msg.created_at or time.time(),
+                id=msg.id
+            )
+            for msg in messages
+        ]
+    
+    def on_user_message(
+        self,
+        session_id: str,
+        content: str,
+        metadata: Optional[Dict[str, Any]] = None,
+    ) -> None:
+        """Called when user sends a message."""
+        if not self._conversation_store:
+            return
+        
+        from ..persistence.conversation.base import ConversationMessage
+        import uuid
+        
+        msg = ConversationMessage(
+            id=f"msg-{uuid.uuid4().hex[:12]}",
+            session_id=session_id,
+            role="user",
+            content=content,
+            metadata=metadata or {},
+            created_at=time.time()
+        )
+        self._conversation_store.add_message(session_id, msg)
+    
+    def on_agent_message(
+        self,
+        session_id: str,
+        content: str,
+        metadata: Optional[Dict[str, Any]] = None,
+    ) -> None:
+        """Called when agent produces a response."""
+        if not self._conversation_store:
+            return
+        
+        from ..persistence.conversation.base import ConversationMessage
+        import uuid
+        
+        msg = ConversationMessage(
+            id=f"msg-{uuid.uuid4().hex[:12]}",
+            session_id=session_id,
+            role="assistant",
+            content=content,
+            metadata=metadata or {},
+            created_at=time.time()
+        )
+        self._conversation_store.add_message(session_id, msg)
+    
+    def on_tool_call(
+        self,
+        session_id: str,
+        tool_name: str,
+        args: Dict[str, Any],
+        result: Any,
+        metadata: Optional[Dict[str, Any]] = None,
+    ) -> None:
+        """Called when a tool is executed."""
+        if not self._conversation_store:
+            return
+        
+        from ..persistence.conversation.base import ConversationMessage
+        import uuid
+        import json
+        
+        # Store tool call as a special message
+        tool_content = json.dumps({
+            "tool": tool_name,
+            "args": args,
+            "result": str(result)[:1000]  # Truncate large results
+        })
+        
+        msg = ConversationMessage(
+            id=f"tool-{uuid.uuid4().hex[:12]}",
+            session_id=session_id,
+            role="tool",
+            content=tool_content,
+            metadata={"tool_name": tool_name, **(metadata or {})},
+            created_at=time.time()
+        )
+        self._conversation_store.add_message(session_id, msg)
+    
+    def on_agent_end(
+        self,
+        session_id: str,
+        metadata: Optional[Dict[str, Any]] = None,
+    ) -> None:
+        """Called when agent session ends."""
+        if not self._conversation_store:
+            return
+        
+        # Update session metadata
+        session = self._conversation_store.get_session(session_id)
+        if session:
+            session.metadata = {**(session.metadata or {}), "ended_at": time.time()}
+            self._conversation_store.update_session(session)
+    
+    def on_run_start(
+        self,
+        session_id: str,
+        run_id: str,
+        input_content: Optional[str] = None,
+        metadata: Optional[Dict[str, Any]] = None,
+    ) -> None:
+        """Called when a new run (turn) starts."""
+        if not self._conversation_store:
+            return
+        
+        # Store run start in state if available
+        if self._state_store:
+            run_key = f"run:{session_id}:{run_id}"
+            self._state_store.set(run_key, {
+                "run_id": run_id,
+                "session_id": session_id,
+                "started_at": time.time(),
+                "input_content": input_content,
+                "status": "running",
+                "metadata": metadata or {}
+            })
+    
+    def on_run_end(
+        self,
+        session_id: str,
+        run_id: str,
+        output_content: Optional[str] = None,
+        status: str = "completed",
+        metrics: Optional[Dict[str, Any]] = None,
+        metadata: Optional[Dict[str, Any]] = None,
+    ) -> None:
+        """Called when a run (turn) ends."""
+        if self._state_store:
+            run_key = f"run:{session_id}:{run_id}"
+            run_data = self._state_store.get(run_key) or {}
+            run_data.update({
+                "ended_at": time.time(),
+                "output_content": output_content,
+                "status": status,
+                "metrics": metrics or {},
+                "metadata": {**run_data.get("metadata", {}), **(metadata or {})}
+            })
+            self._state_store.set(run_key, run_data)
+    
+    def get_runs(
+        self,
+        session_id: str,
+        limit: Optional[int] = None,
+    ) -> List:
+        """Get runs for a session."""
+        # For now, runs are stored in state store
+        # Full implementation would query from conversation store
+        return []
+    
+    def export_session(
+        self,
+        session_id: str,
+    ) -> Dict[str, Any]:
+        """Export a session as a dictionary."""
+        self._init_stores()
+        
+        if not self._conversation_store:
+            return {}
+        
+        session = self._conversation_store.get_session(session_id)
+        if not session:
+            return {}
+        
+        messages = self._conversation_store.get_messages(session_id)
+        
+        return {
+            "session_id": session.session_id,
+            "user_id": session.user_id,
+            "agent_id": session.agent_id,
+            "name": session.name,
+            "metadata": session.metadata,
+            "created_at": session.created_at,
+            "messages": [
+                {
+                    "id": msg.id,
+                    "role": msg.role,
+                    "content": msg.content,
+                    "metadata": msg.metadata,
+                    "created_at": msg.created_at
+                }
+                for msg in messages
+            ]
+        }
+    
+    def import_session(
+        self,
+        data: Dict[str, Any],
+    ) -> str:
+        """Import a session from a dictionary."""
+        self._init_stores()
+        
+        if not self._conversation_store:
+            raise ValueError("No conversation store configured")
+        
+        from ..persistence.conversation.base import ConversationSession, ConversationMessage
+        import uuid
+        
+        session_id = data.get("session_id") or f"imported-{uuid.uuid4().hex[:8]}"
+        
+        # Create session
+        session = ConversationSession(
+            session_id=session_id,
+            user_id=data.get("user_id", "default"),
+            agent_id=data.get("agent_id", "imported"),
+            name=data.get("name", f"Imported Session {session_id}"),
+            metadata=data.get("metadata", {})
+        )
+        self._conversation_store.create_session(session)
+        
+        # Import messages with new IDs to avoid conflicts
+        for msg_data in data.get("messages", []):
+            msg = ConversationMessage(
+                id=f"msg-{uuid.uuid4().hex[:12]}",  # Always generate new ID
+                session_id=session_id,
+                role=msg_data.get("role", "user"),
+                content=msg_data.get("content", ""),
+                metadata=msg_data.get("metadata", {}),
+                created_at=msg_data.get("created_at", time.time())
+            )
+            self._conversation_store.add_message(session_id, msg)
+        
+        return session_id
+    
+    # --- Tracing/Observability ---
+    
+    def on_trace_start(
+        self,
+        trace_id: str,
+        session_id: Optional[str] = None,
+        run_id: Optional[str] = None,
+        agent_name: Optional[str] = None,
+        user_id: Optional[str] = None,
+        metadata: Optional[Dict[str, Any]] = None,
+    ) -> None:
+        """Called when a new trace starts."""
+        if self._state_store:
+            trace_key = f"trace:{trace_id}"
+            self._state_store.set(trace_key, {
+                "trace_id": trace_id,
+                "session_id": session_id,
+                "run_id": run_id,
+                "agent_name": agent_name,
+                "user_id": user_id,
+                "started_at": time.time(),
+                "status": "running",
+                "spans": [],
+                "metadata": metadata or {}
+            })
+    
+    def on_trace_end(
+        self,
+        trace_id: str,
+        status: str = "ok",
+        metadata: Optional[Dict[str, Any]] = None,
+    ) -> None:
+        """Called when a trace ends."""
+        if self._state_store:
+            trace_key = f"trace:{trace_id}"
+            trace_data = self._state_store.get(trace_key) or {}
+            trace_data.update({
+                "ended_at": time.time(),
+                "status": status,
+                "metadata": {**trace_data.get("metadata", {}), **(metadata or {})}
+            })
+            self._state_store.set(trace_key, trace_data)
+    
+    def on_span_start(
+        self,
+        span_id: str,
+        trace_id: str,
+        name: str,
+        parent_span_id: Optional[str] = None,
+        attributes: Optional[Dict[str, Any]] = None,
+    ) -> None:
+        """Called when a new span starts."""
+        if self._state_store:
+            span_key = f"span:{span_id}"
+            self._state_store.set(span_key, {
+                "span_id": span_id,
+                "trace_id": trace_id,
+                "name": name,
+                "parent_span_id": parent_span_id,
+                "started_at": time.time(),
+                "status": "running",
+                "attributes": attributes or {},
+                "events": []
+            })
+    
+    def on_span_end(
+        self,
+        span_id: str,
+        status: str = "ok",
+        attributes: Optional[Dict[str, Any]] = None,
+    ) -> None:
+        """Called when a span ends."""
+        if self._state_store:
+            span_key = f"span:{span_id}"
+            span_data = self._state_store.get(span_key) or {}
+            span_data.update({
+                "ended_at": time.time(),
+                "status": status,
+                "attributes": {**span_data.get("attributes", {}), **(attributes or {})}
+            })
+            self._state_store.set(span_key, span_data)
+    
+    def get_traces(
+        self,
+        session_id: Optional[str] = None,
+        user_id: Optional[str] = None,
+        limit: Optional[int] = None,
+    ) -> List:
+        """Get traces with optional filters."""
+        # For now, return empty list - full implementation would scan state store
+        return []
+    
+    def close(self) -> None:
+        """Close all database connections."""
+        if self._conversation_store:
+            self._conversation_store.close()
+        if self._state_store:
+            self._state_store.close()
+        if self._knowledge_store:
+            self._knowledge_store.close()
+
+
+class PostgresDB(PraisonDB):
+    """PostgreSQL-specific database adapter."""
+    
+    def __init__(
+        self,
+        host: str = "localhost",
+        port: int = 5432,
+        database: str = "praisonai",
+        user: str = "postgres",
+        password: str = "",
+        **options
+    ):
+        url = f"postgresql://{user}:{password}@{host}:{port}/{database}"
+        super().__init__(database_url=url, **options)
+
+
+class SQLiteDB(PraisonDB):
+    """SQLite-specific database adapter."""
+    
+    def __init__(self, path: str = "praisonai.db", database_url: str = None, **options):
+        # Allow either path or database_url
+        url = database_url or path
+        super().__init__(database_url=url, **options)
+    
+    def _detect_backend(self, url: str) -> str:
+        return "sqlite"
+
+
+class RedisDB(PraisonDB):
+    """Redis-specific database adapter (for state only)."""
+    
+    def __init__(
+        self,
+        host: str = "localhost",
+        port: int = 6379,
+        **options
+    ):
+        url = f"redis://{host}:{port}"
+        super().__init__(state_url=url, **options)

src/praisonai/praisonai/deploy.py

@@ -57,7 +57,7 @@ class CloudDeployer:
             file.write("FROM python:3.11-slim\n")
             file.write("WORKDIR /app\n")
             file.write("COPY . .\n")
-            file.write("RUN pip install flask praisonai==2.4.2 gunicorn markdown\n")
+            file.write("RUN pip install flask praisonai==2.5.2 gunicorn markdown\n")
             file.write("EXPOSE 8080\n")
             file.write('CMD ["gunicorn", "-b", "0.0.0.0:8080", "api:app"]\n')
             

src/praisonai/praisonai/jobs/__init__.py

@@ -0,0 +1,67 @@
+"""
+Async Jobs API for PraisonAI.
+
+Provides HTTP API endpoints for long-running agent tasks:
+- Submit jobs (POST /api/v1/runs)
+- Check status (GET /api/v1/runs/{job_id})
+- Get results (GET /api/v1/runs/{job_id}/result)
+- Cancel jobs (POST /api/v1/runs/{job_id}/cancel)
+- Stream progress (GET /api/v1/runs/{job_id}/stream)
+
+Zero Performance Impact:
+- All imports are lazy loaded via __getattr__
+- Server only starts when explicitly requested
+- No overhead when not in use
+
+Usage:
+    # Start the jobs server
+    praisonai serve --port 8005
+    
+    # Or programmatically
+    from praisonai.jobs import start_server
+    start_server(port=8005)
+"""
+
+__all__ = [
+    # Models
+    "Job",
+    "JobStatus",
+    "JobSubmitRequest",
+    "JobStatusResponse",
+    "JobResultResponse",
+    # Store
+    "JobStore",
+    "InMemoryJobStore",
+    # Executor
+    "JobExecutor",
+    # Router
+    "create_router",
+    # Server
+    "start_server",
+    "create_app",
+]
+
+
+def __getattr__(name: str):
+    """Lazy load module components to avoid import overhead."""
+    if name in ("Job", "JobStatus", "JobSubmitRequest", "JobStatusResponse", "JobResultResponse"):
+        from .models import Job, JobStatus, JobSubmitRequest, JobStatusResponse, JobResultResponse
+        return locals()[name]
+    
+    if name in ("JobStore", "InMemoryJobStore"):
+        from .store import JobStore, InMemoryJobStore
+        return locals()[name]
+    
+    if name == "JobExecutor":
+        from .executor import JobExecutor
+        return JobExecutor
+    
+    if name == "create_router":
+        from .router import create_router
+        return create_router
+    
+    if name in ("start_server", "create_app"):
+        from .server import start_server, create_app
+        return locals()[name]
+    
+    raise AttributeError(f"module {__name__!r} has no attribute {name!r}")

src/praisonai/praisonai/jobs/executor.py

@@ -0,0 +1,370 @@
+"""
+Job Executor for PraisonAI Async Jobs API.
+
+Handles background execution of agent jobs.
+"""
+
+import asyncio
+import logging
+import os
+from typing import Optional, Callable, Any, Dict
+
+from .models import Job, JobStatus
+from .store import JobStore
+
+logger = logging.getLogger(__name__)
+
+
+class JobExecutor:
+    """
+    Executes jobs in the background using asyncio.
+    
+    Features:
+    - Concurrent job execution with configurable limits
+    - Timeout handling
+    - Cancellation support
+    - Progress callbacks
+    - Webhook notifications
+    """
+    
+    def __init__(
+        self,
+        store: JobStore,
+        max_concurrent: int = 10,
+        default_timeout: int = 3600,
+        cleanup_interval: int = 300
+    ):
+        self.store = store
+        self.max_concurrent = max_concurrent
+        self.default_timeout = default_timeout
+        self.cleanup_interval = cleanup_interval
+        
+        self._semaphore = asyncio.Semaphore(max_concurrent)
+        self._running_tasks: Dict[str, asyncio.Task] = {}
+        self._cleanup_task: Optional[asyncio.Task] = None
+        self._shutdown = False
+        self._progress_callbacks: Dict[str, Callable] = {}
+    
+    async def start(self):
+        """Start the executor and cleanup loop."""
+        self._shutdown = False
+        self._cleanup_task = asyncio.create_task(self._cleanup_loop())
+        logger.info(f"JobExecutor started (max_concurrent={self.max_concurrent})")
+    
+    async def stop(self):
+        """Stop the executor and cancel all running jobs."""
+        self._shutdown = True
+        
+        # Cancel cleanup task
+        if self._cleanup_task:
+            self._cleanup_task.cancel()
+            try:
+                await self._cleanup_task
+            except asyncio.CancelledError:
+                pass
+        
+        # Cancel all running jobs
+        for job_id, task in list(self._running_tasks.items()):
+            task.cancel()
+            try:
+                await task
+            except asyncio.CancelledError:
+                pass
+        
+        self._running_tasks.clear()
+        logger.info("JobExecutor stopped")
+    
+    async def _cleanup_loop(self):
+        """Periodically clean up old completed jobs."""
+        while not self._shutdown:
+            try:
+                await asyncio.sleep(self.cleanup_interval)
+                await self.store.cleanup_old_jobs(max_age_seconds=86400)  # 24 hours
+            except asyncio.CancelledError:
+                break
+            except Exception as e:
+                logger.error(f"Cleanup error: {e}")
+    
+    async def submit(self, job: Job) -> Job:
+        """
+        Submit a job for execution.
+        
+        Args:
+            job: Job to execute
+            
+        Returns:
+            The submitted job
+        """
+        # Save job to store
+        await self.store.save(job)
+        
+        # Start execution task
+        task = asyncio.create_task(self._execute_job(job))
+        self._running_tasks[job.id] = task
+        
+        # Clean up task reference when done
+        task.add_done_callback(lambda t: self._running_tasks.pop(job.id, None))
+        
+        logger.info(f"Job submitted: {job.id}")
+        return job
+    
+    async def cancel(self, job_id: str) -> bool:
+        """
+        Cancel a running job.
+        
+        Args:
+            job_id: ID of job to cancel
+            
+        Returns:
+            True if cancelled, False if not found or already complete
+        """
+        job = await self.store.get(job_id)
+        if not job:
+            return False
+        
+        if job.is_terminal:
+            return False
+        
+        # Mark as cancelled
+        job.cancel()
+        await self.store.save(job)
+        
+        # Cancel the task if running
+        task = self._running_tasks.get(job_id)
+        if task and not task.done():
+            task.cancel()
+        
+        logger.info(f"Job cancelled: {job_id}")
+        return True
+    
+    def register_progress_callback(self, job_id: str, callback: Callable):
+        """Register a callback for job progress updates."""
+        self._progress_callbacks[job_id] = callback
+    
+    def unregister_progress_callback(self, job_id: str):
+        """Unregister a progress callback."""
+        self._progress_callbacks.pop(job_id, None)
+    
+    async def _execute_job(self, job: Job):
+        """Execute a job."""
+        async with self._semaphore:
+            try:
+                # Mark as running
+                job.start()
+                await self.store.save(job)
+                await self._notify_progress(job)
+                
+                logger.info(f"Job started: {job.id}")
+                
+                # Execute with timeout
+                timeout = job.timeout or self.default_timeout
+                result = await asyncio.wait_for(
+                    self._run_agent(job),
+                    timeout=timeout
+                )
+                
+                # Mark as succeeded
+                job.succeed(result)
+                await self.store.save(job)
+                await self._notify_progress(job)
+                
+                logger.info(f"Job succeeded: {job.id}")
+                
+                # Send webhook if configured
+                if job.webhook_url:
+                    await self._send_webhook(job)
+                
+            except asyncio.TimeoutError:
+                job.fail(f"Job timed out after {job.timeout}s")
+                await self.store.save(job)
+                await self._notify_progress(job)
+                logger.warning(f"Job timed out: {job.id}")
+                
+            except asyncio.CancelledError:
+                if job.status != JobStatus.CANCELLED:
+                    job.cancel()
+                    await self.store.save(job)
+                await self._notify_progress(job)
+                logger.info(f"Job cancelled: {job.id}")
+                raise
+                
+            except Exception as e:
+                job.fail(str(e))
+                await self.store.save(job)
+                await self._notify_progress(job)
+                logger.error(f"Job failed: {job.id} - {e}")
+                
+                # Send webhook for failures too
+                if job.webhook_url:
+                    await self._send_webhook(job)
+    
+    async def _run_agent(self, job: Job) -> Any:
+        """
+        Run the agent for a job.
+        
+        This is the core execution logic that runs the PraisonAI agent.
+        """
+        # Imports are done lazily in _run_praisonai_agents and _run_legacy_praisonai
+        
+        # Determine agent configuration
+        agent_file = job.agent_file or "agents.yaml"
+        framework = job.framework or "praisonai"
+        
+        # Check if we should use inline YAML
+        if job.agent_yaml:
+            # Write to temp file
+            import tempfile
+            with tempfile.NamedTemporaryFile(mode='w', suffix='.yaml', delete=False) as f:
+                f.write(job.agent_yaml)
+                agent_file = f.name
+        
+        # Update progress
+        job.update_progress(percentage=10.0, step="Initializing agent")
+        await self.store.save(job)
+        await self._notify_progress(job)
+        
+        try:
+            # Try praisonaiagents first (preferred)
+            if framework == "praisonai":
+                result = await self._run_praisonai_agents(job, agent_file)
+            else:
+                # Use legacy PraisonAI for crewai/autogen
+                result = await self._run_legacy_praisonai(job, agent_file, framework)
+            
+            return result
+            
+        finally:
+            # Clean up temp file if created
+            if job.agent_yaml and agent_file != "agents.yaml":
+                try:
+                    os.unlink(agent_file)
+                except Exception:
+                    pass
+    
+    async def _run_praisonai_agents(self, job: Job, agent_file: str) -> Any:
+        """Run using praisonaiagents framework."""
+        try:
+            from praisonaiagents import Agent
+        except ImportError:
+            raise RuntimeError("praisonaiagents not installed")
+        
+        # Update progress
+        job.update_progress(percentage=20.0, step="Creating agent")
+        await self.store.save(job)
+        await self._notify_progress(job)
+        
+        # Create agent
+        agent = Agent(
+            instructions="You are a helpful AI assistant.",
+            verbose=False
+        )
+        
+        # Update job with agent info
+        job.agent_id = getattr(agent, 'name', 'agent')
+        job.run_id = getattr(agent, 'run_id', None)
+        
+        # Update progress
+        job.update_progress(percentage=30.0, step="Running agent")
+        await self.store.save(job)
+        await self._notify_progress(job)
+        
+        # Run the agent
+        loop = asyncio.get_event_loop()
+        result = await loop.run_in_executor(
+            None,
+            lambda: agent.start(job.prompt)
+        )
+        
+        # Update progress
+        job.update_progress(percentage=90.0, step="Finalizing")
+        await self.store.save(job)
+        await self._notify_progress(job)
+        
+        return result
+    
+    async def _run_legacy_praisonai(self, job: Job, agent_file: str, framework: str) -> Any:
+        """Run using legacy PraisonAI (crewai/autogen)."""
+        try:
+            from praisonai import PraisonAI
+        except ImportError:
+            raise RuntimeError("praisonai not installed")
+        
+        # Update progress
+        job.update_progress(percentage=20.0, step="Creating PraisonAI instance")
+        await self.store.save(job)
+        await self._notify_progress(job)
+        
+        # Create PraisonAI instance
+        praisonai = PraisonAI(
+            agent_file=agent_file,
+            framework=framework
+        )
+        
+        # Update progress
+        job.update_progress(percentage=30.0, step="Running agents")
+        await self.store.save(job)
+        await self._notify_progress(job)
+        
+        # Run in executor (blocking call)
+        loop = asyncio.get_event_loop()
+        result = await loop.run_in_executor(None, praisonai.run)
+        
+        # Update progress
+        job.update_progress(percentage=90.0, step="Finalizing")
+        await self.store.save(job)
+        await self._notify_progress(job)
+        
+        return result
+    
+    async def _notify_progress(self, job: Job):
+        """Notify progress callback if registered."""
+        callback = self._progress_callbacks.get(job.id)
+        if callback:
+            try:
+                if asyncio.iscoroutinefunction(callback):
+                    await callback(job)
+                else:
+                    callback(job)
+            except Exception as e:
+                logger.warning(f"Progress callback error for {job.id}: {e}")
+    
+    async def _send_webhook(self, job: Job):
+        """Send webhook notification for job completion."""
+        if not job.webhook_url:
+            return
+        
+        try:
+            import httpx
+            
+            payload = {
+                "job_id": job.id,
+                "status": job.status.value,
+                "result": job.result if job.status == JobStatus.SUCCEEDED else None,
+                "error": job.error if job.status == JobStatus.FAILED else None,
+                "completed_at": job.completed_at.isoformat() if job.completed_at else None,
+                "duration_seconds": job.duration_seconds
+            }
+            
+            async with httpx.AsyncClient(timeout=30.0) as client:
+                response = await client.post(
+                    job.webhook_url,
+                    json=payload,
+                    headers={"Content-Type": "application/json"}
+                )
+                
+                if response.status_code >= 400:
+                    logger.warning(f"Webhook failed for {job.id}: {response.status_code}")
+                else:
+                    logger.info(f"Webhook sent for {job.id}")
+                    
+        except Exception as e:
+            logger.error(f"Webhook error for {job.id}: {e}")
+    
+    def get_stats(self) -> Dict[str, Any]:
+        """Get executor statistics."""
+        return {
+            "running_jobs": len(self._running_tasks),
+            "max_concurrent": self.max_concurrent,
+            "default_timeout": self.default_timeout,
+            "shutdown": self._shutdown
+        }

src/praisonai/praisonai/jobs/models.py

@@ -0,0 +1,224 @@
+"""
+Job Models for PraisonAI Async Jobs API.
+
+Defines the data structures for job submission, status, and results.
+"""
+
+import uuid
+from enum import Enum
+from typing import Optional, Dict, Any, List
+from datetime import datetime
+from pydantic import BaseModel, Field
+
+
+class JobStatus(str, Enum):
+    """Status of a job."""
+    QUEUED = "queued"
+    RUNNING = "running"
+    SUCCEEDED = "succeeded"
+    FAILED = "failed"
+    CANCELLED = "cancelled"
+
+
+class JobSubmitRequest(BaseModel):
+    """Request body for submitting a new job."""
+    prompt: str = Field(..., description="The prompt or task for the agent")
+    agent_file: Optional[str] = Field(None, description="Path to agents.yaml file")
+    agent_yaml: Optional[str] = Field(None, description="Inline agent YAML configuration")
+    framework: Optional[str] = Field("praisonai", description="Framework to use (praisonai, crewai, autogen)")
+    config: Optional[Dict[str, Any]] = Field(default_factory=dict, description="Additional configuration")
+    webhook_url: Optional[str] = Field(None, description="URL to POST results when complete")
+    timeout: Optional[int] = Field(3600, description="Timeout in seconds (default: 1 hour)")
+    session_id: Optional[str] = Field(None, description="Session ID for conversation continuity")
+    idempotency_key: Optional[str] = Field(None, description="Idempotency key to prevent duplicate submissions")
+
+
+class JobSubmitResponse(BaseModel):
+    """Response after submitting a job."""
+    job_id: str = Field(..., description="Unique job identifier")
+    status: JobStatus = Field(..., description="Current job status")
+    created_at: datetime = Field(..., description="Job creation timestamp")
+    poll_url: str = Field(..., description="URL to poll for status")
+    stream_url: str = Field(..., description="URL for SSE streaming")
+
+
+class JobProgress(BaseModel):
+    """Progress information for a running job."""
+    percentage: float = Field(0.0, ge=0.0, le=100.0, description="Completion percentage")
+    current_step: Optional[str] = Field(None, description="Current step description")
+    steps_completed: int = Field(0, description="Number of steps completed")
+    steps_total: Optional[int] = Field(None, description="Total number of steps")
+
+
+class JobStatusResponse(BaseModel):
+    """Response for job status query."""
+    job_id: str = Field(..., description="Unique job identifier")
+    status: JobStatus = Field(..., description="Current job status")
+    progress: JobProgress = Field(default_factory=JobProgress, description="Progress information")
+    created_at: datetime = Field(..., description="Job creation timestamp")
+    started_at: Optional[datetime] = Field(None, description="When job started running")
+    completed_at: Optional[datetime] = Field(None, description="When job completed")
+    agent_id: Optional[str] = Field(None, description="Current agent ID")
+    run_id: Optional[str] = Field(None, description="Run ID for tracing")
+    session_id: Optional[str] = Field(None, description="Session ID")
+    error: Optional[str] = Field(None, description="Error message if failed")
+    retry_after: Optional[int] = Field(None, description="Suggested seconds before next poll")
+
+
+class JobResultResponse(BaseModel):
+    """Response for job result query."""
+    job_id: str = Field(..., description="Unique job identifier")
+    status: JobStatus = Field(..., description="Final job status")
+    result: Optional[Any] = Field(None, description="Job result/output")
+    result_url: Optional[str] = Field(None, description="URL to fetch large results")
+    metrics: Optional[Dict[str, Any]] = Field(None, description="Execution metrics")
+    created_at: datetime = Field(..., description="Job creation timestamp")
+    started_at: Optional[datetime] = Field(None, description="When job started running")
+    completed_at: Optional[datetime] = Field(None, description="When job completed")
+    duration_seconds: Optional[float] = Field(None, description="Total execution time")
+    error: Optional[str] = Field(None, description="Error message if failed")
+
+
+class JobListResponse(BaseModel):
+    """Response for listing jobs."""
+    jobs: List[JobStatusResponse] = Field(..., description="List of jobs")
+    total: int = Field(..., description="Total number of jobs matching filter")
+    page: int = Field(1, description="Current page number")
+    page_size: int = Field(20, description="Number of jobs per page")
+
+
+class Job(BaseModel):
+    """Internal job representation with full state."""
+    id: str = Field(default_factory=lambda: f"run_{uuid.uuid4().hex[:12]}")
+    status: JobStatus = Field(default=JobStatus.QUEUED)
+    
+    # Request data
+    prompt: str = Field(...)
+    agent_file: Optional[str] = Field(None)
+    agent_yaml: Optional[str] = Field(None)
+    framework: str = Field("praisonai")
+    config: Dict[str, Any] = Field(default_factory=dict)
+    webhook_url: Optional[str] = Field(None)
+    timeout: int = Field(3600)
+    session_id: Optional[str] = Field(None)
+    idempotency_key: Optional[str] = Field(None)
+    
+    # Progress tracking
+    progress_percentage: float = Field(0.0)
+    progress_step: Optional[str] = Field(None)
+    steps_completed: int = Field(0)
+    steps_total: Optional[int] = Field(None)
+    
+    # Attribution
+    agent_id: Optional[str] = Field(None)
+    run_id: Optional[str] = Field(None)
+    
+    # Timestamps
+    created_at: datetime = Field(default_factory=datetime.utcnow)
+    started_at: Optional[datetime] = Field(None)
+    completed_at: Optional[datetime] = Field(None)
+    
+    # Result
+    result: Optional[Any] = Field(None)
+    error: Optional[str] = Field(None)
+    metrics: Optional[Dict[str, Any]] = Field(None)
+    
+    # Internal
+    _cancel_requested: bool = False
+    
+    @property
+    def duration_seconds(self) -> Optional[float]:
+        """Calculate job duration."""
+        if self.started_at is None:
+            return None
+        end_time = self.completed_at or datetime.utcnow()
+        return (end_time - self.started_at).total_seconds()
+    
+    @property
+    def is_terminal(self) -> bool:
+        """Check if job is in a terminal state."""
+        return self.status in (JobStatus.SUCCEEDED, JobStatus.FAILED, JobStatus.CANCELLED)
+    
+    def to_status_response(self, base_url: str = "") -> JobStatusResponse:
+        """Convert to status response."""
+        retry_after = None
+        if self.status == JobStatus.QUEUED:
+            retry_after = 2
+        elif self.status == JobStatus.RUNNING:
+            retry_after = 5
+        
+        return JobStatusResponse(
+            job_id=self.id,
+            status=self.status,
+            progress=JobProgress(
+                percentage=self.progress_percentage,
+                current_step=self.progress_step,
+                steps_completed=self.steps_completed,
+                steps_total=self.steps_total
+            ),
+            created_at=self.created_at,
+            started_at=self.started_at,
+            completed_at=self.completed_at,
+            agent_id=self.agent_id,
+            run_id=self.run_id,
+            session_id=self.session_id,
+            error=self.error,
+            retry_after=retry_after
+        )
+    
+    def to_result_response(self) -> JobResultResponse:
+        """Convert to result response."""
+        return JobResultResponse(
+            job_id=self.id,
+            status=self.status,
+            result=self.result,
+            result_url=None,  # For large results, would be set
+            metrics=self.metrics,
+            created_at=self.created_at,
+            started_at=self.started_at,
+            completed_at=self.completed_at,
+            duration_seconds=self.duration_seconds,
+            error=self.error
+        )
+    
+    def start(self):
+        """Mark job as started."""
+        self.status = JobStatus.RUNNING
+        self.started_at = datetime.utcnow()
+    
+    def succeed(self, result: Any, metrics: Optional[Dict[str, Any]] = None):
+        """Mark job as succeeded."""
+        self.status = JobStatus.SUCCEEDED
+        self.result = result
+        self.metrics = metrics
+        self.completed_at = datetime.utcnow()
+        self.progress_percentage = 100.0
+    
+    def fail(self, error: str):
+        """Mark job as failed."""
+        self.status = JobStatus.FAILED
+        self.error = error
+        self.completed_at = datetime.utcnow()
+    
+    def cancel(self):
+        """Mark job as cancelled."""
+        self.status = JobStatus.CANCELLED
+        self.completed_at = datetime.utcnow()
+        self._cancel_requested = True
+    
+    def update_progress(
+        self,
+        percentage: Optional[float] = None,
+        step: Optional[str] = None,
+        steps_completed: Optional[int] = None,
+        steps_total: Optional[int] = None
+    ):
+        """Update job progress."""
+        if percentage is not None:
+            self.progress_percentage = min(max(percentage, 0.0), 100.0)
+        if step is not None:
+            self.progress_step = step
+        if steps_completed is not None:
+            self.steps_completed = steps_completed
+        if steps_total is not None:
+            self.steps_total = steps_total

src/praisonai/praisonai/jobs/router.py

@@ -0,0 +1,314 @@
+"""
+FastAPI Router for PraisonAI Async Jobs API.
+
+Provides HTTP endpoints for job management.
+"""
+
+import asyncio
+import logging
+from typing import Optional
+from datetime import datetime
+
+from fastapi import APIRouter, HTTPException, Header, Request, Query, Response
+from sse_starlette.sse import EventSourceResponse
+
+from .models import (
+    Job,
+    JobStatus,
+    JobSubmitRequest,
+    JobSubmitResponse,
+    JobStatusResponse,
+    JobResultResponse,
+    JobListResponse
+)
+from .store import JobStore
+from .executor import JobExecutor
+
+logger = logging.getLogger(__name__)
+
+
+def create_router(store: JobStore, executor: JobExecutor) -> APIRouter:
+    """
+    Create the jobs API router.
+    
+    Args:
+        store: Job storage backend
+        executor: Job executor
+        
+    Returns:
+        FastAPI router with all endpoints
+    """
+    router = APIRouter(prefix="/api/v1/runs", tags=["jobs"])
+    
+    @router.post("", response_model=JobSubmitResponse, status_code=202)
+    async def submit_job(
+        request: Request,
+        response: Response,
+        body: JobSubmitRequest,
+        idempotency_key: Optional[str] = Header(None, alias="Idempotency-Key")
+    ):
+        """
+        Submit a new job for execution.
+        
+        Returns 202 Accepted with job ID and status URLs.
+        Headers returned:
+        - Location: URL to poll for status
+        - Retry-After: Suggested seconds before first poll (default: 2)
+        
+        If an Idempotency-Key header is provided and a job with that key
+        already exists, returns the existing job instead of creating a new one.
+        """
+        # Check idempotency key
+        effective_key = idempotency_key or body.idempotency_key
+        if effective_key:
+            existing = await store.get_by_idempotency_key(effective_key)
+            if existing:
+                logger.info(f"Returning existing job for idempotency key: {effective_key}")
+                base_url = str(request.base_url).rstrip("/")
+                poll_url = f"{base_url}/api/v1/runs/{existing.id}"
+                response.headers["Location"] = poll_url
+                response.headers["Retry-After"] = "2"
+                return JobSubmitResponse(
+                    job_id=existing.id,
+                    status=existing.status,
+                    created_at=existing.created_at,
+                    poll_url=poll_url,
+                    stream_url=f"{base_url}/api/v1/runs/{existing.id}/stream"
+                )
+        
+        # Create new job
+        job = Job(
+            prompt=body.prompt,
+            agent_file=body.agent_file,
+            agent_yaml=body.agent_yaml,
+            framework=body.framework or "praisonai",
+            config=body.config or {},
+            webhook_url=body.webhook_url,
+            timeout=body.timeout or 3600,
+            session_id=body.session_id,
+            idempotency_key=effective_key
+        )
+        
+        # Submit for execution
+        await executor.submit(job)
+        
+        # Build response with Location and Retry-After headers (RFC best practice)
+        base_url = str(request.base_url).rstrip("/")
+        poll_url = f"{base_url}/api/v1/runs/{job.id}"
+        response.headers["Location"] = poll_url
+        response.headers["Retry-After"] = "2"
+        
+        return JobSubmitResponse(
+            job_id=job.id,
+            status=job.status,
+            created_at=job.created_at,
+            poll_url=poll_url,
+            stream_url=f"{base_url}/api/v1/runs/{job.id}/stream"
+        )
+    
+    @router.get("", response_model=JobListResponse)
+    async def list_jobs(
+        status: Optional[str] = Query(None, description="Filter by status"),
+        session_id: Optional[str] = Query(None, description="Filter by session ID"),
+        page: int = Query(1, ge=1, description="Page number"),
+        page_size: int = Query(20, ge=1, le=100, description="Jobs per page")
+    ):
+        """
+        List jobs with optional filters.
+        """
+        # Parse status filter
+        status_filter = None
+        if status:
+            try:
+                status_filter = JobStatus(status)
+            except ValueError:
+                raise HTTPException(
+                    status_code=400,
+                    detail=f"Invalid status: {status}. Valid values: {[s.value for s in JobStatus]}"
+                )
+        
+        # Get jobs
+        offset = (page - 1) * page_size
+        jobs = await store.list_jobs(
+            status=status_filter,
+            session_id=session_id,
+            limit=page_size,
+            offset=offset
+        )
+        
+        total = await store.count(status=status_filter, session_id=session_id)
+        
+        return JobListResponse(
+            jobs=[job.to_status_response() for job in jobs],
+            total=total,
+            page=page,
+            page_size=page_size
+        )
+    
+    @router.get("/{job_id}", response_model=JobStatusResponse)
+    async def get_job_status(job_id: str):
+        """
+        Get the status of a job.
+        
+        Returns current status, progress, and timing information.
+        """
+        job = await store.get(job_id)
+        if not job:
+            raise HTTPException(status_code=404, detail=f"Job not found: {job_id}")
+        
+        return job.to_status_response()
+    
+    @router.get("/{job_id}/result", response_model=JobResultResponse)
+    async def get_job_result(job_id: str):
+        """
+        Get the result of a completed job.
+        
+        Returns 409 Conflict if job is not yet complete.
+        """
+        job = await store.get(job_id)
+        if not job:
+            raise HTTPException(status_code=404, detail=f"Job not found: {job_id}")
+        
+        if not job.is_terminal:
+            raise HTTPException(
+                status_code=409,
+                detail=f"Job is not complete. Current status: {job.status.value}"
+            )
+        
+        return job.to_result_response()
+    
+    @router.post("/{job_id}/cancel", response_model=JobStatusResponse)
+    async def cancel_job(job_id: str):
+        """
+        Cancel a running job.
+        
+        Returns 409 Conflict if job is already complete.
+        """
+        job = await store.get(job_id)
+        if not job:
+            raise HTTPException(status_code=404, detail=f"Job not found: {job_id}")
+        
+        if job.is_terminal:
+            raise HTTPException(
+                status_code=409,
+                detail=f"Job is already complete. Status: {job.status.value}"
+            )
+        
+        success = await executor.cancel(job_id)
+        if not success:
+            raise HTTPException(status_code=500, detail="Failed to cancel job")
+        
+        # Refresh job state
+        job = await store.get(job_id)
+        return job.to_status_response()
+    
+    @router.delete("/{job_id}", status_code=204)
+    async def delete_job(job_id: str):
+        """
+        Delete a job.
+        
+        Only completed jobs can be deleted.
+        """
+        job = await store.get(job_id)
+        if not job:
+            raise HTTPException(status_code=404, detail=f"Job not found: {job_id}")
+        
+        if not job.is_terminal:
+            raise HTTPException(
+                status_code=409,
+                detail="Cannot delete a running job. Cancel it first."
+            )
+        
+        await store.delete(job_id)
+    
+    @router.get("/{job_id}/stream")
+    async def stream_job(job_id: str):
+        """
+        Stream job progress via Server-Sent Events (SSE).
+        
+        Events:
+        - status: Job status updates
+        - progress: Progress percentage updates
+        - result: Final result (when complete)
+        - error: Error message (when failed)
+        """
+        job = await store.get(job_id)
+        if not job:
+            raise HTTPException(status_code=404, detail=f"Job not found: {job_id}")
+        
+        async def event_generator():
+            """Generate SSE events for job progress."""
+            last_status = None
+            last_progress = -1
+            
+            # Register for progress updates
+            progress_queue = asyncio.Queue()
+            
+            async def on_progress(updated_job: Job):
+                await progress_queue.put(updated_job)
+            
+            executor.register_progress_callback(job_id, on_progress)
+            
+            try:
+                while True:
+                    # Get current job state
+                    current_job = await store.get(job_id)
+                    if not current_job:
+                        yield {
+                            "event": "error",
+                            "data": '{"error": "Job not found"}'
+                        }
+                        break
+                    
+                    # Send status update if changed
+                    if current_job.status != last_status:
+                        last_status = current_job.status
+                        yield {
+                            "event": "status",
+                            "data": f'{{"status": "{current_job.status.value}", "job_id": "{job_id}"}}'
+                        }
+                    
+                    # Send progress update if changed
+                    if current_job.progress_percentage != last_progress:
+                        last_progress = current_job.progress_percentage
+                        yield {
+                            "event": "progress",
+                            "data": f'{{"percentage": {current_job.progress_percentage}, "step": "{current_job.progress_step or ""}"}}'
+                        }
+                    
+                    # Check if complete
+                    if current_job.is_terminal:
+                        if current_job.status == JobStatus.SUCCEEDED:
+                            result_str = str(current_job.result).replace('"', '\\"')[:1000]
+                            yield {
+                                "event": "result",
+                                "data": f'{{"result": "{result_str}"}}'
+                            }
+                        elif current_job.status == JobStatus.FAILED:
+                            error_str = (current_job.error or "Unknown error").replace('"', '\\"')
+                            yield {
+                                "event": "error",
+                                "data": f'{{"error": "{error_str}"}}'
+                            }
+                        elif current_job.status == JobStatus.CANCELLED:
+                            yield {
+                                "event": "cancelled",
+                                "data": '{"message": "Job was cancelled"}'
+                            }
+                        break
+                    
+                    # Wait for next update or timeout
+                    try:
+                        await asyncio.wait_for(progress_queue.get(), timeout=5.0)
+                    except asyncio.TimeoutError:
+                        # Send heartbeat
+                        yield {
+                            "event": "heartbeat",
+                            "data": f'{{"timestamp": "{datetime.utcnow().isoformat()}"}}'
+                        }
+            finally:
+                executor.unregister_progress_callback(job_id)
+        
+        return EventSourceResponse(event_generator())
+    
+    return router

src/praisonai/praisonai/jobs/server.py

@@ -0,0 +1,186 @@
+"""
+Server for PraisonAI Async Jobs API.
+
+Provides FastAPI application setup and server startup.
+"""
+
+import logging
+import os
+from typing import Optional
+from contextlib import asynccontextmanager
+
+from fastapi import FastAPI
+from fastapi.middleware.cors import CORSMiddleware
+
+from .store import InMemoryJobStore, JobStore
+from .executor import JobExecutor
+from .router import create_router
+
+logger = logging.getLogger(__name__)
+
+# Global instances (for single-process deployment)
+_store: Optional[JobStore] = None
+_executor: Optional[JobExecutor] = None
+
+
+def get_store() -> JobStore:
+    """Get or create the job store."""
+    global _store
+    if _store is None:
+        _store = InMemoryJobStore(max_jobs=1000)
+    return _store
+
+
+def get_executor() -> JobExecutor:
+    """Get or create the job executor."""
+    global _executor
+    if _executor is None:
+        _executor = JobExecutor(
+            store=get_store(),
+            max_concurrent=int(os.environ.get("PRAISONAI_MAX_CONCURRENT_JOBS", "10")),
+            default_timeout=int(os.environ.get("PRAISONAI_JOB_TIMEOUT", "3600"))
+        )
+    return _executor
+
+
+@asynccontextmanager
+async def lifespan(app: FastAPI):
+    """Application lifespan manager."""
+    executor = get_executor()
+    await executor.start()
+    logger.info("Jobs API server started")
+    
+    yield
+    
+    await executor.stop()
+    logger.info("Jobs API server stopped")
+
+
+def create_app(
+    store: Optional[JobStore] = None,
+    executor: Optional[JobExecutor] = None,
+    cors_origins: Optional[list] = None
+) -> FastAPI:
+    """
+    Create the FastAPI application.
+    
+    Args:
+        store: Optional custom job store
+        executor: Optional custom executor
+        cors_origins: Optional list of allowed CORS origins
+        
+    Returns:
+        FastAPI application
+    """
+    global _store, _executor
+    
+    if store:
+        _store = store
+    if executor:
+        _executor = executor
+    
+    app = FastAPI(
+        title="PraisonAI Jobs API",
+        description="Async Jobs API for long-running agent tasks",
+        version="1.0.0",
+        lifespan=lifespan
+    )
+    
+    # Add CORS middleware
+    origins = cors_origins or ["*"]
+    app.add_middleware(
+        CORSMiddleware,
+        allow_origins=origins,
+        allow_credentials=True,
+        allow_methods=["*"],
+        allow_headers=["*"],
+    )
+    
+    # Add jobs router
+    jobs_router = create_router(get_store(), get_executor())
+    app.include_router(jobs_router)
+    
+    # Health check endpoint
+    @app.get("/health")
+    async def health_check():
+        """Health check endpoint."""
+        return {
+            "status": "healthy",
+            "store": get_store().__class__.__name__,
+            "executor_stats": get_executor().get_stats()
+        }
+    
+    # Stats endpoint
+    @app.get("/stats")
+    async def get_stats():
+        """Get server statistics."""
+        store = get_store()
+        executor = get_executor()
+        
+        stats = {
+            "executor": executor.get_stats()
+        }
+        
+        if hasattr(store, "get_stats"):
+            stats["store"] = store.get_stats()
+        
+        return stats
+    
+    return app
+
+
+def start_server(
+    host: str = "127.0.0.1",
+    port: int = 8005,
+    reload: bool = False,
+    workers: int = 1,
+    log_level: str = "info"
+):
+    """
+    Start the jobs API server.
+    
+    Args:
+        host: Host to bind to
+        port: Port to bind to
+        reload: Enable auto-reload for development
+        workers: Number of worker processes
+        log_level: Logging level
+    """
+    try:
+        import uvicorn
+    except ImportError:
+        raise RuntimeError("uvicorn is required. Install with: pip install uvicorn")
+    
+    logger.info(f"Starting PraisonAI Jobs API on {host}:{port}")
+    
+    uvicorn.run(
+        "praisonai.jobs.server:create_app",
+        host=host,
+        port=port,
+        reload=reload,
+        workers=workers,
+        log_level=log_level,
+        factory=True
+    )
+
+
+# For direct module execution
+if __name__ == "__main__":
+    import argparse
+    
+    parser = argparse.ArgumentParser(description="PraisonAI Jobs API Server")
+    parser.add_argument("--host", default="127.0.0.1", help="Host to bind to")
+    parser.add_argument("--port", type=int, default=8005, help="Port to bind to")
+    parser.add_argument("--reload", action="store_true", help="Enable auto-reload")
+    parser.add_argument("--workers", type=int, default=1, help="Number of workers")
+    parser.add_argument("--log-level", default="info", help="Log level")
+    
+    args = parser.parse_args()
+    
+    start_server(
+        host=args.host,
+        port=args.port,
+        reload=args.reload,
+        workers=args.workers,
+        log_level=args.log_level
+    )

src/praisonai/praisonai/jobs/store.py

@@ -0,0 +1,203 @@
+"""
+Job Store for PraisonAI Async Jobs API.
+
+Provides storage backends for job state persistence.
+"""
+
+import asyncio
+import logging
+from abc import ABC, abstractmethod
+from typing import Optional, List, Dict, Any
+from datetime import datetime, timedelta
+
+from .models import Job, JobStatus
+
+logger = logging.getLogger(__name__)
+
+
+class JobStore(ABC):
+    """Abstract base class for job storage backends."""
+    
+    @abstractmethod
+    async def save(self, job: Job) -> None:
+        """Save or update a job."""
+        pass
+    
+    @abstractmethod
+    async def get(self, job_id: str) -> Optional[Job]:
+        """Get a job by ID."""
+        pass
+    
+    @abstractmethod
+    async def get_by_idempotency_key(self, key: str) -> Optional[Job]:
+        """Get a job by idempotency key."""
+        pass
+    
+    @abstractmethod
+    async def list_jobs(
+        self,
+        status: Optional[JobStatus] = None,
+        session_id: Optional[str] = None,
+        limit: int = 20,
+        offset: int = 0
+    ) -> List[Job]:
+        """List jobs with optional filters."""
+        pass
+    
+    @abstractmethod
+    async def count(
+        self,
+        status: Optional[JobStatus] = None,
+        session_id: Optional[str] = None
+    ) -> int:
+        """Count jobs matching filters."""
+        pass
+    
+    @abstractmethod
+    async def delete(self, job_id: str) -> bool:
+        """Delete a job by ID."""
+        pass
+    
+    @abstractmethod
+    async def cleanup_old_jobs(self, max_age_seconds: int = 86400) -> int:
+        """Remove jobs older than max_age_seconds. Returns count deleted."""
+        pass
+
+
+class InMemoryJobStore(JobStore):
+    """
+    In-memory job store.
+    
+    Suitable for development and single-process deployments.
+    Jobs are lost on restart.
+    """
+    
+    def __init__(self, max_jobs: int = 1000):
+        self._jobs: Dict[str, Job] = {}
+        self._idempotency_keys: Dict[str, str] = {}  # key -> job_id
+        self._max_jobs = max_jobs
+        self._lock = asyncio.Lock()
+    
+    async def save(self, job: Job) -> None:
+        """Save or update a job."""
+        async with self._lock:
+            self._jobs[job.id] = job
+            
+            # Track idempotency key
+            if job.idempotency_key:
+                self._idempotency_keys[job.idempotency_key] = job.id
+            
+            # Enforce max jobs limit by removing oldest completed
+            if len(self._jobs) > self._max_jobs:
+                await self._evict_oldest_completed()
+    
+    async def _evict_oldest_completed(self) -> None:
+        """Remove oldest completed jobs to stay under limit."""
+        completed = [
+            (job_id, job) for job_id, job in self._jobs.items()
+            if job.is_terminal
+        ]
+        
+        if not completed:
+            return
+        
+        # Sort by completed_at, oldest first
+        completed.sort(key=lambda x: x[1].completed_at or datetime.min)
+        
+        # Remove oldest 10%
+        to_remove = max(1, len(completed) // 10)
+        for job_id, job in completed[:to_remove]:
+            del self._jobs[job_id]
+            if job.idempotency_key:
+                self._idempotency_keys.pop(job.idempotency_key, None)
+    
+    async def get(self, job_id: str) -> Optional[Job]:
+        """Get a job by ID."""
+        return self._jobs.get(job_id)
+    
+    async def get_by_idempotency_key(self, key: str) -> Optional[Job]:
+        """Get a job by idempotency key."""
+        job_id = self._idempotency_keys.get(key)
+        if job_id:
+            return self._jobs.get(job_id)
+        return None
+    
+    async def list_jobs(
+        self,
+        status: Optional[JobStatus] = None,
+        session_id: Optional[str] = None,
+        limit: int = 20,
+        offset: int = 0
+    ) -> List[Job]:
+        """List jobs with optional filters."""
+        jobs = list(self._jobs.values())
+        
+        # Apply filters
+        if status:
+            jobs = [j for j in jobs if j.status == status]
+        if session_id:
+            jobs = [j for j in jobs if j.session_id == session_id]
+        
+        # Sort by created_at descending (newest first)
+        jobs.sort(key=lambda j: j.created_at, reverse=True)
+        
+        # Apply pagination
+        return jobs[offset:offset + limit]
+    
+    async def count(
+        self,
+        status: Optional[JobStatus] = None,
+        session_id: Optional[str] = None
+    ) -> int:
+        """Count jobs matching filters."""
+        jobs = list(self._jobs.values())
+        
+        if status:
+            jobs = [j for j in jobs if j.status == status]
+        if session_id:
+            jobs = [j for j in jobs if j.session_id == session_id]
+        
+        return len(jobs)
+    
+    async def delete(self, job_id: str) -> bool:
+        """Delete a job by ID."""
+        async with self._lock:
+            job = self._jobs.pop(job_id, None)
+            if job:
+                if job.idempotency_key:
+                    self._idempotency_keys.pop(job.idempotency_key, None)
+                return True
+            return False
+    
+    async def cleanup_old_jobs(self, max_age_seconds: int = 86400) -> int:
+        """Remove jobs older than max_age_seconds."""
+        async with self._lock:
+            cutoff = datetime.utcnow() - timedelta(seconds=max_age_seconds)
+            to_remove = []
+            
+            for job_id, job in self._jobs.items():
+                if job.is_terminal and job.completed_at and job.completed_at < cutoff:
+                    to_remove.append(job_id)
+            
+            for job_id in to_remove:
+                job = self._jobs.pop(job_id)
+                if job.idempotency_key:
+                    self._idempotency_keys.pop(job.idempotency_key, None)
+            
+            if to_remove:
+                logger.info(f"Cleaned up {len(to_remove)} old jobs")
+            
+            return len(to_remove)
+    
+    def get_stats(self) -> Dict[str, Any]:
+        """Get store statistics."""
+        status_counts = {}
+        for job in self._jobs.values():
+            status_counts[job.status.value] = status_counts.get(job.status.value, 0) + 1
+        
+        return {
+            "total_jobs": len(self._jobs),
+            "idempotency_keys": len(self._idempotency_keys),
+            "max_jobs": self._max_jobs,
+            "status_counts": status_counts
+        }

src/praisonai/praisonai/persistence/conversation/async_mysql.py

@@ -0,0 +1,427 @@
+"""
+Async MySQL implementation of ConversationStore.
+
+Requires: aiomysql
+Install: pip install aiomysql
+"""
+
+import asyncio
+import json
+import logging
+import time
+from typing import List, Optional
+
+from .base import ConversationStore, ConversationSession, ConversationMessage
+
+logger = logging.getLogger(__name__)
+
+
+class AsyncMySQLConversationStore(ConversationStore):
+    """
+    Async MySQL conversation store using aiomysql.
+    
+    Provides high-performance async database operations.
+    
+    Example:
+        store = AsyncMySQLConversationStore(
+            host="localhost",
+            user="root",
+            password="password",
+            database="praisonai"
+        )
+        await store.init()
+    """
+    
+    def __init__(
+        self,
+        url: Optional[str] = None,
+        host: str = "localhost",
+        port: int = 3306,
+        database: str = "praisonai",
+        user: str = "root",
+        password: str = "",
+        table_prefix: str = "praisonai_",
+        pool_size: int = 10,
+    ):
+        """
+        Initialize async MySQL store.
+        
+        Args:
+            url: MySQL connection URL (takes precedence)
+            host: Database host
+            port: Database port
+            database: Database name
+            user: Database user
+            password: Database password
+            table_prefix: Prefix for table names
+            pool_size: Connection pool size
+        """
+        self.url = url
+        self.host = host
+        self.port = port
+        self.database = database
+        self.user = user
+        self.password = password
+        self.table_prefix = table_prefix
+        self.pool_size = pool_size
+        self._pool = None
+        self._initialized = False
+        
+        if url:
+            self._parse_url(url)
+    
+    def _parse_url(self, url: str):
+        """Parse MySQL URL into components."""
+        from urllib.parse import urlparse
+        parsed = urlparse(url)
+        self.host = parsed.hostname or "localhost"
+        self.port = parsed.port or 3306
+        self.database = parsed.path.lstrip('/') or "praisonai"
+        self.user = parsed.username or "root"
+        self.password = parsed.password or ""
+    
+    async def init(self):
+        """Initialize connection pool and create tables."""
+        if self._initialized:
+            return
+        
+        try:
+            import aiomysql
+        except ImportError:
+            raise ImportError(
+                "aiomysql is required for async MySQL support. "
+                "Install with: pip install aiomysql"
+            )
+        
+        self._pool = await aiomysql.create_pool(
+            host=self.host,
+            port=self.port,
+            user=self.user,
+            password=self.password,
+            db=self.database,
+            minsize=1,
+            maxsize=self.pool_size,
+            autocommit=True
+        )
+        
+        await self._create_tables()
+        self._initialized = True
+    
+    async def _create_tables(self):
+        """Create required tables if they don't exist."""
+        sessions_table = f"{self.table_prefix}sessions"
+        messages_table = f"{self.table_prefix}messages"
+        
+        async with self._pool.acquire() as conn:
+            async with conn.cursor() as cur:
+                await cur.execute(f"""
+                    CREATE TABLE IF NOT EXISTS {sessions_table} (
+                        session_id VARCHAR(255) PRIMARY KEY,
+                        user_id VARCHAR(255),
+                        agent_id VARCHAR(255),
+                        name VARCHAR(255),
+                        metadata JSON,
+                        created_at DOUBLE,
+                        updated_at DOUBLE,
+                        INDEX idx_user_id (user_id),
+                        INDEX idx_agent_id (agent_id)
+                    )
+                """)
+                
+                await cur.execute(f"""
+                    CREATE TABLE IF NOT EXISTS {messages_table} (
+                        id VARCHAR(255) PRIMARY KEY,
+                        session_id VARCHAR(255),
+                        role VARCHAR(50),
+                        content TEXT,
+                        metadata JSON,
+                        created_at DOUBLE,
+                        INDEX idx_session_id (session_id),
+                        FOREIGN KEY (session_id) REFERENCES {sessions_table}(session_id) ON DELETE CASCADE
+                    )
+                """)
+    
+    async def async_create_session(self, session: ConversationSession) -> ConversationSession:
+        """Create a new session asynchronously."""
+        if not self._initialized:
+            await self.init()
+        
+        table = f"{self.table_prefix}sessions"
+        async with self._pool.acquire() as conn:
+            async with conn.cursor() as cur:
+                await cur.execute(f"""
+                    INSERT INTO {table} (session_id, user_id, agent_id, name, metadata, created_at, updated_at)
+                    VALUES (%s, %s, %s, %s, %s, %s, %s)
+                """, (session.session_id, session.user_id, session.agent_id, session.name,
+                      json.dumps(session.metadata) if session.metadata else None,
+                      session.created_at, session.updated_at))
+        
+        return session
+    
+    def create_session(self, session: ConversationSession) -> ConversationSession:
+        """Sync wrapper for create_session."""
+        return asyncio.get_event_loop().run_until_complete(
+            self.async_create_session(session)
+        )
+    
+    async def async_get_session(self, session_id: str) -> Optional[ConversationSession]:
+        """Get a session by ID asynchronously."""
+        if not self._initialized:
+            await self.init()
+        
+        table = f"{self.table_prefix}sessions"
+        async with self._pool.acquire() as conn:
+            async with conn.cursor() as cur:
+                await cur.execute(f"""
+                    SELECT session_id, user_id, agent_id, name, metadata, created_at, updated_at
+                    FROM {table} WHERE session_id = %s
+                """, (session_id,))
+                row = await cur.fetchone()
+        
+        if row:
+            return ConversationSession(
+                session_id=row[0],
+                user_id=row[1],
+                agent_id=row[2],
+                name=row[3],
+                metadata=json.loads(row[4]) if row[4] else None,
+                created_at=row[5],
+                updated_at=row[6]
+            )
+        return None
+    
+    def get_session(self, session_id: str) -> Optional[ConversationSession]:
+        """Sync wrapper for get_session."""
+        return asyncio.get_event_loop().run_until_complete(
+            self.async_get_session(session_id)
+        )
+    
+    async def async_update_session(self, session: ConversationSession) -> ConversationSession:
+        """Update an existing session asynchronously."""
+        if not self._initialized:
+            await self.init()
+        
+        session.updated_at = time.time()
+        table = f"{self.table_prefix}sessions"
+        
+        async with self._pool.acquire() as conn:
+            async with conn.cursor() as cur:
+                await cur.execute(f"""
+                    UPDATE {table} 
+                    SET name = %s, metadata = %s, updated_at = %s
+                    WHERE session_id = %s
+                """, (session.name, json.dumps(session.metadata) if session.metadata else None,
+                      session.updated_at, session.session_id))
+        
+        return session
+    
+    def update_session(self, session: ConversationSession) -> ConversationSession:
+        """Sync wrapper for update_session."""
+        return asyncio.get_event_loop().run_until_complete(
+            self.async_update_session(session)
+        )
+    
+    async def async_delete_session(self, session_id: str) -> bool:
+        """Delete a session asynchronously."""
+        if not self._initialized:
+            await self.init()
+        
+        table = f"{self.table_prefix}sessions"
+        async with self._pool.acquire() as conn:
+            async with conn.cursor() as cur:
+                await cur.execute(f"""
+                    DELETE FROM {table} WHERE session_id = %s
+                """, (session_id,))
+                return cur.rowcount > 0
+    
+    def delete_session(self, session_id: str) -> bool:
+        """Sync wrapper for delete_session."""
+        return asyncio.get_event_loop().run_until_complete(
+            self.async_delete_session(session_id)
+        )
+    
+    async def async_list_sessions(
+        self,
+        user_id: Optional[str] = None,
+        agent_id: Optional[str] = None,
+        limit: int = 100,
+        offset: int = 0
+    ) -> List[ConversationSession]:
+        """List sessions asynchronously."""
+        if not self._initialized:
+            await self.init()
+        
+        table = f"{self.table_prefix}sessions"
+        conditions = []
+        params = []
+        
+        if user_id:
+            conditions.append("user_id = %s")
+            params.append(user_id)
+        
+        if agent_id:
+            conditions.append("agent_id = %s")
+            params.append(agent_id)
+        
+        where_clause = f"WHERE {' AND '.join(conditions)}" if conditions else ""
+        params.extend([limit, offset])
+        
+        async with self._pool.acquire() as conn:
+            async with conn.cursor() as cur:
+                await cur.execute(f"""
+                    SELECT session_id, user_id, agent_id, name, metadata, created_at, updated_at
+                    FROM {table} {where_clause}
+                    ORDER BY updated_at DESC
+                    LIMIT %s OFFSET %s
+                """, params)
+                rows = await cur.fetchall()
+        
+        return [
+            ConversationSession(
+                session_id=row[0],
+                user_id=row[1],
+                agent_id=row[2],
+                name=row[3],
+                metadata=json.loads(row[4]) if row[4] else None,
+                created_at=row[5],
+                updated_at=row[6]
+            )
+            for row in rows
+        ]
+    
+    def list_sessions(
+        self,
+        user_id: Optional[str] = None,
+        agent_id: Optional[str] = None,
+        limit: int = 100,
+        offset: int = 0
+    ) -> List[ConversationSession]:
+        """Sync wrapper for list_sessions."""
+        return asyncio.get_event_loop().run_until_complete(
+            self.async_list_sessions(user_id, agent_id, limit, offset)
+        )
+    
+    async def async_add_message(self, session_id: str, message: ConversationMessage) -> ConversationMessage:
+        """Add a message asynchronously."""
+        if not self._initialized:
+            await self.init()
+        
+        message.session_id = session_id
+        table = f"{self.table_prefix}messages"
+        
+        async with self._pool.acquire() as conn:
+            async with conn.cursor() as cur:
+                await cur.execute(f"""
+                    INSERT INTO {table} (id, session_id, role, content, metadata, created_at)
+                    VALUES (%s, %s, %s, %s, %s, %s)
+                """, (message.id, session_id, message.role, message.content,
+                      json.dumps(message.metadata) if message.metadata else None,
+                      message.created_at))
+        
+        return message
+    
+    def add_message(self, session_id: str, message: ConversationMessage) -> ConversationMessage:
+        """Sync wrapper for add_message."""
+        return asyncio.get_event_loop().run_until_complete(
+            self.async_add_message(session_id, message)
+        )
+    
+    async def async_get_messages(
+        self,
+        session_id: str,
+        limit: Optional[int] = None,
+        before: Optional[float] = None,
+        after: Optional[float] = None
+    ) -> List[ConversationMessage]:
+        """Get messages asynchronously."""
+        if not self._initialized:
+            await self.init()
+        
+        table = f"{self.table_prefix}messages"
+        conditions = ["session_id = %s"]
+        params = [session_id]
+        
+        if before:
+            conditions.append("created_at < %s")
+            params.append(before)
+        
+        if after:
+            conditions.append("created_at > %s")
+            params.append(after)
+        
+        where_clause = f"WHERE {' AND '.join(conditions)}"
+        limit_clause = "LIMIT %s" if limit else ""
+        if limit:
+            params.append(limit)
+        
+        async with self._pool.acquire() as conn:
+            async with conn.cursor() as cur:
+                await cur.execute(f"""
+                    SELECT id, session_id, role, content, metadata, created_at
+                    FROM {table} {where_clause}
+                    ORDER BY created_at ASC {limit_clause}
+                """, params)
+                rows = await cur.fetchall()
+        
+        return [
+            ConversationMessage(
+                id=row[0],
+                session_id=row[1],
+                role=row[2],
+                content=row[3],
+                metadata=json.loads(row[4]) if row[4] else None,
+                created_at=row[5]
+            )
+            for row in rows
+        ]
+    
+    def get_messages(
+        self,
+        session_id: str,
+        limit: Optional[int] = None,
+        before: Optional[float] = None,
+        after: Optional[float] = None
+    ) -> List[ConversationMessage]:
+        """Sync wrapper for get_messages."""
+        return asyncio.get_event_loop().run_until_complete(
+            self.async_get_messages(session_id, limit, before, after)
+        )
+    
+    async def async_delete_messages(self, session_id: str, message_ids: Optional[List[str]] = None) -> int:
+        """Delete messages asynchronously."""
+        if not self._initialized:
+            await self.init()
+        
+        table = f"{self.table_prefix}messages"
+        
+        async with self._pool.acquire() as conn:
+            async with conn.cursor() as cur:
+                if message_ids is None:
+                    await cur.execute(f"""
+                        DELETE FROM {table} WHERE session_id = %s
+                    """, (session_id,))
+                else:
+                    placeholders = ','.join(['%s'] * len(message_ids))
+                    await cur.execute(f"""
+                        DELETE FROM {table} WHERE session_id = %s AND id IN ({placeholders})
+                    """, [session_id] + message_ids)
+                return cur.rowcount
+    
+    def delete_messages(self, session_id: str, message_ids: Optional[List[str]] = None) -> int:
+        """Sync wrapper for delete_messages."""
+        return asyncio.get_event_loop().run_until_complete(
+            self.async_delete_messages(session_id, message_ids)
+        )
+    
+    async def async_close(self) -> None:
+        """Close the connection pool."""
+        if self._pool:
+            self._pool.close()
+            await self._pool.wait_closed()
+            self._pool = None
+            self._initialized = False
+    
+    def close(self) -> None:
+        """Sync wrapper for close."""
+        if self._pool:
+            asyncio.get_event_loop().run_until_complete(self.async_close())

src/praisonai/praisonai/persistence/conversation/async_postgres.py

@@ -0,0 +1,410 @@
+"""
+Async PostgreSQL implementation of ConversationStore.
+
+Requires: asyncpg
+Install: pip install asyncpg
+"""
+
+import asyncio
+import json
+import logging
+import time
+from typing import List, Optional
+
+from .base import ConversationStore, ConversationSession, ConversationMessage
+
+logger = logging.getLogger(__name__)
+
+
+class AsyncPostgresConversationStore(ConversationStore):
+    """
+    Async PostgreSQL conversation store using asyncpg.
+    
+    Provides high-performance async database operations.
+    
+    Example:
+        store = AsyncPostgresConversationStore(
+            url="postgresql://user:pass@localhost:5432/praisonai"
+        )
+        await store.init()
+    """
+    
+    def __init__(
+        self,
+        url: Optional[str] = None,
+        host: str = "localhost",
+        port: int = 5432,
+        database: str = "praisonai",
+        user: str = "postgres",
+        password: str = "",
+        table_prefix: str = "praisonai_",
+        pool_size: int = 10,
+    ):
+        """
+        Initialize async PostgreSQL store.
+        
+        Args:
+            url: PostgreSQL connection URL (takes precedence)
+            host: Database host
+            port: Database port
+            database: Database name
+            user: Database user
+            password: Database password
+            table_prefix: Prefix for table names
+            pool_size: Connection pool size
+        """
+        self.url = url
+        self.host = host
+        self.port = port
+        self.database = database
+        self.user = user
+        self.password = password
+        self.table_prefix = table_prefix
+        self.pool_size = pool_size
+        self._pool = None
+        self._initialized = False
+    
+    async def init(self):
+        """Initialize connection pool and create tables."""
+        if self._initialized:
+            return
+        
+        try:
+            import asyncpg
+        except ImportError:
+            raise ImportError(
+                "asyncpg is required for async PostgreSQL support. "
+                "Install with: pip install asyncpg"
+            )
+        
+        if self.url:
+            self._pool = await asyncpg.create_pool(self.url, min_size=1, max_size=self.pool_size)
+        else:
+            self._pool = await asyncpg.create_pool(
+                host=self.host,
+                port=self.port,
+                database=self.database,
+                user=self.user,
+                password=self.password,
+                min_size=1,
+                max_size=self.pool_size
+            )
+        
+        await self._create_tables()
+        self._initialized = True
+    
+    async def _create_tables(self):
+        """Create required tables if they don't exist."""
+        sessions_table = f"{self.table_prefix}sessions"
+        messages_table = f"{self.table_prefix}messages"
+        
+        async with self._pool.acquire() as conn:
+            await conn.execute(f"""
+                CREATE TABLE IF NOT EXISTS {sessions_table} (
+                    session_id VARCHAR(255) PRIMARY KEY,
+                    user_id VARCHAR(255),
+                    agent_id VARCHAR(255),
+                    name VARCHAR(255),
+                    metadata JSONB,
+                    created_at DOUBLE PRECISION,
+                    updated_at DOUBLE PRECISION
+                )
+            """)
+            
+            await conn.execute(f"""
+                CREATE TABLE IF NOT EXISTS {messages_table} (
+                    id VARCHAR(255) PRIMARY KEY,
+                    session_id VARCHAR(255) REFERENCES {sessions_table}(session_id) ON DELETE CASCADE,
+                    role VARCHAR(50),
+                    content TEXT,
+                    metadata JSONB,
+                    created_at DOUBLE PRECISION
+                )
+            """)
+            
+            await conn.execute(f"""
+                CREATE INDEX IF NOT EXISTS idx_{messages_table}_session 
+                ON {messages_table}(session_id)
+            """)
+    
+    async def async_create_session(self, session: ConversationSession) -> ConversationSession:
+        """Create a new session asynchronously."""
+        if not self._initialized:
+            await self.init()
+        
+        table = f"{self.table_prefix}sessions"
+        async with self._pool.acquire() as conn:
+            await conn.execute(f"""
+                INSERT INTO {table} (session_id, user_id, agent_id, name, metadata, created_at, updated_at)
+                VALUES ($1, $2, $3, $4, $5, $6, $7)
+            """, session.session_id, session.user_id, session.agent_id, session.name,
+                json.dumps(session.metadata) if session.metadata else None,
+                session.created_at, session.updated_at)
+        
+        return session
+    
+    def create_session(self, session: ConversationSession) -> ConversationSession:
+        """Sync wrapper for create_session."""
+        return asyncio.get_event_loop().run_until_complete(
+            self.async_create_session(session)
+        )
+    
+    async def async_get_session(self, session_id: str) -> Optional[ConversationSession]:
+        """Get a session by ID asynchronously."""
+        if not self._initialized:
+            await self.init()
+        
+        table = f"{self.table_prefix}sessions"
+        async with self._pool.acquire() as conn:
+            row = await conn.fetchrow(f"""
+                SELECT * FROM {table} WHERE session_id = $1
+            """, session_id)
+        
+        if row:
+            return ConversationSession(
+                session_id=row['session_id'],
+                user_id=row['user_id'],
+                agent_id=row['agent_id'],
+                name=row['name'],
+                metadata=json.loads(row['metadata']) if row['metadata'] else None,
+                created_at=row['created_at'],
+                updated_at=row['updated_at']
+            )
+        return None
+    
+    def get_session(self, session_id: str) -> Optional[ConversationSession]:
+        """Sync wrapper for get_session."""
+        return asyncio.get_event_loop().run_until_complete(
+            self.async_get_session(session_id)
+        )
+    
+    async def async_update_session(self, session: ConversationSession) -> ConversationSession:
+        """Update an existing session asynchronously."""
+        if not self._initialized:
+            await self.init()
+        
+        session.updated_at = time.time()
+        table = f"{self.table_prefix}sessions"
+        
+        async with self._pool.acquire() as conn:
+            await conn.execute(f"""
+                UPDATE {table} 
+                SET name = $2, metadata = $3, updated_at = $4
+                WHERE session_id = $1
+            """, session.session_id, session.name,
+                json.dumps(session.metadata) if session.metadata else None,
+                session.updated_at)
+        
+        return session
+    
+    def update_session(self, session: ConversationSession) -> ConversationSession:
+        """Sync wrapper for update_session."""
+        return asyncio.get_event_loop().run_until_complete(
+            self.async_update_session(session)
+        )
+    
+    async def async_delete_session(self, session_id: str) -> bool:
+        """Delete a session asynchronously."""
+        if not self._initialized:
+            await self.init()
+        
+        table = f"{self.table_prefix}sessions"
+        async with self._pool.acquire() as conn:
+            result = await conn.execute(f"""
+                DELETE FROM {table} WHERE session_id = $1
+            """, session_id)
+        
+        return "DELETE 1" in result
+    
+    def delete_session(self, session_id: str) -> bool:
+        """Sync wrapper for delete_session."""
+        return asyncio.get_event_loop().run_until_complete(
+            self.async_delete_session(session_id)
+        )
+    
+    async def async_list_sessions(
+        self,
+        user_id: Optional[str] = None,
+        agent_id: Optional[str] = None,
+        limit: int = 100,
+        offset: int = 0
+    ) -> List[ConversationSession]:
+        """List sessions asynchronously."""
+        if not self._initialized:
+            await self.init()
+        
+        table = f"{self.table_prefix}sessions"
+        conditions = []
+        params = []
+        param_idx = 1
+        
+        if user_id:
+            conditions.append(f"user_id = ${param_idx}")
+            params.append(user_id)
+            param_idx += 1
+        
+        if agent_id:
+            conditions.append(f"agent_id = ${param_idx}")
+            params.append(agent_id)
+            param_idx += 1
+        
+        where_clause = f"WHERE {' AND '.join(conditions)}" if conditions else ""
+        params.extend([limit, offset])
+        
+        async with self._pool.acquire() as conn:
+            rows = await conn.fetch(f"""
+                SELECT * FROM {table} {where_clause}
+                ORDER BY updated_at DESC
+                LIMIT ${param_idx} OFFSET ${param_idx + 1}
+            """, *params)
+        
+        return [
+            ConversationSession(
+                session_id=row['session_id'],
+                user_id=row['user_id'],
+                agent_id=row['agent_id'],
+                name=row['name'],
+                metadata=json.loads(row['metadata']) if row['metadata'] else None,
+                created_at=row['created_at'],
+                updated_at=row['updated_at']
+            )
+            for row in rows
+        ]
+    
+    def list_sessions(
+        self,
+        user_id: Optional[str] = None,
+        agent_id: Optional[str] = None,
+        limit: int = 100,
+        offset: int = 0
+    ) -> List[ConversationSession]:
+        """Sync wrapper for list_sessions."""
+        return asyncio.get_event_loop().run_until_complete(
+            self.async_list_sessions(user_id, agent_id, limit, offset)
+        )
+    
+    async def async_add_message(self, session_id: str, message: ConversationMessage) -> ConversationMessage:
+        """Add a message asynchronously."""
+        if not self._initialized:
+            await self.init()
+        
+        message.session_id = session_id
+        table = f"{self.table_prefix}messages"
+        
+        async with self._pool.acquire() as conn:
+            await conn.execute(f"""
+                INSERT INTO {table} (id, session_id, role, content, metadata, created_at)
+                VALUES ($1, $2, $3, $4, $5, $6)
+            """, message.id, session_id, message.role, message.content,
+                json.dumps(message.metadata) if message.metadata else None,
+                message.created_at)
+        
+        return message
+    
+    def add_message(self, session_id: str, message: ConversationMessage) -> ConversationMessage:
+        """Sync wrapper for add_message."""
+        return asyncio.get_event_loop().run_until_complete(
+            self.async_add_message(session_id, message)
+        )
+    
+    async def async_get_messages(
+        self,
+        session_id: str,
+        limit: Optional[int] = None,
+        before: Optional[float] = None,
+        after: Optional[float] = None
+    ) -> List[ConversationMessage]:
+        """Get messages asynchronously."""
+        if not self._initialized:
+            await self.init()
+        
+        table = f"{self.table_prefix}messages"
+        conditions = ["session_id = $1"]
+        params = [session_id]
+        param_idx = 2
+        
+        if before:
+            conditions.append(f"created_at < ${param_idx}")
+            params.append(before)
+            param_idx += 1
+        
+        if after:
+            conditions.append(f"created_at > ${param_idx}")
+            params.append(after)
+            param_idx += 1
+        
+        where_clause = f"WHERE {' AND '.join(conditions)}"
+        limit_clause = f"LIMIT ${param_idx}" if limit else ""
+        if limit:
+            params.append(limit)
+        
+        async with self._pool.acquire() as conn:
+            rows = await conn.fetch(f"""
+                SELECT * FROM {table} {where_clause}
+                ORDER BY created_at ASC {limit_clause}
+            """, *params)
+        
+        return [
+            ConversationMessage(
+                id=row['id'],
+                session_id=row['session_id'],
+                role=row['role'],
+                content=row['content'],
+                metadata=json.loads(row['metadata']) if row['metadata'] else None,
+                created_at=row['created_at']
+            )
+            for row in rows
+        ]
+    
+    def get_messages(
+        self,
+        session_id: str,
+        limit: Optional[int] = None,
+        before: Optional[float] = None,
+        after: Optional[float] = None
+    ) -> List[ConversationMessage]:
+        """Sync wrapper for get_messages."""
+        return asyncio.get_event_loop().run_until_complete(
+            self.async_get_messages(session_id, limit, before, after)
+        )
+    
+    async def async_delete_messages(self, session_id: str, message_ids: Optional[List[str]] = None) -> int:
+        """Delete messages asynchronously."""
+        if not self._initialized:
+            await self.init()
+        
+        table = f"{self.table_prefix}messages"
+        
+        async with self._pool.acquire() as conn:
+            if message_ids is None:
+                result = await conn.execute(f"""
+                    DELETE FROM {table} WHERE session_id = $1
+                """, session_id)
+            else:
+                result = await conn.execute(f"""
+                    DELETE FROM {table} WHERE session_id = $1 AND id = ANY($2)
+                """, session_id, message_ids)
+        
+        # Parse count from result like "DELETE 5"
+        try:
+            return int(result.split()[-1])
+        except (ValueError, IndexError):
+            return 0
+    
+    def delete_messages(self, session_id: str, message_ids: Optional[List[str]] = None) -> int:
+        """Sync wrapper for delete_messages."""
+        return asyncio.get_event_loop().run_until_complete(
+            self.async_delete_messages(session_id, message_ids)
+        )
+    
+    async def async_close(self) -> None:
+        """Close the connection pool."""
+        if self._pool:
+            await self._pool.close()
+            self._pool = None
+            self._initialized = False
+    
+    def close(self) -> None:
+        """Sync wrapper for close."""
+        if self._pool:
+            asyncio.get_event_loop().run_until_complete(self.async_close())

src/praisonai/praisonai/persistence/conversation/async_sqlite.py

@@ -0,0 +1,371 @@
+"""
+Async SQLite implementation of ConversationStore.
+
+Requires: aiosqlite
+Install: pip install aiosqlite
+"""
+
+import asyncio
+import json
+import logging
+import time
+from typing import List, Optional
+
+from .base import ConversationStore, ConversationSession, ConversationMessage
+
+logger = logging.getLogger(__name__)
+
+
+class AsyncSQLiteConversationStore(ConversationStore):
+    """
+    Async SQLite conversation store using aiosqlite.
+    
+    Provides async database operations for SQLite.
+    
+    Example:
+        store = AsyncSQLiteConversationStore(path="./conversations.db")
+        await store.init()
+    """
+    
+    def __init__(
+        self,
+        path: str = "praisonai_conversations.db",
+        table_prefix: str = "praisonai_",
+    ):
+        """
+        Initialize async SQLite store.
+        
+        Args:
+            path: Path to SQLite database file
+            table_prefix: Prefix for table names
+        """
+        self.path = path
+        self.table_prefix = table_prefix
+        self._conn = None
+        self._initialized = False
+    
+    async def init(self):
+        """Initialize connection and create tables."""
+        if self._initialized:
+            return
+        
+        try:
+            import aiosqlite
+        except ImportError:
+            raise ImportError(
+                "aiosqlite is required for async SQLite support. "
+                "Install with: pip install aiosqlite"
+            )
+        
+        self._aiosqlite = aiosqlite
+        self._conn = await aiosqlite.connect(self.path)
+        self._conn.row_factory = aiosqlite.Row
+        await self._create_tables()
+        self._initialized = True
+    
+    async def _create_tables(self):
+        """Create required tables if they don't exist."""
+        sessions_table = f"{self.table_prefix}sessions"
+        messages_table = f"{self.table_prefix}messages"
+        
+        await self._conn.execute(f"""
+            CREATE TABLE IF NOT EXISTS {sessions_table} (
+                session_id TEXT PRIMARY KEY,
+                user_id TEXT,
+                agent_id TEXT,
+                name TEXT,
+                metadata TEXT,
+                created_at REAL,
+                updated_at REAL
+            )
+        """)
+        
+        await self._conn.execute(f"""
+            CREATE TABLE IF NOT EXISTS {messages_table} (
+                id TEXT PRIMARY KEY,
+                session_id TEXT REFERENCES {sessions_table}(session_id) ON DELETE CASCADE,
+                role TEXT,
+                content TEXT,
+                metadata TEXT,
+                created_at REAL
+            )
+        """)
+        
+        await self._conn.execute(f"""
+            CREATE INDEX IF NOT EXISTS idx_{messages_table}_session 
+            ON {messages_table}(session_id)
+        """)
+        
+        await self._conn.commit()
+    
+    async def async_create_session(self, session: ConversationSession) -> ConversationSession:
+        """Create a new session asynchronously."""
+        if not self._initialized:
+            await self.init()
+        
+        table = f"{self.table_prefix}sessions"
+        await self._conn.execute(f"""
+            INSERT INTO {table} (session_id, user_id, agent_id, name, metadata, created_at, updated_at)
+            VALUES (?, ?, ?, ?, ?, ?, ?)
+        """, (session.session_id, session.user_id, session.agent_id, session.name,
+              json.dumps(session.metadata) if session.metadata else None,
+              session.created_at, session.updated_at))
+        await self._conn.commit()
+        
+        return session
+    
+    def create_session(self, session: ConversationSession) -> ConversationSession:
+        """Sync wrapper for create_session."""
+        return asyncio.get_event_loop().run_until_complete(
+            self.async_create_session(session)
+        )
+    
+    async def async_get_session(self, session_id: str) -> Optional[ConversationSession]:
+        """Get a session by ID asynchronously."""
+        if not self._initialized:
+            await self.init()
+        
+        table = f"{self.table_prefix}sessions"
+        async with self._conn.execute(f"""
+            SELECT * FROM {table} WHERE session_id = ?
+        """, (session_id,)) as cursor:
+            row = await cursor.fetchone()
+        
+        if row:
+            return ConversationSession(
+                session_id=row['session_id'],
+                user_id=row['user_id'],
+                agent_id=row['agent_id'],
+                name=row['name'],
+                metadata=json.loads(row['metadata']) if row['metadata'] else None,
+                created_at=row['created_at'],
+                updated_at=row['updated_at']
+            )
+        return None
+    
+    def get_session(self, session_id: str) -> Optional[ConversationSession]:
+        """Sync wrapper for get_session."""
+        return asyncio.get_event_loop().run_until_complete(
+            self.async_get_session(session_id)
+        )
+    
+    async def async_update_session(self, session: ConversationSession) -> ConversationSession:
+        """Update an existing session asynchronously."""
+        if not self._initialized:
+            await self.init()
+        
+        session.updated_at = time.time()
+        table = f"{self.table_prefix}sessions"
+        
+        await self._conn.execute(f"""
+            UPDATE {table} 
+            SET name = ?, metadata = ?, updated_at = ?
+            WHERE session_id = ?
+        """, (session.name, json.dumps(session.metadata) if session.metadata else None,
+              session.updated_at, session.session_id))
+        await self._conn.commit()
+        
+        return session
+    
+    def update_session(self, session: ConversationSession) -> ConversationSession:
+        """Sync wrapper for update_session."""
+        return asyncio.get_event_loop().run_until_complete(
+            self.async_update_session(session)
+        )
+    
+    async def async_delete_session(self, session_id: str) -> bool:
+        """Delete a session asynchronously."""
+        if not self._initialized:
+            await self.init()
+        
+        table = f"{self.table_prefix}sessions"
+        cursor = await self._conn.execute(f"""
+            DELETE FROM {table} WHERE session_id = ?
+        """, (session_id,))
+        await self._conn.commit()
+        
+        return cursor.rowcount > 0
+    
+    def delete_session(self, session_id: str) -> bool:
+        """Sync wrapper for delete_session."""
+        return asyncio.get_event_loop().run_until_complete(
+            self.async_delete_session(session_id)
+        )
+    
+    async def async_list_sessions(
+        self,
+        user_id: Optional[str] = None,
+        agent_id: Optional[str] = None,
+        limit: int = 100,
+        offset: int = 0
+    ) -> List[ConversationSession]:
+        """List sessions asynchronously."""
+        if not self._initialized:
+            await self.init()
+        
+        table = f"{self.table_prefix}sessions"
+        conditions = []
+        params = []
+        
+        if user_id:
+            conditions.append("user_id = ?")
+            params.append(user_id)
+        
+        if agent_id:
+            conditions.append("agent_id = ?")
+            params.append(agent_id)
+        
+        where_clause = f"WHERE {' AND '.join(conditions)}" if conditions else ""
+        params.extend([limit, offset])
+        
+        async with self._conn.execute(f"""
+            SELECT * FROM {table} {where_clause}
+            ORDER BY updated_at DESC
+            LIMIT ? OFFSET ?
+        """, params) as cursor:
+            rows = await cursor.fetchall()
+        
+        return [
+            ConversationSession(
+                session_id=row['session_id'],
+                user_id=row['user_id'],
+                agent_id=row['agent_id'],
+                name=row['name'],
+                metadata=json.loads(row['metadata']) if row['metadata'] else None,
+                created_at=row['created_at'],
+                updated_at=row['updated_at']
+            )
+            for row in rows
+        ]
+    
+    def list_sessions(
+        self,
+        user_id: Optional[str] = None,
+        agent_id: Optional[str] = None,
+        limit: int = 100,
+        offset: int = 0
+    ) -> List[ConversationSession]:
+        """Sync wrapper for list_sessions."""
+        return asyncio.get_event_loop().run_until_complete(
+            self.async_list_sessions(user_id, agent_id, limit, offset)
+        )
+    
+    async def async_add_message(self, session_id: str, message: ConversationMessage) -> ConversationMessage:
+        """Add a message asynchronously."""
+        if not self._initialized:
+            await self.init()
+        
+        message.session_id = session_id
+        table = f"{self.table_prefix}messages"
+        
+        await self._conn.execute(f"""
+            INSERT INTO {table} (id, session_id, role, content, metadata, created_at)
+            VALUES (?, ?, ?, ?, ?, ?)
+        """, (message.id, session_id, message.role, message.content,
+              json.dumps(message.metadata) if message.metadata else None,
+              message.created_at))
+        await self._conn.commit()
+        
+        return message
+    
+    def add_message(self, session_id: str, message: ConversationMessage) -> ConversationMessage:
+        """Sync wrapper for add_message."""
+        return asyncio.get_event_loop().run_until_complete(
+            self.async_add_message(session_id, message)
+        )
+    
+    async def async_get_messages(
+        self,
+        session_id: str,
+        limit: Optional[int] = None,
+        before: Optional[float] = None,
+        after: Optional[float] = None
+    ) -> List[ConversationMessage]:
+        """Get messages asynchronously."""
+        if not self._initialized:
+            await self.init()
+        
+        table = f"{self.table_prefix}messages"
+        conditions = ["session_id = ?"]
+        params = [session_id]
+        
+        if before:
+            conditions.append("created_at < ?")
+            params.append(before)
+        
+        if after:
+            conditions.append("created_at > ?")
+            params.append(after)
+        
+        where_clause = f"WHERE {' AND '.join(conditions)}"
+        limit_clause = f"LIMIT ?" if limit else ""
+        if limit:
+            params.append(limit)
+        
+        async with self._conn.execute(f"""
+            SELECT * FROM {table} {where_clause}
+            ORDER BY created_at ASC {limit_clause}
+        """, params) as cursor:
+            rows = await cursor.fetchall()
+        
+        return [
+            ConversationMessage(
+                id=row['id'],
+                session_id=row['session_id'],
+                role=row['role'],
+                content=row['content'],
+                metadata=json.loads(row['metadata']) if row['metadata'] else None,
+                created_at=row['created_at']
+            )
+            for row in rows
+        ]
+    
+    def get_messages(
+        self,
+        session_id: str,
+        limit: Optional[int] = None,
+        before: Optional[float] = None,
+        after: Optional[float] = None
+    ) -> List[ConversationMessage]:
+        """Sync wrapper for get_messages."""
+        return asyncio.get_event_loop().run_until_complete(
+            self.async_get_messages(session_id, limit, before, after)
+        )
+    
+    async def async_delete_messages(self, session_id: str, message_ids: Optional[List[str]] = None) -> int:
+        """Delete messages asynchronously."""
+        if not self._initialized:
+            await self.init()
+        
+        table = f"{self.table_prefix}messages"
+        
+        if message_ids is None:
+            cursor = await self._conn.execute(f"""
+                DELETE FROM {table} WHERE session_id = ?
+            """, (session_id,))
+        else:
+            placeholders = ','.join(['?' for _ in message_ids])
+            cursor = await self._conn.execute(f"""
+                DELETE FROM {table} WHERE session_id = ? AND id IN ({placeholders})
+            """, [session_id] + message_ids)
+        
+        await self._conn.commit()
+        return cursor.rowcount
+    
+    def delete_messages(self, session_id: str, message_ids: Optional[List[str]] = None) -> int:
+        """Sync wrapper for delete_messages."""
+        return asyncio.get_event_loop().run_until_complete(
+            self.async_delete_messages(session_id, message_ids)
+        )
+    
+    async def async_close(self) -> None:
+        """Close the connection."""
+        if self._conn:
+            await self._conn.close()
+            self._conn = None
+            self._initialized = False
+    
+    def close(self) -> None:
+        """Sync wrapper for close."""
+        if self._conn:
+            asyncio.get_event_loop().run_until_complete(self.async_close())

src/praisonai/praisonai/persistence/conversation/json_store.py

@@ -0,0 +1,250 @@
+"""
+JSON file-based implementation of ConversationStore.
+
+Simple file-based storage for development and testing.
+No external dependencies required.
+"""
+
+import json
+import logging
+import time
+from pathlib import Path
+from typing import Dict, List, Optional
+import threading
+
+from .base import ConversationStore, ConversationSession, ConversationMessage
+
+logger = logging.getLogger(__name__)
+
+
+class JSONConversationStore(ConversationStore):
+    """
+    JSON file-based conversation store.
+    
+    Stores sessions and messages in JSON files.
+    Suitable for development, testing, and small-scale deployments.
+    
+    Example:
+        store = JSONConversationStore(path="./data/conversations")
+    """
+    
+    def __init__(
+        self,
+        path: str = "./praisonai_conversations",
+        pretty: bool = True,
+    ):
+        """
+        Initialize JSON conversation store.
+        
+        Args:
+            path: Directory path for JSON files
+            pretty: Use pretty-printed JSON (default: True)
+        """
+        self.path = Path(path)
+        self.pretty = pretty
+        self._lock = threading.Lock()
+        
+        # Create directory if it doesn't exist
+        self.path.mkdir(parents=True, exist_ok=True)
+        
+        # Sessions index file
+        self._index_file = self.path / "_sessions_index.json"
+        self._load_index()
+    
+    def _load_index(self):
+        """Load or create sessions index."""
+        if self._index_file.exists():
+            with open(self._index_file, 'r') as f:
+                self._index = json.load(f)
+        else:
+            self._index = {"sessions": {}}
+            self._save_index()
+    
+    def _save_index(self):
+        """Save sessions index."""
+        with open(self._index_file, 'w') as f:
+            if self.pretty:
+                json.dump(self._index, f, indent=2, default=str)
+            else:
+                json.dump(self._index, f, default=str)
+    
+    def _session_file(self, session_id: str) -> Path:
+        """Get path to session file."""
+        return self.path / f"{session_id}.json"
+    
+    def _load_session_data(self, session_id: str) -> Optional[Dict]:
+        """Load session data from file."""
+        file_path = self._session_file(session_id)
+        if file_path.exists():
+            with open(file_path, 'r') as f:
+                return json.load(f)
+        return None
+    
+    def _save_session_data(self, session_id: str, data: Dict):
+        """Save session data to file."""
+        file_path = self._session_file(session_id)
+        with open(file_path, 'w') as f:
+            if self.pretty:
+                json.dump(data, f, indent=2, default=str)
+            else:
+                json.dump(data, f, default=str)
+    
+    def create_session(self, session: ConversationSession) -> ConversationSession:
+        """Create a new session."""
+        with self._lock:
+            data = {
+                "session": session.to_dict(),
+                "messages": []
+            }
+            self._save_session_data(session.session_id, data)
+            
+            # Update index
+            self._index["sessions"][session.session_id] = {
+                "user_id": session.user_id,
+                "agent_id": session.agent_id,
+                "name": session.name,
+                "created_at": session.created_at,
+                "updated_at": session.updated_at,
+            }
+            self._save_index()
+            
+            return session
+    
+    def get_session(self, session_id: str) -> Optional[ConversationSession]:
+        """Get a session by ID."""
+        data = self._load_session_data(session_id)
+        if data and "session" in data:
+            return ConversationSession.from_dict(data["session"])
+        return None
+    
+    def update_session(self, session: ConversationSession) -> ConversationSession:
+        """Update an existing session."""
+        with self._lock:
+            data = self._load_session_data(session.session_id)
+            if not data:
+                raise ValueError(f"Session {session.session_id} not found")
+            
+            session.updated_at = time.time()
+            data["session"] = session.to_dict()
+            self._save_session_data(session.session_id, data)
+            
+            # Update index
+            self._index["sessions"][session.session_id].update({
+                "name": session.name,
+                "updated_at": session.updated_at,
+            })
+            self._save_index()
+            
+            return session
+    
+    def delete_session(self, session_id: str) -> bool:
+        """Delete a session and all its messages."""
+        with self._lock:
+            file_path = self._session_file(session_id)
+            if file_path.exists():
+                file_path.unlink()
+                
+                # Update index
+                if session_id in self._index["sessions"]:
+                    del self._index["sessions"][session_id]
+                    self._save_index()
+                
+                return True
+            return False
+    
+    def list_sessions(
+        self,
+        user_id: Optional[str] = None,
+        agent_id: Optional[str] = None,
+        limit: int = 100,
+        offset: int = 0
+    ) -> List[ConversationSession]:
+        """List sessions, optionally filtered by user or agent."""
+        sessions = []
+        
+        for sid, info in self._index["sessions"].items():
+            if user_id and info.get("user_id") != user_id:
+                continue
+            if agent_id and info.get("agent_id") != agent_id:
+                continue
+            
+            session = self.get_session(sid)
+            if session:
+                sessions.append(session)
+        
+        # Sort by updated_at descending
+        sessions.sort(key=lambda s: s.updated_at, reverse=True)
+        
+        return sessions[offset:offset + limit]
+    
+    def add_message(self, session_id: str, message: ConversationMessage) -> ConversationMessage:
+        """Add a message to a session."""
+        with self._lock:
+            data = self._load_session_data(session_id)
+            if not data:
+                raise ValueError(f"Session {session_id} not found")
+            
+            message.session_id = session_id
+            data["messages"].append(message.to_dict())
+            
+            # Update session timestamp
+            data["session"]["updated_at"] = time.time()
+            
+            self._save_session_data(session_id, data)
+            
+            # Update index
+            if session_id in self._index["sessions"]:
+                self._index["sessions"][session_id]["updated_at"] = time.time()
+                self._save_index()
+            
+            return message
+    
+    def get_messages(
+        self,
+        session_id: str,
+        limit: Optional[int] = None,
+        before: Optional[float] = None,
+        after: Optional[float] = None
+    ) -> List[ConversationMessage]:
+        """Get messages from a session."""
+        data = self._load_session_data(session_id)
+        if not data:
+            return []
+        
+        messages = [ConversationMessage.from_dict(m) for m in data.get("messages", [])]
+        
+        # Apply filters
+        if before:
+            messages = [m for m in messages if m.created_at < before]
+        if after:
+            messages = [m for m in messages if m.created_at > after]
+        
+        # Sort by created_at
+        messages.sort(key=lambda m: m.created_at)
+        
+        if limit:
+            messages = messages[-limit:]
+        
+        return messages
+    
+    def delete_messages(self, session_id: str, message_ids: Optional[List[str]] = None) -> int:
+        """Delete messages. If message_ids is None, delete all messages in session."""
+        with self._lock:
+            data = self._load_session_data(session_id)
+            if not data:
+                return 0
+            
+            if message_ids is None:
+                count = len(data["messages"])
+                data["messages"] = []
+            else:
+                original_count = len(data["messages"])
+                data["messages"] = [m for m in data["messages"] if m.get("id") not in message_ids]
+                count = original_count - len(data["messages"])
+            
+            self._save_session_data(session_id, data)
+            return count
+    
+    def close(self) -> None:
+        """Close the store."""
+        pass  # No resources to release

src/praisonai/praisonai/persistence/factory.py

@@ -62,10 +62,29 @@ def create_conversation_store(
         from .conversation.surrealdb import SurrealDBConversationStore
         return SurrealDBConversationStore(url=url, **options)
     
+    elif backend == "json":
+        from .conversation.json_store import JSONConversationStore
+        path = url or options.pop("path", None) or "./praisonai_conversations"
+        return JSONConversationStore(path=path, **options)
+    
+    elif backend in ("async_postgres", "asyncpg", "postgres_async"):
+        from .conversation.async_postgres import AsyncPostgresConversationStore
+        return AsyncPostgresConversationStore(url=url, **options)
+    
+    elif backend in ("async_sqlite", "aiosqlite", "sqlite_async"):
+        from .conversation.async_sqlite import AsyncSQLiteConversationStore
+        path = url or options.pop("path", None)
+        return AsyncSQLiteConversationStore(path=path, **options)
+    
+    elif backend in ("async_mysql", "aiomysql", "mysql_async"):
+        from .conversation.async_mysql import AsyncMySQLConversationStore
+        return AsyncMySQLConversationStore(url=url, **options)
+    
     else:
         raise ValueError(
             f"Unknown conversation store backend: {backend}. "
-            f"Supported: postgres, mysql, sqlite, singlestore, supabase, surrealdb"
+            f"Supported: postgres, mysql, sqlite, json, singlestore, supabase, surrealdb, "
+            f"async_postgres, async_sqlite, async_mysql"
         )
 
 
@@ -135,10 +154,47 @@ def create_knowledge_store(
         from .knowledge.clickhouse import ClickHouseKnowledgeStore
         return ClickHouseKnowledgeStore(**options)
     
+    elif backend == "couchbase":
+        from .knowledge.couchbase import CouchbaseKnowledgeStore
+        return CouchbaseKnowledgeStore(**options)
+    
+    elif backend in ("mongodb_vector", "mongodb_atlas", "mongo_vector"):
+        from .knowledge.mongodb_vector import MongoDBVectorKnowledgeStore
+        return MongoDBVectorKnowledgeStore(url=url, **options)
+    
+    elif backend in ("singlestore_vector", "singlestore_v"):
+        from .knowledge.singlestore_vector import SingleStoreVectorKnowledgeStore
+        return SingleStoreVectorKnowledgeStore(url=url, **options)
+    
+    elif backend in ("surrealdb_vector", "surrealdb_v"):
+        from .knowledge.surrealdb_vector import SurrealDBVectorKnowledgeStore
+        return SurrealDBVectorKnowledgeStore(url=url, **options)
+    
+    elif backend in ("upstash_vector", "upstash_v"):
+        from .knowledge.upstash_vector import UpstashVectorKnowledgeStore
+        return UpstashVectorKnowledgeStore(url=url, **options)
+    
+    elif backend == "lightrag":
+        from .knowledge.lightrag_adapter import LightRAGKnowledgeStore
+        return LightRAGKnowledgeStore(**options)
+    
+    elif backend in ("langchain", "langchain_adapter"):
+        from .knowledge.langchain_adapter import LangChainKnowledgeStore
+        return LangChainKnowledgeStore(**options)
+    
+    elif backend in ("llamaindex", "llama_index", "llamaindex_adapter"):
+        from .knowledge.llamaindex_adapter import LlamaIndexKnowledgeStore
+        return LlamaIndexKnowledgeStore(**options)
+    
+    elif backend in ("cosmosdb", "cosmos", "azure_cosmos", "cosmosdb_vector"):
+        from .knowledge.cosmosdb_vector import CosmosDBVectorKnowledgeStore
+        return CosmosDBVectorKnowledgeStore(**options)
+    
     else:
         raise ValueError(
             f"Unknown knowledge store backend: {backend}. "
-            f"Supported: qdrant, pinecone, chroma, weaviate, lancedb, milvus, pgvector, redis, cassandra, clickhouse"
+            f"Supported: qdrant, pinecone, chroma, weaviate, lancedb, milvus, pgvector, redis, cassandra, clickhouse, "
+            f"couchbase, mongodb_vector, singlestore_vector, surrealdb_vector, upstash_vector, lightrag, langchain, llamaindex, cosmosdb"
         )
 
 
@@ -190,10 +246,21 @@ def create_state_store(
         from .state.memory import MemoryStateStore
         return MemoryStateStore(**options)
     
+    elif backend == "gcs":
+        from .state.gcs import GCSStateStore
+        bucket = options.pop("bucket_name", None) or options.pop("bucket", None)
+        if not bucket:
+            raise ValueError("GCS state store requires 'bucket_name' option")
+        return GCSStateStore(bucket_name=bucket, **options)
+    
+    elif backend in ("async_mongodb", "motor", "mongodb_async"):
+        from .state.async_mongodb import AsyncMongoDBStateStore
+        return AsyncMongoDBStateStore(url=url, **options)
+    
     else:
         raise ValueError(
             f"Unknown state store backend: {backend}. "
-            f"Supported: redis, dynamodb, firestore, mongodb, upstash, memory"
+            f"Supported: redis, dynamodb, firestore, mongodb, upstash, memory, gcs, async_mongodb"
         )
 
 

src/praisonai/praisonai/persistence/knowledge/cosmosdb_vector.py

@@ -0,0 +1,438 @@
+"""
+Azure Cosmos DB Vector implementation of KnowledgeStore.
+
+Supports both MongoDB API and vCore modes.
+
+Requires: pymongo (for MongoDB API) or azure-cosmos (for SQL API)
+Install: pip install pymongo  # or pip install azure-cosmos
+"""
+
+import logging
+from typing import Any, Dict, List, Optional
+
+from .base import KnowledgeStore, KnowledgeDocument
+
+logger = logging.getLogger(__name__)
+
+
+class CosmosDBVectorKnowledgeStore(KnowledgeStore):
+    """
+    Azure Cosmos DB vector store for knowledge/RAG.
+    
+    Supports MongoDB API with vector search capabilities.
+    
+    Example:
+        store = CosmosDBVectorKnowledgeStore(
+            connection_string="mongodb+srv://...",
+            database="praisonai",
+            collection="vectors"
+        )
+    """
+    
+    def __init__(
+        self,
+        connection_string: Optional[str] = None,
+        database: str = "praisonai",
+        collection: str = "vectors",
+        index_name: str = "vector_index",
+        embedding_field: str = "embedding",
+        text_field: str = "content",
+        embedding_dim: int = 1536,
+        api_mode: str = "mongodb",  # "mongodb" or "sql"
+    ):
+        """
+        Initialize Cosmos DB Vector store.
+        
+        Args:
+            connection_string: Azure Cosmos DB connection string
+            database: Database name
+            collection: Collection name
+            index_name: Vector search index name
+            embedding_field: Field name for embeddings
+            text_field: Field name for text content
+            embedding_dim: Embedding dimension
+            api_mode: API mode ("mongodb" or "sql")
+        """
+        self.connection_string = connection_string
+        self.database_name = database
+        self.collection_name = collection
+        self.index_name = index_name
+        self.embedding_field = embedding_field
+        self.text_field = text_field
+        self.embedding_dim = embedding_dim
+        self.api_mode = api_mode
+        
+        self._client = None
+        self._db = None
+        self._collection = None
+        self._initialized = False
+    
+    def _init_client(self):
+        """Initialize Cosmos DB client lazily."""
+        if self._initialized:
+            return
+        
+        if not self.connection_string:
+            import os
+            self.connection_string = os.getenv("COSMOS_CONNECTION_STRING")
+            if not self.connection_string:
+                raise ValueError(
+                    "connection_string required. Set COSMOS_CONNECTION_STRING env var."
+                )
+        
+        if self.api_mode == "mongodb":
+            self._init_mongodb_client()
+        else:
+            self._init_sql_client()
+        
+        self._initialized = True
+    
+    def _init_mongodb_client(self):
+        """Initialize MongoDB API client."""
+        try:
+            from pymongo import MongoClient
+        except ImportError:
+            raise ImportError(
+                "pymongo is required for Cosmos DB MongoDB API. "
+                "Install with: pip install pymongo"
+            )
+        
+        self._client = MongoClient(self.connection_string)
+        self._db = self._client[self.database_name]
+        self._collection = self._db[self.collection_name]
+    
+    def _init_sql_client(self):
+        """Initialize SQL API client."""
+        try:
+            from azure.cosmos import CosmosClient
+        except ImportError:
+            raise ImportError(
+                "azure-cosmos is required for Cosmos DB SQL API. "
+                "Install with: pip install azure-cosmos"
+            )
+        
+        self._client = CosmosClient.from_connection_string(self.connection_string)
+        self._db = self._client.get_database_client(self.database_name)
+        self._collection = self._db.get_container_client(self.collection_name)
+    
+    def create_collection(
+        self,
+        name: str,
+        dimension: int = 1536,
+        distance: str = "cosine",
+        metadata: Optional[Dict[str, Any]] = None
+    ) -> None:
+        """Create collection with vector index."""
+        self._init_client()
+        
+        if self.api_mode == "mongodb":
+            # Create vector search index via MongoDB command
+            try:
+                self._db.command({
+                    "createIndexes": name,
+                    "indexes": [{
+                        "name": self.index_name,
+                        "key": {self.embedding_field: "cosmosSearch"},
+                        "cosmosSearchOptions": {
+                            "kind": "vector-ivf",
+                            "numLists": 100,
+                            "similarity": distance,
+                            "dimensions": dimension
+                        }
+                    }]
+                })
+            except Exception as e:
+                logger.warning(f"Index creation may require manual setup: {e}")
+    
+    def delete_collection(self, name: str) -> bool:
+        """Delete collection."""
+        self._init_client()
+        try:
+            if self.api_mode == "mongodb":
+                self._db.drop_collection(name)
+            else:
+                self._db.delete_container(name)
+            return True
+        except Exception as e:
+            logger.error(f"Failed to delete collection: {e}")
+            return False
+    
+    def collection_exists(self, name: str) -> bool:
+        """Check if collection exists."""
+        self._init_client()
+        if self.api_mode == "mongodb":
+            return name in self._db.list_collection_names()
+        return True  # SQL API containers are pre-created
+    
+    def list_collections(self) -> List[str]:
+        """List collections."""
+        self._init_client()
+        if self.api_mode == "mongodb":
+            return self._db.list_collection_names()
+        return [c["id"] for c in self._db.list_containers()]
+    
+    def insert(
+        self,
+        collection: str,
+        documents: List[KnowledgeDocument]
+    ) -> List[str]:
+        """Insert documents."""
+        self._init_client()
+        
+        coll = self._db[collection] if self.api_mode == "mongodb" else self._db.get_container_client(collection)
+        ids = []
+        
+        for doc in documents:
+            item = {
+                "_id" if self.api_mode == "mongodb" else "id": doc.id,
+                self.text_field: doc.content,
+                self.embedding_field: doc.embedding,
+                "metadata": doc.metadata or {},
+                "content_hash": doc.content_hash,
+                "created_at": doc.created_at,
+            }
+            
+            try:
+                if self.api_mode == "mongodb":
+                    coll.insert_one(item)
+                else:
+                    coll.create_item(item)
+                ids.append(doc.id)
+            except Exception as e:
+                logger.warning(f"Failed to insert {doc.id}: {e}")
+        
+        return ids
+    
+    def upsert(
+        self,
+        collection: str,
+        documents: List[KnowledgeDocument]
+    ) -> List[str]:
+        """Upsert documents."""
+        self._init_client()
+        
+        coll = self._db[collection] if self.api_mode == "mongodb" else self._db.get_container_client(collection)
+        ids = []
+        
+        for doc in documents:
+            item = {
+                "_id" if self.api_mode == "mongodb" else "id": doc.id,
+                self.text_field: doc.content,
+                self.embedding_field: doc.embedding,
+                "metadata": doc.metadata or {},
+                "content_hash": doc.content_hash,
+                "created_at": doc.created_at,
+            }
+            
+            try:
+                if self.api_mode == "mongodb":
+                    coll.replace_one({"_id": doc.id}, item, upsert=True)
+                else:
+                    coll.upsert_item(item)
+                ids.append(doc.id)
+            except Exception as e:
+                logger.warning(f"Failed to upsert {doc.id}: {e}")
+        
+        return ids
+    
+    def search(
+        self,
+        collection: str,
+        query_embedding: List[float],
+        limit: int = 5,
+        filters: Optional[Dict[str, Any]] = None,
+        score_threshold: Optional[float] = None
+    ) -> List[KnowledgeDocument]:
+        """Search for similar documents using vector search."""
+        self._init_client()
+        
+        if self.api_mode == "mongodb":
+            return self._search_mongodb(collection, query_embedding, limit, filters, score_threshold)
+        else:
+            return self._search_sql(collection, query_embedding, limit, filters, score_threshold)
+    
+    def _search_mongodb(
+        self,
+        collection: str,
+        query_embedding: List[float],
+        limit: int,
+        filters: Optional[Dict[str, Any]],
+        score_threshold: Optional[float]
+    ) -> List[KnowledgeDocument]:
+        """MongoDB API vector search."""
+        coll = self._db[collection]
+        
+        pipeline = [
+            {
+                "$search": {
+                    "cosmosSearch": {
+                        "vector": query_embedding,
+                        "path": self.embedding_field,
+                        "k": limit
+                    },
+                    "returnStoredSource": True
+                }
+            },
+            {
+                "$project": {
+                    "similarityScore": {"$meta": "searchScore"},
+                    self.text_field: 1,
+                    "metadata": 1,
+                    "content_hash": 1,
+                    "created_at": 1
+                }
+            }
+        ]
+        
+        if filters:
+            pipeline.insert(1, {"$match": filters})
+        
+        try:
+            results = list(coll.aggregate(pipeline))
+            
+            documents = []
+            for doc in results:
+                score = doc.get("similarityScore", 0)
+                if score_threshold and score < score_threshold:
+                    continue
+                
+                documents.append(KnowledgeDocument(
+                    id=str(doc.get("_id", "")),
+                    content=doc.get(self.text_field, ""),
+                    metadata={**(doc.get("metadata") or {}), "score": score},
+                    content_hash=doc.get("content_hash"),
+                    created_at=doc.get("created_at", 0)
+                ))
+            
+            return documents
+        except Exception as e:
+            logger.error(f"Vector search failed: {e}")
+            return []
+    
+    def _search_sql(
+        self,
+        collection: str,
+        query_embedding: List[float],
+        limit: int,
+        filters: Optional[Dict[str, Any]],
+        score_threshold: Optional[float]
+    ) -> List[KnowledgeDocument]:
+        """SQL API vector search (requires DiskANN index)."""
+        coll = self._db.get_container_client(collection)
+        
+        # SQL API uses VectorDistance function
+        query = f"""
+            SELECT TOP {limit} c.id, c.{self.text_field}, c.metadata, c.content_hash, c.created_at,
+                   VectorDistance(c.{self.embedding_field}, @embedding) AS score
+            FROM c
+            ORDER BY VectorDistance(c.{self.embedding_field}, @embedding)
+        """
+        
+        try:
+            results = list(coll.query_items(
+                query=query,
+                parameters=[{"name": "@embedding", "value": query_embedding}],
+                enable_cross_partition_query=True
+            ))
+            
+            documents = []
+            for doc in results:
+                score = doc.get("score", 0)
+                if score_threshold and score < score_threshold:
+                    continue
+                
+                documents.append(KnowledgeDocument(
+                    id=doc.get("id", ""),
+                    content=doc.get(self.text_field, ""),
+                    metadata={**(doc.get("metadata") or {}), "score": score},
+                    content_hash=doc.get("content_hash"),
+                    created_at=doc.get("created_at", 0)
+                ))
+            
+            return documents
+        except Exception as e:
+            logger.error(f"Vector search failed: {e}")
+            return []
+    
+    def get(
+        self,
+        collection: str,
+        ids: List[str]
+    ) -> List[KnowledgeDocument]:
+        """Get documents by IDs."""
+        self._init_client()
+        
+        coll = self._db[collection] if self.api_mode == "mongodb" else self._db.get_container_client(collection)
+        documents = []
+        
+        for doc_id in ids:
+            try:
+                if self.api_mode == "mongodb":
+                    doc = coll.find_one({"_id": doc_id})
+                else:
+                    doc = coll.read_item(item=doc_id, partition_key=doc_id)
+                
+                if doc:
+                    documents.append(KnowledgeDocument(
+                        id=str(doc.get("_id" if self.api_mode == "mongodb" else "id", "")),
+                        content=doc.get(self.text_field, ""),
+                        embedding=doc.get(self.embedding_field),
+                        metadata=doc.get("metadata"),
+                        content_hash=doc.get("content_hash"),
+                        created_at=doc.get("created_at", 0)
+                    ))
+            except Exception as e:
+                logger.warning(f"Failed to get {doc_id}: {e}")
+        
+        return documents
+    
+    def delete(
+        self,
+        collection: str,
+        ids: Optional[List[str]] = None,
+        filters: Optional[Dict[str, Any]] = None
+    ) -> int:
+        """Delete documents."""
+        self._init_client()
+        
+        coll = self._db[collection] if self.api_mode == "mongodb" else self._db.get_container_client(collection)
+        
+        if ids:
+            count = 0
+            for doc_id in ids:
+                try:
+                    if self.api_mode == "mongodb":
+                        coll.delete_one({"_id": doc_id})
+                    else:
+                        coll.delete_item(item=doc_id, partition_key=doc_id)
+                    count += 1
+                except Exception as e:
+                    logger.warning(f"Failed to delete {doc_id}: {e}")
+            return count
+        
+        return 0
+    
+    def count(self, collection: str) -> int:
+        """Count documents."""
+        self._init_client()
+        
+        if self.api_mode == "mongodb":
+            return self._db[collection].count_documents({})
+        else:
+            # SQL API count
+            coll = self._db.get_container_client(collection)
+            result = list(coll.query_items(
+                query="SELECT VALUE COUNT(1) FROM c",
+                enable_cross_partition_query=True
+            ))
+            return result[0] if result else 0
+    
+    def close(self) -> None:
+        """Close the connection."""
+        if self._client:
+            if self.api_mode == "mongodb":
+                self._client.close()
+            self._client = None
+            self._db = None
+            self._collection = None
+            self._initialized = False

src/praisonai/praisonai/persistence/knowledge/couchbase.py

@@ -0,0 +1,286 @@
+"""
+Couchbase implementation of KnowledgeStore.
+
+Requires: couchbase
+Install: pip install couchbase
+"""
+
+import logging
+from typing import Any, Dict, List, Optional
+
+from .base import KnowledgeStore, KnowledgeDocument
+
+logger = logging.getLogger(__name__)
+
+
+class CouchbaseKnowledgeStore(KnowledgeStore):
+    """
+    Couchbase vector store for knowledge/RAG.
+    
+    Uses Couchbase's vector search capabilities.
+    
+    Example:
+        store = CouchbaseKnowledgeStore(
+            connection_string="couchbase://localhost",
+            username="admin",
+            password="password",
+            bucket_name="praisonai"
+        )
+    """
+    
+    def __init__(
+        self,
+        connection_string: str = "couchbase://localhost",
+        username: str = "Administrator",
+        password: str = "",
+        bucket_name: str = "praisonai",
+        scope_name: str = "_default",
+        collection_name: str = "vectors",
+        index_name: str = "vector_index",
+        vector_field: str = "embedding",
+        text_field: str = "content",
+        embedding_dim: int = 1536,
+    ):
+        """
+        Initialize Couchbase knowledge store.
+        
+        Args:
+            connection_string: Couchbase connection string
+            username: Couchbase username
+            password: Couchbase password
+            bucket_name: Bucket name
+            scope_name: Scope name
+            collection_name: Collection name
+            index_name: Vector search index name
+            vector_field: Field name for embeddings
+            text_field: Field name for text content
+            embedding_dim: Embedding dimension
+        """
+        self.connection_string = connection_string
+        self.username = username
+        self.password = password
+        self.bucket_name = bucket_name
+        self.scope_name = scope_name
+        self.collection_name = collection_name
+        self.index_name = index_name
+        self.vector_field = vector_field
+        self.text_field = text_field
+        self.embedding_dim = embedding_dim
+        
+        self._cluster = None
+        self._bucket = None
+        self._collection = None
+        self._initialized = False
+    
+    def _init_client(self):
+        """Initialize Couchbase client lazily."""
+        if self._initialized:
+            return
+        
+        try:
+            from couchbase.cluster import Cluster
+            from couchbase.options import ClusterOptions
+            from couchbase.auth import PasswordAuthenticator
+            from datetime import timedelta
+        except ImportError:
+            raise ImportError(
+                "couchbase is required for Couchbase support. "
+                "Install with: pip install couchbase"
+            )
+        
+        auth = PasswordAuthenticator(self.username, self.password)
+        options = ClusterOptions(auth)
+        
+        self._cluster = Cluster(self.connection_string, options)
+        self._cluster.wait_until_ready(timedelta(seconds=10))
+        
+        self._bucket = self._cluster.bucket(self.bucket_name)
+        scope = self._bucket.scope(self.scope_name)
+        self._collection = scope.collection(self.collection_name)
+        
+        self._initialized = True
+    
+    def insert(
+        self,
+        collection_name: str,
+        documents: List[KnowledgeDocument],
+        embeddings: Optional[List[List[float]]] = None,
+    ) -> List[str]:
+        """Insert documents into the store."""
+        self._init_client()
+        
+        ids = []
+        for i, doc in enumerate(documents):
+            doc_id = doc.id or f"doc_{i}_{hash(doc.content)}"
+            
+            data = {
+                self.text_field: doc.content,
+                "metadata": doc.metadata or {},
+            }
+            
+            if embeddings and i < len(embeddings):
+                data[self.vector_field] = embeddings[i]
+            
+            self._collection.upsert(doc_id, data)
+            ids.append(doc_id)
+        
+        return ids
+    
+    def upsert(
+        self,
+        collection_name: str,
+        documents: List[KnowledgeDocument],
+        embeddings: Optional[List[List[float]]] = None,
+    ) -> List[str]:
+        """Upsert documents (same as insert for Couchbase)."""
+        return self.insert(collection_name, documents, embeddings)
+    
+    def search(
+        self,
+        collection_name: str,
+        query_embedding: List[float],
+        limit: int = 10,
+        filters: Optional[Dict[str, Any]] = None,
+    ) -> List[KnowledgeDocument]:
+        """Search for similar documents using vector search."""
+        self._init_client()
+        
+        try:
+            from couchbase.vector_search import VectorQuery, VectorSearch
+        except ImportError:
+            logger.warning("Vector search not available in this Couchbase version")
+            return []
+        
+        vector_query = VectorQuery(
+            self.vector_field,
+            query_embedding,
+            num_candidates=limit * 2
+        )
+        
+        search_req = VectorSearch.from_vector_query(vector_query)
+        
+        try:
+            scope = self._bucket.scope(self.scope_name)
+            result = scope.search(
+                self.index_name,
+                search_req,
+                limit=limit
+            )
+            
+            documents = []
+            for row in result.rows():
+                doc_id = row.id
+                try:
+                    doc_data = self._collection.get(doc_id).content_as[dict]
+                    documents.append(KnowledgeDocument(
+                        id=doc_id,
+                        content=doc_data.get(self.text_field, ""),
+                        metadata=doc_data.get("metadata", {}),
+                        score=row.score if hasattr(row, 'score') else None
+                    ))
+                except Exception as e:
+                    logger.warning(f"Failed to fetch document {doc_id}: {e}")
+            
+            return documents
+        except Exception as e:
+            logger.error(f"Vector search failed: {e}")
+            return []
+    
+    def delete(
+        self,
+        collection_name: str,
+        ids: Optional[List[str]] = None,
+        filters: Optional[Dict[str, Any]] = None,
+    ) -> int:
+        """Delete documents by ID."""
+        self._init_client()
+        
+        if not ids:
+            return 0
+        
+        count = 0
+        for doc_id in ids:
+            try:
+                self._collection.remove(doc_id)
+                count += 1
+            except Exception as e:
+                logger.warning(f"Failed to delete {doc_id}: {e}")
+        
+        return count
+    
+    def get(
+        self,
+        collection_name: str,
+        ids: List[str],
+    ) -> List[KnowledgeDocument]:
+        """Get documents by ID."""
+        self._init_client()
+        
+        documents = []
+        for doc_id in ids:
+            try:
+                doc_data = self._collection.get(doc_id).content_as[dict]
+                documents.append(KnowledgeDocument(
+                    id=doc_id,
+                    content=doc_data.get(self.text_field, ""),
+                    metadata=doc_data.get("metadata", {})
+                ))
+            except Exception as e:
+                logger.warning(f"Failed to get {doc_id}: {e}")
+        
+        return documents
+    
+    def count(self, collection_name: str) -> int:
+        """Count documents in collection."""
+        self._init_client()
+        
+        try:
+            query = f"SELECT COUNT(*) as count FROM `{self.bucket_name}`.`{self.scope_name}`.`{self.collection_name}`"
+            result = self._cluster.query(query)
+            for row in result:
+                return row.get("count", 0)
+        except Exception as e:
+            logger.error(f"Count query failed: {e}")
+        
+        return 0
+    
+    def create_collection(self, collection_name: str, **kwargs) -> bool:
+        """Create collection (Couchbase collections are pre-created)."""
+        self._init_client()
+        return True
+    
+    def delete_collection(self, collection_name: str) -> bool:
+        """Delete all documents in collection."""
+        self._init_client()
+        
+        try:
+            query = f"DELETE FROM `{self.bucket_name}`.`{self.scope_name}`.`{self.collection_name}`"
+            self._cluster.query(query)
+            return True
+        except Exception as e:
+            logger.error(f"Delete collection failed: {e}")
+            return False
+    
+    def list_collections(self) -> List[str]:
+        """List available collections."""
+        self._init_client()
+        
+        try:
+            collections = self._bucket.collections().get_all_scopes()
+            result = []
+            for scope in collections:
+                for coll in scope.collections:
+                    result.append(f"{scope.name}.{coll.name}")
+            return result
+        except Exception as e:
+            logger.error(f"List collections failed: {e}")
+            return []
+    
+    def close(self) -> None:
+        """Close the connection."""
+        if self._cluster:
+            self._cluster.close()
+            self._cluster = None
+            self._bucket = None
+            self._collection = None
+            self._initialized = False

src/praisonai/praisonai/persistence/knowledge/langchain_adapter.py

@@ -0,0 +1,291 @@
+"""
+LangChain VectorStore adapter implementation of KnowledgeStore.
+
+Wraps any LangChain VectorStore for use with PraisonAI.
+
+Requires: langchain-core
+Install: pip install langchain-core
+"""
+
+import logging
+from typing import Any, Dict, List, Optional
+
+from .base import KnowledgeStore, KnowledgeDocument
+
+logger = logging.getLogger(__name__)
+
+
+class LangChainKnowledgeStore(KnowledgeStore):
+    """
+    LangChain VectorStore adapter for knowledge/RAG.
+    
+    Wraps any LangChain VectorStore for use with PraisonAI's interface.
+    
+    Example:
+        from langchain_chroma import Chroma
+        
+        lc_store = Chroma(persist_directory="./chroma_db")
+        store = LangChainKnowledgeStore(vectorstore=lc_store)
+    """
+    
+    def __init__(
+        self,
+        vectorstore: Any = None,
+        vectorstore_cls: Optional[str] = None,
+        vectorstore_kwargs: Optional[Dict[str, Any]] = None,
+    ):
+        """
+        Initialize LangChain adapter.
+        
+        Args:
+            vectorstore: Pre-initialized LangChain VectorStore instance
+            vectorstore_cls: Class name to import (e.g., "langchain_chroma.Chroma")
+            vectorstore_kwargs: Kwargs for vectorstore initialization
+        """
+        self._vectorstore = vectorstore
+        self._vectorstore_cls = vectorstore_cls
+        self._vectorstore_kwargs = vectorstore_kwargs or {}
+        self._initialized = vectorstore is not None
+    
+    def _init_client(self):
+        """Initialize vectorstore lazily."""
+        if self._initialized:
+            return
+        
+        if not self._vectorstore_cls:
+            raise ValueError(
+                "Either vectorstore or vectorstore_cls must be provided"
+            )
+        
+        # Dynamic import of vectorstore class
+        parts = self._vectorstore_cls.rsplit(".", 1)
+        if len(parts) == 2:
+            module_name, class_name = parts
+            import importlib
+            module = importlib.import_module(module_name)
+            cls = getattr(module, class_name)
+            self._vectorstore = cls(**self._vectorstore_kwargs)
+        else:
+            raise ValueError(f"Invalid vectorstore_cls: {self._vectorstore_cls}")
+        
+        self._initialized = True
+    
+    def create_collection(
+        self,
+        name: str,
+        dimension: int = 1536,
+        distance: str = "cosine",
+        metadata: Optional[Dict[str, Any]] = None
+    ) -> None:
+        """Create collection (handled by underlying vectorstore)."""
+        self._init_client()
+        logger.info(f"Collection creation handled by LangChain vectorstore")
+    
+    def delete_collection(self, name: str) -> bool:
+        """Delete collection."""
+        self._init_client()
+        try:
+            if hasattr(self._vectorstore, 'delete_collection'):
+                self._vectorstore.delete_collection()
+                return True
+            logger.warning("Underlying vectorstore doesn't support delete_collection")
+            return False
+        except Exception as e:
+            logger.error(f"Failed to delete collection: {e}")
+            return False
+    
+    def collection_exists(self, name: str) -> bool:
+        """Check if collection exists."""
+        self._init_client()
+        return self._vectorstore is not None
+    
+    def list_collections(self) -> List[str]:
+        """List collections."""
+        self._init_client()
+        if hasattr(self._vectorstore, 'list_collections'):
+            return self._vectorstore.list_collections()
+        return ["default"]
+    
+    def insert(
+        self,
+        collection: str,
+        documents: List[KnowledgeDocument]
+    ) -> List[str]:
+        """Insert documents."""
+        self._init_client()
+        
+        try:
+            from langchain_core.documents import Document as LCDocument
+        except ImportError:
+            raise ImportError(
+                "langchain-core is required. Install with: pip install langchain-core"
+            )
+        
+        lc_docs = []
+        ids = []
+        for doc in documents:
+            lc_doc = LCDocument(
+                page_content=doc.content,
+                metadata={
+                    **(doc.metadata or {}),
+                    "id": doc.id,
+                    "content_hash": doc.content_hash,
+                    "created_at": doc.created_at,
+                }
+            )
+            lc_docs.append(lc_doc)
+            ids.append(doc.id)
+        
+        if hasattr(self._vectorstore, 'add_documents'):
+            self._vectorstore.add_documents(lc_docs, ids=ids)
+        else:
+            logger.warning("Vectorstore doesn't support add_documents")
+        
+        return ids
+    
+    def upsert(
+        self,
+        collection: str,
+        documents: List[KnowledgeDocument]
+    ) -> List[str]:
+        """Upsert documents."""
+        # Most LangChain vectorstores handle upsert via add_documents
+        return self.insert(collection, documents)
+    
+    def search(
+        self,
+        collection: str,
+        query_embedding: List[float],
+        limit: int = 5,
+        filters: Optional[Dict[str, Any]] = None,
+        score_threshold: Optional[float] = None
+    ) -> List[KnowledgeDocument]:
+        """Search for similar documents."""
+        self._init_client()
+        
+        try:
+            if hasattr(self._vectorstore, 'similarity_search_by_vector'):
+                results = self._vectorstore.similarity_search_by_vector(
+                    query_embedding,
+                    k=limit,
+                    filter=filters
+                )
+            else:
+                logger.warning("Vectorstore doesn't support vector search")
+                return []
+            
+            documents = []
+            for lc_doc in results:
+                metadata = lc_doc.metadata or {}
+                documents.append(KnowledgeDocument(
+                    id=metadata.get("id", ""),
+                    content=lc_doc.page_content,
+                    metadata=metadata,
+                    content_hash=metadata.get("content_hash"),
+                    created_at=metadata.get("created_at", 0)
+                ))
+            
+            return documents
+        except Exception as e:
+            logger.error(f"Search failed: {e}")
+            return []
+    
+    def similarity_search(
+        self,
+        query_text: str,
+        limit: int = 5,
+        filters: Optional[Dict[str, Any]] = None
+    ) -> List[KnowledgeDocument]:
+        """
+        Text-based similarity search (native LangChain method).
+        
+        Args:
+            query_text: Query string
+            limit: Maximum results
+            filters: Metadata filters
+        
+        Returns:
+            List of matching documents
+        """
+        self._init_client()
+        
+        try:
+            results = self._vectorstore.similarity_search(
+                query_text,
+                k=limit,
+                filter=filters
+            )
+            
+            documents = []
+            for lc_doc in results:
+                metadata = lc_doc.metadata or {}
+                documents.append(KnowledgeDocument(
+                    id=metadata.get("id", ""),
+                    content=lc_doc.page_content,
+                    metadata=metadata,
+                    content_hash=metadata.get("content_hash"),
+                    created_at=metadata.get("created_at", 0)
+                ))
+            
+            return documents
+        except Exception as e:
+            logger.error(f"Similarity search failed: {e}")
+            return []
+    
+    def get(
+        self,
+        collection: str,
+        ids: List[str]
+    ) -> List[KnowledgeDocument]:
+        """Get documents by IDs."""
+        self._init_client()
+        
+        if hasattr(self._vectorstore, 'get'):
+            try:
+                results = self._vectorstore.get(ids)
+                documents = []
+                for lc_doc in results:
+                    metadata = lc_doc.metadata or {}
+                    documents.append(KnowledgeDocument(
+                        id=metadata.get("id", ""),
+                        content=lc_doc.page_content,
+                        metadata=metadata
+                    ))
+                return documents
+            except Exception as e:
+                logger.warning(f"Get by ID failed: {e}")
+        
+        return []
+    
+    def delete(
+        self,
+        collection: str,
+        ids: Optional[List[str]] = None,
+        filters: Optional[Dict[str, Any]] = None
+    ) -> int:
+        """Delete documents."""
+        self._init_client()
+        
+        if ids and hasattr(self._vectorstore, 'delete'):
+            try:
+                self._vectorstore.delete(ids)
+                return len(ids)
+            except Exception as e:
+                logger.error(f"Delete failed: {e}")
+        
+        return 0
+    
+    def count(self, collection: str) -> int:
+        """Count documents."""
+        self._init_client()
+        
+        if hasattr(self._vectorstore, '__len__'):
+            return len(self._vectorstore)
+        
+        logger.warning("Vectorstore doesn't support counting")
+        return -1
+    
+    def close(self) -> None:
+        """Close the client."""
+        self._vectorstore = None
+        self._initialized = False

src/praisonai/praisonai/persistence/knowledge/lightrag_adapter.py

@@ -0,0 +1,212 @@
+"""
+LightRAG adapter implementation of KnowledgeStore.
+
+Requires: lightrag
+Install: pip install lightrag
+"""
+
+import logging
+from typing import Any, Dict, List, Optional
+
+from .base import KnowledgeStore, KnowledgeDocument
+
+logger = logging.getLogger(__name__)
+
+
+class LightRAGKnowledgeStore(KnowledgeStore):
+    """
+    LightRAG adapter for knowledge/RAG.
+    
+    Wraps LightRAG for use with PraisonAI's KnowledgeStore interface.
+    
+    Example:
+        store = LightRAGKnowledgeStore(
+            working_dir="./lightrag_data"
+        )
+    """
+    
+    def __init__(
+        self,
+        working_dir: str = "./lightrag_data",
+        llm_model: str = "gpt-4o-mini",
+        embedding_model: str = "text-embedding-3-small",
+        **kwargs
+    ):
+        """
+        Initialize LightRAG adapter.
+        
+        Args:
+            working_dir: Directory for LightRAG data
+            llm_model: LLM model for graph extraction
+            embedding_model: Embedding model name
+            **kwargs: Additional LightRAG options
+        """
+        self.working_dir = working_dir
+        self.llm_model = llm_model
+        self.embedding_model = embedding_model
+        self.kwargs = kwargs
+        
+        self._rag = None
+        self._initialized = False
+    
+    def _init_client(self):
+        """Initialize LightRAG client lazily."""
+        if self._initialized:
+            return
+        
+        try:
+            from lightrag import LightRAG, QueryParam
+            from lightrag.llm import gpt_4o_mini_complete, openai_embedding
+        except ImportError:
+            raise ImportError(
+                "lightrag is required for LightRAG support. "
+                "Install with: pip install lightrag"
+            )
+        
+        self._QueryParam = QueryParam
+        self._rag = LightRAG(
+            working_dir=self.working_dir,
+            llm_model_func=gpt_4o_mini_complete,
+            embedding_func=openai_embedding,
+            **self.kwargs
+        )
+        self._initialized = True
+    
+    def create_collection(
+        self,
+        name: str,
+        dimension: int = 1536,
+        distance: str = "cosine",
+        metadata: Optional[Dict[str, Any]] = None
+    ) -> None:
+        """Create collection (LightRAG uses working directory)."""
+        self._init_client()
+        logger.info(f"LightRAG uses working_dir: {self.working_dir}")
+    
+    def delete_collection(self, name: str) -> bool:
+        """Delete collection data."""
+        import shutil
+        import os
+        
+        try:
+            if os.path.exists(self.working_dir):
+                shutil.rmtree(self.working_dir)
+            return True
+        except Exception as e:
+            logger.error(f"Failed to delete collection: {e}")
+            return False
+    
+    def collection_exists(self, name: str) -> bool:
+        """Check if working directory exists."""
+        import os
+        return os.path.exists(self.working_dir)
+    
+    def list_collections(self) -> List[str]:
+        """List collections (single collection per working_dir)."""
+        import os
+        if os.path.exists(self.working_dir):
+            return ["default"]
+        return []
+    
+    def insert(
+        self,
+        collection: str,
+        documents: List[KnowledgeDocument]
+    ) -> List[str]:
+        """Insert documents into LightRAG."""
+        self._init_client()
+        
+        ids = []
+        for doc in documents:
+            try:
+                self._rag.insert(doc.content)
+                ids.append(doc.id)
+            except Exception as e:
+                logger.warning(f"Failed to insert document {doc.id}: {e}")
+        
+        return ids
+    
+    def upsert(
+        self,
+        collection: str,
+        documents: List[KnowledgeDocument]
+    ) -> List[str]:
+        """Upsert documents (same as insert for LightRAG)."""
+        return self.insert(collection, documents)
+    
+    def search(
+        self,
+        collection: str,
+        query_embedding: List[float],
+        limit: int = 5,
+        filters: Optional[Dict[str, Any]] = None,
+        score_threshold: Optional[float] = None
+    ) -> List[KnowledgeDocument]:
+        """Search using LightRAG's query method."""
+        self._init_client()
+        
+        # LightRAG uses text query, not embeddings directly
+        # This is a limitation - we need the original query text
+        logger.warning(
+            "LightRAG search requires text query, not embeddings. "
+            "Use query() method directly for best results."
+        )
+        return []
+    
+    def query(
+        self,
+        query_text: str,
+        mode: str = "hybrid",
+        limit: int = 5
+    ) -> str:
+        """
+        Query LightRAG with text (native method).
+        
+        Args:
+            query_text: The query string
+            mode: Query mode (naive, local, global, hybrid)
+            limit: Maximum results
+        
+        Returns:
+            Query response as string
+        """
+        self._init_client()
+        
+        try:
+            result = self._rag.query(
+                query_text,
+                param=self._QueryParam(mode=mode, top_k=limit)
+            )
+            return result
+        except Exception as e:
+            logger.error(f"Query failed: {e}")
+            return ""
+    
+    def get(
+        self,
+        collection: str,
+        ids: List[str]
+    ) -> List[KnowledgeDocument]:
+        """Get documents by IDs (not supported by LightRAG)."""
+        logger.warning("LightRAG does not support get by ID")
+        return []
+    
+    def delete(
+        self,
+        collection: str,
+        ids: Optional[List[str]] = None,
+        filters: Optional[Dict[str, Any]] = None
+    ) -> int:
+        """Delete documents (not directly supported)."""
+        logger.warning("LightRAG does not support individual document deletion")
+        return 0
+    
+    def count(self, collection: str) -> int:
+        """Count documents (not directly supported)."""
+        logger.warning("LightRAG does not support document counting")
+        return -1
+    
+    def close(self) -> None:
+        """Close the client."""
+        self._rag = None
+        self._initialized = False

src/praisonai/praisonai/persistence/knowledge/llamaindex_adapter.py

@@ -0,0 +1,256 @@
+"""
+LlamaIndex VectorStore adapter implementation of KnowledgeStore.
+
+Wraps any LlamaIndex VectorStore for use with PraisonAI.
+
+Requires: llama-index-core
+Install: pip install llama-index-core
+"""
+
+import logging
+from typing import Any, Dict, List, Optional
+
+from .base import KnowledgeStore, KnowledgeDocument
+
+logger = logging.getLogger(__name__)
+
+
+class LlamaIndexKnowledgeStore(KnowledgeStore):
+    """
+    LlamaIndex VectorStore adapter for knowledge/RAG.
+    
+    Wraps any LlamaIndex VectorStore for use with PraisonAI's interface.
+    
+    Example:
+        from llama_index.vector_stores.chroma import ChromaVectorStore
+        
+        li_store = ChromaVectorStore(chroma_collection=collection)
+        store = LlamaIndexKnowledgeStore(vector_store=li_store)
+    """
+    
+    def __init__(
+        self,
+        vector_store: Any = None,
+        vector_store_cls: Optional[str] = None,
+        vector_store_kwargs: Optional[Dict[str, Any]] = None,
+    ):
+        """
+        Initialize LlamaIndex adapter.
+        
+        Args:
+            vector_store: Pre-initialized LlamaIndex VectorStore instance
+            vector_store_cls: Class name to import (e.g., "llama_index.vector_stores.chroma.ChromaVectorStore")
+            vector_store_kwargs: Kwargs for vector_store initialization
+        """
+        self._vector_store = vector_store
+        self._vector_store_cls = vector_store_cls
+        self._vector_store_kwargs = vector_store_kwargs or {}
+        self._initialized = vector_store is not None
+    
+    def _init_client(self):
+        """Initialize vector store lazily."""
+        if self._initialized:
+            return
+        
+        if not self._vector_store_cls:
+            raise ValueError(
+                "Either vector_store or vector_store_cls must be provided"
+            )
+        
+        # Dynamic import of vector store class
+        parts = self._vector_store_cls.rsplit(".", 1)
+        if len(parts) == 2:
+            module_name, class_name = parts
+            import importlib
+            module = importlib.import_module(module_name)
+            cls = getattr(module, class_name)
+            self._vector_store = cls(**self._vector_store_kwargs)
+        else:
+            raise ValueError(f"Invalid vector_store_cls: {self._vector_store_cls}")
+        
+        self._initialized = True
+    
+    def create_collection(
+        self,
+        name: str,
+        dimension: int = 1536,
+        distance: str = "cosine",
+        metadata: Optional[Dict[str, Any]] = None
+    ) -> None:
+        """Create collection (handled by underlying vector store)."""
+        self._init_client()
+        logger.info("Collection creation handled by LlamaIndex vector store")
+    
+    def delete_collection(self, name: str) -> bool:
+        """Delete collection."""
+        self._init_client()
+        try:
+            if hasattr(self._vector_store, 'delete'):
+                self._vector_store.delete(delete_all=True)
+                return True
+            logger.warning("Underlying vector store doesn't support delete")
+            return False
+        except Exception as e:
+            logger.error(f"Failed to delete collection: {e}")
+            return False
+    
+    def collection_exists(self, name: str) -> bool:
+        """Check if collection exists."""
+        self._init_client()
+        return self._vector_store is not None
+    
+    def list_collections(self) -> List[str]:
+        """List collections."""
+        self._init_client()
+        return ["default"]
+    
+    def insert(
+        self,
+        collection: str,
+        documents: List[KnowledgeDocument]
+    ) -> List[str]:
+        """Insert documents."""
+        self._init_client()
+        
+        try:
+            from llama_index.core.schema import TextNode
+        except ImportError:
+            raise ImportError(
+                "llama-index-core is required. Install with: pip install llama-index-core"
+            )
+        
+        nodes = []
+        ids = []
+        for doc in documents:
+            node = TextNode(
+                text=doc.content,
+                id_=doc.id,
+                metadata={
+                    **(doc.metadata or {}),
+                    "content_hash": doc.content_hash,
+                    "created_at": doc.created_at,
+                },
+                embedding=doc.embedding
+            )
+            nodes.append(node)
+            ids.append(doc.id)
+        
+        if hasattr(self._vector_store, 'add'):
+            self._vector_store.add(nodes)
+        else:
+            logger.warning("Vector store doesn't support add")
+        
+        return ids
+    
+    def upsert(
+        self,
+        collection: str,
+        documents: List[KnowledgeDocument]
+    ) -> List[str]:
+        """Upsert documents."""
+        return self.insert(collection, documents)
+    
+    def search(
+        self,
+        collection: str,
+        query_embedding: List[float],
+        limit: int = 5,
+        filters: Optional[Dict[str, Any]] = None,
+        score_threshold: Optional[float] = None
+    ) -> List[KnowledgeDocument]:
+        """Search for similar documents."""
+        self._init_client()
+        
+        try:
+            from llama_index.core.vector_stores import VectorStoreQuery
+        except ImportError:
+            raise ImportError(
+                "llama-index-core is required. Install with: pip install llama-index-core"
+            )
+        
+        try:
+            query = VectorStoreQuery(
+                query_embedding=query_embedding,
+                similarity_top_k=limit,
+            )
+            
+            result = self._vector_store.query(query)
+            
+            documents = []
+            for i, node in enumerate(result.nodes or []):
+                score = result.similarities[i] if result.similarities else None
+                if score_threshold and score and score < score_threshold:
+                    continue
+                
+                metadata = node.metadata or {}
+                if score:
+                    metadata["score"] = score
+                
+                documents.append(KnowledgeDocument(
+                    id=node.node_id or node.id_,
+                    content=node.text or node.get_content(),
+                    embedding=node.embedding,
+                    metadata=metadata,
+                    content_hash=metadata.get("content_hash"),
+                    created_at=metadata.get("created_at", 0)
+                ))
+            
+            return documents
+        except Exception as e:
+            logger.error(f"Search failed: {e}")
+            return []
+    
+    def get(
+        self,
+        collection: str,
+        ids: List[str]
+    ) -> List[KnowledgeDocument]:
+        """Get documents by IDs."""
+        self._init_client()
+        
+        documents = []
+        for doc_id in ids:
+            try:
+                if hasattr(self._vector_store, 'get'):
+                    node = self._vector_store.get(doc_id)
+                    if node:
+                        metadata = node.metadata or {}
+                        documents.append(KnowledgeDocument(
+                            id=node.node_id or node.id_,
+                            content=node.text or node.get_content(),
+                            embedding=node.embedding,
+                            metadata=metadata
+                        ))
+            except Exception as e:
+                logger.warning(f"Get by ID failed for {doc_id}: {e}")
+        
+        return documents
+    
+    def delete(
+        self,
+        collection: str,
+        ids: Optional[List[str]] = None,
+        filters: Optional[Dict[str, Any]] = None
+    ) -> int:
+        """Delete documents."""
+        self._init_client()
+        
+        if ids and hasattr(self._vector_store, 'delete'):
+            try:
+                self._vector_store.delete(ref_doc_id=ids[0] if len(ids) == 1 else None)
+                return len(ids)
+            except Exception as e:
+                logger.error(f"Delete failed: {e}")
+        
+        return 0
+    
+    def count(self, collection: str) -> int:
+        """Count documents."""
+        self._init_client()
+        logger.warning("LlamaIndex vector stores don't typically support counting")
+        return -1
+    
+    def close(self) -> None:
+        """Close the client."""
+        self._vector_store = None
+        self._initialized = False

src/praisonai/praisonai/persistence/knowledge/mongodb_vector.py

@@ -0,0 +1,306 @@
+"""
+MongoDB Vector Search implementation of KnowledgeStore.
+
+Requires: pymongo
+Install: pip install pymongo
+"""
+
+import logging
+from typing import Any, Dict, List, Optional
+
+from .base import KnowledgeStore, KnowledgeDocument
+
+logger = logging.getLogger(__name__)
+
+
+class MongoDBVectorKnowledgeStore(KnowledgeStore):
+    """
+    MongoDB Atlas Vector Search store for knowledge/RAG.
+    
+    Uses MongoDB Atlas Vector Search capabilities.
+    
+    Example:
+        store = MongoDBVectorKnowledgeStore(
+            url="mongodb+srv://...",
+            database="praisonai",
+            collection="vectors"
+        )
+    """
+    
+    def __init__(
+        self,
+        url: str = "mongodb://localhost:27017",
+        database: str = "praisonai",
+        collection: str = "vectors",
+        index_name: str = "vector_index",
+        embedding_field: str = "embedding",
+        text_field: str = "content",
+        embedding_dim: int = 1536,
+    ):
+        """
+        Initialize MongoDB Vector store.
+        
+        Args:
+            url: MongoDB connection URL
+            database: Database name
+            collection: Collection name
+            index_name: Vector search index name
+            embedding_field: Field name for embeddings
+            text_field: Field name for text content
+            embedding_dim: Embedding dimension
+        """
+        self.url = url
+        self.database_name = database
+        self.collection_name = collection
+        self.index_name = index_name
+        self.embedding_field = embedding_field
+        self.text_field = text_field
+        self.embedding_dim = embedding_dim
+        
+        self._client = None
+        self._db = None
+        self._collection = None
+        self._initialized = False
+    
+    def _init_client(self):
+        """Initialize MongoDB client lazily."""
+        if self._initialized:
+            return
+        
+        try:
+            from pymongo import MongoClient
+        except ImportError:
+            raise ImportError(
+                "pymongo is required for MongoDB Vector support. "
+                "Install with: pip install pymongo"
+            )
+        
+        self._client = MongoClient(self.url)
+        self._db = self._client[self.database_name]
+        self._collection = self._db[self.collection_name]
+        self._initialized = True
+    
+    def create_collection(
+        self,
+        name: str,
+        dimension: int = 1536,
+        distance: str = "cosine",
+        metadata: Optional[Dict[str, Any]] = None
+    ) -> None:
+        """Create collection (MongoDB creates on first insert)."""
+        self._init_client()
+        # MongoDB creates collections automatically on first insert
+        # Vector search index must be created via Atlas UI or API
+        logger.info(
+            f"Collection '{name}' will be created on first insert. "
+            f"Note: Vector search index must be created via MongoDB Atlas."
+        )
+    
+    def delete_collection(self, name: str) -> bool:
+        """Delete a collection."""
+        self._init_client()
+        try:
+            self._db.drop_collection(name)
+            return True
+        except Exception as e:
+            logger.error(f"Failed to delete collection {name}: {e}")
+            return False
+    
+    def collection_exists(self, name: str) -> bool:
+        """Check if collection exists."""
+        self._init_client()
+        return name in self._db.list_collection_names()
+    
+    def list_collections(self) -> List[str]:
+        """List all collections."""
+        self._init_client()
+        return self._db.list_collection_names()
+    
+    def insert(
+        self,
+        collection: str,
+        documents: List[KnowledgeDocument]
+    ) -> List[str]:
+        """Insert documents into collection."""
+        self._init_client()
+        
+        coll = self._db[collection]
+        ids = []
+        
+        for doc in documents:
+            mongo_doc = {
+                "_id": doc.id,
+                self.text_field: doc.content,
+                "metadata": doc.metadata or {},
+                "content_hash": doc.content_hash,
+                "created_at": doc.created_at,
+            }
+            
+            if doc.embedding:
+                mongo_doc[self.embedding_field] = doc.embedding
+            
+            try:
+                coll.insert_one(mongo_doc)
+                ids.append(doc.id)
+            except Exception as e:
+                logger.warning(f"Failed to insert document {doc.id}: {e}")
+        
+        return ids
+    
+    def upsert(
+        self,
+        collection: str,
+        documents: List[KnowledgeDocument]
+    ) -> List[str]:
+        """Upsert documents into collection."""
+        self._init_client()
+        
+        coll = self._db[collection]
+        ids = []
+        
+        for doc in documents:
+            mongo_doc = {
+                self.text_field: doc.content,
+                "metadata": doc.metadata or {},
+                "content_hash": doc.content_hash,
+                "created_at": doc.created_at,
+            }
+            
+            if doc.embedding:
+                mongo_doc[self.embedding_field] = doc.embedding
+            
+            try:
+                coll.replace_one(
+                    {"_id": doc.id},
+                    {"_id": doc.id, **mongo_doc},
+                    upsert=True
+                )
+                ids.append(doc.id)
+            except Exception as e:
+                logger.warning(f"Failed to upsert document {doc.id}: {e}")
+        
+        return ids
+    
+    def search(
+        self,
+        collection: str,
+        query_embedding: List[float],
+        limit: int = 5,
+        filters: Optional[Dict[str, Any]] = None,
+        score_threshold: Optional[float] = None
+    ) -> List[KnowledgeDocument]:
+        """Search for similar documents using vector search."""
+        self._init_client()
+        
+        coll = self._db[collection]
+        
+        # Build aggregation pipeline for vector search
+        pipeline = [
+            {
+                "$vectorSearch": {
+                    "index": self.index_name,
+                    "path": self.embedding_field,
+                    "queryVector": query_embedding,
+                    "numCandidates": limit * 10,
+                    "limit": limit
+                }
+            },
+            {
+                "$project": {
+                    "_id": 1,
+                    self.text_field: 1,
+                    "metadata": 1,
+                    "content_hash": 1,
+                    "created_at": 1,
+                    "score": {"$meta": "vectorSearchScore"}
+                }
+            }
+        ]
+        
+        # Add filter if provided
+        if filters:
+            pipeline[0]["$vectorSearch"]["filter"] = filters
+        
+        try:
+            results = list(coll.aggregate(pipeline))
+            
+            documents = []
+            for doc in results:
+                score = doc.get("score", 0)
+                if score_threshold and score < score_threshold:
+                    continue
+                
+                documents.append(KnowledgeDocument(
+                    id=str(doc["_id"]),
+                    content=doc.get(self.text_field, ""),
+                    metadata={**(doc.get("metadata") or {}), "score": score},
+                    content_hash=doc.get("content_hash"),
+                    created_at=doc.get("created_at", 0)
+                ))
+            
+            return documents
+        except Exception as e:
+            logger.error(f"Vector search failed: {e}")
+            return []
+    
+    def get(
+        self,
+        collection: str,
+        ids: List[str]
+    ) -> List[KnowledgeDocument]:
+        """Get documents by IDs."""
+        self._init_client()
+        
+        coll = self._db[collection]
+        documents = []
+        
+        for doc_id in ids:
+            try:
+                doc = coll.find_one({"_id": doc_id})
+                if doc:
+                    documents.append(KnowledgeDocument(
+                        id=str(doc["_id"]),
+                        content=doc.get(self.text_field, ""),
+                        embedding=doc.get(self.embedding_field),
+                        metadata=doc.get("metadata"),
+                        content_hash=doc.get("content_hash"),
+                        created_at=doc.get("created_at", 0)
+                    ))
+            except Exception as e:
+                logger.warning(f"Failed to get document {doc_id}: {e}")
+        
+        return documents
+    
+    def delete(
+        self,
+        collection: str,
+        ids: Optional[List[str]] = None,
+        filters: Optional[Dict[str, Any]] = None
+    ) -> int:
+        """Delete documents by IDs or filters."""
+        self._init_client()
+        
+        coll = self._db[collection]
+        
+        if ids:
+            result = coll.delete_many({"_id": {"$in": ids}})
+            return result.deleted_count
+        elif filters:
+            result = coll.delete_many(filters)
+            return result.deleted_count
+        else:
+            return 0
+    
+    def count(self, collection: str) -> int:
+        """Count documents in collection."""
+        self._init_client()
+        return self._db[collection].count_documents({})
+    
+    def close(self) -> None:
+        """Close the connection."""
+        if self._client:
+            self._client.close()
+            self._client = None
+            self._db = None
+            self._collection = None
+            self._initialized = False