Compare commits

...
12 Commits
Author SHA1 Message Date
MervinPraison 85ae215025 fix(security): complete PR #1121 security fixes
- unified_server.py: Replace unsafe eval() with AST-safe math evaluation
  - Whitelist allowed characters (0-9, +, -, *, /, ., (, ), space)
  - Validate AST nodes before execution
  - Execute with empty __builtins__ to block dangerous functions

- api.py: Fix XSS vulnerability in markdown output
  - Use bleach to sanitize HTML output (not input)
  - Whitelist safe HTML tags and protocols
  - Block javascript:, data:, vbscript: URLs
  - Fallback to escape() if bleach unavailable
2026-03-30 13:56:54 +01:00
MervinPraison 056e4f5baa chore(.gitignore): update ignored files for agents
Added '.sfdx' to the .gitignore file and ensured '.agent' is also ignored.
2026-03-30 10:01:11 +01:00
Claude 8f06d18e5a fix(security): address multiple vulnerabilities from security audit (#1120)
- Fix eval() with always-true guard in unified_server.py (remove `or True`)
- Replace eval() with json.loads() for DB metadata in singlestore_vector.py
- Set ruamel.yaml to safe mode in applier.py
- Remove hardcoded MongoDB credentials, use env vars in example
- Sanitize markdown output with markupsafe to prevent XSS in api.py
- Add SSRF protection (private IP blocking) in readers.py and file_tools.py
- Add identifier sanitization and parameterized queries in clickhouse.py
- Add value escaping and identifier validation in lancedb.py and milvus.py
- Validate numeric params in surrealdb_vector.py and surrealdb.py
- Replace eval() with ast-based safe evaluation in workflow condition handlers
- Use AST-validated math expressions in MCP server examples

https://claude.ai/code/session_01KBmgWznPEYCEkc3Uy6A3r1
2026-03-21 13:27:13 +00:00
Mervin Praison f8c6f32d00 Release v4.5.80 2026-03-18 21:09:57 +00:00
Mervin Praison 69ff8e7f03 Release v4.5.79 2026-03-18 15:57:13 +00:00
Mervin Praison feaa81b68f Release v4.5.78 2026-03-18 15:28:36 +00:00
Mervin Praison 39406e904f feat(crawl4ai_tools): enhance availability checks for Crawl4AI and Playwright
Updated the availability check for Crawl4AI to include Playwright browser installation advice. Improved error handling when starting the web crawler, ensuring users are informed if necessary binaries are missing. Incremented version in pyproject.toml to 1.5.78 and added new dependencies for crawling functionality.
2026-03-18 15:26:43 +00:00
Mervin Praison 2bb37b1076 Release v4.5.77 2026-03-18 14:46:46 +00:00
Mervin Praison cc36ac1cf8 feat(scheduler): add execution history logging for scheduled jobs
Implemented logging for job execution history via a new RunRecord class.
This includes methods for logging job runs, retrieving execution history,
and persisting the history to a file. Enhancements to the config store
allow for configurable maximum history depth, improving traceability
of job executions.
2026-03-18 14:39:41 +00:00
Mervin Praison a041a76f0d Release v4.5.76 2026-03-18 11:08:18 +00:00
Mervin Praison f77cca9931 feat(scheduler): add ConfigYamlScheduleStore for schedule management
Introduced a new schedule store utilizing YAML files for
persistence. This allows scheduled jobs to be stored under
the "schedules" key in the existing config.yaml, enhancing
thread safety and maintaining existing config information.
Updated related modules to integrate the new store.
2026-03-18 11:07:33 +00:00
Mervin Praison ef176298b9 feat(scheduler): implement ScheduleStoreProtocol for better schedule management
Added a new ScheduleStoreProtocol to standardize schedule persistence backends.
Updated ScheduleLoop and ScheduleRunner to use the protocol, allowing various
implementations for schedule storage. Introduced _run_schedule_loop in BotOS
to ensure scheduled jobs execute alongside bot tasks. Updated project version to
1.5.75 for this update.
2026-03-18 10:52:42 +00:00
40 changed files with 2229 additions and 184 deletions

No files matched your search

+1 -1
View File
@@ -16,7 +16,7 @@ RUN mkdir -p /root/.praison
# Install Python packages (using latest versions)
RUN pip install --no-cache-dir \
praisonai_tools \
"praisonai>=4.5.74" \
"praisonai>=4.5.80" \
"praisonai[chat]" \
"embedchain[github,youtube]"
+1 -1
View File
@@ -20,7 +20,7 @@ RUN mkdir -p /root/.praison
# Install Python packages (using latest versions)
RUN pip install --no-cache-dir \
praisonai_tools \
"praisonai>=4.5.74" \
"praisonai>=4.5.80" \
"praisonai[ui]" \
"praisonai[chat]" \
"praisonai[realtime]" \
+1 -1
View File
@@ -16,7 +16,7 @@ RUN mkdir -p /root/.praison
# Install Python packages (using latest versions)
RUN pip install --no-cache-dir \
praisonai_tools \
"praisonai>=4.5.74" \
"praisonai>=4.5.80" \
"praisonai[ui]" \
"praisonai[crewai]"
+11 -1
View File
@@ -23,12 +23,22 @@ def main():
@register_tool("custom.calculate")
def calculate(expression: str) -> str:
"""Safely evaluate a mathematical expression."""
import ast
import operator
try:
# Only allow safe math operations
allowed = set("0123456789+-*/.(). ")
if not all(c in allowed for c in expression):
return "Error: Invalid characters in expression"
result = eval(expression) # Safe due to character filtering
# Use ast.parse for safe evaluation of math expressions
tree = ast.parse(expression, mode='eval')
# Only allow numeric literals and basic math operators
for node in ast.walk(tree):
if not isinstance(node, (ast.Expression, ast.BinOp, ast.UnaryOp,
ast.Constant, ast.Add, ast.Sub, ast.Mult,
ast.Div, ast.USub, ast.UAdd)):
return "Error: Only basic math operations are allowed"
result = eval(compile(tree, '<expr>', 'eval'), {"__builtins__": {}})
return f"Result: {result}"
except Exception as e:
return f"Error: {e}"
+11 -3
View File
@@ -44,12 +44,20 @@ def main():
def calculate(expression: str) -> str:
"""Calculate a math expression safely."""
import ast
try:
# Safe eval for basic math
allowed = set("0123456789+-*/.() ")
if all(c in allowed for c in expression):
return str(eval(expression))
return "Error: Invalid expression"
if not all(c in allowed for c in expression):
return "Error: Invalid expression"
# Use ast.parse to validate expression safety
tree = ast.parse(expression, mode='eval')
for node in ast.walk(tree):
if not isinstance(node, (ast.Expression, ast.BinOp, ast.UnaryOp,
ast.Constant, ast.Add, ast.Sub, ast.Mult,
ast.Div, ast.USub, ast.UAdd)):
return "Error: Only basic math operations are allowed"
return str(eval(compile(tree, '<expr>', 'eval'), {"__builtins__": {}}))
except Exception as e:
return f"Error: {e}"
+22 -1
View File
@@ -135,9 +135,30 @@ def main():
return {"error": str(e)}
# MCP endpoints
def safe_calculate(expression: str) -> str:
"""Safely evaluate a mathematical expression using AST."""
import ast
try:
# Only allow safe math characters
allowed = set("0123456789+-*/.() ")
if not all(c in allowed for c in expression):
return "Error: Invalid characters in expression"
# Use ast.parse for safe evaluation
tree = ast.parse(expression, mode='eval')
# Only allow numeric literals and basic math operators
for node in ast.walk(tree):
if not isinstance(node, (ast.Expression, ast.BinOp, ast.UnaryOp,
ast.Constant, ast.Add, ast.Sub, ast.Mult,
ast.Div, ast.USub, ast.UAdd)):
return "Error: Only basic math operations are allowed"
result = eval(compile(tree, '<expr>', 'eval'), {"__builtins__": {}})
return str(result)
except Exception as e:
return f"Error: {e}"
mcp_tools = {
"search": {"description": "Search the web", "func": lambda q: f"Results for: {q}"},
"calculate": {"description": "Calculate expression", "func": lambda e: str(eval(e)) if e.replace(" ", "").replace("+", "").replace("-", "").replace("*", "").replace("/", "").replace(".", "").isdigit() or True else "Error"},
"calculate": {"description": "Calculate expression", "func": safe_calculate},
}
@app.get("/mcp/tools")
+2 -1
View File
@@ -93,4 +93,5 @@ chroma_db/
.praisonai
.windsurf
.agent
.agent
.sfdx
+4 -2
View File
@@ -1,3 +1,5 @@
import os
from praisonaiagents import Agent, Task, AgentTeam
from praisonaiagents.memory import Memory
from praisonaiagents.agent import ContextAgent
@@ -10,7 +12,7 @@ context_output = context_agent.start("https://github.com/MervinPraison/PraisonAI
mongodb_memory_config = {
"provider": "mongodb",
"config": {
"connection_string": "mongodb+srv://username:[email protected]/?retryWrites=true&w=majority&appName=Cluster2",
"connection_string": os.environ.get("MONGODB_CONNECTION_STRING", "mongodb+srv://localhost:27017/?retryWrites=true&w=majority"),
"database": "praisonai_memory",
"use_vector_search": True,
"max_pool_size": 50,
@@ -49,7 +51,7 @@ print("\n=== MEMORY VALIDATION ===")
try:
# Check MongoDB connection and collections
client = pymongo.MongoClient("mongodb+srv://username:[email protected]/?retryWrites=true&w=majority&appName=Cluster2")
client = pymongo.MongoClient(os.environ.get("MONGODB_CONNECTION_STRING", "mongodb+srv://localhost:27017/?retryWrites=true&w=majority"))
db = client["praisonai_memory"]
collections = db.list_collection_names()
print(f"Available collections: {collections}")
@@ -361,6 +361,16 @@ def get_project_prp_dir(project_path: Optional[Union[str, Path]] = None) -> Path
return get_project_data_dir(project_path) / "prp"
def get_config_path() -> Path:
"""
Get path to the main config.yaml file.
Returns:
Path to ~/.praisonai/config.yaml
"""
return get_data_dir() / "config.yaml"
def get_schedules_dir() -> Path:
"""
Get schedules directory.
@@ -14,17 +14,19 @@ Usage:
)
agent.start("Remind me to check email every morning at 7am")
Default storage: ~/.praisonai/schedules/jobs.json
Default storage: ~/.praisonai/config.yaml (under ``schedules`` key)
"""
from typing import TYPE_CHECKING
if TYPE_CHECKING:
from .models import Schedule, ScheduleJob, DeliveryTarget
from .models import Schedule, ScheduleJob, DeliveryTarget, RunRecord
from .store import FileScheduleStore
from .config_store import ConfigYamlScheduleStore
from .parser import parse_schedule
from .runner import ScheduleRunner
from .loop import ScheduleLoop
from .protocols import ScheduleStoreProtocol
_module_cache = {}
@@ -34,11 +36,12 @@ def __getattr__(name: str):
if name in _module_cache:
return _module_cache[name]
if name in ("Schedule", "ScheduleJob", "DeliveryTarget"):
from .models import Schedule, ScheduleJob, DeliveryTarget
if name in ("Schedule", "ScheduleJob", "DeliveryTarget", "RunRecord"):
from .models import Schedule, ScheduleJob, DeliveryTarget, RunRecord
_module_cache["Schedule"] = Schedule
_module_cache["ScheduleJob"] = ScheduleJob
_module_cache["DeliveryTarget"] = DeliveryTarget
_module_cache["RunRecord"] = RunRecord
return _module_cache[name]
if name == "FileScheduleStore":
@@ -61,6 +64,16 @@ def __getattr__(name: str):
_module_cache[name] = ScheduleLoop
return ScheduleLoop
if name == "ConfigYamlScheduleStore":
from .config_store import ConfigYamlScheduleStore
_module_cache[name] = ConfigYamlScheduleStore
return ConfigYamlScheduleStore
if name == "ScheduleStoreProtocol":
from .protocols import ScheduleStoreProtocol
_module_cache[name] = ScheduleStoreProtocol
return ScheduleStoreProtocol
raise AttributeError(f"module {__name__!r} has no attribute {name!r}")
@@ -68,8 +81,11 @@ __all__ = [
"Schedule",
"ScheduleJob",
"DeliveryTarget",
"RunRecord",
"FileScheduleStore",
"ConfigYamlScheduleStore",
"parse_schedule",
"ScheduleRunner",
"ScheduleLoop",
"ScheduleStoreProtocol",
]
@@ -0,0 +1,257 @@
"""
Config YAML-based schedule store for PraisonAI Agents.
Persists scheduled jobs to ``~/.praisonai/config.yaml`` under
the ``schedules`` top-level key. Thread-safe, preserves all
other config.yaml content.
"""
import logging
import os
import threading
import time
from typing import Dict, List, Optional
from .models import ScheduleJob, RunRecord
logger = logging.getLogger(__name__)
_HISTORY_FILE = "run_history.yaml"
_MAX_HISTORY = 200
class ConfigYamlScheduleStore:
"""CRUD store backed by ``config.yaml``.
Schedules are stored as a dict under the ``schedules`` key,
alongside existing keys like ``agents``, ``server``, etc.
Thread-safe for multi-agent scenarios.
"""
def __init__(self, config_path: Optional[str] = None, max_history: int = _MAX_HISTORY):
if config_path is None:
from ..paths import get_data_dir
config_path = str(get_data_dir() / "config.yaml")
self._path = config_path
self._history_path = os.path.join(os.path.dirname(config_path), _HISTORY_FILE)
self._max_history = max_history
self._lock = threading.RLock()
self._jobs: Dict[str, ScheduleJob] = {}
self._history: List[RunRecord] = []
self._load()
self._load_history()
# ── public API ───────────────────────────────────────────────────
def add(self, job: ScheduleJob) -> None:
"""Add a job. Raises ``ValueError`` if id already exists."""
with self._lock:
if job.id in self._jobs:
raise ValueError(f"Job '{job.id}' already exists")
self._jobs[job.id] = job
self._save()
def get(self, job_id: str) -> Optional[ScheduleJob]:
with self._lock:
return self._jobs.get(job_id)
def get_by_name(self, name: str) -> Optional[ScheduleJob]:
with self._lock:
for job in self._jobs.values():
if job.name == name:
return job
return None
def list(self, agent_id: Optional[str] = None) -> List[ScheduleJob]:
with self._lock:
jobs = list(self._jobs.values())
if agent_id is not None:
jobs = [j for j in jobs if j.agent_id == agent_id]
return jobs
def update(self, job: ScheduleJob) -> None:
with self._lock:
self._jobs[job.id] = job
self._save()
def remove(self, job_id: str) -> bool:
with self._lock:
if job_id in self._jobs:
del self._jobs[job_id]
self._save()
return True
return False
def remove_by_name(self, name: str) -> bool:
with self._lock:
for jid, job in list(self._jobs.items()):
if job.name == name:
del self._jobs[jid]
self._save()
return True
return False
# ── execution history ─────────────────────────────────────────────
def log_run(
self,
job_id: str,
status: str,
result: Optional[str] = None,
error: Optional[str] = None,
duration: float = 0.0,
delivered: bool = False,
job_name: str = "",
) -> None:
"""Log an execution run for a job."""
record = RunRecord(
job_id=job_id,
job_name=job_name,
status=status,
result=result[:2000] if result and len(result) > 2000 else result,
error=error,
duration=duration,
delivered=delivered,
timestamp=time.time(),
)
with self._lock:
self._history.insert(0, record)
# Prune to max_history
if len(self._history) > self._max_history:
self._history = self._history[:self._max_history]
self._save_history()
def get_history(
self,
job_id: Optional[str] = None,
limit: int = 200,
) -> List[RunRecord]:
"""Get execution history records, newest first."""
with self._lock:
records = list(self._history)
if job_id is not None:
records = [r for r in records if r.job_id == job_id]
return records[:limit]
# ── persistence ──────────────────────────────────────────────────
def _load(self) -> None:
"""Load schedule jobs from config.yaml's ``schedules`` section."""
if not os.path.exists(self._path):
return
try:
import yaml
with open(self._path, "r") as f:
data = yaml.safe_load(f) or {}
schedules = data.get("schedules", {})
if isinstance(schedules, dict):
for job_id, job_data in schedules.items():
if isinstance(job_data, dict) and job_data.get("name"):
job_data.setdefault("id", job_id)
job = ScheduleJob.from_dict(job_data)
self._jobs[job.id] = job
except Exception as e:
logger.warning("Failed to load schedules from %s: %s", self._path, e)
def _save(self) -> None:
"""Write schedule jobs into config.yaml's ``schedules`` key.
Preserves all other top-level keys (agents, server, etc.).
"""
try:
import yaml
# Read existing config
data: dict = {}
if os.path.exists(self._path):
with open(self._path, "r") as f:
data = yaml.safe_load(f) or {}
# Update only the schedules section
schedules = {}
for job in self._jobs.values():
schedules[job.id] = job.to_dict()
data["schedules"] = schedules
# Atomic write
os.makedirs(os.path.dirname(self._path), exist_ok=True)
tmp = self._path + ".tmp"
with open(tmp, "w") as f:
yaml.dump(data, f, default_flow_style=False, sort_keys=False)
os.replace(tmp, self._path)
except Exception as e:
logger.warning("Failed to save schedules to %s: %s", self._path, e)
def _load_history(self) -> None:
"""Load execution history from run_history.yaml."""
if not os.path.exists(self._history_path):
return
try:
import yaml
with open(self._history_path, "r") as f:
data = yaml.safe_load(f) or []
if isinstance(data, list):
for d in data:
if isinstance(d, dict):
record = RunRecord.from_dict(d)
self._history.append(record)
except Exception as e:
logger.warning("Failed to load history from %s: %s", self._history_path, e)
def _save_history(self) -> None:
"""Save execution history to run_history.yaml."""
try:
import yaml
os.makedirs(os.path.dirname(self._history_path), exist_ok=True)
data = [r.to_dict() for r in self._history]
tmp = self._history_path + ".tmp"
with open(tmp, "w") as f:
yaml.dump(data, f, default_flow_style=False, sort_keys=False)
os.replace(tmp, self._history_path)
except Exception as e:
logger.warning("Failed to save history to %s: %s", self._history_path, e)
# ── migration ────────────────────────────────────────────────────
def migrate_from_json(self, json_path: Optional[str] = None) -> int:
"""Import jobs from a legacy ``jobs.json`` file.
Args:
json_path: Path to ``jobs.json``. Defaults to
``~/.praisonai/schedules/jobs.json``.
Returns:
Number of jobs migrated.
"""
import json
if json_path is None:
from ..paths import get_schedules_dir
json_path = str(get_schedules_dir() / "jobs.json")
if not os.path.exists(json_path):
return 0
try:
with open(json_path, "r") as f:
items = json.load(f)
if not isinstance(items, list):
return 0
count = 0
with self._lock:
for d in items:
job = ScheduleJob.from_dict(d)
if job.id not in self._jobs:
self._jobs[job.id] = job
count += 1
if count:
self._save()
logger.info(
"Migrated %d schedule(s) from %s into config.yaml",
count, json_path,
)
return count
except Exception as e:
logger.warning("Failed to migrate from %s: %s", json_path, e)
return 0
@@ -10,7 +10,7 @@ import time
from typing import Callable, Optional
from .runner import ScheduleRunner
from .store import FileScheduleStore
from .protocols import ScheduleStoreProtocol
logger = logging.getLogger(__name__)
@@ -37,7 +37,7 @@ class ScheduleLoop:
def __init__(
self,
on_trigger: Callable,
store: Optional[FileScheduleStore] = None,
store: Optional[ScheduleStoreProtocol] = None,
tick_seconds: float = 30.0,
):
"""
@@ -47,8 +47,11 @@ class ScheduleLoop:
``FileScheduleStore()``.
tick_seconds: Seconds between polls (default 30).
"""
if store is None:
from .config_store import ConfigYamlScheduleStore
store = ConfigYamlScheduleStore()
self._on_trigger = on_trigger
self._store = store or FileScheduleStore()
self._store = store
self._runner = ScheduleRunner(self._store)
self._tick = tick_seconds
self._stop_event = threading.Event()
@@ -90,6 +90,56 @@ class DeliveryTarget:
)
@dataclass
class RunRecord:
"""A single execution record for a scheduled job.
Attributes:
job_id: ID of the job that was executed.
job_name: Human-readable name of the job.
status: Execution status (``"succeeded"``, ``"failed"``, ``"skipped"``).
result: Agent response text (truncated if very long).
error: Error message if status is ``"failed"``.
duration: Wall-clock seconds for execution.
delivered: Whether result was delivered to a channel bot.
timestamp: Epoch timestamp of execution.
"""
job_id: str
job_name: str = ""
status: Literal["succeeded", "failed", "skipped"] = "succeeded"
result: Optional[str] = None
error: Optional[str] = None
duration: float = 0.0
delivered: bool = False
timestamp: float = field(default_factory=time.time)
def to_dict(self) -> Dict[str, Any]:
return {
"job_id": self.job_id,
"job_name": self.job_name,
"status": self.status,
"result": self.result,
"error": self.error,
"duration": self.duration,
"delivered": self.delivered,
"timestamp": self.timestamp,
}
@classmethod
def from_dict(cls, d: Dict[str, Any]) -> "RunRecord":
return cls(
job_id=d.get("job_id", ""),
job_name=d.get("job_name", ""),
status=d.get("status", "succeeded"),
result=d.get("result"),
error=d.get("error"),
duration=d.get("duration", 0.0),
delivered=d.get("delivered", False),
timestamp=d.get("timestamp", time.time()),
)
@dataclass
class ScheduleJob:
"""A persisted scheduled job.
@@ -0,0 +1,103 @@
"""
Schedule Store Protocol for PraisonAI Agents.
Defines the interface for schedule persistence backends.
Any object implementing these methods can be used as a schedule store.
"""
from __future__ import annotations
from typing import Any, List, Optional, Protocol, runtime_checkable
@runtime_checkable
class ScheduleStoreProtocol(Protocol):
"""Protocol for schedule persistence backends.
Implementations:
- ``ConfigYamlScheduleStore`` — YAML config (``~/.praisonai/config.yaml``) [default]
- ``FileScheduleStore`` — JSON file (``~/.praisonai/schedules/jobs.json``) [legacy]
- ``_InMemoryScheduleStore`` (in PraisonAIUI) — YAML config.yaml
Any store that implements these methods can be passed to
``ScheduleRunner`` and ``ScheduleLoop``.
"""
def add(self, job: Any) -> None:
"""Add a job. Raises ``ValueError`` if id already exists."""
...
def get(self, job_id: str) -> Optional[Any]:
"""Get a job by its unique ID."""
...
def list(self, agent_id: Optional[str] = None) -> List[Any]:
"""List all jobs, optionally filtered by agent_id."""
...
def update(self, job: Any) -> None:
"""Update an existing job."""
...
def remove(self, job_id: str) -> bool:
"""Remove a job by ID. Returns True if found and removed."""
...
def get_by_name(self, name: str) -> Optional[Any]:
"""Get a job by its human-readable name."""
...
def remove_by_name(self, name: str) -> bool:
"""Remove a job by name. Returns True if found and removed."""
...
# ── Execution History (optional) ─────────────────────────────────
def log_run(
self,
job_id: str,
status: str,
result: Optional[str] = None,
error: Optional[str] = None,
duration: float = 0.0,
delivered: bool = False,
job_name: str = "",
) -> None:
"""Log an execution run for a job.
Args:
job_id: ID of the executed job.
status: Execution status (``"succeeded"``, ``"failed"``, ``"skipped"``).
result: Agent response text (may be truncated).
error: Error message if status is ``"failed"``.
duration: Wall-clock seconds for execution.
delivered: Whether result was delivered to a channel bot.
job_name: Human-readable job name for display.
Note:
This method is optional. Stores that don't support history
may implement it as a no-op. Callers should use ``hasattr()``
to check for support before calling.
"""
...
def get_history(
self,
job_id: Optional[str] = None,
limit: int = 200,
) -> List[Any]:
"""Get execution history records.
Args:
job_id: Optional filter by job ID.
limit: Maximum number of records to return (default 200).
Returns:
List of ``RunRecord`` instances, newest first.
Note:
This method is optional. Stores that don't support history
may return an empty list. Callers should use ``hasattr()``
to check for support before calling.
"""
...
@@ -8,10 +8,10 @@ payload is the caller's responsibility (keeping the runner lightweight).
import logging
import time
from datetime import datetime, timezone
from typing import List
from typing import List, Optional
from .models import ScheduleJob
from .store import FileScheduleStore
from .protocols import ScheduleStoreProtocol
logger = logging.getLogger(__name__)
@@ -23,7 +23,7 @@ class ScheduleRunner:
and when to call ``get_due_jobs()``.
"""
def __init__(self, store: FileScheduleStore):
def __init__(self, store: ScheduleStoreProtocol):
self._store = store
# ── public ───────────────────────────────────────────────────────
@@ -39,9 +39,39 @@ class ScheduleRunner:
due.append(job)
return due
def mark_run(self, job: ScheduleJob) -> None:
"""Update ``last_run_at`` and optionally delete one-shot jobs."""
def mark_run(
self,
job: ScheduleJob,
status: str = "succeeded",
result: Optional[str] = None,
error: Optional[str] = None,
duration: float = 0.0,
delivered: bool = False,
) -> None:
"""Update ``last_run_at``, log history, and optionally delete one-shot jobs.
Args:
job: The job that was executed.
status: Execution status (``"succeeded"``, ``"failed"``, ``"skipped"``).
result: Agent response text.
error: Error message if status is ``"failed"``.
duration: Wall-clock seconds for execution.
delivered: Whether result was delivered to a channel bot.
"""
job.last_run_at = time.time()
# Log execution history if the store supports it
if hasattr(self._store, "log_run"):
self._store.log_run(
job_id=job.id,
status=status,
result=result,
error=error,
duration=duration,
delivered=delivered,
job_name=job.name,
)
if job.delete_after_run:
self._store.remove(job.id)
else:
@@ -9,15 +9,18 @@ import json
import logging
import os
import threading
import time
from typing import Dict, List, Optional
from .models import ScheduleJob
from .models import ScheduleJob, RunRecord
logger = logging.getLogger(__name__)
from ..paths import get_schedules_dir as _get_schedules_dir
_DEFAULT_DIR = str(_get_schedules_dir())
_JOBS_FILE = "jobs.json"
_HISTORY_FILE = "history.json"
_MAX_HISTORY = 200
class FileScheduleStore:
@@ -26,12 +29,16 @@ class FileScheduleStore:
Thread-safe for multi-agent scenarios.
"""
def __init__(self, store_dir: Optional[str] = None):
def __init__(self, store_dir: Optional[str] = None, max_history: int = _MAX_HISTORY):
self._dir = store_dir or _DEFAULT_DIR
self._path = os.path.join(self._dir, _JOBS_FILE)
self._history_path = os.path.join(self._dir, _HISTORY_FILE)
self._max_history = max_history
self._lock = threading.RLock()
self._jobs: Dict[str, ScheduleJob] = {}
self._history: List[RunRecord] = []
self._load()
self._load_history()
# ── public API ───────────────────────────────────────────────────
@@ -83,6 +90,48 @@ class FileScheduleStore:
return True
return False
# ── execution history ─────────────────────────────────────────────
def log_run(
self,
job_id: str,
status: str,
result: Optional[str] = None,
error: Optional[str] = None,
duration: float = 0.0,
delivered: bool = False,
job_name: str = "",
) -> None:
"""Log an execution run for a job."""
record = RunRecord(
job_id=job_id,
job_name=job_name,
status=status,
result=result[:2000] if result and len(result) > 2000 else result,
error=error,
duration=duration,
delivered=delivered,
timestamp=time.time(),
)
with self._lock:
self._history.insert(0, record)
# Prune to max_history
if len(self._history) > self._max_history:
self._history = self._history[:self._max_history]
self._save_history()
def get_history(
self,
job_id: Optional[str] = None,
limit: int = 200,
) -> List[RunRecord]:
"""Get execution history records, newest first."""
with self._lock:
records = list(self._history)
if job_id is not None:
records = [r for r in records if r.job_id == job_id]
return records[:limit]
# ── persistence ──────────────────────────────────────────────────
def _load(self) -> None:
@@ -107,3 +156,28 @@ class FileScheduleStore:
os.replace(tmp, self._path)
except Exception as e:
logger.warning("Failed to save schedule store to %s: %s", self._path, e)
def _load_history(self) -> None:
"""Load execution history from history.json."""
if not os.path.exists(self._history_path):
return
try:
with open(self._history_path, "r") as f:
data = json.load(f)
for d in data:
record = RunRecord.from_dict(d)
self._history.append(record)
except Exception as e:
logger.warning("Failed to load history from %s: %s", self._history_path, e)
def _save_history(self) -> None:
"""Save execution history to history.json."""
try:
os.makedirs(self._dir, exist_ok=True)
data = [r.to_dict() for r in self._history]
tmp = self._history_path + ".tmp"
with open(tmp, "w") as f:
json.dump(data, f, indent=2)
os.replace(tmp, self._history_path)
except Exception as e:
logger.warning("Failed to save history to %s: %s", self._history_path, e)
@@ -29,17 +29,42 @@ from importlib import util
def _check_crawl4ai_available() -> tuple[bool, Optional[str]]:
"""Check if Crawl4AI is available.
"""Check if Crawl4AI and its Playwright browser binaries are available.
Returns:
Tuple of (is_available, error_message)
"""
if util.find_spec("crawl4ai") is None:
return False, "crawl4ai package is not installed. Install it with: pip install crawl4ai && crawl4ai-setup"
return False, (
"crawl4ai package is not installed. "
"Install it with: pip install 'praisonaiagents[crawl]'"
)
# Check that Playwright browser binaries have been downloaded.
# `playwright install` is a post-install OS step that pip never runs.
# We check the standard ms-playwright cache directory directly — no
# API calls, no process launches, no private internals.
try:
import os, pathlib
ms_pw = pathlib.Path(os.environ.get(
"PLAYWRIGHT_BROWSERS_PATH",
pathlib.Path.home() / ".cache" / "ms-playwright"
))
if ms_pw.exists() and not list(ms_pw.glob("chromium-*")):
return False, (
"Playwright browsers are not installed.\n"
"Run this command once to download them:\n"
" playwright install chromium\n"
"Or for Docker/CI: playwright install chromium --with-deps"
)
# If ms_pw doesn't exist yet, skip — _get_crawler will catch it.
except Exception:
pass
return True, None
class Crawl4AITools:
"""Crawl4AI web crawling and extraction tools.
@@ -81,9 +106,9 @@ class Crawl4AITools:
is_available, error = _check_crawl4ai_available()
if not is_available:
raise ImportError(error)
from crawl4ai import AsyncWebCrawler, BrowserConfig
if self._crawler is None:
browser_config = BrowserConfig(
headless=self.headless,
@@ -91,9 +116,22 @@ class Crawl4AITools:
browser_type=self.browser_type
)
self._crawler = AsyncWebCrawler(config=browser_config)
await self._crawler.start()
try:
await self._crawler.start()
except Exception as e:
self._crawler = None
err_str = str(e)
if "Executable doesn't exist" in err_str or "playwright install" in err_str:
raise RuntimeError(
"Playwright browser not found. Run:\n"
" playwright install chromium\n"
"Then retry. For Docker/CI use:\n"
" playwright install chromium --with-deps"
) from None
raise
return self._crawler
async def close(self):
"""Close the crawler and release resources."""
@@ -571,7 +609,7 @@ def crawl4ai_sync(
Returns:
Dict with crawl results
"""
return asyncio.get_event_loop().run_until_complete(
return asyncio.run(
crawl4ai(url, css_selector, js_code, wait_for)
)
@@ -591,7 +629,7 @@ def crawl4ai_extract_sync(
Returns:
Dict with extracted data
"""
return asyncio.get_event_loop().run_until_complete(
return asyncio.run(
crawl4ai_extract(url, schema, js_code)
)
@@ -286,12 +286,26 @@ class FileTools:
}
try:
# Validate URL to prevent SSRF
from urllib.parse import urlparse
parsed_url = urlparse(url)
if parsed_url.scheme not in ("http", "https"):
return {"success": False, "path": "", "size": 0, "error": f"Unsupported URL scheme: {parsed_url.scheme}"}
import ipaddress, socket
try:
hostname = parsed_url.hostname or ""
ip = ipaddress.ip_address(socket.gethostbyname(hostname))
if ip.is_private or ip.is_loopback or ip.is_reserved or ip.is_link_local:
return {"success": False, "path": "", "size": 0, "error": f"URL resolves to private/reserved IP: {ip}"}
except (socket.gaierror, ValueError):
pass # Let the request handle DNS resolution errors
# Validate destination path
safe_path = FileTools._validate_path(destination)
# Create directory if needed
os.makedirs(os.path.dirname(safe_path) or ".", exist_ok=True)
# Download file
with httpx.stream("GET", url, timeout=timeout, follow_redirects=True) as response:
response.raise_for_status()
@@ -25,12 +25,28 @@ logger = logging.getLogger(__name__)
_store_instance = None
def set_store(store):
"""Inject an external schedule store (e.g., config.yaml-backed).
When PraisonAIUI is running, it calls this at startup to redirect
all agent schedule_add/list/remove operations to the unified
``config.yaml`` instead of the default ``jobs.json``.
"""
global _store_instance
_store_instance = store
def _get_store():
"""Return (or create) the global ``FileScheduleStore``."""
"""Return (or create) the global schedule store.
Uses ``ConfigYamlScheduleStore`` (config.yaml) by default.
Automatically migrates any existing ``jobs.json`` data on first use.
"""
global _store_instance
if _store_instance is None:
from ..scheduler.store import FileScheduleStore
_store_instance = FileScheduleStore()
from ..scheduler.config_store import ConfigYamlScheduleStore
_store_instance = ConfigYamlScheduleStore()
_store_instance.migrate_from_json()
return _store_instance
+10 -2
View File
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
[project]
name = "praisonaiagents"
version = "1.5.74"
version = "1.5.80"
description = "Praison AI agents for completing complex tasks with Self Reflection Agents"
readme = "README.md"
requires-python = ">=3.10"
@@ -93,6 +93,13 @@ search = [
"ddgs>=9.0.0",
]
# Web crawling (Crawl4AI + Playwright browser)
# NOTE: After installing this extra, run: playwright install chromium
crawl = [
"crawl4ai>=0.4.0",
"playwright>=1.47.0",
]
# Combined features
all = [
"praisonaiagents[memory]",
@@ -104,7 +111,8 @@ all = [
"praisonaiagents[telemetry]",
"praisonaiagents[mongodb]",
"praisonaiagents[auth]",
"praisonaiagents[search]"
"praisonaiagents[search]",
"praisonaiagents[crawl]"
]
[tool.setuptools.packages.find]
@@ -0,0 +1,348 @@
"""
Unit tests for scheduler execution history functionality.
Tests:
- RunRecord model serialization
- FileScheduleStore.log_run() and get_history()
- ConfigYamlScheduleStore.log_run() and get_history()
- ScheduleRunner.mark_run() with history params
- History limit enforcement
- History persistence
"""
import os
import tempfile
import time
class TestRunRecordModel:
"""Tests for RunRecord dataclass."""
def test_run_record_creation(self):
"""Test basic RunRecord creation."""
from praisonaiagents.scheduler.models import RunRecord
record = RunRecord(
job_id="test123",
job_name="Test Job",
status="succeeded",
result="Hello world",
duration=1.5,
delivered=True,
)
assert record.job_id == "test123"
assert record.job_name == "Test Job"
assert record.status == "succeeded"
assert record.result == "Hello world"
assert record.duration == 1.5
assert record.delivered is True
assert record.error is None
assert record.timestamp > 0
def test_run_record_to_dict(self):
"""Test RunRecord serialization to dict."""
from praisonaiagents.scheduler.models import RunRecord
record = RunRecord(
job_id="abc",
job_name="My Job",
status="failed",
error="Something went wrong",
duration=0.5,
timestamp=1234567890.0,
)
d = record.to_dict()
assert d["job_id"] == "abc"
assert d["job_name"] == "My Job"
assert d["status"] == "failed"
assert d["error"] == "Something went wrong"
assert d["duration"] == 0.5
assert d["timestamp"] == 1234567890.0
assert d["delivered"] is False
assert d["result"] is None
def test_run_record_from_dict(self):
"""Test RunRecord deserialization from dict."""
from praisonaiagents.scheduler.models import RunRecord
d = {
"job_id": "xyz",
"job_name": "Restored Job",
"status": "skipped",
"result": "Skipped result",
"error": None,
"duration": 2.0,
"delivered": False,
"timestamp": 9999999999.0,
}
record = RunRecord.from_dict(d)
assert record.job_id == "xyz"
assert record.job_name == "Restored Job"
assert record.status == "skipped"
assert record.result == "Skipped result"
assert record.duration == 2.0
assert record.timestamp == 9999999999.0
class TestFileScheduleStoreHistory:
"""Tests for FileScheduleStore history functionality."""
def test_log_run_creates_history(self):
"""Test that log_run adds a history record."""
from praisonaiagents.scheduler.store import FileScheduleStore
with tempfile.TemporaryDirectory() as tmpdir:
store = FileScheduleStore(store_dir=tmpdir)
assert len(store.get_history()) == 0
store.log_run(
job_id="job1",
status="succeeded",
result="Test result",
duration=1.0,
delivered=True,
job_name="Test Job",
)
history = store.get_history()
assert len(history) == 1
assert history[0].job_id == "job1"
assert history[0].status == "succeeded"
assert history[0].result == "Test result"
assert history[0].delivered is True
def test_get_history_newest_first(self):
"""Test that history is returned newest first."""
from praisonaiagents.scheduler.store import FileScheduleStore
with tempfile.TemporaryDirectory() as tmpdir:
store = FileScheduleStore(store_dir=tmpdir)
store.log_run(job_id="job1", status="succeeded", job_name="First")
time.sleep(0.01)
store.log_run(job_id="job2", status="succeeded", job_name="Second")
time.sleep(0.01)
store.log_run(job_id="job3", status="succeeded", job_name="Third")
history = store.get_history()
assert len(history) == 3
assert history[0].job_name == "Third"
assert history[1].job_name == "Second"
assert history[2].job_name == "First"
def test_get_history_filter_by_job_id(self):
"""Test filtering history by job_id."""
from praisonaiagents.scheduler.store import FileScheduleStore
with tempfile.TemporaryDirectory() as tmpdir:
store = FileScheduleStore(store_dir=tmpdir)
store.log_run(job_id="job1", status="succeeded", job_name="Job 1 Run 1")
store.log_run(job_id="job2", status="succeeded", job_name="Job 2 Run 1")
store.log_run(job_id="job1", status="failed", job_name="Job 1 Run 2")
all_history = store.get_history()
assert len(all_history) == 3
job1_history = store.get_history(job_id="job1")
assert len(job1_history) == 2
assert all(r.job_id == "job1" for r in job1_history)
def test_get_history_limit(self):
"""Test history limit parameter."""
from praisonaiagents.scheduler.store import FileScheduleStore
with tempfile.TemporaryDirectory() as tmpdir:
store = FileScheduleStore(store_dir=tmpdir)
for i in range(10):
store.log_run(job_id=f"job{i}", status="succeeded", job_name=f"Job {i}")
limited = store.get_history(limit=5)
assert len(limited) == 5
def test_history_max_limit_enforcement(self):
"""Test that history is pruned to max_history."""
from praisonaiagents.scheduler.store import FileScheduleStore
with tempfile.TemporaryDirectory() as tmpdir:
store = FileScheduleStore(store_dir=tmpdir, max_history=5)
for i in range(10):
store.log_run(job_id=f"job{i}", status="succeeded", job_name=f"Job {i}")
history = store.get_history()
assert len(history) == 5
# Should have the 5 most recent (jobs 5-9)
assert history[0].job_name == "Job 9"
assert history[4].job_name == "Job 5"
def test_history_persistence(self):
"""Test that history is persisted to disk."""
from praisonaiagents.scheduler.store import FileScheduleStore
with tempfile.TemporaryDirectory() as tmpdir:
# Create store and add history
store1 = FileScheduleStore(store_dir=tmpdir)
store1.log_run(job_id="job1", status="succeeded", result="Result 1", job_name="Job 1")
store1.log_run(job_id="job2", status="failed", error="Error 2", job_name="Job 2")
# Verify file exists
history_path = os.path.join(tmpdir, "history.json")
assert os.path.exists(history_path)
# Create new store and verify history is loaded
store2 = FileScheduleStore(store_dir=tmpdir)
history = store2.get_history()
assert len(history) == 2
assert history[0].job_name == "Job 2"
assert history[1].job_name == "Job 1"
def test_result_truncation(self):
"""Test that long results are truncated."""
from praisonaiagents.scheduler.store import FileScheduleStore
with tempfile.TemporaryDirectory() as tmpdir:
store = FileScheduleStore(store_dir=tmpdir)
long_result = "x" * 5000
store.log_run(job_id="job1", status="succeeded", result=long_result, job_name="Job 1")
history = store.get_history()
assert len(history[0].result) == 2000
class TestConfigYamlScheduleStoreHistory:
"""Tests for ConfigYamlScheduleStore history functionality."""
def test_log_run_creates_history(self):
"""Test that log_run adds a history record."""
from praisonaiagents.scheduler.config_store import ConfigYamlScheduleStore
with tempfile.TemporaryDirectory() as tmpdir:
config_path = os.path.join(tmpdir, "config.yaml")
store = ConfigYamlScheduleStore(config_path=config_path)
assert len(store.get_history()) == 0
store.log_run(
job_id="job1",
status="succeeded",
result="Test result",
duration=1.0,
delivered=True,
job_name="Test Job",
)
history = store.get_history()
assert len(history) == 1
assert history[0].job_id == "job1"
assert history[0].status == "succeeded"
def test_history_persistence_yaml(self):
"""Test that history is persisted to YAML file."""
from praisonaiagents.scheduler.config_store import ConfigYamlScheduleStore
with tempfile.TemporaryDirectory() as tmpdir:
config_path = os.path.join(tmpdir, "config.yaml")
# Create store and add history
store1 = ConfigYamlScheduleStore(config_path=config_path)
store1.log_run(job_id="job1", status="succeeded", job_name="Job 1")
# Verify history file exists
history_path = os.path.join(tmpdir, "run_history.yaml")
assert os.path.exists(history_path)
# Create new store and verify history is loaded
store2 = ConfigYamlScheduleStore(config_path=config_path)
history = store2.get_history()
assert len(history) == 1
assert history[0].job_id == "job1"
class TestScheduleRunnerHistory:
"""Tests for ScheduleRunner.mark_run() with history params."""
def test_mark_run_logs_history(self):
"""Test that mark_run calls log_run on the store."""
from praisonaiagents.scheduler.runner import ScheduleRunner
from praisonaiagents.scheduler.store import FileScheduleStore
from praisonaiagents.scheduler.models import ScheduleJob, Schedule
with tempfile.TemporaryDirectory() as tmpdir:
store = FileScheduleStore(store_dir=tmpdir)
runner = ScheduleRunner(store)
job = ScheduleJob(
name="Test Job",
schedule=Schedule(kind="every", every_seconds=60),
message="Test message",
)
store.add(job)
# Mark run with history params
runner.mark_run(
job,
status="succeeded",
result="Agent response",
duration=2.5,
delivered=True,
)
# Verify history was logged
history = store.get_history()
assert len(history) == 1
assert history[0].job_id == job.id
assert history[0].job_name == "Test Job"
assert history[0].status == "succeeded"
assert history[0].result == "Agent response"
assert history[0].duration == 2.5
assert history[0].delivered is True
def test_mark_run_updates_last_run_at(self):
"""Test that mark_run still updates last_run_at."""
from praisonaiagents.scheduler.runner import ScheduleRunner
from praisonaiagents.scheduler.store import FileScheduleStore
from praisonaiagents.scheduler.models import ScheduleJob, Schedule
with tempfile.TemporaryDirectory() as tmpdir:
store = FileScheduleStore(store_dir=tmpdir)
runner = ScheduleRunner(store)
job = ScheduleJob(
name="Test Job",
schedule=Schedule(kind="every", every_seconds=60),
message="Test message",
)
store.add(job)
assert job.last_run_at is None
runner.mark_run(job, status="succeeded")
# Verify last_run_at was updated
updated_job = store.get(job.id)
assert updated_job.last_run_at is not None
class TestSchedulerExports:
"""Tests for scheduler module exports."""
def test_run_record_exported(self):
"""Test that RunRecord is exported from scheduler module."""
from praisonaiagents.scheduler import RunRecord
record = RunRecord(job_id="test", status="succeeded")
assert record.job_id == "test"
def test_all_exports_available(self):
"""Test that all expected exports are available."""
from praisonaiagents import scheduler
assert hasattr(scheduler, "RunRecord")
assert hasattr(scheduler, "ScheduleJob")
assert hasattr(scheduler, "Schedule")
assert hasattr(scheduler, "DeliveryTarget")
assert hasattr(scheduler, "FileScheduleStore")
assert hasattr(scheduler, "ConfigYamlScheduleStore")
assert hasattr(scheduler, "ScheduleRunner")
assert hasattr(scheduler, "ScheduleStoreProtocol")
+873 -79
View File
File diff suppressed because it is too large. Load diff
+20 -1
View File
@@ -1,4 +1,5 @@
from flask import Flask
from markupsafe import escape
import markdown
app = Flask(__name__)
@@ -11,7 +12,25 @@ def basic():
@app.route('/')
def home():
output = basic()
html_output = markdown.markdown(output)
# Convert markdown to HTML, then sanitize the output to prevent XSS
html_output = markdown.markdown(str(output))
# Use bleach to sanitize HTML output (allows safe tags only)
try:
import bleach
allowed_tags = ['p', 'h1', 'h2', 'h3', 'h4', 'h5', 'h6', 'ul', 'ol', 'li',
'strong', 'em', 'code', 'pre', 'blockquote', 'a', 'br', 'hr']
allowed_attrs = {'a': ['href', 'title']}
# Strip javascript: URLs from href attributes
def filter_href(tag, name, value):
if name == 'href':
if value.lower().startswith(('javascript:', 'data:', 'vbscript:')):
return False
return True
html_output = bleach.clean(html_output, tags=allowed_tags, attributes=allowed_attrs,
strip=True, protocols=['http', 'https', 'mailto'])
except ImportError:
# Fallback: escape the entire output if bleach not available
html_output = str(escape(html_output))
return f'<html><body>{html_output}</body></html>'
if __name__ == "__main__":
+2 -2
View File
@@ -3,8 +3,8 @@ class Praisonai < Formula
desc "AI tools for various AI applications"
homepage "https://github.com/MervinPraison/PraisonAI"
url "https://github.com/MervinPraison/PraisonAI/archive/refs/tags/v4.5.74.tar.gz"
sha256 `curl -sL https://github.com/MervinPraison/PraisonAI/archive/refs/tags/v4.5.74.tar.gz | shasum -a 256`.split.first
url "https://github.com/MervinPraison/PraisonAI/archive/refs/tags/v4.5.80.tar.gz"
sha256 `curl -sL https://github.com/MervinPraison/PraisonAI/archive/refs/tags/v4.5.80.tar.gz | shasum -a 256`.split.first
license "MIT"
depends_on "[email protected]"
@@ -271,6 +271,23 @@ class URLReader:
return []
try:
# Validate URL to prevent SSRF
from urllib.parse import urlparse
parsed = urlparse(source)
if parsed.scheme not in ("http", "https"):
logger.error(f"Unsupported URL scheme: {parsed.scheme}")
return []
import ipaddress
try:
import socket
hostname = parsed.hostname or ""
ip = ipaddress.ip_address(socket.gethostbyname(hostname))
if ip.is_private or ip.is_loopback or ip.is_reserved or ip.is_link_local:
logger.error(f"URL resolves to private/reserved IP: {ip}")
return []
except (socket.gaierror, ValueError):
pass # Let the request handle DNS resolution errors
response = httpx.get(source, timeout=30, follow_redirects=True)
response.raise_for_status()
content = response.text
+116
View File
@@ -137,6 +137,13 @@ class BotOS:
)
self._tasks.append(task)
# Start schedule loop alongside bots so cron jobs execute
schedule_task = asyncio.create_task(
self._run_schedule_loop(),
name="botos-scheduler",
)
self._tasks.append(schedule_task)
try:
await asyncio.gather(*self._tasks, return_exceptions=True)
except asyncio.CancelledError:
@@ -152,6 +159,115 @@ class BotOS:
except Exception as e:
logger.error(f"BotOS: {platform} failed: {e}")
async def _run_schedule_loop(self) -> None:
"""Poll for due scheduled jobs and execute them.
Runs alongside bot tasks so cron/interval schedules fire even
without the UI. On trigger, the agent processes the message
and results are delivered back to the originating platform.
When the UI scheduler is already running (PraisonAIUI started),
this loop defers to it to avoid double-firing jobs.
"""
try:
from praisonaiagents.tools.schedule_tools import _get_store
from praisonaiagents.scheduler import ScheduleRunner
except ImportError:
logger.debug("BotOS: schedule module not available, skipping scheduler")
return
store = _get_store()
runner = ScheduleRunner(store)
logger.info("BotOS: schedule loop started (30s tick)")
while True:
try:
# Skip if the UI scheduler is already running (avoids double-fire)
ui_running = False
try:
from praisonaiui.features.schedules import _scheduler_running
ui_running = _scheduler_running
except ImportError:
pass # UI not installed — we're the only scheduler
if ui_running:
await asyncio.sleep(30)
continue
due = runner.get_due_jobs()
for job in due:
import time as _time
_start = _time.time()
_status = "succeeded"
_result = None
_error = None
_delivered = False
try:
_result, _delivered = await self._execute_schedule_job(job)
except Exception as e:
_status = "failed"
_error = str(e)
logger.warning(f"BotOS: schedule job {job.name} failed: {e}")
_duration = _time.time() - _start
runner.mark_run(
job,
status=_status,
result=_result,
error=_error,
duration=_duration,
delivered=_delivered,
)
except Exception as e:
logger.debug(f"BotOS: schedule tick error: {e}")
await asyncio.sleep(30)
async def _execute_schedule_job(self, job) -> tuple:
"""Execute a single due schedule job.
Returns:
Tuple of (result_str, delivered_bool).
"""
if not job.message:
return (None, False)
# Find the agent to execute with
agent = None
if job.agent_id:
# Look for a bot whose agent matches
for bot in self._bots.values():
a = bot.get_agent() if hasattr(bot, 'get_agent') else getattr(bot, '_agent', None)
if a and getattr(a, 'name', None) == job.agent_id:
agent = a
break
if agent is None:
# Use the first available agent
for bot in self._bots.values():
a = bot.get_agent() if hasattr(bot, 'get_agent') else getattr(bot, '_agent', None)
if a:
agent = a
break
if agent is None:
logger.debug(f"BotOS: no agent for schedule job {job.name}")
return (None, False)
# Run the agent
result = await asyncio.to_thread(agent.chat, job.message)
result_str = str(result) if result else None
# Deliver to originating platform if delivery target is set
delivered = False
delivery = job.delivery
if delivery and delivery.channel and delivery.channel_id:
bot = self.get_bot(delivery.channel)
if bot:
try:
await bot.send_message(delivery.channel_id, str(result))
delivered = True
except Exception as e:
logger.warning(f"BotOS: delivery to {delivery.channel} failed: {e}")
logger.info(f"BotOS: executed schedule job '{job.name}'")
return (result_str, delivered)
async def stop(self) -> None:
"""Gracefully stop all running bots."""
if not self._is_running:
@@ -312,13 +312,23 @@ class HybridWorkflowExecutor:
return flags
def _eval_condition(self, condition: str, flags: Dict) -> bool:
"""Evaluate a simple condition expression."""
"""Evaluate a simple condition expression safely."""
import ast
condition = condition.strip()
if condition.startswith("{{") and condition.endswith("}}"):
condition = condition[2:-2].strip()
context = {"flags": type("Flags", (), flags)(), "env": os.environ}
# Handle simple attribute lookups like "flags.verbose" or "env.DEBUG"
context = {"flags": flags, "env": dict(os.environ)}
try:
return bool(eval(condition, {"__builtins__": {}}, context))
except Exception:
# Only allow simple attribute access patterns: obj.attr
parts = condition.split(".")
if len(parts) == 2 and parts[0] in context:
obj = context[parts[0]]
if isinstance(obj, dict):
return bool(obj.get(parts[1], False))
# Try literal eval for simple boolean/numeric expressions
return bool(ast.literal_eval(condition))
except (ValueError, KeyError, TypeError):
return False
@@ -719,17 +719,26 @@ class JobWorkflowExecutor:
# ------------------------------------------------------------------
def _eval_condition(self, condition: str, flags: Dict) -> bool:
"""Evaluate a simple condition expression."""
"""Evaluate a simple condition expression safely."""
import ast
# Strip {{ }} if present
condition = condition.strip()
if condition.startswith("{{") and condition.endswith("}}"):
condition = condition[2:-2].strip()
# Build evaluation context
context = {"flags": type("Flags", (), flags)(), "env": os.environ}
# Handle simple attribute lookups like "flags.verbose" or "env.DEBUG"
context = {"flags": flags, "env": dict(os.environ)}
try:
return bool(eval(condition, {"__builtins__": {}}, context))
except Exception:
# Only allow simple attribute access patterns: obj.attr
parts = condition.split(".")
if len(parts) == 2 and parts[0] in context:
obj = context[parts[0]]
if isinstance(obj, dict):
return bool(obj.get(parts[1], False))
# Try literal eval for simple boolean/numeric expressions
return bool(ast.literal_eval(condition))
except (ValueError, KeyError, TypeError):
return False
# ------------------------------------------------------------------
+1 -1
View File
@@ -57,7 +57,7 @@ class CloudDeployer:
file.write("FROM python:3.11-slim\n")
file.write("WORKDIR /app\n")
file.write("COPY . .\n")
file.write("RUN pip install flask praisonai==4.5.74 gunicorn markdown\n")
file.write("RUN pip install flask praisonai==4.5.80 gunicorn markdown\n")
file.write("EXPOSE 8080\n")
file.write('CMD ["gunicorn", "-b", "0.0.0.0:8080", "api:app"]\n')
@@ -237,7 +237,7 @@ class SurrealDBConversationStore(ConversationStore):
params["after"] = after
where = "WHERE " + " AND ".join(conditions)
limit_clause = f"LIMIT {limit}" if limit else ""
limit_clause = f"LIMIT {int(limit)}" if limit else ""
result = self._run_sync(
self._client.query(
@@ -6,6 +6,7 @@ Install: pip install clickhouse-connect
"""
import logging
import re
from typing import Any, Dict, List, Optional
from .base import KnowledgeStore, KnowledgeDocument
@@ -13,6 +14,13 @@ from .base import KnowledgeStore, KnowledgeDocument
logger = logging.getLogger(__name__)
def _sanitize_identifier(name: str) -> str:
"""Sanitize a SQL identifier (table/database name) to prevent injection."""
if not re.match(r'^[a-zA-Z_][a-zA-Z0-9_.]*$', name):
raise ValueError(f"Invalid identifier: {name}")
return name
class ClickHouseKnowledgeStore(KnowledgeStore):
"""
ClickHouse-based knowledge store using vector search.
@@ -52,11 +60,11 @@ class ClickHouseKnowledgeStore(KnowledgeStore):
self.database = database
# Create database if not exists
self._client.command(f"CREATE DATABASE IF NOT EXISTS {database}")
self._client.command(f"CREATE DATABASE IF NOT EXISTS {_sanitize_identifier(database)}")
logger.info(f"Connected to ClickHouse database: {database}")
def _table_name(self, collection: str) -> str:
return f"{self.database}.praison_vec_{collection}"
return _sanitize_identifier(f"{self.database}.praison_vec_{collection}")
def create_collection(
self,
@@ -98,10 +106,11 @@ class ClickHouseKnowledgeStore(KnowledgeStore):
def list_collections(self) -> List[str]:
"""List all collections."""
result = self._client.query(f"""
SELECT name FROM system.tables
WHERE database = '{self.database}' AND name LIKE 'praison_vec_%'
""")
db = _sanitize_identifier(self.database)
result = self._client.query(
"SELECT name FROM system.tables WHERE database = %(db)s AND name LIKE 'praison_vec_%%'",
parameters={"db": db}
)
return [row[0].replace("praison_vec_", "") for row in result.result_rows]
def insert(
@@ -156,19 +165,20 @@ class ClickHouseKnowledgeStore(KnowledgeStore):
"""Search for similar documents using cosine distance."""
table = self._table_name(collection)
# Convert embedding to string for query
embedding_str = "[" + ",".join(map(str, query_embedding)) + "]"
# Convert embedding to validated float array for query
embedding_str = "[" + ",".join(str(float(x)) for x in query_embedding) + "]"
limit = int(limit)
query = f"""
SELECT
SELECT
id, content, content_hash, created_at,
1 - cosineDistance(embedding, {embedding_str}) as score
FROM {table}
"""
if score_threshold:
query += f" WHERE 1 - cosineDistance(embedding, {embedding_str}) >= {score_threshold}"
query += f" WHERE 1 - cosineDistance(embedding, {embedding_str}) >= {float(score_threshold)}"
query += f" ORDER BY score DESC LIMIT {limit}"
result = self._client.query(query)
@@ -195,11 +205,12 @@ class ClickHouseKnowledgeStore(KnowledgeStore):
"""Get documents by IDs."""
table = self._table_name(collection)
id_list = ", ".join([f"'{i}'" for i in ids])
result = self._client.query(f"""
SELECT id, content, content_hash, created_at, embedding
FROM {table} WHERE id IN ({id_list})
""")
placeholders = ", ".join(["%(id_{})s".format(idx) for idx in range(len(ids))])
params = {"id_{}".format(idx): i for idx, i in enumerate(ids)}
result = self._client.query(
f"SELECT id, content, content_hash, created_at, embedding FROM {table} WHERE id IN ({placeholders})",
parameters=params
)
documents = []
for row in result.result_rows:
@@ -225,8 +236,12 @@ class ClickHouseKnowledgeStore(KnowledgeStore):
table = self._table_name(collection)
if ids:
id_list = ", ".join([f"'{i}'" for i in ids])
self._client.command(f"ALTER TABLE {table} DELETE WHERE id IN ({id_list})")
placeholders = ", ".join(["%(id_{})s".format(idx) for idx in range(len(ids))])
params = {"id_{}".format(idx): i for idx, i in enumerate(ids)}
self._client.command(
f"ALTER TABLE {table} DELETE WHERE id IN ({placeholders})",
parameters=params
)
return len(ids)
return 0
@@ -6,6 +6,7 @@ Install: pip install lancedb
"""
import logging
import re
from typing import Any, Dict, List, Optional
from .base import KnowledgeStore, KnowledgeDocument
@@ -13,6 +14,18 @@ from .base import KnowledgeStore, KnowledgeDocument
logger = logging.getLogger(__name__)
def _sanitize_identifier(name: str) -> str:
"""Sanitize a SQL identifier to prevent injection."""
if not re.match(r'^[a-zA-Z_][a-zA-Z0-9_]*$', name):
raise ValueError(f"Invalid identifier: {name}")
return name
def _escape_value(val: str) -> str:
"""Escape a string value for use in LanceDB WHERE clauses."""
return str(val).replace("'", "''").replace("\\", "\\\\")
class LanceDBKnowledgeStore(KnowledgeStore):
"""
LanceDB-based knowledge store for vector search.
@@ -140,9 +153,9 @@ class LanceDBKnowledgeStore(KnowledgeStore):
query = table.search(query_embedding).limit(limit)
if filters:
where_clauses = [f"{k} = '{v}'" for k, v in filters.items()]
where_clauses = [f"{_sanitize_identifier(k)} = '{_escape_value(v)}'" for k, v in filters.items()]
query = query.where(" AND ".join(where_clauses))
results = query.to_pandas()
documents = []
@@ -170,7 +183,7 @@ class LanceDBKnowledgeStore(KnowledgeStore):
"""Get documents by IDs."""
table = self._get_table(collection)
id_list = ", ".join([f"'{i}'" for i in ids])
id_list = ", ".join([f"'{_escape_value(i)}'" for i in ids])
results = table.search().where(f"id IN ({id_list})").to_pandas()
documents = []
@@ -197,11 +210,11 @@ class LanceDBKnowledgeStore(KnowledgeStore):
table = self._get_table(collection)
if ids:
id_list = ", ".join([f"'{i}'" for i in ids])
id_list = ", ".join([f"'{_escape_value(i)}'" for i in ids])
table.delete(f"id IN ({id_list})")
return len(ids)
elif filters:
where_clauses = [f"{k} = '{v}'" for k, v in filters.items()]
where_clauses = [f"{_sanitize_identifier(k)} = '{_escape_value(v)}'" for k, v in filters.items()]
table.delete(" AND ".join(where_clauses))
return -1
return 0
@@ -6,6 +6,7 @@ Install: pip install pymilvus
"""
import logging
import re
from typing import Any, Dict, List, Optional
from .base import KnowledgeStore, KnowledgeDocument
@@ -13,6 +14,18 @@ from .base import KnowledgeStore, KnowledgeDocument
logger = logging.getLogger(__name__)
def _sanitize_identifier(name: str) -> str:
"""Sanitize a Milvus field name to prevent injection."""
if not re.match(r'^[a-zA-Z_][a-zA-Z0-9_]*$', name):
raise ValueError(f"Invalid field name: {name}")
return name
def _escape_value(val: str) -> str:
"""Escape a string value for use in Milvus expressions."""
return str(val).replace('\\', '\\\\').replace('"', '\\"')
class MilvusKnowledgeStore(KnowledgeStore):
"""
Milvus-based knowledge store for vector search.
@@ -180,9 +193,9 @@ class MilvusKnowledgeStore(KnowledgeStore):
expr = None
if filters:
conditions = [f'{k} == "{v}"' for k, v in filters.items()]
conditions = [f'{_sanitize_identifier(k)} == "{_escape_value(v)}"' for k, v in filters.items()]
expr = " and ".join(conditions)
results = col.search(
data=[query_embedding],
anns_field="vector",
@@ -220,9 +233,9 @@ class MilvusKnowledgeStore(KnowledgeStore):
col = Collection(collection)
col.load()
id_list = ", ".join([f'"{i}"' for i in ids])
id_list = ", ".join([f'"{_escape_value(i)}"' for i in ids])
expr = f"id in [{id_list}]"
results = col.query(
expr=expr,
output_fields=["id", "content", "content_hash", "created_at", "vector"],
@@ -254,12 +267,12 @@ class MilvusKnowledgeStore(KnowledgeStore):
col.load()
if ids:
id_list = ", ".join([f'"{i}"' for i in ids])
id_list = ", ".join([f'"{_escape_value(i)}"' for i in ids])
expr = f"id in [{id_list}]"
col.delete(expr)
return len(ids)
elif filters:
conditions = [f'{k} == "{v}"' for k, v in filters.items()]
conditions = [f'{_sanitize_identifier(k)} == "{_escape_value(v)}"' for k, v in filters.items()]
expr = " and ".join(conditions)
col.delete(expr)
return -1
@@ -5,6 +5,7 @@ Requires: singlestoredb
Install: pip install singlestoredb
"""
import json
import logging
from typing import Any, Dict, List, Optional
@@ -227,7 +228,7 @@ class SingleStoreVectorKnowledgeStore(KnowledgeStore):
documents.append(KnowledgeDocument(
id=row[0],
content=row[1],
metadata=eval(row[2]) if row[2] else None,
metadata=json.loads(row[2]) if row[2] else None,
content_hash=row[3],
created_at=row[4]
))
@@ -256,7 +257,7 @@ class SingleStoreVectorKnowledgeStore(KnowledgeStore):
KnowledgeDocument(
id=row[0],
content=row[1],
metadata=eval(row[2]) if row[2] else None,
metadata=json.loads(row[2]) if row[2] else None,
content_hash=row[3],
created_at=row[4]
)
@@ -199,8 +199,9 @@ class SurrealDBVectorKnowledgeStore(KnowledgeStore):
"""Search for similar documents using vector search."""
self._init_client()
# SurrealDB vector search query
embedding_str = str(query_embedding)
# SurrealDB vector search query - validate numeric inputs
embedding_str = "[" + ",".join(str(float(x)) for x in query_embedding) + "]"
limit = int(limit)
query = f"""
SELECT *, vector::similarity::cosine(embedding, {embedding_str}) AS score
FROM {collection}
+1 -1
View File
@@ -56,7 +56,7 @@ class PlanApplier:
# Try to use ruamel.yaml for comment preservation
try:
from ruamel.yaml import YAML
self._yaml = YAML()
self._yaml = YAML(typ='safe')
self._yaml.preserve_quotes = True
self._use_ruamel = True
except ImportError:
+27 -7
View File
@@ -173,12 +173,18 @@ class ScheduledAgentExecutor:
# Skip jobs with no message
if not message:
self._runner.mark_run(job)
duration = time.time() - started
self._runner.mark_run(
job,
status="skipped",
error="No message configured",
duration=duration,
)
return JobResult(
job=job,
status="skipped",
error="No message configured",
duration=time.time() - started,
duration=duration,
)
# Resolve the agent
@@ -187,19 +193,23 @@ class ScheduledAgentExecutor:
except Exception as e:
logger.warning("Agent resolution failed for job '%s': %s", job.id, e)
err = f"Agent resolution failed: {e}"
duration = time.time() - started
self._runner.mark_run(job, status="failed", error=err, duration=duration)
if self._on_failure:
self._on_failure(job, err)
return JobResult(
job=job, status="failed", error=err, duration=time.time() - started,
job=job, status="failed", error=err, duration=duration,
)
if agent is None:
err = f"No agent found for agent_id={agent_id!r}"
logger.warning("Skipping job '%s': %s", job.id, err)
duration = time.time() - started
self._runner.mark_run(job, status="failed", error=err, duration=duration)
if self._on_failure:
self._on_failure(job, err)
return JobResult(
job=job, status="failed", error=err, duration=time.time() - started,
job=job, status="failed", error=err, duration=duration,
)
# Execute via agent.chat() in a thread (sync-safe)
@@ -227,14 +237,15 @@ class ScheduledAgentExecutor:
except Exception as e:
logger.warning("Job '%s' execution failed: %s", job.id, e)
err = str(e)
duration = time.time() - started
self._runner.mark_run(job, status="failed", error=err, duration=duration)
if self._on_failure:
self._on_failure(job, err)
return JobResult(
job=job, status="failed", error=err, duration=time.time() - started,
job=job, status="failed", error=err, duration=duration,
)
# Success
self._runner.mark_run(job)
# Success - calculate duration before delivery
duration = time.time() - started
# Deliver to channel bot if delivery target exists
@@ -256,6 +267,15 @@ class ScheduledAgentExecutor:
job.id, delivery.channel, delivery.channel_id, e,
)
# Log history with execution results
self._runner.mark_run(
job,
status="succeeded",
result=result_str,
duration=duration,
delivered=delivered,
)
if self._on_success:
self._on_success(job, result_str)
+1 -1
View File
@@ -1 +1 @@
__version__ = "4.5.74"
__version__ = "4.5.80"
+3 -1
View File
@@ -12,7 +12,7 @@ dependencies = [
"rich>=13.7",
"markdown>=3.5",
"pyparsing>=3.0.0",
"praisonaiagents>=1.5.74",
"praisonaiagents>=1.5.80",
"python-dotenv>=0.19.0",
"litellm>=1.81.0",
"PyYAML>=6.0",
@@ -72,6 +72,7 @@ realtime = [
"greenlet>=3.0.3",
"tavily-python==0.5.0",
"crawl4ai>=0.7.0",
"playwright>=1.47.0", # required by crawl4ai — browsers still need: playwright install chromium
"websockets>=12.0",
"plotly>=5.24.0",
"yfinance>=0.2.44",
@@ -158,6 +159,7 @@ claw = [
# Search + browsing
"tavily-python==0.5.0",
"crawl4ai>=0.7.0",
"playwright>=1.47.0", # required by crawl4ai — browsers still need: playwright install chromium
"websockets>=12.0",
"ddgs>=9.0.0",
# Shell tools
+10 -4
View File
@@ -4811,6 +4811,7 @@ claw = [
{ name = "fastapi" },
{ name = "flask" },
{ name = "greenlet" },
{ name = "playwright" },
{ name = "praisonaiagents", extra = ["all"] },
{ name = "psutil" },
{ name = "python-telegram-bot" },
@@ -4855,6 +4856,7 @@ realtime = [
{ name = "crawl4ai" },
{ name = "ddgs" },
{ name = "greenlet" },
{ name = "playwright" },
{ name = "plotly" },
{ name = "sqlalchemy" },
{ name = "tavily-python" },
@@ -4929,13 +4931,15 @@ requires-dist = [
{ name = "neonize", marker = "extra == 'bot-whatsapp-web'", specifier = ">=0.3.14" },
{ name = "playwright", marker = "extra == 'all'", specifier = ">=1.47.0" },
{ name = "playwright", marker = "extra == 'chat'", specifier = ">=1.47.0" },
{ name = "playwright", marker = "extra == 'claw'", specifier = ">=1.47.0" },
{ name = "playwright", marker = "extra == 'code'", specifier = ">=1.47.0" },
{ name = "playwright", marker = "extra == 'realtime'", specifier = ">=1.47.0" },
{ name = "plotly", marker = "extra == 'all'", specifier = ">=5.24.0" },
{ name = "plotly", marker = "extra == 'realtime'", specifier = ">=5.24.0" },
{ name = "praisonai-tools", marker = "extra == 'autogen'", specifier = ">=0.1.0" },
{ name = "praisonai-tools", marker = "extra == 'autogen-v4'", specifier = ">=0.1.0" },
{ name = "praisonai-tools", marker = "extra == 'crewai'", specifier = ">=0.1.0" },
{ name = "praisonaiagents", specifier = ">=1.5.74" },
{ name = "praisonaiagents", specifier = ">=1.5.80" },
{ name = "praisonaiagents", extras = ["all"], marker = "extra == 'claw'", specifier = ">=1.5.40" },
{ name = "psutil", marker = "extra == 'all'", specifier = ">=5.9.0" },
{ name = "psutil", marker = "extra == 'bot'", specifier = ">=5.9.0" },
@@ -5005,7 +5009,7 @@ wheels = [
[[package]]
name = "praisonaiagents"
version = "1.5.74"
version = "1.5.80"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "aiohttp" },
@@ -5014,15 +5018,16 @@ dependencies = [
{ name = "pydantic" },
{ name = "rich" },
]
sdist = { url = "https://files.pythonhosted.org/packages/f7/8c/d69223724d5938ace639f281a5823f1c78dee230e475583afb7add893ef0/praisonaiagents-1.5.74.tar.gz", hash = "sha256:1c00780fd3e2d43d1c243b62e85de2bd5cc10ace4c5d5a25c051c97fb27f2a33", size = 1199751, upload-time = "2026-03-17T14:54:18.185Z" }
sdist = { url = "https://files.pythonhosted.org/packages/6e/3b/6c88188227bac16f88e7b141568cebb03d6ef46f38935e3bb8408a400017/praisonaiagents-1.5.80.tar.gz", hash = "sha256:fa0b803943e5aab8ce4099c8257e8f5378648a19668ec046749aa3babad99d57", size = 1204298, upload-time = "2026-03-18T21:06:55.71Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/c4/22/d4e368db9c72ba596f268b5df6c75f352d41adbbe4beebaa1489cc5c80be/praisonaiagents-1.5.74-py3-none-any.whl", hash = "sha256:a9c359cf5d0612d866872f63e6bcd95cf2105a7b76694ccb5a9716f89c31b699", size = 1242788, upload-time = "2026-03-17T14:54:16.071Z" },
{ url = "https://files.pythonhosted.org/packages/b3/76/ab37e313ad3e756c39df8f5b180613169ff8074a293aaba52fbdef88928e/praisonaiagents-1.5.80-py3-none-any.whl", hash = "sha256:e1590e6bbbda526e13d3654ab17cab6a2a602af4508fabe7a8cd389213658efe", size = 1248857, upload-time = "2026-03-18T21:06:53.898Z" },
]
[package.optional-dependencies]
all = [
{ name = "chonkie" },
{ name = "chromadb" },
{ name = "crawl4ai" },
{ name = "ddgs" },
{ name = "fastapi" },
{ name = "litellm" },
@@ -5031,6 +5036,7 @@ all = [
{ name = "mem0ai", extra = ["graph"] },
{ name = "motor" },
{ name = "passlib", extra = ["bcrypt"] },
{ name = "playwright" },
{ name = "posthog" },
{ name = "pydantic" },
{ name = "pyjwt" },