Compare commits

...
12 Commits
Author SHA1 Message Date
Mervin Praison 6d7cc9c3ba Release v2.3.2 2025-12-15 13:34:15 +00:00
Mervin Praison 690e4497cc feat: add --save support for direct prompts
- Save output to output/prompts/ folder with timestamped filename
- Works with all flags: --web-search, --expand-prompt, etc.

Usage:
  praisonai 'latest AI trends' --web-search --save
  praisonai 'write a poem' --expand-prompt --save
2025-12-15 13:33:28 +00:00
Mervin Praison 041f5f0b08 fix: add current date to PromptExpanderAgent for time-sensitive context
- Add datetime import and current_date to all expansion prompts
- Ensures expanded prompts reference the correct current date (e.g., December 15, 2025)
- Fixes issue where prompts defaulted to outdated year references
2025-12-15 13:26:13 +00:00
Mervin Praison bc0d9a6213 feat: add CLI flags for web-search, web-fetch, and prompt-caching
- Add --web-search flag for native web search (OpenAI, Gemini, Anthropic, xAI, Perplexity)
- Add --web-fetch flag for URL content retrieval (Anthropic only)
- Add --prompt-caching flag to reduce costs (OpenAI, Anthropic, Bedrock, Deepseek)
- Update README with CLI and programmatic usage examples

Usage:
  praisonai 'latest AI news' --web-search --llm openai/gpt-4o-search-preview
  praisonai 'summarize https://example.com' --web-fetch --llm anthropic/claude-sonnet-4-20250514
  praisonai 'analyze...' --prompt-caching --llm anthropic/claude-sonnet-4-20250514
2025-12-15 12:56:02 +00:00
Mervin Praison a4427fa135 Release v2.3.1 2025-12-15 12:45:22 +00:00
Mervin Praison 51e8c65c8c feat: refactor model capabilities to use LiteLLM helper functions
- Replace static model lists with LiteLLM's built-in capability checks (supports_response_schema, supports_function_calling, supports_parallel_function_calling, supports_web_search)
- Add supports_function_calling() and supports_parallel_function_calling() functions
- Keep static list for web_fetch (no LiteLLM helper available yet)
- Update documentation to reference LiteLLM as primary source
- Add Web Fetch feature to README features
2025-12-15 12:32:09 +00:00
Mervin Praison 285ed672c5 feat: add native web search and web fetch documentation to README
- Add Native Web Search section with supported providers (OpenAI, Anthropic, Gemini, xAI, Perplexity)
- Add Web Fetch section for Anthropic models with configuration options
- Add code examples for web_search and web_fetch parameters
- Document web_fetch options (max_uses, allowed_domains, citations)
2025-12-15 12:20:41 +00:00
Mervin Praison c03e51e337 feat: add native web fetch support for Anthropic Claude models
- Add web_fetch parameter to Agent and LLM classes
- Add model_capabilities.py with MODELS_SUPPORTING_WEB_FETCH list (Claude 3.5+, 4+)
- Add supports_web_fetch() function to check model capabilities
- Add web_fetch tool definition with configurable options (max_uses, allowed_domains, blocked_domains, citations, max_content_tokens)
- Add _model_supports_web_fetch() method to Agent class
- Add warning log for unsupported models when web_
2025-12-15 12:18:08 +00:00
Mervin Praison f3ba05f6aa feat: expand structured outputs support for OpenAI, Anthropic, and Gemini models
- Add comprehensive list of OpenAI models with structured outputs (gpt-4o-2024-08-06+, gpt-4.1+, gpt-5+, o-series)
- Add Anthropic Claude 4+ models with structured outputs support (Sonnet 4.5, Opus 4.1/4.5, Haiku 4.5)
- Add all Google Gemini models (2.5-pro, 2.5-flash, 2.0-flash, 1.5-pro/flash)
- Add auto-detection for newer model versions (gpt-4.1+, gpt-5+, o-series, Claude 4+, all Gemini)
- Add azure/ prefix support
2025-12-15 12:07:46 +00:00
Mervin Praison 9cea3edcd6 feat: add native web search support for OpenAI, Gemini, Anthropic, xAI, and Perplexity models
- Add web_search parameter to Agent and LLM classes
- Add model_capabilities.py with MODELS_SUPPORTING_WEB_SEARCH list
- Add supports_web_search() function to check model capabilities
- Add automatic DuckDuckGo fallback for unsupported models
- Add web_search_options to LiteLLM completion params
- Remove incompatible parameters (temperature, top_p, etc.) for web search models
- Update README with Native
2025-12-15 11:59:19 +00:00
Mervin Praison 02a3bd3597 chore: add uv cache clean before uv lock in release.py 2025-12-15 11:26:08 +00:00
Mervin Praison 128c77926d chore: add uv cache clean before uv lock in bump_and_release.py 2025-12-15 11:25:42 +00:00
24 changed files with 1483 additions and 94 deletions

No files matched your search

+53
View File
@@ -55,6 +55,9 @@ PraisonAI is a production-ready Multi-AI Agents framework with self-reflection,
- 💯 100+ LLM Support
- 🔬 **Deep Research Agents** (OpenAI & Gemini)
- 🔄 **Query Rewriter Agent** (HyDE, Step-back, Multi-query)
- 🌐 **Native Web Search** (OpenAI, Gemini, Anthropic, xAI, Perplexity)
- 📥 **Web Fetch** (Retrieve full content from URLs - Anthropic)
- 💾 **Prompt Caching** (Reduce costs & latency - OpenAI, Anthropic, Bedrock, Deepseek)
## Using Python Code
@@ -759,6 +762,56 @@ result = agent.expand("blog about AI", strategy=ExpandStrategy.DETAILED)
- `--expand-prompt`: Expands prompts for **detailed task execution**
## Web Search, Web Fetch & Prompt Caching
### CLI Usage
```bash
# Web Search - Get real-time information
praisonai "What are the latest AI news today?" --web-search --llm openai/gpt-4o-search-preview
# Web Fetch - Retrieve and analyze URL content (Anthropic only)
praisonai "Summarize https://docs.praison.ai" --web-fetch --llm anthropic/claude-sonnet-4-20250514
# Prompt Caching - Reduce costs for repeated prompts
praisonai "Analyze this document..." --prompt-caching --llm anthropic/claude-sonnet-4-20250514
```
### Programmatic Usage
```python
from praisonaiagents import Agent
# Web Search
agent = Agent(
instructions="You are a research assistant",
llm="openai/gpt-4o-search-preview",
web_search=True
)
# Web Fetch (Anthropic only)
agent = Agent(
instructions="You are a content analyzer",
llm="anthropic/claude-sonnet-4-20250514",
web_fetch=True
)
# Prompt Caching
agent = Agent(
instructions="You are an AI assistant..." * 50, # Long system prompt
llm="anthropic/claude-sonnet-4-20250514",
prompt_caching=True
)
```
**Supported Providers:**
| Feature | Providers |
|---------|-----------|
| Web Search | OpenAI, Gemini, Anthropic, xAI, Perplexity |
| Web Fetch | Anthropic |
| Prompt Caching | OpenAI (auto), Anthropic, Bedrock, Deepseek |
## Development:
Below is used for development only.
+1 -1
View File
@@ -16,7 +16,7 @@ RUN mkdir -p /root/.praison
# Install Python packages (using latest versions)
RUN pip install --no-cache-dir \
flask \
"praisonai>=2.3.0" \
"praisonai>=2.3.2" \
"praisonai[api]" \
gunicorn \
markdown
+1 -1
View File
@@ -16,7 +16,7 @@ RUN mkdir -p /root/.praison
# Install Python packages (using latest versions)
RUN pip install --no-cache-dir \
praisonai_tools \
"praisonai>=2.3.0" \
"praisonai>=2.3.2" \
"praisonai[chat]" \
"embedchain[github,youtube]"
+1 -1
View File
@@ -20,7 +20,7 @@ RUN mkdir -p /root/.praison
# Install Python packages (using latest versions)
RUN pip install --no-cache-dir \
praisonai_tools \
"praisonai>=2.3.0" \
"praisonai>=2.3.2" \
"praisonai[ui]" \
"praisonai[chat]" \
"praisonai[realtime]" \
+1 -1
View File
@@ -16,7 +16,7 @@ RUN mkdir -p /root/.praison
# Install Python packages (using latest versions)
RUN pip install --no-cache-dir \
praisonai_tools \
"praisonai>=2.3.0" \
"praisonai>=2.3.2" \
"praisonai[ui]" \
"praisonai[crewai]"
+86
View File
@@ -347,6 +347,92 @@ agent = Agent(
---
## Native Web Search
Enable real-time web search capabilities using LiteLLM's native web search.
```python
from praisonaiagents import Agent
# Enable web search with a supported model
agent = Agent(
name="Researcher",
instructions="You are a helpful research assistant.",
llm="openai/gpt-4o-search-preview",
web_search=True
)
result = agent.start("What are the latest developments in AI today?")
```
**Supported Providers:**
- **OpenAI**: `gpt-4o-search-preview`, `gpt-4o-mini-search-preview`
- **Anthropic**: `claude-3-5-sonnet-latest`, `claude-sonnet-4`
- **Google/Gemini**: `gemini-2.0-flash`, `gemini-2.5-pro`
- **xAI**: `grok-3`
- **Perplexity**: All models
---
## Web Fetch
Retrieve full content from specific URLs (Anthropic models only).
```python
from praisonaiagents import Agent
# Enable web fetch with an Anthropic model
agent = Agent(
name="Content Analyzer",
instructions="You can fetch and analyze web content.",
llm="anthropic/claude-sonnet-4-20250514",
web_fetch=True
)
result = agent.start("Fetch and summarize https://example.com")
```
**Web Fetch Options:**
```python
agent = Agent(
llm="anthropic/claude-sonnet-4-20250514",
web_fetch={
"max_uses": 10,
"allowed_domains": ["example.com"],
"citations": {"enabled": True}
}
)
```
---
## Prompt Caching
Reduce costs and latency by caching parts of prompts.
```python
from praisonaiagents import Agent
# Enable prompt caching for Anthropic models
agent = Agent(
name="Legal Analyst",
instructions="You are an AI assistant analyzing legal documents." * 50, # Long prompt
llm="anthropic/claude-3-5-sonnet-latest",
prompt_caching=True
)
# First call creates cache, subsequent calls use cached tokens
result = agent.start("Summarize the key terms")
```
**Supported Providers:**
- **OpenAI** (`openai/`) - Automatic caching for prompts ≥1024 tokens
- **Anthropic** (`anthropic/`) - Manual caching with `cache_control`
- **Bedrock** (`bedrock/`) - All models supporting prompt caching
- **Deepseek** (`deepseek/`) - Works like OpenAI
---
## Guardrails
Add safety checks and validation to agent responses.
@@ -234,7 +234,10 @@ class Agent:
max_guardrail_retries: int = 3,
handoffs: Optional[List[Union['Agent', 'Handoff']]] = None,
base_url: Optional[str] = None,
api_key: Optional[str] = None
api_key: Optional[str] = None,
web_search: Optional[Union[bool, Dict[str, Any]]] = None,
web_fetch: Optional[Union[bool, Dict[str, Any]]] = None,
prompt_caching: Optional[bool] = None
):
"""Initialize an Agent instance.
@@ -400,7 +403,10 @@ class Agent:
model=model_name,
base_url=base_url,
api_key=api_key,
metrics=metrics
metrics=metrics,
web_search=web_search,
web_fetch=web_fetch,
prompt_caching=prompt_caching
)
self._using_custom_llm = True
except ImportError as e:
@@ -435,6 +441,9 @@ class Agent:
if api_key:
llm_params['api_key'] = api_key
llm_params['metrics'] = metrics
llm_params['web_search'] = web_search
llm_params['web_fetch'] = web_fetch
llm_params['prompt_caching'] = prompt_caching
self.llm_instance = LLM(**llm_params)
self._using_custom_llm = True
@@ -511,6 +520,22 @@ Your Goal: {self.goal}
# Store user_id
self.user_id = user_id or "praison"
self.reasoning_steps = reasoning_steps
self.web_search = web_search
self.web_fetch = web_fetch
self.prompt_caching = prompt_caching
# Handle web_search fallback: inject DuckDuckGo tool for unsupported models
if web_search and not self._model_supports_web_search():
from ..tools.duckduckgo_tools import internet_search
# Check if internet_search is not already in tools
tool_names = [getattr(t, '__name__', str(t)) for t in self.tools]
if 'internet_search' not in tool_names and 'duckduckgo' not in tool_names:
self.tools.append(internet_search)
logging.info("Model does not support native web search. Added DuckDuckGo fallback tool.")
# Log warning if web_fetch is enabled but model doesn't support it
if web_fetch and not self._model_supports_web_fetch():
logging.warning(f"Model '{self.llm}' does not support native web fetch. Web fetch will be ignored.")
# Initialize guardrail settings
self.guardrail = guardrail
@@ -576,6 +601,69 @@ Your Goal: {self.goal}
self._agent_id = str(uuid.uuid4())
return self._agent_id
def _model_supports_web_search(self) -> bool:
"""
Check if the agent's model supports native web search via LiteLLM.
Returns:
bool: True if the model supports native web search, False otherwise
"""
from ..llm.model_capabilities import supports_web_search
# Get the model name
if hasattr(self, 'llm_instance') and self.llm_instance:
model_name = self.llm_instance.model
elif hasattr(self, 'llm') and self.llm:
model_name = self.llm
else:
model_name = "gpt-5-nano"
return supports_web_search(model_name)
def _model_supports_web_fetch(self) -> bool:
"""
Check if the agent's model supports web fetch via LiteLLM.
Web fetch allows the model to retrieve full content from specific URLs.
Currently only supported by Anthropic Claude models.
Returns:
bool: True if the model supports web fetch, False otherwise
"""
from ..llm.model_capabilities import supports_web_fetch
# Get the model name
if hasattr(self, 'llm_instance') and self.llm_instance:
model_name = self.llm_instance.model
elif hasattr(self, 'llm') and self.llm:
model_name = self.llm
else:
model_name = "gpt-5-nano"
return supports_web_fetch(model_name)
def _model_supports_prompt_caching(self) -> bool:
"""
Check if the agent's model supports prompt caching via LiteLLM.
Prompt caching allows caching parts of prompts to reduce costs and latency.
Supported by OpenAI, Anthropic, Bedrock, and Deepseek.
Returns:
bool: True if the model supports prompt caching, False otherwise
"""
from ..llm.model_capabilities import supports_prompt_caching
# Get the model name
if hasattr(self, 'llm_instance') and self.llm_instance:
model_name = self.llm_instance.model
elif hasattr(self, 'llm') and self.llm:
model_name = self.llm
else:
model_name = "gpt-4o-mini"
return supports_prompt_caching(model_name)
@property
def llm_model(self):
"""Unified property to get the LLM model regardless of configuration type.
@@ -37,7 +37,7 @@ import logging
from typing import List, Optional, Any, Dict
from dataclasses import dataclass, field
from enum import Enum
from datetime import datetime
class ExpandStrategy(Enum):
"""Enumeration of available prompt expansion strategies."""
@@ -88,6 +88,8 @@ class PromptExpanderAgent:
PROMPTS = {
"basic": """You are an expert at improving task prompts.
Current date: {current_date}
Given the following task prompt, expand it to be clearer and more actionable while preserving the original intent.
Guidelines:
@@ -95,6 +97,7 @@ Guidelines:
- Add clarity without changing the core request
- Fix any ambiguity
- Keep it concise but complete
- Use the current date for any time-sensitive context
Original prompt: {prompt}
@@ -102,6 +105,8 @@ Expanded prompt:""",
"detailed": """You are an expert at creating comprehensive task prompts.
Current date: {current_date}
Given the following brief task prompt, expand it into a detailed, actionable prompt that provides rich context and clear guidance.
Guidelines:
@@ -111,6 +116,7 @@ Guidelines:
- Add relevant constraints or requirements
- Include format guidance if applicable
- Mention style or tone if relevant
- Use the current date for any time-sensitive context
Original prompt: {prompt}
@@ -118,6 +124,8 @@ Detailed expanded prompt:""",
"structured": """You are an expert at creating well-structured task prompts.
Current date: {current_date}
Given the following task prompt, expand it into a structured format with clear sections.
Guidelines:
@@ -129,6 +137,7 @@ Guidelines:
* Style: Tone or style guidance
* Constraints: Any limitations
- Be specific and actionable
- Use the current date for any time-sensitive context
Original prompt: {prompt}
@@ -136,6 +145,8 @@ Structured expanded prompt:""",
"creative": """You are an expert at crafting inspiring, creative task prompts.
Current date: {current_date}
Given the following task prompt, expand it with vivid language and creative direction while keeping it actionable.
Guidelines:
@@ -145,6 +156,7 @@ Guidelines:
- Add creative direction and artistic guidance
- Make it exciting and engaging
- Preserve the core task intent
- Use the current date for any time-sensitive context
Original prompt: {prompt}
@@ -152,6 +164,8 @@ Creative expanded prompt:""",
"auto": """You are an expert at analyzing and expanding task prompts.
Current date: {current_date}
Analyze the following prompt and expand it appropriately based on its nature.
For short/vague prompts: Add significant detail and context
@@ -160,6 +174,7 @@ For creative tasks: Add artistic direction
For technical tasks: Add precision and requirements
IMPORTANT: Keep it as a TASK/COMMAND. Never convert to a question.
Use the current date for any time-sensitive context.
Original prompt: {prompt}
{context}
@@ -306,9 +321,13 @@ Expanded prompt:"""
# Fallback if auto-detect returns AUTO
strategy = ExpandStrategy.DETAILED
# Get current date for time-sensitive context
current_date = datetime.now().strftime("%B %d, %Y")
expansion_prompt = self.PROMPTS[strategy.value].format(
prompt=prompt,
context=context or ""
context=context or "",
current_date=current_date
)
# Call the agent
+117 -1
View File
@@ -239,6 +239,9 @@ class LLM:
api_key: Optional[str] = None,
base_url: Optional[str] = None,
events: List[Any] = [],
web_search: Optional[Union[bool, Dict[str, Any]]] = None,
web_fetch: Optional[Union[bool, Dict[str, Any]]] = None,
prompt_caching: Optional[bool] = None,
**extra_settings
):
# Configure logging only once at the class level
@@ -273,6 +276,9 @@ class LLM:
self.base_url = base_url
self.events = events
self.extra_settings = extra_settings
self.web_search = web_search
self.web_fetch = web_fetch
self.prompt_caching = prompt_caching
self._console = None # Lazy load console when needed
self.chat_history = []
self.verbose = extra_settings.get('verbose', True)
@@ -381,6 +387,45 @@ class LLM:
else:
return False
def _supports_web_search(self) -> bool:
"""
Check if the current model supports native web search via LiteLLM.
Native web search allows the model to search the web in real-time
without requiring external tools like DuckDuckGo.
Returns:
bool: True if the model supports native web search, False otherwise
"""
from .model_capabilities import supports_web_search
return supports_web_search(self.model)
def _supports_web_fetch(self) -> bool:
"""
Check if the current model supports web fetch via LiteLLM.
Web fetch allows the model to retrieve full content from specific URLs
(web pages and PDF documents). Currently only supported by Anthropic Claude models.
Returns:
bool: True if the model supports web fetch, False otherwise
"""
from .model_capabilities import supports_web_fetch
return supports_web_fetch(self.model)
def _supports_prompt_caching(self) -> bool:
"""
Check if the current model supports prompt caching via LiteLLM.
Prompt caching allows caching parts of prompts to reduce costs and latency.
Supported by OpenAI, Anthropic, Bedrock, and Deepseek.
Returns:
bool: True if the model supports prompt caching, False otherwise
"""
from .model_capabilities import supports_prompt_caching
return supports_prompt_caching(self.model)
def _generate_ollama_tool_summary(self, tool_results: List[Any], response_text: str) -> Optional[str]:
"""
Generate a summary from tool results for Ollama to prevent infinite loops.
@@ -724,7 +769,21 @@ class LLM:
# Skip system messages for legacy o1 models as they don't support them
if not self._needs_system_message_skip():
messages.append({"role": "system", "content": system_prompt})
# Apply prompt caching for Anthropic models if enabled
if self.prompt_caching and self._supports_prompt_caching() and self._is_anthropic_model():
# Anthropic requires cache_control in content array format
messages.append({
"role": "system",
"content": [
{
"type": "text",
"text": system_prompt,
"cache_control": {"type": "ephemeral"}
}
]
})
else:
messages.append({"role": "system", "content": system_prompt})
# Add chat history if provided
if chat_history:
@@ -2295,6 +2354,12 @@ Output MUST be JSON with 'reflection' and 'satisfactory'.
return False
return any(prefix in self.model.lower() for prefix in ['gemini', 'gemini/', 'google/gemini'])
def _is_anthropic_model(self) -> bool:
"""Check if the model is an Anthropic Claude model."""
if not self.model:
return False
return any(prefix in self.model.lower() for prefix in ['claude', 'anthropic/', 'anthropic/claude'])
def _is_streaming_error_recoverable(self, error: Exception) -> bool:
"""Check if a streaming error is recoverable (e.g., malformed chunk vs connection error)."""
error_msg = str(error).lower()
@@ -3171,6 +3236,57 @@ Output MUST be JSON with 'reflection' and 'satisfactory'.
logging.debug(f"Could not verify function calling support: {e}. Setting tool_choice anyway.")
params['tool_choice'] = 'auto'
# Add web_search_options if web_search is enabled and model supports it
if self.web_search and self._supports_web_search():
if isinstance(self.web_search, dict):
# Use custom web_search_options provided by user
params['web_search_options'] = self.web_search
else:
# Use default web_search_options
params['web_search_options'] = {"search_context_size": "medium"}
logging.debug(f"Web search enabled with options: {params['web_search_options']}")
# Some web search models (like gpt-4o-search-preview) don't support certain parameters
# Remove incompatible parameters to avoid BadRequestError
incompatible_params = ['temperature', 'top_p', 'presence_penalty', 'frequency_penalty', 'logit_bias']
for param in incompatible_params:
if param in params:
del params[param]
logging.debug(f"Removed incompatible parameter '{param}' for web search model")
# Add web_fetch tool if web_fetch is enabled and model supports it
# Web fetch is passed as a tool (unlike web_search which uses web_search_options)
if self.web_fetch and self._supports_web_fetch():
# Build the web_fetch tool definition
web_fetch_tool = {
"type": "web_fetch_20250910",
"name": "web_fetch",
}
# Add optional parameters from web_fetch dict if provided
if isinstance(self.web_fetch, dict):
if 'max_uses' in self.web_fetch:
web_fetch_tool['max_uses'] = self.web_fetch['max_uses']
if 'allowed_domains' in self.web_fetch:
web_fetch_tool['allowed_domains'] = self.web_fetch['allowed_domains']
if 'blocked_domains' in self.web_fetch:
web_fetch_tool['blocked_domains'] = self.web_fetch['blocked_domains']
if 'citations' in self.web_fetch:
web_fetch_tool['citations'] = self.web_fetch['citations']
if 'max_content_tokens' in self.web_fetch:
web_fetch_tool['max_content_tokens'] = self.web_fetch['max_content_tokens']
else:
# Default max_uses if web_fetch=True
web_fetch_tool['max_uses'] = 5
# Add web_fetch tool to existing tools or create tools list
if 'tools' in params and params['tools']:
params['tools'].append(web_fetch_tool)
else:
params['tools'] = [web_fetch_tool]
logging.debug(f"Web fetch enabled with tool: {web_fetch_tool}")
return params
def _prepare_response_logging(self, temperature: float, stream: bool, verbose: bool, markdown: bool, **kwargs) -> Optional[Dict[str, Any]]:
@@ -1,62 +1,30 @@
"""
Model capabilities configuration for different LLM providers.
This module defines which models support specific features like structured outputs.
This module uses LiteLLM's helper functions as the primary source for model capability detection.
LiteLLM is maintained by many contributors and is more accurate and up-to-date.
LiteLLM Helper Functions:
- litellm.supports_web_search(model=) - Check web search support
- litellm.supports_function_calling(model=) - Check function calling support
- litellm.supports_parallel_function_calling(model=) - Check parallel function calling
- litellm.supports_response_schema(model=) - Check JSON schema/structured outputs support
- litellm.utils.supports_prompt_caching(model=) - Check prompt caching support
- litellm.get_supported_openai_params(model=) - Get all supported params
Sources:
- https://docs.litellm.ai/docs/completion/web_search
- https://docs.litellm.ai/docs/completion/json_mode
- https://docs.litellm.ai/docs/completion/function_call
- https://docs.litellm.ai/docs/completion/prompt_caching
"""
# Models that support OpenAI-style structured outputs (response_format with Pydantic models)
MODELS_SUPPORTING_STRUCTURED_OUTPUTS = {
# OpenAI models
"gpt-5-nano",
"gpt-5-nano",
"gpt-4-turbo",
"gpt-4-turbo-preview",
"gpt-4-turbo-2024-04-09",
"gpt-4-1106-preview",
"gpt-4-0125-preview",
"gpt-3.5-turbo",
"gpt-3.5-turbo-1106",
"gpt-3.5-turbo-0125",
# New/Future OpenAI models (as mentioned by user)
"codex-mini",
"o3-pro",
"gpt-4.5-preview",
"o3-mini",
"o1",
"o1-preview",
"o1-mini",
"gpt-4.1",
"gpt-4.1-nano",
"gpt-4.1-mini",
"o4-mini",
"o3",
# Gemini models that support structured outputs
"gemini-2.0-flash",
"gemini-2.0-flash-exp",
"gemini-1.5-pro",
"gemini-1.5-pro-latest",
"gemini-1.5-flash",
"gemini-1.5-flash-latest",
"gemini-1.5-flash-8b",
"gemini-1.5-flash-8b-latest",
}
# Models that explicitly DON'T support structured outputs
MODELS_NOT_SUPPORTING_STRUCTURED_OUTPUTS = {
# Audio preview models
"gpt-4o-audio-preview",
"gpt-5-nano-audio-preview",
# Legacy o1 models (don't support system messages either)
"o1-preview-2024-09-12",
"o1-mini-2024-09-12",
}
def supports_structured_outputs(model_name: str) -> bool:
"""
Check if a model supports OpenAI-style structured outputs.
Check if a model supports structured outputs (JSON schema).
Uses LiteLLM's supports_response_schema() as the primary check.
Args:
model_name: The name of the model to check
@@ -67,34 +35,72 @@ def supports_structured_outputs(model_name: str) -> bool:
if not model_name:
return False
# Strip provider prefixes (e.g., 'google/', 'openai/', etc.)
model_without_provider = model_name
for prefix in ['google/', 'openai/', 'anthropic/', 'gemini/', 'mistral/', 'deepseek/', 'groq/']:
if model_name.startswith(prefix):
model_without_provider = model_name[len(prefix):]
break
try:
import litellm
# Use LiteLLM's built-in check - most accurate and up-to-date
if hasattr(litellm, 'supports_response_schema'):
return litellm.supports_response_schema(model=model_name)
except Exception:
pass
# First check if it's explicitly in the NOT supporting list
if model_without_provider in MODELS_NOT_SUPPORTING_STRUCTURED_OUTPUTS:
return False
def supports_function_calling(model_name: str) -> bool:
"""
Check if a model supports function calling.
Uses LiteLLM's supports_function_calling() as the primary check.
Args:
model_name: The name of the model to check
Returns:
bool: True if the model supports function calling, False otherwise
"""
if not model_name:
return False
# Then check if it's in the supporting list
if model_without_provider in MODELS_SUPPORTING_STRUCTURED_OUTPUTS:
return True
try:
import litellm
# Use LiteLLM's built-in check - most accurate and up-to-date
if hasattr(litellm, 'supports_function_calling'):
return litellm.supports_function_calling(model=model_name)
except Exception:
pass
# For models with version suffixes, check the base model name
base_model = model_without_provider.split('-2024-')[0].split('-2025-')[0]
if base_model in MODELS_SUPPORTING_STRUCTURED_OUTPUTS:
return True
return False
def supports_parallel_function_calling(model_name: str) -> bool:
"""
Check if a model supports parallel function calling.
Uses LiteLLM's supports_parallel_function_calling() as the primary check.
Args:
model_name: The name of the model to check
Returns:
bool: True if the model supports parallel function calling, False otherwise
"""
if not model_name:
return False
try:
import litellm
# Use LiteLLM's built-in check - most accurate and up-to-date
if hasattr(litellm, 'supports_parallel_function_calling'):
return litellm.supports_parallel_function_calling(model=model_name)
except Exception:
pass
# Default to False for unknown models
return False
def supports_streaming_with_tools(model_name: str) -> bool:
"""
Check if a model supports streaming when tools are provided.
Most models that support structured outputs also support streaming with tools.
Args:
model_name: The name of the model to check
@@ -102,15 +108,153 @@ def supports_streaming_with_tools(model_name: str) -> bool:
Returns:
bool: True if the model supports streaming with tools, False otherwise
"""
# For now, use the same logic as structured outputs
# In the future, this could be a separate list if needed
return supports_structured_outputs(model_name)
# Models that support function calling generally support streaming with tools
return supports_function_calling(model_name)
# Supported Gemini internal tools
GEMINI_INTERNAL_TOOLS = {'googleSearch', 'urlContext', 'codeExecution'}
def supports_web_search(model_name: str) -> bool:
"""
Check if a model supports native web search via LiteLLM.
Uses LiteLLM's supports_web_search() as the primary check.
Native web search allows the model to search the web in real-time
without requiring external tools like DuckDuckGo.
Supported providers:
- OpenAI (gpt-4o-search-preview, gpt-4o-mini-search-preview)
- xAI (grok-3)
- Anthropic (claude-3-5-sonnet-latest, claude-sonnet-4, etc.)
- Google/Vertex AI (gemini-2.0-flash, gemini-2.5-*, etc.)
- Perplexity (all models)
Args:
model_name: The name of the model to check (with or without provider prefix)
Returns:
bool: True if the model supports native web search, False otherwise
"""
if not model_name:
return False
try:
import litellm
# Use LiteLLM's built-in check - most accurate and up-to-date
if hasattr(litellm, 'supports_web_search'):
return litellm.supports_web_search(model=model_name)
except Exception:
pass
return False
def supports_prompt_caching(model_name: str) -> bool:
"""
Check if a model supports prompt caching via LiteLLM.
Uses LiteLLM's supports_prompt_caching() as the primary check.
Prompt caching allows caching parts of prompts to reduce costs and latency
on subsequent requests with similar prompts.
Supported providers:
- OpenAI (openai/) - Automatic caching for prompts ≥1024 tokens
- Anthropic (anthropic/) - Manual caching with cache_control
- Bedrock (bedrock/) - All models that support prompt caching
- Deepseek (deepseek/) - Works like OpenAI
Args:
model_name: The name of the model to check (with or without provider prefix)
Returns:
bool: True if the model supports prompt caching, False otherwise
"""
if not model_name:
return False
try:
from litellm.utils import supports_prompt_caching as litellm_supports_prompt_caching
return litellm_supports_prompt_caching(model=model_name)
except Exception:
pass
return False
# Models that support web fetch via LiteLLM (Anthropic only)
# Web fetch retrieves full content from specific URLs (web pages and PDFs)
# Source: https://docs.litellm.ai/docs/completion/web_fetch
# Note: LiteLLM doesn't have a supports_web_fetch() helper yet, so we maintain a static list
MODELS_SUPPORTING_WEB_FETCH = {
# Anthropic Claude models with web fetch support
"claude-opus-4-1-20250805",
"claude-opus-4-1",
"claude-opus-4-20250514",
"claude-opus-4",
"claude-sonnet-4-20250514",
"claude-sonnet-4",
"claude-3-7-sonnet-20250219",
"claude-3-7-sonnet-latest",
"claude-3-5-sonnet-latest",
"claude-3-5-sonnet-20241022",
"claude-3-5-haiku-latest",
"claude-3-5-haiku-20241022",
}
def supports_web_fetch(model_name: str) -> bool:
"""
Check if a model supports web fetch via LiteLLM.
Web fetch allows the model to retrieve full content from specific URLs
(web pages and PDF documents). Currently only supported by Anthropic Claude models.
Note: LiteLLM doesn't have a supports_web_fetch() helper yet, so we use a static list
with auto-detection for Claude 4+ models.
Args:
model_name: The name of the model to check (with or without provider prefix)
Returns:
bool: True if the model supports web fetch, False otherwise
"""
if not model_name:
return False
# Strip provider prefixes
model_without_provider = model_name
for prefix in ['anthropic/', 'bedrock/', 'vertex_ai/']:
if model_name.startswith(prefix):
model_without_provider = model_name[len(prefix):]
break
# Check our static list
if model_without_provider in MODELS_SUPPORTING_WEB_FETCH:
return True
# Check base model name (without version suffix)
base_model = model_without_provider.split('-2024-')[0].split('-2025-')[0]
if base_model in MODELS_SUPPORTING_WEB_FETCH:
return True
# Auto-support for Claude 4+ models (Anthropic only)
model_lower = model_without_provider.lower()
if 'claude' in model_lower:
import re
# Match patterns like claude-4, claude-5, claude-sonnet-4, claude-opus-4, etc.
version_match = re.search(r'claude-(?:sonnet-|opus-|haiku-)?(\d+)', model_lower)
if version_match:
version = int(version_match.group(1))
if version >= 4: # Claude 4 and later
return True
return False
def is_gemini_internal_tool(tool) -> bool:
"""
Check if a tool is a Gemini internal tool and should be included in formatted tools.
+1 -1
View File
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
[project]
name = "praisonaiagents"
version = "0.0.170"
version = "0.0.175"
description = "Praison AI agents for completing complex tasks with Self Reflection Agents"
requires-python = ">=3.10"
authors = [
@@ -0,0 +1,239 @@
"""
Tests for prompt caching feature.
Prompt caching allows caching parts of prompts to reduce costs and latency.
Supported by OpenAI, Anthropic, Bedrock, and Deepseek.
Source: https://docs.litellm.ai/docs/completion/prompt_caching
"""
import pytest
from unittest.mock import patch, MagicMock
class TestSupportsPromptCaching:
"""Tests for the supports_prompt_caching function in model_capabilities.py"""
def test_supports_prompt_caching_anthropic(self):
"""Test that Anthropic Claude models support prompt caching."""
from praisonaiagents.llm.model_capabilities import supports_prompt_caching
# Mock litellm.utils.supports_prompt_caching
with patch('praisonaiagents.llm.model_capabilities.supports_prompt_caching') as mock_func:
# Just test that the function exists and can be called
mock_func.return_value = True
result = mock_func(model="anthropic/claude-3-5-sonnet-latest")
assert result == True
def test_supports_prompt_caching_openai(self):
"""Test that OpenAI models support prompt caching."""
from praisonaiagents.llm.model_capabilities import supports_prompt_caching
with patch('litellm.utils.supports_prompt_caching', return_value=True):
# The function should delegate to litellm
result = supports_prompt_caching("openai/gpt-4o")
# Result depends on litellm's implementation
assert isinstance(result, bool)
def test_supports_prompt_caching_empty_model(self):
"""Test that empty model name returns False."""
from praisonaiagents.llm.model_capabilities import supports_prompt_caching
assert supports_prompt_caching("") == False
assert supports_prompt_caching(None) == False
class TestLLMPromptCachingParameter:
"""Tests for prompt_caching parameter in LLM class."""
def test_llm_init_with_prompt_caching_true(self):
"""Test LLM initialization with prompt_caching=True."""
from praisonaiagents.llm.llm import LLM
llm = LLM(model="anthropic/claude-3-5-sonnet-latest", prompt_caching=True)
assert llm.prompt_caching == True
def test_llm_init_with_prompt_caching_false(self):
"""Test LLM initialization with prompt_caching=False."""
from praisonaiagents.llm.llm import LLM
llm = LLM(model="anthropic/claude-3-5-sonnet-latest", prompt_caching=False)
assert llm.prompt_caching == False
def test_llm_init_without_prompt_caching(self):
"""Test LLM initialization without prompt_caching parameter."""
from praisonaiagents.llm.llm import LLM
llm = LLM(model="gpt-4o-mini")
assert llm.prompt_caching is None
def test_llm_supports_prompt_caching_method(self):
"""Test _supports_prompt_caching method exists and works."""
from praisonaiagents.llm.llm import LLM
llm = LLM(model="anthropic/claude-3-5-sonnet-latest")
# Method should exist and return a boolean
result = llm._supports_prompt_caching()
assert isinstance(result, bool)
def test_llm_is_anthropic_model_method(self):
"""Test _is_anthropic_model method."""
from praisonaiagents.llm.llm import LLM
# Anthropic model
llm_anthropic = LLM(model="anthropic/claude-3-5-sonnet-latest")
assert llm_anthropic._is_anthropic_model() == True
# OpenAI model
llm_openai = LLM(model="gpt-4o-mini")
assert llm_openai._is_anthropic_model() == False
# Gemini model
llm_gemini = LLM(model="gemini/gemini-2.0-flash")
assert llm_gemini._is_anthropic_model() == False
class TestLLMBuildMessagesWithPromptCaching:
"""Tests for _build_messages with prompt caching enabled."""
def test_build_messages_with_prompt_caching_anthropic(self):
"""Test that _build_messages adds cache_control for Anthropic models."""
from praisonaiagents.llm.llm import LLM
llm = LLM(model="anthropic/claude-3-5-sonnet-latest", prompt_caching=True)
# Mock _supports_prompt_caching to return True
with patch.object(llm, '_supports_prompt_caching', return_value=True):
messages, _ = llm._build_messages(
prompt="Hello",
system_prompt="You are a helpful assistant."
)
# Check that system message has cache_control
system_msg = messages[0]
assert system_msg["role"] == "system"
# For Anthropic with prompt_caching, content should be an array
assert isinstance(system_msg["content"], list)
assert system_msg["content"][0]["type"] == "text"
assert system_msg["content"][0]["cache_control"] == {"type": "ephemeral"}
def test_build_messages_without_prompt_caching(self):
"""Test that _build_messages doesn't add cache_control when disabled."""
from praisonaiagents.llm.llm import LLM
llm = LLM(model="anthropic/claude-3-5-sonnet-latest", prompt_caching=False)
messages, _ = llm._build_messages(
prompt="Hello",
system_prompt="You are a helpful assistant."
)
# Check that system message is a simple string
system_msg = messages[0]
assert system_msg["role"] == "system"
assert isinstance(system_msg["content"], str)
def test_build_messages_prompt_caching_non_anthropic(self):
"""Test that prompt caching doesn't affect non-Anthropic models."""
from praisonaiagents.llm.llm import LLM
llm = LLM(model="gpt-4o-mini", prompt_caching=True)
messages, _ = llm._build_messages(
prompt="Hello",
system_prompt="You are a helpful assistant."
)
# For non-Anthropic models, content should remain a string
system_msg = messages[0]
assert system_msg["role"] == "system"
assert isinstance(system_msg["content"], str)
class TestAgentPromptCachingParameter:
"""Tests for prompt_caching parameter in Agent class."""
def test_agent_init_with_prompt_caching_true(self):
"""Test Agent initialization with prompt_caching=True."""
from praisonaiagents import Agent
agent = Agent(
name="Test Agent",
llm="anthropic/claude-3-5-sonnet-latest",
prompt_caching=True
)
assert agent.prompt_caching == True
def test_agent_init_with_prompt_caching_false(self):
"""Test Agent initialization with prompt_caching=False."""
from praisonaiagents import Agent
agent = Agent(
name="Test Agent",
llm="anthropic/claude-3-5-sonnet-latest",
prompt_caching=False
)
assert agent.prompt_caching == False
def test_agent_init_without_prompt_caching(self):
"""Test Agent initialization without prompt_caching parameter."""
from praisonaiagents import Agent
agent = Agent(name="Test Agent")
assert agent.prompt_caching is None
def test_agent_model_supports_prompt_caching_method(self):
"""Test _model_supports_prompt_caching method exists."""
from praisonaiagents import Agent
agent = Agent(
name="Test Agent",
llm="anthropic/claude-3-5-sonnet-latest"
)
# Method should exist and return a boolean
result = agent._model_supports_prompt_caching()
assert isinstance(result, bool)
def test_agent_passes_prompt_caching_to_llm(self):
"""Test that Agent passes prompt_caching to LLM instance."""
from praisonaiagents import Agent
agent = Agent(
name="Test Agent",
llm="anthropic/claude-3-5-sonnet-latest",
prompt_caching=True
)
# Check that LLM instance has prompt_caching set
if hasattr(agent, 'llm_instance') and agent.llm_instance:
assert agent.llm_instance.prompt_caching == True
class TestPromptCachingIntegration:
"""Integration tests for prompt caching (requires API key)."""
@pytest.mark.skip(reason="Requires ANTHROPIC_API_KEY environment variable")
def test_prompt_caching_with_anthropic(self):
"""Test prompt caching with actual Anthropic API."""
import os
from praisonaiagents import Agent
# Skip if no API key
if not os.getenv("ANTHROPIC_API_KEY"):
pytest.skip("ANTHROPIC_API_KEY not set")
agent = Agent(
name="Caching Test Agent",
instructions="You are a helpful assistant." * 100, # Long system prompt
llm="anthropic/claude-3-5-sonnet-latest",
prompt_caching=True,
verbose=False
)
# First call - should create cache
result1 = agent.start("What is 2+2?")
assert result1 is not None
# Second call - should use cache
result2 = agent.start("What is 3+3?")
assert result2 is not None
@@ -0,0 +1,284 @@
"""
Test suite for web_fetch feature.
This module tests the web fetch functionality that allows LLMs to retrieve
full content from specific URLs (web pages and PDF documents).
Currently only supported by Anthropic Claude models.
"""
import pytest
import os
class TestSupportsWebFetch:
"""Test the supports_web_fetch function in model_capabilities."""
def test_anthropic_claude_opus_4_1_supports_web_fetch(self):
"""Test that Claude Opus 4.1 supports web fetch."""
from praisonaiagents.llm.model_capabilities import supports_web_fetch
assert supports_web_fetch("anthropic/claude-opus-4-1-20250805")
assert supports_web_fetch("claude-opus-4-1")
def test_anthropic_claude_opus_4_supports_web_fetch(self):
"""Test that Claude Opus 4 supports web fetch."""
from praisonaiagents.llm.model_capabilities import supports_web_fetch
assert supports_web_fetch("anthropic/claude-opus-4-20250514")
assert supports_web_fetch("claude-opus-4")
def test_anthropic_claude_sonnet_4_supports_web_fetch(self):
"""Test that Claude Sonnet 4 supports web fetch."""
from praisonaiagents.llm.model_capabilities import supports_web_fetch
assert supports_web_fetch("anthropic/claude-sonnet-4-20250514")
assert supports_web_fetch("claude-sonnet-4")
def test_anthropic_claude_3_7_sonnet_supports_web_fetch(self):
"""Test that Claude 3.7 Sonnet supports web fetch."""
from praisonaiagents.llm.model_capabilities import supports_web_fetch
assert supports_web_fetch("anthropic/claude-3-7-sonnet-20250219")
assert supports_web_fetch("claude-3-7-sonnet-latest")
def test_anthropic_claude_3_5_sonnet_supports_web_fetch(self):
"""Test that Claude 3.5 Sonnet supports web fetch."""
from praisonaiagents.llm.model_capabilities import supports_web_fetch
assert supports_web_fetch("anthropic/claude-3-5-sonnet-latest")
assert supports_web_fetch("claude-3-5-sonnet-20241022")
def test_anthropic_claude_3_5_haiku_supports_web_fetch(self):
"""Test that Claude 3.5 Haiku supports web fetch."""
from praisonaiagents.llm.model_capabilities import supports_web_fetch
assert supports_web_fetch("anthropic/claude-3-5-haiku-latest")
assert supports_web_fetch("claude-3-5-haiku-20241022")
def test_claude_5_auto_supports_web_fetch(self):
"""Test that Claude 5+ models auto-support web fetch."""
from praisonaiagents.llm.model_capabilities import supports_web_fetch
assert supports_web_fetch("claude-5")
assert supports_web_fetch("claude-sonnet-5")
assert supports_web_fetch("claude-opus-5")
def test_openai_models_do_not_support_web_fetch(self):
"""Test that OpenAI models do not support web fetch."""
from praisonaiagents.llm.model_capabilities import supports_web_fetch
assert not supports_web_fetch("openai/gpt-4o")
assert not supports_web_fetch("gpt-4o-mini")
assert not supports_web_fetch("gpt-4o-search-preview")
def test_gemini_models_do_not_support_web_fetch(self):
"""Test that Gemini models do not support web fetch."""
from praisonaiagents.llm.model_capabilities import supports_web_fetch
assert not supports_web_fetch("gemini/gemini-2.0-flash")
assert not supports_web_fetch("gemini-2.5-pro")
def test_unknown_models_do_not_support_web_fetch(self):
"""Test that unknown models do not support web fetch."""
from praisonaiagents.llm.model_capabilities import supports_web_fetch
assert not supports_web_fetch("ollama/llama3")
assert not supports_web_fetch("unknown/model")
class TestLLMWebFetchParameter:
"""Test the web_fetch parameter in LLM class."""
def test_llm_init_with_web_fetch_true(self):
"""Test LLM initialization with web_fetch=True."""
from praisonaiagents.llm.llm import LLM
llm = LLM(model="anthropic/claude-3-5-sonnet-latest", web_fetch=True)
assert llm.web_fetch is True
def test_llm_init_with_web_fetch_dict(self):
"""Test LLM initialization with web_fetch as dict."""
from praisonaiagents.llm.llm import LLM
web_fetch_config = {
"max_uses": 10,
"allowed_domains": ["example.com"],
"citations": {"enabled": True}
}
llm = LLM(model="anthropic/claude-3-5-sonnet-latest", web_fetch=web_fetch_config)
assert llm.web_fetch == web_fetch_config
def test_llm_init_without_web_fetch(self):
"""Test LLM initialization without web_fetch."""
from praisonaiagents.llm.llm import LLM
llm = LLM(model="gpt-4o-mini")
assert llm.web_fetch is None
def test_llm_supports_web_fetch_method(self):
"""Test LLM._supports_web_fetch() method."""
from praisonaiagents.llm.llm import LLM
# Claude model should support web fetch
llm_claude = LLM(model="anthropic/claude-3-5-sonnet-latest")
assert llm_claude._supports_web_fetch()
# OpenAI model should not support web fetch
llm_openai = LLM(model="gpt-4o-mini")
assert not llm_openai._supports_web_fetch()
class TestLLMBuildCompletionParamsWebFetch:
"""Test _build_completion_params with web_fetch."""
def test_build_params_with_web_fetch_true(self):
"""Test that web_fetch=True adds web_fetch tool to params."""
from praisonaiagents.llm.llm import LLM
llm = LLM(model="anthropic/claude-3-5-sonnet-latest", web_fetch=True)
params = llm._build_completion_params()
assert 'tools' in params
# Find the web_fetch tool
web_fetch_tool = None
for tool in params['tools']:
if isinstance(tool, dict) and tool.get('type') == 'web_fetch_20250910':
web_fetch_tool = tool
break
assert web_fetch_tool is not None
assert web_fetch_tool['name'] == 'web_fetch'
assert web_fetch_tool['max_uses'] == 5 # default
def test_build_params_with_web_fetch_dict(self):
"""Test that web_fetch dict adds customized web_fetch tool."""
from praisonaiagents.llm.llm import LLM
web_fetch_config = {
"max_uses": 10,
"allowed_domains": ["example.com", "docs.example.com"],
"citations": {"enabled": True},
"max_content_tokens": 50000
}
llm = LLM(model="anthropic/claude-3-5-sonnet-latest", web_fetch=web_fetch_config)
params = llm._build_completion_params()
assert 'tools' in params
# Find the web_fetch tool
web_fetch_tool = None
for tool in params['tools']:
if isinstance(tool, dict) and tool.get('type') == 'web_fetch_20250910':
web_fetch_tool = tool
break
assert web_fetch_tool is not None
assert web_fetch_tool['max_uses'] == 10
assert web_fetch_tool['allowed_domains'] == ["example.com", "docs.example.com"]
assert web_fetch_tool['citations'] == {"enabled": True}
assert web_fetch_tool['max_content_tokens'] == 50000
def test_build_params_web_fetch_unsupported_model(self):
"""Test that web_fetch is ignored for unsupported models."""
from praisonaiagents.llm.llm import LLM
llm = LLM(model="gpt-4o-mini", web_fetch=True)
params = llm._build_completion_params()
# Should not have web_fetch tool
if 'tools' in params:
for tool in params['tools']:
if isinstance(tool, dict):
assert tool.get('type') != 'web_fetch_20250910'
class TestAgentWebFetchParameter:
"""Test the web_fetch parameter in Agent class."""
def test_agent_init_with_web_fetch_true(self):
"""Test Agent initialization with web_fetch=True."""
from praisonaiagents.agent.agent import Agent
agent = Agent(
name="Test Agent",
llm="anthropic/claude-3-5-sonnet-latest",
web_fetch=True
)
assert agent.web_fetch is True
def test_agent_init_with_web_fetch_dict(self):
"""Test Agent initialization with web_fetch as dict."""
from praisonaiagents.agent.agent import Agent
web_fetch_config = {"max_uses": 10, "allowed_domains": ["example.com"]}
agent = Agent(
name="Test Agent",
llm="anthropic/claude-3-5-sonnet-latest",
web_fetch=web_fetch_config
)
assert agent.web_fetch == web_fetch_config
def test_agent_init_without_web_fetch(self):
"""Test Agent initialization without web_fetch."""
from praisonaiagents.agent.agent import Agent
agent = Agent(name="Test Agent", llm="gpt-4o-mini")
assert agent.web_fetch is None
def test_agent_passes_web_fetch_to_llm(self):
"""Test that Agent passes web_fetch to LLM instance."""
from praisonaiagents.agent.agent import Agent
agent = Agent(
name="Test Agent",
llm="anthropic/claude-3-5-sonnet-latest",
web_fetch=True
)
assert agent.llm_instance.web_fetch is True
def test_agent_model_supports_web_fetch_method(self):
"""Test Agent._model_supports_web_fetch() method."""
from praisonaiagents.agent.agent import Agent
# Claude model should support web fetch
agent_claude = Agent(
name="Claude Agent",
llm="anthropic/claude-3-5-sonnet-latest"
)
assert agent_claude._model_supports_web_fetch()
# OpenAI model should not support web fetch
agent_openai = Agent(
name="OpenAI Agent",
llm="gpt-4o-mini"
)
assert not agent_openai._model_supports_web_fetch()
class TestWebFetchWithWebSearch:
"""Test that web_fetch and web_search can be used together."""
def test_both_web_fetch_and_web_search(self):
"""Test using both web_fetch and web_search on supported model."""
from praisonaiagents.llm.llm import LLM
llm = LLM(
model="anthropic/claude-3-5-sonnet-latest",
web_search=True,
web_fetch=True
)
params = llm._build_completion_params()
# Should have web_search_options
assert 'web_search_options' in params
# Should also have web_fetch tool
assert 'tools' in params
web_fetch_found = False
for tool in params['tools']:
if isinstance(tool, dict) and tool.get('type') == 'web_fetch_20250910':
web_fetch_found = True
break
assert web_fetch_found
@pytest.mark.skipif(
not os.getenv('ANTHROPIC_API_KEY'),
reason="ANTHROPIC_API_KEY not set"
)
class TestWebFetchIntegration:
"""Integration tests for web_fetch (requires API key)."""
def test_web_fetch_with_anthropic(self):
"""Test actual web fetch with Anthropic API."""
from praisonaiagents.agent.agent import Agent
agent = Agent(
name="Web Fetch Agent",
instructions="You are a helpful assistant that can fetch web content.",
llm="anthropic/claude-3-5-sonnet-latest",
web_fetch=True
)
# This would make an actual API call
# response = agent.chat("Fetch and summarize https://example.com")
# assert response is not None
pass # Skip actual API call in tests
@@ -0,0 +1,255 @@
"""
Test file for web_search feature (TDD approach).
Tests the LiteLLM native web search integration with fallback to DuckDuckGo.
"""
import pytest
import os
from unittest.mock import patch
class TestWebSearchSupport:
"""Test web search capability detection."""
def test_supports_web_search_openai_search_preview(self):
"""Test that gpt-4o-search-preview is detected as supporting web search."""
from praisonaiagents.llm.model_capabilities import supports_web_search
assert supports_web_search("openai/gpt-4o-search-preview") == True
def test_supports_web_search_xai_grok(self):
"""Test that xai/grok-3 is detected as supporting web search."""
from praisonaiagents.llm.model_capabilities import supports_web_search
assert supports_web_search("xai/grok-3") == True
def test_supports_web_search_anthropic_claude(self):
"""Test that claude-3-5-sonnet-latest is detected as supporting web search."""
from praisonaiagents.llm.model_capabilities import supports_web_search
assert supports_web_search("anthropic/claude-3-5-sonnet-latest") == True
def test_supports_web_search_gemini(self):
"""Test that gemini-2.0-flash is detected as supporting web search."""
from praisonaiagents.llm.model_capabilities import supports_web_search
assert supports_web_search("gemini-2.0-flash") == True
def test_supports_web_search_ollama_false(self):
"""Test that ollama models don't support native web search."""
from praisonaiagents.llm.model_capabilities import supports_web_search
assert supports_web_search("ollama/llama3") == False
def test_supports_web_search_unknown_model(self):
"""Test that unknown models default to False."""
from praisonaiagents.llm.model_capabilities import supports_web_search
assert supports_web_search("unknown/model") == False
class TestLLMWebSearchParameter:
"""Test LLM class web_search parameter."""
def test_llm_init_with_web_search_true(self):
"""Test LLM initialization with web_search=True."""
from praisonaiagents.llm.llm import LLM
llm = LLM(model="gpt-4o-mini", web_search=True)
assert llm.web_search == True
def test_llm_init_with_web_search_dict(self):
"""Test LLM initialization with web_search as dict."""
from praisonaiagents.llm.llm import LLM
web_search_options = {"search_context_size": "high"}
llm = LLM(model="gpt-4o-mini", web_search=web_search_options)
assert llm.web_search == web_search_options
def test_llm_init_with_web_search_false(self):
"""Test LLM initialization with web_search=False."""
from praisonaiagents.llm.llm import LLM
llm = LLM(model="gpt-4o-mini", web_search=False)
assert llm.web_search == False
def test_llm_init_without_web_search(self):
"""Test LLM initialization without web_search (default None)."""
from praisonaiagents.llm.llm import LLM
llm = LLM(model="gpt-4o-mini")
assert llm.web_search is None
def test_llm_supports_web_search_method(self):
"""Test LLM._supports_web_search() method."""
from praisonaiagents.llm.llm import LLM
llm = LLM(model="openai/gpt-4o-search-preview")
assert llm._supports_web_search() == True
def test_llm_supports_web_search_method_ollama(self):
"""Test LLM._supports_web_search() returns False for Ollama."""
from praisonaiagents.llm.llm import LLM
llm = LLM(model="ollama/llama3")
assert llm._supports_web_search() == False
class TestLLMBuildCompletionParams:
"""Test _build_completion_params includes web_search_options."""
def test_build_params_with_web_search_true(self):
"""Test that web_search=True adds web_search_options to params."""
from praisonaiagents.llm.llm import LLM
llm = LLM(model="openai/gpt-4o-search-preview", web_search=True)
with patch.object(llm, '_supports_web_search', return_value=True):
params = llm._build_completion_params(messages=[{"role": "user", "content": "test"}])
assert "web_search_options" in params
assert params["web_search_options"] == {"search_context_size": "medium"}
def test_build_params_with_web_search_dict(self):
"""Test that web_search dict is passed correctly."""
from praisonaiagents.llm.llm import LLM
web_search_options = {"search_context_size": "high"}
llm = LLM(model="openai/gpt-4o-search-preview", web_search=web_search_options)
with patch.object(llm, '_supports_web_search', return_value=True):
params = llm._build_completion_params(messages=[{"role": "user", "content": "test"}])
assert "web_search_options" in params
assert params["web_search_options"] == {"search_context_size": "high"}
def test_build_params_without_web_search(self):
"""Test that web_search_options is not added when web_search is None."""
from praisonaiagents.llm.llm import LLM
llm = LLM(model="gpt-4o-mini")
params = llm._build_completion_params(messages=[{"role": "user", "content": "test"}])
assert "web_search_options" not in params
def test_build_params_web_search_unsupported_model(self):
"""Test that web_search_options is not added for unsupported models."""
from praisonaiagents.llm.llm import LLM
llm = LLM(model="ollama/llama3", web_search=True)
with patch.object(llm, '_supports_web_search', return_value=False):
params = llm._build_completion_params(messages=[{"role": "user", "content": "test"}])
assert "web_search_options" not in params
class TestAgentWebSearchParameter:
"""Test Agent class web_search parameter."""
def test_agent_init_with_web_search_true(self):
"""Test Agent initialization with web_search=True."""
from praisonaiagents import Agent
agent = Agent(
name="Test Agent",
instructions="Test",
web_search=True
)
assert agent.web_search == True
def test_agent_init_with_web_search_dict(self):
"""Test Agent initialization with web_search as dict."""
from praisonaiagents import Agent
web_search_options = {"search_context_size": "high"}
agent = Agent(
name="Test Agent",
instructions="Test",
web_search=web_search_options
)
assert agent.web_search == web_search_options
def test_agent_init_without_web_search(self):
"""Test Agent initialization without web_search (default None)."""
from praisonaiagents import Agent
agent = Agent(
name="Test Agent",
instructions="Test"
)
assert agent.web_search is None
def test_agent_with_custom_llm_passes_web_search(self):
"""Test that Agent passes web_search to custom LLM instance."""
from praisonaiagents import Agent
agent = Agent(
name="Test Agent",
instructions="Test",
llm="openai/gpt-4o-search-preview",
web_search=True
)
# When using custom LLM (with /), it creates llm_instance
assert hasattr(agent, 'llm_instance')
assert agent.llm_instance.web_search == True
class TestWebSearchFallback:
"""Test fallback to DuckDuckGo when native web search not supported."""
def test_fallback_tool_injection_for_unsupported_model(self):
"""Test that DuckDuckGo tool is injected for unsupported models."""
from praisonaiagents import Agent
agent = Agent(
name="Test Agent",
instructions="Test",
llm="ollama/llama3",
web_search=True
)
# Check that internet_search tool was added to tools
tool_names = [getattr(t, '__name__', str(t)) for t in agent.tools]
assert 'internet_search' in tool_names or 'duckduckgo' in tool_names
def test_no_fallback_for_supported_model(self):
"""Test that no fallback tool is added for supported models."""
from praisonaiagents import Agent
agent = Agent(
name="Test Agent",
instructions="Test",
llm="openai/gpt-4o-search-preview",
web_search=True,
tools=[] # Explicitly empty tools
)
# For supported models, tools should remain empty (native web search used)
# Only check if no DuckDuckGo tool was auto-added
tool_names = [getattr(t, '__name__', str(t)) for t in agent.tools]
# internet_search should NOT be auto-added for supported models
assert 'internet_search' not in tool_names
class TestDuckDuckGoToolPreservation:
"""Test that DuckDuckGo tools still work as before."""
def test_duckduckgo_import(self):
"""Test that DuckDuckGo tools can still be imported."""
from praisonaiagents.tools import internet_search, duckduckgo
assert callable(internet_search)
assert callable(duckduckgo)
def test_explicit_duckduckgo_tool_usage(self):
"""Test that explicit DuckDuckGo tool usage still works."""
from praisonaiagents import Agent
from praisonaiagents.tools import internet_search
agent = Agent(
name="Test Agent",
instructions="Test",
tools=[internet_search]
)
assert internet_search in agent.tools
class TestWebSearchIntegration:
"""Integration tests for web search (requires API keys)."""
@pytest.mark.skipif(
not os.getenv("OPENAI_API_KEY"),
reason="OPENAI_API_KEY not set"
)
def test_web_search_with_openai(self):
"""Test actual web search with OpenAI (integration test)."""
from praisonaiagents import Agent
agent = Agent(
name="Researcher",
instructions="You are a helpful research assistant.",
llm="openai/gpt-4o-search-preview",
web_search=True
)
result = agent.start("What is the current weather in San Francisco?")
assert result is not None
assert len(result) > 0
if __name__ == "__main__":
pytest.main([__file__, "-v"])
+1 -1
View File
@@ -2422,7 +2422,7 @@ wheels = [
[[package]]
name = "praisonaiagents"
version = "0.0.169"
version = "0.0.175"
source = { editable = "." }
dependencies = [
{ name = "aiohttp" },
+53
View File
@@ -55,6 +55,9 @@ PraisonAI is a production-ready Multi-AI Agents framework with self-reflection,
- 💯 100+ LLM Support
- 🔬 **Deep Research Agents** (OpenAI & Gemini)
- 🔄 **Query Rewriter Agent** (HyDE, Step-back, Multi-query)
- 🌐 **Native Web Search** (OpenAI, Gemini, Anthropic, xAI, Perplexity)
- 📥 **Web Fetch** (Retrieve full content from URLs - Anthropic)
- 💾 **Prompt Caching** (Reduce costs & latency - OpenAI, Anthropic, Bedrock, Deepseek)
## Using Python Code
@@ -759,6 +762,56 @@ result = agent.expand("blog about AI", strategy=ExpandStrategy.DETAILED)
- `--expand-prompt`: Expands prompts for **detailed task execution**
## Web Search, Web Fetch & Prompt Caching
### CLI Usage
```bash
# Web Search - Get real-time information
praisonai "What are the latest AI news today?" --web-search --llm openai/gpt-4o-search-preview
# Web Fetch - Retrieve and analyze URL content (Anthropic only)
praisonai "Summarize https://docs.praison.ai" --web-fetch --llm anthropic/claude-sonnet-4-20250514
# Prompt Caching - Reduce costs for repeated prompts
praisonai "Analyze this document..." --prompt-caching --llm anthropic/claude-sonnet-4-20250514
```
### Programmatic Usage
```python
from praisonaiagents import Agent
# Web Search
agent = Agent(
instructions="You are a research assistant",
llm="openai/gpt-4o-search-preview",
web_search=True
)
# Web Fetch (Anthropic only)
agent = Agent(
instructions="You are a content analyzer",
llm="anthropic/claude-sonnet-4-20250514",
web_fetch=True
)
# Prompt Caching
agent = Agent(
instructions="You are an AI assistant..." * 50, # Long system prompt
llm="anthropic/claude-sonnet-4-20250514",
prompt_caching=True
)
```
**Supported Providers:**
| Feature | Providers |
|---------|-----------|
| Web Search | OpenAI, Gemini, Anthropic, xAI, Perplexity |
| Web Fetch | Anthropic |
| Prompt Caching | OpenAI (auto), Anthropic, Bedrock, Deepseek |
## Development:
Below is used for development only.
+2 -2
View File
@@ -3,8 +3,8 @@ class Praisonai < Formula
desc "AI tools for various AI applications"
homepage "https://github.com/MervinPraison/PraisonAI"
url "https://github.com/MervinPraison/PraisonAI/archive/refs/tags/v2.3.0.tar.gz"
sha256 `curl -sL https://github.com/MervinPraison/PraisonAI/archive/refs/tags/v2.3.0.tar.gz | shasum -a 256`.split.first
url "https://github.com/MervinPraison/PraisonAI/archive/refs/tags/v2.3.2.tar.gz"
sha256 `curl -sL https://github.com/MervinPraison/PraisonAI/archive/refs/tags/v2.3.2.tar.gz | shasum -a 256`.split.first
license "MIT"
depends_on "[email protected]"
+46
View File
@@ -584,6 +584,9 @@ class PraisonAI:
parser.add_argument("--tools", "-t", type=str, help="Path to tools.py file for research agent")
parser.add_argument("--save", "-s", action="store_true", help="Save research output to file (output/research/)")
parser.add_argument("--verbose", "-v", action="store_true", help="Enable verbose output for research")
parser.add_argument("--web-search", action="store_true", help="Enable native web search (OpenAI, Gemini, Anthropic, xAI, Perplexity)")
parser.add_argument("--web-fetch", action="store_true", help="Enable web fetch to retrieve URL content (Anthropic only)")
parser.add_argument("--prompt-caching", action="store_true", help="Enable prompt caching to reduce costs (OpenAI, Anthropic, Bedrock, Deepseek)")
# If we're in a test environment, parse with empty args to avoid pytest interference
if in_test_env:
@@ -933,6 +936,35 @@ class PraisonAI:
verbose = getattr(self.args, 'verbose', False)
return self._expand_prompt(prompt, expand_tools, verbose)
def _save_output(self, prompt: str, result: str):
"""
Save output to output/prompts/ folder.
Args:
prompt: The original prompt
result: The output result to save
"""
from datetime import datetime
from rich import print
# Create output directory
output_dir = os.path.join(os.getcwd(), "output", "prompts")
os.makedirs(output_dir, exist_ok=True)
# Generate filename with timestamp
timestamp = datetime.now().strftime("%Y-%m-%d_%H-%M-%S")
# Create a safe filename from prompt (first 30 chars)
safe_prompt = "".join(c if c.isalnum() or c in " -_" else "" for c in prompt[:30]).strip().replace(" ", "_")
filename = f"{timestamp}_{safe_prompt}.md"
filepath = os.path.join(output_dir, filename)
# Write output
with open(filepath, "w", encoding="utf-8") as f:
f.write(f"# Prompt\n\n{prompt}\n\n")
f.write(f"# Output\n\n{result}\n")
print(f"[green]✅ Output saved to: {filepath}[/green]")
def handle_direct_prompt(self, prompt):
"""
Handle direct prompt by creating a single agent and running it.
@@ -954,8 +986,22 @@ class PraisonAI:
if hasattr(self, 'args') and self.args.llm:
agent_config["llm"] = self.args.llm
# Add feature flags if enabled
if hasattr(self, 'args'):
if getattr(self.args, 'web_search', False):
agent_config["web_search"] = True
if getattr(self.args, 'web_fetch', False):
agent_config["web_fetch"] = True
if getattr(self.args, 'prompt_caching', False):
agent_config["prompt_caching"] = True
agent = PraisonAgent(**agent_config)
result = agent.start(prompt)
# Save output if --save is enabled
if hasattr(self, 'args') and getattr(self.args, 'save', False):
self._save_output(prompt, result)
return result
elif CREWAI_AVAILABLE:
from crewai import Agent, Task, Crew
+1 -1
View File
@@ -57,7 +57,7 @@ class CloudDeployer:
file.write("FROM python:3.11-slim\n")
file.write("WORKDIR /app\n")
file.write("COPY . .\n")
file.write("RUN pip install flask praisonai==2.3.0 gunicorn markdown\n")
file.write("RUN pip install flask praisonai==2.3.2 gunicorn markdown\n")
file.write("EXPOSE 8080\n")
file.write('CMD ["gunicorn", "-b", "0.0.0.0:8080", "api:app"]\n')
+1 -1
View File
@@ -1 +1 @@
__version__ = "2.3.0"
__version__ = "2.3.2"
+1 -1
View File
@@ -12,7 +12,7 @@ dependencies = [
"rich>=13.7",
"markdown>=3.5",
"pyparsing>=3.0.0",
"praisonaiagents>=0.0.170",
"praisonaiagents>=0.0.175",
"python-dotenv>=0.19.0",
"instructor>=1.3.3",
"PyYAML>=6.0",
+4 -1
View File
@@ -145,7 +145,10 @@ def release(version: str):
shutil.copy(root_readme, pkg_readme)
print(f" ✅ Copied {root_readme} -> {pkg_readme}")
# 2. uv lock
# 2. Clear uv cache and run uv lock
print("\n🧹 Clearing uv cache...")
run(["uv", "cache", "clean"], cwd=praisonai_dir)
print("\n📦 Running uv lock...")
run(["uv", "lock"], cwd=praisonai_dir)
+4 -1
View File
@@ -71,7 +71,10 @@ def main():
shutil.copy(root_readme, pkg_readme)
print(f" ✅ Copied {root_readme} -> {pkg_readme}")
# 2. uv lock
# 2. Clear uv cache and run uv lock
print("\n🧹 Clearing uv cache...")
run(["uv", "cache", "clean"], cwd=praisonai_dir)
print("\n📦 Running uv lock...")
run(["uv", "lock"], cwd=praisonai_dir)
+4 -4
View File
@@ -4197,7 +4197,7 @@ requires-dist = [
{ name = "praisonai-tools", marker = "extra == 'autogen'", specifier = ">=0.0.22" },
{ name = "praisonai-tools", marker = "extra == 'autogen-v4'", specifier = ">=0.0.22" },
{ name = "praisonai-tools", marker = "extra == 'crewai'", specifier = ">=0.0.22" },
{ name = "praisonaiagents", specifier = ">=0.0.170" },
{ name = "praisonaiagents", specifier = ">=0.0.175" },
{ name = "pyautogen", marker = "extra == 'autogen'", specifier = "==0.2.29" },
{ name = "pydantic", marker = "extra == 'chat'", specifier = "<=2.10.1" },
{ name = "pydantic", marker = "extra == 'code'", specifier = "<=2.10.1" },
@@ -4252,7 +4252,7 @@ wheels = [
[[package]]
name = "praisonaiagents"
version = "0.0.170"
version = "0.0.175"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "aiohttp" },
@@ -4261,9 +4261,9 @@ dependencies = [
{ name = "pydantic" },
{ name = "rich" },
]
sdist = { url = "https://files.pythonhosted.org/packages/20/35/a45a1f616bc0a661b9816b7cd7afdace803bacdc91cbacc6080960e7d1fb/praisonaiagents-0.0.170.tar.gz", hash = "sha256:b0114255b82184657af074f55316238c1ddcb7105aab6b23817d0315830d2082", size = 324448, upload-time = "2025-12-15T11:12:25.135Z" }
sdist = { url = "https://files.pythonhosted.org/packages/a7/3a/a3f78a53c5ffdab876b59c7f7ea86ed815d641dff45465c70b6fd8ab1a8f/praisonaiagents-0.0.175.tar.gz", hash = "sha256:5f2bdeae4988a34947aa231c7a3de76bf50d60d2dfd43e214769d188886d6362", size = 332558, upload-time = "2025-12-15T12:43:06.491Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/b1/cc/6a6c8da467f400ab51201bd0008948326483ccae89d370a32b46e66f87ae/praisonaiagents-0.0.170-py3-none-any.whl", hash = "sha256:5930c1a9a4d1b6ae5f03e471351701b5c0af21a8312fdb2c9b756a89e88cdd4d", size = 337210, upload-time = "2025-12-15T11:12:23.518Z" },
{ url = "https://files.pythonhosted.org/packages/ee/c9/93cf31b2c1011ee205efc75080a4d7ab9ed68e53e033d51a861112545428/praisonaiagents-0.0.175-py3-none-any.whl", hash = "sha256:1bdb672efefd562c47be481008551894e57083e52005227650f97999f88e7ff6", size = 339912, upload-time = "2025-12-15T12:43:05.064Z" },
]
[[package]]