mirror of
https://github.com/MervinPraison/PraisonAI.git
synced 2026-11-10 03:31:28 -10:00
Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
 Mervin Praison
|
585b68d02c | ||
|
Mervin Praison
|
66481cc51c | ||
|
Mervin Praison
|
8e637daecd | ||
|
Mervin Praison
|
ba482895f7 | ||
|
Mervin Praison
|
ba09407981 |
No files matched your search
@@ -1,105 +0,0 @@
|
||||
# Agent Migration Ledger
|
||||
|
||||
Generated: 2026-01-09
|
||||
|
||||
## 1. Agent Definition Files (Source of Truth)
|
||||
|
||||
### Core SDK (praisonai-agents)
|
||||
| File | Line | Description |
|
||||
|------|------|-------------|
|
||||
| `/Users/praison/praisonai-package/src/praisonai-agents/praisonaiagents/agent/agent.py:50` | `class Agent` | Main Agent class definition |
|
||||
|
||||
### Related Classes
|
||||
| File | Line | Description |
|
||||
|------|------|-------------|
|
||||
| `/Users/praison/praisonai-package/src/praisonai-agents/praisonaiagents/agents/agents.py:129` | `class Agents` | Multi-agent orchestrator (PraisonAIAgents) |
|
||||
| `/Users/praison/praisonai-package/src/praisonai-agents/praisonaiagents/agents/autoagents.py:26` | `class AgentConfig` | Auto-generated agent config |
|
||||
|
||||
## 2. Agent Example Files Count
|
||||
|
||||
- **Total example files with Agent()**: 467+
|
||||
- **Key example directories**:
|
||||
- `/examples/python/` - Main Python examples
|
||||
- `/examples/context/` - Context management examples
|
||||
- `/examples/knowledge/` - Knowledge/RAG examples
|
||||
- `/examples/guardrails/` - Guardrail examples
|
||||
- `/examples/execution/` - Execution config examples
|
||||
- `/examples/eval/` - Evaluation examples
|
||||
|
||||
## 3. Agent Docs Pages Count
|
||||
|
||||
- **Total docs pages with Agent()**: 806+
|
||||
- **Key docs directories**:
|
||||
- `/docs/features/` - Feature documentation
|
||||
- `/docs/sdk/` - SDK reference
|
||||
- `/docs/guides/` - User guides
|
||||
- `/docs/examples/` - Example documentation
|
||||
|
||||
## 4. Agent API Signature (Current)
|
||||
|
||||
The Agent class uses a consolidated API pattern. Key parameters:
|
||||
|
||||
```python
|
||||
class Agent:
|
||||
def __init__(
|
||||
self,
|
||||
name: str = "Agent",
|
||||
role: str = "Assistant",
|
||||
goal: str = "Help the user",
|
||||
backstory: str = "",
|
||||
instructions: str = "",
|
||||
llm: Optional[str] = None,
|
||||
tools: Optional[List] = None,
|
||||
|
||||
# Consolidated feature params (Instance > Config > String > Bool > Default)
|
||||
memory: Optional[Any] = None, # bool | str | MemoryConfig | Memory instance
|
||||
knowledge: Optional[Any] = None, # bool | str | KnowledgeConfig | Knowledge instance
|
||||
output: Optional[Any] = None, # bool | str | OutputConfig
|
||||
execution: Optional[Any] = None, # bool | str | ExecutionConfig
|
||||
guardrails: Optional[Any] = None, # bool | str | GuardrailConfig | List
|
||||
context: Optional[Any] = None, # bool | str | ContextConfig
|
||||
|
||||
# Other params
|
||||
verbose: bool = False,
|
||||
markdown: bool = True,
|
||||
self_reflect: bool = False,
|
||||
max_reflect: int = 3,
|
||||
min_reflect: int = 1,
|
||||
...
|
||||
)
|
||||
```
|
||||
|
||||
## 5. Migration Status
|
||||
|
||||
| Category | Total | Verified | Status |
|
||||
|----------|-------|----------|--------|
|
||||
| Agent Definition | 1 | 1 | ✅ Complete |
|
||||
| Agent Examples | 467+ | 0 | ⏳ Pending |
|
||||
| Agent Docs | 806+ | 0 | ⏳ Pending |
|
||||
| CLI | TBD | 0 | ⏳ Pending |
|
||||
|
||||
## 6. Action Log
|
||||
|
||||
| Date | Action | Status |
|
||||
|------|--------|--------|
|
||||
| 2026-01-09 | Created Agent Migration Ledger | ✅ Complete |
|
||||
| 2026-01-09 | Verified Agent class definition | ✅ Complete |
|
||||
| 2026-01-09 | Workflow markdown parser fixed | ✅ Complete |
|
||||
|
||||
## 7. Final Verification (2026-01-09)
|
||||
|
||||
```
|
||||
=== AGENT API VERIFICATION ===
|
||||
✅ Agent class has consolidated params: memory, knowledge, planning, reflection, guardrails, web, context, autonomy, output, execution, caching, hooks, skills
|
||||
✅ Agent follows Instance > Config > String > Bool > Default precedence
|
||||
✅ Agent creation with consolidated params works correctly
|
||||
```
|
||||
|
||||
## 8. Notes
|
||||
|
||||
The Agent class already uses a well-designed consolidated API pattern:
|
||||
- All feature params support: `False` (disabled), `True` (defaults), `Config` (custom)
|
||||
- Precedence: Instance > Config > String > Bool > Default
|
||||
- Agent-centric imports: `from praisonaiagents import Agent`
|
||||
|
||||
No legacy patterns found in examples or docs that require updating.
|
||||
@@ -1,258 +0,0 @@
|
||||
# Workflow Migration Ledger
|
||||
|
||||
Generated: 2026-01-09
|
||||
|
||||
## 1. Workflow Definition Files (Source of Truth)
|
||||
|
||||
### Core SDK (praisonai-agents)
|
||||
| File | Classes/Configs Defined |
|
||||
|------|------------------------|
|
||||
| `/Users/praison/praisonai-package/src/praisonai-agents/praisonaiagents/workflows/workflows.py:184` | `WorkflowStep` dataclass |
|
||||
| `/Users/praison/praisonai-package/src/praisonai-agents/praisonaiagents/workflows/workflows.py:381` | `Workflow` dataclass |
|
||||
| `/Users/praison/praisonai-package/src/praisonai-agents/praisonaiagents/workflows/workflow_configs.py` | `WorkflowOutputConfig`, `WorkflowPlanningConfig`, `WorkflowMemoryConfig`, `WorkflowHooksConfig`, `WorkflowStepContextConfig`, `WorkflowStepOutputConfig`, `WorkflowStepExecutionConfig`, `WorkflowStepRoutingConfig` |
|
||||
|
||||
### Wrapper/CLI (praisonai)
|
||||
| File | Description |
|
||||
|------|-------------|
|
||||
| `/Users/praison/praisonai-package/src/praisonai/praisonai/cli/features/workflow.py` | `WorkflowHandler` CLI handler |
|
||||
| `/Users/praison/praisonai-package/src/praisonai/praisonai/agents_generator.py` | Uses `YAMLWorkflowParser` |
|
||||
|
||||
## 2. Workflow Callsites (Files Using Workflow/WorkflowStep)
|
||||
|
||||
### Core SDK Internal
|
||||
- `/Users/praison/praisonai-package/src/praisonai-agents/praisonaiagents/workflows/__init__.py`
|
||||
- `/Users/praison/praisonai-package/src/praisonai-agents/praisonaiagents/workflows/yaml_parser.py`
|
||||
- `/Users/praison/praisonai-package/src/praisonai-agents/praisonaiagents/__init__.py`
|
||||
|
||||
### Wrapper/CLI
|
||||
- `/Users/praison/praisonai-package/src/praisonai/praisonai/agents_generator.py`
|
||||
- `/Users/praison/praisonai-package/src/praisonai/praisonai/cli/features/workflow.py`
|
||||
- `/Users/praison/praisonai-package/src/praisonai/praisonai/cli/features/templates.py`
|
||||
- `/Users/praison/praisonai-package/src/praisonai/praisonai/cli/features/benchmark.py`
|
||||
- `/Users/praison/praisonai-package/src/praisonai/praisonai/cli/features/flow_display.py`
|
||||
- `/Users/praison/praisonai-package/src/praisonai/praisonai/cli/features/n8n.py`
|
||||
|
||||
### Tests
|
||||
- `/Users/praison/praisonai-package/src/praisonai-agents/tests/test_workflow_patterns.py`
|
||||
- `/Users/praison/praisonai-package/src/praisonai-agents/tests/unit/workflows/test_workflow_consolidated_api.py`
|
||||
|
||||
## 3. Workflow-Related Examples (67 files)
|
||||
|
||||
### Primary Workflow Examples (examples/python/workflows/)
|
||||
1. `/Users/praison/praisonai-package/examples/python/workflows/simple_workflow.py`
|
||||
2. `/Users/praison/praisonai-package/examples/python/workflows/simple_workflow_non_agentic.py`
|
||||
3. `/Users/praison/praisonai-package/examples/python/workflows/workflow_routing.py`
|
||||
4. `/Users/praison/praisonai-package/examples/python/workflows/workflow_routing_non_agentic.py`
|
||||
5. `/Users/praison/praisonai-package/examples/python/workflows/workflow_parallel.py`
|
||||
6. `/Users/praison/praisonai-package/examples/python/workflows/workflow_parallel_non_agentic.py`
|
||||
7. `/Users/praison/praisonai-package/examples/python/workflows/workflow_repeat.py`
|
||||
8. `/Users/praison/praisonai-package/examples/python/workflows/workflow_repeat_non_agentic.py`
|
||||
9. `/Users/praison/praisonai-package/examples/python/workflows/workflow_loop_csv.py`
|
||||
10. `/Users/praison/praisonai-package/examples/python/workflows/workflow_loop_csv_non_agentic.py`
|
||||
11. `/Users/praison/praisonai-package/examples/python/workflows/workflow_loop_list.py`
|
||||
12. `/Users/praison/praisonai-package/examples/python/workflows/workflow_loops.py`
|
||||
13. `/Users/praison/praisonai-package/examples/python/workflows/workflow_mixed_steps.py`
|
||||
14. `/Users/praison/praisonai-package/examples/python/workflows/workflow_branching.py`
|
||||
15. `/Users/praison/praisonai-package/examples/python/workflows/workflow_conditional.py`
|
||||
16. `/Users/praison/praisonai-package/examples/python/workflows/workflow_conditional_non_agentic.py`
|
||||
17. `/Users/praison/praisonai-package/examples/python/workflows/workflow_early_stop.py`
|
||||
18. `/Users/praison/praisonai-package/examples/python/workflows/workflow_checkpoints.py`
|
||||
19. `/Users/praison/praisonai-package/examples/python/workflows/workflow_with_agents.py`
|
||||
20. `/Users/praison/praisonai-package/examples/python/workflows/task_callbacks.py`
|
||||
|
||||
### Other Examples with Workflow References
|
||||
21. `/Users/praison/praisonai-package/examples/python/general/workflow_example_basic.py`
|
||||
22. `/Users/praison/praisonai-package/examples/python/general/workflow_example_detailed.py`
|
||||
23. `/Users/praison/praisonai-package/examples/python/general/prompt_chaining.py`
|
||||
24. `/Users/praison/praisonai-package/examples/python/general/async_example.py`
|
||||
25. `/Users/praison/praisonai-package/examples/python/general/autonomous-agent.py`
|
||||
26. `/Users/praison/praisonai-package/examples/python/processes/advanced-workflow-patterns.py`
|
||||
27. `/Users/praison/praisonai-package/examples/python/agents/autoagents_workflow_patterns.py`
|
||||
28. `/Users/praison/praisonai-package/examples/python/concepts/context-engineering-workflow.py`
|
||||
29. `/Users/praison/praisonai-package/examples/python/stateful/workflow-state-example.py`
|
||||
30. `/Users/praison/praisonai-package/examples/routing/routellm_workflow.py`
|
||||
31. `/Users/praison/praisonai-package/examples/yaml/workflows/example_usage.py`
|
||||
|
||||
### Use Case Examples (mention workflow in comments/prints)
|
||||
32-67. Various use case examples in `/Users/praison/praisonai-package/examples/python/usecases/`
|
||||
|
||||
## 4. Workflow-Related Docs Pages (Mintlify)
|
||||
|
||||
### Primary Workflow Docs
|
||||
1. `/Users/praison/PraisonAIDocs/docs/features/workflows.mdx`
|
||||
2. `/Users/praison/PraisonAIDocs/docs/features/workflow-patterns.mdx`
|
||||
3. `/Users/praison/PraisonAIDocs/docs/features/workflow-routing.mdx`
|
||||
4. `/Users/praison/PraisonAIDocs/docs/features/workflow-parallel.mdx`
|
||||
5. `/Users/praison/PraisonAIDocs/docs/features/workflow-loop.mdx`
|
||||
6. `/Users/praison/PraisonAIDocs/docs/features/workflow-repeat.mdx`
|
||||
7. `/Users/praison/PraisonAIDocs/docs/features/workflow-validation.mdx`
|
||||
8. `/Users/praison/PraisonAIDocs/docs/features/yaml-workflows.mdx`
|
||||
9. `/Users/praison/PraisonAIDocs/docs/features/autonomous-workflow.mdx`
|
||||
10. `/Users/praison/PraisonAIDocs/docs/cli/workflow.mdx`
|
||||
|
||||
### SDK Reference Docs
|
||||
11. `/Users/praison/PraisonAIDocs/docs/sdk/praisonaiagents/workflows/workflows.mdx`
|
||||
12. `/Users/praison/PraisonAIDocs/docs/sdk/praisonaiagents/workflows/workflow-manager.mdx`
|
||||
|
||||
### Guide Docs
|
||||
13. `/Users/praison/PraisonAIDocs/docs/guides/workflows/index.mdx`
|
||||
14. `/Users/praison/PraisonAIDocs/docs/guides/workflows/sequential.mdx`
|
||||
15. `/Users/praison/PraisonAIDocs/docs/guides/workflows/parallel.mdx`
|
||||
16. `/Users/praison/PraisonAIDocs/docs/guides/workflows/routing.mdx`
|
||||
17. `/Users/praison/PraisonAIDocs/docs/guides/workflows/orchestrator.mdx`
|
||||
|
||||
### JS/TypeScript Docs
|
||||
18. `/Users/praison/PraisonAIDocs/docs/js/workflows.mdx`
|
||||
19. `/Users/praison/PraisonAIDocs/docs/js/workflows-cli.mdx`
|
||||
|
||||
### Other Docs with Workflow References
|
||||
20. `/Users/praison/PraisonAIDocs/docs/concepts/process.mdx`
|
||||
21. `/Users/praison/PraisonAIDocs/docs/features/routing.mdx`
|
||||
22. `/Users/praison/PraisonAIDocs/docs/features/promptchaining.mdx`
|
||||
23. `/Users/praison/PraisonAIDocs/docs/features/evaluator-optimiser.mdx`
|
||||
24. `/Users/praison/PraisonAIDocs/docs/features/parallelisation.mdx`
|
||||
|
||||
## 5. Google Docs URLs
|
||||
|
||||
**None found in repo code/docs** (only in node_modules third-party dependencies)
|
||||
|
||||
## 6. Baseline API Signatures
|
||||
|
||||
### Workflow Dataclass Fields (Consolidated API - CURRENT)
|
||||
```
|
||||
name: str
|
||||
description: str = ""
|
||||
steps: List[Any] = field(default_factory=list)
|
||||
variables: Dict[str, Any] = field(default_factory=dict)
|
||||
file_path: Optional[str] = None
|
||||
default_agent_config: Optional[Dict[str, Any]] = None
|
||||
default_llm: Optional[str] = None
|
||||
context: Optional[Any] = None # Consolidated context param
|
||||
|
||||
# CONSOLIDATED FEATURE PARAMS
|
||||
output: Optional[Any] = None # Union[bool, str, WorkflowOutputConfig]
|
||||
planning: Optional[Any] = None # Union[bool, WorkflowPlanningConfig]
|
||||
memory: Optional[Any] = None # Union[bool, WorkflowMemoryConfig]
|
||||
hooks: Optional[Any] = None # WorkflowHooksConfig
|
||||
|
||||
# Internal resolved fields (private)
|
||||
_verbose, _stream, _planning_enabled, _planning_llm, _reasoning, _memory_config, _hooks_config
|
||||
```
|
||||
|
||||
### WorkflowStep Dataclass Fields (Consolidated API - CURRENT)
|
||||
```
|
||||
name: str
|
||||
description: str = ""
|
||||
action: str = ""
|
||||
condition: Optional[str] = None
|
||||
should_run: Optional[Callable] = None
|
||||
handler: Optional[Callable] = None
|
||||
agent_config: Optional[Dict[str, Any]] = None
|
||||
agent: Optional[Any] = None
|
||||
tools: Optional[List[Any]] = None
|
||||
loop_over: Optional[str] = None
|
||||
loop_var: str = "item"
|
||||
guardrail: Optional[Callable] = None
|
||||
images: Optional[List[str]] = None
|
||||
|
||||
# CONSOLIDATED FEATURE PARAMS
|
||||
context: Optional[Any] = None # Union[bool, List[str], WorkflowStepContextConfig]
|
||||
output: Optional[Any] = None # Union[str, WorkflowStepOutputConfig]
|
||||
execution: Optional[Any] = None # Union[str, WorkflowStepExecutionConfig]
|
||||
routing: Optional[Any] = None # Union[List[str], WorkflowStepRoutingConfig]
|
||||
|
||||
# Internal resolved fields (private)
|
||||
_context_from, _retain_full_context, _output_variable, _output_file, _output_json, _output_pydantic
|
||||
_async_execution, _quality_check, _rerun, _max_retries, _on_error, _next_steps, _branch_condition
|
||||
```
|
||||
|
||||
## 7. Migration Status
|
||||
|
||||
| Category | Total | Updated | Remaining |
|
||||
|----------|-------|---------|-----------|
|
||||
| Definition Files | 2 | 2 | 0 |
|
||||
| Callsites | 10 | 10 | 0 |
|
||||
| Examples | 67 | 67 | 0 |
|
||||
| Docs Pages | 24+ | 24+ | 0 |
|
||||
| CLI | 1 | 1 | 0 |
|
||||
| Tests | 2 | 2 | 0 |
|
||||
|
||||
## 8. Verification Commands
|
||||
|
||||
```bash
|
||||
# Check for legacy params in Workflow/WorkflowStep definitions
|
||||
grep -rn "verbose=\|stream=\|planning_llm=\|memory_config=\|on_workflow_start=\|on_step_" \
|
||||
/Users/praison/praisonai-package/src/praisonai-agents/praisonaiagents/workflows/workflows.py
|
||||
|
||||
# Check examples for legacy API usage
|
||||
grep -rn "Workflow(" /Users/praison/praisonai-package/examples/ | grep -E "verbose=|stream=|planning_llm="
|
||||
|
||||
# Run workflow tests
|
||||
cd /Users/praison/praisonai-package/src/praisonai-agents && python -m pytest tests/unit/workflows/ -v
|
||||
```
|
||||
|
||||
## 9. Action Log
|
||||
|
||||
| Date | Action | Status |
|
||||
|------|--------|--------|
|
||||
| 2026-01-09 | Created workflow_configs.py with consolidated config classes | ✅ Complete |
|
||||
| 2026-01-09 | Updated Workflow dataclass with consolidated params | ✅ Complete |
|
||||
| 2026-01-09 | Updated WorkflowStep dataclass with consolidated params | ✅ Complete |
|
||||
| 2026-01-09 | Updated workflows/__init__.py exports | ✅ Complete |
|
||||
| 2026-01-09 | Added test_workflow_consolidated_api.py (39 tests) | ✅ Complete |
|
||||
| 2026-01-09 | Updated test_workflow_patterns.py to use new API | ✅ Complete |
|
||||
| 2026-01-09 | Updated workflow examples (fixed imports) | ✅ Complete |
|
||||
| 2026-01-09 | Updated workflow_branching.py to use routing= param | ✅ Complete |
|
||||
| 2026-01-09 | Updated workflows.mdx docs with consolidated API | ✅ Complete |
|
||||
| 2026-01-09 | Updated SDK reference docs | ✅ Complete |
|
||||
| 2026-01-09 | Updated WorkflowManager._load_workflow to use consolidated API | ✅ Complete |
|
||||
| 2026-01-09 | Final verification: missing=0 for legacy dataclass fields | ✅ Complete |
|
||||
|
||||
## 10. Final Verification Results
|
||||
|
||||
```
|
||||
=== Workflow Dataclass Fields ===
|
||||
Total fields: 23
|
||||
Legacy params found: 0
|
||||
✅ No legacy params in dataclass fields
|
||||
|
||||
=== WorkflowStep Dataclass Fields ===
|
||||
Total fields: 32
|
||||
Legacy params found: 0
|
||||
✅ No legacy params in dataclass fields
|
||||
|
||||
=== Consolidated Params Present ===
|
||||
Workflow consolidated params: ['output', 'planning', 'memory', 'hooks', 'context']
|
||||
WorkflowStep consolidated params: ['context', 'output', 'execution', 'routing']
|
||||
|
||||
✅ VERIFICATION PASSED: missing=0 for legacy dataclass fields
|
||||
```
|
||||
|
||||
## 11. Test Results
|
||||
|
||||
- **39 consolidated API tests**: All passed
|
||||
- **86 workflow pattern tests**: All passed
|
||||
- **6 markdown parser tests**: All passed (FIXED)
|
||||
|
||||
**Total: 125 tests passed**
|
||||
|
||||
## 12. Final Verification (2026-01-09)
|
||||
|
||||
```
|
||||
=== WORKFLOW API VERIFICATION ===
|
||||
✅ Workflow consolidated params: verbose=True, planning=True
|
||||
✅ WorkflowStep consolidated params: max_retries=1, context_from=['step1']
|
||||
✅ WorkflowManager._parse_steps uses consolidated API
|
||||
✅ WorkflowManager._load_workflow uses consolidated API
|
||||
✅ All 199 workflow tests pass
|
||||
|
||||
=== TEST BREAKDOWN ===
|
||||
- test_workflow_patterns.py: 125 tests passed
|
||||
- test_workflow_planning.py: 16 tests passed
|
||||
- test_workflow_context_passing.py: 13 tests passed
|
||||
- test_workflow_markdown_parser.py: 10 tests passed
|
||||
- test_workflow_async.py: 6 tests passed
|
||||
- test_workflow_agents.py: 10 tests passed
|
||||
- test_workflow_tools.py: 10 tests passed
|
||||
- test_workflow_consolidated_api.py: 39 tests passed
|
||||
```
|
||||
@@ -16,7 +16,7 @@ RUN mkdir -p /root/.praison
|
||||
# Install Python packages (using latest versions)
|
||||
RUN pip install --no-cache-dir \
|
||||
praisonai_tools \
|
||||
"praisonai>=3.5.7" \
|
||||
"praisonai>=3.6.2" \
|
||||
"praisonai[chat]" \
|
||||
"embedchain[github,youtube]"
|
||||
|
||||
|
||||
@@ -20,7 +20,7 @@ RUN mkdir -p /root/.praison
|
||||
# Install Python packages (using latest versions)
|
||||
RUN pip install --no-cache-dir \
|
||||
praisonai_tools \
|
||||
"praisonai>=3.5.7" \
|
||||
"praisonai>=3.6.2" \
|
||||
"praisonai[ui]" \
|
||||
"praisonai[chat]" \
|
||||
"praisonai[realtime]" \
|
||||
|
||||
@@ -16,7 +16,7 @@ RUN mkdir -p /root/.praison
|
||||
# Install Python packages (using latest versions)
|
||||
RUN pip install --no-cache-dir \
|
||||
praisonai_tools \
|
||||
"praisonai>=3.5.7" \
|
||||
"praisonai>=3.6.2" \
|
||||
"praisonai[ui]" \
|
||||
"praisonai[crewai]"
|
||||
|
||||
|
||||
@@ -23,6 +23,7 @@ examples/
|
||||
|
||||
| Category | Examples | Docs |
|
||||
|----------|----------|------|
|
||||
| **Consolidated Params** | [consolidated_params/](consolidated_params/) | [📖](https://docs.praison.ai/api/consolidated-params) |
|
||||
| **Agents** | [python/agents/](python/agents/) | [📖](https://docs.praison.ai/concepts/agents) |
|
||||
| **Workflows** | [python/workflows/](python/workflows/) | [📖](https://docs.praison.ai/features/workflows) |
|
||||
| **Model Router** | [python/agents/router-agent-cost-optimization.py](python/agents/router-agent-cost-optimization.py) | [📖](https://docs.praison.ai/features/model-router) |
|
||||
@@ -35,6 +36,21 @@ examples/
|
||||
| **Serve** | [serve/](serve/) | [📖](https://docs.praison.ai/cli/serve) |
|
||||
| **Endpoints** | [serve/](serve/) | [📖](https://docs.praison.ai/cli/endpoints) |
|
||||
|
||||
## Consolidated Params Examples
|
||||
|
||||
Agent-centric API with unified parameter resolution. Precedence: Instance > Config > Array > Dict > String > Bool > Default
|
||||
|
||||
| Example | Description |
|
||||
|---------|-------------|
|
||||
| [basic_agent.py](consolidated_params/basic_agent.py) | Minimal agent with memory |
|
||||
| [basic_agents.py](consolidated_params/basic_agents.py) | Multi-agent with memory+planning |
|
||||
| [basic_workflow.py](consolidated_params/basic_workflow.py) | Workflow with consolidated params |
|
||||
| [basic_memory.py](consolidated_params/basic_memory.py) | Memory presets (file, redis, postgres) |
|
||||
| [basic_guardrails.py](consolidated_params/basic_guardrails.py) | Guardrails with callable or config |
|
||||
| [basic_workflow_agentlike.py](consolidated_params/basic_workflow_agentlike.py) | Workflow with agent-like params |
|
||||
| [basic_step_override.py](consolidated_params/basic_step_override.py) | Step-level override of workflow defaults |
|
||||
| [advanced_workflow_full_features.py](consolidated_params/advanced_workflow_full_features.py) | All consolidated params |
|
||||
|
||||
## Serve Examples
|
||||
|
||||
| Example | Description | CLI Command |
|
||||
|
||||
@@ -0,0 +1,17 @@
|
||||
"""
|
||||
Advanced Output + Execution Example - Agent-Centric API
|
||||
|
||||
Demonstrates preset + override array syntax.
|
||||
"""
|
||||
|
||||
from praisonaiagents import Agent
|
||||
|
||||
# Advanced: Preset with overrides using array syntax
|
||||
agent = Agent(
|
||||
instructions="You are a helpful assistant.",
|
||||
output=["verbose", {"stream": True, "metrics": True}],
|
||||
execution=["thorough", {"max_iter": 30}],
|
||||
)
|
||||
|
||||
response = agent.start("Explain quantum computing in simple terms.")
|
||||
print(response)
|
||||
@@ -0,0 +1,55 @@
|
||||
"""Advanced Workflow with all consolidated params configured."""
|
||||
from praisonaiagents import Agent
|
||||
from praisonaiagents.workflows import Workflow, WorkflowStep
|
||||
from praisonaiagents.workflows.workflow_configs import (
|
||||
WorkflowOutputConfig, WorkflowPlanningConfig, WorkflowMemoryConfig,
|
||||
)
|
||||
|
||||
# Full-featured workflow with all consolidated params
|
||||
workflow = Workflow(
|
||||
name="FullFeaturedWorkflow",
|
||||
description="Demonstrates all agent-like consolidated params",
|
||||
steps=[
|
||||
WorkflowStep(
|
||||
name="researcher",
|
||||
action="Research: {{input}}",
|
||||
agent=Agent(instructions="You are a researcher."),
|
||||
knowledge=["research_docs/"], # Step-specific knowledge
|
||||
web="duckduckgo", # Use DuckDuckGo for this step
|
||||
),
|
||||
WorkflowStep(
|
||||
name="analyzer",
|
||||
action="Analyze the research",
|
||||
agent=Agent(instructions="You are an analyst."),
|
||||
reflection="thorough", # Thorough reflection for analysis
|
||||
guardrails="strict", # Strict validation
|
||||
),
|
||||
WorkflowStep(
|
||||
name="writer",
|
||||
action="Write the final report",
|
||||
agent=Agent(instructions="You are a writer."),
|
||||
autonomy=True, # Enable autonomy for writing
|
||||
),
|
||||
],
|
||||
# Workflow-level consolidated params
|
||||
output=WorkflowOutputConfig(verbose=True, stream=True),
|
||||
planning=WorkflowPlanningConfig(enabled=True, reasoning=True),
|
||||
memory=WorkflowMemoryConfig(backend="file"),
|
||||
# Agent-like params
|
||||
autonomy=True,
|
||||
knowledge=["shared_docs/"],
|
||||
guardrails=True,
|
||||
web=True,
|
||||
reflection=True,
|
||||
)
|
||||
|
||||
if __name__ == "__main__":
|
||||
print(f"Workflow: {workflow.name}")
|
||||
print(f"Output verbose: {workflow._verbose}")
|
||||
print(f"Planning enabled: {workflow._planning_enabled}")
|
||||
print(f"Memory config: {workflow._memory_config}")
|
||||
print(f"Autonomy: {workflow.autonomy}")
|
||||
print(f"Knowledge: {workflow.knowledge}")
|
||||
print(f"Guardrails: {workflow.guardrails}")
|
||||
print(f"Web: {workflow.web}")
|
||||
print(f"Reflection: {workflow.reflection}")
|
||||
@@ -0,0 +1,12 @@
|
||||
"""Basic Agent example with consolidated params."""
|
||||
from praisonaiagents import Agent
|
||||
|
||||
# Minimal agent with memory
|
||||
agent = Agent(
|
||||
instructions="You are a helpful assistant.",
|
||||
memory=True,
|
||||
)
|
||||
|
||||
if __name__ == "__main__":
|
||||
response = agent.chat("Hello!")
|
||||
print(response)
|
||||
@@ -0,0 +1,17 @@
|
||||
"""Basic Agents (multi-agent) example with consolidated params."""
|
||||
from praisonaiagents import Agent, Agents
|
||||
|
||||
# Create agents
|
||||
writer = Agent(instructions="You write content.")
|
||||
editor = Agent(instructions="You edit content.")
|
||||
|
||||
# Multi-agent with consolidated params
|
||||
agents = Agents(
|
||||
agents=[writer, editor],
|
||||
memory=True,
|
||||
planning=True,
|
||||
)
|
||||
|
||||
if __name__ == "__main__":
|
||||
result = agents.start("Write a haiku about AI")
|
||||
print(result)
|
||||
@@ -0,0 +1,16 @@
|
||||
"""
|
||||
Basic Execution Example - Agent-Centric API
|
||||
|
||||
Demonstrates execution preset usage.
|
||||
"""
|
||||
|
||||
from praisonaiagents import Agent
|
||||
|
||||
# Basic: Use fast execution preset
|
||||
agent = Agent(
|
||||
instructions="You are a helpful assistant.",
|
||||
execution="fast", # Presets: fast, balanced, thorough, unlimited
|
||||
)
|
||||
|
||||
response = agent.start("What is the capital of France?")
|
||||
print(response)
|
||||
@@ -0,0 +1,26 @@
|
||||
"""
|
||||
Basic Guardrails Example - Agent-Centric API
|
||||
|
||||
Demonstrates guardrails with callable validator.
|
||||
"""
|
||||
|
||||
from praisonaiagents import Agent
|
||||
|
||||
# Define a simple guardrail validator
|
||||
def validate_response(output):
|
||||
"""Validate that response doesn't contain forbidden words."""
|
||||
forbidden = ["error", "fail", "cannot"]
|
||||
text = str(output).lower()
|
||||
for word in forbidden:
|
||||
if word in text:
|
||||
return (False, f"Response contains forbidden word: {word}")
|
||||
return (True, output)
|
||||
|
||||
# Basic: Use callable guardrail
|
||||
agent = Agent(
|
||||
instructions="You are a helpful assistant.",
|
||||
guardrails=validate_response,
|
||||
)
|
||||
|
||||
response = agent.start("What is 2 + 2?")
|
||||
print(response)
|
||||
@@ -0,0 +1,21 @@
|
||||
"""
|
||||
Basic Memory Example - Agent-Centric API
|
||||
|
||||
Demonstrates the simplest memory usage with consolidated params.
|
||||
"""
|
||||
|
||||
from praisonaiagents import Agent
|
||||
|
||||
# Basic: Enable memory with preset
|
||||
agent = Agent(
|
||||
instructions="You are a helpful assistant with memory.",
|
||||
memory="sqlite", # Preset: sqlite, redis, postgres, file
|
||||
)
|
||||
|
||||
# Run a simple task
|
||||
response = agent.start("Remember that my favorite color is blue.")
|
||||
print(response)
|
||||
|
||||
# Follow-up to test memory
|
||||
response = agent.start("What is my favorite color?")
|
||||
print(response)
|
||||
@@ -0,0 +1,16 @@
|
||||
"""
|
||||
Basic Output Example - Agent-Centric API
|
||||
|
||||
Demonstrates output preset usage.
|
||||
"""
|
||||
|
||||
from praisonaiagents import Agent
|
||||
|
||||
# Basic: Use verbose output preset
|
||||
agent = Agent(
|
||||
instructions="You are a helpful assistant.",
|
||||
output="verbose", # Presets: minimal, normal, verbose, debug, silent
|
||||
)
|
||||
|
||||
response = agent.start("What is 2 + 2?")
|
||||
print(response)
|
||||
@@ -0,0 +1,33 @@
|
||||
"""Basic example showing step-level override of workflow defaults."""
|
||||
from praisonaiagents import Agent
|
||||
from praisonaiagents.workflows import Workflow, WorkflowStep
|
||||
|
||||
# Workflow with defaults
|
||||
workflow = Workflow(
|
||||
name="StepOverrideWorkflow",
|
||||
steps=[
|
||||
WorkflowStep(
|
||||
name="step1",
|
||||
action="Process with workflow defaults",
|
||||
agent=Agent(instructions="You are a processor."),
|
||||
# Uses workflow-level web=True (inherited)
|
||||
),
|
||||
WorkflowStep(
|
||||
name="step2",
|
||||
action="Process with step override",
|
||||
agent=Agent(instructions="You are a processor."),
|
||||
# Override workflow default
|
||||
web=False, # Disable web for this step
|
||||
reflection="thorough", # Use thorough reflection preset
|
||||
),
|
||||
],
|
||||
# Workflow defaults
|
||||
web=True,
|
||||
reflection=True,
|
||||
)
|
||||
|
||||
if __name__ == "__main__":
|
||||
print(f"Workflow web default: {workflow.web}")
|
||||
print(f"Step1 web: {workflow.steps[0].web}") # None (uses workflow default)
|
||||
print(f"Step2 web: {workflow.steps[1].web}") # False (overridden)
|
||||
print(f"Step2 reflection: {workflow.steps[1].reflection}") # thorough
|
||||
@@ -0,0 +1,26 @@
|
||||
"""
|
||||
Basic Workflow Example - Agent-Centric API
|
||||
|
||||
Demonstrates minimal workflow with consolidated params.
|
||||
"""
|
||||
|
||||
from praisonaiagents import Agent
|
||||
from praisonaiagents.workflows import Workflow, WorkflowStep
|
||||
|
||||
# Create a simple workflow
|
||||
workflow = Workflow(
|
||||
name="simple_workflow",
|
||||
output="verbose", # Workflow-level output preset
|
||||
)
|
||||
|
||||
# Add a step
|
||||
step = WorkflowStep(
|
||||
name="writer",
|
||||
action="Write a haiku about programming",
|
||||
agent=Agent(instructions="You are a creative poet."),
|
||||
output="result.txt", # Save output to file
|
||||
)
|
||||
|
||||
# Note: This is a structural example - actual execution requires workflow.run()
|
||||
print("Workflow created with output preset:", workflow.output)
|
||||
print("Step output file:", step.output_file)
|
||||
@@ -0,0 +1,32 @@
|
||||
"""Basic Workflow with Agent-like consolidated params."""
|
||||
from praisonaiagents import Agent
|
||||
from praisonaiagents.workflows import Workflow, WorkflowStep
|
||||
|
||||
# Workflow with agent-like params (knowledge, web, guardrails, reflection)
|
||||
workflow = Workflow(
|
||||
name="AgentLikeWorkflow",
|
||||
steps=[
|
||||
WorkflowStep(
|
||||
name="researcher",
|
||||
action="Research the topic: {{input}}",
|
||||
agent=Agent(instructions="You are a researcher."),
|
||||
),
|
||||
WorkflowStep(
|
||||
name="writer",
|
||||
action="Write about the research findings",
|
||||
agent=Agent(instructions="You are a writer."),
|
||||
),
|
||||
],
|
||||
# Agent-like consolidated params
|
||||
knowledge=["docs/"], # Enable RAG
|
||||
web=True, # Enable web search
|
||||
guardrails=True, # Enable guardrails
|
||||
reflection=True, # Enable self-reflection
|
||||
)
|
||||
|
||||
if __name__ == "__main__":
|
||||
print(f"Workflow: {workflow.name}")
|
||||
print(f"Knowledge: {workflow.knowledge}")
|
||||
print(f"Web: {workflow.web}")
|
||||
print(f"Guardrails: {workflow.guardrails}")
|
||||
print(f"Reflection: {workflow.reflection}")
|
||||
@@ -0,0 +1,22 @@
|
||||
"""Sample application."""
|
||||
|
||||
def greet(name: str) -> str:
|
||||
"""Greet someone."""
|
||||
return f"Hello, {name}!"
|
||||
|
||||
def add(a: int, b: int) -> int:
|
||||
"""Add two numbers."""
|
||||
return a + b
|
||||
|
||||
class Calculator:
|
||||
"""Simple calculator."""
|
||||
|
||||
def __init__(self):
|
||||
self.value = 0
|
||||
|
||||
def add(self, x):
|
||||
self.value += x
|
||||
return self.value
|
||||
|
||||
if __name__ == "__main__":
|
||||
print(greet("World"))
|
||||
@@ -0,0 +1,10 @@
|
||||
"""
|
||||
demo_output.py - Auto-generated file
|
||||
"""
|
||||
|
||||
def main():
|
||||
pass
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,10 @@
|
||||
"""
|
||||
test_demo.py - Auto-generated file
|
||||
"""
|
||||
|
||||
def main():
|
||||
pass
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,117 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Dict Config Example - Demonstrates dict shorthand for consolidated parameters.
|
||||
|
||||
This example shows how to use dict shorthand instead of importing config classes.
|
||||
Dict input is strictly validated - unknown keys raise helpful errors.
|
||||
|
||||
Requirements:
|
||||
- Set OPENAI_API_KEY environment variable
|
||||
"""
|
||||
|
||||
import os
|
||||
|
||||
|
||||
def example_dict_output_config():
|
||||
"""Example: Using dict for output configuration."""
|
||||
from praisonaiagents import Agent
|
||||
|
||||
# Dict shorthand - equivalent to OutputConfig(verbose=True, stream=False)
|
||||
agent = Agent(
|
||||
name="Assistant",
|
||||
instructions="You are a helpful assistant.",
|
||||
output={"verbose": True, "stream": False},
|
||||
)
|
||||
|
||||
result = agent.start("Say hello in one word.")
|
||||
print(f"Result: {result}")
|
||||
return agent
|
||||
|
||||
|
||||
def example_dict_execution_config():
|
||||
"""Example: Using dict for execution configuration."""
|
||||
from praisonaiagents import Agent
|
||||
|
||||
# Dict shorthand for execution settings
|
||||
agent = Agent(
|
||||
name="FastAgent",
|
||||
instructions="You are a fast responder.",
|
||||
execution={"max_iter": 5, "timeout": 30},
|
||||
)
|
||||
|
||||
result = agent.start("What is 2+2?")
|
||||
print(f"Result: {result}")
|
||||
return agent
|
||||
|
||||
|
||||
def example_dict_validation_error():
|
||||
"""Example: Dict with unknown key raises helpful error."""
|
||||
from praisonaiagents import Agent
|
||||
|
||||
print("\n--- Testing dict validation error ---")
|
||||
try:
|
||||
# This will raise TypeError with helpful message
|
||||
Agent(
|
||||
name="Test",
|
||||
instructions="Test agent",
|
||||
output={"verbose": True, "invalid_key": "value"},
|
||||
)
|
||||
except TypeError as e:
|
||||
print(f"Expected error caught: {e}")
|
||||
# Shows: Unknown keys for output: ['invalid_key']. Valid keys: ...
|
||||
|
||||
|
||||
def example_array_preset_override():
|
||||
"""Example: Array [preset, {overrides}] pattern still works."""
|
||||
from praisonaiagents import Agent
|
||||
|
||||
# Array pattern: preset name + override dict
|
||||
agent = Agent(
|
||||
name="VerboseAgent",
|
||||
instructions="You are verbose.",
|
||||
output=["verbose", {"stream": False}],
|
||||
)
|
||||
|
||||
result = agent.start("Say hi.")
|
||||
print(f"Result: {result}")
|
||||
return agent
|
||||
|
||||
|
||||
def example_base_url_api_key_separate():
|
||||
"""Example: base_url and api_key remain separate parameters."""
|
||||
from praisonaiagents import Agent
|
||||
|
||||
# base_url and api_key are NOT consolidated - they stay separate
|
||||
# This is intentional for security and clarity
|
||||
agent = Agent(
|
||||
name="CustomEndpoint",
|
||||
instructions="You use a custom endpoint.",
|
||||
llm="gpt-4o-mini",
|
||||
# base_url="http://localhost:11434/v1", # Uncomment for Ollama
|
||||
# api_key=os.getenv("CUSTOM_API_KEY"), # Separate parameter
|
||||
)
|
||||
|
||||
return agent
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
# Check for API key
|
||||
if not os.getenv("OPENAI_API_KEY"):
|
||||
print("Warning: OPENAI_API_KEY not set. Some examples may fail.")
|
||||
|
||||
print("=== Dict Config Examples ===\n")
|
||||
|
||||
# Run validation error example (doesn't need API key)
|
||||
example_dict_validation_error()
|
||||
|
||||
# Run examples that need API key
|
||||
if os.getenv("OPENAI_API_KEY"):
|
||||
print("\n--- Dict output config ---")
|
||||
example_dict_output_config()
|
||||
|
||||
print("\n--- Array preset override ---")
|
||||
example_array_preset_override()
|
||||
else:
|
||||
print("\nSkipping API examples (no OPENAI_API_KEY)")
|
||||
|
||||
print("\n=== Examples Complete ===")
|
||||
@@ -0,0 +1,14 @@
|
||||
#!/usr/bin/env python3
|
||||
|
||||
from praisonaiagents import Agent
|
||||
|
||||
print("Creating agent...")
|
||||
agent = Agent(
|
||||
instructions="You are a helpful assistant"
|
||||
)
|
||||
|
||||
print("Starting agent...")
|
||||
result = agent.start("Why is the sky blue?")
|
||||
|
||||
print("Agent completed successfully!")
|
||||
print(f"Result: {result}")
|
||||
@@ -0,0 +1,23 @@
|
||||
"""Advanced memory example - with config and overrides."""
|
||||
from praisonaiagents import Agent, MemoryConfig
|
||||
|
||||
# With full config
|
||||
agent = Agent(
|
||||
instructions="You are a helpful assistant with memory.",
|
||||
memory=MemoryConfig(
|
||||
backend="redis",
|
||||
user_id="user123",
|
||||
session_id="session456",
|
||||
auto_memory=True,
|
||||
),
|
||||
)
|
||||
|
||||
# With array [preset, overrides]
|
||||
agent_override = Agent(
|
||||
instructions="You are a helpful assistant.",
|
||||
memory=["redis", {"user_id": "custom_user"}],
|
||||
)
|
||||
|
||||
if __name__ == "__main__":
|
||||
response = agent.chat("Remember that my favorite color is blue.")
|
||||
print(response)
|
||||
@@ -0,0 +1,24 @@
|
||||
"""Basic memory example - minimal usage."""
|
||||
from praisonaiagents import Agent
|
||||
|
||||
# Simple enable (uses default file-based memory)
|
||||
agent = Agent(
|
||||
instructions="You are a helpful assistant with memory.",
|
||||
memory=True,
|
||||
)
|
||||
|
||||
# With preset
|
||||
agent_redis = Agent(
|
||||
instructions="You are a helpful assistant.",
|
||||
memory="redis", # Uses redis preset
|
||||
)
|
||||
|
||||
# With URL
|
||||
agent_postgres = Agent(
|
||||
instructions="You are a helpful assistant.",
|
||||
memory="postgresql://localhost/mydb", # URL auto-detected
|
||||
)
|
||||
|
||||
if __name__ == "__main__":
|
||||
response = agent.chat("Remember that my favorite color is blue.")
|
||||
print(response)
|
||||
@@ -0,0 +1,13 @@
|
||||
"""Basic output and execution example."""
|
||||
from praisonaiagents import Agent
|
||||
|
||||
# Simple preset usage
|
||||
agent = Agent(
|
||||
instructions="You are a helpful assistant.",
|
||||
output="verbose", # Preset: verbose output
|
||||
execution="thorough", # Preset: thorough execution
|
||||
)
|
||||
|
||||
if __name__ == "__main__":
|
||||
response = agent.chat("What is 2+2?")
|
||||
print(response)
|
||||
@@ -0,0 +1,22 @@
|
||||
"""Basic workflow example with consolidated params."""
|
||||
from praisonaiagents import Agent
|
||||
from praisonaiagents.workflows import Workflow, WorkflowStep
|
||||
|
||||
# Create agents
|
||||
writer = Agent(instructions="You are a content writer.")
|
||||
editor = Agent(instructions="You are an editor.")
|
||||
|
||||
# Create workflow with consolidated params
|
||||
workflow = Workflow(
|
||||
name="Content Pipeline",
|
||||
steps=[
|
||||
WorkflowStep(name="write", agent=writer, action="Write about {{topic}}"),
|
||||
WorkflowStep(name="edit", agent=editor, action="Edit the content", context=["write"]),
|
||||
],
|
||||
output="verbose",
|
||||
planning=True,
|
||||
)
|
||||
|
||||
if __name__ == "__main__":
|
||||
result = workflow.run(variables={"topic": "AI agents"})
|
||||
print(result)
|
||||
@@ -177,6 +177,13 @@ def __getattr__(name):
|
||||
_lazy_cache["PraisonAIAgents"] = Agents # Also cache alias
|
||||
return Agents
|
||||
elif name == "PraisonAIAgents":
|
||||
import warnings
|
||||
warnings.warn(
|
||||
"PraisonAIAgents is deprecated, use Agents instead. "
|
||||
"Example: from praisonaiagents import Agents",
|
||||
DeprecationWarning,
|
||||
stacklevel=2
|
||||
)
|
||||
from .agents.agents import Agents
|
||||
_lazy_cache["Agents"] = Agents
|
||||
_lazy_cache[name] = Agents
|
||||
|
||||
@@ -0,0 +1,49 @@
|
||||
"""
|
||||
Resolver helper functions for consolidated parameters.
|
||||
|
||||
These helpers wrap the canonical resolver for specific parameter types,
|
||||
handling special cases like callable validators for guardrails.
|
||||
"""
|
||||
|
||||
from typing import Any, Type
|
||||
|
||||
from .config.param_resolver import resolve, ArrayMode
|
||||
from .config.presets import GUARDRAIL_PRESETS
|
||||
|
||||
|
||||
def resolve_guardrails(value: Any, config_class: Type) -> Any:
|
||||
"""
|
||||
Resolve guardrails parameter.
|
||||
|
||||
Handles special case of callable validators which have highest precedence.
|
||||
|
||||
Args:
|
||||
value: The guardrails parameter value
|
||||
config_class: GuardrailConfig class
|
||||
|
||||
Returns:
|
||||
Resolved config, callable, or string
|
||||
"""
|
||||
# Handle None
|
||||
if value is None:
|
||||
return None
|
||||
|
||||
# Handle False (disabled)
|
||||
if value is False:
|
||||
return None
|
||||
|
||||
# Handle callable (highest precedence after None check)
|
||||
# This includes functions, lambdas, and callable objects
|
||||
if callable(value) and not isinstance(value, type):
|
||||
return value
|
||||
|
||||
# Use canonical resolver for everything else
|
||||
return resolve(
|
||||
value=value,
|
||||
param_name="guardrails",
|
||||
config_class=config_class,
|
||||
presets=GUARDRAIL_PRESETS,
|
||||
array_mode=ArrayMode.PRESET_OVERRIDE,
|
||||
string_mode="llm_prompt",
|
||||
default=None,
|
||||
)
|
||||
@@ -522,34 +522,45 @@ class Agent:
|
||||
cache, prompt_caching = True, None
|
||||
|
||||
# ─────────────────────────────────────────────────────────────────────
|
||||
# Resolve HOOKS param
|
||||
# Supports: None, list of callables, HooksConfig
|
||||
# Resolve HOOKS param using unified resolver
|
||||
# Supports: None, list of callables, HooksConfig, dict
|
||||
# ─────────────────────────────────────────────────────────────────────
|
||||
_hooks_list = []
|
||||
step_callback = None
|
||||
if hooks is not None:
|
||||
if isinstance(hooks, list):
|
||||
_hooks_list = hooks
|
||||
elif isinstance(hooks, HooksConfig):
|
||||
step_callback = hooks.on_step
|
||||
_hooks_list = hooks.middleware or []
|
||||
elif isinstance(hooks, dict):
|
||||
step_callback = hooks.get('on_step')
|
||||
_hooks_list = hooks.get('middleware', [])
|
||||
_hooks_config = resolve(
|
||||
value=hooks,
|
||||
param_name="hooks",
|
||||
config_class=HooksConfig,
|
||||
array_mode=ArrayMode.PASSTHROUGH,
|
||||
default=None,
|
||||
)
|
||||
if _hooks_config is not None:
|
||||
if isinstance(_hooks_config, list):
|
||||
_hooks_list = _hooks_config
|
||||
elif isinstance(_hooks_config, HooksConfig):
|
||||
step_callback = _hooks_config.on_step
|
||||
_hooks_list = _hooks_config.middleware or []
|
||||
|
||||
# ─────────────────────────────────────────────────────────────────────
|
||||
# Resolve SKILLS param
|
||||
# Supports: None, list of paths, SkillsConfig
|
||||
# Resolve SKILLS param using unified resolver
|
||||
# Supports: None, list of paths, SkillsConfig, str
|
||||
# ─────────────────────────────────────────────────────────────────────
|
||||
_skills = None
|
||||
if skills is not None:
|
||||
if isinstance(skills, list):
|
||||
_skills = skills
|
||||
elif isinstance(skills, SkillsConfig):
|
||||
_skills = skills.paths
|
||||
skills_dirs = skills.dirs
|
||||
elif isinstance(skills, str):
|
||||
_skills = [skills]
|
||||
skills_dirs = None
|
||||
_skills_config = resolve(
|
||||
value=skills,
|
||||
param_name="skills",
|
||||
config_class=SkillsConfig,
|
||||
array_mode=ArrayMode.SOURCES,
|
||||
string_mode="path_as_source",
|
||||
default=None,
|
||||
)
|
||||
if _skills_config is not None:
|
||||
if isinstance(_skills_config, SkillsConfig):
|
||||
_skills = _skills_config.paths
|
||||
skills_dirs = _skills_config.dirs
|
||||
elif isinstance(_skills_config, list):
|
||||
_skills = _skills_config
|
||||
|
||||
# ─────────────────────────────────────────────────────────────────────
|
||||
# Resolve MEMORY param using unified resolver
|
||||
@@ -595,36 +606,42 @@ class Agent:
|
||||
memory = None
|
||||
|
||||
# ─────────────────────────────────────────────────────────────────────
|
||||
# Resolve KNOWLEDGE param
|
||||
# Resolve KNOWLEDGE param using unified resolver
|
||||
# Supports: None, bool, str path, list of sources, KnowledgeConfig, Instance
|
||||
# ─────────────────────────────────────────────────────────────────────
|
||||
retrieval_config = None
|
||||
embedder_config = None
|
||||
|
||||
if knowledge is not None and knowledge is not False:
|
||||
if isinstance(knowledge, list):
|
||||
# List of sources - use directly
|
||||
pass
|
||||
elif hasattr(knowledge, 'search') and hasattr(knowledge, 'add'):
|
||||
_knowledge_config = resolve(
|
||||
value=knowledge,
|
||||
param_name="knowledge",
|
||||
config_class=KnowledgeConfig,
|
||||
instance_check=lambda v: hasattr(v, 'search') and hasattr(v, 'add'),
|
||||
array_mode=ArrayMode.SOURCES_WITH_CONFIG,
|
||||
string_mode="path_as_source",
|
||||
default=None,
|
||||
)
|
||||
|
||||
if _knowledge_config is not None:
|
||||
if hasattr(_knowledge_config, 'search') and hasattr(_knowledge_config, 'add'):
|
||||
# Knowledge instance - pass through
|
||||
pass
|
||||
elif isinstance(knowledge, KnowledgeConfig):
|
||||
embedder_config = knowledge.embedder_config
|
||||
if knowledge.config:
|
||||
retrieval_config = knowledge.config
|
||||
knowledge = _knowledge_config
|
||||
elif isinstance(_knowledge_config, KnowledgeConfig):
|
||||
embedder_config = _knowledge_config.embedder_config
|
||||
if _knowledge_config.config:
|
||||
retrieval_config = _knowledge_config.config
|
||||
else:
|
||||
retrieval_config = {
|
||||
'top_k': knowledge.retrieval_k,
|
||||
'threshold': knowledge.retrieval_threshold,
|
||||
'rerank': knowledge.rerank,
|
||||
'rerank_model': knowledge.rerank_model,
|
||||
'top_k': _knowledge_config.retrieval_k,
|
||||
'threshold': _knowledge_config.retrieval_threshold,
|
||||
'rerank': _knowledge_config.rerank,
|
||||
'rerank_model': _knowledge_config.rerank_model,
|
||||
}
|
||||
knowledge = knowledge.sources if knowledge.sources else None
|
||||
elif isinstance(knowledge, str):
|
||||
# Single path - convert to list
|
||||
knowledge = [knowledge]
|
||||
elif knowledge is True:
|
||||
knowledge = None
|
||||
knowledge = _knowledge_config.sources if _knowledge_config.sources else None
|
||||
elif isinstance(_knowledge_config, list):
|
||||
knowledge = _knowledge_config
|
||||
elif knowledge is False:
|
||||
knowledge = None
|
||||
|
||||
# ─────────────────────────────────────────────────────────────────────
|
||||
# Resolve PLANNING param
|
||||
@@ -681,22 +698,25 @@ class Agent:
|
||||
self_reflect = False
|
||||
|
||||
# ─────────────────────────────────────────────────────────────────────
|
||||
# Resolve GUARDRAILS param
|
||||
# Resolve GUARDRAILS param using unified resolver
|
||||
# Supports: None, bool, callable, str prompt, GuardrailConfig
|
||||
# ─────────────────────────────────────────────────────────────────────
|
||||
if guardrails is not None:
|
||||
if guardrails is False:
|
||||
guardrail = None
|
||||
elif callable(guardrails):
|
||||
guardrail = guardrails
|
||||
elif isinstance(guardrails, GuardrailConfig):
|
||||
guardrail = guardrails.validator or guardrails.llm_validator
|
||||
max_guardrail_retries = guardrails.max_retries
|
||||
elif isinstance(guardrails, str):
|
||||
# Long string = LLM validator prompt
|
||||
guardrail = guardrails
|
||||
elif guardrails is True:
|
||||
pass # No-op, user must provide validator
|
||||
from .._resolver_helpers import resolve_guardrails as _resolve_guardrails
|
||||
_guardrails_config = _resolve_guardrails(
|
||||
value=guardrails,
|
||||
config_class=GuardrailConfig,
|
||||
)
|
||||
|
||||
if _guardrails_config is not None:
|
||||
if callable(_guardrails_config) and not isinstance(_guardrails_config, type):
|
||||
# Callable validator function
|
||||
guardrail = _guardrails_config
|
||||
elif isinstance(_guardrails_config, GuardrailConfig):
|
||||
guardrail = _guardrails_config.validator or _guardrails_config.llm_validator
|
||||
max_guardrail_retries = _guardrails_config.max_retries
|
||||
elif isinstance(_guardrails_config, str):
|
||||
# LLM validator prompt
|
||||
guardrail = _guardrails_config
|
||||
|
||||
# ─────────────────────────────────────────────────────────────────────
|
||||
# Resolve WEB param using unified resolver
|
||||
|
||||
@@ -142,6 +142,12 @@ class Agents:
|
||||
output: Optional[Any] = None, # Union[str, MultiAgentOutputConfig]
|
||||
execution: Optional[Any] = None, # Union[str, MultiAgentExecutionConfig]
|
||||
hooks: Optional[Any] = None, # MultiAgentHooksConfig
|
||||
# Additional consolidated params for feature parity with Agent
|
||||
autonomy: Optional[Any] = None, # Union[bool, AutonomyConfig] - agent autonomy
|
||||
knowledge: Optional[Any] = None, # Union[bool, List[str], KnowledgeConfig] - RAG
|
||||
guardrails: Optional[Any] = None, # Union[bool, Callable, GuardrailConfig] - validation
|
||||
web: Optional[Any] = None, # Union[bool, WebConfig] - web search/fetch
|
||||
reflection: Optional[Any] = None, # Union[bool, ReflectionConfig] - self-reflection
|
||||
):
|
||||
"""
|
||||
Initialize PraisonAIAgents with consolidated feature parameters.
|
||||
@@ -159,13 +165,25 @@ class Agents:
|
||||
output: Output configuration (str | MultiAgentOutputConfig)
|
||||
execution: Execution configuration (str | MultiAgentExecutionConfig)
|
||||
hooks: Hooks configuration (MultiAgentHooksConfig)
|
||||
autonomy: Autonomy configuration (bool | AutonomyConfig)
|
||||
knowledge: Knowledge/RAG configuration (bool | List[str] | KnowledgeConfig)
|
||||
guardrails: Guardrails configuration (bool | Callable | GuardrailConfig)
|
||||
web: Web search/fetch configuration (bool | WebConfig)
|
||||
reflection: Self-reflection configuration (bool | ReflectionConfig)
|
||||
"""
|
||||
# Store new params for propagation to agents
|
||||
self._autonomy = autonomy
|
||||
self._knowledge = knowledge
|
||||
self._guardrails = guardrails
|
||||
self._web = web
|
||||
self._reflection = reflection
|
||||
# ─────────────────────────────────────────────────────────────────────
|
||||
# Extract values from consolidated params using unified resolver
|
||||
# Precedence: Instance > Config > Array > String > Bool > Default
|
||||
# Extract values from consolidated params using UNIFIED CANONICAL resolver
|
||||
# Precedence: Instance > Config > Dict > Array > String > Bool > Default
|
||||
# ─────────────────────────────────────────────────────────────────────
|
||||
|
||||
# Import presets for unified resolution
|
||||
# Import canonical resolver and presets
|
||||
from ..config.param_resolver import resolve, ArrayMode
|
||||
from ..config.presets import (
|
||||
MULTI_AGENT_OUTPUT_PRESETS, MULTI_AGENT_EXECUTION_PRESETS,
|
||||
MEMORY_PRESETS, MEMORY_URL_SCHEMES,
|
||||
@@ -186,156 +204,118 @@ class Agents:
|
||||
MultiAgentHooksConfig = None
|
||||
|
||||
# ─────────────────────────────────────────────────────────────────────
|
||||
# Resolve OUTPUT param
|
||||
# Resolve OUTPUT param using canonical resolver
|
||||
# Supports: None, str preset, list [preset, overrides], Config, dict
|
||||
# ─────────────────────────────────────────────────────────────────────
|
||||
_verbose = 0
|
||||
_stream = True
|
||||
if output is None:
|
||||
pass # Use defaults
|
||||
elif isinstance(output, str):
|
||||
preset = MULTI_AGENT_OUTPUT_PRESETS.get(output, {})
|
||||
_verbose = preset.get("verbose", 0)
|
||||
_stream = preset.get("stream", True)
|
||||
elif isinstance(output, list) and len(output) >= 1:
|
||||
# Array mode: [preset] or [preset, overrides]
|
||||
preset_name = output[0] if isinstance(output[0], str) else None
|
||||
overrides = output[-1] if len(output) > 1 and isinstance(output[-1], dict) else {}
|
||||
if preset_name and preset_name in MULTI_AGENT_OUTPUT_PRESETS:
|
||||
preset = MULTI_AGENT_OUTPUT_PRESETS[preset_name].copy()
|
||||
preset.update(overrides)
|
||||
_verbose = preset.get("verbose", 0)
|
||||
_stream = preset.get("stream", True)
|
||||
elif MultiAgentOutputConfig and isinstance(output, MultiAgentOutputConfig):
|
||||
_verbose = output.verbose
|
||||
_stream = output.stream
|
||||
elif isinstance(output, dict):
|
||||
_verbose = output.get("verbose", 0)
|
||||
_stream = output.get("stream", True)
|
||||
_output_config = resolve(
|
||||
value=output,
|
||||
param_name="output",
|
||||
config_class=MultiAgentOutputConfig,
|
||||
presets=MULTI_AGENT_OUTPUT_PRESETS,
|
||||
array_mode=ArrayMode.PRESET_OVERRIDE,
|
||||
default=MultiAgentOutputConfig() if MultiAgentOutputConfig else None,
|
||||
)
|
||||
if _output_config and hasattr(_output_config, 'verbose'):
|
||||
_verbose = _output_config.verbose
|
||||
_stream = _output_config.stream
|
||||
else:
|
||||
_verbose = 0
|
||||
_stream = True
|
||||
|
||||
# ─────────────────────────────────────────────────────────────────────
|
||||
# Resolve EXECUTION param
|
||||
# Resolve EXECUTION param using canonical resolver
|
||||
# Supports: None, str preset, list [preset, overrides], Config, dict
|
||||
# ─────────────────────────────────────────────────────────────────────
|
||||
_max_iter = 10
|
||||
_max_retries = 5
|
||||
if execution is None:
|
||||
pass # Use defaults
|
||||
elif isinstance(execution, str):
|
||||
preset = MULTI_AGENT_EXECUTION_PRESETS.get(execution, {})
|
||||
_max_iter = preset.get("max_iter", 10)
|
||||
_max_retries = preset.get("max_retries", 5)
|
||||
elif isinstance(execution, list) and len(execution) >= 1:
|
||||
# Array mode: [preset] or [preset, overrides]
|
||||
preset_name = execution[0] if isinstance(execution[0], str) else None
|
||||
overrides = execution[-1] if len(execution) > 1 and isinstance(execution[-1], dict) else {}
|
||||
if preset_name and preset_name in MULTI_AGENT_EXECUTION_PRESETS:
|
||||
preset = MULTI_AGENT_EXECUTION_PRESETS[preset_name].copy()
|
||||
preset.update(overrides)
|
||||
_max_iter = preset.get("max_iter", 10)
|
||||
_max_retries = preset.get("max_retries", 5)
|
||||
elif MultiAgentExecutionConfig and isinstance(execution, MultiAgentExecutionConfig):
|
||||
_max_iter = execution.max_iter
|
||||
_max_retries = execution.max_retries
|
||||
elif isinstance(execution, dict):
|
||||
_max_iter = execution.get("max_iter", 10)
|
||||
_max_retries = execution.get("max_retries", 5)
|
||||
_exec_config = resolve(
|
||||
value=execution,
|
||||
param_name="execution",
|
||||
config_class=MultiAgentExecutionConfig,
|
||||
presets=MULTI_AGENT_EXECUTION_PRESETS,
|
||||
array_mode=ArrayMode.PRESET_OVERRIDE,
|
||||
default=MultiAgentExecutionConfig() if MultiAgentExecutionConfig else None,
|
||||
)
|
||||
if _exec_config and hasattr(_exec_config, 'max_iter'):
|
||||
_max_iter = _exec_config.max_iter
|
||||
_max_retries = _exec_config.max_retries
|
||||
else:
|
||||
_max_iter = 10
|
||||
_max_retries = 5
|
||||
|
||||
# ─────────────────────────────────────────────────────────────────────
|
||||
# Resolve HOOKS param
|
||||
# Resolve HOOKS param using canonical resolver
|
||||
# Supports: None, list, Config, dict
|
||||
# ─────────────────────────────────────────────────────────────────────
|
||||
_completion_checker = None
|
||||
_on_task_start = None
|
||||
_on_task_complete = None
|
||||
if hooks is not None:
|
||||
if MultiAgentHooksConfig and isinstance(hooks, MultiAgentHooksConfig):
|
||||
_completion_checker = hooks.completion_checker
|
||||
_on_task_start = hooks.on_task_start
|
||||
_on_task_complete = hooks.on_task_complete
|
||||
elif isinstance(hooks, dict):
|
||||
_completion_checker = hooks.get("completion_checker")
|
||||
_on_task_start = hooks.get("on_task_start")
|
||||
_on_task_complete = hooks.get("on_task_complete")
|
||||
_hooks_config = resolve(
|
||||
value=hooks,
|
||||
param_name="hooks",
|
||||
config_class=MultiAgentHooksConfig,
|
||||
array_mode=ArrayMode.PASSTHROUGH,
|
||||
default=None,
|
||||
)
|
||||
if _hooks_config and hasattr(_hooks_config, 'completion_checker'):
|
||||
_completion_checker = _hooks_config.completion_checker
|
||||
_on_task_start = _hooks_config.on_task_start
|
||||
_on_task_complete = _hooks_config.on_task_complete
|
||||
else:
|
||||
_completion_checker = None
|
||||
_on_task_start = None
|
||||
_on_task_complete = None
|
||||
|
||||
# ─────────────────────────────────────────────────────────────────────
|
||||
# Resolve MEMORY param
|
||||
# Resolve MEMORY param using canonical resolver
|
||||
# Supports: None, bool, str preset/URL, list, Config, dict, Instance
|
||||
# ─────────────────────────────────────────────────────────────────────
|
||||
_memory_config_resolved = resolve(
|
||||
value=memory,
|
||||
param_name="memory",
|
||||
config_class=MultiAgentMemoryConfig,
|
||||
presets=MEMORY_PRESETS,
|
||||
url_schemes=MEMORY_URL_SCHEMES,
|
||||
instance_check=lambda v: hasattr(v, 'database_url'),
|
||||
array_mode=ArrayMode.SINGLE_OR_LIST,
|
||||
default=None,
|
||||
)
|
||||
|
||||
# Extract values from resolved memory config
|
||||
_user_id = "praison"
|
||||
_memory_config = None
|
||||
_embedder = None
|
||||
|
||||
if memory is None or memory is False:
|
||||
pass # Disabled
|
||||
elif memory is True:
|
||||
pass # Use defaults
|
||||
elif isinstance(memory, str):
|
||||
# Check for URL
|
||||
if "://" in memory:
|
||||
# URL - extract backend from scheme
|
||||
scheme = memory.split("://")[0].lower()
|
||||
backend = MEMORY_URL_SCHEMES.get(scheme)
|
||||
if backend:
|
||||
_memory_config = {"url": memory, "backend": backend}
|
||||
elif memory in MEMORY_PRESETS:
|
||||
# Preset name
|
||||
_memory_config = MEMORY_PRESETS[memory]
|
||||
elif isinstance(memory, list) and len(memory) >= 1:
|
||||
# Array mode: [URL] or [preset, overrides]
|
||||
first = memory[0]
|
||||
if isinstance(first, str) and "://" in first:
|
||||
scheme = first.split("://")[0].lower()
|
||||
backend = MEMORY_URL_SCHEMES.get(scheme)
|
||||
if backend:
|
||||
_memory_config = {"url": first, "backend": backend}
|
||||
elif MultiAgentMemoryConfig and isinstance(memory, MultiAgentMemoryConfig):
|
||||
_user_id = memory.user_id or "praison"
|
||||
_memory_config = memory.config
|
||||
_embedder = memory.embedder
|
||||
elif isinstance(memory, dict):
|
||||
_user_id = memory.get("user_id", "praison")
|
||||
_memory_config = memory.get("config")
|
||||
_embedder = memory.get("embedder")
|
||||
elif hasattr(memory, 'database_url'):
|
||||
# db() instance
|
||||
_memory_config = {"db_instance": memory}
|
||||
if _memory_config_resolved is not None:
|
||||
if hasattr(_memory_config_resolved, 'database_url'):
|
||||
# db() instance - pass through
|
||||
_memory_config = {"db_instance": _memory_config_resolved}
|
||||
elif MultiAgentMemoryConfig and isinstance(_memory_config_resolved, MultiAgentMemoryConfig):
|
||||
_user_id = _memory_config_resolved.user_id or "praison"
|
||||
_memory_config = _memory_config_resolved.config
|
||||
_embedder = _memory_config_resolved.embedder
|
||||
elif isinstance(_memory_config_resolved, dict):
|
||||
# Dict from preset resolution
|
||||
_memory_config = _memory_config_resolved
|
||||
|
||||
# ─────────────────────────────────────────────────────────────────────
|
||||
# Resolve PLANNING param
|
||||
# Resolve PLANNING param using canonical resolver
|
||||
# Supports: None, bool, str LLM, list, Config, dict
|
||||
# ─────────────────────────────────────────────────────────────────────
|
||||
_planning_config = resolve(
|
||||
value=planning,
|
||||
param_name="planning",
|
||||
config_class=MultiAgentPlanningConfig,
|
||||
string_mode="llm_model",
|
||||
array_mode=ArrayMode.PRESET_OVERRIDE,
|
||||
default=None,
|
||||
)
|
||||
|
||||
# Extract values from resolved planning config
|
||||
_planning_llm = "gpt-4o-mini"
|
||||
_auto_approve_plan = False
|
||||
_planning_tools = None
|
||||
_planning_reasoning = False
|
||||
|
||||
if planning is None or planning is False:
|
||||
pass # Disabled
|
||||
elif planning is True:
|
||||
pass # Use defaults
|
||||
elif isinstance(planning, str):
|
||||
# String = LLM model name
|
||||
_planning_llm = planning
|
||||
elif isinstance(planning, list) and len(planning) >= 1:
|
||||
# Array mode: [llm] or [llm, overrides]
|
||||
if isinstance(planning[0], str):
|
||||
_planning_llm = planning[0]
|
||||
if len(planning) > 1 and isinstance(planning[-1], dict):
|
||||
overrides = planning[-1]
|
||||
_auto_approve_plan = overrides.get("auto_approve", False)
|
||||
_planning_tools = overrides.get("tools")
|
||||
_planning_reasoning = overrides.get("reasoning", False)
|
||||
elif MultiAgentPlanningConfig and isinstance(planning, MultiAgentPlanningConfig):
|
||||
_planning_llm = planning.llm or "gpt-4o-mini"
|
||||
_auto_approve_plan = planning.auto_approve
|
||||
_planning_tools = planning.tools
|
||||
_planning_reasoning = planning.reasoning
|
||||
elif isinstance(planning, dict):
|
||||
_planning_llm = planning.get("llm", "gpt-4o-mini")
|
||||
_auto_approve_plan = planning.get("auto_approve", False)
|
||||
_planning_tools = planning.get("tools")
|
||||
_planning_reasoning = planning.get("reasoning", False)
|
||||
if _planning_config is not None:
|
||||
if MultiAgentPlanningConfig and isinstance(_planning_config, MultiAgentPlanningConfig):
|
||||
_planning_llm = _planning_config.llm or "gpt-4o-mini"
|
||||
_auto_approve_plan = _planning_config.auto_approve
|
||||
_planning_tools = _planning_config.tools
|
||||
_planning_reasoning = _planning_config.reasoning
|
||||
|
||||
# ─────────────────────────────────────────────────────────────────────
|
||||
# Memory dependency check
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
"""
|
||||
Unified Parameter Resolver for Consolidated Parameters.
|
||||
|
||||
Implements the precedence rules: Instance > Config > Array > String > Bool > Default
|
||||
Implements the precedence rules: Instance > Config > Dict > Array > String > Bool > Default
|
||||
|
||||
This is the SINGLE, DRY resolver used by:
|
||||
- Agent
|
||||
@@ -12,7 +12,8 @@ This is the SINGLE, DRY resolver used by:
|
||||
Performance: O(1) happy path. Typo suggestions only on error path.
|
||||
"""
|
||||
|
||||
from typing import Any, Callable, Dict, Optional, Type, Union
|
||||
from typing import Any, Callable, Dict, Optional, Set, Type, Union
|
||||
import inspect
|
||||
|
||||
from .parse_utils import (
|
||||
detect_url_scheme,
|
||||
@@ -53,7 +54,7 @@ def resolve(
|
||||
) -> Any:
|
||||
"""
|
||||
Resolve a consolidated parameter following precedence rules:
|
||||
Instance > Config > Array > String > Bool > Default
|
||||
Instance > Config > Array > Dict > String > Bool > Default
|
||||
|
||||
Args:
|
||||
value: The parameter value
|
||||
@@ -91,19 +92,7 @@ def resolve(
|
||||
return value
|
||||
|
||||
# =========================================================================
|
||||
# 4. Dict -> convert to config
|
||||
# =========================================================================
|
||||
if isinstance(value, dict) and config_class:
|
||||
try:
|
||||
return config_class(**value)
|
||||
except TypeError as e:
|
||||
raise ValueError(
|
||||
f"Invalid {param_name} dict: {e}. "
|
||||
f"Valid fields: {_get_config_fields(config_class)}"
|
||||
)
|
||||
|
||||
# =========================================================================
|
||||
# 5. Array handling (before string to respect precedence)
|
||||
# 4. Array handling (BEFORE Dict per AC-1 precedence)
|
||||
# =========================================================================
|
||||
if isinstance(value, (list, tuple)):
|
||||
return _resolve_array(
|
||||
@@ -115,6 +104,35 @@ def resolve(
|
||||
array_mode=array_mode,
|
||||
)
|
||||
|
||||
# =========================================================================
|
||||
# 5. Dict -> convert to config (STRICT validation)
|
||||
# =========================================================================
|
||||
if isinstance(value, dict):
|
||||
if config_class:
|
||||
# Validate keys BEFORE construction for helpful errors
|
||||
unknown_keys = _validate_dict_keys(value, config_class)
|
||||
if unknown_keys:
|
||||
valid_fields = _get_config_fields(config_class)
|
||||
example = _get_example_dict(config_class)
|
||||
raise TypeError(
|
||||
f"Unknown keys for {param_name}: {unknown_keys}. "
|
||||
f"Valid keys: {valid_fields}. "
|
||||
f"Example: {param_name}={{{example}}}"
|
||||
)
|
||||
try:
|
||||
return config_class(**value)
|
||||
except TypeError as e:
|
||||
raise TypeError(
|
||||
f"Invalid {param_name} dict: {e}. "
|
||||
f"Valid fields: {_get_config_fields(config_class)}"
|
||||
)
|
||||
else:
|
||||
# No config class - dict input not supported
|
||||
raise TypeError(
|
||||
f"Dict input not supported for {param_name} (no config class defined). "
|
||||
f"Use a supported type: bool, str, or instance."
|
||||
)
|
||||
|
||||
# =========================================================================
|
||||
# 6. String handling
|
||||
# =========================================================================
|
||||
@@ -203,7 +221,7 @@ def _resolve_array(
|
||||
return config_class(sources=sources, **config_override)
|
||||
return sources
|
||||
|
||||
# Single or list mode
|
||||
# Single or list mode - SINGLE item only, multiple items raise error
|
||||
if array_mode == ArrayMode.SINGLE_OR_LIST:
|
||||
if len(value) == 1:
|
||||
single_value = value[0]
|
||||
@@ -215,10 +233,17 @@ def _resolve_array(
|
||||
# Check if single item is a preset
|
||||
if isinstance(single_value, str) and presets and single_value in presets:
|
||||
return _apply_preset(single_value, presets, config_class)
|
||||
# Multiple items - treat as sources
|
||||
if config_class:
|
||||
return config_class(sources=list(value))
|
||||
return list(value)
|
||||
# Single non-preset string - try as URL or return as-is
|
||||
if isinstance(single_value, str):
|
||||
if config_class:
|
||||
return config_class()
|
||||
return single_value
|
||||
# Multiple items - ERROR (not allowed for SINGLE_OR_LIST mode)
|
||||
raise TypeError(
|
||||
f"Multiple values not allowed for {param_name}. "
|
||||
f"Use a single value: {param_name}='preset' or {param_name}='url://...' "
|
||||
f"Got {len(value)} items: {value}"
|
||||
)
|
||||
|
||||
# Preset override mode - [preset, {overrides}] or [preset]
|
||||
if array_mode == ArrayMode.PRESET_OVERRIDE:
|
||||
@@ -384,6 +409,81 @@ def _get_config_fields(config_class: Type) -> str:
|
||||
return "unknown"
|
||||
|
||||
|
||||
def _get_valid_keys(config_class: Type) -> Optional[Set[str]]:
|
||||
"""
|
||||
Get valid keys from a config class.
|
||||
|
||||
Supports:
|
||||
- Dataclasses (via __dataclass_fields__)
|
||||
- Regular classes (via __init__ signature)
|
||||
- Classes with __annotations__
|
||||
|
||||
Returns:
|
||||
Set of valid key names, or None if cannot determine.
|
||||
"""
|
||||
# 1. Dataclass fields (most common)
|
||||
if hasattr(config_class, '__dataclass_fields__'):
|
||||
return set(config_class.__dataclass_fields__.keys())
|
||||
|
||||
# 2. __init__ signature parameters
|
||||
try:
|
||||
sig = inspect.signature(config_class.__init__)
|
||||
params = set(sig.parameters.keys()) - {'self'}
|
||||
if params:
|
||||
return params
|
||||
except (ValueError, TypeError):
|
||||
pass
|
||||
|
||||
# 3. Class annotations
|
||||
if hasattr(config_class, '__annotations__'):
|
||||
return set(config_class.__annotations__.keys())
|
||||
|
||||
# Cannot determine valid keys
|
||||
return None
|
||||
|
||||
|
||||
def _validate_dict_keys(value: dict, config_class: Type) -> list:
|
||||
"""
|
||||
Validate dict keys against config class fields.
|
||||
Returns list of unknown keys (empty if all valid).
|
||||
O(n) where n = number of keys in dict (typically small).
|
||||
|
||||
Works for both dataclasses and regular classes.
|
||||
"""
|
||||
valid_keys = _get_valid_keys(config_class)
|
||||
if valid_keys is None:
|
||||
return [] # Cannot validate, allow all keys
|
||||
|
||||
provided_keys = set(value.keys())
|
||||
unknown = provided_keys - valid_keys
|
||||
return list(unknown)
|
||||
|
||||
|
||||
def _get_example_dict(config_class: Type) -> str:
|
||||
"""
|
||||
Generate a minimal example dict snippet for error messages.
|
||||
Shows first 2-3 fields with placeholder values.
|
||||
"""
|
||||
if not hasattr(config_class, '__dataclass_fields__'):
|
||||
return "..."
|
||||
|
||||
fields = list(config_class.__dataclass_fields__.items())[:3]
|
||||
examples = []
|
||||
for name, field in fields:
|
||||
# Generate appropriate placeholder based on type hint
|
||||
field_type = field.type if hasattr(field, 'type') else None
|
||||
if field_type is bool or str(field_type) == 'bool':
|
||||
examples.append(f"'{name}': True")
|
||||
elif field_type is int or str(field_type) == 'int':
|
||||
examples.append(f"'{name}': 1")
|
||||
elif field_type is str or 'str' in str(field_type):
|
||||
examples.append(f"'{name}': '...'")
|
||||
else:
|
||||
examples.append(f"'{name}': ...")
|
||||
|
||||
return ", ".join(examples)
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# Convenience Functions for Specific Parameters
|
||||
# =============================================================================
|
||||
|
||||
@@ -12,7 +12,9 @@ Provides dataclasses for consolidated workflow feature configuration:
|
||||
- WorkflowStepRoutingConfig: Step routing/branching settings
|
||||
|
||||
All configs follow the workflow-centric pattern with precedence:
|
||||
Instance > Config > String > Bool > Default
|
||||
Instance > Config > Dict > Array > String > Bool > Default
|
||||
|
||||
Uses the CANONICAL resolver from param_resolver.py for DRY resolution.
|
||||
|
||||
Usage:
|
||||
from praisonaiagents import Workflow, WorkflowOutputConfig
|
||||
@@ -263,7 +265,7 @@ class WorkflowStepRoutingConfig:
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# Precedence Resolution Helper
|
||||
# Precedence Resolution Helper - Uses CANONICAL resolver
|
||||
# =============================================================================
|
||||
|
||||
def resolve_param(
|
||||
@@ -275,245 +277,173 @@ def resolve_param(
|
||||
array_mode: Optional[str] = None,
|
||||
) -> Any:
|
||||
"""
|
||||
Resolve a consolidated parameter following precedence rules:
|
||||
Instance > Config > Array > String > Bool > Default
|
||||
Resolve a consolidated parameter following precedence rules.
|
||||
|
||||
Args:
|
||||
value: The parameter value (can be Instance, Config, Array, String, Bool, or None)
|
||||
config_class: The expected config dataclass type
|
||||
presets: Dict mapping preset strings to config instances or dicts
|
||||
default: Default value if None/unset
|
||||
instance_check: Optional function to check if value is an instance
|
||||
array_mode: How to handle arrays ("preset_override", "step_names", "passthrough")
|
||||
|
||||
Returns:
|
||||
Resolved config object or value
|
||||
|
||||
Example:
|
||||
# Resolve output param
|
||||
output_config = resolve_param(
|
||||
value=output,
|
||||
config_class=WorkflowOutputConfig,
|
||||
presets={
|
||||
"silent": WorkflowOutputConfig(verbose=False, stream=False),
|
||||
"verbose": WorkflowOutputConfig(verbose=True, stream=True),
|
||||
},
|
||||
default=WorkflowOutputConfig(),
|
||||
)
|
||||
|
||||
# With array override
|
||||
output_config = resolve_param(
|
||||
value=["verbose", {"stream": False}],
|
||||
config_class=WorkflowOutputConfig,
|
||||
presets=presets,
|
||||
array_mode="preset_override",
|
||||
)
|
||||
DEPRECATED: This is a wrapper around the canonical resolver for backward compatibility.
|
||||
New code should use `from ..config.param_resolver import resolve` directly.
|
||||
|
||||
Precedence: Instance > Config > Dict > Array > String > Bool > Default
|
||||
"""
|
||||
# None/unset -> Default
|
||||
if value is None:
|
||||
return default
|
||||
# Import canonical resolver
|
||||
from ..config.param_resolver import resolve as canonical_resolve, ArrayMode
|
||||
|
||||
# Instance check (e.g., pre-built manager object)
|
||||
if instance_check and instance_check(value):
|
||||
return value
|
||||
# Map string array_mode to ArrayMode enum
|
||||
array_mode_map = {
|
||||
"preset_override": ArrayMode.PRESET_OVERRIDE,
|
||||
"step_names": ArrayMode.STEP_NAMES,
|
||||
"passthrough": ArrayMode.PASSTHROUGH,
|
||||
}
|
||||
canonical_array_mode = array_mode_map.get(array_mode) if array_mode else None
|
||||
|
||||
# Config dataclass
|
||||
if isinstance(value, config_class):
|
||||
return value
|
||||
# Convert presets from config instances to dicts for canonical resolver
|
||||
presets_dict = None
|
||||
if presets:
|
||||
presets_dict = {}
|
||||
for key, val in presets.items():
|
||||
if hasattr(val, 'to_dict'):
|
||||
presets_dict[key] = val.to_dict()
|
||||
elif hasattr(val, '__dataclass_fields__'):
|
||||
from dataclasses import asdict
|
||||
presets_dict[key] = asdict(val)
|
||||
else:
|
||||
presets_dict[key] = val
|
||||
|
||||
# Dict -> convert to config
|
||||
if isinstance(value, dict):
|
||||
try:
|
||||
return config_class(**value)
|
||||
except TypeError:
|
||||
return default
|
||||
|
||||
# Array handling (NEW - before string to respect precedence)
|
||||
if isinstance(value, (list, tuple)):
|
||||
if not value:
|
||||
return None # Empty array = disabled
|
||||
|
||||
# Passthrough mode - return list as-is
|
||||
if array_mode == "passthrough":
|
||||
return list(value)
|
||||
|
||||
# Step names mode - for context/routing
|
||||
if array_mode == "step_names":
|
||||
# All strings = step names
|
||||
if all(isinstance(item, str) for item in value):
|
||||
return value # Return list directly for step names
|
||||
|
||||
# Preset override mode - [preset, {overrides}]
|
||||
if array_mode == "preset_override" and presets:
|
||||
first = value[0]
|
||||
if isinstance(first, str) and first in presets:
|
||||
# Get base config from preset
|
||||
preset_value = presets[first]
|
||||
if isinstance(preset_value, dict):
|
||||
base_config = config_class(**preset_value)
|
||||
elif isinstance(preset_value, config_class):
|
||||
base_config = preset_value
|
||||
else:
|
||||
base_config = config_class()
|
||||
|
||||
# Apply overrides if present
|
||||
if len(value) >= 2 and isinstance(value[-1], dict):
|
||||
overrides = value[-1]
|
||||
# Merge base with overrides
|
||||
from dataclasses import asdict
|
||||
base_dict = asdict(base_config)
|
||||
base_dict.update(overrides)
|
||||
return config_class(**base_dict)
|
||||
|
||||
return base_config
|
||||
|
||||
# Default: treat first item as value
|
||||
if len(value) == 1:
|
||||
return resolve_param(value[0], config_class, presets, default, instance_check)
|
||||
|
||||
return default
|
||||
|
||||
# String -> preset lookup
|
||||
if isinstance(value, str) and presets:
|
||||
preset_value = presets.get(value)
|
||||
if preset_value is not None:
|
||||
if isinstance(preset_value, dict):
|
||||
return config_class(**preset_value)
|
||||
return preset_value
|
||||
return default
|
||||
|
||||
# Bool -> True enables defaults, False disables
|
||||
if isinstance(value, bool):
|
||||
if value:
|
||||
return config_class() if config_class else True
|
||||
return None # Disabled
|
||||
|
||||
# Fallback to default
|
||||
return default
|
||||
return canonical_resolve(
|
||||
value=value,
|
||||
param_name="workflow_param",
|
||||
config_class=config_class,
|
||||
presets=presets_dict,
|
||||
default=default,
|
||||
instance_check=instance_check,
|
||||
array_mode=canonical_array_mode,
|
||||
)
|
||||
|
||||
|
||||
def resolve_output_config(value: Any) -> WorkflowOutputConfig:
|
||||
"""Resolve output parameter to WorkflowOutputConfig.
|
||||
"""Resolve output parameter to WorkflowOutputConfig using canonical resolver.
|
||||
|
||||
Supports: None, str preset, list [preset, overrides], Config, dict
|
||||
"""
|
||||
presets = {
|
||||
"silent": WorkflowOutputConfig(verbose=False, stream=False),
|
||||
"minimal": WorkflowOutputConfig(verbose=False, stream=True),
|
||||
"normal": WorkflowOutputConfig(verbose=False, stream=True),
|
||||
"verbose": WorkflowOutputConfig(verbose=True, stream=True),
|
||||
"debug": WorkflowOutputConfig(verbose=True, stream=True),
|
||||
}
|
||||
result = resolve_param(
|
||||
value, WorkflowOutputConfig, presets, WorkflowOutputConfig(),
|
||||
array_mode="preset_override"
|
||||
from ..config.param_resolver import resolve, ArrayMode
|
||||
from ..config.presets import WORKFLOW_OUTPUT_PRESETS
|
||||
|
||||
result = resolve(
|
||||
value=value,
|
||||
param_name="output",
|
||||
config_class=WorkflowOutputConfig,
|
||||
presets=WORKFLOW_OUTPUT_PRESETS,
|
||||
array_mode=ArrayMode.PRESET_OVERRIDE,
|
||||
default=WorkflowOutputConfig(),
|
||||
)
|
||||
return result if result else WorkflowOutputConfig(verbose=False, stream=False)
|
||||
|
||||
|
||||
def resolve_planning_config(value: Any) -> Optional[WorkflowPlanningConfig]:
|
||||
"""Resolve planning parameter to WorkflowPlanningConfig or None."""
|
||||
if value is None or value is False:
|
||||
return None
|
||||
if value is True:
|
||||
return WorkflowPlanningConfig(enabled=True)
|
||||
if isinstance(value, WorkflowPlanningConfig):
|
||||
return value
|
||||
if isinstance(value, dict):
|
||||
return WorkflowPlanningConfig(**value)
|
||||
if isinstance(value, str):
|
||||
# String is treated as LLM model name
|
||||
return WorkflowPlanningConfig(enabled=True, llm=value)
|
||||
return None
|
||||
"""Resolve planning parameter to WorkflowPlanningConfig using canonical resolver."""
|
||||
from ..config.param_resolver import resolve, ArrayMode
|
||||
|
||||
result = resolve(
|
||||
value=value,
|
||||
param_name="planning",
|
||||
config_class=WorkflowPlanningConfig,
|
||||
string_mode="llm_model",
|
||||
array_mode=ArrayMode.PRESET_OVERRIDE,
|
||||
default=None,
|
||||
)
|
||||
return result
|
||||
|
||||
|
||||
def resolve_memory_config(value: Any) -> Optional[WorkflowMemoryConfig]:
|
||||
"""Resolve memory parameter to WorkflowMemoryConfig or None."""
|
||||
if value is None or value is False:
|
||||
return None
|
||||
if value is True:
|
||||
return WorkflowMemoryConfig()
|
||||
if isinstance(value, WorkflowMemoryConfig):
|
||||
return value
|
||||
if isinstance(value, dict):
|
||||
return WorkflowMemoryConfig(**value)
|
||||
# Instance check - if it has search/add methods, it's a memory instance
|
||||
if hasattr(value, 'search') and hasattr(value, 'add'):
|
||||
return value # Return instance directly
|
||||
return None
|
||||
"""Resolve memory parameter to WorkflowMemoryConfig using canonical resolver."""
|
||||
from ..config.param_resolver import resolve, ArrayMode
|
||||
from ..config.presets import MEMORY_PRESETS, MEMORY_URL_SCHEMES
|
||||
|
||||
result = resolve(
|
||||
value=value,
|
||||
param_name="memory",
|
||||
config_class=WorkflowMemoryConfig,
|
||||
presets=MEMORY_PRESETS,
|
||||
url_schemes=MEMORY_URL_SCHEMES,
|
||||
instance_check=lambda v: hasattr(v, 'search') and hasattr(v, 'add'),
|
||||
array_mode=ArrayMode.SINGLE_OR_LIST,
|
||||
default=None,
|
||||
)
|
||||
return result
|
||||
|
||||
|
||||
def resolve_hooks_config(value: Any) -> Optional[WorkflowHooksConfig]:
|
||||
"""Resolve hooks parameter to WorkflowHooksConfig or None."""
|
||||
if value is None:
|
||||
return None
|
||||
if isinstance(value, WorkflowHooksConfig):
|
||||
return value
|
||||
if isinstance(value, dict):
|
||||
return WorkflowHooksConfig(**value)
|
||||
return None
|
||||
"""Resolve hooks parameter to WorkflowHooksConfig using canonical resolver."""
|
||||
from ..config.param_resolver import resolve, ArrayMode
|
||||
|
||||
result = resolve(
|
||||
value=value,
|
||||
param_name="hooks",
|
||||
config_class=WorkflowHooksConfig,
|
||||
array_mode=ArrayMode.PASSTHROUGH,
|
||||
default=None,
|
||||
)
|
||||
return result
|
||||
|
||||
|
||||
def resolve_step_context_config(value: Any) -> Optional[WorkflowStepContextConfig]:
|
||||
"""Resolve step context parameter."""
|
||||
if value is None or value is False:
|
||||
return None
|
||||
if value is True:
|
||||
return WorkflowStepContextConfig()
|
||||
if isinstance(value, WorkflowStepContextConfig):
|
||||
return value
|
||||
if isinstance(value, dict):
|
||||
return WorkflowStepContextConfig(**value)
|
||||
if isinstance(value, list):
|
||||
# List of step names
|
||||
return WorkflowStepContextConfig(from_steps=value)
|
||||
return None
|
||||
"""Resolve step context parameter using canonical resolver."""
|
||||
from ..config.param_resolver import resolve, ArrayMode
|
||||
|
||||
result = resolve(
|
||||
value=value,
|
||||
param_name="context",
|
||||
config_class=WorkflowStepContextConfig,
|
||||
array_mode=ArrayMode.STEP_NAMES,
|
||||
default=None,
|
||||
)
|
||||
return result
|
||||
|
||||
|
||||
def resolve_step_output_config(value: Any) -> Optional[WorkflowStepOutputConfig]:
|
||||
"""Resolve step output parameter."""
|
||||
if value is None:
|
||||
return None
|
||||
if isinstance(value, WorkflowStepOutputConfig):
|
||||
return value
|
||||
if isinstance(value, dict):
|
||||
return WorkflowStepOutputConfig(**value)
|
||||
if isinstance(value, str):
|
||||
# String is treated as output file
|
||||
"""Resolve step output parameter using canonical resolver."""
|
||||
from ..config.param_resolver import resolve
|
||||
|
||||
result = resolve(
|
||||
value=value,
|
||||
param_name="output",
|
||||
config_class=WorkflowStepOutputConfig,
|
||||
string_mode="path_as_source", # String treated as output file
|
||||
default=None,
|
||||
)
|
||||
# Handle string -> file conversion for backward compatibility
|
||||
if isinstance(value, str) and result is None:
|
||||
return WorkflowStepOutputConfig(file=value)
|
||||
return None
|
||||
return result
|
||||
|
||||
|
||||
def resolve_step_execution_config(value: Any) -> WorkflowStepExecutionConfig:
|
||||
"""Resolve step execution parameter."""
|
||||
presets = {
|
||||
"fast": WorkflowStepExecutionConfig(max_retries=1, quality_check=False),
|
||||
"balanced": WorkflowStepExecutionConfig(max_retries=3, quality_check=True),
|
||||
"thorough": WorkflowStepExecutionConfig(max_retries=5, quality_check=True),
|
||||
}
|
||||
if value is None:
|
||||
return WorkflowStepExecutionConfig()
|
||||
if isinstance(value, WorkflowStepExecutionConfig):
|
||||
return value
|
||||
if isinstance(value, dict):
|
||||
return WorkflowStepExecutionConfig(**value)
|
||||
if isinstance(value, str) and value in presets:
|
||||
return presets[value]
|
||||
return WorkflowStepExecutionConfig()
|
||||
"""Resolve step execution parameter using canonical resolver."""
|
||||
from ..config.param_resolver import resolve, ArrayMode
|
||||
from ..config.presets import WORKFLOW_STEP_EXECUTION_PRESETS
|
||||
|
||||
result = resolve(
|
||||
value=value,
|
||||
param_name="execution",
|
||||
config_class=WorkflowStepExecutionConfig,
|
||||
presets=WORKFLOW_STEP_EXECUTION_PRESETS,
|
||||
array_mode=ArrayMode.PRESET_OVERRIDE,
|
||||
default=WorkflowStepExecutionConfig(),
|
||||
)
|
||||
return result if result else WorkflowStepExecutionConfig()
|
||||
|
||||
|
||||
def resolve_step_routing_config(value: Any) -> Optional[WorkflowStepRoutingConfig]:
|
||||
"""Resolve step routing parameter."""
|
||||
if value is None:
|
||||
return None
|
||||
if isinstance(value, WorkflowStepRoutingConfig):
|
||||
return value
|
||||
if isinstance(value, dict):
|
||||
return WorkflowStepRoutingConfig(**value)
|
||||
if isinstance(value, list):
|
||||
# List of next step names
|
||||
return WorkflowStepRoutingConfig(next_steps=value)
|
||||
return None
|
||||
"""Resolve step routing parameter using canonical resolver."""
|
||||
from ..config.param_resolver import resolve, ArrayMode
|
||||
|
||||
result = resolve(
|
||||
value=value,
|
||||
param_name="routing",
|
||||
config_class=WorkflowStepRoutingConfig,
|
||||
array_mode=ArrayMode.STEP_NAMES,
|
||||
default=None,
|
||||
)
|
||||
return result
|
||||
|
||||
|
||||
# =============================================================================
|
||||
|
||||
@@ -231,20 +231,29 @@ class WorkflowStep:
|
||||
loop_over: Optional[str] = None # Variable name to iterate over (e.g., "items")
|
||||
loop_var: str = "item" # Variable name for current item in loop
|
||||
|
||||
# Guardrail (like process="workflow")
|
||||
guardrail: Optional[Callable[['StepResult'], Tuple[bool, Any]]] = None # Returns (is_valid, feedback)
|
||||
# Guardrails (standardized plural naming)
|
||||
# Accepts: callable, str (LLM prompt), GuardrailConfig, or list [preset, overrides]
|
||||
guardrails: Optional[Any] = None # Union[Callable, str, GuardrailConfig] - validation
|
||||
|
||||
# Backward compatibility alias (deprecated - use guardrails instead)
|
||||
guardrail: Optional[Callable[['StepResult'], Tuple[bool, Any]]] = None # DEPRECATED: use guardrails
|
||||
|
||||
# Image handling (migrated from Task)
|
||||
images: Optional[List[str]] = None # Image paths/URLs for vision tasks
|
||||
|
||||
# ============================================================
|
||||
# CONSOLIDATED FEATURE PARAMS (workflow-centric API)
|
||||
# Precedence: Instance > Config > String > Bool > Default
|
||||
# Precedence: Instance > Config > Array > Dict > String > Bool > Default
|
||||
# ============================================================
|
||||
context: Optional[Any] = None # Union[bool, List[str], WorkflowStepContextConfig] - context handling
|
||||
output: Optional[Any] = None # Union[str, WorkflowStepOutputConfig] - output handling
|
||||
execution: Optional[Any] = None # Union[str, WorkflowStepExecutionConfig] - execution control
|
||||
routing: Optional[Any] = None # Union[List[str], WorkflowStepRoutingConfig] - branching
|
||||
# NEW: Agent-like consolidated params for feature parity
|
||||
autonomy: Optional[Any] = None # Union[bool, AutonomyConfig] - agent autonomy
|
||||
knowledge: Optional[Any] = None # Union[bool, List[str], KnowledgeConfig] - RAG
|
||||
web: Optional[Any] = None # Union[bool, WebConfig] - web search/fetch
|
||||
reflection: Optional[Any] = None # Union[bool, ReflectionConfig] - self-reflection
|
||||
|
||||
# Status tracking
|
||||
status: str = "pending" # pending, running, completed, failed, skipped
|
||||
@@ -264,6 +273,11 @@ class WorkflowStep:
|
||||
_on_error: str = field(default="stop", repr=False)
|
||||
_next_steps: Optional[List[str]] = field(default=None, repr=False)
|
||||
_branch_condition: Optional[Dict[str, List[str]]] = field(default=None, repr=False)
|
||||
# NEW: Resolved configs for agent-like params
|
||||
_autonomy_config: Optional[Any] = field(default=None, repr=False)
|
||||
_knowledge_config: Optional[Any] = field(default=None, repr=False)
|
||||
_web_config: Optional[Any] = field(default=None, repr=False)
|
||||
_reflection_config: Optional[Any] = field(default=None, repr=False)
|
||||
|
||||
def __post_init__(self):
|
||||
"""Resolve consolidated params to internal values."""
|
||||
@@ -272,6 +286,18 @@ class WorkflowStep:
|
||||
resolve_step_execution_config, resolve_step_routing_config,
|
||||
)
|
||||
|
||||
# Handle guardrails/guardrail compatibility
|
||||
# guardrails (plural) takes precedence; guardrail (singular) is deprecated
|
||||
if self.guardrails is None and self.guardrail is not None:
|
||||
# Use deprecated guardrail value, map to guardrails
|
||||
import warnings
|
||||
warnings.warn(
|
||||
"WorkflowStep 'guardrail' parameter is deprecated. Use 'guardrails' instead.",
|
||||
DeprecationWarning,
|
||||
stacklevel=2,
|
||||
)
|
||||
object.__setattr__(self, 'guardrails', self.guardrail)
|
||||
|
||||
# Resolve context param
|
||||
ctx_cfg = resolve_step_context_config(self.context)
|
||||
if ctx_cfg:
|
||||
@@ -300,6 +326,50 @@ class WorkflowStep:
|
||||
if route_cfg:
|
||||
self._next_steps = route_cfg.next_steps
|
||||
self._branch_condition = route_cfg.branches
|
||||
|
||||
# Resolve NEW agent-like params using canonical resolver
|
||||
from ..config.param_resolver import resolve, ArrayMode
|
||||
from ..config.presets import (
|
||||
WEB_PRESETS, REFLECTION_PRESETS, AUTONOMY_PRESETS, KNOWLEDGE_PRESETS,
|
||||
)
|
||||
try:
|
||||
from ..config.feature_configs import (
|
||||
AutonomyConfig, KnowledgeConfig, WebConfig, ReflectionConfig,
|
||||
)
|
||||
except ImportError:
|
||||
AutonomyConfig = KnowledgeConfig = WebConfig = ReflectionConfig = None
|
||||
|
||||
# Resolve autonomy
|
||||
if AutonomyConfig and self.autonomy is not None:
|
||||
self._autonomy_config = resolve(
|
||||
value=self.autonomy, param_name="autonomy",
|
||||
config_class=AutonomyConfig, presets=AUTONOMY_PRESETS,
|
||||
default=None,
|
||||
)
|
||||
|
||||
# Resolve knowledge
|
||||
if KnowledgeConfig and self.knowledge is not None:
|
||||
self._knowledge_config = resolve(
|
||||
value=self.knowledge, param_name="knowledge",
|
||||
config_class=KnowledgeConfig, presets=KNOWLEDGE_PRESETS,
|
||||
array_mode=ArrayMode.SOURCES, default=None,
|
||||
)
|
||||
|
||||
# Resolve web
|
||||
if WebConfig and self.web is not None:
|
||||
self._web_config = resolve(
|
||||
value=self.web, param_name="web",
|
||||
config_class=WebConfig, presets=WEB_PRESETS,
|
||||
default=None,
|
||||
)
|
||||
|
||||
# Resolve reflection
|
||||
if ReflectionConfig and self.reflection is not None:
|
||||
self._reflection_config = resolve(
|
||||
value=self.reflection, param_name="reflection",
|
||||
config_class=ReflectionConfig, presets=REFLECTION_PRESETS,
|
||||
default=None,
|
||||
)
|
||||
|
||||
# Property accessors for backward compatibility
|
||||
@property
|
||||
@@ -423,13 +493,20 @@ class Workflow:
|
||||
|
||||
# ============================================================
|
||||
# CONSOLIDATED FEATURE PARAMS (workflow-centric API)
|
||||
# Precedence: Instance > Config > String > Bool > Default
|
||||
# Precedence: Instance > Config > Array > Dict > String > Bool > Default
|
||||
# Workflow is "Agents in an organised way" - supports same params as Agents
|
||||
# ============================================================
|
||||
output: Optional[Any] = None # Union[str, WorkflowOutputConfig] - verbose/stream
|
||||
planning: Optional[Any] = False # Union[bool, str, WorkflowPlanningConfig] - planning mode
|
||||
memory: Optional[Any] = None # Union[bool, WorkflowMemoryConfig, instance] - memory
|
||||
hooks: Optional[Any] = None # WorkflowHooksConfig - lifecycle callbacks
|
||||
context: Optional[Any] = False # Union[bool, ManagerConfig, ContextManager] - context management
|
||||
# NEW: Agent-like consolidated params for feature parity
|
||||
autonomy: Optional[Any] = None # Union[bool, AutonomyConfig] - agent autonomy
|
||||
knowledge: Optional[Any] = None # Union[bool, List[str], KnowledgeConfig] - RAG
|
||||
guardrails: Optional[Any] = None # Union[bool, Callable, GuardrailConfig] - validation
|
||||
web: Optional[Any] = None # Union[bool, WebConfig] - web search/fetch
|
||||
reflection: Optional[Any] = None # Union[bool, ReflectionConfig] - self-reflection
|
||||
|
||||
# Status tracking
|
||||
status: str = "not_started" # not_started, running, completed, failed
|
||||
@@ -445,6 +522,12 @@ class Workflow:
|
||||
_hooks_config: Optional[Any] = field(default=None, repr=False)
|
||||
_context_manager: Optional[Any] = field(default=None, repr=False)
|
||||
_context_manager_initialized: bool = field(default=False, repr=False)
|
||||
# NEW: Resolved configs for agent-like params (for propagation to steps/agents)
|
||||
_autonomy_config: Optional[Any] = field(default=None, repr=False)
|
||||
_knowledge_config: Optional[Any] = field(default=None, repr=False)
|
||||
_guardrails_config: Optional[Any] = field(default=None, repr=False)
|
||||
_web_config: Optional[Any] = field(default=None, repr=False)
|
||||
_reflection_config: Optional[Any] = field(default=None, repr=False)
|
||||
|
||||
def __post_init__(self):
|
||||
"""Resolve consolidated params to internal values."""
|
||||
@@ -474,6 +557,62 @@ class Workflow:
|
||||
|
||||
# Resolve hooks param
|
||||
self._hooks_config = resolve_hooks_config(self.hooks)
|
||||
|
||||
# Resolve NEW agent-like params using canonical resolver
|
||||
# These are stored for propagation to steps/agents
|
||||
from ..config.param_resolver import resolve, ArrayMode
|
||||
from ..config.presets import (
|
||||
WEB_PRESETS, REFLECTION_PRESETS, GUARDRAIL_PRESETS,
|
||||
AUTONOMY_PRESETS, KNOWLEDGE_PRESETS,
|
||||
)
|
||||
try:
|
||||
from ..config.feature_configs import (
|
||||
AutonomyConfig, KnowledgeConfig, GuardrailConfig,
|
||||
WebConfig, ReflectionConfig,
|
||||
)
|
||||
except ImportError:
|
||||
AutonomyConfig = KnowledgeConfig = GuardrailConfig = None
|
||||
WebConfig = ReflectionConfig = None
|
||||
|
||||
# Resolve autonomy
|
||||
if AutonomyConfig and self.autonomy is not None:
|
||||
self._autonomy_config = resolve(
|
||||
value=self.autonomy, param_name="autonomy",
|
||||
config_class=AutonomyConfig, presets=AUTONOMY_PRESETS,
|
||||
default=None,
|
||||
)
|
||||
|
||||
# Resolve knowledge
|
||||
if KnowledgeConfig and self.knowledge is not None:
|
||||
self._knowledge_config = resolve(
|
||||
value=self.knowledge, param_name="knowledge",
|
||||
config_class=KnowledgeConfig, presets=KNOWLEDGE_PRESETS,
|
||||
array_mode=ArrayMode.SOURCES, default=None,
|
||||
)
|
||||
|
||||
# Resolve guardrails
|
||||
if GuardrailConfig and self.guardrails is not None:
|
||||
self._guardrails_config = resolve(
|
||||
value=self.guardrails, param_name="guardrails",
|
||||
config_class=GuardrailConfig, presets=GUARDRAIL_PRESETS,
|
||||
default=None,
|
||||
)
|
||||
|
||||
# Resolve web
|
||||
if WebConfig and self.web is not None:
|
||||
self._web_config = resolve(
|
||||
value=self.web, param_name="web",
|
||||
config_class=WebConfig, presets=WEB_PRESETS,
|
||||
default=None,
|
||||
)
|
||||
|
||||
# Resolve reflection
|
||||
if ReflectionConfig and self.reflection is not None:
|
||||
self._reflection_config = resolve(
|
||||
value=self.reflection, param_name="reflection",
|
||||
config_class=ReflectionConfig, presets=REFLECTION_PRESETS,
|
||||
default=None,
|
||||
)
|
||||
|
||||
def to_dict(self) -> Dict[str, Any]:
|
||||
return {
|
||||
|
||||
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
|
||||
|
||||
[project]
|
||||
name = "praisonaiagents"
|
||||
version = "0.10.6"
|
||||
version = "0.10.10"
|
||||
description = "Praison AI agents for completing complex tasks with Self Reflection Agents"
|
||||
readme = "README.md"
|
||||
requires-python = ">=3.10"
|
||||
|
||||
@@ -0,0 +1,333 @@
|
||||
"""
|
||||
Integration tests for Dict support in consolidated param resolution.
|
||||
|
||||
Tests that Agent, Agents, and Workflow classes correctly handle dict params
|
||||
through the unified canonical resolver.
|
||||
|
||||
Note: Agent tests are skipped as they require LLM setup. The core dict
|
||||
resolution is tested in unit tests. These tests focus on Workflow/WorkflowStep
|
||||
which don't require LLM setup.
|
||||
"""
|
||||
|
||||
import pytest
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# AGENT DICT PARAM TESTS - Using resolver directly to avoid LLM setup
|
||||
# =============================================================================
|
||||
|
||||
class TestAgentDictParamsResolver:
|
||||
"""Test Agent dict param resolution using the resolver directly."""
|
||||
|
||||
def test_agent_output_dict_resolution(self):
|
||||
"""Output dict resolves correctly through canonical resolver."""
|
||||
from praisonaiagents.config.param_resolver import resolve, ArrayMode
|
||||
from praisonaiagents.config.feature_configs import OutputConfig
|
||||
from praisonaiagents.config.presets import OUTPUT_PRESETS
|
||||
|
||||
result = resolve(
|
||||
value={"verbose": True, "markdown": False},
|
||||
param_name="output",
|
||||
config_class=OutputConfig,
|
||||
presets=OUTPUT_PRESETS,
|
||||
array_mode=ArrayMode.PRESET_OVERRIDE,
|
||||
)
|
||||
assert isinstance(result, OutputConfig)
|
||||
assert result.verbose is True
|
||||
assert result.markdown is False
|
||||
|
||||
def test_agent_execution_dict_resolution(self):
|
||||
"""Execution dict resolves correctly through canonical resolver."""
|
||||
from praisonaiagents.config.param_resolver import resolve, ArrayMode
|
||||
from praisonaiagents.config.feature_configs import ExecutionConfig
|
||||
from praisonaiagents.config.presets import EXECUTION_PRESETS
|
||||
|
||||
result = resolve(
|
||||
value={"max_iter": 50, "max_retry_limit": 10},
|
||||
param_name="execution",
|
||||
config_class=ExecutionConfig,
|
||||
presets=EXECUTION_PRESETS,
|
||||
array_mode=ArrayMode.PRESET_OVERRIDE,
|
||||
)
|
||||
assert isinstance(result, ExecutionConfig)
|
||||
assert result.max_iter == 50
|
||||
|
||||
def test_agent_dict_with_unknown_key_raises_error(self):
|
||||
"""Dict with unknown key raises TypeError."""
|
||||
from praisonaiagents.config.param_resolver import resolve, ArrayMode
|
||||
from praisonaiagents.config.feature_configs import OutputConfig
|
||||
from praisonaiagents.config.presets import OUTPUT_PRESETS
|
||||
|
||||
with pytest.raises(TypeError) as exc_info:
|
||||
resolve(
|
||||
value={"verbose": True, "invalid_key": "value"},
|
||||
param_name="output",
|
||||
config_class=OutputConfig,
|
||||
presets=OUTPUT_PRESETS,
|
||||
array_mode=ArrayMode.PRESET_OVERRIDE,
|
||||
)
|
||||
assert "Unknown keys" in str(exc_info.value)
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# AGENTS (MULTI-AGENT) DICT PARAM TESTS - Using resolver directly
|
||||
# =============================================================================
|
||||
|
||||
class TestAgentsDictParamsResolver:
|
||||
"""Test Agents dict param resolution using the resolver directly."""
|
||||
|
||||
def test_agents_output_dict_resolution(self):
|
||||
"""Multi-agent output dict resolves correctly."""
|
||||
from praisonaiagents.config.param_resolver import resolve, ArrayMode
|
||||
from praisonaiagents.config.feature_configs import MultiAgentOutputConfig
|
||||
from praisonaiagents.config.presets import MULTI_AGENT_OUTPUT_PRESETS
|
||||
|
||||
result = resolve(
|
||||
value={"verbose": 2, "stream": False},
|
||||
param_name="output",
|
||||
config_class=MultiAgentOutputConfig,
|
||||
presets=MULTI_AGENT_OUTPUT_PRESETS,
|
||||
array_mode=ArrayMode.PRESET_OVERRIDE,
|
||||
)
|
||||
assert isinstance(result, MultiAgentOutputConfig)
|
||||
assert result.verbose == 2
|
||||
assert result.stream is False
|
||||
|
||||
def test_agents_execution_dict_resolution(self):
|
||||
"""Multi-agent execution dict resolves correctly."""
|
||||
from praisonaiagents.config.param_resolver import resolve, ArrayMode
|
||||
from praisonaiagents.config.feature_configs import MultiAgentExecutionConfig
|
||||
from praisonaiagents.config.presets import MULTI_AGENT_EXECUTION_PRESETS
|
||||
|
||||
result = resolve(
|
||||
value={"max_iter": 25, "max_retries": 8},
|
||||
param_name="execution",
|
||||
config_class=MultiAgentExecutionConfig,
|
||||
presets=MULTI_AGENT_EXECUTION_PRESETS,
|
||||
array_mode=ArrayMode.PRESET_OVERRIDE,
|
||||
)
|
||||
assert isinstance(result, MultiAgentExecutionConfig)
|
||||
assert result.max_iter == 25
|
||||
assert result.max_retries == 8
|
||||
|
||||
def test_agents_hooks_dict_resolution(self):
|
||||
"""Multi-agent hooks dict resolves correctly."""
|
||||
from praisonaiagents.config.param_resolver import resolve, ArrayMode
|
||||
from praisonaiagents.config.feature_configs import MultiAgentHooksConfig
|
||||
|
||||
def my_checker(result):
|
||||
return True
|
||||
|
||||
result = resolve(
|
||||
value={"completion_checker": my_checker},
|
||||
param_name="hooks",
|
||||
config_class=MultiAgentHooksConfig,
|
||||
array_mode=ArrayMode.PASSTHROUGH,
|
||||
)
|
||||
assert isinstance(result, MultiAgentHooksConfig)
|
||||
assert result.completion_checker is my_checker
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# WORKFLOW DICT PARAM TESTS
|
||||
# =============================================================================
|
||||
|
||||
class TestWorkflowDictParams:
|
||||
"""Test Workflow and WorkflowStep with dict params."""
|
||||
|
||||
def test_workflow_output_dict(self):
|
||||
"""Workflow accepts output as dict."""
|
||||
from praisonaiagents.workflows.workflows import Workflow
|
||||
|
||||
workflow = Workflow(
|
||||
name="test",
|
||||
steps=[],
|
||||
output={"verbose": True, "stream": False},
|
||||
)
|
||||
# Output config should be resolved
|
||||
assert workflow.output is not None
|
||||
|
||||
def test_workflow_planning_dict(self):
|
||||
"""Workflow accepts planning as dict."""
|
||||
from praisonaiagents.workflows.workflows import Workflow
|
||||
|
||||
workflow = Workflow(
|
||||
name="test",
|
||||
steps=[],
|
||||
planning={"enabled": True, "llm": "gpt-4o"},
|
||||
)
|
||||
# Planning should be configured
|
||||
assert workflow.planning is not None
|
||||
|
||||
def test_workflow_memory_dict(self):
|
||||
"""Workflow accepts memory as dict."""
|
||||
from praisonaiagents.workflows.workflows import Workflow
|
||||
|
||||
workflow = Workflow(
|
||||
name="test",
|
||||
steps=[],
|
||||
memory={"backend": "sqlite", "user_id": "test_user"},
|
||||
)
|
||||
# Memory should be configured
|
||||
assert workflow.memory is not None
|
||||
|
||||
def test_workflowstep_context_dict(self):
|
||||
"""WorkflowStep accepts context as dict."""
|
||||
from praisonaiagents.workflows.workflows import WorkflowStep
|
||||
|
||||
step = WorkflowStep(
|
||||
name="step1",
|
||||
context={"from_steps": ["step0"], "retain_full": True},
|
||||
)
|
||||
assert step.context_from == ["step0"]
|
||||
assert step.retain_full_context is True
|
||||
|
||||
def test_workflowstep_output_dict(self):
|
||||
"""WorkflowStep accepts output as dict."""
|
||||
from praisonaiagents.workflows.workflows import WorkflowStep
|
||||
|
||||
step = WorkflowStep(
|
||||
name="step1",
|
||||
output={"file": "output.txt", "variable": "result"},
|
||||
)
|
||||
assert step.output_file == "output.txt"
|
||||
assert step.output_variable == "result"
|
||||
|
||||
def test_workflowstep_execution_dict(self):
|
||||
"""WorkflowStep accepts execution as dict."""
|
||||
from praisonaiagents.workflows.workflows import WorkflowStep
|
||||
|
||||
step = WorkflowStep(
|
||||
name="step1",
|
||||
execution={"max_retries": 5, "quality_check": True},
|
||||
)
|
||||
assert step.max_retries == 5
|
||||
assert step.quality_check is True
|
||||
|
||||
def test_workflowstep_routing_dict(self):
|
||||
"""WorkflowStep accepts routing as dict."""
|
||||
from praisonaiagents.workflows.workflows import WorkflowStep
|
||||
|
||||
step = WorkflowStep(
|
||||
name="step1",
|
||||
routing={"next_steps": ["step2", "step3"]},
|
||||
)
|
||||
assert step.next_steps == ["step2", "step3"]
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# BACKWARD COMPATIBILITY TESTS - Using resolver directly
|
||||
# =============================================================================
|
||||
|
||||
class TestBackwardCompatibility:
|
||||
"""Test that existing usage patterns still work via resolver."""
|
||||
|
||||
def test_bool_true_creates_default_config(self):
|
||||
"""Bool True creates default config instance."""
|
||||
from praisonaiagents.config.param_resolver import resolve
|
||||
from praisonaiagents.config.feature_configs import PlanningConfig
|
||||
|
||||
result = resolve(
|
||||
value=True,
|
||||
param_name="planning",
|
||||
config_class=PlanningConfig,
|
||||
)
|
||||
assert isinstance(result, PlanningConfig)
|
||||
|
||||
def test_bool_false_returns_none(self):
|
||||
"""Bool False returns None (disabled)."""
|
||||
from praisonaiagents.config.param_resolver import resolve
|
||||
from praisonaiagents.config.feature_configs import ReflectionConfig
|
||||
|
||||
result = resolve(
|
||||
value=False,
|
||||
param_name="reflection",
|
||||
config_class=ReflectionConfig,
|
||||
)
|
||||
assert result is None
|
||||
|
||||
def test_string_preset_resolves(self):
|
||||
"""String preset resolves to config."""
|
||||
from praisonaiagents.config.param_resolver import resolve, ArrayMode
|
||||
from praisonaiagents.config.feature_configs import OutputConfig
|
||||
from praisonaiagents.config.presets import OUTPUT_PRESETS
|
||||
|
||||
result = resolve(
|
||||
value="verbose",
|
||||
param_name="output",
|
||||
config_class=OutputConfig,
|
||||
presets=OUTPUT_PRESETS,
|
||||
array_mode=ArrayMode.PRESET_OVERRIDE,
|
||||
)
|
||||
assert isinstance(result, OutputConfig)
|
||||
assert result.verbose is True
|
||||
|
||||
def test_array_preset_override_works(self):
|
||||
"""[preset, overrides] pattern works."""
|
||||
from praisonaiagents.config.param_resolver import resolve, ArrayMode
|
||||
from praisonaiagents.config.feature_configs import OutputConfig
|
||||
from praisonaiagents.config.presets import OUTPUT_PRESETS
|
||||
|
||||
result = resolve(
|
||||
value=["verbose", {"stream": False}],
|
||||
param_name="output",
|
||||
config_class=OutputConfig,
|
||||
presets=OUTPUT_PRESETS,
|
||||
array_mode=ArrayMode.PRESET_OVERRIDE,
|
||||
)
|
||||
assert isinstance(result, OutputConfig)
|
||||
assert result.verbose is True
|
||||
assert result.stream is False
|
||||
|
||||
def test_workflow_bool_params_still_work(self):
|
||||
"""Workflow with bool params still works."""
|
||||
from praisonaiagents.workflows.workflows import Workflow
|
||||
|
||||
workflow = Workflow(
|
||||
name="test",
|
||||
steps=[],
|
||||
planning=True,
|
||||
)
|
||||
# Should not raise - workflow created successfully
|
||||
assert workflow is not None
|
||||
|
||||
def test_workflowstep_list_context_still_works(self):
|
||||
"""WorkflowStep with list context still works."""
|
||||
from praisonaiagents.workflows.workflows import WorkflowStep
|
||||
|
||||
step = WorkflowStep(
|
||||
name="step1",
|
||||
context=["step0", "step_prev"],
|
||||
)
|
||||
assert step.context_from == ["step0", "step_prev"]
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# BASE_URL / API_KEY SEPARATION TESTS
|
||||
# =============================================================================
|
||||
|
||||
class TestBaseUrlApiKeySeparation:
|
||||
"""Test that base_url and api_key remain separate parameters."""
|
||||
|
||||
def test_agent_base_url_separate(self):
|
||||
"""Agent base_url is a separate parameter, not consolidated."""
|
||||
from praisonaiagents.agent.agent import Agent
|
||||
import inspect
|
||||
|
||||
sig = inspect.signature(Agent.__init__)
|
||||
params = list(sig.parameters.keys())
|
||||
|
||||
# base_url should be a separate parameter
|
||||
assert "base_url" in params
|
||||
# It should NOT be part of any consolidated config
|
||||
|
||||
def test_agent_api_key_separate(self):
|
||||
"""Agent api_key is a separate parameter, not consolidated."""
|
||||
from praisonaiagents.agent.agent import Agent
|
||||
import inspect
|
||||
|
||||
sig = inspect.signature(Agent.__init__)
|
||||
params = list(sig.parameters.keys())
|
||||
|
||||
# api_key should be a separate parameter
|
||||
assert "api_key" in params
|
||||
@@ -0,0 +1,526 @@
|
||||
"""
|
||||
Comprehensive tests for param_resolver.py - TDD for outstanding gaps.
|
||||
|
||||
Tests cover:
|
||||
1. Dict strict validation (dataclass and non-dataclass configs)
|
||||
2. Array PRESET_OVERRIDE invalid preset error UX
|
||||
3. Precedence ladder for all consolidated params
|
||||
4. Agent consolidated params via canonical resolver
|
||||
5. Workflow/WorkflowStep resolution consistency
|
||||
"""
|
||||
|
||||
import pytest
|
||||
from dataclasses import dataclass, field
|
||||
from typing import List
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# Test Fixtures - Config Classes
|
||||
# =============================================================================
|
||||
|
||||
@dataclass
|
||||
class MockConfig:
|
||||
"""Mock config for testing."""
|
||||
enabled: bool = True
|
||||
value: str = "default"
|
||||
count: int = 0
|
||||
|
||||
|
||||
@dataclass
|
||||
class MockConfigWithSources:
|
||||
"""Mock config with sources field."""
|
||||
sources: List[str] = field(default_factory=list)
|
||||
rerank: bool = False
|
||||
|
||||
|
||||
class NonDataclassConfig:
|
||||
"""Non-dataclass config for testing validation."""
|
||||
def __init__(self, enabled: bool = True, value: str = "default"):
|
||||
self.enabled = enabled
|
||||
self.value = value
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# PHASE 1A: Dict Strict Validation Tests
|
||||
# =============================================================================
|
||||
|
||||
class TestDictStrictValidation:
|
||||
"""Tests for strict dict key validation."""
|
||||
|
||||
def test_dict_unknown_keys_error_dataclass(self):
|
||||
"""Dict with unknown keys should raise TypeError with helpful message."""
|
||||
from praisonaiagents.config.param_resolver import resolve
|
||||
|
||||
with pytest.raises(TypeError) as exc_info:
|
||||
resolve(
|
||||
value={"enabled": True, "unknown_key": "bad"},
|
||||
param_name="test",
|
||||
config_class=MockConfig,
|
||||
)
|
||||
|
||||
error_msg = str(exc_info.value)
|
||||
assert "unknown_key" in error_msg.lower() or "Unknown keys" in error_msg
|
||||
assert "enabled" in error_msg or "Valid keys" in error_msg
|
||||
|
||||
def test_dict_valid_keys_accepted(self):
|
||||
"""Dict with valid keys should create config."""
|
||||
from praisonaiagents.config.param_resolver import resolve
|
||||
|
||||
result = resolve(
|
||||
value={"enabled": False, "value": "custom"},
|
||||
param_name="test",
|
||||
config_class=MockConfig,
|
||||
)
|
||||
|
||||
assert isinstance(result, MockConfig)
|
||||
assert result.enabled is False
|
||||
assert result.value == "custom"
|
||||
|
||||
def test_dict_no_config_class_error(self):
|
||||
"""Dict without config_class should raise TypeError."""
|
||||
from praisonaiagents.config.param_resolver import resolve
|
||||
|
||||
with pytest.raises(TypeError) as exc_info:
|
||||
resolve(
|
||||
value={"key": "value"},
|
||||
param_name="test",
|
||||
config_class=None,
|
||||
)
|
||||
|
||||
assert "not supported" in str(exc_info.value).lower() or "no config class" in str(exc_info.value).lower()
|
||||
|
||||
def test_dict_validation_non_dataclass_config(self):
|
||||
"""Non-dataclass config should still validate keys via signature/annotations."""
|
||||
from praisonaiagents.config.param_resolver import _get_valid_keys
|
||||
|
||||
# This tests the _get_valid_keys helper handles non-dataclass
|
||||
# The implementation should use inspect.signature or __annotations__
|
||||
# For now, verify the helper exists and returns something useful
|
||||
keys = _get_valid_keys(NonDataclassConfig)
|
||||
# Should return valid keys or empty (graceful degradation)
|
||||
assert isinstance(keys, (set, list, frozenset)) or keys is None
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# PHASE 1B: Array PRESET_OVERRIDE Error UX Tests
|
||||
# =============================================================================
|
||||
|
||||
class TestArrayPresetOverrideErrorUX:
|
||||
"""Tests for array preset error messages with suggestions."""
|
||||
|
||||
def test_invalid_preset_in_array_raises_error_with_suggestions(self):
|
||||
"""Invalid preset string in array should raise error with typo suggestions."""
|
||||
from praisonaiagents.config.param_resolver import resolve, ArrayMode
|
||||
|
||||
presets = {
|
||||
"fast": {"enabled": True, "value": "fast"},
|
||||
"slow": {"enabled": True, "value": "slow"},
|
||||
"balanced": {"enabled": True, "value": "balanced"},
|
||||
}
|
||||
|
||||
with pytest.raises((ValueError, TypeError)) as exc_info:
|
||||
resolve(
|
||||
value=["fats"], # Typo for "fast"
|
||||
param_name="execution",
|
||||
config_class=MockConfig,
|
||||
presets=presets,
|
||||
array_mode=ArrayMode.PRESET_OVERRIDE,
|
||||
)
|
||||
|
||||
error_msg = str(exc_info.value).lower()
|
||||
# Should suggest similar preset
|
||||
assert "fast" in error_msg or "fats" in error_msg
|
||||
# Should list valid presets
|
||||
assert "valid" in error_msg or "available" in error_msg or any(p in error_msg for p in presets.keys())
|
||||
|
||||
def test_valid_preset_in_array_works(self):
|
||||
"""Valid preset string in array should work."""
|
||||
from praisonaiagents.config.param_resolver import resolve, ArrayMode
|
||||
|
||||
presets = {
|
||||
"fast": {"enabled": True, "value": "fast"},
|
||||
}
|
||||
|
||||
result = resolve(
|
||||
value=["fast"],
|
||||
param_name="execution",
|
||||
config_class=MockConfig,
|
||||
presets=presets,
|
||||
array_mode=ArrayMode.PRESET_OVERRIDE,
|
||||
)
|
||||
|
||||
assert isinstance(result, MockConfig)
|
||||
assert result.value == "fast"
|
||||
|
||||
def test_preset_with_overrides_in_array(self):
|
||||
"""Preset with override dict in array should merge correctly."""
|
||||
from praisonaiagents.config.param_resolver import resolve, ArrayMode
|
||||
|
||||
presets = {
|
||||
"fast": {"enabled": True, "value": "fast", "count": 1},
|
||||
}
|
||||
|
||||
result = resolve(
|
||||
value=["fast", {"count": 99}],
|
||||
param_name="execution",
|
||||
config_class=MockConfig,
|
||||
presets=presets,
|
||||
array_mode=ArrayMode.PRESET_OVERRIDE,
|
||||
)
|
||||
|
||||
assert isinstance(result, MockConfig)
|
||||
assert result.value == "fast"
|
||||
assert result.count == 99
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# PHASE 1C: Precedence Ladder Tests
|
||||
# =============================================================================
|
||||
|
||||
class TestPrecedenceLadder:
|
||||
"""Tests for Instance > Config > Dict > Array > String > Bool > Default precedence."""
|
||||
|
||||
def test_instance_highest_precedence(self):
|
||||
"""Instance check should have highest precedence."""
|
||||
from praisonaiagents.config.param_resolver import resolve
|
||||
|
||||
class MockInstance:
|
||||
def search(self): pass
|
||||
def add(self): pass
|
||||
|
||||
instance = MockInstance()
|
||||
result = resolve(
|
||||
value=instance,
|
||||
param_name="memory",
|
||||
config_class=MockConfig,
|
||||
instance_check=lambda v: hasattr(v, 'search') and hasattr(v, 'add'),
|
||||
)
|
||||
|
||||
assert result is instance
|
||||
|
||||
def test_config_class_instance_precedence(self):
|
||||
"""Config class instance should be returned as-is."""
|
||||
from praisonaiagents.config.param_resolver import resolve
|
||||
|
||||
config = MockConfig(enabled=False, value="custom")
|
||||
result = resolve(
|
||||
value=config,
|
||||
param_name="test",
|
||||
config_class=MockConfig,
|
||||
)
|
||||
|
||||
assert result is config
|
||||
|
||||
def test_dict_before_string(self):
|
||||
"""Dict should be processed before string."""
|
||||
from praisonaiagents.config.param_resolver import resolve
|
||||
|
||||
# Dict input
|
||||
result = resolve(
|
||||
value={"enabled": False},
|
||||
param_name="test",
|
||||
config_class=MockConfig,
|
||||
)
|
||||
|
||||
assert isinstance(result, MockConfig)
|
||||
assert result.enabled is False
|
||||
|
||||
def test_bool_true_returns_default_config(self):
|
||||
"""Bool True should return default config instance."""
|
||||
from praisonaiagents.config.param_resolver import resolve
|
||||
|
||||
result = resolve(
|
||||
value=True,
|
||||
param_name="test",
|
||||
config_class=MockConfig,
|
||||
)
|
||||
|
||||
assert isinstance(result, MockConfig)
|
||||
assert result.enabled is True # Default
|
||||
|
||||
def test_bool_false_returns_none(self):
|
||||
"""Bool False should return None (disabled)."""
|
||||
from praisonaiagents.config.param_resolver import resolve
|
||||
|
||||
result = resolve(
|
||||
value=False,
|
||||
param_name="test",
|
||||
config_class=MockConfig,
|
||||
)
|
||||
|
||||
assert result is None
|
||||
|
||||
def test_none_returns_default(self):
|
||||
"""None should return the default value."""
|
||||
from praisonaiagents.config.param_resolver import resolve
|
||||
|
||||
default = MockConfig(value="default_value")
|
||||
result = resolve(
|
||||
value=None,
|
||||
param_name="test",
|
||||
config_class=MockConfig,
|
||||
default=default,
|
||||
)
|
||||
|
||||
assert result is default
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# PHASE 1D: Agent Consolidated Params Tests
|
||||
# =============================================================================
|
||||
|
||||
class TestAgentConsolidatedParams:
|
||||
"""Tests for Agent consolidated params using canonical resolver."""
|
||||
|
||||
def test_agent_hooks_via_resolver(self):
|
||||
"""Agent hooks param should use canonical resolver."""
|
||||
from praisonaiagents.config.feature_configs import HooksConfig
|
||||
from praisonaiagents.config.param_resolver import resolve, ArrayMode
|
||||
|
||||
# Test list passthrough
|
||||
hooks_list = [lambda x: x]
|
||||
result = resolve(
|
||||
value=hooks_list,
|
||||
param_name="hooks",
|
||||
config_class=HooksConfig,
|
||||
array_mode=ArrayMode.PASSTHROUGH,
|
||||
)
|
||||
assert result == hooks_list
|
||||
|
||||
# Test HooksConfig instance
|
||||
hooks_config = HooksConfig(on_step=lambda x: x)
|
||||
result = resolve(
|
||||
value=hooks_config,
|
||||
param_name="hooks",
|
||||
config_class=HooksConfig,
|
||||
)
|
||||
assert result is hooks_config
|
||||
|
||||
# Test dict
|
||||
result = resolve(
|
||||
value={"on_step": None},
|
||||
param_name="hooks",
|
||||
config_class=HooksConfig,
|
||||
)
|
||||
assert isinstance(result, HooksConfig)
|
||||
|
||||
def test_agent_skills_via_resolver(self):
|
||||
"""Agent skills param should use canonical resolver."""
|
||||
from praisonaiagents.config.feature_configs import SkillsConfig
|
||||
from praisonaiagents.config.param_resolver import resolve
|
||||
|
||||
# Test SkillsConfig instance directly
|
||||
config = SkillsConfig(paths=["./skill1", "./skill2"])
|
||||
result = resolve(
|
||||
value=config,
|
||||
param_name="skills",
|
||||
config_class=SkillsConfig,
|
||||
)
|
||||
assert isinstance(result, SkillsConfig)
|
||||
assert result.paths == ["./skill1", "./skill2"]
|
||||
|
||||
# Test dict input
|
||||
result = resolve(
|
||||
value={"paths": ["./my-skill"]},
|
||||
param_name="skills",
|
||||
config_class=SkillsConfig,
|
||||
)
|
||||
assert isinstance(result, SkillsConfig)
|
||||
assert result.paths == ["./my-skill"]
|
||||
|
||||
def test_agent_knowledge_via_resolver(self):
|
||||
"""Agent knowledge param should use canonical resolver."""
|
||||
from praisonaiagents.config.feature_configs import KnowledgeConfig
|
||||
from praisonaiagents.config.param_resolver import resolve, ArrayMode
|
||||
|
||||
# Test list of sources
|
||||
result = resolve(
|
||||
value=["docs/", "data.pdf"],
|
||||
param_name="knowledge",
|
||||
config_class=KnowledgeConfig,
|
||||
array_mode=ArrayMode.SOURCES_WITH_CONFIG,
|
||||
)
|
||||
assert isinstance(result, KnowledgeConfig)
|
||||
assert "docs/" in result.sources
|
||||
|
||||
# Test list with config override
|
||||
result = resolve(
|
||||
value=["docs/", {"rerank": True}],
|
||||
param_name="knowledge",
|
||||
config_class=KnowledgeConfig,
|
||||
array_mode=ArrayMode.SOURCES_WITH_CONFIG,
|
||||
)
|
||||
assert isinstance(result, KnowledgeConfig)
|
||||
assert result.rerank is True
|
||||
|
||||
def test_agent_guardrails_via_resolver(self):
|
||||
"""Agent guardrails param should use canonical resolver."""
|
||||
from praisonaiagents.config.feature_configs import GuardrailConfig
|
||||
from praisonaiagents.config.param_resolver import resolve, ArrayMode
|
||||
from praisonaiagents.config.presets import GUARDRAIL_PRESETS
|
||||
|
||||
# Test string preset
|
||||
if GUARDRAIL_PRESETS:
|
||||
preset_name = list(GUARDRAIL_PRESETS.keys())[0]
|
||||
result = resolve(
|
||||
value=preset_name,
|
||||
param_name="guardrails",
|
||||
config_class=GuardrailConfig,
|
||||
presets=GUARDRAIL_PRESETS,
|
||||
array_mode=ArrayMode.PRESET_OVERRIDE,
|
||||
)
|
||||
assert result is not None
|
||||
|
||||
# Test callable (should be handled via instance_check)
|
||||
def validator_fn(x):
|
||||
return (True, x)
|
||||
|
||||
result = resolve(
|
||||
value=validator_fn,
|
||||
param_name="guardrails",
|
||||
config_class=GuardrailConfig,
|
||||
instance_check=callable,
|
||||
)
|
||||
assert result is validator_fn
|
||||
|
||||
# Test GuardrailConfig instance
|
||||
config = GuardrailConfig(max_retries=5)
|
||||
result = resolve(
|
||||
value=config,
|
||||
param_name="guardrails",
|
||||
config_class=GuardrailConfig,
|
||||
)
|
||||
assert result is config
|
||||
|
||||
# Test string as LLM prompt
|
||||
result = resolve(
|
||||
value="Ensure response is safe and helpful",
|
||||
param_name="guardrails",
|
||||
config_class=GuardrailConfig,
|
||||
string_mode="llm_prompt",
|
||||
)
|
||||
assert isinstance(result, GuardrailConfig)
|
||||
assert result.llm_validator == "Ensure response is safe and helpful"
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# PHASE 1E: Workflow/WorkflowStep Resolution Tests
|
||||
# =============================================================================
|
||||
|
||||
class TestWorkflowResolution:
|
||||
"""Tests for Workflow and WorkflowStep resolution consistency."""
|
||||
|
||||
def test_workflow_step_context_resolved(self):
|
||||
"""WorkflowStep context should be resolved to config."""
|
||||
from praisonaiagents.workflows.workflows import WorkflowStep
|
||||
|
||||
step = WorkflowStep(
|
||||
name="test",
|
||||
context=["step1", "step2"],
|
||||
)
|
||||
|
||||
# Should have resolved _context_from
|
||||
assert step._context_from == ["step1", "step2"]
|
||||
|
||||
def test_workflow_step_execution_resolved(self):
|
||||
"""WorkflowStep execution should be resolved to config."""
|
||||
from praisonaiagents.workflows.workflows import WorkflowStep
|
||||
|
||||
step = WorkflowStep(
|
||||
name="test",
|
||||
execution="fast",
|
||||
)
|
||||
|
||||
# Should have resolved execution config
|
||||
assert step._async_execution is not None or step._max_retries is not None
|
||||
|
||||
def test_workflow_step_output_resolved(self):
|
||||
"""WorkflowStep output should be resolved to config."""
|
||||
from praisonaiagents.workflows.workflows import WorkflowStep
|
||||
from praisonaiagents.workflows.workflow_configs import WorkflowStepOutputConfig
|
||||
|
||||
# Test with config object
|
||||
step = WorkflowStep(
|
||||
name="test",
|
||||
output=WorkflowStepOutputConfig(file="result.txt"),
|
||||
)
|
||||
|
||||
# Should have resolved output file
|
||||
assert step._output_file == "result.txt"
|
||||
|
||||
def test_workflow_output_resolved(self):
|
||||
"""Workflow output should be resolved to config."""
|
||||
from praisonaiagents.workflows.workflows import Workflow
|
||||
|
||||
workflow = Workflow(
|
||||
name="test",
|
||||
output="verbose",
|
||||
)
|
||||
|
||||
# Should have resolved verbose setting
|
||||
assert workflow._verbose is True
|
||||
|
||||
def test_workflow_planning_resolved(self):
|
||||
"""Workflow planning should be resolved to config."""
|
||||
from praisonaiagents.workflows.workflows import Workflow
|
||||
|
||||
workflow = Workflow(
|
||||
name="test",
|
||||
planning=True,
|
||||
)
|
||||
|
||||
# Should have resolved planning enabled
|
||||
assert workflow._planning_enabled is True
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# PHASE 1F: Naming/Alias Tests
|
||||
# =============================================================================
|
||||
|
||||
class TestNamingAlias:
|
||||
"""Tests for Agents vs PraisonAIAgents naming."""
|
||||
|
||||
def test_agents_is_primary(self):
|
||||
"""Agents should be the primary import."""
|
||||
from praisonaiagents import Agents
|
||||
assert Agents is not None
|
||||
|
||||
def test_praisonaiagents_alias_works(self):
|
||||
"""PraisonAIAgents alias should still work."""
|
||||
from praisonaiagents import PraisonAIAgents
|
||||
assert PraisonAIAgents is not None
|
||||
|
||||
def test_both_are_same_class(self):
|
||||
"""Both names should refer to the same class."""
|
||||
from praisonaiagents import Agents, PraisonAIAgents
|
||||
assert Agents is PraisonAIAgents
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# PHASE 1G: Helper Function Tests
|
||||
# =============================================================================
|
||||
|
||||
class TestHelperFunctions:
|
||||
"""Tests for resolver helper functions."""
|
||||
|
||||
def test_get_valid_keys_dataclass(self):
|
||||
"""_get_valid_keys should return dataclass fields."""
|
||||
from praisonaiagents.config.param_resolver import _get_valid_keys
|
||||
|
||||
keys = _get_valid_keys(MockConfig)
|
||||
assert "enabled" in keys
|
||||
assert "value" in keys
|
||||
assert "count" in keys
|
||||
|
||||
def test_get_valid_keys_non_dataclass(self):
|
||||
"""_get_valid_keys should handle non-dataclass via signature."""
|
||||
from praisonaiagents.config.param_resolver import _get_valid_keys
|
||||
|
||||
keys = _get_valid_keys(NonDataclassConfig)
|
||||
# Should return keys from __init__ signature or __annotations__
|
||||
# or gracefully return empty/None
|
||||
assert keys is None or isinstance(keys, (set, list, frozenset))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
pytest.main([__file__, "-v"])
|
||||
@@ -0,0 +1,720 @@
|
||||
"""
|
||||
TDD Tests for Resolver Standardization.
|
||||
|
||||
These tests verify:
|
||||
1. Precedence order: None → Instance → Config → Array → Dict → String → Bool → Default
|
||||
2. Memory array semantics: single=ok, multiple=error
|
||||
3. Caching array support with PRESET_OVERRIDE
|
||||
4. Workflow/WorkflowStep using canonical resolve
|
||||
5. CLI resolver integration
|
||||
6. Naming standardization (guardrails plural)
|
||||
7. All precedence levels per consolidated param
|
||||
|
||||
Run with: pytest tests/unit/config/test_resolver_standardization.py -v
|
||||
"""
|
||||
|
||||
import pytest
|
||||
from dataclasses import dataclass
|
||||
from typing import Any, Dict, List, Optional
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# Test Fixtures - Config Classes
|
||||
# =============================================================================
|
||||
|
||||
@dataclass
|
||||
class MockConfig:
|
||||
"""Mock config class for testing."""
|
||||
enabled: bool = True
|
||||
value: str = "default"
|
||||
count: int = 0
|
||||
|
||||
|
||||
@dataclass
|
||||
class MockMemoryConfig:
|
||||
"""Mock memory config for testing."""
|
||||
backend: str = "file"
|
||||
user_id: Optional[str] = None
|
||||
config: Optional[Dict[str, Any]] = None
|
||||
sources: Optional[List[str]] = None
|
||||
|
||||
|
||||
@dataclass
|
||||
class MockCachingConfig:
|
||||
"""Mock caching config for testing."""
|
||||
enabled: bool = True
|
||||
ttl: int = 3600
|
||||
prompt_caching: bool = False
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# TODO 1.1: Tests for Precedence Order (Array BEFORE Dict)
|
||||
# =============================================================================
|
||||
|
||||
class TestPrecedenceOrder:
|
||||
"""Test that precedence order is: None → Instance → Config → Array → Dict → String → Bool → Default"""
|
||||
|
||||
def test_array_takes_precedence_over_dict_interpretation(self):
|
||||
"""Array input should be processed as array, not as dict."""
|
||||
from praisonaiagents.config.param_resolver import resolve, ArrayMode
|
||||
|
||||
# When given an array, it should use array handling, not dict handling
|
||||
result = resolve(
|
||||
value=["preset1"],
|
||||
param_name="test",
|
||||
config_class=MockConfig,
|
||||
presets={"preset1": {"enabled": True, "value": "from_preset"}},
|
||||
array_mode=ArrayMode.PRESET_OVERRIDE,
|
||||
)
|
||||
|
||||
# Should resolve via array path (preset), not dict path
|
||||
assert isinstance(result, MockConfig)
|
||||
assert result.value == "from_preset"
|
||||
|
||||
def test_precedence_instance_over_config(self):
|
||||
"""Instance check should take precedence over config class check."""
|
||||
from praisonaiagents.config.param_resolver import resolve
|
||||
|
||||
class CustomInstance:
|
||||
def __init__(self):
|
||||
self.is_custom = True
|
||||
|
||||
custom = CustomInstance()
|
||||
|
||||
result = resolve(
|
||||
value=custom,
|
||||
param_name="test",
|
||||
config_class=MockConfig,
|
||||
instance_check=lambda v: hasattr(v, 'is_custom'),
|
||||
)
|
||||
|
||||
# Should return instance as-is, not try to convert
|
||||
assert result is custom
|
||||
assert result.is_custom is True
|
||||
|
||||
def test_precedence_config_over_array(self):
|
||||
"""Config class instance should take precedence over array handling."""
|
||||
from praisonaiagents.config.param_resolver import resolve, ArrayMode
|
||||
|
||||
config = MockConfig(enabled=False, value="config_value")
|
||||
|
||||
result = resolve(
|
||||
value=config,
|
||||
param_name="test",
|
||||
config_class=MockConfig,
|
||||
array_mode=ArrayMode.PRESET_OVERRIDE,
|
||||
)
|
||||
|
||||
# Should return config as-is
|
||||
assert result is config
|
||||
assert result.value == "config_value"
|
||||
|
||||
def test_precedence_array_over_dict(self):
|
||||
"""Array should be checked BEFORE dict in precedence."""
|
||||
from praisonaiagents.config.param_resolver import resolve, ArrayMode
|
||||
|
||||
# This test verifies the fix: Array must be checked before Dict
|
||||
# A list input should go through array handling, not dict handling
|
||||
result = resolve(
|
||||
value=["verbose", {"count": 5}],
|
||||
param_name="test",
|
||||
config_class=MockConfig,
|
||||
presets={"verbose": {"enabled": True, "value": "verbose_mode"}},
|
||||
array_mode=ArrayMode.PRESET_OVERRIDE,
|
||||
)
|
||||
|
||||
assert isinstance(result, MockConfig)
|
||||
assert result.value == "verbose_mode"
|
||||
assert result.count == 5 # Override applied
|
||||
|
||||
def test_precedence_dict_over_string(self):
|
||||
"""Dict should take precedence over string handling."""
|
||||
from praisonaiagents.config.param_resolver import resolve
|
||||
|
||||
result = resolve(
|
||||
value={"enabled": False, "value": "dict_value"},
|
||||
param_name="test",
|
||||
config_class=MockConfig,
|
||||
presets={"dict_value": {"enabled": True}}, # Should NOT use this
|
||||
)
|
||||
|
||||
assert isinstance(result, MockConfig)
|
||||
assert result.value == "dict_value"
|
||||
assert result.enabled is False
|
||||
|
||||
def test_precedence_string_over_bool(self):
|
||||
"""String preset should take precedence over bool handling."""
|
||||
from praisonaiagents.config.param_resolver import resolve
|
||||
|
||||
result = resolve(
|
||||
value="verbose",
|
||||
param_name="test",
|
||||
config_class=MockConfig,
|
||||
presets={"verbose": {"enabled": True, "value": "verbose_mode"}},
|
||||
)
|
||||
|
||||
assert isinstance(result, MockConfig)
|
||||
assert result.value == "verbose_mode"
|
||||
|
||||
def test_precedence_bool_over_default(self):
|
||||
"""Bool True should create default config, not return default value."""
|
||||
from praisonaiagents.config.param_resolver import resolve
|
||||
|
||||
result = resolve(
|
||||
value=True,
|
||||
param_name="test",
|
||||
config_class=MockConfig,
|
||||
default={"should": "not_use"},
|
||||
)
|
||||
|
||||
assert isinstance(result, MockConfig)
|
||||
# Should be default MockConfig(), not the default dict
|
||||
|
||||
def test_precedence_none_returns_default(self):
|
||||
"""None should return the default value."""
|
||||
from praisonaiagents.config.param_resolver import resolve
|
||||
|
||||
default_config = MockConfig(value="default_value")
|
||||
|
||||
result = resolve(
|
||||
value=None,
|
||||
param_name="test",
|
||||
config_class=MockConfig,
|
||||
default=default_config,
|
||||
)
|
||||
|
||||
assert result is default_config
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# TODO 1.2: Tests for Memory Array Semantics
|
||||
# =============================================================================
|
||||
|
||||
class TestMemoryArraySemantics:
|
||||
"""Test memory array semantics: single=ok, multiple=error."""
|
||||
|
||||
def test_memory_single_item_array_resolves_as_scalar(self):
|
||||
"""memory=["redis"] should resolve as scalar preset."""
|
||||
from praisonaiagents.config.param_resolver import resolve, ArrayMode
|
||||
|
||||
result = resolve(
|
||||
value=["redis"],
|
||||
param_name="memory",
|
||||
config_class=MockMemoryConfig,
|
||||
presets={"redis": {"backend": "redis"}},
|
||||
array_mode=ArrayMode.SINGLE_OR_LIST,
|
||||
)
|
||||
|
||||
assert isinstance(result, MockMemoryConfig)
|
||||
assert result.backend == "redis"
|
||||
|
||||
def test_memory_multiple_items_raises_error(self):
|
||||
"""memory=["redis", "file"] should raise TypeError."""
|
||||
from praisonaiagents.config.param_resolver import resolve, ArrayMode
|
||||
|
||||
with pytest.raises(TypeError) as exc_info:
|
||||
resolve(
|
||||
value=["redis", "file"],
|
||||
param_name="memory",
|
||||
config_class=MockMemoryConfig,
|
||||
presets={"redis": {"backend": "redis"}, "file": {"backend": "file"}},
|
||||
array_mode=ArrayMode.SINGLE_OR_LIST,
|
||||
)
|
||||
|
||||
# Error message should be helpful
|
||||
assert "memory" in str(exc_info.value).lower()
|
||||
assert "multiple" in str(exc_info.value).lower() or "single" in str(exc_info.value).lower()
|
||||
|
||||
def test_memory_single_url_in_array_resolves(self):
|
||||
"""memory=["postgresql://..."] should resolve as URL."""
|
||||
from praisonaiagents.config.param_resolver import resolve, ArrayMode
|
||||
|
||||
result = resolve(
|
||||
value=["postgresql://localhost/db"],
|
||||
param_name="memory",
|
||||
config_class=MockMemoryConfig,
|
||||
url_schemes={"postgresql": "postgres"},
|
||||
array_mode=ArrayMode.SINGLE_OR_LIST,
|
||||
)
|
||||
|
||||
assert isinstance(result, MockMemoryConfig)
|
||||
assert result.backend == "postgres"
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# TODO 1.3: Tests for Caching Array Support
|
||||
# =============================================================================
|
||||
|
||||
class TestCachingArraySupport:
|
||||
"""Test caching array support with PRESET_OVERRIDE."""
|
||||
|
||||
def test_caching_array_preset_override(self):
|
||||
"""caching=["enabled", {"ttl": 7200}] should work."""
|
||||
from praisonaiagents.config.param_resolver import resolve, ArrayMode
|
||||
|
||||
result = resolve(
|
||||
value=["enabled", {"ttl": 7200}],
|
||||
param_name="caching",
|
||||
config_class=MockCachingConfig,
|
||||
presets={"enabled": {"enabled": True, "ttl": 3600}},
|
||||
array_mode=ArrayMode.PRESET_OVERRIDE,
|
||||
)
|
||||
|
||||
assert isinstance(result, MockCachingConfig)
|
||||
assert result.enabled is True
|
||||
assert result.ttl == 7200 # Override applied
|
||||
|
||||
def test_caching_string_preset(self):
|
||||
"""caching="enabled" should work."""
|
||||
from praisonaiagents.config.param_resolver import resolve, ArrayMode
|
||||
|
||||
result = resolve(
|
||||
value="enabled",
|
||||
param_name="caching",
|
||||
config_class=MockCachingConfig,
|
||||
presets={"enabled": {"enabled": True}},
|
||||
array_mode=ArrayMode.PRESET_OVERRIDE,
|
||||
)
|
||||
|
||||
assert isinstance(result, MockCachingConfig)
|
||||
assert result.enabled is True
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# TODO 1.4: Tests for Workflow/WorkflowStep Using Canonical Resolve
|
||||
# =============================================================================
|
||||
|
||||
class TestWorkflowCanonicalResolve:
|
||||
"""Test that Workflow and WorkflowStep use canonical resolve directly."""
|
||||
|
||||
def test_workflow_output_uses_canonical_resolve(self):
|
||||
"""Workflow output param should use canonical resolve."""
|
||||
from praisonaiagents.workflows.workflows import Workflow
|
||||
|
||||
workflow = Workflow(
|
||||
name="test",
|
||||
steps=[],
|
||||
output="verbose",
|
||||
)
|
||||
|
||||
assert workflow._verbose is True
|
||||
|
||||
def test_workflow_output_dict_validation(self):
|
||||
"""Workflow output dict should have strict validation."""
|
||||
from praisonaiagents.workflows.workflows import Workflow
|
||||
|
||||
with pytest.raises(TypeError) as exc_info:
|
||||
Workflow(
|
||||
name="test",
|
||||
steps=[],
|
||||
output={"invalid_key": True},
|
||||
)
|
||||
|
||||
assert "invalid_key" in str(exc_info.value).lower() or "unknown" in str(exc_info.value).lower()
|
||||
|
||||
def test_workflowstep_context_uses_canonical_resolve(self):
|
||||
"""WorkflowStep context param should use canonical resolve."""
|
||||
from praisonaiagents.workflows.workflows import WorkflowStep
|
||||
|
||||
step = WorkflowStep(
|
||||
name="test",
|
||||
context=["step1", "step2"],
|
||||
)
|
||||
|
||||
assert step._context_from == ["step1", "step2"]
|
||||
|
||||
def test_workflowstep_execution_preset(self):
|
||||
"""WorkflowStep execution preset should work."""
|
||||
from praisonaiagents.workflows.workflows import WorkflowStep
|
||||
|
||||
step = WorkflowStep(
|
||||
name="test",
|
||||
execution="thorough",
|
||||
)
|
||||
|
||||
# Should have thorough preset values
|
||||
assert step._max_retries >= 3 # Thorough has more retries
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# TODO 1.5: Tests for CLI Resolver Integration
|
||||
# =============================================================================
|
||||
|
||||
class TestCLIResolverIntegration:
|
||||
"""Test CLI flags use canonical resolver."""
|
||||
|
||||
def test_cli_memory_flag_true(self):
|
||||
"""CLI --memory flag (true) should resolve correctly."""
|
||||
# This tests the integration point - CLI should pass value to canonical resolver
|
||||
from praisonaiagents.config.param_resolver import resolve, ArrayMode
|
||||
from praisonaiagents.config.presets import MEMORY_PRESETS, MEMORY_URL_SCHEMES
|
||||
|
||||
# Simulate CLI passing True
|
||||
result = resolve(
|
||||
value=True,
|
||||
param_name="memory",
|
||||
config_class=MockMemoryConfig,
|
||||
presets=MEMORY_PRESETS,
|
||||
url_schemes=MEMORY_URL_SCHEMES,
|
||||
array_mode=ArrayMode.SINGLE_OR_LIST,
|
||||
)
|
||||
|
||||
assert isinstance(result, MockMemoryConfig)
|
||||
|
||||
def test_cli_memory_flag_preset_string(self):
|
||||
"""CLI --memory=redis should resolve as preset."""
|
||||
from praisonaiagents.config.param_resolver import resolve, ArrayMode
|
||||
from praisonaiagents.config.presets import MEMORY_PRESETS, MEMORY_URL_SCHEMES
|
||||
|
||||
result = resolve(
|
||||
value="redis",
|
||||
param_name="memory",
|
||||
config_class=MockMemoryConfig,
|
||||
presets=MEMORY_PRESETS,
|
||||
url_schemes=MEMORY_URL_SCHEMES,
|
||||
array_mode=ArrayMode.SINGLE_OR_LIST,
|
||||
)
|
||||
|
||||
assert isinstance(result, MockMemoryConfig)
|
||||
assert result.backend == "redis"
|
||||
|
||||
def test_cli_memory_flag_url(self):
|
||||
"""CLI --memory=postgresql://... should resolve as URL."""
|
||||
from praisonaiagents.config.param_resolver import resolve, ArrayMode
|
||||
from praisonaiagents.config.presets import MEMORY_PRESETS, MEMORY_URL_SCHEMES
|
||||
|
||||
result = resolve(
|
||||
value="postgresql://localhost/db",
|
||||
param_name="memory",
|
||||
config_class=MockMemoryConfig,
|
||||
presets=MEMORY_PRESETS,
|
||||
url_schemes=MEMORY_URL_SCHEMES,
|
||||
array_mode=ArrayMode.SINGLE_OR_LIST,
|
||||
)
|
||||
|
||||
assert isinstance(result, MockMemoryConfig)
|
||||
assert result.backend == "postgres"
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# TODO 1.6: Tests for Naming Standardization (guardrails plural)
|
||||
# =============================================================================
|
||||
|
||||
class TestNamingStandardization:
|
||||
"""Test guardrail → guardrails naming standardization."""
|
||||
|
||||
def test_workflowstep_accepts_guardrails_plural(self):
|
||||
"""WorkflowStep should accept guardrails= (plural)."""
|
||||
from praisonaiagents.workflows.workflows import WorkflowStep
|
||||
|
||||
def my_guardrail(result):
|
||||
return (True, result)
|
||||
|
||||
# Should accept guardrails= (plural)
|
||||
step = WorkflowStep(
|
||||
name="test",
|
||||
guardrails=my_guardrail,
|
||||
)
|
||||
|
||||
# Should store in guardrails attribute
|
||||
assert step.guardrails is my_guardrail or step.guardrail is my_guardrail
|
||||
|
||||
def test_workflowstep_guardrail_singular_compat(self):
|
||||
"""WorkflowStep should accept guardrail= (singular) for compatibility."""
|
||||
from praisonaiagents.workflows.workflows import WorkflowStep
|
||||
|
||||
def my_guardrail(result):
|
||||
return (True, result)
|
||||
|
||||
# Should still accept guardrail= (singular) for backward compat
|
||||
step = WorkflowStep(
|
||||
name="test",
|
||||
guardrail=my_guardrail,
|
||||
)
|
||||
|
||||
# Should work (backward compat)
|
||||
assert step.guardrail is my_guardrail
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# TODO 1.7: Tests for All Precedence Levels Per Consolidated Param
|
||||
# =============================================================================
|
||||
|
||||
class TestAllPrecedenceLevels:
|
||||
"""Test each consolidated param has all 7 precedence levels working."""
|
||||
|
||||
def test_output_all_precedence_levels(self):
|
||||
"""Test output param at all precedence levels."""
|
||||
from praisonaiagents.config.param_resolver import resolve, ArrayMode
|
||||
from praisonaiagents.config.feature_configs import OutputConfig
|
||||
from praisonaiagents.config.presets import OUTPUT_PRESETS
|
||||
|
||||
# 1. None → Default
|
||||
result = resolve(value=None, param_name="output", config_class=OutputConfig,
|
||||
presets=OUTPUT_PRESETS, default=OutputConfig())
|
||||
assert isinstance(result, OutputConfig)
|
||||
|
||||
# 2. Instance (custom object with expected attributes)
|
||||
class CustomOutput:
|
||||
verbose = True
|
||||
stream = True
|
||||
custom = CustomOutput()
|
||||
result = resolve(value=custom, param_name="output", config_class=OutputConfig,
|
||||
presets=OUTPUT_PRESETS, instance_check=lambda v: hasattr(v, 'verbose'))
|
||||
assert result is custom
|
||||
|
||||
# 3. Config class instance
|
||||
config = OutputConfig(verbose=True, stream=False)
|
||||
result = resolve(value=config, param_name="output", config_class=OutputConfig,
|
||||
presets=OUTPUT_PRESETS)
|
||||
assert result is config
|
||||
|
||||
# 4. Array [preset, overrides]
|
||||
result = resolve(value=["verbose", {"stream": False}], param_name="output",
|
||||
config_class=OutputConfig, presets=OUTPUT_PRESETS,
|
||||
array_mode=ArrayMode.PRESET_OVERRIDE)
|
||||
assert isinstance(result, OutputConfig)
|
||||
|
||||
# 5. Dict
|
||||
result = resolve(value={"verbose": True, "stream": False}, param_name="output",
|
||||
config_class=OutputConfig, presets=OUTPUT_PRESETS)
|
||||
assert isinstance(result, OutputConfig)
|
||||
assert result.verbose is True
|
||||
|
||||
# 6. String preset
|
||||
result = resolve(value="verbose", param_name="output", config_class=OutputConfig,
|
||||
presets=OUTPUT_PRESETS)
|
||||
assert isinstance(result, OutputConfig)
|
||||
|
||||
# 7. Bool True
|
||||
result = resolve(value=True, param_name="output", config_class=OutputConfig,
|
||||
presets=OUTPUT_PRESETS)
|
||||
assert isinstance(result, OutputConfig)
|
||||
|
||||
# 8. Bool False → None
|
||||
result = resolve(value=False, param_name="output", config_class=OutputConfig,
|
||||
presets=OUTPUT_PRESETS)
|
||||
assert result is None
|
||||
|
||||
def test_memory_all_precedence_levels(self):
|
||||
"""Test memory param at all precedence levels."""
|
||||
from praisonaiagents.config.param_resolver import resolve, ArrayMode
|
||||
from praisonaiagents.config.feature_configs import MemoryConfig
|
||||
from praisonaiagents.config.presets import MEMORY_PRESETS, MEMORY_URL_SCHEMES
|
||||
|
||||
# 1. None → Default
|
||||
result = resolve(value=None, param_name="memory", config_class=MemoryConfig,
|
||||
presets=MEMORY_PRESETS, url_schemes=MEMORY_URL_SCHEMES,
|
||||
array_mode=ArrayMode.SINGLE_OR_LIST, default=None)
|
||||
assert result is None
|
||||
|
||||
# 2. Config class instance
|
||||
config = MemoryConfig(backend="redis")
|
||||
result = resolve(value=config, param_name="memory", config_class=MemoryConfig,
|
||||
presets=MEMORY_PRESETS, url_schemes=MEMORY_URL_SCHEMES,
|
||||
array_mode=ArrayMode.SINGLE_OR_LIST)
|
||||
assert result is config
|
||||
|
||||
# 3. Dict
|
||||
result = resolve(value={"backend": "sqlite"}, param_name="memory",
|
||||
config_class=MemoryConfig, presets=MEMORY_PRESETS,
|
||||
url_schemes=MEMORY_URL_SCHEMES, array_mode=ArrayMode.SINGLE_OR_LIST)
|
||||
assert isinstance(result, MemoryConfig)
|
||||
assert result.backend == "sqlite"
|
||||
|
||||
# 4. String preset
|
||||
result = resolve(value="redis", param_name="memory", config_class=MemoryConfig,
|
||||
presets=MEMORY_PRESETS, url_schemes=MEMORY_URL_SCHEMES,
|
||||
array_mode=ArrayMode.SINGLE_OR_LIST)
|
||||
assert isinstance(result, MemoryConfig)
|
||||
|
||||
# 5. String URL
|
||||
result = resolve(value="postgresql://localhost/db", param_name="memory",
|
||||
config_class=MemoryConfig, presets=MEMORY_PRESETS,
|
||||
url_schemes=MEMORY_URL_SCHEMES, array_mode=ArrayMode.SINGLE_OR_LIST)
|
||||
assert isinstance(result, MemoryConfig)
|
||||
|
||||
# 6. Bool True
|
||||
result = resolve(value=True, param_name="memory", config_class=MemoryConfig,
|
||||
presets=MEMORY_PRESETS, url_schemes=MEMORY_URL_SCHEMES,
|
||||
array_mode=ArrayMode.SINGLE_OR_LIST)
|
||||
assert isinstance(result, MemoryConfig)
|
||||
|
||||
# 7. Bool False → None
|
||||
result = resolve(value=False, param_name="memory", config_class=MemoryConfig,
|
||||
presets=MEMORY_PRESETS, url_schemes=MEMORY_URL_SCHEMES,
|
||||
array_mode=ArrayMode.SINGLE_OR_LIST)
|
||||
assert result is None
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# Tests for Error Messages and Typo Suggestions
|
||||
# =============================================================================
|
||||
|
||||
class TestErrorMessages:
|
||||
"""Test error messages are helpful with typo suggestions."""
|
||||
|
||||
def test_invalid_preset_suggests_similar(self):
|
||||
"""Invalid preset should suggest similar valid presets."""
|
||||
from praisonaiagents.config.param_resolver import resolve
|
||||
|
||||
with pytest.raises(ValueError) as exc_info:
|
||||
resolve(
|
||||
value="verbos", # Typo
|
||||
param_name="output",
|
||||
config_class=MockConfig,
|
||||
presets={"verbose": {}, "minimal": {}, "debug": {}},
|
||||
)
|
||||
|
||||
error_msg = str(exc_info.value).lower()
|
||||
assert "verbose" in error_msg # Should suggest "verbose"
|
||||
|
||||
def test_unknown_dict_keys_lists_valid_keys(self):
|
||||
"""Unknown dict keys should list valid keys."""
|
||||
from praisonaiagents.config.param_resolver import resolve
|
||||
|
||||
with pytest.raises(TypeError) as exc_info:
|
||||
resolve(
|
||||
value={"invalid_key": True},
|
||||
param_name="test",
|
||||
config_class=MockConfig,
|
||||
)
|
||||
|
||||
error_msg = str(exc_info.value).lower()
|
||||
assert "invalid_key" in error_msg or "unknown" in error_msg
|
||||
assert "enabled" in error_msg or "valid" in error_msg
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# Tests for Strict Dict Validation
|
||||
# =============================================================================
|
||||
|
||||
class TestStrictDictValidation:
|
||||
"""Test strict dict validation."""
|
||||
|
||||
def test_dict_without_config_class_raises_error(self):
|
||||
"""Dict input without config_class should raise TypeError."""
|
||||
from praisonaiagents.config.param_resolver import resolve
|
||||
|
||||
with pytest.raises(TypeError) as exc_info:
|
||||
resolve(
|
||||
value={"key": "value"},
|
||||
param_name="test",
|
||||
config_class=None, # No config class
|
||||
)
|
||||
|
||||
assert "dict" in str(exc_info.value).lower()
|
||||
|
||||
def test_dict_with_unknown_keys_raises_error(self):
|
||||
"""Dict with unknown keys should raise TypeError."""
|
||||
from praisonaiagents.config.param_resolver import resolve
|
||||
|
||||
with pytest.raises(TypeError) as exc_info:
|
||||
resolve(
|
||||
value={"enabled": True, "unknown_field": "value"},
|
||||
param_name="test",
|
||||
config_class=MockConfig,
|
||||
)
|
||||
|
||||
error_msg = str(exc_info.value)
|
||||
assert "unknown_field" in error_msg.lower() or "unknown" in error_msg.lower()
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# PHASE 1 TDD: Tests for Workflow/WorkflowStep New Params (ALL GREEN target)
|
||||
# =============================================================================
|
||||
|
||||
class TestWorkflowNewParams:
|
||||
"""Test Workflow supports all consolidated params like Agents."""
|
||||
|
||||
def test_workflow_has_autonomy_param(self):
|
||||
"""Workflow should have autonomy param."""
|
||||
from praisonaiagents.workflows.workflows import Workflow
|
||||
|
||||
# Should accept autonomy param without error
|
||||
workflow = Workflow(steps=[], autonomy=True)
|
||||
assert hasattr(workflow, 'autonomy') or hasattr(workflow, '_autonomy_config')
|
||||
|
||||
def test_workflow_has_knowledge_param(self):
|
||||
"""Workflow should have knowledge param."""
|
||||
from praisonaiagents.workflows.workflows import Workflow
|
||||
|
||||
workflow = Workflow(steps=[], knowledge=["docs/"])
|
||||
assert hasattr(workflow, 'knowledge') or hasattr(workflow, '_knowledge_config')
|
||||
|
||||
def test_workflow_has_guardrails_param(self):
|
||||
"""Workflow should have guardrails param."""
|
||||
from praisonaiagents.workflows.workflows import Workflow
|
||||
|
||||
workflow = Workflow(steps=[], guardrails=True)
|
||||
assert hasattr(workflow, 'guardrails') or hasattr(workflow, '_guardrails_config')
|
||||
|
||||
def test_workflow_has_web_param(self):
|
||||
"""Workflow should have web param."""
|
||||
from praisonaiagents.workflows.workflows import Workflow
|
||||
|
||||
workflow = Workflow(steps=[], web=True)
|
||||
assert hasattr(workflow, 'web') or hasattr(workflow, '_web_config')
|
||||
|
||||
def test_workflow_has_reflection_param(self):
|
||||
"""Workflow should have reflection param."""
|
||||
from praisonaiagents.workflows.workflows import Workflow
|
||||
|
||||
workflow = Workflow(steps=[], reflection=True)
|
||||
assert hasattr(workflow, 'reflection') or hasattr(workflow, '_reflection_config')
|
||||
|
||||
|
||||
class TestWorkflowStepNewParams:
|
||||
"""Test WorkflowStep supports all consolidated params."""
|
||||
|
||||
def test_workflowstep_has_autonomy_param(self):
|
||||
"""WorkflowStep should have autonomy param."""
|
||||
from praisonaiagents.workflows.workflows import WorkflowStep
|
||||
|
||||
step = WorkflowStep(name="test", autonomy=True)
|
||||
assert hasattr(step, 'autonomy') or hasattr(step, '_autonomy_config')
|
||||
|
||||
def test_workflowstep_has_knowledge_param(self):
|
||||
"""WorkflowStep should have knowledge param."""
|
||||
from praisonaiagents.workflows.workflows import WorkflowStep
|
||||
|
||||
step = WorkflowStep(name="test", knowledge=["docs/"])
|
||||
assert hasattr(step, 'knowledge') or hasattr(step, '_knowledge_config')
|
||||
|
||||
def test_workflowstep_has_web_param(self):
|
||||
"""WorkflowStep should have web param."""
|
||||
from praisonaiagents.workflows.workflows import WorkflowStep
|
||||
|
||||
step = WorkflowStep(name="test", web=True)
|
||||
assert hasattr(step, 'web') or hasattr(step, '_web_config')
|
||||
|
||||
def test_workflowstep_has_reflection_param(self):
|
||||
"""WorkflowStep should have reflection param."""
|
||||
from praisonaiagents.workflows.workflows import WorkflowStep
|
||||
|
||||
step = WorkflowStep(name="test", reflection=True)
|
||||
assert hasattr(step, 'reflection') or hasattr(step, '_reflection_config')
|
||||
|
||||
|
||||
class TestWorkflowPropagation:
|
||||
"""Test workflow defaults propagate to steps/agents."""
|
||||
|
||||
def test_workflow_defaults_available_for_steps(self):
|
||||
"""Workflow-level configs should be accessible for step propagation."""
|
||||
from praisonaiagents.workflows.workflows import Workflow
|
||||
|
||||
workflow = Workflow(
|
||||
steps=[],
|
||||
knowledge=["docs/"],
|
||||
web=True,
|
||||
guardrails=True,
|
||||
)
|
||||
|
||||
# Workflow should store resolved configs for propagation
|
||||
assert workflow.knowledge is not None or hasattr(workflow, '_knowledge_config')
|
||||
assert workflow.web is not None or hasattr(workflow, '_web_config')
|
||||
assert workflow.guardrails is not None or hasattr(workflow, '_guardrails_config')
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
pytest.main([__file__, "-v"])
|
||||
@@ -0,0 +1,423 @@
|
||||
"""
|
||||
Unit tests for Dict support in the canonical param resolver.
|
||||
|
||||
Tests the NEW_FEATURE_SPEC:
|
||||
1. dict → config success
|
||||
2. dict with unknown key → error
|
||||
3. dict where no config class → error
|
||||
|
||||
Also tests:
|
||||
- Precedence ordering (Instance > Config > Dict > Array > String > Bool > Default)
|
||||
- Array semantics unchanged
|
||||
"""
|
||||
|
||||
import pytest
|
||||
from dataclasses import dataclass
|
||||
from typing import Optional, List
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# Test Config Classes (mimicking real config classes)
|
||||
# =============================================================================
|
||||
|
||||
@dataclass
|
||||
class TestOutputConfig:
|
||||
"""Test config for output parameter."""
|
||||
verbose: bool = False
|
||||
stream: bool = True
|
||||
markdown: bool = False
|
||||
|
||||
|
||||
@dataclass
|
||||
class TestExecutionConfig:
|
||||
"""Test config for execution parameter."""
|
||||
max_iter: int = 10
|
||||
max_retries: int = 3
|
||||
|
||||
|
||||
@dataclass
|
||||
class TestMemoryConfig:
|
||||
"""Test config for memory parameter."""
|
||||
backend: str = "file"
|
||||
user_id: Optional[str] = None
|
||||
|
||||
|
||||
@dataclass
|
||||
class TestContextConfig:
|
||||
"""Test config for context parameter with list field."""
|
||||
from_steps: Optional[List[str]] = None
|
||||
retain_full: bool = True
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# Test Fixtures
|
||||
# =============================================================================
|
||||
|
||||
@pytest.fixture
|
||||
def resolver():
|
||||
"""Import the canonical resolver."""
|
||||
from praisonaiagents.config.param_resolver import resolve, ArrayMode
|
||||
return resolve, ArrayMode
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def output_presets():
|
||||
"""Sample presets for output parameter."""
|
||||
return {
|
||||
"verbose": {"verbose": True, "stream": True, "markdown": True},
|
||||
"silent": {"verbose": False, "stream": False, "markdown": False},
|
||||
"minimal": {"verbose": False, "stream": True, "markdown": False},
|
||||
}
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# DICT SUCCESS TESTS
|
||||
# =============================================================================
|
||||
|
||||
class TestDictSuccess:
|
||||
"""Test dict → config conversion success cases."""
|
||||
|
||||
def test_dict_to_config_basic(self, resolver):
|
||||
"""Dict with valid keys converts to config."""
|
||||
resolve, _ = resolver
|
||||
result = resolve(
|
||||
value={"verbose": True, "stream": False},
|
||||
param_name="output",
|
||||
config_class=TestOutputConfig,
|
||||
)
|
||||
assert isinstance(result, TestOutputConfig)
|
||||
assert result.verbose is True
|
||||
assert result.stream is False
|
||||
assert result.markdown is False # default
|
||||
|
||||
def test_dict_to_config_all_fields(self, resolver):
|
||||
"""Dict with all fields converts correctly."""
|
||||
resolve, _ = resolver
|
||||
result = resolve(
|
||||
value={"verbose": True, "stream": True, "markdown": True},
|
||||
param_name="output",
|
||||
config_class=TestOutputConfig,
|
||||
)
|
||||
assert isinstance(result, TestOutputConfig)
|
||||
assert result.verbose is True
|
||||
assert result.stream is True
|
||||
assert result.markdown is True
|
||||
|
||||
def test_dict_to_config_partial_fields(self, resolver):
|
||||
"""Dict with partial fields uses defaults for missing."""
|
||||
resolve, _ = resolver
|
||||
result = resolve(
|
||||
value={"max_iter": 50},
|
||||
param_name="execution",
|
||||
config_class=TestExecutionConfig,
|
||||
)
|
||||
assert isinstance(result, TestExecutionConfig)
|
||||
assert result.max_iter == 50
|
||||
assert result.max_retries == 3 # default
|
||||
|
||||
def test_dict_to_config_empty_dict(self, resolver):
|
||||
"""Empty dict creates config with all defaults."""
|
||||
resolve, _ = resolver
|
||||
result = resolve(
|
||||
value={},
|
||||
param_name="output",
|
||||
config_class=TestOutputConfig,
|
||||
)
|
||||
assert isinstance(result, TestOutputConfig)
|
||||
assert result.verbose is False
|
||||
assert result.stream is True
|
||||
assert result.markdown is False
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# DICT ERROR TESTS - Unknown Keys
|
||||
# =============================================================================
|
||||
|
||||
class TestDictUnknownKeyError:
|
||||
"""Test dict with unknown key raises helpful error."""
|
||||
|
||||
def test_unknown_key_raises_typeerror(self, resolver):
|
||||
"""Unknown key in dict raises TypeError with helpful message."""
|
||||
resolve, _ = resolver
|
||||
with pytest.raises(TypeError) as exc_info:
|
||||
resolve(
|
||||
value={"verbose": True, "unknown_key": "value"},
|
||||
param_name="output",
|
||||
config_class=TestOutputConfig,
|
||||
)
|
||||
error_msg = str(exc_info.value)
|
||||
assert "Unknown keys for output" in error_msg
|
||||
assert "unknown_key" in error_msg
|
||||
assert "Valid keys" in error_msg
|
||||
|
||||
def test_multiple_unknown_keys(self, resolver):
|
||||
"""Multiple unknown keys are all reported."""
|
||||
resolve, _ = resolver
|
||||
with pytest.raises(TypeError) as exc_info:
|
||||
resolve(
|
||||
value={"foo": 1, "bar": 2, "verbose": True},
|
||||
param_name="output",
|
||||
config_class=TestOutputConfig,
|
||||
)
|
||||
error_msg = str(exc_info.value)
|
||||
assert "foo" in error_msg or "bar" in error_msg
|
||||
|
||||
def test_typo_in_key_raises_error(self, resolver):
|
||||
"""Typo in key name raises error with valid keys listed."""
|
||||
resolve, _ = resolver
|
||||
with pytest.raises(TypeError) as exc_info:
|
||||
resolve(
|
||||
value={"verbos": True}, # typo: verbos instead of verbose
|
||||
param_name="output",
|
||||
config_class=TestOutputConfig,
|
||||
)
|
||||
error_msg = str(exc_info.value)
|
||||
assert "verbos" in error_msg
|
||||
assert "verbose" in error_msg # valid key should be shown
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# DICT ERROR TESTS - No Config Class
|
||||
# =============================================================================
|
||||
|
||||
class TestDictNoConfigClassError:
|
||||
"""Test dict where no config class is defined raises error."""
|
||||
|
||||
def test_dict_without_config_class_raises_typeerror(self, resolver):
|
||||
"""Dict input without config_class raises TypeError."""
|
||||
resolve, _ = resolver
|
||||
with pytest.raises(TypeError) as exc_info:
|
||||
resolve(
|
||||
value={"some": "dict"},
|
||||
param_name="custom_param",
|
||||
config_class=None,
|
||||
)
|
||||
error_msg = str(exc_info.value)
|
||||
assert "Dict input not supported" in error_msg
|
||||
assert "custom_param" in error_msg
|
||||
assert "no config class" in error_msg
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# PRECEDENCE TESTS
|
||||
# =============================================================================
|
||||
|
||||
class TestPrecedence:
|
||||
"""Test precedence ordering: Instance > Config > Dict > Array > String > Bool > Default."""
|
||||
|
||||
def test_instance_beats_dict(self, resolver):
|
||||
"""Instance check takes precedence over dict."""
|
||||
resolve, _ = resolver
|
||||
|
||||
class CustomInstance:
|
||||
def __init__(self):
|
||||
self.custom = True
|
||||
|
||||
instance = CustomInstance()
|
||||
result = resolve(
|
||||
value=instance,
|
||||
param_name="test",
|
||||
config_class=TestOutputConfig,
|
||||
instance_check=lambda v: isinstance(v, CustomInstance),
|
||||
)
|
||||
assert result is instance
|
||||
|
||||
def test_config_instance_beats_dict(self, resolver):
|
||||
"""Config instance takes precedence over dict conversion."""
|
||||
resolve, _ = resolver
|
||||
config = TestOutputConfig(verbose=True, stream=False)
|
||||
result = resolve(
|
||||
value=config,
|
||||
param_name="output",
|
||||
config_class=TestOutputConfig,
|
||||
)
|
||||
assert result is config
|
||||
|
||||
def test_dict_beats_string(self, resolver, output_presets):
|
||||
"""Dict takes precedence over string preset lookup."""
|
||||
resolve, _ = resolver
|
||||
# Dict should be processed, not treated as string
|
||||
result = resolve(
|
||||
value={"verbose": False, "stream": False},
|
||||
param_name="output",
|
||||
config_class=TestOutputConfig,
|
||||
presets=output_presets,
|
||||
)
|
||||
assert isinstance(result, TestOutputConfig)
|
||||
assert result.verbose is False
|
||||
assert result.stream is False
|
||||
|
||||
def test_dict_beats_bool(self, resolver):
|
||||
"""Dict takes precedence over bool handling."""
|
||||
resolve, _ = resolver
|
||||
# Dict should be processed, not treated as bool
|
||||
result = resolve(
|
||||
value={"verbose": True},
|
||||
param_name="output",
|
||||
config_class=TestOutputConfig,
|
||||
)
|
||||
assert isinstance(result, TestOutputConfig)
|
||||
assert result.verbose is True
|
||||
|
||||
def test_bool_true_creates_default_config(self, resolver):
|
||||
"""Bool True creates default config instance."""
|
||||
resolve, _ = resolver
|
||||
result = resolve(
|
||||
value=True,
|
||||
param_name="output",
|
||||
config_class=TestOutputConfig,
|
||||
)
|
||||
assert isinstance(result, TestOutputConfig)
|
||||
assert result.verbose is False # default
|
||||
|
||||
def test_bool_false_returns_none(self, resolver):
|
||||
"""Bool False returns None (disabled)."""
|
||||
resolve, _ = resolver
|
||||
result = resolve(
|
||||
value=False,
|
||||
param_name="output",
|
||||
config_class=TestOutputConfig,
|
||||
)
|
||||
assert result is None
|
||||
|
||||
def test_none_returns_default(self, resolver):
|
||||
"""None returns the default value."""
|
||||
resolve, _ = resolver
|
||||
default = TestOutputConfig(verbose=True)
|
||||
result = resolve(
|
||||
value=None,
|
||||
param_name="output",
|
||||
config_class=TestOutputConfig,
|
||||
default=default,
|
||||
)
|
||||
assert result is default
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# ARRAY SEMANTICS UNCHANGED TESTS
|
||||
# =============================================================================
|
||||
|
||||
class TestArraySemanticsUnchanged:
|
||||
"""Test that array semantics remain unchanged."""
|
||||
|
||||
def test_preset_override_array(self, resolver, output_presets):
|
||||
"""[preset, {overrides}] pattern still works."""
|
||||
resolve, ArrayMode = resolver
|
||||
result = resolve(
|
||||
value=["verbose", {"stream": False}],
|
||||
param_name="output",
|
||||
config_class=TestOutputConfig,
|
||||
presets=output_presets,
|
||||
array_mode=ArrayMode.PRESET_OVERRIDE,
|
||||
)
|
||||
assert isinstance(result, TestOutputConfig)
|
||||
assert result.verbose is True # from preset
|
||||
assert result.stream is False # overridden
|
||||
|
||||
def test_step_names_array(self, resolver):
|
||||
"""List of step names creates config with from_steps."""
|
||||
resolve, ArrayMode = resolver
|
||||
result = resolve(
|
||||
value=["step1", "step2"],
|
||||
param_name="context",
|
||||
config_class=TestContextConfig,
|
||||
array_mode=ArrayMode.STEP_NAMES,
|
||||
)
|
||||
assert isinstance(result, TestContextConfig)
|
||||
assert result.from_steps == ["step1", "step2"]
|
||||
|
||||
def test_passthrough_array(self, resolver):
|
||||
"""Passthrough mode returns list as-is."""
|
||||
resolve, ArrayMode = resolver
|
||||
input_list = ["a", "b", "c"]
|
||||
result = resolve(
|
||||
value=input_list,
|
||||
param_name="hooks",
|
||||
config_class=None,
|
||||
array_mode=ArrayMode.PASSTHROUGH,
|
||||
)
|
||||
assert result == input_list
|
||||
|
||||
def test_empty_array_returns_none(self, resolver):
|
||||
"""Empty array returns None (disabled)."""
|
||||
resolve, ArrayMode = resolver
|
||||
result = resolve(
|
||||
value=[],
|
||||
param_name="output",
|
||||
config_class=TestOutputConfig,
|
||||
array_mode=ArrayMode.PRESET_OVERRIDE,
|
||||
)
|
||||
assert result is None
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# HELPER FUNCTION TESTS
|
||||
# =============================================================================
|
||||
|
||||
class TestHelperFunctions:
|
||||
"""Test helper functions for dict validation."""
|
||||
|
||||
def test_get_config_fields(self):
|
||||
"""_get_config_fields returns field names."""
|
||||
from praisonaiagents.config.param_resolver import _get_config_fields
|
||||
fields = _get_config_fields(TestOutputConfig)
|
||||
assert "verbose" in fields
|
||||
assert "stream" in fields
|
||||
assert "markdown" in fields
|
||||
|
||||
def test_validate_dict_keys_valid(self):
|
||||
"""_validate_dict_keys returns empty list for valid keys."""
|
||||
from praisonaiagents.config.param_resolver import _validate_dict_keys
|
||||
unknown = _validate_dict_keys(
|
||||
{"verbose": True, "stream": False},
|
||||
TestOutputConfig
|
||||
)
|
||||
assert unknown == []
|
||||
|
||||
def test_validate_dict_keys_invalid(self):
|
||||
"""_validate_dict_keys returns unknown keys."""
|
||||
from praisonaiagents.config.param_resolver import _validate_dict_keys
|
||||
unknown = _validate_dict_keys(
|
||||
{"verbose": True, "bad_key": "value"},
|
||||
TestOutputConfig
|
||||
)
|
||||
assert "bad_key" in unknown
|
||||
|
||||
def test_get_example_dict(self):
|
||||
"""_get_example_dict returns example snippet."""
|
||||
from praisonaiagents.config.param_resolver import _get_example_dict
|
||||
example = _get_example_dict(TestOutputConfig)
|
||||
assert "verbose" in example or "stream" in example
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# STRING PRESET TESTS (regression)
|
||||
# =============================================================================
|
||||
|
||||
class TestStringPresets:
|
||||
"""Test string preset resolution still works."""
|
||||
|
||||
def test_string_preset_resolves(self, resolver, output_presets):
|
||||
"""String preset name resolves to config."""
|
||||
resolve, _ = resolver
|
||||
result = resolve(
|
||||
value="verbose",
|
||||
param_name="output",
|
||||
config_class=TestOutputConfig,
|
||||
presets=output_presets,
|
||||
)
|
||||
assert isinstance(result, TestOutputConfig)
|
||||
assert result.verbose is True
|
||||
|
||||
def test_invalid_preset_raises_error(self, resolver, output_presets):
|
||||
"""Invalid preset name raises helpful error."""
|
||||
resolve, _ = resolver
|
||||
with pytest.raises(ValueError) as exc_info:
|
||||
resolve(
|
||||
value="invalid_preset",
|
||||
param_name="output",
|
||||
config_class=TestOutputConfig,
|
||||
presets=output_presets,
|
||||
)
|
||||
error_msg = str(exc_info.value)
|
||||
assert "invalid_preset" in error_msg
|
||||
Generated
+1
-1
@@ -2422,7 +2422,7 @@ wheels = [
|
||||
|
||||
[[package]]
|
||||
name = "praisonaiagents"
|
||||
version = "0.10.6"
|
||||
version = "0.10.10"
|
||||
source = { editable = "." }
|
||||
dependencies = [
|
||||
{ name = "aiohttp" },
|
||||
|
||||
@@ -1,2 +1,3 @@
|
||||
chainlit.md
|
||||
tests/fixtures
|
||||
tests/fixtures
|
||||
reports
|
||||
@@ -3,8 +3,8 @@ class Praisonai < Formula
|
||||
|
||||
desc "AI tools for various AI applications"
|
||||
homepage "https://github.com/MervinPraison/PraisonAI"
|
||||
url "https://github.com/MervinPraison/PraisonAI/archive/refs/tags/v3.5.7.tar.gz"
|
||||
sha256 `curl -sL https://github.com/MervinPraison/PraisonAI/archive/refs/tags/v3.5.7.tar.gz | shasum -a 256`.split.first
|
||||
url "https://github.com/MervinPraison/PraisonAI/archive/refs/tags/v3.6.2.tar.gz"
|
||||
sha256 `curl -sL https://github.com/MervinPraison/PraisonAI/archive/refs/tags/v3.6.2.tar.gz | shasum -a 256`.split.first
|
||||
license "MIT"
|
||||
|
||||
depends_on "[email protected]"
|
||||
|
||||
@@ -5,20 +5,60 @@ Provides terminal-native interactive chat mode.
|
||||
This command NEVER opens a browser - it runs entirely in the terminal.
|
||||
"""
|
||||
|
||||
from typing import List, Optional
|
||||
from typing import List, Optional, Union
|
||||
|
||||
import typer
|
||||
|
||||
app = typer.Typer(help="Terminal-native interactive chat mode")
|
||||
|
||||
|
||||
def _parse_memory_flag(memory: Optional[str], no_memory: bool) -> Union[bool, str, None]:
|
||||
"""
|
||||
Parse memory CLI flag to value for Agent.
|
||||
|
||||
Precedence: --no-memory > --memory=value > --memory (flag) > None
|
||||
|
||||
Args:
|
||||
memory: Memory flag value (None, "true", preset string, or URL)
|
||||
no_memory: Whether --no-memory was specified
|
||||
|
||||
Returns:
|
||||
- False if --no-memory
|
||||
- True if --memory (flag only)
|
||||
- str if --memory=preset or --memory=URL
|
||||
- None if neither specified
|
||||
"""
|
||||
if no_memory:
|
||||
return False
|
||||
|
||||
if memory is None:
|
||||
return None
|
||||
|
||||
# --memory flag without value sets "true"
|
||||
if memory.lower() == "true":
|
||||
return True
|
||||
|
||||
# --memory=false explicitly disables
|
||||
if memory.lower() == "false":
|
||||
return False
|
||||
|
||||
# Otherwise it's a preset or URL string
|
||||
return memory
|
||||
|
||||
|
||||
@app.callback(invoke_without_command=True)
|
||||
def chat_main(
|
||||
ctx: typer.Context,
|
||||
prompt: Optional[str] = typer.Argument(None, help="Initial prompt for chat"),
|
||||
model: Optional[str] = typer.Option(None, "--model", "-m", help="LLM model to use"),
|
||||
verbose: bool = typer.Option(False, "--verbose", "-v", help="Verbose output"),
|
||||
memory: bool = typer.Option(False, "--memory", help="Enable memory"),
|
||||
memory: Optional[str] = typer.Option(
|
||||
None, "--memory",
|
||||
help="Enable memory. Use --memory for default, --memory=redis for preset, --memory=postgresql://... for URL",
|
||||
is_flag=False,
|
||||
flag_value="true",
|
||||
),
|
||||
no_memory: bool = typer.Option(False, "--no-memory", help="Disable memory"),
|
||||
tools: Optional[str] = typer.Option(None, "--tools", "-t", help="Tools file path"),
|
||||
user_id: Optional[str] = typer.Option(None, "--user-id", help="User ID for memory isolation"),
|
||||
session_id: Optional[str] = typer.Option(None, "--session", "-s", help="Session ID to resume"),
|
||||
@@ -28,6 +68,31 @@ def chat_main(
|
||||
no_acp: bool = typer.Option(False, "--no-acp", help="Disable ACP tools"),
|
||||
no_lsp: bool = typer.Option(False, "--no-lsp", help="Disable LSP tools"),
|
||||
autonomy: bool = typer.Option(True, "--autonomy/--no-autonomy", help="Enable agent autonomy for complex tasks"),
|
||||
# NEW: Agent-like consolidated params for ALL GREEN consistency
|
||||
knowledge: Optional[str] = typer.Option(
|
||||
None, "--knowledge", "-k",
|
||||
help="Enable knowledge/RAG. Use --knowledge for default, --knowledge=docs/ for sources",
|
||||
is_flag=False,
|
||||
flag_value="true",
|
||||
),
|
||||
guardrails: Optional[str] = typer.Option(
|
||||
None, "--guardrails",
|
||||
help="Enable guardrails. Use --guardrails for default, --guardrails=strict for preset",
|
||||
is_flag=False,
|
||||
flag_value="true",
|
||||
),
|
||||
web: Optional[str] = typer.Option(
|
||||
None, "--web",
|
||||
help="Enable web search. Use --web for default, --web=duckduckgo for preset",
|
||||
is_flag=False,
|
||||
flag_value="true",
|
||||
),
|
||||
reflection: Optional[str] = typer.Option(
|
||||
None, "--reflection",
|
||||
help="Enable self-reflection. Use --reflection for default, --reflection=thorough for preset",
|
||||
is_flag=False,
|
||||
flag_value="true",
|
||||
),
|
||||
profile: bool = typer.Option(False, "--profile", help="Enable CLI profiling (timing breakdown)"),
|
||||
profile_deep: bool = typer.Option(False, "--profile-deep", help="Enable deep profiling (cProfile stats, higher overhead)"),
|
||||
):
|
||||
@@ -69,6 +134,9 @@ def chat_main(
|
||||
typer.echo("⚠️ Profiling is only supported for single prompt mode.", err=True)
|
||||
typer.echo(" Use: praisonai chat \"your prompt\" --profile", err=True)
|
||||
|
||||
# Parse memory flag: --no-memory takes precedence, then --memory value
|
||||
memory_value = _parse_memory_flag(memory, no_memory)
|
||||
|
||||
# Try InteractiveCore first (preferred terminal-native implementation)
|
||||
try:
|
||||
from praisonai.cli.interactive import InteractiveCore, InteractiveConfig
|
||||
@@ -80,7 +148,7 @@ def chat_main(
|
||||
continue_session=continue_session,
|
||||
workspace=workspace or None,
|
||||
verbose=verbose,
|
||||
memory=memory,
|
||||
memory=memory_value,
|
||||
files=list(file) if file else [],
|
||||
autonomy=autonomy,
|
||||
)
|
||||
|
||||
@@ -21,7 +21,8 @@ app = typer.Typer(help="Documentation management and code validation")
|
||||
def _get_default_docs_path() -> Path:
|
||||
"""Get default docs path."""
|
||||
candidates = [
|
||||
Path("/Users/praison/PraisonAIDocs"),
|
||||
Path.home() / "PraisonAIDocs" / "docs",
|
||||
Path("/Users/praison/PraisonAIDocs/docs"),
|
||||
Path.cwd() / "docs",
|
||||
Path(__file__).parent.parent.parent.parent.parent / "docs",
|
||||
]
|
||||
@@ -36,7 +37,7 @@ def _get_default_docs_path() -> Path:
|
||||
def _get_default_report_dir() -> Path:
|
||||
"""Get default report directory with timestamp."""
|
||||
timestamp = datetime.now().strftime("%Y%m%d_%H%M%S")
|
||||
return Path.cwd() / "reports" / "docs" / timestamp
|
||||
return Path.home() / "Downloads" / "reports" / "docs" / timestamp
|
||||
|
||||
|
||||
@app.command("run")
|
||||
@@ -56,6 +57,11 @@ def docs_run(
|
||||
"--exclude", "-e",
|
||||
help="Exclude patterns (glob), can be specified multiple times",
|
||||
),
|
||||
group: Optional[List[str]] = typer.Option(
|
||||
None,
|
||||
"--group", "-g",
|
||||
help="Run only specific groups (top-level dirs), can be repeated",
|
||||
),
|
||||
languages: str = typer.Option(
|
||||
"python",
|
||||
"--languages", "-l",
|
||||
@@ -66,26 +72,21 @@ def docs_run(
|
||||
"--timeout", "-t",
|
||||
help="Per-snippet timeout in seconds",
|
||||
),
|
||||
max_snippets: Optional[int] = typer.Option(
|
||||
None,
|
||||
"--max-snippets",
|
||||
help="Maximum snippets to process",
|
||||
),
|
||||
fail_fast: bool = typer.Option(
|
||||
False,
|
||||
"--fail-fast", "-x",
|
||||
help="Stop on first failure",
|
||||
),
|
||||
dry_run: bool = typer.Option(
|
||||
False,
|
||||
"--dry-run",
|
||||
help="Extract only, don't execute",
|
||||
),
|
||||
mode: str = typer.Option(
|
||||
"lenient",
|
||||
"--mode", "-m",
|
||||
help="Mode: 'lenient' or 'strict' (strict fails on not_run)",
|
||||
),
|
||||
max_snippets: Optional[int] = typer.Option(
|
||||
None,
|
||||
"--max-snippets",
|
||||
help="Maximum snippets to process",
|
||||
),
|
||||
no_stream: bool = typer.Option(
|
||||
False,
|
||||
"--no-stream",
|
||||
@@ -106,11 +107,21 @@ def docs_run(
|
||||
"--no-md",
|
||||
help="Skip Markdown report generation",
|
||||
),
|
||||
no_csv: bool = typer.Option(
|
||||
False,
|
||||
"--no-csv",
|
||||
help="Skip CSV report generation",
|
||||
),
|
||||
require_env: Optional[List[str]] = typer.Option(
|
||||
None,
|
||||
"--require-env",
|
||||
help="Required env vars (skip all if missing)",
|
||||
),
|
||||
python: Optional[str] = typer.Option(
|
||||
None,
|
||||
"--python",
|
||||
help="Python executable to use (default: current interpreter)",
|
||||
),
|
||||
quiet: bool = typer.Option(
|
||||
False,
|
||||
"--quiet", "-q",
|
||||
@@ -121,17 +132,16 @@ def docs_run(
|
||||
Run Python code blocks from documentation files.
|
||||
|
||||
Extracts code blocks from Mintlify docs, classifies them as runnable,
|
||||
executes them, and generates reports.
|
||||
executes them, and generates reports (JSON, Markdown, CSV).
|
||||
|
||||
Examples:
|
||||
praisonai docs run
|
||||
praisonai docs run --docs-path /path/to/docs --timeout 120
|
||||
praisonai docs run --dry-run --max-snippets 10
|
||||
praisonai docs run --group models --group tools --max-snippets 10
|
||||
praisonai docs run --include "quickstart*" --fail-fast
|
||||
"""
|
||||
# Lazy import to avoid loading at CLI startup
|
||||
from praisonai.docs_runner.executor import DocsExecutor
|
||||
from praisonai.docs_runner.reporter import SnippetResult
|
||||
from praisonai.suite_runner import DocsSource, SuiteExecutor, RunResult
|
||||
|
||||
# Resolve paths
|
||||
docs = docs_path or _get_default_docs_path()
|
||||
@@ -144,22 +154,34 @@ def docs_run(
|
||||
# Parse languages
|
||||
lang_list = [lang.strip() for lang in languages.split(',') if lang.strip()]
|
||||
|
||||
# Create executor
|
||||
executor = DocsExecutor(
|
||||
docs_path=docs,
|
||||
# Create source
|
||||
source = DocsSource(
|
||||
root=docs,
|
||||
languages=lang_list,
|
||||
include_patterns=list(include) if include else None,
|
||||
exclude_patterns=list(exclude) if exclude else None,
|
||||
languages=lang_list,
|
||||
groups=list(group) if group else None,
|
||||
)
|
||||
|
||||
# Discover items
|
||||
items = source.discover()
|
||||
|
||||
# Create executor
|
||||
executor = SuiteExecutor(
|
||||
suite="docs",
|
||||
source_path=docs,
|
||||
timeout=timeout,
|
||||
fail_fast=fail_fast,
|
||||
stream_output=not no_stream,
|
||||
dry_run=dry_run,
|
||||
mode=mode,
|
||||
max_snippets=max_snippets,
|
||||
max_items=max_snippets,
|
||||
require_env=list(require_env) if require_env else None,
|
||||
report_dir=output_dir,
|
||||
generate_json=not no_json,
|
||||
generate_md=not no_md,
|
||||
generate_csv=not no_csv,
|
||||
python_executable=python,
|
||||
pythonpath_additions=source.get_pythonpath(),
|
||||
groups=list(group) if group else None,
|
||||
)
|
||||
|
||||
# Status icons
|
||||
@@ -169,27 +191,27 @@ def docs_run(
|
||||
"skipped": "⏭️",
|
||||
"timeout": "⏱️",
|
||||
"not_run": "📝",
|
||||
"xfail": "⚠️",
|
||||
}
|
||||
|
||||
def on_snippet_start(block, idx: int, total: int):
|
||||
def on_item_start(item, idx: int, total: int):
|
||||
if not quiet:
|
||||
doc_name = block.doc_path.name
|
||||
typer.echo(f"\n[{idx}/{total}] {doc_name} block {block.block_index}")
|
||||
typer.echo(f"\n[{idx}/{total}] {item.display_name}")
|
||||
|
||||
def on_snippet_end(result: SnippetResult, idx: int, total: int):
|
||||
def on_item_end(result: RunResult, idx: int, total: int):
|
||||
icon = icons.get(result.status, "?")
|
||||
duration = f"{result.duration_seconds:.2f}s" if result.duration_seconds else ""
|
||||
|
||||
if quiet:
|
||||
typer.echo(f"{icon} {result.doc_path.name}:{result.block_index}")
|
||||
typer.echo(f"{icon} {result.display_name}")
|
||||
else:
|
||||
msg = f" {icon} {result.status.upper()}"
|
||||
if duration:
|
||||
msg += f" ({duration})"
|
||||
if result.skip_reason:
|
||||
msg += f" - {result.skip_reason[:50]}"
|
||||
if result.error_summary and not no_stream:
|
||||
msg += f"\n Error: {result.error_summary[:100]}"
|
||||
if result.error_message and not no_stream:
|
||||
msg += f"\n Error: {result.error_message[:100]}"
|
||||
typer.echo(msg)
|
||||
|
||||
def on_output(line: str, stream: str):
|
||||
@@ -205,17 +227,25 @@ def docs_run(
|
||||
typer.echo(f"Docs Path: {docs}")
|
||||
typer.echo(f"Timeout: {timeout}s")
|
||||
typer.echo(f"Mode: {mode}")
|
||||
if dry_run:
|
||||
typer.echo("Mode: DRY RUN (no execution)")
|
||||
if group:
|
||||
typer.echo(f"Groups: {', '.join(group)}")
|
||||
typer.echo(f"Reports: {output_dir}")
|
||||
typer.echo(f"Items: {len(items)}")
|
||||
|
||||
# Run
|
||||
report = executor.run(
|
||||
on_snippet_start=on_snippet_start,
|
||||
on_snippet_end=on_snippet_end,
|
||||
items=items,
|
||||
on_item_start=on_item_start,
|
||||
on_item_end=on_item_end,
|
||||
on_output=on_output if not no_stream else None,
|
||||
)
|
||||
|
||||
# Update report with CLI args
|
||||
report.cli_args = [f"--docs-path={docs}", f"--timeout={timeout}"]
|
||||
if group:
|
||||
for g in group:
|
||||
report.cli_args.append(f"--group={g}")
|
||||
|
||||
# Print summary
|
||||
totals = report.totals
|
||||
total_count = sum(totals.values())
|
||||
@@ -263,6 +293,11 @@ def docs_list(
|
||||
"--exclude", "-e",
|
||||
help="Exclude patterns (glob)",
|
||||
),
|
||||
group: Optional[List[str]] = typer.Option(
|
||||
None,
|
||||
"--group", "-g",
|
||||
help="Filter by group (top-level dir)",
|
||||
),
|
||||
languages: str = typer.Option(
|
||||
"python",
|
||||
"--languages", "-l",
|
||||
@@ -273,16 +308,21 @@ def docs_list(
|
||||
"--code", "-c",
|
||||
help="Show code preview",
|
||||
),
|
||||
show_groups: bool = typer.Option(
|
||||
False,
|
||||
"--groups",
|
||||
help="Show available groups only",
|
||||
),
|
||||
):
|
||||
"""
|
||||
List discovered code blocks from documentation.
|
||||
|
||||
Examples:
|
||||
praisonai docs list
|
||||
praisonai docs list --docs-path /path/to/docs --code
|
||||
praisonai docs list --groups
|
||||
praisonai docs list --group models --code
|
||||
"""
|
||||
from praisonai.docs_runner.extractor import FenceExtractor
|
||||
from praisonai.docs_runner.classifier import RunnableClassifier
|
||||
from praisonai.suite_runner import DocsSource
|
||||
|
||||
docs = docs_path or _get_default_docs_path()
|
||||
|
||||
@@ -293,38 +333,354 @@ def docs_list(
|
||||
# Parse languages
|
||||
lang_list = [lang.strip() for lang in languages.split(',') if lang.strip()]
|
||||
|
||||
extractor = FenceExtractor(languages=lang_list)
|
||||
classifier = RunnableClassifier(target_languages=tuple(lang_list))
|
||||
|
||||
blocks = extractor.extract_from_directory(
|
||||
docs,
|
||||
source = DocsSource(
|
||||
root=docs,
|
||||
languages=lang_list,
|
||||
include_patterns=list(include) if include else None,
|
||||
exclude_patterns=list(exclude) if exclude else None,
|
||||
groups=list(group) if group else None,
|
||||
)
|
||||
|
||||
# Filter to target languages
|
||||
blocks = [b for b in blocks if b.language in lang_list]
|
||||
# Show groups only
|
||||
if show_groups:
|
||||
groups = source.get_groups()
|
||||
typer.echo(f"Available groups in {docs}:\n")
|
||||
for g in groups:
|
||||
typer.echo(f" - {g}")
|
||||
return
|
||||
|
||||
typer.echo(f"Found {len(blocks)} code blocks in {docs}\n")
|
||||
items = source.discover()
|
||||
|
||||
for idx, block in enumerate(blocks, 1):
|
||||
rel_path = block.doc_path.relative_to(docs) if docs in block.doc_path.parents else block.doc_path
|
||||
classification = classifier.classify(block)
|
||||
typer.echo(f"Found {len(items)} code blocks in {docs}\n")
|
||||
|
||||
for idx, item in enumerate(items, 1):
|
||||
rel_path = item.source_path.relative_to(docs) if docs in item.source_path.parents else item.source_path
|
||||
|
||||
status = "✅ Runnable" if classification.is_runnable else "📝 Partial"
|
||||
status = "✅ Runnable" if item.runnable else "📝 Partial"
|
||||
|
||||
typer.echo(f"{idx:3}. {rel_path}:{block.line_start}-{block.line_end}")
|
||||
typer.echo(f" {status} ({classification.reason})")
|
||||
typer.echo(f"{idx:3}. [{item.group}] {rel_path}:{item.line_start}-{item.line_end}")
|
||||
typer.echo(f" {status} ({item.runnable_decision})")
|
||||
|
||||
if show_code:
|
||||
preview = block.code[:100].replace('\n', ' ')
|
||||
if len(block.code) > 100:
|
||||
preview = item.code[:100].replace('\n', ' ')
|
||||
if len(item.code) > 100:
|
||||
preview += "..."
|
||||
typer.echo(f" Code: {preview}")
|
||||
|
||||
typer.echo()
|
||||
|
||||
|
||||
@app.command("stats")
|
||||
def docs_stats(
|
||||
docs_path: Optional[Path] = typer.Option(
|
||||
None,
|
||||
"--docs-path", "-p",
|
||||
help="Path to documentation directory",
|
||||
),
|
||||
group: Optional[List[str]] = typer.Option(
|
||||
None,
|
||||
"--group", "-g",
|
||||
help="Filter by group (top-level dir)",
|
||||
),
|
||||
languages: str = typer.Option(
|
||||
"python",
|
||||
"--languages", "-l",
|
||||
help="Languages to show (comma-separated)",
|
||||
),
|
||||
):
|
||||
"""
|
||||
Show statistics for documentation code blocks.
|
||||
|
||||
Displays counts by group, runnable status, and agent-centric usage.
|
||||
|
||||
Examples:
|
||||
praisonai docs stats
|
||||
praisonai docs stats --group models --group tools
|
||||
"""
|
||||
from collections import Counter
|
||||
from praisonai.suite_runner import DocsSource
|
||||
|
||||
docs = docs_path or _get_default_docs_path()
|
||||
|
||||
if not docs.exists():
|
||||
typer.echo(f"❌ Docs path not found: {docs}")
|
||||
raise typer.Exit(2)
|
||||
|
||||
lang_list = [lang.strip() for lang in languages.split(',') if lang.strip()]
|
||||
|
||||
source = DocsSource(
|
||||
root=docs,
|
||||
languages=lang_list,
|
||||
groups=list(group) if group else None,
|
||||
)
|
||||
|
||||
items = source.discover()
|
||||
|
||||
# Calculate stats
|
||||
group_counts = Counter(item.group for item in items)
|
||||
runnable_by_group = Counter(item.group for item in items if item.runnable)
|
||||
|
||||
# Agent-centric stats
|
||||
agent_count = sum(1 for item in items if item.uses_agent)
|
||||
agents_count = sum(1 for item in items if item.uses_agents)
|
||||
workflow_count = sum(1 for item in items if item.uses_workflow)
|
||||
|
||||
agent_by_group = Counter(item.group for item in items if item.uses_agent)
|
||||
agents_by_group = Counter(item.group for item in items if item.uses_agents)
|
||||
workflow_by_group = Counter(item.group for item in items if item.uses_workflow)
|
||||
|
||||
typer.echo("\n📊 Documentation Code Block Statistics")
|
||||
typer.echo(f"Path: {docs}")
|
||||
typer.echo("=" * 80)
|
||||
|
||||
typer.echo(f"\n{'Group':<20} {'Total':>8} {'Runnable':>10} {'Agent':>8} {'Agents':>8} {'Workflow':>10}")
|
||||
typer.echo("-" * 80)
|
||||
|
||||
for g in sorted(group_counts.keys()):
|
||||
typer.echo(
|
||||
f"{g:<20} {group_counts[g]:>8} {runnable_by_group[g]:>10} "
|
||||
f"{agent_by_group[g]:>8} {agents_by_group[g]:>8} {workflow_by_group[g]:>10}"
|
||||
)
|
||||
|
||||
typer.echo("-" * 80)
|
||||
typer.echo(
|
||||
f"{'TOTAL':<20} {len(items):>8} {sum(1 for i in items if i.runnable):>10} "
|
||||
f"{agent_count:>8} {agents_count:>8} {workflow_count:>10}"
|
||||
)
|
||||
typer.echo()
|
||||
|
||||
|
||||
@app.command("run-all")
|
||||
def docs_run_all(
|
||||
docs_path: Optional[Path] = typer.Option(
|
||||
None,
|
||||
"--docs-path", "-p",
|
||||
help="Path to documentation directory",
|
||||
),
|
||||
timeout: int = typer.Option(
|
||||
60,
|
||||
"--timeout", "-t",
|
||||
help="Per-snippet timeout in seconds",
|
||||
),
|
||||
report_dir: Optional[Path] = typer.Option(
|
||||
None,
|
||||
"--report-dir", "-r",
|
||||
help="Directory for reports (default: ~/Downloads/reports/docs/<timestamp>)",
|
||||
),
|
||||
parallel: bool = typer.Option(
|
||||
True,
|
||||
"--parallel/--sequential",
|
||||
help="Run groups in parallel (default: parallel)",
|
||||
),
|
||||
max_workers: int = typer.Option(
|
||||
4,
|
||||
"--workers", "-w",
|
||||
help="Max parallel workers (default: 4)",
|
||||
),
|
||||
quiet: bool = typer.Option(
|
||||
False,
|
||||
"--quiet", "-q",
|
||||
help="Minimal output",
|
||||
),
|
||||
):
|
||||
"""
|
||||
Run all documentation code blocks group-by-group.
|
||||
|
||||
Executes all groups and generates a comprehensive report.
|
||||
Uses parallel execution by default for faster results.
|
||||
|
||||
Examples:
|
||||
praisonai docs run-all
|
||||
praisonai docs run-all --sequential
|
||||
praisonai docs run-all --workers 8 --timeout 120
|
||||
"""
|
||||
import json
|
||||
from concurrent.futures import ProcessPoolExecutor, as_completed
|
||||
from praisonai.suite_runner import DocsSource, SuiteExecutor, RunResult
|
||||
|
||||
docs = docs_path or _get_default_docs_path()
|
||||
output_dir = report_dir or _get_default_report_dir()
|
||||
|
||||
if not docs.exists():
|
||||
typer.echo(f"❌ Docs path not found: {docs}")
|
||||
raise typer.Exit(2)
|
||||
|
||||
# Get all groups
|
||||
source = DocsSource(root=docs)
|
||||
all_groups = sorted(source.get_groups())
|
||||
|
||||
typer.echo(f"Found {len(all_groups)} groups to process")
|
||||
typer.echo(f"Mode: {'parallel' if parallel else 'sequential'}")
|
||||
typer.echo("=" * 60)
|
||||
|
||||
# Track overall results
|
||||
overall_results = {
|
||||
'passed': 0, 'failed': 0, 'skipped': 0,
|
||||
'timeout': 0, 'not_run': 0, 'total': 0,
|
||||
}
|
||||
group_summaries = []
|
||||
|
||||
def run_group(group_name: str) -> dict:
|
||||
"""Run a single group and return summary."""
|
||||
group_source = DocsSource(root=docs, groups=[group_name])
|
||||
items = group_source.discover()
|
||||
|
||||
if not items:
|
||||
return {
|
||||
'group': group_name,
|
||||
'total': 0, 'passed': 0, 'failed': 0,
|
||||
'skipped': 0, 'timeout': 0, 'not_run': 0,
|
||||
}
|
||||
|
||||
group_report_dir = output_dir / group_name
|
||||
group_report_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
executor = SuiteExecutor(
|
||||
suite='docs',
|
||||
source_path=docs,
|
||||
report_dir=group_report_dir,
|
||||
timeout=timeout,
|
||||
stream_output=False,
|
||||
generate_json=True,
|
||||
generate_md=True,
|
||||
generate_csv=True,
|
||||
groups=[group_name],
|
||||
)
|
||||
|
||||
report = executor.run(items=items)
|
||||
totals = report.totals
|
||||
|
||||
return {
|
||||
'group': group_name,
|
||||
**totals,
|
||||
}
|
||||
|
||||
if parallel and len(all_groups) > 1:
|
||||
# Parallel execution
|
||||
with ProcessPoolExecutor(max_workers=min(max_workers, len(all_groups))) as pool:
|
||||
futures = {pool.submit(run_group, g): g for g in all_groups}
|
||||
|
||||
for future in as_completed(futures):
|
||||
group_name = futures[future]
|
||||
try:
|
||||
summary = future.result()
|
||||
group_summaries.append(summary)
|
||||
|
||||
# Update overall
|
||||
for key in overall_results:
|
||||
if key in summary:
|
||||
overall_results[key] += summary[key]
|
||||
|
||||
if not quiet:
|
||||
typer.echo(
|
||||
f"✅ {group_name}: "
|
||||
f"✅{summary['passed']} ❌{summary['failed']} "
|
||||
f"⏭️{summary['skipped']} ⏱️{summary['timeout']} 📝{summary['not_run']}"
|
||||
)
|
||||
except Exception as e:
|
||||
typer.echo(f"❌ {group_name}: Error - {e}")
|
||||
else:
|
||||
# Sequential execution with real-time output
|
||||
for group_name in all_groups:
|
||||
if not quiet:
|
||||
typer.echo(f"\n{'='*60}")
|
||||
typer.echo(f"GROUP: {group_name}")
|
||||
typer.echo(f"{'='*60}")
|
||||
|
||||
group_source = DocsSource(root=docs, groups=[group_name])
|
||||
items = group_source.discover()
|
||||
|
||||
if not items:
|
||||
if not quiet:
|
||||
typer.echo(" No items found, skipping")
|
||||
group_summaries.append({
|
||||
'group': group_name,
|
||||
'total': 0, 'passed': 0, 'failed': 0,
|
||||
'skipped': 0, 'timeout': 0, 'not_run': 0,
|
||||
})
|
||||
continue
|
||||
|
||||
group_report_dir = output_dir / group_name
|
||||
group_report_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
executor = SuiteExecutor(
|
||||
suite='docs',
|
||||
source_path=docs,
|
||||
report_dir=group_report_dir,
|
||||
timeout=timeout,
|
||||
stream_output=False,
|
||||
generate_json=True,
|
||||
generate_md=True,
|
||||
generate_csv=True,
|
||||
groups=[group_name],
|
||||
)
|
||||
|
||||
icons = {'passed': '✅', 'failed': '❌', 'skipped': '⏭️', 'timeout': '⏱️', 'not_run': '📝'}
|
||||
|
||||
def on_start(item, idx, total):
|
||||
if not quiet:
|
||||
typer.echo(f" [{idx}/{total}] {item.display_name} ", nl=False)
|
||||
|
||||
def on_end(result: RunResult, idx, total):
|
||||
if not quiet:
|
||||
icon = icons.get(result.status, '❓')
|
||||
typer.echo(f"{icon}")
|
||||
|
||||
report = executor.run(items=items, on_item_start=on_start, on_item_end=on_end)
|
||||
totals = report.totals
|
||||
|
||||
group_summaries.append({'group': group_name, **totals})
|
||||
|
||||
for key in overall_results:
|
||||
if key in totals:
|
||||
overall_results[key] += totals[key]
|
||||
|
||||
if not quiet:
|
||||
typer.echo(
|
||||
f"\n Summary: ✅{totals['passed']} ❌{totals['failed']} "
|
||||
f"⏭️{totals['skipped']} ⏱️{totals['timeout']} 📝{totals['not_run']}"
|
||||
)
|
||||
|
||||
# Final report
|
||||
typer.echo("\n" + "=" * 80)
|
||||
typer.echo("FINAL REPORT - ALL GROUPS")
|
||||
typer.echo("=" * 80)
|
||||
|
||||
typer.echo(f"\n{'Group':<20} {'Total':>8} {'Passed':>8} {'Failed':>8} {'Skip':>8} {'Timeout':>8} {'NotRun':>8}")
|
||||
typer.echo("-" * 80)
|
||||
|
||||
for gs in sorted(group_summaries, key=lambda x: x['group']):
|
||||
typer.echo(
|
||||
f"{gs['group']:<20} {gs['total']:>8} {gs['passed']:>8} {gs['failed']:>8} "
|
||||
f"{gs['skipped']:>8} {gs['timeout']:>8} {gs['not_run']:>8}"
|
||||
)
|
||||
|
||||
typer.echo("-" * 80)
|
||||
typer.echo(
|
||||
f"{'TOTAL':<20} {overall_results['total']:>8} {overall_results['passed']:>8} "
|
||||
f"{overall_results['failed']:>8} {overall_results['skipped']:>8} "
|
||||
f"{overall_results['timeout']:>8} {overall_results['not_run']:>8}"
|
||||
)
|
||||
|
||||
# Save final summary
|
||||
output_dir.mkdir(parents=True, exist_ok=True)
|
||||
summary_path = output_dir / 'final_summary.json'
|
||||
with open(summary_path, 'w') as f:
|
||||
json.dump({
|
||||
'timestamp': datetime.now().isoformat(),
|
||||
'overall': overall_results,
|
||||
'groups': group_summaries,
|
||||
}, f, indent=2)
|
||||
|
||||
typer.echo(f"\n📁 Final summary saved to: {summary_path}")
|
||||
typer.echo(f"📁 Individual group reports in: {output_dir}")
|
||||
|
||||
# Exit code
|
||||
if overall_results['failed'] > 0 or overall_results['timeout'] > 0:
|
||||
raise typer.Exit(1)
|
||||
raise typer.Exit(0)
|
||||
|
||||
|
||||
@app.command("generate")
|
||||
def docs_generate(
|
||||
source: str = typer.Argument(".", help="Source directory"),
|
||||
@@ -368,3 +724,92 @@ def docs_serve(
|
||||
pass
|
||||
finally:
|
||||
sys.argv = original_argv
|
||||
|
||||
|
||||
@app.command("report")
|
||||
def docs_report(
|
||||
report_dir: Optional[Path] = typer.Argument(
|
||||
None,
|
||||
help="Report directory path (default: auto-detect latest)",
|
||||
),
|
||||
base_dir: Optional[Path] = typer.Option(
|
||||
None,
|
||||
"--base-dir", "-b",
|
||||
help="Base reports directory (default: ~/Downloads/reports/docs)",
|
||||
),
|
||||
limit: int = typer.Option(
|
||||
50,
|
||||
"--limit", "-n",
|
||||
help="Limit rows in tables (0 = no limit)",
|
||||
),
|
||||
errors: bool = typer.Option(
|
||||
True,
|
||||
"--errors/--no-errors",
|
||||
help="Show error grouping section",
|
||||
),
|
||||
wide: bool = typer.Option(
|
||||
False,
|
||||
"--wide/--no-wide",
|
||||
help="Wide output: full error messages and item lists",
|
||||
),
|
||||
show_paths: bool = typer.Option(
|
||||
True,
|
||||
"--show-paths/--no-show-paths",
|
||||
help="Show affected item paths in error groups",
|
||||
),
|
||||
match: Optional[List[str]] = typer.Option(
|
||||
None,
|
||||
"--match", "-m",
|
||||
help="Filter by status (repeatable: --match failed --match timeout)",
|
||||
),
|
||||
group: Optional[List[str]] = typer.Option(
|
||||
None,
|
||||
"--group", "-g",
|
||||
help="Filter by group name (repeatable)",
|
||||
),
|
||||
contains: Optional[str] = typer.Option(
|
||||
None,
|
||||
"--contains", "-c",
|
||||
help="Filter items whose error contains text (case-insensitive)",
|
||||
),
|
||||
open_dir: bool = typer.Option(
|
||||
False,
|
||||
"--open",
|
||||
help="Show report artifacts listing",
|
||||
),
|
||||
output_format: str = typer.Option(
|
||||
"table",
|
||||
"--format", "-f",
|
||||
help="Output format: table or json",
|
||||
),
|
||||
):
|
||||
"""
|
||||
View execution report with failures and error grouping.
|
||||
|
||||
Examples:
|
||||
praisonai docs report # View latest report
|
||||
praisonai docs report ./my-report # View specific report
|
||||
praisonai docs report --wide --limit 0 # Full details
|
||||
praisonai docs report --match failed # Only failures
|
||||
praisonai docs report --format json # JSON output
|
||||
"""
|
||||
# Lazy import to avoid loading at CLI startup
|
||||
from praisonai.suite_runner.report_viewer import view_report
|
||||
|
||||
output, exit_code = view_report(
|
||||
report_dir=report_dir,
|
||||
suite="docs",
|
||||
base_dir=base_dir,
|
||||
limit=limit,
|
||||
show_errors=errors,
|
||||
wide=wide,
|
||||
show_paths=show_paths,
|
||||
match_statuses=match,
|
||||
match_groups=group,
|
||||
contains=contains,
|
||||
show_artifacts=open_dir,
|
||||
output_format=output_format,
|
||||
)
|
||||
|
||||
typer.echo(output)
|
||||
raise typer.Exit(exit_code)
|
||||
@@ -20,8 +20,9 @@ app = typer.Typer(help="Run and manage example files")
|
||||
|
||||
def _get_default_examples_path() -> Path:
|
||||
"""Get default examples path (repo examples/ or cwd)."""
|
||||
# Try to find examples directory
|
||||
candidates = [
|
||||
Path.home() / "praisonai-package" / "examples",
|
||||
Path("/Users/praison/praisonai-package/examples"),
|
||||
Path.cwd() / "examples",
|
||||
Path(__file__).parent.parent.parent.parent.parent / "examples", # repo root
|
||||
]
|
||||
@@ -36,7 +37,7 @@ def _get_default_examples_path() -> Path:
|
||||
def _get_default_report_dir() -> Path:
|
||||
"""Get default report directory with timestamp."""
|
||||
timestamp = datetime.now().strftime("%Y%m%d_%H%M%S")
|
||||
return Path.cwd() / "reports" / "examples" / timestamp
|
||||
return Path.home() / "Downloads" / "reports" / "examples" / timestamp
|
||||
|
||||
|
||||
@app.command()
|
||||
@@ -56,11 +57,21 @@ def run(
|
||||
"--exclude", "-e",
|
||||
help="Exclude patterns (glob), can be specified multiple times",
|
||||
),
|
||||
group: Optional[List[str]] = typer.Option(
|
||||
None,
|
||||
"--group", "-g",
|
||||
help="Run only specific groups (top-level dirs), can be repeated",
|
||||
),
|
||||
timeout: int = typer.Option(
|
||||
60,
|
||||
"--timeout", "-t",
|
||||
help="Per-example timeout in seconds",
|
||||
),
|
||||
max_items: Optional[int] = typer.Option(
|
||||
None,
|
||||
"--max-items",
|
||||
help="Maximum examples to run",
|
||||
),
|
||||
fail_fast: bool = typer.Option(
|
||||
False,
|
||||
"--fail-fast", "-x",
|
||||
@@ -86,11 +97,21 @@ def run(
|
||||
"--no-md",
|
||||
help="Skip Markdown report generation",
|
||||
),
|
||||
no_csv: bool = typer.Option(
|
||||
False,
|
||||
"--no-csv",
|
||||
help="Skip CSV report generation",
|
||||
),
|
||||
require_env: Optional[List[str]] = typer.Option(
|
||||
None,
|
||||
"--require-env",
|
||||
help="Required env vars (skip all if missing)",
|
||||
),
|
||||
python: Optional[str] = typer.Option(
|
||||
None,
|
||||
"--python",
|
||||
help="Python executable to use (default: current interpreter)",
|
||||
),
|
||||
quiet: bool = typer.Option(
|
||||
False,
|
||||
"--quiet", "-q",
|
||||
@@ -103,11 +124,12 @@ def run(
|
||||
Examples:
|
||||
praisonai examples run
|
||||
praisonai examples run --path ./examples --timeout 120
|
||||
praisonai examples run --group python --group mcp --max-items 5
|
||||
praisonai examples run --include "context/*" --exclude "*_wow.py"
|
||||
praisonai examples run --fail-fast --no-stream
|
||||
"""
|
||||
# Lazy import to avoid loading at CLI startup
|
||||
from praisonai.cli.features.examples import ExamplesExecutor, ExampleResult
|
||||
from praisonai.suite_runner import ExamplesSource, SuiteExecutor, RunResult
|
||||
|
||||
# Resolve paths
|
||||
examples_path = path or _get_default_examples_path()
|
||||
@@ -117,18 +139,33 @@ def run(
|
||||
typer.echo(f"❌ Examples path not found: {examples_path}")
|
||||
raise typer.Exit(2)
|
||||
|
||||
# Create source
|
||||
source = ExamplesSource(
|
||||
root=examples_path,
|
||||
include_patterns=list(include) if include else None,
|
||||
exclude_patterns=list(exclude) if exclude else None,
|
||||
groups=list(group) if group else None,
|
||||
)
|
||||
|
||||
# Discover items
|
||||
items = source.discover()
|
||||
|
||||
# Create executor
|
||||
executor = ExamplesExecutor(
|
||||
path=examples_path,
|
||||
executor = SuiteExecutor(
|
||||
suite="examples",
|
||||
source_path=examples_path,
|
||||
timeout=timeout,
|
||||
fail_fast=fail_fast,
|
||||
stream_output=not no_stream,
|
||||
include_patterns=list(include) if include else None,
|
||||
exclude_patterns=list(exclude) if exclude else None,
|
||||
max_items=max_items,
|
||||
require_env=list(require_env) if require_env else None,
|
||||
report_dir=output_dir,
|
||||
generate_json=not no_json,
|
||||
generate_md=not no_md,
|
||||
generate_csv=not no_csv,
|
||||
python_executable=python,
|
||||
pythonpath_additions=source.get_pythonpath(),
|
||||
groups=list(group) if group else None,
|
||||
)
|
||||
|
||||
# Status icons
|
||||
@@ -138,26 +175,27 @@ def run(
|
||||
"skipped": "⏭️",
|
||||
"timeout": "⏱️",
|
||||
"xfail": "⚠️",
|
||||
"not_run": "📝",
|
||||
}
|
||||
|
||||
def on_example_start(path: Path, idx: int, total: int):
|
||||
def on_item_start(item, idx: int, total: int):
|
||||
if not quiet:
|
||||
typer.echo(f"\n[{idx}/{total}] Running: {path.name}")
|
||||
typer.echo(f"\n[{idx}/{total}] Running: {item.display_name}")
|
||||
|
||||
def on_example_end(result: ExampleResult, idx: int, total: int):
|
||||
def on_item_end(result: RunResult, idx: int, total: int):
|
||||
icon = icons.get(result.status, "?")
|
||||
duration = f"{result.duration_seconds:.2f}s" if result.duration_seconds else ""
|
||||
|
||||
if quiet:
|
||||
typer.echo(f"{icon} {result.path.name}")
|
||||
typer.echo(f"{icon} {result.display_name}")
|
||||
else:
|
||||
msg = f" {icon} {result.status.upper()}"
|
||||
if duration:
|
||||
msg += f" ({duration})"
|
||||
if result.skip_reason:
|
||||
msg += f" - {result.skip_reason}"
|
||||
if result.error_summary and not no_stream:
|
||||
msg += f"\n Error: {result.error_summary[:100]}"
|
||||
if result.error_message and not no_stream:
|
||||
msg += f"\n Error: {result.error_message[:100]}"
|
||||
typer.echo(msg)
|
||||
|
||||
def on_output(line: str, stream: str):
|
||||
@@ -172,28 +210,26 @@ def run(
|
||||
typer.echo("=" * 60)
|
||||
typer.echo(f"Path: {examples_path}")
|
||||
typer.echo(f"Timeout: {timeout}s")
|
||||
if group:
|
||||
typer.echo(f"Groups: {', '.join(group)}")
|
||||
typer.echo(f"Reports: {output_dir}")
|
||||
typer.echo(f"Items: {len(items)}")
|
||||
|
||||
# Run examples
|
||||
report = executor.run(
|
||||
on_example_start=on_example_start,
|
||||
on_example_end=on_example_end,
|
||||
items=items,
|
||||
on_item_start=on_item_start,
|
||||
on_item_end=on_item_end,
|
||||
on_output=on_output if not no_stream else None,
|
||||
)
|
||||
|
||||
# Update report with CLI args
|
||||
report.cli_args = [
|
||||
f"--path={examples_path}",
|
||||
f"--timeout={timeout}",
|
||||
]
|
||||
report.cli_args = [f"--path={examples_path}", f"--timeout={timeout}"]
|
||||
if fail_fast:
|
||||
report.cli_args.append("--fail-fast")
|
||||
if include:
|
||||
for p in include:
|
||||
report.cli_args.append(f"--include={p}")
|
||||
if exclude:
|
||||
for p in exclude:
|
||||
report.cli_args.append(f"--exclude={p}")
|
||||
if group:
|
||||
for g in group:
|
||||
report.cli_args.append(f"--group={g}")
|
||||
|
||||
# Print summary
|
||||
totals = report.totals
|
||||
@@ -239,20 +275,31 @@ def list_examples(
|
||||
"--exclude", "-e",
|
||||
help="Exclude patterns (glob)",
|
||||
),
|
||||
group: Optional[List[str]] = typer.Option(
|
||||
None,
|
||||
"--group", "-g",
|
||||
help="Filter by group (top-level dir)",
|
||||
),
|
||||
show_metadata: bool = typer.Option(
|
||||
False,
|
||||
"--metadata", "-m",
|
||||
help="Show parsed metadata for each example",
|
||||
),
|
||||
show_groups: bool = typer.Option(
|
||||
False,
|
||||
"--groups",
|
||||
help="Show available groups only",
|
||||
),
|
||||
):
|
||||
"""
|
||||
List discovered examples without running them.
|
||||
|
||||
Examples:
|
||||
praisonai examples list
|
||||
praisonai examples list --path ./examples --metadata
|
||||
praisonai examples list --groups
|
||||
praisonai examples list --group python --metadata
|
||||
"""
|
||||
from praisonai.cli.features.examples import ExampleDiscovery, ExampleMetadata
|
||||
from praisonai.suite_runner import ExamplesSource
|
||||
|
||||
examples_path = path or _get_default_examples_path()
|
||||
|
||||
@@ -260,37 +307,45 @@ def list_examples(
|
||||
typer.echo(f"❌ Examples path not found: {examples_path}")
|
||||
raise typer.Exit(2)
|
||||
|
||||
discovery = ExampleDiscovery(
|
||||
examples_path,
|
||||
source = ExamplesSource(
|
||||
root=examples_path,
|
||||
include_patterns=list(include) if include else None,
|
||||
exclude_patterns=list(exclude) if exclude else None,
|
||||
groups=list(group) if group else None,
|
||||
)
|
||||
|
||||
examples = discovery.discover()
|
||||
# Show groups only
|
||||
if show_groups:
|
||||
groups = source.get_groups()
|
||||
typer.echo(f"Available groups in {examples_path}:\n")
|
||||
for g in groups:
|
||||
typer.echo(f" - {g}")
|
||||
return
|
||||
|
||||
typer.echo(f"Found {len(examples)} examples in {examples_path}\n")
|
||||
items = source.discover()
|
||||
|
||||
for idx, ex in enumerate(examples, 1):
|
||||
rel_path = ex.relative_to(examples_path)
|
||||
typer.echo(f"Found {len(items)} examples in {examples_path}\n")
|
||||
|
||||
for idx, item in enumerate(items, 1):
|
||||
rel_path = item.source_path.relative_to(examples_path)
|
||||
|
||||
if show_metadata:
|
||||
meta = ExampleMetadata.from_file(ex)
|
||||
flags = []
|
||||
if meta.skip:
|
||||
if item.skip:
|
||||
flags.append("skip")
|
||||
if meta.timeout:
|
||||
flags.append(f"timeout={meta.timeout}")
|
||||
if meta.require_env:
|
||||
flags.append(f"env={','.join(meta.require_env)}")
|
||||
if meta.xfail:
|
||||
flags.append(f"xfail={meta.xfail}")
|
||||
if meta.is_interactive:
|
||||
if item.timeout:
|
||||
flags.append(f"timeout={item.timeout}")
|
||||
if item.require_env:
|
||||
flags.append(f"env={','.join(item.require_env)}")
|
||||
if item.xfail:
|
||||
flags.append(f"xfail={item.xfail}")
|
||||
if item.is_interactive:
|
||||
flags.append("interactive")
|
||||
|
||||
flag_str = f" [{', '.join(flags)}]" if flags else ""
|
||||
typer.echo(f" {idx:3}. {rel_path}{flag_str}")
|
||||
typer.echo(f" {idx:3}. [{item.group}] {rel_path}{flag_str}")
|
||||
else:
|
||||
typer.echo(f" {idx:3}. {rel_path}")
|
||||
typer.echo(f" {idx:3}. [{item.group}] {rel_path}")
|
||||
|
||||
|
||||
@app.command()
|
||||
@@ -306,25 +361,423 @@ def info(
|
||||
Examples:
|
||||
praisonai examples info ./examples/context/01_basic.py
|
||||
"""
|
||||
from praisonai.cli.features.examples import ExampleMetadata
|
||||
from praisonai.suite_runner import ExamplesSource
|
||||
|
||||
if not example.exists():
|
||||
typer.echo(f"❌ Example not found: {example}")
|
||||
raise typer.Exit(2)
|
||||
|
||||
meta = ExampleMetadata.from_file(example)
|
||||
# Create a source just to parse the file
|
||||
source = ExamplesSource(root=example.parent)
|
||||
item = source._create_item(example)
|
||||
|
||||
typer.echo(f"Example: {example.name}")
|
||||
typer.echo(f"Path: {example}")
|
||||
typer.echo(f"Group: {item.group}")
|
||||
typer.echo("")
|
||||
typer.echo("Metadata:")
|
||||
typer.echo(f" Skip: {meta.skip}" if meta.skip else " Skip: False")
|
||||
if meta.skip_reason:
|
||||
typer.echo(f" Skip Reason: {meta.skip_reason}")
|
||||
typer.echo(f" Timeout: {meta.timeout or 'default'}")
|
||||
typer.echo(f" Required Env: {', '.join(meta.require_env) if meta.require_env else 'none'}")
|
||||
typer.echo(f" XFail: {meta.xfail or 'no'}")
|
||||
typer.echo(f" Interactive: {meta.is_interactive}")
|
||||
typer.echo(f" Runnable: {item.runnable}")
|
||||
typer.echo(f" Decision: {item.runnable_decision}")
|
||||
typer.echo(f" Skip: {item.skip}")
|
||||
if item.skip_reason:
|
||||
typer.echo(f" Skip Reason: {item.skip_reason}")
|
||||
typer.echo(f" Timeout: {item.timeout or 'default'}")
|
||||
typer.echo(f" Required Env: {', '.join(item.require_env) if item.require_env else 'none'}")
|
||||
typer.echo(f" XFail: {item.xfail or 'no'}")
|
||||
typer.echo(f" Interactive: {item.is_interactive}")
|
||||
typer.echo(f" Code Hash: {item.code_hash}")
|
||||
|
||||
|
||||
@app.command("stats")
|
||||
def examples_stats(
|
||||
path: Optional[Path] = typer.Option(
|
||||
None,
|
||||
"--path", "-p",
|
||||
help="Path to examples directory",
|
||||
),
|
||||
group: Optional[List[str]] = typer.Option(
|
||||
None,
|
||||
"--group", "-g",
|
||||
help="Filter by group (top-level dir)",
|
||||
),
|
||||
):
|
||||
"""
|
||||
Show statistics for examples.
|
||||
|
||||
Displays counts by group, runnable status, and agent-centric usage.
|
||||
|
||||
Examples:
|
||||
praisonai examples stats
|
||||
praisonai examples stats --group python --group mcp
|
||||
"""
|
||||
from collections import Counter
|
||||
from praisonai.suite_runner import ExamplesSource
|
||||
|
||||
examples_path = path or _get_default_examples_path()
|
||||
|
||||
if not examples_path.exists():
|
||||
typer.echo(f"❌ Examples path not found: {examples_path}")
|
||||
raise typer.Exit(2)
|
||||
|
||||
source = ExamplesSource(
|
||||
root=examples_path,
|
||||
groups=list(group) if group else None,
|
||||
)
|
||||
|
||||
items = source.discover()
|
||||
|
||||
# Calculate stats
|
||||
group_counts = Counter(item.group for item in items)
|
||||
runnable_by_group = Counter(item.group for item in items if item.runnable)
|
||||
|
||||
# Agent-centric stats
|
||||
agent_count = sum(1 for item in items if item.uses_agent)
|
||||
agents_count = sum(1 for item in items if item.uses_agents)
|
||||
workflow_count = sum(1 for item in items if item.uses_workflow)
|
||||
|
||||
agent_by_group = Counter(item.group for item in items if item.uses_agent)
|
||||
agents_by_group = Counter(item.group for item in items if item.uses_agents)
|
||||
workflow_by_group = Counter(item.group for item in items if item.uses_workflow)
|
||||
|
||||
typer.echo("\n📊 Examples Statistics")
|
||||
typer.echo(f"Path: {examples_path}")
|
||||
typer.echo("=" * 80)
|
||||
|
||||
typer.echo(f"\n{'Group':<20} {'Total':>8} {'Runnable':>10} {'Agent':>8} {'Agents':>8} {'Workflow':>10}")
|
||||
typer.echo("-" * 80)
|
||||
|
||||
for g in sorted(group_counts.keys()):
|
||||
typer.echo(
|
||||
f"{g:<20} {group_counts[g]:>8} {runnable_by_group[g]:>10} "
|
||||
f"{agent_by_group[g]:>8} {agents_by_group[g]:>8} {workflow_by_group[g]:>10}"
|
||||
)
|
||||
|
||||
typer.echo("-" * 80)
|
||||
typer.echo(
|
||||
f"{'TOTAL':<20} {len(items):>8} {sum(1 for i in items if i.runnable):>10} "
|
||||
f"{agent_count:>8} {agents_count:>8} {workflow_count:>10}"
|
||||
)
|
||||
typer.echo()
|
||||
|
||||
|
||||
@app.command("run-all")
|
||||
def examples_run_all(
|
||||
path: Optional[Path] = typer.Option(
|
||||
None,
|
||||
"--path", "-p",
|
||||
help="Path to examples directory",
|
||||
),
|
||||
timeout: int = typer.Option(
|
||||
60,
|
||||
"--timeout", "-t",
|
||||
help="Per-example timeout in seconds",
|
||||
),
|
||||
report_dir: Optional[Path] = typer.Option(
|
||||
None,
|
||||
"--report-dir", "-r",
|
||||
help="Directory for reports (default: ~/Downloads/reports/examples/<timestamp>)",
|
||||
),
|
||||
parallel: bool = typer.Option(
|
||||
True,
|
||||
"--parallel/--sequential",
|
||||
help="Run groups in parallel (default: parallel)",
|
||||
),
|
||||
max_workers: int = typer.Option(
|
||||
4,
|
||||
"--workers", "-w",
|
||||
help="Max parallel workers (default: 4)",
|
||||
),
|
||||
quiet: bool = typer.Option(
|
||||
False,
|
||||
"--quiet", "-q",
|
||||
help="Minimal output",
|
||||
),
|
||||
):
|
||||
"""
|
||||
Run all examples group-by-group.
|
||||
|
||||
Executes all groups and generates a comprehensive report.
|
||||
Uses parallel execution by default for faster results.
|
||||
|
||||
Examples:
|
||||
praisonai examples run-all
|
||||
praisonai examples run-all --sequential
|
||||
praisonai examples run-all --workers 8 --timeout 120
|
||||
"""
|
||||
import json
|
||||
from concurrent.futures import ProcessPoolExecutor, as_completed
|
||||
from praisonai.suite_runner import ExamplesSource, SuiteExecutor, RunResult
|
||||
|
||||
examples_path = path or _get_default_examples_path()
|
||||
output_dir = report_dir or _get_default_report_dir()
|
||||
|
||||
if not examples_path.exists():
|
||||
typer.echo(f"❌ Examples path not found: {examples_path}")
|
||||
raise typer.Exit(2)
|
||||
|
||||
# Get all groups
|
||||
source = ExamplesSource(root=examples_path)
|
||||
all_groups = sorted(source.get_groups())
|
||||
|
||||
typer.echo(f"Found {len(all_groups)} groups to process")
|
||||
typer.echo(f"Mode: {'parallel' if parallel else 'sequential'}")
|
||||
typer.echo("=" * 60)
|
||||
|
||||
# Track overall results
|
||||
overall_results = {
|
||||
'passed': 0, 'failed': 0, 'skipped': 0,
|
||||
'timeout': 0, 'not_run': 0, 'total': 0, 'xfail': 0,
|
||||
}
|
||||
group_summaries = []
|
||||
|
||||
def run_group(group_name: str) -> dict:
|
||||
"""Run a single group and return summary."""
|
||||
group_source = ExamplesSource(root=examples_path, groups=[group_name])
|
||||
items = group_source.discover()
|
||||
|
||||
if not items:
|
||||
return {
|
||||
'group': group_name,
|
||||
'total': 0, 'passed': 0, 'failed': 0,
|
||||
'skipped': 0, 'timeout': 0, 'not_run': 0, 'xfail': 0,
|
||||
}
|
||||
|
||||
group_report_dir = output_dir / group_name
|
||||
group_report_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
executor = SuiteExecutor(
|
||||
suite='examples',
|
||||
source_path=examples_path,
|
||||
report_dir=group_report_dir,
|
||||
timeout=timeout,
|
||||
stream_output=False,
|
||||
generate_json=True,
|
||||
generate_md=True,
|
||||
generate_csv=True,
|
||||
groups=[group_name],
|
||||
)
|
||||
|
||||
report = executor.run(items=items)
|
||||
totals = report.totals
|
||||
|
||||
return {
|
||||
'group': group_name,
|
||||
**totals,
|
||||
}
|
||||
|
||||
if parallel and len(all_groups) > 1:
|
||||
# Parallel execution
|
||||
with ProcessPoolExecutor(max_workers=min(max_workers, len(all_groups))) as pool:
|
||||
futures = {pool.submit(run_group, g): g for g in all_groups}
|
||||
|
||||
for future in as_completed(futures):
|
||||
group_name = futures[future]
|
||||
try:
|
||||
summary = future.result()
|
||||
group_summaries.append(summary)
|
||||
|
||||
# Update overall
|
||||
for key in overall_results:
|
||||
if key in summary:
|
||||
overall_results[key] += summary[key]
|
||||
|
||||
if not quiet:
|
||||
typer.echo(
|
||||
f"✅ {group_name}: "
|
||||
f"✅{summary['passed']} ❌{summary['failed']} "
|
||||
f"⏭️{summary['skipped']} ⏱️{summary['timeout']} ⚠️{summary.get('xfail', 0)}"
|
||||
)
|
||||
except Exception as e:
|
||||
typer.echo(f"❌ {group_name}: Error - {e}")
|
||||
else:
|
||||
# Sequential execution with real-time output
|
||||
for group_name in all_groups:
|
||||
if not quiet:
|
||||
typer.echo(f"\n{'='*60}")
|
||||
typer.echo(f"GROUP: {group_name}")
|
||||
typer.echo(f"{'='*60}")
|
||||
|
||||
group_source = ExamplesSource(root=examples_path, groups=[group_name])
|
||||
items = group_source.discover()
|
||||
|
||||
if not items:
|
||||
if not quiet:
|
||||
typer.echo(" No items found, skipping")
|
||||
group_summaries.append({
|
||||
'group': group_name,
|
||||
'total': 0, 'passed': 0, 'failed': 0,
|
||||
'skipped': 0, 'timeout': 0, 'not_run': 0, 'xfail': 0,
|
||||
})
|
||||
continue
|
||||
|
||||
group_report_dir = output_dir / group_name
|
||||
group_report_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
executor = SuiteExecutor(
|
||||
suite='examples',
|
||||
source_path=examples_path,
|
||||
report_dir=group_report_dir,
|
||||
timeout=timeout,
|
||||
stream_output=False,
|
||||
generate_json=True,
|
||||
generate_md=True,
|
||||
generate_csv=True,
|
||||
groups=[group_name],
|
||||
)
|
||||
|
||||
icons = {'passed': '✅', 'failed': '❌', 'skipped': '⏭️', 'timeout': '⏱️', 'xfail': '⚠️'}
|
||||
|
||||
def on_start(item, idx, total):
|
||||
if not quiet:
|
||||
typer.echo(f" [{idx}/{total}] {item.display_name} ", nl=False)
|
||||
|
||||
def on_end(result: RunResult, idx, total):
|
||||
if not quiet:
|
||||
icon = icons.get(result.status, '❓')
|
||||
typer.echo(f"{icon}")
|
||||
|
||||
report = executor.run(items=items, on_item_start=on_start, on_item_end=on_end)
|
||||
totals = report.totals
|
||||
|
||||
group_summaries.append({'group': group_name, **totals})
|
||||
|
||||
for key in overall_results:
|
||||
if key in totals:
|
||||
overall_results[key] += totals[key]
|
||||
|
||||
if not quiet:
|
||||
typer.echo(
|
||||
f"\n Summary: ✅{totals['passed']} ❌{totals['failed']} "
|
||||
f"⏭️{totals['skipped']} ⏱️{totals['timeout']} ⚠️{totals.get('xfail', 0)}"
|
||||
)
|
||||
|
||||
# Final report
|
||||
typer.echo("\n" + "=" * 80)
|
||||
typer.echo("FINAL REPORT - ALL GROUPS")
|
||||
typer.echo("=" * 80)
|
||||
|
||||
typer.echo(f"\n{'Group':<20} {'Total':>8} {'Passed':>8} {'Failed':>8} {'Skip':>8} {'Timeout':>8} {'XFail':>8}")
|
||||
typer.echo("-" * 80)
|
||||
|
||||
for gs in sorted(group_summaries, key=lambda x: x['group']):
|
||||
typer.echo(
|
||||
f"{gs['group']:<20} {gs['total']:>8} {gs['passed']:>8} {gs['failed']:>8} "
|
||||
f"{gs['skipped']:>8} {gs['timeout']:>8} {gs.get('xfail', 0):>8}"
|
||||
)
|
||||
|
||||
typer.echo("-" * 80)
|
||||
typer.echo(
|
||||
f"{'TOTAL':<20} {overall_results['total']:>8} {overall_results['passed']:>8} "
|
||||
f"{overall_results['failed']:>8} {overall_results['skipped']:>8} "
|
||||
f"{overall_results['timeout']:>8} {overall_results.get('xfail', 0):>8}"
|
||||
)
|
||||
|
||||
# Save final summary
|
||||
output_dir.mkdir(parents=True, exist_ok=True)
|
||||
summary_path = output_dir / 'final_summary.json'
|
||||
with open(summary_path, 'w') as f:
|
||||
json.dump({
|
||||
'timestamp': datetime.now().isoformat(),
|
||||
'overall': overall_results,
|
||||
'groups': group_summaries,
|
||||
}, f, indent=2)
|
||||
|
||||
typer.echo(f"\n📁 Final summary saved to: {summary_path}")
|
||||
typer.echo(f"📁 Individual group reports in: {output_dir}")
|
||||
|
||||
# Exit code
|
||||
if overall_results['failed'] > 0 or overall_results['timeout'] > 0:
|
||||
raise typer.Exit(1)
|
||||
raise typer.Exit(0)
|
||||
|
||||
|
||||
@app.command("report")
|
||||
def examples_report(
|
||||
report_dir: Optional[Path] = typer.Argument(
|
||||
None,
|
||||
help="Report directory path (default: auto-detect latest)",
|
||||
),
|
||||
base_dir: Optional[Path] = typer.Option(
|
||||
None,
|
||||
"--base-dir", "-b",
|
||||
help="Base reports directory (default: ~/Downloads/reports/examples)",
|
||||
),
|
||||
limit: int = typer.Option(
|
||||
50,
|
||||
"--limit", "-n",
|
||||
help="Limit rows in tables (0 = no limit)",
|
||||
),
|
||||
errors: bool = typer.Option(
|
||||
True,
|
||||
"--errors/--no-errors",
|
||||
help="Show error grouping section",
|
||||
),
|
||||
wide: bool = typer.Option(
|
||||
False,
|
||||
"--wide/--no-wide",
|
||||
help="Wide output: full error messages and item lists",
|
||||
),
|
||||
show_paths: bool = typer.Option(
|
||||
True,
|
||||
"--show-paths/--no-show-paths",
|
||||
help="Show affected item paths in error groups",
|
||||
),
|
||||
match: Optional[List[str]] = typer.Option(
|
||||
None,
|
||||
"--match", "-m",
|
||||
help="Filter by status (repeatable: --match failed --match timeout)",
|
||||
),
|
||||
group: Optional[List[str]] = typer.Option(
|
||||
None,
|
||||
"--group", "-g",
|
||||
help="Filter by group name (repeatable)",
|
||||
),
|
||||
contains: Optional[str] = typer.Option(
|
||||
None,
|
||||
"--contains", "-c",
|
||||
help="Filter items whose error contains text (case-insensitive)",
|
||||
),
|
||||
open_dir: bool = typer.Option(
|
||||
False,
|
||||
"--open",
|
||||
help="Show report artifacts listing",
|
||||
),
|
||||
output_format: str = typer.Option(
|
||||
"table",
|
||||
"--format", "-f",
|
||||
help="Output format: table or json",
|
||||
),
|
||||
):
|
||||
"""
|
||||
View execution report with failures and error grouping.
|
||||
|
||||
Examples:
|
||||
praisonai examples report # View latest report
|
||||
praisonai examples report ./my-report # View specific report
|
||||
praisonai examples report --wide --limit 0 # Full details
|
||||
praisonai examples report --match failed # Only failures
|
||||
praisonai examples report --format json # JSON output
|
||||
"""
|
||||
# Lazy import to avoid loading at CLI startup
|
||||
from praisonai.suite_runner.report_viewer import view_report
|
||||
|
||||
output, exit_code = view_report(
|
||||
report_dir=report_dir,
|
||||
suite="examples",
|
||||
base_dir=base_dir,
|
||||
limit=limit,
|
||||
show_errors=errors,
|
||||
wide=wide,
|
||||
show_paths=show_paths,
|
||||
match_statuses=match,
|
||||
match_groups=group,
|
||||
contains=contains,
|
||||
show_artifacts=open_dir,
|
||||
output_format=output_format,
|
||||
)
|
||||
|
||||
typer.echo(output)
|
||||
raise typer.Exit(exit_code)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
|
||||
@@ -57,7 +57,7 @@ class CloudDeployer:
|
||||
file.write("FROM python:3.11-slim\n")
|
||||
file.write("WORKDIR /app\n")
|
||||
file.write("COPY . .\n")
|
||||
file.write("RUN pip install flask praisonai==3.5.7 gunicorn markdown\n")
|
||||
file.write("RUN pip install flask praisonai==3.6.2 gunicorn markdown\n")
|
||||
file.write("EXPOSE 8080\n")
|
||||
file.write('CMD ["gunicorn", "-b", "0.0.0.0:8080", "api:app"]\n')
|
||||
|
||||
|
||||
@@ -0,0 +1,63 @@
|
||||
"""
|
||||
Suite Runner - Unified execution engine for examples and docs.
|
||||
|
||||
Provides shared infrastructure for running code suites with:
|
||||
- Subprocess execution with streaming and timeouts
|
||||
- Report generation (JSON, Markdown, CSV)
|
||||
- Deterministic ordering and logging
|
||||
|
||||
All imports are lazy-loaded for zero performance impact.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
__all__ = [
|
||||
# Dataclasses
|
||||
"RunItem",
|
||||
"RunResult",
|
||||
"RunReport",
|
||||
# Runner
|
||||
"ScriptRunner",
|
||||
# Reporter
|
||||
"SuiteReporter",
|
||||
# Discovery
|
||||
"FileDiscovery",
|
||||
# Sources
|
||||
"ExamplesSource",
|
||||
"DocsSource",
|
||||
# Executor
|
||||
"SuiteExecutor",
|
||||
]
|
||||
|
||||
|
||||
def __getattr__(name: str):
|
||||
"""Lazy import for zero startup cost."""
|
||||
if name in ("RunItem", "RunResult", "RunReport"):
|
||||
from .models import RunItem, RunResult, RunReport
|
||||
return {"RunItem": RunItem, "RunResult": RunResult, "RunReport": RunReport}[name]
|
||||
|
||||
if name == "ScriptRunner":
|
||||
from .runner import ScriptRunner
|
||||
return ScriptRunner
|
||||
|
||||
if name == "SuiteReporter":
|
||||
from .reporter import SuiteReporter
|
||||
return SuiteReporter
|
||||
|
||||
if name == "FileDiscovery":
|
||||
from .discovery import FileDiscovery
|
||||
return FileDiscovery
|
||||
|
||||
if name == "ExamplesSource":
|
||||
from .examples_source import ExamplesSource
|
||||
return ExamplesSource
|
||||
|
||||
if name == "DocsSource":
|
||||
from .docs_source import DocsSource
|
||||
return DocsSource
|
||||
|
||||
if name == "SuiteExecutor":
|
||||
from .executor import SuiteExecutor
|
||||
return SuiteExecutor
|
||||
|
||||
raise AttributeError(f"module {__name__!r} has no attribute {name!r}")
|
||||
@@ -0,0 +1,215 @@
|
||||
"""
|
||||
Unified File Discovery for suite execution.
|
||||
|
||||
Provides common file discovery logic with filtering and grouping.
|
||||
Used by both examples and docs runners.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import fnmatch
|
||||
from pathlib import Path
|
||||
from typing import List, Optional, Set
|
||||
|
||||
|
||||
class FileDiscovery:
|
||||
"""
|
||||
Discovers files in a directory with filtering.
|
||||
|
||||
Unified discovery for both examples and docs suites.
|
||||
"""
|
||||
|
||||
# Directories to always ignore
|
||||
IGNORE_DIRS: Set[str] = {
|
||||
'__pycache__', '.git', '.svn', '.hg',
|
||||
'venv', '.venv', 'env', '.env',
|
||||
'node_modules', '.tox', '.pytest_cache',
|
||||
'.mypy_cache', '.ruff_cache', 'dist', 'build',
|
||||
'egg-info', '.eggs', '.netlify',
|
||||
}
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
root: Path,
|
||||
extensions: Optional[List[str]] = None,
|
||||
include_patterns: Optional[List[str]] = None,
|
||||
exclude_patterns: Optional[List[str]] = None,
|
||||
ignore_underscore: bool = True,
|
||||
):
|
||||
"""
|
||||
Initialize discovery.
|
||||
|
||||
Args:
|
||||
root: Root directory to search.
|
||||
extensions: File extensions to include (e.g., ['.py', '.md']).
|
||||
include_patterns: Glob patterns to include.
|
||||
exclude_patterns: Glob patterns to exclude.
|
||||
ignore_underscore: Skip files starting with underscore.
|
||||
"""
|
||||
self.root = Path(root).resolve()
|
||||
self.extensions = extensions or ['.py']
|
||||
self.include_patterns = include_patterns or []
|
||||
self.exclude_patterns = exclude_patterns or []
|
||||
self.ignore_underscore = ignore_underscore
|
||||
|
||||
def discover(self) -> List[Path]:
|
||||
"""
|
||||
Discover all matching files, sorted deterministically.
|
||||
|
||||
Returns:
|
||||
List of file paths.
|
||||
"""
|
||||
files = []
|
||||
|
||||
for ext in self.extensions:
|
||||
pattern = f"*{ext}" if not ext.startswith('*') else ext
|
||||
|
||||
for path in self.root.rglob(pattern):
|
||||
if not path.is_file():
|
||||
continue
|
||||
|
||||
# Skip files starting with underscore
|
||||
if self.ignore_underscore and path.name.startswith('_'):
|
||||
continue
|
||||
|
||||
# Skip ignored directories
|
||||
rel_parts = path.relative_to(self.root).parts
|
||||
if any(
|
||||
part in self.IGNORE_DIRS or
|
||||
part.endswith('.egg-info') or
|
||||
part.startswith('.')
|
||||
for part in rel_parts
|
||||
):
|
||||
continue
|
||||
|
||||
# Apply include patterns (if any)
|
||||
rel_path = path.relative_to(self.root).as_posix()
|
||||
if self.include_patterns:
|
||||
if not any(
|
||||
fnmatch.fnmatch(rel_path, p) or
|
||||
fnmatch.fnmatch(path.name, p)
|
||||
for p in self.include_patterns
|
||||
):
|
||||
continue
|
||||
|
||||
# Apply exclude patterns
|
||||
if self.exclude_patterns:
|
||||
if any(fnmatch.fnmatch(rel_path, p) for p in self.exclude_patterns):
|
||||
continue
|
||||
|
||||
files.append(path)
|
||||
|
||||
# Sort for deterministic order
|
||||
return sorted(files, key=lambda p: p.relative_to(self.root).as_posix())
|
||||
|
||||
def discover_by_group(
|
||||
self,
|
||||
groups: Optional[List[str]] = None,
|
||||
) -> dict:
|
||||
"""
|
||||
Discover files grouped by top-level directory.
|
||||
|
||||
Args:
|
||||
groups: Specific groups to include (None = all).
|
||||
|
||||
Returns:
|
||||
Dict mapping group name to list of files.
|
||||
"""
|
||||
all_files = self.discover()
|
||||
grouped = {}
|
||||
|
||||
for path in all_files:
|
||||
rel_path = path.relative_to(self.root)
|
||||
|
||||
# Get group (first directory component or 'root')
|
||||
if len(rel_path.parts) > 1:
|
||||
group = rel_path.parts[0]
|
||||
else:
|
||||
group = "root"
|
||||
|
||||
# Filter by requested groups
|
||||
if groups and group not in groups:
|
||||
continue
|
||||
|
||||
if group not in grouped:
|
||||
grouped[group] = []
|
||||
grouped[group].append(path)
|
||||
|
||||
return grouped
|
||||
|
||||
def get_groups(self) -> List[str]:
|
||||
"""
|
||||
Get list of available groups (top-level directories).
|
||||
|
||||
Returns:
|
||||
Sorted list of group names.
|
||||
"""
|
||||
groups = set()
|
||||
|
||||
for path in self.discover():
|
||||
rel_path = path.relative_to(self.root)
|
||||
if len(rel_path.parts) > 1:
|
||||
groups.add(rel_path.parts[0])
|
||||
else:
|
||||
groups.add("root")
|
||||
|
||||
return sorted(groups)
|
||||
|
||||
@staticmethod
|
||||
def get_group_for_path(path: Path, root: Path) -> str:
|
||||
"""
|
||||
Get group name for a file path.
|
||||
|
||||
Args:
|
||||
path: File path.
|
||||
root: Root directory.
|
||||
|
||||
Returns:
|
||||
Group name (first directory or 'root').
|
||||
"""
|
||||
try:
|
||||
rel_path = path.relative_to(root)
|
||||
if len(rel_path.parts) > 1:
|
||||
return rel_path.parts[0]
|
||||
return "root"
|
||||
except ValueError:
|
||||
return "unknown"
|
||||
|
||||
|
||||
def get_pythonpath_for_dev(start_path: Path) -> List[str]:
|
||||
"""
|
||||
Get PYTHONPATH additions for local package imports in dev mode.
|
||||
|
||||
Searches up from start_path to find src/ directories.
|
||||
|
||||
Args:
|
||||
start_path: Starting path to search from.
|
||||
|
||||
Returns:
|
||||
List of paths to add to PYTHONPATH.
|
||||
"""
|
||||
paths = []
|
||||
current = Path(start_path).resolve()
|
||||
|
||||
for _ in range(10): # Look up to 10 levels
|
||||
# Check for src directory
|
||||
src_dir = current / "src"
|
||||
if src_dir.exists():
|
||||
# Add all immediate subdirs of src
|
||||
for subdir in src_dir.iterdir():
|
||||
if subdir.is_dir() and not subdir.name.startswith('.'):
|
||||
paths.append(str(subdir))
|
||||
break
|
||||
|
||||
# Also check for common package directories
|
||||
for pkg in ["praisonai-agents", "praisonai"]:
|
||||
pkg_dir = current / "src" / pkg
|
||||
if pkg_dir.exists():
|
||||
paths.append(str(pkg_dir))
|
||||
|
||||
parent = current.parent
|
||||
if parent == current:
|
||||
break
|
||||
current = parent
|
||||
|
||||
return paths
|
||||
@@ -0,0 +1,357 @@
|
||||
"""
|
||||
Docs Source - Adapter for docs code execution suite.
|
||||
|
||||
Extracts and prepares Python code blocks from Markdown/MDX documentation.
|
||||
Uses the shared suite_runner engine.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
import tempfile
|
||||
import textwrap
|
||||
from pathlib import Path
|
||||
from typing import List, Optional, Tuple
|
||||
|
||||
from .models import RunItem
|
||||
from .discovery import FileDiscovery, get_pythonpath_for_dev
|
||||
|
||||
|
||||
# Patterns for fence detection
|
||||
FENCE_START_PATTERN = re.compile(r'^(\s*)(`{3,}|~{3,})(\w*)\s*(.*)$')
|
||||
|
||||
# Directive pattern for HTML comments
|
||||
DIRECTIVE_PATTERN = re.compile(
|
||||
r'<!--\s*praisonai:\s*([^>]+)\s*-->',
|
||||
re.IGNORECASE
|
||||
)
|
||||
DIRECTIVE_KV_PATTERN = re.compile(r'(\w+)=([^\s]+)')
|
||||
|
||||
# Interactive detection
|
||||
INPUT_PATTERN = re.compile(r'\binput\s*\(|\bgetpass\s*\.')
|
||||
|
||||
# Import detection
|
||||
IMPORT_PATTERN = re.compile(r'^(?:from\s+\S+\s+)?import\s+', re.MULTILINE)
|
||||
|
||||
# Terminal action detection
|
||||
TERMINAL_ACTIONS = [
|
||||
r'\.start\s*\(',
|
||||
r'\.run\s*\(',
|
||||
r'\.chat\s*\(',
|
||||
r'\bprint\s*\(',
|
||||
r'asyncio\.run\s*\(',
|
||||
r'if\s+__name__\s*==',
|
||||
r'\.execute\s*\(',
|
||||
r'\.main\s*\(',
|
||||
]
|
||||
TERMINAL_PATTERN = re.compile('|'.join(TERMINAL_ACTIONS))
|
||||
|
||||
# Agent-centric detection patterns
|
||||
AGENT_PATTERN = re.compile(r'\bAgent\s*\(')
|
||||
AGENTS_PATTERN = re.compile(r'\b(?:Agents|PraisonAIAgents)\s*\(')
|
||||
WORKFLOW_PATTERN = re.compile(r'\bWorkflow\s*\(')
|
||||
|
||||
|
||||
class DocsSource:
|
||||
"""
|
||||
Extracts and prepares Python code blocks from documentation.
|
||||
|
||||
Parses directives from HTML comments:
|
||||
- <!-- praisonai: runnable=true -->
|
||||
- <!-- praisonai: skip=true -->
|
||||
- <!-- praisonai: timeout=120 -->
|
||||
- <!-- praisonai: require_env=KEY1,KEY2 -->
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
root: Path,
|
||||
languages: Optional[List[str]] = None,
|
||||
include_patterns: Optional[List[str]] = None,
|
||||
exclude_patterns: Optional[List[str]] = None,
|
||||
groups: Optional[List[str]] = None,
|
||||
workspace_dir: Optional[Path] = None,
|
||||
):
|
||||
"""
|
||||
Initialize docs source.
|
||||
|
||||
Args:
|
||||
root: Root documentation directory.
|
||||
languages: Languages to extract (default: ['python']).
|
||||
include_patterns: Glob patterns to include.
|
||||
exclude_patterns: Glob patterns to exclude.
|
||||
groups: Specific groups (subdirs) to include.
|
||||
workspace_dir: Directory for extracted scripts.
|
||||
"""
|
||||
self.root = Path(root).resolve()
|
||||
self.languages = languages or ['python']
|
||||
self.include_patterns = include_patterns
|
||||
self.exclude_patterns = exclude_patterns
|
||||
self.groups = groups
|
||||
|
||||
if workspace_dir:
|
||||
self.workspace_dir = Path(workspace_dir)
|
||||
else:
|
||||
self.workspace_dir = Path(tempfile.mkdtemp(prefix="praisonai_docs_"))
|
||||
|
||||
self._discovery = FileDiscovery(
|
||||
root=self.root,
|
||||
extensions=['.md', '.mdx'],
|
||||
include_patterns=include_patterns,
|
||||
exclude_patterns=exclude_patterns,
|
||||
ignore_underscore=False,
|
||||
)
|
||||
|
||||
def discover(self) -> List[RunItem]:
|
||||
"""
|
||||
Discover all code blocks and create RunItems.
|
||||
|
||||
Returns:
|
||||
List of RunItem objects.
|
||||
"""
|
||||
if self.groups:
|
||||
grouped = self._discovery.discover_by_group(self.groups)
|
||||
files = []
|
||||
for group_files in grouped.values():
|
||||
files.extend(group_files)
|
||||
files = sorted(files, key=lambda p: p.relative_to(self.root).as_posix())
|
||||
else:
|
||||
files = self._discovery.discover()
|
||||
|
||||
items = []
|
||||
for doc_path in files:
|
||||
blocks = self._extract_blocks(doc_path)
|
||||
items.extend(blocks)
|
||||
|
||||
return items
|
||||
|
||||
def _extract_blocks(self, doc_path: Path) -> List[RunItem]:
|
||||
"""Extract code blocks from a document."""
|
||||
try:
|
||||
content = doc_path.read_text(encoding='utf-8', errors='ignore')
|
||||
except Exception:
|
||||
return []
|
||||
|
||||
group = FileDiscovery.get_group_for_path(doc_path, self.root)
|
||||
blocks = []
|
||||
lines = content.split('\n')
|
||||
|
||||
i = 0
|
||||
block_index = 0
|
||||
|
||||
while i < len(lines):
|
||||
line = lines[i]
|
||||
match = FENCE_START_PATTERN.match(line)
|
||||
|
||||
if match:
|
||||
indent = match.group(1)
|
||||
fence = match.group(2)
|
||||
language = match.group(3) or ''
|
||||
title = match.group(4).strip() if match.group(4) else None
|
||||
|
||||
# Find closing fence
|
||||
code_lines = []
|
||||
fence_start_line = i + 1 # 1-indexed
|
||||
j = i + 1
|
||||
|
||||
while j < len(lines):
|
||||
close_line = lines[j]
|
||||
# Check if this is the closing fence
|
||||
if close_line.strip() == fence or (
|
||||
close_line.startswith(indent) and
|
||||
close_line[len(indent):].startswith(fence) and
|
||||
close_line[len(indent):].strip() == fence
|
||||
):
|
||||
break
|
||||
code_lines.append(close_line)
|
||||
j += 1
|
||||
|
||||
if j < len(lines): # Found closing fence
|
||||
code_content = '\n'.join(code_lines)
|
||||
code_start = fence_start_line + 1
|
||||
code_end = code_start + len(code_lines) - 1 if code_lines else code_start
|
||||
|
||||
# Check if language matches target
|
||||
if language.lower() in [lang.lower() for lang in self.languages]:
|
||||
# Look for directive in previous lines
|
||||
directive = self._parse_directive(lines, i)
|
||||
|
||||
# Create item
|
||||
item = self._create_item(
|
||||
doc_path=doc_path,
|
||||
group=group,
|
||||
block_index=block_index,
|
||||
language=language.lower(),
|
||||
code=code_content,
|
||||
line_start=code_start,
|
||||
line_end=code_end,
|
||||
title=title,
|
||||
directive=directive,
|
||||
)
|
||||
blocks.append(item)
|
||||
block_index += 1
|
||||
|
||||
i = j + 1
|
||||
continue
|
||||
|
||||
i += 1
|
||||
|
||||
return blocks
|
||||
|
||||
def _parse_directive(self, lines: List[str], fence_line: int) -> dict:
|
||||
"""Parse directive from lines before fence."""
|
||||
directive = {
|
||||
'runnable': None,
|
||||
'skip': False,
|
||||
'timeout': None,
|
||||
'require_env': [],
|
||||
}
|
||||
|
||||
# Check up to 5 lines before
|
||||
start = max(0, fence_line - 5)
|
||||
pre_text = '\n'.join(lines[start:fence_line])
|
||||
|
||||
match = DIRECTIVE_PATTERN.search(pre_text)
|
||||
if match:
|
||||
directive_text = match.group(1)
|
||||
for kv_match in DIRECTIVE_KV_PATTERN.finditer(directive_text):
|
||||
key, value = kv_match.group(1).lower(), kv_match.group(2)
|
||||
|
||||
if key == 'runnable':
|
||||
directive['runnable'] = value.lower() in ('true', '1', 'yes')
|
||||
elif key == 'skip':
|
||||
directive['skip'] = value.lower() in ('true', '1', 'yes')
|
||||
elif key == 'timeout':
|
||||
try:
|
||||
directive['timeout'] = int(value)
|
||||
except ValueError:
|
||||
pass
|
||||
elif key == 'require_env':
|
||||
directive['require_env'] = [e.strip() for e in value.split(',') if e.strip()]
|
||||
|
||||
return directive
|
||||
|
||||
def _create_item(
|
||||
self,
|
||||
doc_path: Path,
|
||||
group: str,
|
||||
block_index: int,
|
||||
language: str,
|
||||
code: str,
|
||||
line_start: int,
|
||||
line_end: int,
|
||||
title: Optional[str],
|
||||
directive: dict,
|
||||
) -> RunItem:
|
||||
"""Create RunItem from extracted code block."""
|
||||
rel_path = doc_path.relative_to(self.root).as_posix()
|
||||
item_id = f"{rel_path.replace('/', '__').replace('.mdx', '').replace('.md', '')}__{block_index}"
|
||||
|
||||
# Dedent code for MDX indentation
|
||||
dedented_code = textwrap.dedent(code)
|
||||
|
||||
# Classify runnable status
|
||||
runnable, runnable_decision = self._classify(dedented_code, directive)
|
||||
|
||||
# Check for interactive
|
||||
is_interactive = bool(INPUT_PATTERN.search(dedented_code))
|
||||
if is_interactive and runnable:
|
||||
runnable = False
|
||||
runnable_decision = "interactive_input"
|
||||
|
||||
# Handle directive skip
|
||||
skip = directive.get('skip', False)
|
||||
skip_reason = None
|
||||
if skip:
|
||||
runnable = False
|
||||
runnable_decision = "directive_skip"
|
||||
skip_reason = "Directive skip"
|
||||
|
||||
# Detect agent-centric usage
|
||||
uses_agent = bool(AGENT_PATTERN.search(dedented_code))
|
||||
uses_agents = bool(AGENTS_PATTERN.search(dedented_code))
|
||||
uses_workflow = bool(WORKFLOW_PATTERN.search(dedented_code))
|
||||
|
||||
# Write script to workspace
|
||||
script_path = None
|
||||
if runnable:
|
||||
script_path = self._write_script(item_id, dedented_code)
|
||||
|
||||
return RunItem(
|
||||
item_id=item_id,
|
||||
suite="docs",
|
||||
group=group,
|
||||
source_path=doc_path,
|
||||
block_index=block_index,
|
||||
language=language,
|
||||
line_start=line_start,
|
||||
line_end=line_end,
|
||||
code=dedented_code,
|
||||
script_path=script_path,
|
||||
runnable=runnable,
|
||||
runnable_decision=runnable_decision,
|
||||
skip=skip,
|
||||
skip_reason=skip_reason,
|
||||
timeout=directive.get('timeout'),
|
||||
require_env=directive.get('require_env', []),
|
||||
is_interactive=is_interactive,
|
||||
title=title,
|
||||
uses_agent=uses_agent,
|
||||
uses_agents=uses_agents,
|
||||
uses_workflow=uses_workflow,
|
||||
)
|
||||
|
||||
def _classify(self, code: str, directive: dict) -> Tuple[bool, str]:
|
||||
"""
|
||||
Classify if code block is runnable.
|
||||
|
||||
Returns:
|
||||
(is_runnable, reason)
|
||||
"""
|
||||
# Directive override
|
||||
if directive.get('runnable') is True:
|
||||
return True, "directive_runnable"
|
||||
if directive.get('runnable') is False:
|
||||
return False, "directive_not_runnable"
|
||||
|
||||
# Too short
|
||||
lines = [line for line in code.strip().split('\n') if line.strip()]
|
||||
if len(lines) < 2:
|
||||
return False, "too_short"
|
||||
|
||||
# Check for imports
|
||||
has_import = bool(IMPORT_PATTERN.search(code))
|
||||
|
||||
# Check for terminal action
|
||||
has_terminal = bool(TERMINAL_PATTERN.search(code))
|
||||
|
||||
# Heuristic: needs both import and terminal action
|
||||
if has_import and has_terminal:
|
||||
return True, "heuristic_standalone"
|
||||
|
||||
if not has_import:
|
||||
return False, "no_import_partial"
|
||||
|
||||
if not has_terminal:
|
||||
return False, "no_terminal_action_partial"
|
||||
|
||||
return False, "partial"
|
||||
|
||||
def _write_script(self, item_id: str, code: str) -> Path:
|
||||
"""Write code to workspace as executable script."""
|
||||
self.workspace_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
# Sanitize filename
|
||||
safe_id = item_id.replace('/', '_').replace('\\', '_')
|
||||
script_path = self.workspace_dir / f"{safe_id}.py"
|
||||
|
||||
script_path.write_text(code, encoding='utf-8')
|
||||
return script_path
|
||||
|
||||
def get_groups(self) -> List[str]:
|
||||
"""Get available groups."""
|
||||
return self._discovery.get_groups()
|
||||
|
||||
def get_pythonpath(self) -> List[str]:
|
||||
"""Get PYTHONPATH additions for dev mode."""
|
||||
return get_pythonpath_for_dev(self.root)
|
||||
@@ -0,0 +1,173 @@
|
||||
"""
|
||||
Examples Source - Adapter for examples suite.
|
||||
|
||||
Discovers and prepares Python example files for execution.
|
||||
Uses the shared suite_runner engine.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
from pathlib import Path
|
||||
from typing import List, Optional
|
||||
|
||||
from .models import RunItem
|
||||
from .discovery import FileDiscovery, get_pythonpath_for_dev
|
||||
|
||||
|
||||
# Directive parsing regex
|
||||
DIRECTIVE_PATTERN = re.compile(r'^#\s*praisonai:\s*(\w+)=(.+)$', re.MULTILINE)
|
||||
INPUT_PATTERN = re.compile(r'\binput\s*\(')
|
||||
|
||||
# Agent-centric detection patterns
|
||||
AGENT_PATTERN = re.compile(r'\bAgent\s*\(')
|
||||
AGENTS_PATTERN = re.compile(r'\b(?:Agents|PraisonAIAgents)\s*\(')
|
||||
WORKFLOW_PATTERN = re.compile(r'\bWorkflow\s*\(')
|
||||
|
||||
|
||||
class ExamplesSource:
|
||||
"""
|
||||
Discovers and prepares example Python files.
|
||||
|
||||
Parses directives from comments:
|
||||
- # praisonai: skip=true
|
||||
- # praisonai: timeout=120
|
||||
- # praisonai: require_env=KEY1,KEY2
|
||||
- # praisonai: xfail=reason
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
root: Path,
|
||||
include_patterns: Optional[List[str]] = None,
|
||||
exclude_patterns: Optional[List[str]] = None,
|
||||
groups: Optional[List[str]] = None,
|
||||
):
|
||||
"""
|
||||
Initialize examples source.
|
||||
|
||||
Args:
|
||||
root: Root examples directory.
|
||||
include_patterns: Glob patterns to include.
|
||||
exclude_patterns: Glob patterns to exclude.
|
||||
groups: Specific groups (subdirs) to include.
|
||||
"""
|
||||
self.root = Path(root).resolve()
|
||||
self.include_patterns = include_patterns
|
||||
self.exclude_patterns = exclude_patterns
|
||||
self.groups = groups
|
||||
|
||||
self._discovery = FileDiscovery(
|
||||
root=self.root,
|
||||
extensions=['.py'],
|
||||
include_patterns=include_patterns,
|
||||
exclude_patterns=exclude_patterns,
|
||||
ignore_underscore=True,
|
||||
)
|
||||
|
||||
def discover(self) -> List[RunItem]:
|
||||
"""
|
||||
Discover all example files and create RunItems.
|
||||
|
||||
Returns:
|
||||
List of RunItem objects.
|
||||
"""
|
||||
if self.groups:
|
||||
grouped = self._discovery.discover_by_group(self.groups)
|
||||
files = []
|
||||
for group_files in grouped.values():
|
||||
files.extend(group_files)
|
||||
files = sorted(files, key=lambda p: p.relative_to(self.root).as_posix())
|
||||
else:
|
||||
files = self._discovery.discover()
|
||||
|
||||
items = []
|
||||
for path in files:
|
||||
item = self._create_item(path)
|
||||
items.append(item)
|
||||
|
||||
return items
|
||||
|
||||
def _create_item(self, path: Path) -> RunItem:
|
||||
"""Create RunItem from example file."""
|
||||
group = FileDiscovery.get_group_for_path(path, self.root)
|
||||
rel_path = path.relative_to(self.root).as_posix()
|
||||
item_id = rel_path.replace('/', '__').replace('.py', '')
|
||||
|
||||
# Read file content
|
||||
try:
|
||||
content = path.read_text(encoding='utf-8', errors='ignore')
|
||||
except Exception:
|
||||
content = ""
|
||||
|
||||
# Parse directives from first 30 lines
|
||||
lines = content.split('\n')[:30]
|
||||
header = '\n'.join(lines)
|
||||
|
||||
skip = False
|
||||
skip_reason = None
|
||||
timeout = None
|
||||
require_env = []
|
||||
xfail = None
|
||||
|
||||
for match in DIRECTIVE_PATTERN.finditer(header):
|
||||
key, value = match.group(1), match.group(2).strip()
|
||||
|
||||
if key == 'skip':
|
||||
skip = value.lower() in ('true', '1', 'yes')
|
||||
if skip:
|
||||
skip_reason = "skip=true directive"
|
||||
elif key == 'timeout':
|
||||
try:
|
||||
timeout = int(value)
|
||||
except ValueError:
|
||||
pass
|
||||
elif key == 'require_env':
|
||||
require_env = [k.strip() for k in value.split(',') if k.strip()]
|
||||
elif key == 'xfail':
|
||||
xfail = value
|
||||
|
||||
# Detect interactive input() calls
|
||||
is_interactive = bool(INPUT_PATTERN.search(content))
|
||||
if is_interactive and not skip:
|
||||
skip = True
|
||||
skip_reason = "Interactive example (contains input())"
|
||||
|
||||
# Detect agent-centric usage
|
||||
uses_agent = bool(AGENT_PATTERN.search(content))
|
||||
uses_agents = bool(AGENTS_PATTERN.search(content))
|
||||
uses_workflow = bool(WORKFLOW_PATTERN.search(content))
|
||||
|
||||
# Determine runnable status
|
||||
runnable = not skip
|
||||
runnable_decision = "runnable" if runnable else (skip_reason or "skipped")
|
||||
|
||||
return RunItem(
|
||||
item_id=item_id,
|
||||
suite="examples",
|
||||
group=group,
|
||||
source_path=path,
|
||||
block_index=0,
|
||||
language="python",
|
||||
code=content,
|
||||
script_path=path, # Examples run directly
|
||||
runnable=runnable,
|
||||
runnable_decision=runnable_decision,
|
||||
skip=skip,
|
||||
skip_reason=skip_reason,
|
||||
timeout=timeout,
|
||||
require_env=require_env,
|
||||
xfail=xfail,
|
||||
is_interactive=is_interactive,
|
||||
uses_agent=uses_agent,
|
||||
uses_agents=uses_agents,
|
||||
uses_workflow=uses_workflow,
|
||||
)
|
||||
|
||||
def get_groups(self) -> List[str]:
|
||||
"""Get available groups."""
|
||||
return self._discovery.get_groups()
|
||||
|
||||
def get_pythonpath(self) -> List[str]:
|
||||
"""Get PYTHONPATH additions for dev mode."""
|
||||
return get_pythonpath_for_dev(self.root)
|
||||
@@ -0,0 +1,232 @@
|
||||
"""
|
||||
Unified Suite Executor.
|
||||
|
||||
Main orchestrator that runs items from any source (examples or docs)
|
||||
using the shared runner and reporter.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import sys
|
||||
from pathlib import Path
|
||||
from typing import List, Optional, Callable
|
||||
|
||||
from .models import RunItem, RunResult, RunReport
|
||||
from .runner import ScriptRunner
|
||||
from .reporter import SuiteReporter
|
||||
|
||||
|
||||
class SuiteExecutor:
|
||||
"""
|
||||
Unified executor for running code suites.
|
||||
|
||||
Works with any source (examples, docs) that provides RunItems.
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
suite: str,
|
||||
source_path: Path,
|
||||
timeout: int = 60,
|
||||
fail_fast: bool = False,
|
||||
stream_output: bool = True,
|
||||
max_items: Optional[int] = None,
|
||||
require_env: Optional[List[str]] = None,
|
||||
env_overrides: Optional[dict] = None,
|
||||
report_dir: Optional[Path] = None,
|
||||
generate_json: bool = True,
|
||||
generate_md: bool = True,
|
||||
generate_csv: bool = True,
|
||||
python_executable: Optional[str] = None,
|
||||
pythonpath_additions: Optional[List[str]] = None,
|
||||
groups: Optional[List[str]] = None,
|
||||
):
|
||||
"""
|
||||
Initialize executor.
|
||||
|
||||
Args:
|
||||
suite: Suite name ("examples" or "docs").
|
||||
source_path: Path to source directory.
|
||||
timeout: Per-item timeout in seconds.
|
||||
fail_fast: Stop on first failure.
|
||||
stream_output: Stream output to terminal.
|
||||
max_items: Maximum items to process.
|
||||
require_env: Global required env vars.
|
||||
env_overrides: Environment variable overrides.
|
||||
report_dir: Directory for reports.
|
||||
generate_json: Generate JSON report.
|
||||
generate_md: Generate Markdown report.
|
||||
generate_csv: Generate CSV report.
|
||||
python_executable: Python interpreter to use.
|
||||
pythonpath_additions: Paths to add to PYTHONPATH.
|
||||
groups: Specific groups to run.
|
||||
"""
|
||||
self.suite = suite
|
||||
self.source_path = Path(source_path).resolve()
|
||||
self.timeout = timeout
|
||||
self.fail_fast = fail_fast
|
||||
self.stream_output = stream_output
|
||||
self.max_items = max_items
|
||||
self.require_env = require_env or []
|
||||
self.env_overrides = env_overrides or {}
|
||||
self.report_dir = Path(report_dir) if report_dir else None
|
||||
self.generate_json = generate_json
|
||||
self.generate_md = generate_md
|
||||
self.generate_csv = generate_csv
|
||||
self.python_executable = python_executable or sys.executable
|
||||
self.pythonpath_additions = pythonpath_additions or []
|
||||
self.groups = groups
|
||||
|
||||
def run(
|
||||
self,
|
||||
items: List[RunItem],
|
||||
on_item_start: Optional[Callable[[RunItem, int, int], None]] = None,
|
||||
on_item_end: Optional[Callable[[RunResult, int, int], None]] = None,
|
||||
on_output: Optional[Callable[[str, str], None]] = None,
|
||||
) -> RunReport:
|
||||
"""
|
||||
Run all items and return report.
|
||||
|
||||
Args:
|
||||
items: List of RunItems to execute.
|
||||
on_item_start: Callback when starting an item.
|
||||
on_item_end: Callback when item completes.
|
||||
on_output: Callback for streaming output.
|
||||
|
||||
Returns:
|
||||
RunReport with all results.
|
||||
"""
|
||||
# Apply max_items limit
|
||||
if self.max_items and len(items) > self.max_items:
|
||||
items = items[:self.max_items]
|
||||
|
||||
# Check global required env
|
||||
import os
|
||||
for key in self.require_env:
|
||||
if not os.environ.get(key):
|
||||
# Return empty report with error
|
||||
return RunReport(
|
||||
results=[],
|
||||
suite=self.suite,
|
||||
source_path=self.source_path,
|
||||
cli_args=[f"--require-env={key} (missing)"],
|
||||
groups_run=self.groups or [],
|
||||
)
|
||||
|
||||
# Create runner
|
||||
runner = ScriptRunner(
|
||||
timeout=self.timeout,
|
||||
capture_output=True,
|
||||
stream_output=self.stream_output,
|
||||
env_overrides=self.env_overrides,
|
||||
pythonpath_additions=self.pythonpath_additions,
|
||||
python_executable=self.python_executable,
|
||||
)
|
||||
|
||||
results: List[RunResult] = []
|
||||
total = len(items)
|
||||
|
||||
for idx, item in enumerate(items, 1):
|
||||
if on_item_start:
|
||||
on_item_start(item, idx, total)
|
||||
|
||||
# Handle non-runnable items
|
||||
if not item.runnable:
|
||||
result = RunResult(
|
||||
item_id=item.item_id,
|
||||
suite=item.suite,
|
||||
group=item.group,
|
||||
source_path=item.source_path,
|
||||
block_index=item.block_index,
|
||||
language=item.language,
|
||||
line_start=item.line_start,
|
||||
line_end=item.line_end,
|
||||
runnable_decision=item.runnable_decision,
|
||||
status="skipped" if item.skip else "not_run",
|
||||
skip_reason=item.skip_reason or item.runnable_decision,
|
||||
code_hash=item.code_hash,
|
||||
)
|
||||
results.append(result)
|
||||
|
||||
if on_item_end:
|
||||
on_item_end(result, idx, total)
|
||||
continue
|
||||
|
||||
# Check required env from item
|
||||
if item.require_env:
|
||||
missing = runner.check_required_env(item.require_env)
|
||||
if missing:
|
||||
result = RunResult(
|
||||
item_id=item.item_id,
|
||||
suite=item.suite,
|
||||
group=item.group,
|
||||
source_path=item.source_path,
|
||||
block_index=item.block_index,
|
||||
language=item.language,
|
||||
line_start=item.line_start,
|
||||
line_end=item.line_end,
|
||||
runnable_decision=item.runnable_decision,
|
||||
status="skipped",
|
||||
skip_reason=f"Missing env: {missing}",
|
||||
env_requirements=",".join(item.require_env),
|
||||
code_hash=item.code_hash,
|
||||
)
|
||||
results.append(result)
|
||||
|
||||
if on_item_end:
|
||||
on_item_end(result, idx, total)
|
||||
continue
|
||||
|
||||
# Execute
|
||||
result = runner.run(
|
||||
item,
|
||||
on_output=on_output if self.stream_output else None,
|
||||
)
|
||||
results.append(result)
|
||||
|
||||
if on_item_end:
|
||||
on_item_end(result, idx, total)
|
||||
|
||||
# Check fail-fast
|
||||
if self.fail_fast and result.status == "failed":
|
||||
# Mark remaining as skipped
|
||||
for remaining_item in items[idx:]:
|
||||
results.append(RunResult(
|
||||
item_id=remaining_item.item_id,
|
||||
suite=remaining_item.suite,
|
||||
group=remaining_item.group,
|
||||
source_path=remaining_item.source_path,
|
||||
block_index=remaining_item.block_index,
|
||||
language=remaining_item.language,
|
||||
runnable_decision=remaining_item.runnable_decision,
|
||||
status="skipped",
|
||||
skip_reason="Skipped due to --fail-fast",
|
||||
code_hash=remaining_item.code_hash,
|
||||
))
|
||||
break
|
||||
|
||||
# Create report
|
||||
report = RunReport(
|
||||
results=results,
|
||||
suite=self.suite,
|
||||
source_path=self.source_path,
|
||||
groups_run=self.groups or [],
|
||||
)
|
||||
|
||||
# Generate reports if configured
|
||||
if self.report_dir:
|
||||
reporter = SuiteReporter(self.report_dir)
|
||||
reporter.save_logs(results)
|
||||
|
||||
if self.generate_json:
|
||||
reporter.generate_json(report)
|
||||
|
||||
if self.generate_md:
|
||||
reporter.generate_markdown(report)
|
||||
|
||||
if self.generate_csv:
|
||||
reporter.generate_csv(report)
|
||||
|
||||
report.report_path = self.report_dir
|
||||
|
||||
return report
|
||||
@@ -0,0 +1,254 @@
|
||||
"""
|
||||
Shared data models for suite execution.
|
||||
|
||||
Provides unified dataclasses used by both examples and docs runners.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import platform
|
||||
import subprocess
|
||||
from dataclasses import dataclass, field
|
||||
from datetime import datetime, timezone
|
||||
from pathlib import Path
|
||||
from typing import List, Optional, Dict, Any
|
||||
|
||||
|
||||
@dataclass
|
||||
class RunItem:
|
||||
"""
|
||||
Represents a single runnable item (example file or doc code block).
|
||||
|
||||
Unified structure for both examples and docs suites.
|
||||
"""
|
||||
|
||||
# Identity
|
||||
item_id: str # Unique stable identifier
|
||||
suite: str # "examples" or "docs"
|
||||
group: str # Top-level directory/category
|
||||
|
||||
# Source
|
||||
source_path: Path # Original file path
|
||||
block_index: int = 0 # For docs: which code block (0 for examples)
|
||||
language: str = "python"
|
||||
line_start: int = 0
|
||||
line_end: int = 0
|
||||
|
||||
# Content
|
||||
code: str = "" # The actual code content
|
||||
script_path: Optional[Path] = None # Path to materialized script
|
||||
|
||||
# Classification
|
||||
runnable: bool = True
|
||||
runnable_decision: str = "" # Reason for classification
|
||||
|
||||
# Directives
|
||||
skip: bool = False
|
||||
skip_reason: Optional[str] = None
|
||||
timeout: Optional[int] = None
|
||||
require_env: List[str] = field(default_factory=list)
|
||||
xfail: Optional[str] = None
|
||||
is_interactive: bool = False
|
||||
|
||||
# Metadata
|
||||
title: Optional[str] = None
|
||||
|
||||
# Agent-centric detection
|
||||
uses_agent: bool = False # Agent() class
|
||||
uses_agents: bool = False # Agents() / PraisonAIAgents class
|
||||
uses_workflow: bool = False # Workflow class
|
||||
|
||||
@property
|
||||
def agent_type(self) -> str:
|
||||
"""Get agent type used in this item."""
|
||||
types = []
|
||||
if self.uses_agent:
|
||||
types.append("Agent")
|
||||
if self.uses_agents:
|
||||
types.append("Agents")
|
||||
if self.uses_workflow:
|
||||
types.append("Workflow")
|
||||
return ",".join(types) if types else "none"
|
||||
|
||||
@property
|
||||
def code_hash(self) -> str:
|
||||
"""Generate short hash of code content."""
|
||||
return hashlib.sha256(self.code.encode()).hexdigest()[:16]
|
||||
|
||||
@property
|
||||
def display_name(self) -> str:
|
||||
"""Human-readable display name."""
|
||||
if self.suite == "docs":
|
||||
return f"{self.source_path.name}:{self.block_index}"
|
||||
return self.source_path.name
|
||||
|
||||
|
||||
@dataclass
|
||||
class RunResult:
|
||||
"""
|
||||
Result of executing a single RunItem.
|
||||
|
||||
Unified structure for both examples and docs execution results.
|
||||
"""
|
||||
|
||||
# Link to item
|
||||
item_id: str
|
||||
suite: str
|
||||
group: str
|
||||
source_path: Path
|
||||
block_index: int = 0
|
||||
language: str = "python"
|
||||
line_start: int = 0
|
||||
line_end: int = 0
|
||||
|
||||
# Classification
|
||||
runnable_decision: str = ""
|
||||
|
||||
# Execution result
|
||||
status: str = "not_run" # passed, failed, skipped, timeout, not_run, xfail
|
||||
exit_code: int = 0
|
||||
duration_seconds: float = 0.0
|
||||
start_time: str = ""
|
||||
end_time: str = ""
|
||||
|
||||
# Error info
|
||||
skip_reason: Optional[str] = None
|
||||
error_type: Optional[str] = None
|
||||
error_message: Optional[str] = None
|
||||
|
||||
# Output
|
||||
stdout: str = ""
|
||||
stderr: str = ""
|
||||
stdout_path: Optional[str] = None
|
||||
stderr_path: Optional[str] = None
|
||||
|
||||
# Execution context
|
||||
python_executable: str = ""
|
||||
cwd: str = ""
|
||||
env_requirements: str = "" # Comma-separated
|
||||
code_hash: str = ""
|
||||
|
||||
@property
|
||||
def display_name(self) -> str:
|
||||
"""Human-readable display name."""
|
||||
if self.suite == "docs":
|
||||
return f"{self.source_path.name}:{self.block_index}"
|
||||
return self.source_path.name
|
||||
|
||||
|
||||
@dataclass
|
||||
class RunReport:
|
||||
"""
|
||||
Complete report from a suite execution run.
|
||||
|
||||
Aggregates all results with metadata.
|
||||
"""
|
||||
|
||||
# Results
|
||||
results: List[RunResult] = field(default_factory=list)
|
||||
|
||||
# Context
|
||||
suite: str = "" # "examples", "docs", or "mixed"
|
||||
source_path: Path = field(default_factory=Path)
|
||||
report_path: Optional[Path] = None
|
||||
cli_args: List[str] = field(default_factory=list)
|
||||
|
||||
# Metadata (populated on creation)
|
||||
timestamp: str = field(default_factory=lambda: datetime.now(timezone.utc).isoformat())
|
||||
platform_info: str = field(default_factory=lambda: f"{platform.system()}-{platform.release()}-{platform.machine()}")
|
||||
python_version: str = field(default_factory=lambda: platform.python_version())
|
||||
python_executable: str = ""
|
||||
praisonai_version: str = ""
|
||||
git_commit: Optional[str] = None
|
||||
|
||||
# Grouping
|
||||
groups_run: List[str] = field(default_factory=list)
|
||||
|
||||
def __post_init__(self):
|
||||
"""Populate version info."""
|
||||
import sys
|
||||
self.python_executable = sys.executable
|
||||
|
||||
try:
|
||||
from praisonai.version import __version__
|
||||
self.praisonai_version = __version__
|
||||
except ImportError:
|
||||
self.praisonai_version = "unknown"
|
||||
|
||||
if self.git_commit is None:
|
||||
try:
|
||||
result = subprocess.run(
|
||||
['git', 'rev-parse', '--short', 'HEAD'],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=5,
|
||||
)
|
||||
if result.returncode == 0:
|
||||
self.git_commit = result.stdout.strip()
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
@property
|
||||
def totals(self) -> Dict[str, int]:
|
||||
"""Calculate totals by status."""
|
||||
counts = {
|
||||
"passed": 0,
|
||||
"failed": 0,
|
||||
"skipped": 0,
|
||||
"timeout": 0,
|
||||
"not_run": 0,
|
||||
"xfail": 0,
|
||||
"total": 0,
|
||||
}
|
||||
for r in self.results:
|
||||
counts["total"] += 1
|
||||
if r.status in counts:
|
||||
counts[r.status] += 1
|
||||
return counts
|
||||
|
||||
def to_dict(self) -> Dict[str, Any]:
|
||||
"""Convert to dictionary for JSON serialization."""
|
||||
return {
|
||||
"metadata": {
|
||||
"suite": self.suite,
|
||||
"source_path": str(self.source_path),
|
||||
"timestamp": self.timestamp,
|
||||
"platform": self.platform_info,
|
||||
"python_version": self.python_version,
|
||||
"python_executable": self.python_executable,
|
||||
"praisonai_version": self.praisonai_version,
|
||||
"git_commit": self.git_commit,
|
||||
"cli_args": self.cli_args,
|
||||
"groups_run": self.groups_run,
|
||||
"totals": self.totals,
|
||||
},
|
||||
"results": [
|
||||
{
|
||||
"item_id": r.item_id,
|
||||
"suite": r.suite,
|
||||
"group": r.group,
|
||||
"source_path": str(r.source_path),
|
||||
"block_index": r.block_index,
|
||||
"language": r.language,
|
||||
"line_start": r.line_start,
|
||||
"line_end": r.line_end,
|
||||
"runnable_decision": r.runnable_decision,
|
||||
"status": r.status,
|
||||
"exit_code": r.exit_code,
|
||||
"duration_seconds": r.duration_seconds,
|
||||
"start_time": r.start_time,
|
||||
"end_time": r.end_time,
|
||||
"skip_reason": r.skip_reason,
|
||||
"error_type": r.error_type,
|
||||
"error_message": r.error_message,
|
||||
"stdout_path": r.stdout_path,
|
||||
"stderr_path": r.stderr_path,
|
||||
"python_executable": r.python_executable,
|
||||
"cwd": r.cwd,
|
||||
"env_requirements": r.env_requirements,
|
||||
"code_hash": r.code_hash,
|
||||
}
|
||||
for r in self.results
|
||||
],
|
||||
}
|
||||
@@ -0,0 +1,793 @@
|
||||
"""
|
||||
Report Viewer for suite execution reports.
|
||||
|
||||
Provides CLI-friendly viewing of JSON/CSV reports with:
|
||||
- Latest report auto-detection
|
||||
- Failure tables with filtering
|
||||
- Error grouping and deduplication
|
||||
- ASCII table rendering (no external deps)
|
||||
|
||||
Used by both `praisonai examples report` and `praisonai docs report`.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import re
|
||||
from dataclasses import dataclass, field
|
||||
from pathlib import Path
|
||||
from typing import Dict, List, Optional, Tuple
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# Data Models
|
||||
# =============================================================================
|
||||
|
||||
@dataclass
|
||||
class ReportItem:
|
||||
"""Normalized report item from JSON/CSV."""
|
||||
item_id: str
|
||||
suite: str
|
||||
group: str
|
||||
source_path: str
|
||||
block_index: int = 0
|
||||
status: str = "not_run"
|
||||
duration_seconds: float = 0.0
|
||||
exit_code: int = 0
|
||||
error_type: Optional[str] = None
|
||||
error_message: Optional[str] = None
|
||||
error_signature: str = "" # Normalized for grouping
|
||||
skip_reason: Optional[str] = None
|
||||
stdout_path: Optional[str] = None
|
||||
stderr_path: Optional[str] = None
|
||||
start_time: str = ""
|
||||
end_time: str = ""
|
||||
runnable_decision: str = ""
|
||||
|
||||
@property
|
||||
def display_name(self) -> str:
|
||||
"""Human-readable display name."""
|
||||
path = Path(self.source_path)
|
||||
if self.suite == "docs" and self.block_index > 0:
|
||||
return f"{path.name}:{self.block_index}"
|
||||
return path.name
|
||||
|
||||
|
||||
@dataclass
|
||||
class ReportMetadata:
|
||||
"""Report metadata from JSON."""
|
||||
suite: str = ""
|
||||
source_path: str = ""
|
||||
timestamp: str = ""
|
||||
platform: str = ""
|
||||
python_version: str = ""
|
||||
python_executable: str = ""
|
||||
praisonai_version: str = ""
|
||||
git_commit: Optional[str] = None
|
||||
groups_run: List[str] = field(default_factory=list)
|
||||
totals: Dict[str, int] = field(default_factory=dict)
|
||||
|
||||
|
||||
@dataclass
|
||||
class ErrorGroup:
|
||||
"""Group of items with similar errors."""
|
||||
signature: str
|
||||
error_type: str
|
||||
sample_message: str
|
||||
count: int = 0
|
||||
items: List[ReportItem] = field(default_factory=list)
|
||||
|
||||
|
||||
@dataclass
|
||||
class LoadedReport:
|
||||
"""Complete loaded and normalized report."""
|
||||
metadata: ReportMetadata
|
||||
items: List[ReportItem]
|
||||
report_dir: Path
|
||||
|
||||
@property
|
||||
def failures(self) -> List[ReportItem]:
|
||||
"""Get failed/timeout/error items."""
|
||||
return [i for i in self.items if i.status in ("failed", "timeout", "error")]
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# Error Signature Normalization
|
||||
# =============================================================================
|
||||
|
||||
# Patterns to strip from error messages for grouping
|
||||
_STRIP_PATTERNS = [
|
||||
# File paths
|
||||
(r'File "[^"]+[/\\][^"]+\.py"', 'File "...py"'),
|
||||
(r'/[^\s:]+\.py', '...py'),
|
||||
(r'[A-Z]:\\[^\s:]+\.py', '...py'),
|
||||
# Line numbers
|
||||
(r', line \d+', ', line N'),
|
||||
(r'line \d+', 'line N'),
|
||||
# Memory addresses
|
||||
(r'0x[0-9a-fA-F]+', '0x...'),
|
||||
# Request IDs, UUIDs
|
||||
(r'[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}', 'UUID'),
|
||||
(r'req_[a-zA-Z0-9]+', 'req_...'),
|
||||
# Timestamps
|
||||
(r'\d{4}-\d{2}-\d{2}[T ]\d{2}:\d{2}:\d{2}[^\s]*', 'TIMESTAMP'),
|
||||
# Numbers in brackets
|
||||
(r'\[\d+\]', '[N]'),
|
||||
# Port numbers
|
||||
(r':\d{4,5}(?=/|$|\s)', ':PORT'),
|
||||
# Whitespace normalization
|
||||
(r'\s+', ' '),
|
||||
]
|
||||
|
||||
|
||||
def normalize_error_signature(error_type: Optional[str], error_message: Optional[str]) -> str:
|
||||
"""
|
||||
Create normalized error signature for grouping.
|
||||
|
||||
Strips variable parts (paths, line numbers, addresses, timestamps)
|
||||
to group similar errors together.
|
||||
"""
|
||||
if not error_message and not error_type:
|
||||
return "unknown_error"
|
||||
|
||||
# Start with error type
|
||||
sig_parts = []
|
||||
if error_type:
|
||||
sig_parts.append(error_type)
|
||||
|
||||
if error_message:
|
||||
msg = error_message.strip()
|
||||
# Apply all normalization patterns
|
||||
for pattern, replacement in _STRIP_PATTERNS:
|
||||
msg = re.sub(pattern, replacement, msg)
|
||||
# Take first 100 chars of normalized message
|
||||
msg = msg[:100].strip()
|
||||
if msg:
|
||||
sig_parts.append(msg)
|
||||
|
||||
return ": ".join(sig_parts) if sig_parts else "unknown_error"
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# Report Discovery
|
||||
# =============================================================================
|
||||
|
||||
def find_latest_report_dir(suite: str, base_dir: Optional[Path] = None) -> Optional[Path]:
|
||||
"""
|
||||
Find the latest report directory for a suite.
|
||||
|
||||
Args:
|
||||
suite: "examples" or "docs"
|
||||
base_dir: Base reports directory (default: ~/Downloads/reports/<suite>)
|
||||
|
||||
Returns:
|
||||
Path to latest report directory, or None if not found.
|
||||
"""
|
||||
if base_dir is None:
|
||||
base_dir = Path.home() / "Downloads" / "reports" / suite
|
||||
|
||||
if not base_dir.exists():
|
||||
return None
|
||||
|
||||
# Check if base_dir itself contains report.json (user passed specific run dir)
|
||||
if (base_dir / "report.json").exists():
|
||||
return base_dir
|
||||
|
||||
# Find subdirectories with report.json
|
||||
candidates = []
|
||||
for subdir in base_dir.iterdir():
|
||||
if subdir.is_dir():
|
||||
# Check for report.json in subdir or in group subdirs
|
||||
if (subdir / "report.json").exists():
|
||||
candidates.append(subdir)
|
||||
else:
|
||||
# Check if this is a run dir with group subdirs
|
||||
for group_dir in subdir.iterdir():
|
||||
if group_dir.is_dir() and (group_dir / "report.json").exists():
|
||||
candidates.append(subdir)
|
||||
break
|
||||
|
||||
if not candidates:
|
||||
return None
|
||||
|
||||
# Sort by mtime (newest first)
|
||||
candidates.sort(key=lambda p: p.stat().st_mtime, reverse=True)
|
||||
return candidates[0]
|
||||
|
||||
|
||||
def find_report_files(report_dir: Path) -> List[Path]:
|
||||
"""
|
||||
Find all report.json files in a report directory.
|
||||
|
||||
Handles both single-group and multi-group report structures.
|
||||
"""
|
||||
report_files = []
|
||||
|
||||
# Check for report.json directly in report_dir
|
||||
direct = report_dir / "report.json"
|
||||
if direct.exists():
|
||||
report_files.append(direct)
|
||||
|
||||
# Check for group subdirectories
|
||||
for subdir in report_dir.iterdir():
|
||||
if subdir.is_dir():
|
||||
group_report = subdir / "report.json"
|
||||
if group_report.exists():
|
||||
report_files.append(group_report)
|
||||
|
||||
return report_files
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# Report Loading
|
||||
# =============================================================================
|
||||
|
||||
def load_report(report_dir: Path) -> LoadedReport:
|
||||
"""
|
||||
Load and normalize a report from a directory.
|
||||
|
||||
Aggregates all group reports if multiple exist.
|
||||
|
||||
Args:
|
||||
report_dir: Path to report run directory.
|
||||
|
||||
Returns:
|
||||
LoadedReport with normalized items.
|
||||
|
||||
Raises:
|
||||
FileNotFoundError: If no report.json found.
|
||||
ValueError: If report cannot be parsed.
|
||||
"""
|
||||
report_files = find_report_files(report_dir)
|
||||
|
||||
if not report_files:
|
||||
raise FileNotFoundError(f"No report.json found in {report_dir}")
|
||||
|
||||
all_items: List[ReportItem] = []
|
||||
metadata = ReportMetadata()
|
||||
groups_seen = set()
|
||||
|
||||
for report_file in report_files:
|
||||
try:
|
||||
data = json.loads(report_file.read_text(encoding='utf-8'))
|
||||
except json.JSONDecodeError as e:
|
||||
raise ValueError(f"Invalid JSON in {report_file}: {e}")
|
||||
|
||||
# Parse metadata from first report
|
||||
if not metadata.suite and "metadata" in data:
|
||||
meta = data["metadata"]
|
||||
metadata = ReportMetadata(
|
||||
suite=meta.get("suite", ""),
|
||||
source_path=meta.get("source_path", ""),
|
||||
timestamp=meta.get("timestamp", ""),
|
||||
platform=meta.get("platform", ""),
|
||||
python_version=meta.get("python_version", ""),
|
||||
python_executable=meta.get("python_executable", ""),
|
||||
praisonai_version=meta.get("praisonai_version", ""),
|
||||
git_commit=meta.get("git_commit"),
|
||||
groups_run=meta.get("groups_run", []),
|
||||
totals=meta.get("totals", {}),
|
||||
)
|
||||
|
||||
# Parse results
|
||||
for r in data.get("results", []):
|
||||
error_sig = normalize_error_signature(
|
||||
r.get("error_type"),
|
||||
r.get("error_message")
|
||||
)
|
||||
|
||||
item = ReportItem(
|
||||
item_id=r.get("item_id", ""),
|
||||
suite=r.get("suite", ""),
|
||||
group=r.get("group", ""),
|
||||
source_path=r.get("source_path", ""),
|
||||
block_index=r.get("block_index", 0),
|
||||
status=r.get("status", "not_run"),
|
||||
duration_seconds=r.get("duration_seconds", 0.0),
|
||||
exit_code=r.get("exit_code", 0),
|
||||
error_type=r.get("error_type"),
|
||||
error_message=r.get("error_message"),
|
||||
error_signature=error_sig,
|
||||
skip_reason=r.get("skip_reason"),
|
||||
stdout_path=r.get("stdout_path"),
|
||||
stderr_path=r.get("stderr_path"),
|
||||
start_time=r.get("start_time", ""),
|
||||
end_time=r.get("end_time", ""),
|
||||
runnable_decision=r.get("runnable_decision", ""),
|
||||
)
|
||||
all_items.append(item)
|
||||
groups_seen.add(item.group)
|
||||
|
||||
# Update metadata with aggregated groups
|
||||
if groups_seen:
|
||||
metadata.groups_run = sorted(groups_seen)
|
||||
|
||||
# Recalculate totals from all items
|
||||
totals = {"passed": 0, "failed": 0, "skipped": 0, "timeout": 0, "not_run": 0, "xfail": 0, "total": 0}
|
||||
for item in all_items:
|
||||
totals["total"] += 1
|
||||
if item.status in totals:
|
||||
totals[item.status] += 1
|
||||
metadata.totals = totals
|
||||
|
||||
return LoadedReport(
|
||||
metadata=metadata,
|
||||
items=all_items,
|
||||
report_dir=report_dir,
|
||||
)
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# Error Grouping
|
||||
# =============================================================================
|
||||
|
||||
def group_errors(items: List[ReportItem]) -> List[ErrorGroup]:
|
||||
"""
|
||||
Group items by error signature.
|
||||
|
||||
Returns groups sorted by count (descending), then signature.
|
||||
"""
|
||||
groups: Dict[str, ErrorGroup] = {}
|
||||
|
||||
for item in items:
|
||||
if item.status not in ("failed", "timeout", "error"):
|
||||
continue
|
||||
|
||||
sig = item.error_signature or "unknown_error"
|
||||
|
||||
if sig not in groups:
|
||||
groups[sig] = ErrorGroup(
|
||||
signature=sig,
|
||||
error_type=item.error_type or "",
|
||||
sample_message=item.error_message or "",
|
||||
)
|
||||
|
||||
groups[sig].count += 1
|
||||
groups[sig].items.append(item)
|
||||
|
||||
# Sort by count desc, then signature asc
|
||||
sorted_groups = sorted(
|
||||
groups.values(),
|
||||
key=lambda g: (-g.count, g.signature)
|
||||
)
|
||||
|
||||
return sorted_groups
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# ASCII Table Rendering
|
||||
# =============================================================================
|
||||
|
||||
def _truncate(text: str, max_len: int, ellipsis: str = "...") -> str:
|
||||
"""Truncate text with ellipsis if too long."""
|
||||
if not text:
|
||||
return ""
|
||||
text = text.replace("\n", " ").replace("\r", "")
|
||||
if len(text) <= max_len:
|
||||
return text
|
||||
return text[:max_len - len(ellipsis)] + ellipsis
|
||||
|
||||
|
||||
def _render_table(
|
||||
headers: List[str],
|
||||
rows: List[List[str]],
|
||||
col_widths: Optional[List[int]] = None,
|
||||
max_width: int = 120,
|
||||
) -> str:
|
||||
"""
|
||||
Render a simple ASCII table.
|
||||
|
||||
Args:
|
||||
headers: Column headers.
|
||||
rows: List of row data (list of strings).
|
||||
col_widths: Optional fixed column widths.
|
||||
max_width: Maximum total table width.
|
||||
|
||||
Returns:
|
||||
Formatted table string.
|
||||
"""
|
||||
if not headers:
|
||||
return ""
|
||||
|
||||
num_cols = len(headers)
|
||||
|
||||
# Calculate column widths
|
||||
if col_widths is None:
|
||||
col_widths = [len(h) for h in headers]
|
||||
for row in rows:
|
||||
for i, cell in enumerate(row[:num_cols]):
|
||||
col_widths[i] = max(col_widths[i], len(str(cell)))
|
||||
|
||||
# Ensure we don't exceed max_width
|
||||
total_width = sum(col_widths) + (num_cols - 1) * 3 + 4 # separators + borders
|
||||
if total_width > max_width and col_widths:
|
||||
# Shrink largest columns proportionally
|
||||
excess = total_width - max_width
|
||||
while excess > 0 and max(col_widths) > 10:
|
||||
max_idx = col_widths.index(max(col_widths))
|
||||
col_widths[max_idx] -= 1
|
||||
excess -= 1
|
||||
|
||||
lines = []
|
||||
|
||||
# Header separator
|
||||
sep = "+" + "+".join("-" * (w + 2) for w in col_widths) + "+"
|
||||
lines.append(sep)
|
||||
|
||||
# Header row
|
||||
header_cells = [
|
||||
f" {_truncate(h, w).ljust(w)} " for h, w in zip(headers, col_widths)
|
||||
]
|
||||
lines.append("|" + "|".join(header_cells) + "|")
|
||||
lines.append(sep)
|
||||
|
||||
# Data rows
|
||||
for row in rows:
|
||||
cells = []
|
||||
for i, w in enumerate(col_widths):
|
||||
cell = str(row[i]) if i < len(row) else ""
|
||||
cells.append(f" {_truncate(cell, w).ljust(w)} ")
|
||||
lines.append("|" + "|".join(cells) + "|")
|
||||
|
||||
lines.append(sep)
|
||||
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# Report Rendering
|
||||
# =============================================================================
|
||||
|
||||
STATUS_ICONS = {
|
||||
"passed": "✅",
|
||||
"failed": "❌",
|
||||
"skipped": "⏭️",
|
||||
"timeout": "⏱️",
|
||||
"not_run": "📝",
|
||||
"xfail": "⚠️",
|
||||
"error": "💥",
|
||||
}
|
||||
|
||||
|
||||
def render_overview(report: LoadedReport, wide: bool = False) -> str:
|
||||
"""Render overview section with totals and metadata."""
|
||||
meta = report.metadata
|
||||
totals = meta.totals
|
||||
|
||||
lines = [
|
||||
"",
|
||||
"=" * 60,
|
||||
f" REPORT: {meta.suite.upper()} EXECUTION",
|
||||
"=" * 60,
|
||||
"",
|
||||
f" Timestamp: {meta.timestamp}",
|
||||
f" Platform: {meta.platform}",
|
||||
f" Python: {meta.python_version}",
|
||||
f" PraisonAI: {meta.praisonai_version}",
|
||||
]
|
||||
|
||||
if meta.git_commit:
|
||||
lines.append(f" Git Commit: {meta.git_commit}")
|
||||
|
||||
if meta.groups_run:
|
||||
groups_str = ", ".join(meta.groups_run[:10])
|
||||
if len(meta.groups_run) > 10:
|
||||
groups_str += f" (+{len(meta.groups_run) - 10} more)"
|
||||
lines.append(f" Groups: {groups_str}")
|
||||
|
||||
lines.append(f" Report Dir: {report.report_dir}")
|
||||
lines.append("")
|
||||
|
||||
# Summary table
|
||||
lines.append(" SUMMARY")
|
||||
lines.append(" " + "-" * 40)
|
||||
|
||||
total = totals.get("total", 0)
|
||||
for status in ["passed", "failed", "timeout", "skipped", "not_run", "xfail"]:
|
||||
count = totals.get(status, 0)
|
||||
icon = STATUS_ICONS.get(status, "?")
|
||||
pct = f"({count * 100 / total:.1f}%)" if total > 0 else ""
|
||||
lines.append(f" {icon} {status.capitalize():12} {count:5} {pct}")
|
||||
|
||||
lines.append(" " + "-" * 40)
|
||||
lines.append(f" {'Total':15} {total:5}")
|
||||
lines.append("")
|
||||
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
def render_failures_table(
|
||||
report: LoadedReport,
|
||||
limit: int = 50,
|
||||
match_statuses: Optional[List[str]] = None,
|
||||
match_groups: Optional[List[str]] = None,
|
||||
contains: Optional[str] = None,
|
||||
wide: bool = False,
|
||||
) -> str:
|
||||
"""
|
||||
Render failures table.
|
||||
|
||||
Args:
|
||||
report: Loaded report.
|
||||
limit: Max rows to show (0 = no limit).
|
||||
match_statuses: Filter by status(es).
|
||||
match_groups: Filter by group(s).
|
||||
contains: Filter by error message substring.
|
||||
wide: Show full error messages.
|
||||
|
||||
Returns:
|
||||
Formatted table string.
|
||||
"""
|
||||
# Default to failure statuses
|
||||
if match_statuses is None:
|
||||
match_statuses = ["failed", "timeout", "error"]
|
||||
|
||||
# Filter items
|
||||
items = []
|
||||
for item in report.items:
|
||||
if item.status not in match_statuses:
|
||||
continue
|
||||
if match_groups and item.group not in match_groups:
|
||||
continue
|
||||
if contains:
|
||||
msg = (item.error_message or "").lower()
|
||||
if contains.lower() not in msg:
|
||||
continue
|
||||
items.append(item)
|
||||
|
||||
if not items:
|
||||
return "\n No matching failures found.\n"
|
||||
|
||||
# Apply limit
|
||||
total_count = len(items)
|
||||
if limit > 0 and len(items) > limit:
|
||||
items = items[:limit]
|
||||
|
||||
lines = [
|
||||
"",
|
||||
f" FAILURES ({total_count} total, showing {len(items)})",
|
||||
" " + "-" * 50,
|
||||
"",
|
||||
]
|
||||
|
||||
# Build table
|
||||
headers = ["Group", "Item", "Status", "Duration", "Error"]
|
||||
rows = []
|
||||
|
||||
error_width = 60 if wide else 30
|
||||
|
||||
for item in items:
|
||||
icon = STATUS_ICONS.get(item.status, "?")
|
||||
duration = f"{item.duration_seconds:.2f}s" if item.duration_seconds else "-"
|
||||
error = item.error_message or item.skip_reason or "-"
|
||||
|
||||
rows.append([
|
||||
item.group,
|
||||
item.display_name,
|
||||
f"{icon} {item.status}",
|
||||
duration,
|
||||
_truncate(error, error_width) if not wide else error[:200],
|
||||
])
|
||||
|
||||
col_widths = [15, 30, 12, 10, error_width]
|
||||
lines.append(_render_table(headers, rows, col_widths if not wide else None))
|
||||
|
||||
if limit > 0 and total_count > limit:
|
||||
lines.append(f"\n ... and {total_count - limit} more (use --limit 0 to show all)")
|
||||
|
||||
lines.append("")
|
||||
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
def render_error_groups(
|
||||
report: LoadedReport,
|
||||
limit: int = 20,
|
||||
wide: bool = False,
|
||||
show_paths: bool = True,
|
||||
) -> str:
|
||||
"""
|
||||
Render error grouping section.
|
||||
|
||||
Args:
|
||||
report: Loaded report.
|
||||
limit: Max groups to show (0 = no limit).
|
||||
wide: Show full error messages and all paths.
|
||||
show_paths: Show affected item paths.
|
||||
|
||||
Returns:
|
||||
Formatted error groups string.
|
||||
"""
|
||||
groups = group_errors(report.items)
|
||||
|
||||
if not groups:
|
||||
return "\n No errors to group.\n"
|
||||
|
||||
total_groups = len(groups)
|
||||
if limit > 0 and len(groups) > limit:
|
||||
groups = groups[:limit]
|
||||
|
||||
lines = [
|
||||
"",
|
||||
f" ERROR GROUPS ({total_groups} unique errors)",
|
||||
" " + "=" * 50,
|
||||
"",
|
||||
]
|
||||
|
||||
for i, grp in enumerate(groups, 1):
|
||||
lines.append(f" [{i}] {grp.error_type or 'Error'} ({grp.count} occurrences)")
|
||||
lines.append(" " + "-" * 40)
|
||||
|
||||
# Show signature/sample message
|
||||
if wide:
|
||||
msg = grp.sample_message or grp.signature
|
||||
lines.append(f" {msg[:500]}")
|
||||
else:
|
||||
lines.append(f" {_truncate(grp.signature, 80)}")
|
||||
|
||||
# Show affected items
|
||||
if show_paths:
|
||||
lines.append("")
|
||||
lines.append(" Affected items:")
|
||||
|
||||
items_to_show = grp.items if wide else grp.items[:5]
|
||||
for item in items_to_show:
|
||||
lines.append(f" - {item.group}/{item.display_name}")
|
||||
|
||||
if not wide and len(grp.items) > 5:
|
||||
lines.append(f" ... and {len(grp.items) - 5} more")
|
||||
|
||||
lines.append("")
|
||||
|
||||
if limit > 0 and total_groups > limit:
|
||||
lines.append(f" ... and {total_groups - limit} more error groups (use --limit 0 to show all)")
|
||||
|
||||
lines.append("")
|
||||
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
def render_artifacts(report: LoadedReport) -> str:
|
||||
"""Render list of report artifacts."""
|
||||
lines = [
|
||||
"",
|
||||
" REPORT ARTIFACTS",
|
||||
" " + "-" * 40,
|
||||
f" Directory: {report.report_dir}",
|
||||
"",
|
||||
]
|
||||
|
||||
# List files
|
||||
for f in sorted(report.report_dir.rglob("*")):
|
||||
if f.is_file():
|
||||
rel = f.relative_to(report.report_dir)
|
||||
size = f.stat().st_size
|
||||
size_str = f"{size:,} bytes" if size < 1024 else f"{size / 1024:.1f} KB"
|
||||
lines.append(f" {rel}: {size_str}")
|
||||
|
||||
lines.append("")
|
||||
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# Main View Function
|
||||
# =============================================================================
|
||||
|
||||
def view_report(
|
||||
report_dir: Optional[Path] = None,
|
||||
suite: str = "examples",
|
||||
base_dir: Optional[Path] = None,
|
||||
limit: int = 50,
|
||||
show_errors: bool = True,
|
||||
wide: bool = False,
|
||||
show_paths: bool = True,
|
||||
match_statuses: Optional[List[str]] = None,
|
||||
match_groups: Optional[List[str]] = None,
|
||||
contains: Optional[str] = None,
|
||||
show_artifacts: bool = False,
|
||||
output_format: str = "table",
|
||||
) -> Tuple[str, int]:
|
||||
"""
|
||||
Main entry point for viewing a report.
|
||||
|
||||
Args:
|
||||
report_dir: Explicit report directory (or None for auto-detect).
|
||||
suite: Suite name for auto-detection.
|
||||
base_dir: Base reports directory for auto-detection.
|
||||
limit: Max rows in tables.
|
||||
show_errors: Show error grouping section.
|
||||
wide: Wide output mode.
|
||||
show_paths: Show paths in error groups.
|
||||
match_statuses: Filter by status(es).
|
||||
match_groups: Filter by group(s).
|
||||
contains: Filter by error message substring.
|
||||
show_artifacts: Show artifact listing.
|
||||
output_format: "table" or "json".
|
||||
|
||||
Returns:
|
||||
Tuple of (output_string, exit_code).
|
||||
Exit code is 0 if no failures, 1 if failures exist.
|
||||
"""
|
||||
# Find report directory
|
||||
if report_dir is None:
|
||||
report_dir = find_latest_report_dir(suite, base_dir)
|
||||
if report_dir is None:
|
||||
return f"Error: No reports found for suite '{suite}'", 1
|
||||
|
||||
if not report_dir.exists():
|
||||
return f"Error: Report directory not found: {report_dir}", 1
|
||||
|
||||
# Load report
|
||||
try:
|
||||
report = load_report(report_dir)
|
||||
except FileNotFoundError as e:
|
||||
return f"Error: {e}", 1
|
||||
except ValueError as e:
|
||||
return f"Error: {e}", 1
|
||||
|
||||
# JSON output
|
||||
if output_format == "json":
|
||||
output = {
|
||||
"metadata": {
|
||||
"suite": report.metadata.suite,
|
||||
"timestamp": report.metadata.timestamp,
|
||||
"totals": report.metadata.totals,
|
||||
"groups_run": report.metadata.groups_run,
|
||||
},
|
||||
"failures": [
|
||||
{
|
||||
"item_id": i.item_id,
|
||||
"group": i.group,
|
||||
"display_name": i.display_name,
|
||||
"status": i.status,
|
||||
"error_type": i.error_type,
|
||||
"error_message": i.error_message,
|
||||
}
|
||||
for i in report.failures
|
||||
],
|
||||
"error_groups": [
|
||||
{
|
||||
"signature": g.signature,
|
||||
"error_type": g.error_type,
|
||||
"count": g.count,
|
||||
"items": [i.display_name for i in g.items],
|
||||
}
|
||||
for g in group_errors(report.items)
|
||||
],
|
||||
}
|
||||
return json.dumps(output, indent=2), 0 if not report.failures else 1
|
||||
|
||||
# Table output
|
||||
parts = []
|
||||
|
||||
# Overview
|
||||
parts.append(render_overview(report, wide=wide))
|
||||
|
||||
# Failures table
|
||||
parts.append(render_failures_table(
|
||||
report,
|
||||
limit=limit,
|
||||
match_statuses=match_statuses,
|
||||
match_groups=match_groups,
|
||||
contains=contains,
|
||||
wide=wide,
|
||||
))
|
||||
|
||||
# Error groups
|
||||
if show_errors:
|
||||
parts.append(render_error_groups(
|
||||
report,
|
||||
limit=limit,
|
||||
wide=wide,
|
||||
show_paths=show_paths,
|
||||
))
|
||||
|
||||
# Artifacts
|
||||
if show_artifacts:
|
||||
parts.append(render_artifacts(report))
|
||||
|
||||
output = "\n".join(parts)
|
||||
exit_code = 0 if not report.failures else 1
|
||||
|
||||
return output, exit_code
|
||||
@@ -0,0 +1,301 @@
|
||||
"""
|
||||
Unified Report Builder for suite execution.
|
||||
|
||||
Generates JSON, Markdown, and CSV reports from execution results.
|
||||
Used by both examples and docs runners.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import csv
|
||||
import json
|
||||
from pathlib import Path
|
||||
from typing import List
|
||||
|
||||
from .models import RunResult, RunReport
|
||||
|
||||
|
||||
class SuiteReporter:
|
||||
"""
|
||||
Generates reports from suite execution results.
|
||||
|
||||
Supports JSON, Markdown, and CSV output formats.
|
||||
"""
|
||||
|
||||
def __init__(self, output_dir: Path):
|
||||
"""
|
||||
Initialize reporter.
|
||||
|
||||
Args:
|
||||
output_dir: Directory to write reports to.
|
||||
"""
|
||||
self.output_dir = Path(output_dir)
|
||||
self.output_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
def generate_json(self, report: RunReport) -> Path:
|
||||
"""
|
||||
Generate JSON report.
|
||||
|
||||
Args:
|
||||
report: RunReport with all results.
|
||||
|
||||
Returns:
|
||||
Path to generated JSON file.
|
||||
"""
|
||||
json_path = self.output_dir / "report.json"
|
||||
json_path.write_text(
|
||||
json.dumps(report.to_dict(), indent=2),
|
||||
encoding='utf-8'
|
||||
)
|
||||
return json_path
|
||||
|
||||
def generate_markdown(self, report: RunReport) -> Path:
|
||||
"""
|
||||
Generate Markdown report.
|
||||
|
||||
Args:
|
||||
report: RunReport with all results.
|
||||
|
||||
Returns:
|
||||
Path to generated Markdown file.
|
||||
"""
|
||||
totals = report.totals
|
||||
total_count = sum(totals.values())
|
||||
|
||||
suite_name = report.suite.title() if report.suite else "Suite"
|
||||
|
||||
lines = [
|
||||
f"# {suite_name} Execution Report",
|
||||
"",
|
||||
f"**Generated**: {report.timestamp}",
|
||||
f"**Platform**: {report.platform_info}",
|
||||
f"**Python**: {report.python_version}",
|
||||
f"**Executable**: `{report.python_executable}`",
|
||||
f"**PraisonAI**: {report.praisonai_version}",
|
||||
f"**Source**: `{report.source_path}`",
|
||||
]
|
||||
|
||||
if report.git_commit:
|
||||
lines.append(f"**Git Commit**: {report.git_commit}")
|
||||
|
||||
if report.groups_run:
|
||||
lines.append(f"**Groups**: {', '.join(report.groups_run)}")
|
||||
|
||||
lines.extend([
|
||||
"",
|
||||
"## Summary",
|
||||
"",
|
||||
"| Status | Count |",
|
||||
"|--------|-------|",
|
||||
f"| ✅ Passed | {totals['passed']} |",
|
||||
f"| ❌ Failed | {totals['failed']} |",
|
||||
f"| ⏭️ Skipped | {totals['skipped']} |",
|
||||
f"| ⏱️ Timeout | {totals['timeout']} |",
|
||||
f"| 📝 Not Run | {totals['not_run']} |",
|
||||
f"| ⚠️ XFail | {totals['xfail']} |",
|
||||
f"| **Total** | **{total_count}** |",
|
||||
"",
|
||||
"## Results",
|
||||
"",
|
||||
"| Item | Group | Status | Duration | Reason |",
|
||||
"|------|-------|--------|----------|--------|",
|
||||
])
|
||||
|
||||
status_icons = {
|
||||
"passed": "✅",
|
||||
"failed": "❌",
|
||||
"skipped": "⏭️",
|
||||
"timeout": "⏱️",
|
||||
"not_run": "📝",
|
||||
"xfail": "⚠️",
|
||||
}
|
||||
|
||||
for r in report.results:
|
||||
icon = status_icons.get(r.status, "?")
|
||||
duration = f"{r.duration_seconds:.2f}s" if r.duration_seconds else "-"
|
||||
reason = r.skip_reason or r.runnable_decision or r.error_message or "-"
|
||||
reason = reason[:40] + "..." if len(reason) > 40 else reason
|
||||
display = r.display_name
|
||||
lines.append(
|
||||
f"| `{display}` | {r.group} | {icon} {r.status} | {duration} | {reason} |"
|
||||
)
|
||||
|
||||
# Add failures section
|
||||
failures = [r for r in report.results if r.status == "failed"]
|
||||
if failures:
|
||||
lines.extend([
|
||||
"",
|
||||
"## Failures",
|
||||
"",
|
||||
])
|
||||
for r in failures:
|
||||
lines.extend([
|
||||
f"### {r.display_name}",
|
||||
"",
|
||||
f"**Source**: `{r.source_path}`",
|
||||
f"**Group**: {r.group}",
|
||||
f"**Exit Code**: {r.exit_code}",
|
||||
])
|
||||
if r.line_start:
|
||||
lines.append(f"**Lines**: {r.line_start}-{r.line_end}")
|
||||
lines.append("")
|
||||
|
||||
if r.error_type:
|
||||
lines.append(f"**Error Type**: `{r.error_type}`")
|
||||
if r.error_message:
|
||||
lines.extend([
|
||||
"**Error**:",
|
||||
"```",
|
||||
r.error_message[:500],
|
||||
"```",
|
||||
"",
|
||||
])
|
||||
if r.stderr_path:
|
||||
lines.append(f"**Logs**: `{r.stderr_path}`")
|
||||
lines.append("")
|
||||
|
||||
# Add not_run section (brief)
|
||||
not_run = [r for r in report.results if r.status == "not_run"]
|
||||
if not_run:
|
||||
lines.extend([
|
||||
"",
|
||||
f"## Not Run ({len(not_run)} items)",
|
||||
"",
|
||||
"Partial snippets or items that were not executed:",
|
||||
"",
|
||||
])
|
||||
for r in not_run[:20]: # Limit to first 20
|
||||
lines.append(f"- `{r.display_name}`: {r.runnable_decision}")
|
||||
if len(not_run) > 20:
|
||||
lines.append(f"- ... and {len(not_run) - 20} more")
|
||||
|
||||
md_path = self.output_dir / "report.md"
|
||||
md_path.write_text('\n'.join(lines), encoding='utf-8')
|
||||
return md_path
|
||||
|
||||
def generate_csv(self, report: RunReport) -> Path:
|
||||
"""
|
||||
Generate CSV report.
|
||||
|
||||
One row per item for easy scanning and filtering.
|
||||
|
||||
Args:
|
||||
report: RunReport with all results.
|
||||
|
||||
Returns:
|
||||
Path to generated CSV file.
|
||||
"""
|
||||
csv_path = self.output_dir / "report.csv"
|
||||
|
||||
fieldnames = [
|
||||
"suite",
|
||||
"group",
|
||||
"item_id",
|
||||
"source_path",
|
||||
"block_index",
|
||||
"language",
|
||||
"line_start",
|
||||
"line_end",
|
||||
"runnable_decision",
|
||||
"status",
|
||||
"duration_seconds",
|
||||
"exit_code",
|
||||
"error_type",
|
||||
"error_message",
|
||||
"skip_reason",
|
||||
"stdout_log_path",
|
||||
"stderr_log_path",
|
||||
"started_at",
|
||||
"finished_at",
|
||||
"python_executable",
|
||||
"cwd",
|
||||
"env_requirements",
|
||||
"code_hash",
|
||||
]
|
||||
|
||||
with open(csv_path, 'w', newline='', encoding='utf-8') as f:
|
||||
writer = csv.DictWriter(f, fieldnames=fieldnames)
|
||||
writer.writeheader()
|
||||
|
||||
for r in report.results:
|
||||
writer.writerow({
|
||||
"suite": r.suite,
|
||||
"group": r.group,
|
||||
"item_id": r.item_id,
|
||||
"source_path": str(r.source_path),
|
||||
"block_index": r.block_index,
|
||||
"language": r.language,
|
||||
"line_start": r.line_start,
|
||||
"line_end": r.line_end,
|
||||
"runnable_decision": r.runnable_decision,
|
||||
"status": r.status,
|
||||
"duration_seconds": r.duration_seconds,
|
||||
"exit_code": r.exit_code,
|
||||
"error_type": r.error_type or "",
|
||||
"error_message": (r.error_message or "")[:500], # Limit length
|
||||
"skip_reason": r.skip_reason or "",
|
||||
"stdout_log_path": r.stdout_path or "",
|
||||
"stderr_log_path": r.stderr_path or "",
|
||||
"started_at": r.start_time,
|
||||
"finished_at": r.end_time,
|
||||
"python_executable": r.python_executable,
|
||||
"cwd": r.cwd,
|
||||
"env_requirements": r.env_requirements,
|
||||
"code_hash": r.code_hash,
|
||||
})
|
||||
|
||||
return csv_path
|
||||
|
||||
def save_logs(self, results: List[RunResult]) -> Path:
|
||||
"""
|
||||
Save stdout/stderr logs for each result.
|
||||
|
||||
Args:
|
||||
results: List of RunResult objects.
|
||||
|
||||
Returns:
|
||||
Path to logs directory.
|
||||
"""
|
||||
logs_dir = self.output_dir / "logs"
|
||||
logs_dir.mkdir(exist_ok=True)
|
||||
|
||||
for r in results:
|
||||
# Create safe filename
|
||||
if r.suite == "docs":
|
||||
slug = f"{r.source_path.stem}__{r.block_index}"
|
||||
else:
|
||||
slug = r.source_path.stem
|
||||
|
||||
# Sanitize slug
|
||||
slug = slug.replace(" ", "_").replace("-", "_").replace("/", "__")
|
||||
|
||||
if r.stdout:
|
||||
stdout_path = logs_dir / f"{slug}.stdout.log"
|
||||
stdout_path.write_text(r.stdout, encoding='utf-8')
|
||||
r.stdout_path = f"logs/{slug}.stdout.log"
|
||||
|
||||
if r.stderr:
|
||||
stderr_path = logs_dir / f"{slug}.stderr.log"
|
||||
stderr_path.write_text(r.stderr, encoding='utf-8')
|
||||
r.stderr_path = f"logs/{slug}.stderr.log"
|
||||
|
||||
return logs_dir
|
||||
|
||||
def generate_all(self, report: RunReport) -> dict:
|
||||
"""
|
||||
Generate all report formats.
|
||||
|
||||
Args:
|
||||
report: RunReport with all results.
|
||||
|
||||
Returns:
|
||||
Dict with paths to generated files.
|
||||
"""
|
||||
self.save_logs(report.results)
|
||||
|
||||
return {
|
||||
"json": self.generate_json(report),
|
||||
"markdown": self.generate_markdown(report),
|
||||
"csv": self.generate_csv(report),
|
||||
"logs": self.output_dir / "logs",
|
||||
}
|
||||
@@ -0,0 +1,294 @@
|
||||
"""
|
||||
Unified Script Runner for suite execution.
|
||||
|
||||
Provides subprocess execution with streaming, timeouts, and output capture.
|
||||
Used by both examples and docs runners.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
import time
|
||||
from datetime import datetime, timezone
|
||||
from pathlib import Path
|
||||
from typing import Optional, Callable, List
|
||||
|
||||
from .models import RunItem, RunResult
|
||||
|
||||
|
||||
class ScriptRunner:
|
||||
"""
|
||||
Runs scripts in isolated subprocesses.
|
||||
|
||||
Unified runner for both examples and docs suites.
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
timeout: int = 60,
|
||||
capture_output: bool = True,
|
||||
stream_output: bool = False,
|
||||
env_overrides: Optional[dict] = None,
|
||||
pythonpath_additions: Optional[List[str]] = None,
|
||||
cwd: Optional[Path] = None,
|
||||
python_executable: Optional[str] = None,
|
||||
):
|
||||
"""
|
||||
Initialize runner.
|
||||
|
||||
Args:
|
||||
timeout: Per-script timeout in seconds.
|
||||
capture_output: Whether to capture stdout/stderr.
|
||||
stream_output: Whether to stream output in real-time.
|
||||
env_overrides: Environment variable overrides.
|
||||
pythonpath_additions: Paths to add to PYTHONPATH.
|
||||
cwd: Working directory for script execution.
|
||||
python_executable: Python interpreter to use (default: sys.executable).
|
||||
"""
|
||||
self.timeout = timeout
|
||||
self.capture_output = capture_output
|
||||
self.stream_output = stream_output
|
||||
self.env_overrides = env_overrides or {}
|
||||
self.pythonpath_additions = pythonpath_additions or []
|
||||
self.cwd = cwd
|
||||
self.python_executable = python_executable or sys.executable
|
||||
|
||||
def _build_env(self) -> dict:
|
||||
"""Build environment for subprocess."""
|
||||
env = os.environ.copy()
|
||||
|
||||
# Add PYTHONPATH
|
||||
pythonpath_parts = list(self.pythonpath_additions)
|
||||
existing = env.get('PYTHONPATH', '')
|
||||
if existing:
|
||||
pythonpath_parts.append(existing)
|
||||
|
||||
if pythonpath_parts:
|
||||
env['PYTHONPATH'] = os.pathsep.join(pythonpath_parts)
|
||||
|
||||
# Apply overrides
|
||||
env.update(self.env_overrides)
|
||||
|
||||
return env
|
||||
|
||||
def check_required_env(self, require_env: List[str]) -> Optional[str]:
|
||||
"""
|
||||
Check if required env vars are present.
|
||||
|
||||
Returns:
|
||||
Missing var name or None if all present.
|
||||
"""
|
||||
for key in require_env:
|
||||
if not os.environ.get(key):
|
||||
return key
|
||||
return None
|
||||
|
||||
def run(
|
||||
self,
|
||||
item: RunItem,
|
||||
on_output: Optional[Callable[[str, str], None]] = None,
|
||||
) -> RunResult:
|
||||
"""
|
||||
Run a script and return result.
|
||||
|
||||
Args:
|
||||
item: RunItem with script_path set.
|
||||
on_output: Callback for streaming output (line, stream_type).
|
||||
|
||||
Returns:
|
||||
RunResult with execution details.
|
||||
"""
|
||||
script_path = item.script_path
|
||||
if not script_path or not script_path.exists():
|
||||
return RunResult(
|
||||
item_id=item.item_id,
|
||||
suite=item.suite,
|
||||
group=item.group,
|
||||
source_path=item.source_path,
|
||||
block_index=item.block_index,
|
||||
language=item.language,
|
||||
line_start=item.line_start,
|
||||
line_end=item.line_end,
|
||||
runnable_decision=item.runnable_decision,
|
||||
status="failed",
|
||||
error_type="FileNotFound",
|
||||
error_message=f"Script not found: {script_path}",
|
||||
code_hash=item.code_hash,
|
||||
)
|
||||
|
||||
# Check required env
|
||||
if item.require_env:
|
||||
missing = self.check_required_env(item.require_env)
|
||||
if missing:
|
||||
return RunResult(
|
||||
item_id=item.item_id,
|
||||
suite=item.suite,
|
||||
group=item.group,
|
||||
source_path=item.source_path,
|
||||
block_index=item.block_index,
|
||||
language=item.language,
|
||||
line_start=item.line_start,
|
||||
line_end=item.line_end,
|
||||
runnable_decision=item.runnable_decision,
|
||||
status="skipped",
|
||||
skip_reason=f"Missing required env: {missing}",
|
||||
env_requirements=",".join(item.require_env),
|
||||
code_hash=item.code_hash,
|
||||
)
|
||||
|
||||
# Determine timeout
|
||||
timeout = item.timeout if item.timeout else self.timeout
|
||||
|
||||
# Build command
|
||||
cmd = [self.python_executable, '-u', str(script_path)]
|
||||
env = self._build_env()
|
||||
cwd = self.cwd or script_path.parent
|
||||
|
||||
# Record start time
|
||||
start_time = datetime.now(timezone.utc)
|
||||
start_ts = start_time.isoformat()
|
||||
|
||||
stdout_data = []
|
||||
stderr_data = []
|
||||
exit_code = 0
|
||||
status = "passed"
|
||||
error_type = None
|
||||
error_message = None
|
||||
|
||||
try:
|
||||
if self.stream_output and on_output:
|
||||
# Stream output in real-time
|
||||
process = subprocess.Popen(
|
||||
cmd,
|
||||
stdout=subprocess.PIPE,
|
||||
stderr=subprocess.PIPE,
|
||||
env=env,
|
||||
cwd=cwd,
|
||||
text=True,
|
||||
bufsize=1,
|
||||
)
|
||||
|
||||
import selectors
|
||||
sel = selectors.DefaultSelector()
|
||||
sel.register(process.stdout, selectors.EVENT_READ)
|
||||
sel.register(process.stderr, selectors.EVENT_READ)
|
||||
|
||||
deadline = time.time() + timeout
|
||||
|
||||
while process.poll() is None:
|
||||
remaining = deadline - time.time()
|
||||
if remaining <= 0:
|
||||
process.terminate()
|
||||
try:
|
||||
process.wait(timeout=2)
|
||||
except subprocess.TimeoutExpired:
|
||||
process.kill()
|
||||
status = "timeout"
|
||||
break
|
||||
|
||||
events = sel.select(timeout=min(0.1, remaining))
|
||||
for key, _ in events:
|
||||
line = key.fileobj.readline()
|
||||
if line:
|
||||
if key.fileobj == process.stdout:
|
||||
stdout_data.append(line)
|
||||
on_output(line, 'stdout')
|
||||
else:
|
||||
stderr_data.append(line)
|
||||
on_output(line, 'stderr')
|
||||
|
||||
sel.close()
|
||||
|
||||
# Read remaining output
|
||||
if process.stdout:
|
||||
remaining_stdout = process.stdout.read()
|
||||
if remaining_stdout:
|
||||
stdout_data.append(remaining_stdout)
|
||||
if process.stderr:
|
||||
remaining_stderr = process.stderr.read()
|
||||
if remaining_stderr:
|
||||
stderr_data.append(remaining_stderr)
|
||||
|
||||
exit_code = process.returncode or 0
|
||||
|
||||
else:
|
||||
# Capture output without streaming
|
||||
result = subprocess.run(
|
||||
cmd,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=timeout,
|
||||
env=env,
|
||||
cwd=cwd,
|
||||
)
|
||||
exit_code = result.returncode
|
||||
stdout_data = [result.stdout] if result.stdout else []
|
||||
stderr_data = [result.stderr] if result.stderr else []
|
||||
|
||||
except subprocess.TimeoutExpired:
|
||||
status = "timeout"
|
||||
error_type = "TimeoutError"
|
||||
error_message = f"Exceeded {timeout}s timeout"
|
||||
except Exception as e:
|
||||
status = "failed"
|
||||
error_type = type(e).__name__
|
||||
error_message = str(e)
|
||||
exit_code = 1
|
||||
|
||||
# Record end time
|
||||
end_time = datetime.now(timezone.utc)
|
||||
end_ts = end_time.isoformat()
|
||||
duration = (end_time - start_time).total_seconds()
|
||||
|
||||
# Process output
|
||||
stdout_str = ''.join(stdout_data)
|
||||
stderr_str = ''.join(stderr_data)
|
||||
|
||||
# Determine final status
|
||||
if status != "timeout":
|
||||
if exit_code != 0:
|
||||
if item.xfail:
|
||||
status = "xfail"
|
||||
else:
|
||||
status = "failed"
|
||||
# Extract error info from stderr
|
||||
if stderr_str:
|
||||
lines = stderr_str.strip().split('\n')
|
||||
if lines:
|
||||
error_message = lines[-1][:500]
|
||||
# Try to extract error type
|
||||
for line in reversed(lines):
|
||||
if ': ' in line:
|
||||
parts = line.split(': ', 1)
|
||||
if parts[0].endswith('Error') or parts[0].endswith('Exception'):
|
||||
error_type = parts[0].split('.')[-1]
|
||||
break
|
||||
else:
|
||||
status = "passed"
|
||||
|
||||
return RunResult(
|
||||
item_id=item.item_id,
|
||||
suite=item.suite,
|
||||
group=item.group,
|
||||
source_path=item.source_path,
|
||||
block_index=item.block_index,
|
||||
language=item.language,
|
||||
line_start=item.line_start,
|
||||
line_end=item.line_end,
|
||||
runnable_decision=item.runnable_decision,
|
||||
status=status,
|
||||
exit_code=exit_code,
|
||||
duration_seconds=duration,
|
||||
start_time=start_ts,
|
||||
end_time=end_ts,
|
||||
error_type=error_type,
|
||||
error_message=error_message,
|
||||
stdout=stdout_str if self.capture_output else "",
|
||||
stderr=stderr_str if self.capture_output else "",
|
||||
python_executable=self.python_executable,
|
||||
cwd=str(cwd),
|
||||
env_requirements=",".join(item.require_env) if item.require_env else "",
|
||||
code_hash=item.code_hash,
|
||||
)
|
||||
@@ -1 +1 @@
|
||||
__version__ = "3.5.7"
|
||||
__version__ = "3.6.2"
|
||||
@@ -12,7 +12,7 @@ dependencies = [
|
||||
"rich>=13.7",
|
||||
"markdown>=3.5",
|
||||
"pyparsing>=3.0.0",
|
||||
"praisonaiagents>=0.10.6",
|
||||
"praisonaiagents>=0.10.10",
|
||||
"python-dotenv>=0.19.0",
|
||||
"instructor>=1.3.3",
|
||||
"PyYAML>=6.0",
|
||||
|
||||
@@ -130,6 +130,19 @@ def bump_version(new_version: str, agents_version: Optional[str] = None):
|
||||
print("\n✨ Version bump complete!")
|
||||
|
||||
|
||||
def validate_dependencies() -> bool:
|
||||
"""Validate that uv lock will succeed before making any changes."""
|
||||
praisonai_dir = get_praisonai_dir()
|
||||
|
||||
print("\n🔍 Validating dependencies (dry-run)...")
|
||||
result = run(["uv", "lock", "--dry-run"], cwd=praisonai_dir, check=False)
|
||||
if result.returncode != 0:
|
||||
print("\n❌ Dependency validation failed. Fix the issues above before releasing.")
|
||||
return False
|
||||
print(" ✅ Dependencies validated successfully")
|
||||
return True
|
||||
|
||||
|
||||
def release(version: str):
|
||||
"""Run the release process."""
|
||||
root = get_project_root()
|
||||
@@ -215,6 +228,11 @@ def main():
|
||||
# Run bump version
|
||||
bump_version(args.version, args.agents)
|
||||
|
||||
# Validate dependencies after version bump (before git operations)
|
||||
if not validate_dependencies():
|
||||
print("\n💡 Tip: Revert changes with 'git checkout .' if needed")
|
||||
sys.exit(1)
|
||||
|
||||
# Run release
|
||||
release(args.version)
|
||||
|
||||
|
||||
@@ -0,0 +1 @@
|
||||
"""Tests for suite_runner module."""
|
||||
@@ -0,0 +1,557 @@
|
||||
"""
|
||||
Unit tests for report_viewer module.
|
||||
|
||||
Tests cover:
|
||||
- Latest report directory detection
|
||||
- JSON parsing and loading
|
||||
- Error signature normalization
|
||||
- Error grouping
|
||||
- Table rendering
|
||||
- CLI integration smoke test
|
||||
"""
|
||||
|
||||
import json
|
||||
import pytest
|
||||
|
||||
from praisonai.suite_runner.report_viewer import (
|
||||
normalize_error_signature,
|
||||
find_latest_report_dir,
|
||||
find_report_files,
|
||||
load_report,
|
||||
group_errors,
|
||||
render_overview,
|
||||
render_failures_table,
|
||||
render_error_groups,
|
||||
view_report,
|
||||
_truncate,
|
||||
_render_table,
|
||||
)
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# Fixtures
|
||||
# =============================================================================
|
||||
|
||||
@pytest.fixture
|
||||
def sample_report_json():
|
||||
"""Sample report JSON structure."""
|
||||
return {
|
||||
"metadata": {
|
||||
"suite": "examples",
|
||||
"source_path": "/path/to/examples",
|
||||
"timestamp": "2026-01-09T14:26:07.808316+00:00",
|
||||
"platform": "Darwin-25.3.0-arm64",
|
||||
"python_version": "3.12.11",
|
||||
"python_executable": "/usr/bin/python",
|
||||
"praisonai_version": "3.5.8",
|
||||
"git_commit": "abc1234",
|
||||
"cli_args": [],
|
||||
"groups_run": ["acp", "tools"],
|
||||
"totals": {
|
||||
"passed": 5,
|
||||
"failed": 2,
|
||||
"skipped": 1,
|
||||
"timeout": 1,
|
||||
"not_run": 0,
|
||||
"xfail": 0,
|
||||
"total": 9,
|
||||
},
|
||||
},
|
||||
"results": [
|
||||
{
|
||||
"item_id": "acp__test1",
|
||||
"suite": "examples",
|
||||
"group": "acp",
|
||||
"source_path": "/path/to/examples/acp/test1.py",
|
||||
"block_index": 0,
|
||||
"language": "python",
|
||||
"status": "passed",
|
||||
"duration_seconds": 1.5,
|
||||
"exit_code": 0,
|
||||
},
|
||||
{
|
||||
"item_id": "acp__test2",
|
||||
"suite": "examples",
|
||||
"group": "acp",
|
||||
"source_path": "/path/to/examples/acp/test2.py",
|
||||
"block_index": 0,
|
||||
"language": "python",
|
||||
"status": "failed",
|
||||
"duration_seconds": 2.0,
|
||||
"exit_code": 1,
|
||||
"error_type": "ImportError",
|
||||
"error_message": "No module named 'missing_module'",
|
||||
},
|
||||
{
|
||||
"item_id": "tools__test3",
|
||||
"suite": "examples",
|
||||
"group": "tools",
|
||||
"source_path": "/path/to/examples/tools/test3.py",
|
||||
"block_index": 0,
|
||||
"language": "python",
|
||||
"status": "failed",
|
||||
"duration_seconds": 0.5,
|
||||
"exit_code": 1,
|
||||
"error_type": "ImportError",
|
||||
"error_message": "No module named 'missing_module'",
|
||||
},
|
||||
{
|
||||
"item_id": "tools__test4",
|
||||
"suite": "examples",
|
||||
"group": "tools",
|
||||
"source_path": "/path/to/examples/tools/test4.py",
|
||||
"block_index": 0,
|
||||
"language": "python",
|
||||
"status": "timeout",
|
||||
"duration_seconds": 60.0,
|
||||
"exit_code": -1,
|
||||
"error_type": "TimeoutError",
|
||||
"error_message": "Execution timed out after 60 seconds",
|
||||
},
|
||||
],
|
||||
}
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def report_dir(tmp_path, sample_report_json):
|
||||
"""Create a temporary report directory with sample data."""
|
||||
report_path = tmp_path / "report.json"
|
||||
report_path.write_text(json.dumps(sample_report_json), encoding='utf-8')
|
||||
return tmp_path
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def multi_group_report_dir(tmp_path, sample_report_json):
|
||||
"""Create a report directory with multiple group subdirs."""
|
||||
# Create group subdirectories
|
||||
acp_dir = tmp_path / "acp"
|
||||
acp_dir.mkdir()
|
||||
tools_dir = tmp_path / "tools"
|
||||
tools_dir.mkdir()
|
||||
|
||||
# Write separate reports for each group
|
||||
acp_report = {
|
||||
"metadata": sample_report_json["metadata"].copy(),
|
||||
"results": [r for r in sample_report_json["results"] if r["group"] == "acp"],
|
||||
}
|
||||
acp_report["metadata"]["groups_run"] = ["acp"]
|
||||
(acp_dir / "report.json").write_text(json.dumps(acp_report), encoding='utf-8')
|
||||
|
||||
tools_report = {
|
||||
"metadata": sample_report_json["metadata"].copy(),
|
||||
"results": [r for r in sample_report_json["results"] if r["group"] == "tools"],
|
||||
}
|
||||
tools_report["metadata"]["groups_run"] = ["tools"]
|
||||
(tools_dir / "report.json").write_text(json.dumps(tools_report), encoding='utf-8')
|
||||
|
||||
return tmp_path
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# Error Signature Normalization Tests
|
||||
# =============================================================================
|
||||
|
||||
class TestNormalizeErrorSignature:
|
||||
"""Tests for error signature normalization."""
|
||||
|
||||
def test_basic_error(self):
|
||||
"""Test basic error type and message."""
|
||||
sig = normalize_error_signature("ImportError", "No module named 'foo'")
|
||||
assert "ImportError" in sig
|
||||
assert "No module named" in sig
|
||||
|
||||
def test_strips_file_paths(self):
|
||||
"""Test that file paths are normalized."""
|
||||
msg = 'File "/Users/praison/project/test.py", line 42, in foo'
|
||||
sig = normalize_error_signature("SyntaxError", msg)
|
||||
assert "/Users/praison" not in sig
|
||||
assert "...py" in sig or "line N" in sig
|
||||
|
||||
def test_strips_line_numbers(self):
|
||||
"""Test that line numbers are normalized."""
|
||||
msg = "Error at line 123 in function"
|
||||
sig = normalize_error_signature("Error", msg)
|
||||
assert "123" not in sig
|
||||
assert "line N" in sig
|
||||
|
||||
def test_strips_memory_addresses(self):
|
||||
"""Test that memory addresses are normalized."""
|
||||
msg = "Object at 0x7f8b1c2d3e4f crashed"
|
||||
sig = normalize_error_signature("RuntimeError", msg)
|
||||
assert "0x7f8b1c2d3e4f" not in sig
|
||||
assert "0x..." in sig
|
||||
|
||||
def test_strips_uuids(self):
|
||||
"""Test that UUIDs are normalized."""
|
||||
msg = "Request 550e8400-e29b-41d4-a716-446655440000 failed"
|
||||
sig = normalize_error_signature("Error", msg)
|
||||
assert "550e8400" not in sig
|
||||
assert "UUID" in sig
|
||||
|
||||
def test_strips_timestamps(self):
|
||||
"""Test that timestamps are normalized."""
|
||||
msg = "Error at 2026-01-09T14:30:00Z"
|
||||
sig = normalize_error_signature("Error", msg)
|
||||
assert "2026-01-09" not in sig
|
||||
assert "TIMESTAMP" in sig
|
||||
|
||||
def test_empty_message(self):
|
||||
"""Test with empty message."""
|
||||
sig = normalize_error_signature("Error", "")
|
||||
assert sig == "Error"
|
||||
|
||||
def test_none_values(self):
|
||||
"""Test with None values."""
|
||||
sig = normalize_error_signature(None, None)
|
||||
assert sig == "unknown_error"
|
||||
|
||||
def test_truncates_long_messages(self):
|
||||
"""Test that long messages are truncated."""
|
||||
long_msg = "x" * 200
|
||||
sig = normalize_error_signature("Error", long_msg)
|
||||
assert len(sig) <= 110 # Error: + 100 chars
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# Report Discovery Tests
|
||||
# =============================================================================
|
||||
|
||||
class TestFindLatestReportDir:
|
||||
"""Tests for latest report directory detection."""
|
||||
|
||||
def test_finds_direct_report(self, report_dir):
|
||||
"""Test finding report.json directly in directory."""
|
||||
result = find_latest_report_dir("examples", report_dir)
|
||||
assert result == report_dir
|
||||
|
||||
def test_finds_latest_by_mtime(self, tmp_path):
|
||||
"""Test finding latest directory by modification time."""
|
||||
# Create older directory
|
||||
old_dir = tmp_path / "20260101_120000"
|
||||
old_dir.mkdir()
|
||||
(old_dir / "report.json").write_text('{"metadata": {}, "results": []}')
|
||||
|
||||
# Create newer directory
|
||||
new_dir = tmp_path / "20260109_140000"
|
||||
new_dir.mkdir()
|
||||
(new_dir / "report.json").write_text('{"metadata": {}, "results": []}')
|
||||
|
||||
result = find_latest_report_dir("examples", tmp_path)
|
||||
assert result == new_dir
|
||||
|
||||
def test_returns_none_for_empty_dir(self, tmp_path):
|
||||
"""Test returns None when no reports found."""
|
||||
result = find_latest_report_dir("examples", tmp_path)
|
||||
assert result is None
|
||||
|
||||
def test_returns_none_for_nonexistent_dir(self, tmp_path):
|
||||
"""Test returns None for nonexistent directory."""
|
||||
result = find_latest_report_dir("examples", tmp_path / "nonexistent")
|
||||
assert result is None
|
||||
|
||||
|
||||
class TestFindReportFiles:
|
||||
"""Tests for finding report files in a directory."""
|
||||
|
||||
def test_finds_direct_report(self, report_dir):
|
||||
"""Test finding report.json directly."""
|
||||
files = find_report_files(report_dir)
|
||||
assert len(files) == 1
|
||||
assert files[0].name == "report.json"
|
||||
|
||||
def test_finds_group_reports(self, multi_group_report_dir):
|
||||
"""Test finding reports in group subdirectories."""
|
||||
files = find_report_files(multi_group_report_dir)
|
||||
assert len(files) == 2
|
||||
names = {f.parent.name for f in files}
|
||||
assert names == {"acp", "tools"}
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# Report Loading Tests
|
||||
# =============================================================================
|
||||
|
||||
class TestLoadReport:
|
||||
"""Tests for loading and parsing reports."""
|
||||
|
||||
def test_loads_single_report(self, report_dir):
|
||||
"""Test loading a single report.json."""
|
||||
report = load_report(report_dir)
|
||||
assert report.metadata.suite == "examples"
|
||||
assert len(report.items) == 4
|
||||
|
||||
def test_loads_multi_group_report(self, multi_group_report_dir):
|
||||
"""Test loading and aggregating multiple group reports."""
|
||||
report = load_report(multi_group_report_dir)
|
||||
assert len(report.items) == 4
|
||||
assert set(report.metadata.groups_run) == {"acp", "tools"}
|
||||
|
||||
def test_calculates_totals(self, report_dir):
|
||||
"""Test that totals are calculated correctly."""
|
||||
report = load_report(report_dir)
|
||||
totals = report.metadata.totals
|
||||
assert totals["passed"] == 1
|
||||
assert totals["failed"] == 2
|
||||
assert totals["timeout"] == 1
|
||||
assert totals["total"] == 4
|
||||
|
||||
def test_raises_for_missing_report(self, tmp_path):
|
||||
"""Test raises FileNotFoundError for missing report."""
|
||||
with pytest.raises(FileNotFoundError):
|
||||
load_report(tmp_path)
|
||||
|
||||
def test_raises_for_invalid_json(self, tmp_path):
|
||||
"""Test raises ValueError for invalid JSON."""
|
||||
(tmp_path / "report.json").write_text("not valid json")
|
||||
with pytest.raises(ValueError):
|
||||
load_report(tmp_path)
|
||||
|
||||
def test_normalizes_error_signatures(self, report_dir):
|
||||
"""Test that error signatures are normalized during load."""
|
||||
report = load_report(report_dir)
|
||||
failed_items = [i for i in report.items if i.status == "failed"]
|
||||
assert all(i.error_signature for i in failed_items)
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# Error Grouping Tests
|
||||
# =============================================================================
|
||||
|
||||
class TestGroupErrors:
|
||||
"""Tests for error grouping functionality."""
|
||||
|
||||
def test_groups_similar_errors(self, report_dir):
|
||||
"""Test that similar errors are grouped together."""
|
||||
report = load_report(report_dir)
|
||||
groups = group_errors(report.items)
|
||||
|
||||
# Should have 2 groups: ImportError and TimeoutError
|
||||
assert len(groups) == 2
|
||||
|
||||
def test_counts_occurrences(self, report_dir):
|
||||
"""Test that occurrence counts are correct."""
|
||||
report = load_report(report_dir)
|
||||
groups = group_errors(report.items)
|
||||
|
||||
# Find ImportError group (should have 2 occurrences)
|
||||
import_group = next((g for g in groups if "ImportError" in g.error_type), None)
|
||||
assert import_group is not None
|
||||
assert import_group.count == 2
|
||||
|
||||
def test_sorts_by_count_desc(self, report_dir):
|
||||
"""Test that groups are sorted by count descending."""
|
||||
report = load_report(report_dir)
|
||||
groups = group_errors(report.items)
|
||||
|
||||
counts = [g.count for g in groups]
|
||||
assert counts == sorted(counts, reverse=True)
|
||||
|
||||
def test_ignores_passed_items(self, report_dir):
|
||||
"""Test that passed items are not grouped."""
|
||||
report = load_report(report_dir)
|
||||
groups = group_errors(report.items)
|
||||
|
||||
all_items = [i for g in groups for i in g.items]
|
||||
assert all(i.status in ("failed", "timeout", "error") for i in all_items)
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# Rendering Tests
|
||||
# =============================================================================
|
||||
|
||||
class TestTruncate:
|
||||
"""Tests for text truncation."""
|
||||
|
||||
def test_short_text_unchanged(self):
|
||||
"""Test that short text is not truncated."""
|
||||
assert _truncate("hello", 10) == "hello"
|
||||
|
||||
def test_long_text_truncated(self):
|
||||
"""Test that long text is truncated with ellipsis."""
|
||||
result = _truncate("hello world", 8)
|
||||
assert len(result) == 8
|
||||
assert result.endswith("...")
|
||||
|
||||
def test_empty_text(self):
|
||||
"""Test empty text handling."""
|
||||
assert _truncate("", 10) == ""
|
||||
assert _truncate(None, 10) == ""
|
||||
|
||||
def test_newlines_removed(self):
|
||||
"""Test that newlines are removed."""
|
||||
assert "\n" not in _truncate("hello\nworld", 20)
|
||||
|
||||
|
||||
class TestRenderTable:
|
||||
"""Tests for ASCII table rendering."""
|
||||
|
||||
def test_renders_headers(self):
|
||||
"""Test that headers are rendered."""
|
||||
table = _render_table(["A", "B"], [["1", "2"]])
|
||||
assert "A" in table
|
||||
assert "B" in table
|
||||
|
||||
def test_renders_data(self):
|
||||
"""Test that data rows are rendered."""
|
||||
table = _render_table(["Col"], [["value1"], ["value2"]])
|
||||
assert "value1" in table
|
||||
assert "value2" in table
|
||||
|
||||
def test_empty_table(self):
|
||||
"""Test empty table handling."""
|
||||
table = _render_table([], [])
|
||||
assert table == ""
|
||||
|
||||
|
||||
class TestRenderOverview:
|
||||
"""Tests for overview rendering."""
|
||||
|
||||
def test_includes_metadata(self, report_dir):
|
||||
"""Test that overview includes metadata."""
|
||||
report = load_report(report_dir)
|
||||
output = render_overview(report)
|
||||
|
||||
assert "EXAMPLES" in output.upper()
|
||||
assert "Python" in output or "python" in output
|
||||
|
||||
def test_includes_totals(self, report_dir):
|
||||
"""Test that overview includes totals."""
|
||||
report = load_report(report_dir)
|
||||
output = render_overview(report)
|
||||
|
||||
assert "Passed" in output or "passed" in output
|
||||
assert "Failed" in output or "failed" in output
|
||||
|
||||
|
||||
class TestRenderFailuresTable:
|
||||
"""Tests for failures table rendering."""
|
||||
|
||||
def test_shows_failures(self, report_dir):
|
||||
"""Test that failures are shown."""
|
||||
report = load_report(report_dir)
|
||||
output = render_failures_table(report)
|
||||
|
||||
assert "test2.py" in output or "test3.py" in output
|
||||
|
||||
def test_respects_limit(self, report_dir):
|
||||
"""Test that limit is respected."""
|
||||
report = load_report(report_dir)
|
||||
output = render_failures_table(report, limit=1)
|
||||
|
||||
# Should mention there are more
|
||||
assert "more" in output.lower() or "1" in output
|
||||
|
||||
def test_filters_by_status(self, report_dir):
|
||||
"""Test filtering by status."""
|
||||
report = load_report(report_dir)
|
||||
output = render_failures_table(report, match_statuses=["timeout"])
|
||||
|
||||
assert "timeout" in output.lower()
|
||||
|
||||
|
||||
class TestRenderErrorGroups:
|
||||
"""Tests for error groups rendering."""
|
||||
|
||||
def test_shows_groups(self, report_dir):
|
||||
"""Test that error groups are shown."""
|
||||
report = load_report(report_dir)
|
||||
output = render_error_groups(report)
|
||||
|
||||
assert "ERROR GROUPS" in output
|
||||
assert "ImportError" in output or "TimeoutError" in output
|
||||
|
||||
def test_shows_counts(self, report_dir):
|
||||
"""Test that occurrence counts are shown."""
|
||||
report = load_report(report_dir)
|
||||
output = render_error_groups(report)
|
||||
|
||||
assert "occurrence" in output.lower()
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# Integration Tests
|
||||
# =============================================================================
|
||||
|
||||
class TestViewReport:
|
||||
"""Integration tests for view_report function."""
|
||||
|
||||
def test_table_output(self, report_dir):
|
||||
"""Test table output format."""
|
||||
output, exit_code = view_report(
|
||||
report_dir=report_dir,
|
||||
suite="examples",
|
||||
output_format="table",
|
||||
)
|
||||
|
||||
assert "EXAMPLES" in output.upper()
|
||||
assert exit_code == 1 # Has failures
|
||||
|
||||
def test_json_output(self, report_dir):
|
||||
"""Test JSON output format."""
|
||||
output, exit_code = view_report(
|
||||
report_dir=report_dir,
|
||||
suite="examples",
|
||||
output_format="json",
|
||||
)
|
||||
|
||||
data = json.loads(output)
|
||||
assert "metadata" in data
|
||||
assert "failures" in data
|
||||
assert "error_groups" in data
|
||||
|
||||
def test_exit_code_zero_for_no_failures(self, tmp_path):
|
||||
"""Test exit code is 0 when no failures."""
|
||||
report = {
|
||||
"metadata": {"suite": "examples", "totals": {"passed": 1, "total": 1}},
|
||||
"results": [
|
||||
{
|
||||
"item_id": "test1",
|
||||
"suite": "examples",
|
||||
"group": "test",
|
||||
"source_path": "/test.py",
|
||||
"status": "passed",
|
||||
}
|
||||
],
|
||||
}
|
||||
(tmp_path / "report.json").write_text(json.dumps(report))
|
||||
|
||||
output, exit_code = view_report(report_dir=tmp_path, suite="examples")
|
||||
assert exit_code == 0
|
||||
|
||||
def test_error_for_missing_report(self, tmp_path):
|
||||
"""Test error message for missing report."""
|
||||
output, exit_code = view_report(
|
||||
report_dir=tmp_path / "nonexistent",
|
||||
suite="examples",
|
||||
)
|
||||
|
||||
assert "Error" in output
|
||||
assert exit_code == 1
|
||||
|
||||
def test_auto_detect_latest(self, tmp_path):
|
||||
"""Test auto-detection of latest report."""
|
||||
# Create a report directory
|
||||
run_dir = tmp_path / "20260109_140000"
|
||||
run_dir.mkdir()
|
||||
report = {
|
||||
"metadata": {"suite": "examples", "totals": {"passed": 1, "total": 1}},
|
||||
"results": [
|
||||
{
|
||||
"item_id": "test1",
|
||||
"suite": "examples",
|
||||
"group": "test",
|
||||
"source_path": "/test.py",
|
||||
"status": "passed",
|
||||
}
|
||||
],
|
||||
}
|
||||
(run_dir / "report.json").write_text(json.dumps(report))
|
||||
|
||||
output, exit_code = view_report(
|
||||
report_dir=None,
|
||||
suite="examples",
|
||||
base_dir=tmp_path,
|
||||
)
|
||||
|
||||
assert "EXAMPLES" in output.upper()
|
||||
assert exit_code == 0
|
||||
@@ -0,0 +1,481 @@
|
||||
"""
|
||||
Tests for the unified suite_runner engine.
|
||||
|
||||
Tests shared infrastructure used by both examples and docs runners.
|
||||
"""
|
||||
|
||||
import os
|
||||
import tempfile
|
||||
from pathlib import Path
|
||||
from unittest.mock import patch
|
||||
|
||||
class TestRunItem:
|
||||
"""Tests for RunItem dataclass."""
|
||||
|
||||
def test_create_run_item(self):
|
||||
"""Test creating a RunItem."""
|
||||
from praisonai.suite_runner import RunItem
|
||||
|
||||
item = RunItem(
|
||||
item_id="test_item",
|
||||
suite="examples",
|
||||
group="test_group",
|
||||
source_path=Path("/test/path.py"),
|
||||
code="print('hello')",
|
||||
)
|
||||
|
||||
assert item.item_id == "test_item"
|
||||
assert item.suite == "examples"
|
||||
assert item.group == "test_group"
|
||||
assert item.runnable is True
|
||||
assert item.code_hash # Should have a hash
|
||||
|
||||
def test_display_name_examples(self):
|
||||
"""Test display_name for examples."""
|
||||
from praisonai.suite_runner import RunItem
|
||||
|
||||
item = RunItem(
|
||||
item_id="test",
|
||||
suite="examples",
|
||||
group="test",
|
||||
source_path=Path("/test/example.py"),
|
||||
)
|
||||
|
||||
assert item.display_name == "example.py"
|
||||
|
||||
def test_display_name_docs(self):
|
||||
"""Test display_name for docs."""
|
||||
from praisonai.suite_runner import RunItem
|
||||
|
||||
item = RunItem(
|
||||
item_id="test",
|
||||
suite="docs",
|
||||
group="test",
|
||||
source_path=Path("/test/doc.mdx"),
|
||||
block_index=2,
|
||||
)
|
||||
|
||||
assert item.display_name == "doc.mdx:2"
|
||||
|
||||
|
||||
class TestRunResult:
|
||||
"""Tests for RunResult dataclass."""
|
||||
|
||||
def test_create_run_result(self):
|
||||
"""Test creating a RunResult."""
|
||||
from praisonai.suite_runner import RunResult
|
||||
|
||||
result = RunResult(
|
||||
item_id="test",
|
||||
suite="examples",
|
||||
group="test",
|
||||
source_path=Path("/test/path.py"),
|
||||
status="passed",
|
||||
exit_code=0,
|
||||
duration_seconds=1.5,
|
||||
)
|
||||
|
||||
assert result.status == "passed"
|
||||
assert result.exit_code == 0
|
||||
assert result.duration_seconds == 1.5
|
||||
|
||||
|
||||
class TestRunReport:
|
||||
"""Tests for RunReport dataclass."""
|
||||
|
||||
def test_create_run_report(self):
|
||||
"""Test creating a RunReport."""
|
||||
from praisonai.suite_runner import RunReport, RunResult
|
||||
|
||||
results = [
|
||||
RunResult(item_id="1", suite="examples", group="a", source_path=Path("/a"), status="passed"),
|
||||
RunResult(item_id="2", suite="examples", group="a", source_path=Path("/b"), status="failed"),
|
||||
RunResult(item_id="3", suite="examples", group="b", source_path=Path("/c"), status="skipped"),
|
||||
]
|
||||
|
||||
report = RunReport(results=results, suite="examples", source_path=Path("/test"))
|
||||
|
||||
totals = report.totals
|
||||
assert totals["passed"] == 1
|
||||
assert totals["failed"] == 1
|
||||
assert totals["skipped"] == 1
|
||||
|
||||
def test_to_dict(self):
|
||||
"""Test converting report to dict."""
|
||||
from praisonai.suite_runner import RunReport, RunResult
|
||||
|
||||
results = [
|
||||
RunResult(item_id="1", suite="examples", group="a", source_path=Path("/a"), status="passed"),
|
||||
]
|
||||
|
||||
report = RunReport(results=results, suite="examples", source_path=Path("/test"))
|
||||
data = report.to_dict()
|
||||
|
||||
assert "metadata" in data
|
||||
assert "results" in data
|
||||
assert data["metadata"]["suite"] == "examples"
|
||||
assert len(data["results"]) == 1
|
||||
|
||||
|
||||
class TestFileDiscovery:
|
||||
"""Tests for FileDiscovery."""
|
||||
|
||||
def test_discover_python_files(self):
|
||||
"""Test discovering Python files."""
|
||||
from praisonai.suite_runner import FileDiscovery
|
||||
|
||||
with tempfile.TemporaryDirectory() as tmpdir:
|
||||
# Create test files
|
||||
Path(tmpdir, "test1.py").write_text("# test")
|
||||
Path(tmpdir, "test2.py").write_text("# test")
|
||||
Path(tmpdir, "_private.py").write_text("# test")
|
||||
Path(tmpdir, "subdir").mkdir()
|
||||
Path(tmpdir, "subdir", "test3.py").write_text("# test")
|
||||
|
||||
discovery = FileDiscovery(root=tmpdir, extensions=[".py"])
|
||||
files = discovery.discover()
|
||||
|
||||
# Should find test1, test2, test3 but not _private
|
||||
assert len(files) == 3
|
||||
names = [f.name for f in files]
|
||||
assert "test1.py" in names
|
||||
assert "test2.py" in names
|
||||
assert "test3.py" in names
|
||||
assert "_private.py" not in names
|
||||
|
||||
def test_discover_by_group(self):
|
||||
"""Test discovering files by group."""
|
||||
from praisonai.suite_runner import FileDiscovery
|
||||
|
||||
with tempfile.TemporaryDirectory() as tmpdir:
|
||||
# Create test structure
|
||||
Path(tmpdir, "group_a").mkdir()
|
||||
Path(tmpdir, "group_b").mkdir()
|
||||
Path(tmpdir, "group_a", "test1.py").write_text("# test")
|
||||
Path(tmpdir, "group_b", "test2.py").write_text("# test")
|
||||
|
||||
discovery = FileDiscovery(root=tmpdir, extensions=[".py"])
|
||||
grouped = discovery.discover_by_group(groups=["group_a"])
|
||||
|
||||
assert "group_a" in grouped
|
||||
assert "group_b" not in grouped
|
||||
assert len(grouped["group_a"]) == 1
|
||||
|
||||
def test_get_groups(self):
|
||||
"""Test getting available groups."""
|
||||
from praisonai.suite_runner import FileDiscovery
|
||||
|
||||
with tempfile.TemporaryDirectory() as tmpdir:
|
||||
Path(tmpdir, "alpha").mkdir()
|
||||
Path(tmpdir, "beta").mkdir()
|
||||
Path(tmpdir, "alpha", "test.py").write_text("# test")
|
||||
Path(tmpdir, "beta", "test.py").write_text("# test")
|
||||
|
||||
discovery = FileDiscovery(root=tmpdir, extensions=[".py"])
|
||||
groups = discovery.get_groups()
|
||||
|
||||
assert "alpha" in groups
|
||||
assert "beta" in groups
|
||||
|
||||
|
||||
class TestScriptRunner:
|
||||
"""Tests for ScriptRunner."""
|
||||
|
||||
def test_run_passing_script(self):
|
||||
"""Test running a passing script."""
|
||||
from praisonai.suite_runner import ScriptRunner, RunItem
|
||||
|
||||
with tempfile.TemporaryDirectory() as tmpdir:
|
||||
script = Path(tmpdir, "test.py")
|
||||
script.write_text("print('hello')")
|
||||
|
||||
item = RunItem(
|
||||
item_id="test",
|
||||
suite="examples",
|
||||
group="test",
|
||||
source_path=script,
|
||||
script_path=script,
|
||||
)
|
||||
|
||||
runner = ScriptRunner(timeout=10)
|
||||
result = runner.run(item)
|
||||
|
||||
assert result.status == "passed"
|
||||
assert result.exit_code == 0
|
||||
assert "hello" in result.stdout
|
||||
|
||||
def test_run_failing_script(self):
|
||||
"""Test running a failing script."""
|
||||
from praisonai.suite_runner import ScriptRunner, RunItem
|
||||
|
||||
with tempfile.TemporaryDirectory() as tmpdir:
|
||||
script = Path(tmpdir, "test.py")
|
||||
script.write_text("raise ValueError('test error')")
|
||||
|
||||
item = RunItem(
|
||||
item_id="test",
|
||||
suite="examples",
|
||||
group="test",
|
||||
source_path=script,
|
||||
script_path=script,
|
||||
)
|
||||
|
||||
runner = ScriptRunner(timeout=10)
|
||||
result = runner.run(item)
|
||||
|
||||
assert result.status == "failed"
|
||||
assert result.exit_code != 0
|
||||
assert "ValueError" in (result.error_message or "")
|
||||
|
||||
def test_run_timeout(self):
|
||||
"""Test script timeout."""
|
||||
from praisonai.suite_runner import ScriptRunner, RunItem
|
||||
|
||||
with tempfile.TemporaryDirectory() as tmpdir:
|
||||
script = Path(tmpdir, "test.py")
|
||||
script.write_text("import time; time.sleep(10)")
|
||||
|
||||
item = RunItem(
|
||||
item_id="test",
|
||||
suite="examples",
|
||||
group="test",
|
||||
source_path=script,
|
||||
script_path=script,
|
||||
)
|
||||
|
||||
runner = ScriptRunner(timeout=1)
|
||||
result = runner.run(item)
|
||||
|
||||
assert result.status == "timeout"
|
||||
|
||||
def test_check_required_env(self):
|
||||
"""Test checking required environment variables."""
|
||||
from praisonai.suite_runner import ScriptRunner
|
||||
|
||||
runner = ScriptRunner()
|
||||
|
||||
# Should return None if env var exists
|
||||
with patch.dict(os.environ, {"TEST_VAR": "value"}):
|
||||
assert runner.check_required_env(["TEST_VAR"]) is None
|
||||
|
||||
# Should return missing var name
|
||||
result = runner.check_required_env(["NONEXISTENT_VAR_12345"])
|
||||
assert result == "NONEXISTENT_VAR_12345"
|
||||
|
||||
|
||||
class TestSuiteReporter:
|
||||
"""Tests for SuiteReporter."""
|
||||
|
||||
def test_generate_json(self):
|
||||
"""Test generating JSON report."""
|
||||
from praisonai.suite_runner import SuiteReporter, RunReport, RunResult
|
||||
|
||||
with tempfile.TemporaryDirectory() as tmpdir:
|
||||
results = [
|
||||
RunResult(item_id="1", suite="examples", group="a", source_path=Path("/a"), status="passed"),
|
||||
]
|
||||
report = RunReport(results=results, suite="examples", source_path=Path("/test"))
|
||||
|
||||
reporter = SuiteReporter(Path(tmpdir))
|
||||
json_path = reporter.generate_json(report)
|
||||
|
||||
assert json_path.exists()
|
||||
assert json_path.name == "report.json"
|
||||
|
||||
def test_generate_markdown(self):
|
||||
"""Test generating Markdown report."""
|
||||
from praisonai.suite_runner import SuiteReporter, RunReport, RunResult
|
||||
|
||||
with tempfile.TemporaryDirectory() as tmpdir:
|
||||
results = [
|
||||
RunResult(item_id="1", suite="examples", group="a", source_path=Path("/a"), status="passed"),
|
||||
]
|
||||
report = RunReport(results=results, suite="examples", source_path=Path("/test"))
|
||||
|
||||
reporter = SuiteReporter(Path(tmpdir))
|
||||
md_path = reporter.generate_markdown(report)
|
||||
|
||||
assert md_path.exists()
|
||||
assert md_path.name == "report.md"
|
||||
content = md_path.read_text()
|
||||
assert "Examples Execution Report" in content
|
||||
|
||||
def test_generate_csv(self):
|
||||
"""Test generating CSV report."""
|
||||
from praisonai.suite_runner import SuiteReporter, RunReport, RunResult
|
||||
|
||||
with tempfile.TemporaryDirectory() as tmpdir:
|
||||
results = [
|
||||
RunResult(item_id="1", suite="examples", group="a", source_path=Path("/a"), status="passed"),
|
||||
RunResult(item_id="2", suite="examples", group="b", source_path=Path("/b"), status="failed"),
|
||||
]
|
||||
report = RunReport(results=results, suite="examples", source_path=Path("/test"))
|
||||
|
||||
reporter = SuiteReporter(Path(tmpdir))
|
||||
csv_path = reporter.generate_csv(report)
|
||||
|
||||
assert csv_path.exists()
|
||||
assert csv_path.name == "report.csv"
|
||||
content = csv_path.read_text()
|
||||
assert "suite,group,item_id" in content
|
||||
assert "passed" in content
|
||||
assert "failed" in content
|
||||
|
||||
def test_save_logs(self):
|
||||
"""Test saving logs."""
|
||||
from praisonai.suite_runner import SuiteReporter, RunResult
|
||||
|
||||
with tempfile.TemporaryDirectory() as tmpdir:
|
||||
results = [
|
||||
RunResult(
|
||||
item_id="1",
|
||||
suite="examples",
|
||||
group="a",
|
||||
source_path=Path("/test.py"),
|
||||
status="passed",
|
||||
stdout="hello stdout",
|
||||
stderr="hello stderr",
|
||||
),
|
||||
]
|
||||
|
||||
reporter = SuiteReporter(Path(tmpdir))
|
||||
logs_dir = reporter.save_logs(results)
|
||||
|
||||
assert logs_dir.exists()
|
||||
assert (logs_dir / "test.stdout.log").exists()
|
||||
assert (logs_dir / "test.stderr.log").exists()
|
||||
|
||||
|
||||
class TestExamplesSource:
|
||||
"""Tests for ExamplesSource."""
|
||||
|
||||
def test_discover_examples(self):
|
||||
"""Test discovering examples."""
|
||||
from praisonai.suite_runner import ExamplesSource
|
||||
|
||||
with tempfile.TemporaryDirectory() as tmpdir:
|
||||
# Create test examples
|
||||
Path(tmpdir, "group_a").mkdir()
|
||||
Path(tmpdir, "group_a", "example1.py").write_text("print('hello')")
|
||||
Path(tmpdir, "group_a", "example2.py").write_text("print('world')")
|
||||
|
||||
source = ExamplesSource(root=tmpdir)
|
||||
items = source.discover()
|
||||
|
||||
assert len(items) == 2
|
||||
assert all(item.suite == "examples" for item in items)
|
||||
|
||||
def test_parse_directives(self):
|
||||
"""Test parsing directives from examples."""
|
||||
from praisonai.suite_runner import ExamplesSource
|
||||
|
||||
with tempfile.TemporaryDirectory() as tmpdir:
|
||||
example = Path(tmpdir, "test.py")
|
||||
example.write_text("""# praisonai: skip=true
|
||||
# praisonai: timeout=120
|
||||
print('hello')
|
||||
""")
|
||||
|
||||
source = ExamplesSource(root=tmpdir)
|
||||
items = source.discover()
|
||||
|
||||
assert len(items) == 1
|
||||
item = items[0]
|
||||
assert item.skip is True
|
||||
assert item.timeout == 120
|
||||
|
||||
def test_detect_interactive(self):
|
||||
"""Test detecting interactive examples."""
|
||||
from praisonai.suite_runner import ExamplesSource
|
||||
|
||||
with tempfile.TemporaryDirectory() as tmpdir:
|
||||
example = Path(tmpdir, "test.py")
|
||||
example.write_text("name = input('Enter name: ')")
|
||||
|
||||
source = ExamplesSource(root=tmpdir)
|
||||
items = source.discover()
|
||||
|
||||
assert len(items) == 1
|
||||
assert items[0].is_interactive is True
|
||||
assert items[0].runnable is False
|
||||
|
||||
|
||||
class TestDocsSource:
|
||||
"""Tests for DocsSource."""
|
||||
|
||||
def test_extract_code_blocks(self):
|
||||
"""Test extracting code blocks from docs."""
|
||||
from praisonai.suite_runner import DocsSource
|
||||
|
||||
with tempfile.TemporaryDirectory() as tmpdir:
|
||||
doc = Path(tmpdir, "test.mdx")
|
||||
doc.write_text("""# Test Doc
|
||||
|
||||
```python
|
||||
from praisonaiagents import Agent
|
||||
agent = Agent()
|
||||
agent.start("Hello")
|
||||
```
|
||||
|
||||
Some text
|
||||
|
||||
```python
|
||||
print("partial")
|
||||
```
|
||||
""")
|
||||
|
||||
source = DocsSource(root=tmpdir)
|
||||
items = source.discover()
|
||||
|
||||
assert len(items) == 2
|
||||
assert all(item.suite == "docs" for item in items)
|
||||
# First block should be runnable (has import + terminal action)
|
||||
assert items[0].runnable is True
|
||||
# Second block is partial (no import)
|
||||
assert items[1].runnable is False
|
||||
|
||||
def test_parse_directive(self):
|
||||
"""Test parsing directives from docs."""
|
||||
from praisonai.suite_runner import DocsSource
|
||||
|
||||
with tempfile.TemporaryDirectory() as tmpdir:
|
||||
doc = Path(tmpdir, "test.mdx")
|
||||
doc.write_text("""# Test
|
||||
|
||||
<!-- praisonai: skip=true timeout=120 -->
|
||||
```python
|
||||
print("hello")
|
||||
```
|
||||
""")
|
||||
|
||||
source = DocsSource(root=tmpdir)
|
||||
items = source.discover()
|
||||
|
||||
assert len(items) == 1
|
||||
item = items[0]
|
||||
assert item.skip is True
|
||||
assert item.timeout == 120
|
||||
|
||||
def test_dedent_indented_code(self):
|
||||
"""Test that indented code blocks are dedented."""
|
||||
from praisonai.suite_runner import DocsSource
|
||||
|
||||
with tempfile.TemporaryDirectory() as tmpdir:
|
||||
doc = Path(tmpdir, "test.mdx")
|
||||
doc.write_text("""# Test
|
||||
|
||||
<Tab>
|
||||
```python
|
||||
from praisonaiagents import Agent
|
||||
agent = Agent()
|
||||
agent.start("test")
|
||||
```
|
||||
</Tab>
|
||||
""")
|
||||
|
||||
source = DocsSource(root=tmpdir)
|
||||
items = source.discover()
|
||||
|
||||
assert len(items) == 1
|
||||
# Code should be dedented
|
||||
assert not items[0].code.startswith(" ")
|
||||
Generated
+4
-4
@@ -4276,7 +4276,7 @@ requires-dist = [
|
||||
{ name = "praisonai-tools", marker = "extra == 'autogen'", specifier = ">=0.1.0" },
|
||||
{ name = "praisonai-tools", marker = "extra == 'autogen-v4'", specifier = ">=0.1.0" },
|
||||
{ name = "praisonai-tools", marker = "extra == 'crewai'", specifier = ">=0.1.0" },
|
||||
{ name = "praisonaiagents", specifier = ">=0.10.6" },
|
||||
{ name = "praisonaiagents", specifier = ">=0.10.10" },
|
||||
{ name = "pyautogen", marker = "extra == 'autogen'", specifier = "==0.2.29" },
|
||||
{ name = "pydantic", marker = "extra == 'chat'", specifier = "<=2.10.1" },
|
||||
{ name = "pydantic", marker = "extra == 'code'", specifier = "<=2.10.1" },
|
||||
@@ -4324,7 +4324,7 @@ wheels = [
|
||||
|
||||
[[package]]
|
||||
name = "praisonaiagents"
|
||||
version = "0.10.6"
|
||||
version = "0.10.10"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
dependencies = [
|
||||
{ name = "aiohttp" },
|
||||
@@ -4333,9 +4333,9 @@ dependencies = [
|
||||
{ name = "pydantic" },
|
||||
{ name = "rich" },
|
||||
]
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/a1/02/7736463640fbc2f425dfd904b9ce6ca7435ca47d7621b524db25ee469f07/praisonaiagents-0.10.6.tar.gz", hash = "sha256:81a9970a15797c1f70fe8c6f2c2bd7196a83eb2ffd22713acb7ee41531fc095d", size = 960433, upload-time = "2026-01-09T11:45:51.17Z" }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/56/4f/d01c588827a74ddc58a8bf30cb7eb97671d210b7f5ef46f9eb2a7249f919/praisonaiagents-0.10.10.tar.gz", hash = "sha256:b6cfea93fef40b898575d0c1574134fdc9ec49754d2287e9ade90ba4381ca3d7", size = 962453, upload-time = "2026-01-10T08:57:35.82Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/71/6b/b0eda6eaae0026fba56fb0a65293c4e5f8943c922c0e5f9723f54f67e6de/praisonaiagents-0.10.6-py3-none-any.whl", hash = "sha256:f5eea08e5d560d4218adc639034f45ec31b45a494144992fe179267b9956bc30", size = 939491, upload-time = "2026-01-09T11:45:49.074Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/07/fe/edd179c792d7f6143d3278c01daf37015fe4a2d98c5dc326fb309b263336/praisonaiagents-0.10.10-py3-none-any.whl", hash = "sha256:fa9bc6db1fb12ae25ea491db8e30eb98ac16c1e9798952cf617594c09999b826", size = 941797, upload-time = "2026-01-10T08:57:34.07Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
|
||||
Reference in new issue
Block a user