agno-agi/agno
1
0
Fork 0
mirror of https://github.com/agno-agi/agno.git synced 2026-10-12 02:10:54 +00:00
Code Issues Packages Projects Releases Wiki Activity

cleanup: cookbook

Browse Source
This commit is contained in:
Ashpreet
2026-02-11 21:44:08 +00:00
parent a623dc16a0
commit 431df85d59
24 changed files with 382 additions and 1509 deletions
Download Patch File Download Diff File Expand all files Collapse all files
cookbook/00_quickstart/CLEANUP_PROMPT.md
-58
View File
@@ -1,58 +0,0 @@
Goal: Clean up and standardize `cookbook/00_quickstart` so all Python examples pass the structure checker and use current model conventions.
Context files (read these first):
- `AGENTS.md` — Project conventions, virtual environments, testing workflow
- `cookbook/STYLE_GUIDE.md` — Python file structure rules (canonical reference)
Environment:
- Python: `.venvs/demo/bin/python`
- API keys: loaded via `direnv allow`
Execution requirements:
1. **Read every `.py` file** in the target cookbook directory before making any changes.
Do not rely solely on grep or the structure checker — open and read each file to understand its full contents. This ensures you catch issues the automated checker might miss (e.g., imports inside sections, stale model references in comments, inconsistent patterns).
2. Run the structure checker and fix all violations:
```
.venvs/demo/bin/python cookbook/scripts/check_cookbook_pattern.py --base-dir cookbook/00_quickstart --recursive
```
Common fixes:
- Add module docstring if missing
- Add `if __name__ == "__main__":` gate if missing
- Add "Create" and "Run" section banners if missing
- Ensure "Create" section appears before "Run" section
- Remove emoji characters
- Ensure section banners use `# ---------------------------------------------------------------------------` style (75 dashes)
3. Ensure imports are between the module docstring and the first section banner.
No imports should appear inside sections.
4. Migrate OpenAI model references. Only change files that use OpenAI models.
Do NOT modify files using other providers (Gemini, Claude, Groq, Ollama, etc.).
| Before | After |
|--------|-------|
| `from agno.models.openai import OpenAIChat` | `from agno.models.openai import OpenAIResponses` |
| `OpenAIChat(id="gpt-4o")` | `OpenAIResponses(id="gpt-5.2")` |
| `OpenAIChat(id="gpt-4o-mini")` | `OpenAIResponses(id="gpt-5.2-mini")` |
| Any other `OpenAIChat(` usage | `OpenAIResponses(` with updated model ID |
If a file imports both `OpenAIChat` and another provider, only change the OpenAI parts.
5. Also check non-Python files (`README.md`, etc.) in the directory for stale `OpenAIChat` references and update them.
6. Make only minimal, behavior-preserving edits.
Do not change agent logic, tool configurations, prompts, or runtime behavior.
Validation commands (must all pass before finishing):
- `.venvs/demo/bin/python cookbook/scripts/check_cookbook_pattern.py --base-dir cookbook/00_quickstart --recursive`
Final response format:
1. Summary of changes made (structural fixes, model migrations).
2. Validation command output showing zero violations.
3. Files changed:
| File | Changes |
|------|---------|
| `path/to/file.py` | OpenAIChat -> OpenAIResponses, added main gate |
cookbook/00_quickstart/TEST_PROMPT.md
+381 -35
View File
@@ -1,48 +1,394 @@
Goal: Thoroughly test and validate `cookbook/00_quickstart` so it aligns with our cookbook standards.
# Prompt: Improve Agno Test Suite
Context files (read these first):
- `AGENTS.md` — Project conventions, virtual environments, testing workflow
- `cookbook/STYLE_GUIDE.md` — Python file structure rules
You are improving the test suite for the **Agno** AI agent framework. Your primary goal is to **eliminate flaky tests** (mostly caused by API rate limits in integration tests) and **add new unit tests** for coverage gaps.
Environment:
- Python: `.venvs/demo/bin/python`
- API keys: loaded via `direnv allow`
- Database: `./cookbook/scripts/run_pgvector.sh` (needed for knowledge agent)
---
Execution requirements:
## Environment Setup
1. **Read every `.py` file** in the target cookbook directory before making any changes.
Do not rely solely on grep or the structure checker — open and read each file to understand its full contents. This ensures you catch issues the automated checker might miss (e.g., imports inside sections, stale model references in comments, inconsistent patterns).
Run these commands at the start of every session, in order:
2. Run `.venvs/demo/bin/python cookbook/scripts/check_cookbook_pattern.py --base-dir cookbook/00_quickstart` and fix any violations.
```bash
# 1. Load environment variables (API keys, etc.)
direnv allow
eval "$(direnv export bash 2>/dev/null)"
3. Run all `cookbook/00_quickstart/*.py` examples using `.venvs/demo/bin/python` and capture outcomes. Skip `__init__.py`. For `run.py`, validate startup only (it's a long-running server).
# 2. Activate the test virtual environment
source .venv/bin/activate
4. Ensure Python examples align with `cookbook/STYLE_GUIDE.md`:
- Module docstring with `=====` underline
- Section banners: `# ---------------------------------------------------------------------------`
- Imports between docstring and first banner
- `if __name__ == "__main__":` gate
- No emoji characters
# 3. If .venv doesn't exist, create it first:
# ./scripts/test_setup.sh
5. Also check non-Python files (`README.md`, etc.) in the directory for stale `OpenAIChat` references and update them.
# 4. Install any missing libraries as needed:
# uv pip install <package>
```
6. Make only minimal, behavior-preserving edits where needed for style compliance.
---
7. Update `cookbook/00_quickstart/TEST_LOG.md` with fresh PASS/FAIL entries per file.
## Running Tests
Validation commands (must all pass before finishing):
- `.venvs/demo/bin/python cookbook/scripts/check_cookbook_pattern.py --base-dir cookbook/00_quickstart`
- `source .venvs/demo/bin/python && ./scripts/format.sh` — format all code (ruff format)
- `source .venvs/demo/bin/python && ./scripts/validate.sh` — validate all code (ruff check, mypy)
```bash
# Full unit test suite with coverage
./scripts/test.sh
Final response format:
1. Findings (inconsistencies, failures, risks) with file references.
2. Test/validation commands run with results.
3. Any remaining gaps or manual follow-ups.
4. Results table in this format:
# Single subdirectory
pytest libs/agno/tests/unit/<subdir> -v
| File | Status | Notes |
|------|--------|-------|
| `agent_with_tools.py` | PASS | Produced investment brief for NVDA |
| `agent_with_storage.py` | FAIL | SQLite lock error on concurrent access |
# Single file
pytest libs/agno/tests/unit/<path>/test_file.py -v
# Single test
pytest libs/agno/tests/unit/<path>/test_file.py::TestClass::test_name -v
# With output visible
pytest ... -s
# Stop on first failure
pytest ... -x
# Integration tests (require API keys via direnv)
pytest libs/agno/tests/integration/<subdir> -v
# Coverage for a specific module
pytest libs/agno/tests/unit/<subdir> --cov=libs/agno/agno/<module> --cov-report=term-missing -v
```
---
## Test Suite Structure
```
libs/agno/tests/
├── unit/ # ~240 files, fully mocked, no real API calls
├── integration/ # ~360 files, real API calls, real databases
└── system/ # ~24 files, end-to-end with running services
```
**Pytest config** is in `libs/agno/pyproject.toml`:
```toml
[tool.pytest.ini_options]
log_cli = true
asyncio_mode = "auto"
asyncio_default_fixture_loop_scope = "function"
```
**Key fixtures** live in:
- `libs/agno/tests/integration/conftest.py` — shared DB fixtures, async client reset
- `libs/agno/tests/unit/*/conftest.py` — per-module fixtures
---
## The Flakiness Problem
**Root cause:** Integration tests make real API calls to 35+ LLM providers. Rate limits (HTTP 429) cause intermittent failures that are NOT bugs in the code.
### Current State of Rate Limit Protection
**Only 4 out of 35 model providers have rate-limit protection:**
- `libs/agno/tests/integration/models/google/conftest.py`
- `libs/agno/tests/integration/models/groq/conftest.py`
- `libs/agno/tests/integration/models/cerebras/conftest.py`
- `libs/agno/tests/integration/models/sambanova/conftest.py`
**31 providers have ZERO protection**, including the most heavily used ones: OpenAI, Anthropic, Azure, AWS, Meta, Deepseek, Mistral, Cohere, xAI, etc.
The protection pattern is a `conftest.py` pytest hook that converts rate-limit failures into skips:
```python
import pytest
@pytest.hookimpl(hookwrapper=True)
def pytest_runtest_makereport(item, call):
"""Skip tests that hit <Provider> rate limits (429) instead of failing."""
outcome = yield
report = outcome.get_result()
if report.when == "call" and report.failed:
if call.excinfo is not None:
error_msg = str(call.excinfo.value)
full_repr = str(report.longrepr) if report.longrepr else ""
sections_text = " ".join(content for _, content in report.sections)
combined = (error_msg + full_repr + sections_text).lower()
if any(p in combined for p in ["429", "rate limit", "rate_limit", "quota", "resource_exhausted"]):
report.outcome = "skipped"
report.longrepr = ("", -1, "Skipped: <Provider> rate limit (429)")
```
### Additional Issue: No Retry/Backoff on Most Tests
Google integration tests configure retry/backoff on the Agent:
```python
agent = Agent(
model=Gemini(id="gemini-2.0-flash"),
exponential_backoff=True,
delay_between_retries=5,
)
```
But OpenAI, Anthropic, and most other provider tests do NOT:
```python
# No backoff — a single 429 = test failure
agent = Agent(model=OpenAIChat(id="gpt-4o-mini"), telemetry=False)
```
---
## Task 1: Add Rate-Limit Protection to All Integration Model Tests
**For every provider directory under `libs/agno/tests/integration/models/` that does NOT already have a `conftest.py` with the rate-limit hook**, create one.
The full list of provider directories to check:
```
aimlapi, anthropic, aws, azure, cerebras*, cohere, cometapi, dashscope,
deepinfra, deepseek, fireworks, google*, groq*, huggingface, ibm, langdb,
litellm, litellm_openai, lmstudio, meta, mistral, nebius, nvidia, ollama,
openai, openrouter, perplexity, portkey, sambanova*, together, vercel,
vertexai, vllm, xai
(* = already has protection)
```
**For each unprotected directory:**
1. Check if a `conftest.py` already exists in that directory
2. If it exists, ADD the `pytest_runtest_makereport` hook to it (don't overwrite existing fixtures)
3. If it doesn't exist, CREATE a new `conftest.py` with just the hook
4. Use the provider name in the skip message (e.g., `"Skipped: OpenAI rate limit (429)"`)
**Also add the hook to these non-model integration test directories** that make real API calls:
- `libs/agno/tests/integration/agent/` (uses OpenAI)
- `libs/agno/tests/integration/teams/` (uses OpenAI)
- `libs/agno/tests/integration/workflows/` (uses OpenAI)
- `libs/agno/tests/integration/embedder/` (uses OpenAI)
- `libs/agno/tests/integration/reranker/`
For these, use a generic message: `"Skipped: rate limit (429)"`.
---
## Task 2: Add Retry/Backoff to Integration Tests That Lack It
For integration tests that create real Agent instances without retry config, add `retries` and `exponential_backoff`:
**Before:**
```python
agent = Agent(model=OpenAIChat(id="gpt-4o-mini"), telemetry=False)
```
**After:**
```python
agent = Agent(
model=OpenAIChat(id="gpt-4o-mini"),
retries=3,
delay_between_retries=2,
exponential_backoff=True,
telemetry=False,
)
```
**Scope:** Apply this to tests under:
- `libs/agno/tests/integration/models/openai/`
- `libs/agno/tests/integration/models/anthropic/`
- `libs/agno/tests/integration/models/azure/`
- `libs/agno/tests/integration/models/aws/`
- `libs/agno/tests/integration/models/deepseek/`
- `libs/agno/tests/integration/models/mistral/`
- `libs/agno/tests/integration/models/cohere/`
- `libs/agno/tests/integration/models/xai/`
- `libs/agno/tests/integration/models/together/`
- `libs/agno/tests/integration/models/meta/`
- Any other provider tests that instantiate Agents without retry config
- `libs/agno/tests/integration/agent/` (all agent integration tests)
- `libs/agno/tests/integration/teams/` (all team integration tests)
- `libs/agno/tests/integration/workflows/` (all workflow integration tests)
**Use fixtures where possible.** If a file already has a model fixture like:
```python
@pytest.fixture(scope="module")
def openai_model():
return OpenAIChat(id="gpt-4o-mini")
```
Then add retry config to the Agent creation, not the model fixture. The Agent-level config is preferred.
**For tests that create Agents inline** (not via fixture), add the retry params directly.
**Do NOT add retries to:**
- Tests that specifically test error handling (e.g., `test_exception_handling`)
- Tests that explicitly test retry behavior (e.g., `test_retries.py`)
- Unit tests (they're all mocked)
---
## Task 3: Fix or Remove Genuinely Flaky Unit Tests
Run the full unit test suite and fix any failures:
```bash
pytest libs/agno/tests/unit/ -v 2>&1
```
**IMPORTANT: Only modify test files, not production code in `libs/agno/agno/`.**
### Known Failure Patterns (fix these first)
**1. Environment variable leaking into default arguments**
A tool class uses `getenv("SOME_KEY")` as a default parameter value in `__init__`. Default args are evaluated at import time, so any value set by direnv gets baked in permanently. Tests that manipulate `os.environ` at runtime can't override it.
Example: `libs/agno/agno/tools/jina.py`:
```python
def __init__(self, api_key: Optional[str] = getenv("JINA_API_KEY"), ...):
```
Fix: In the test, pass the value explicitly and don't rely on the default. Or use `monkeypatch.setattr` to patch the module-level default before construction. Or use `monkeypatch.delenv` / `monkeypatch.setenv` combined with re-importing.
Known files: `libs/agno/tests/unit/tools/test_jina.py`
**2. `time.time` mock exhaustion (StopIteration)**
Tests mock `time.time` with `side_effect=[list of values]`, but Python's `logging` module calls `time.time()` internally in `LogRecord.__init__`. These hidden calls consume values from the mock, causing `StopIteration`.
Fix: Use a callable that never exhausts instead of a finite list:
```python
# Instead of: mock_time.side_effect = [0, 0, 1, 2, 3, 4, 5, 6]
# Use:
call_count = 0
def fake_time():
nonlocal call_count
call_count += 1
return call_count * 0.5
mock_time.side_effect = fake_time
```
Or use `itertools.count()` or a generator that never exhausts.
Known files: `libs/agno/tests/unit/tools/test_opencv.py`
**3. Patching the class being tested (no-op patches)**
Tests do `with patch("agno.tools.jina.JinaReaderTools"):` but then call `JinaReaderTools()` using the already-imported reference. This patch does nothing useful.
Fix: Remove these no-op patches entirely.
Known files: `libs/agno/tests/unit/tools/test_jina.py`
**4. Rate limit / network issues in unit tests**
Tests making real HTTP calls may fail due to rate limits or network issues.
Fix: Ensure all external calls are properly mocked. Unit tests should never hit the network.
### General Flakiness Patterns
| Pattern | Fix |
|---------|-----|
| Tests that depend on execution order | Add proper fixtures / setup |
| Tests with hardcoded file paths | Use `tmp_path` or `tempfile` fixtures |
| Async tests with event loop conflicts | The `reset_async_client` autouse fixture in integration conftest handles this; check if unit tests need similar |
| Tests that mock interfaces that have changed | Update mock to match current signature |
| Import errors from optional dependencies | Add `@pytest.mark.skipif` guards |
**If a test is fundamentally unreliable and cannot be made deterministic, remove it** and note what it was testing so a replacement can be written.
---
## Task 4: Add New Unit Tests for Coverage Gaps
After stabilizing existing tests, check coverage:
```bash
pytest libs/agno/tests/unit/ --cov=libs/agno/agno --cov-report=term-missing -v
```
Focus new tests on **core modules** with the most impact:
1. **`agno/agent/`** — Agent initialization, run lifecycle, tool dispatch, error handling
2. **`agno/models/base.py`** — Retry logic, error classification, backoff calculation
3. **`agno/team/`** — Team coordination, member routing, delegation
4. **`agno/memory/`** — Memory storage, retrieval, summarization
5. **`agno/knowledge/`** — Knowledge base loading, chunking, retrieval
### Test Writing Guidelines
- Follow existing patterns in neighboring test files
- Use class-based organization: `class TestFeatureName:`
- Use `@pytest.mark.parametrize` for multiple inputs
- Mock ALL external dependencies — unit tests must never make network calls
- Write both sync AND async variants for public methods
- Use descriptive names: `test_<what>_<condition>_<expected_result>`
- Don't use f-strings where there are no variables
- Don't use emojis in test output or comments
---
## Task 5: Verify Everything Passes
After all changes, run:
```bash
# Full unit test suite
./scripts/test.sh
# Integration tests (expect some skips from rate limits — that's the point)
pytest libs/agno/tests/integration/ -v 2>&1 | tail -50
# Formatting and validation
./scripts/format.sh
./scripts/validate.sh
```
**Success criteria:**
- All unit tests pass (zero failures)
- Integration test failures are ONLY from missing API keys or services, never from rate limits (those should be skipped)
- `format.sh` and `validate.sh` pass clean
---
## Rules
- **Only modify test files** — do not change production code in `libs/agno/agno/`
- Keep fixes minimal and targeted
- Don't add unnecessary dependencies
- Run `format.sh` and `validate.sh` after making changes
---
## Working Order
Process tasks in this order:
1. **Task 1** first (rate-limit conftest hooks) — biggest bang for buck, purely additive
2. **Task 2** next (retry/backoff on agents) — reduces rate-limit hits in the first place
3. **Task 3** then (fix broken unit tests) — stabilize the base
4. **Task 4** last (new unit tests) — build on a stable foundation
5. **Task 5** final verification
---
## Output
Maintain a running log at `.context/test-improvement-log.md` with:
```markdown
## Rate-Limit Protection Added
- [ ] provider_name — conftest.py created/updated
## Retry/Backoff Added
- [ ] path/to/test_file.py — N agents updated
## Tests Fixed
- [ ] path/to/test_file.py::test_name — description of fix
## Tests Removed
- [ ] path/to/test_file.py::test_name — reason for removal
## Tests Added
- [ ] path/to/test_file.py::test_name — what gap it fills
## Final Status
- Unit tests: PASS/FAIL (N passed, N failed, N skipped)
- Integration tests: PASS/FAIL (N passed, N failed, N skipped)
- format.sh: PASS/FAIL
- validate.sh: PASS/FAIL
```
cookbook/02_agents/CLEANUP_PROMPT.md
-53
View File
@@ -1,53 +0,0 @@
Goal: Clean up and standardize `cookbook/02_agents` so all Python examples pass the structure checker and use current model conventions.
Context files (read these first):
- `AGENTS.md` — Project conventions, virtual environments, testing workflow
- `cookbook/STYLE_GUIDE.md` — Python file structure rules (canonical reference)
Environment:
- Python: `.venvs/demo/bin/python`
- API keys: loaded via `direnv allow`
Execution requirements:
1. Run the structure checker and fix all violations:
```
.venvs/demo/bin/python cookbook/scripts/check_cookbook_pattern.py --base-dir cookbook/02_agents --recursive
```
Common fixes:
- Add module docstring if missing
- Add `if __name__ == "__main__":` gate if missing
- Add "Create" and "Run" section banners if missing
- Ensure "Create" section appears before "Run" section
- Remove emoji characters
- Ensure section banners use `# ---------------------------------------------------------------------------` style (75 dashes)
2. Ensure imports are between the module docstring and the first section banner.
No imports should appear inside sections.
3. Migrate OpenAI model references. Only change files that use OpenAI models.
Do NOT modify files using other providers (Gemini, Claude, Groq, Ollama, etc.).
| Before | After |
|--------|-------|
| `from agno.models.openai import OpenAIChat` | `from agno.models.openai import OpenAIResponses` |
| `OpenAIChat(id="gpt-4o")` | `OpenAIResponses(id="gpt-5.2")` |
| `OpenAIChat(id="gpt-4o-mini")` | `OpenAIResponses(id="gpt-5.2-mini")` |
| Any other `OpenAIChat(` usage | `OpenAIResponses(` with updated model ID |
If a file imports both `OpenAIChat` and another provider, only change the OpenAI parts.
4. Make only minimal, behavior-preserving edits.
Do not change agent logic, tool configurations, prompts, or runtime behavior.
Validation commands (must all pass before finishing):
- `.venvs/demo/bin/python cookbook/scripts/check_cookbook_pattern.py --base-dir cookbook/02_agents --recursive`
Final response format:
1. Summary of changes made (structural fixes, model migrations).
2. Validation command output showing zero violations.
3. Files changed:
| File | Changes |
|------|---------|
| `path/to/file.py` | OpenAIChat -> OpenAIResponses, added main gate |
cookbook/02_agents/TEST_PROMPT.md
+1
View File
@@ -6,6 +6,7 @@ Context files (read these first):
Environment:
- Python: `.venvs/demo/bin/python`
- Created using `./scripts/demo_setup.sh`
- API keys: loaded via `direnv allow`
- Database: `./cookbook/scripts/run_pgvector.sh` (needed for RAG, session, knowledge examples)
cookbook/03_teams/CLEANUP_PROMPT.md
-53
View File
@@ -1,53 +0,0 @@
Goal: Clean up and standardize `cookbook/03_teams` so all Python examples pass the structure checker and use current model conventions.
Context files (read these first):
- `AGENTS.md` — Project conventions, virtual environments, testing workflow
- `cookbook/STYLE_GUIDE.md` — Python file structure rules (canonical reference)
Environment:
- Python: `.venvs/demo/bin/python`
- API keys: loaded via `direnv allow`
Execution requirements:
1. Run the structure checker and fix all violations:
```
.venvs/demo/bin/python cookbook/scripts/check_cookbook_pattern.py --base-dir cookbook/03_teams --recursive
```
Common fixes:
- Add module docstring if missing
- Add `if __name__ == "__main__":` gate if missing
- Add "Create" and "Run" section banners if missing
- Ensure "Create" section appears before "Run" section
- Remove emoji characters
- Ensure section banners use `# ---------------------------------------------------------------------------` style (75 dashes)
2. Ensure imports are between the module docstring and the first section banner.
No imports should appear inside sections.
3. Migrate OpenAI model references. Only change files that use OpenAI models.
Do NOT modify files using other providers (Gemini, Claude, Groq, Ollama, etc.).
| Before | After |
|--------|-------|
| `from agno.models.openai import OpenAIChat` | `from agno.models.openai import OpenAIResponses` |
| `OpenAIChat(id="gpt-4o")` | `OpenAIResponses(id="gpt-5.2")` |
| `OpenAIChat(id="gpt-4o-mini")` | `OpenAIResponses(id="gpt-5.2-mini")` |
| Any other `OpenAIChat(` usage | `OpenAIResponses(` with updated model ID |
If a file imports both `OpenAIChat` and another provider, only change the OpenAI parts.
4. Make only minimal, behavior-preserving edits.
Do not change agent logic, tool configurations, prompts, or runtime behavior.
Validation commands (must all pass before finishing):
- `.venvs/demo/bin/python cookbook/scripts/check_cookbook_pattern.py --base-dir cookbook/03_teams --recursive`
Final response format:
1. Summary of changes made (structural fixes, model migrations).
2. Validation command output showing zero violations.
3. Files changed:
| File | Changes |
|------|---------|
| `path/to/file.py` | OpenAIChat -> OpenAIResponses, added main gate |
cookbook/04_workflows/CLEANUP_PROMPT.md
-53
View File
@@ -1,53 +0,0 @@
Goal: Clean up and standardize `cookbook/04_workflows` so all Python examples pass the structure checker and use current model conventions.
Context files (read these first):
- `AGENTS.md` — Project conventions, virtual environments, testing workflow
- `cookbook/STYLE_GUIDE.md` — Python file structure rules (canonical reference)
Environment:
- Python: `.venvs/demo/bin/python`
- API keys: loaded via `direnv allow`
Execution requirements:
1. Run the structure checker and fix all violations:
```
.venvs/demo/bin/python cookbook/scripts/check_cookbook_pattern.py --base-dir cookbook/04_workflows --recursive
```
Common fixes:
- Add module docstring if missing
- Add `if __name__ == "__main__":` gate if missing
- Add "Create" and "Run" section banners if missing
- Ensure "Create" section appears before "Run" section
- Remove emoji characters
- Ensure section banners use `# ---------------------------------------------------------------------------` style (75 dashes)
2. Ensure imports are between the module docstring and the first section banner.
No imports should appear inside sections.
3. Migrate OpenAI model references. Only change files that use OpenAI models.
Do NOT modify files using other providers (Gemini, Claude, Groq, Ollama, etc.).
| Before | After |
|--------|-------|
| `from agno.models.openai import OpenAIChat` | `from agno.models.openai import OpenAIResponses` |
| `OpenAIChat(id="gpt-4o")` | `OpenAIResponses(id="gpt-5.2")` |
| `OpenAIChat(id="gpt-4o-mini")` | `OpenAIResponses(id="gpt-5.2-mini")` |
| Any other `OpenAIChat(` usage | `OpenAIResponses(` with updated model ID |
If a file imports both `OpenAIChat` and another provider, only change the OpenAI parts.
4. Make only minimal, behavior-preserving edits.
Do not change agent logic, tool configurations, prompts, or runtime behavior.
Validation commands (must all pass before finishing):
- `.venvs/demo/bin/python cookbook/scripts/check_cookbook_pattern.py --base-dir cookbook/04_workflows --recursive`
Final response format:
1. Summary of changes made (structural fixes, model migrations).
2. Validation command output showing zero violations.
3. Files changed:
| File | Changes |
|------|---------|
| `path/to/file.py` | OpenAIChat -> OpenAIResponses, added main gate |
cookbook/05_agent_os/CLEANUP_PROMPT.md
-53
View File
@@ -1,53 +0,0 @@
Goal: Clean up and standardize `cookbook/05_agent_os` so all Python examples pass the structure checker and use current model conventions.
Context files (read these first):
- `AGENTS.md` — Project conventions, virtual environments, testing workflow
- `cookbook/STYLE_GUIDE.md` — Python file structure rules (canonical reference)
Environment:
- Python: `.venvs/demo/bin/python`
- API keys: loaded via `direnv allow`
Execution requirements:
1. Run the structure checker and fix all violations:
```
.venvs/demo/bin/python cookbook/scripts/check_cookbook_pattern.py --base-dir cookbook/05_agent_os --recursive
```
Common fixes:
- Add module docstring if missing
- Add `if __name__ == "__main__":` gate if missing
- Add "Create" and "Run" section banners if missing
- Ensure "Create" section appears before "Run" section
- Remove emoji characters
- Ensure section banners use `# ---------------------------------------------------------------------------` style (75 dashes)
2. Ensure imports are between the module docstring and the first section banner.
No imports should appear inside sections.
3. Migrate OpenAI model references. Only change files that use OpenAI models.
Do NOT modify files using other providers (Gemini, Claude, Groq, Ollama, etc.).
| Before | After |
|--------|-------|
| `from agno.models.openai import OpenAIChat` | `from agno.models.openai import OpenAIResponses` |
| `OpenAIChat(id="gpt-4o")` | `OpenAIResponses(id="gpt-5.2")` |
| `OpenAIChat(id="gpt-4o-mini")` | `OpenAIResponses(id="gpt-5.2-mini")` |
| Any other `OpenAIChat(` usage | `OpenAIResponses(` with updated model ID |
If a file imports both `OpenAIChat` and another provider, only change the OpenAI parts.
4. Make only minimal, behavior-preserving edits.
Do not change agent logic, tool configurations, prompts, or runtime behavior.
Validation commands (must all pass before finishing):
- `.venvs/demo/bin/python cookbook/scripts/check_cookbook_pattern.py --base-dir cookbook/05_agent_os --recursive`
Final response format:
1. Summary of changes made (structural fixes, model migrations).
2. Validation command output showing zero violations.
3. Files changed:
| File | Changes |
|------|---------|
| `path/to/file.py` | OpenAIChat -> OpenAIResponses, added main gate |
cookbook/06_storage/CLEANUP_PROMPT.md
-53
View File
@@ -1,53 +0,0 @@
Goal: Clean up and standardize `cookbook/06_storage` so all Python examples pass the structure checker and use current model conventions.
Context files (read these first):
- `AGENTS.md` — Project conventions, virtual environments, testing workflow
- `cookbook/STYLE_GUIDE.md` — Python file structure rules (canonical reference)
Environment:
- Python: `.venvs/demo/bin/python`
- API keys: loaded via `direnv allow`
Execution requirements:
1. Run the structure checker and fix all violations:
```
.venvs/demo/bin/python cookbook/scripts/check_cookbook_pattern.py --base-dir cookbook/06_storage --recursive
```
Common fixes:
- Add module docstring if missing
- Add `if __name__ == "__main__":` gate if missing
- Add "Create" and "Run" section banners if missing
- Ensure "Create" section appears before "Run" section
- Remove emoji characters
- Ensure section banners use `# ---------------------------------------------------------------------------` style (75 dashes)
2. Ensure imports are between the module docstring and the first section banner.
No imports should appear inside sections.
3. Migrate OpenAI model references. Only change files that use OpenAI models.
Do NOT modify files using other providers (Gemini, Claude, Groq, Ollama, etc.).
| Before | After |
|--------|-------|
| `from agno.models.openai import OpenAIChat` | `from agno.models.openai import OpenAIResponses` |
| `OpenAIChat(id="gpt-4o")` | `OpenAIResponses(id="gpt-5.2")` |
| `OpenAIChat(id="gpt-4o-mini")` | `OpenAIResponses(id="gpt-5.2-mini")` |
| Any other `OpenAIChat(` usage | `OpenAIResponses(` with updated model ID |
If a file imports both `OpenAIChat` and another provider, only change the OpenAI parts.
4. Make only minimal, behavior-preserving edits.
Do not change agent logic, tool configurations, prompts, or runtime behavior.
Validation commands (must all pass before finishing):
- `.venvs/demo/bin/python cookbook/scripts/check_cookbook_pattern.py --base-dir cookbook/06_storage --recursive`
Final response format:
1. Summary of changes made (structural fixes, model migrations).
2. Validation command output showing zero violations.
3. Files changed:
| File | Changes |
|------|---------|
| `path/to/file.py` | OpenAIChat -> OpenAIResponses, added main gate |
cookbook/07_knowledge/CLEANUP_PROMPT.md
-54
View File
@@ -1,54 +0,0 @@
Goal: Clean up and standardize `cookbook/07_knowledge` so all Python examples pass the structure checker and use current model conventions.
Context files (read these first):
- `AGENTS.md` — Project conventions, virtual environments, testing workflow
- `cookbook/STYLE_GUIDE.md` — Python file structure rules (canonical reference)
Environment:
- Python: `.venvs/demo/bin/python`
- API keys: loaded via `direnv allow`
- Database: `./cookbook/scripts/run_pgvector.sh` (needed for knowledge examples)
Execution requirements:
1. Run the structure checker and fix all violations:
```
.venvs/demo/bin/python cookbook/scripts/check_cookbook_pattern.py --base-dir cookbook/07_knowledge --recursive
```
Common fixes:
- Add module docstring if missing
- Add `if __name__ == "__main__":` gate if missing
- Add "Create" and "Run" section banners if missing
- Ensure "Create" section appears before "Run" section
- Remove emoji characters
- Ensure section banners use `# ---------------------------------------------------------------------------` style (75 dashes)
2. Ensure imports are between the module docstring and the first section banner.
No imports should appear inside sections.
3. Migrate OpenAI model references. Only change files that use OpenAI models.
Do NOT modify files using other providers (Gemini, Claude, Groq, Ollama, etc.).
| Before | After |
|--------|-------|
| `from agno.models.openai import OpenAIChat` | `from agno.models.openai import OpenAIResponses` |
| `OpenAIChat(id="gpt-4o")` | `OpenAIResponses(id="gpt-5.2")` |
| `OpenAIChat(id="gpt-4o-mini")` | `OpenAIResponses(id="gpt-5.2-mini")` |
| Any other `OpenAIChat(` usage | `OpenAIResponses(` with updated model ID |
If a file imports both `OpenAIChat` and another provider, only change the OpenAI parts.
4. Make only minimal, behavior-preserving edits.
Do not change agent logic, tool configurations, prompts, or runtime behavior.
Validation commands (must all pass before finishing):
- `.venvs/demo/bin/python cookbook/scripts/check_cookbook_pattern.py --base-dir cookbook/07_knowledge --recursive`
Final response format:
1. Summary of changes made (structural fixes, model migrations).
2. Validation command output showing zero violations.
3. Files changed:
| File | Changes |
|------|---------|
| `path/to/file.py` | OpenAIChat -> OpenAIResponses, added main gate |
cookbook/08_learning/CLAUDE.md
-236
View File
@@ -1,236 +0,0 @@
# CLAUDE.md - Learning Machines Cookbook
Instructions for Claude Code when testing the Learning Machine cookbooks.
---
## Quick Reference
**Test Environment:**
```bash
# Virtual environment with all dependencies
.venvs/demo/bin/python
# Database (must be running)
PostgreSQL with PgVector at localhost:5532
```
**Run a cookbook:**
```bash
.venvs/demo/bin/python cookbook/08_learning/01_basics/1a_user_profile_always.py
```
**Test results file:**
```
cookbook/08_learning/TEST_LOG.md
```
---
## Testing Workflow
### 1. Before Testing
Ensure the database is running:
```bash
# If using Docker
./cookbook/scripts/run_pgvector.sh
```
### 2. Running Tests
Run individual cookbooks with the demo environment:
```bash
.venvs/demo/bin/python cookbook/08_learning/<folder>/<file>.py
```
Tail output for long tests:
```bash
.venvs/demo/bin/python cookbook/08_learning/01_basics/1a_user_profile_always.py 2>&1 | tail -100
```
### 3. Updating TEST_LOG.md
After each test, update `cookbook/08_learning/TEST_LOG.md` with:
- Test name and path
- Status: PASS or FAIL
- Brief description of what was tested
- Any notable observations or issues
Format:
```markdown
### filename.py
**Status:** PASS/FAIL
**Description:** What the test does and what was observed.
**Result:** Summary of success/failure.
---
```
---
## Code Locations
| Component | Location |
|-----------|----------|
| LearningMachine | `libs/agno/agno/learn/machine.py` |
| Config classes | `libs/agno/agno/learn/config.py` |
| Schema classes | `libs/agno/agno/learn/schemas.py` |
| Store Protocol | `libs/agno/agno/learn/stores/protocol.py` |
| UserProfileStore | `libs/agno/agno/learn/stores/user_profile.py` |
| UserMemoryStore | `libs/agno/agno/learn/stores/user_memory.py` |
| SessionContextStore | `libs/agno/agno/learn/stores/session_context.py` |
| EntityMemoryStore | `libs/agno/agno/learn/stores/entity_memory.py` |
| LearnedKnowledgeStore | `libs/agno/agno/learn/stores/learned_knowledge.py` |
| Agent integration | `libs/agno/agno/agent/agent.py` |
---
## Store Mode Support
| Store | ALWAYS | AGENTIC | PROPOSE | HITL |
|-------|--------|---------|---------|------|
| UserProfileStore | Yes | Yes | No | No |
| UserMemoryStore | Yes | Yes | No | No |
| SessionContextStore | Yes | No | No | No |
| EntityMemoryStore | Yes | Yes | No | No |
| LearnedKnowledgeStore | Yes | Yes | Yes | No |
---
## Cookbook Structure
```
cookbook/08_learning/
├── 01_basics/ # Basic tests for each store and mode
├── 02_user_profile/ # User profile specific tests
├── 03_session_context/ # Session context specific tests
├── 04_entity_memory/ # Entity memory specific tests
├── 05_learned_knowledge/ # Learned knowledge specific tests
├── 06_quick_tests/ # Quick validation tests (async, shorthand, edge cases)
├── 07_patterns/ # Combined pattern tests
├── 08_custom_stores/ # Custom store implementations
├── TEST_LOG.md # Test results log
└── CLAUDE.md # This file
```
---
## Key API Pattern
Access the learning machine from an agent:
```python
# Get the resolved LearningMachine instance
learning_machine = agent.learning_machine
# Access individual stores
learning_machine.user_profile_store.print(user_id=user_id)
learning_machine.user_memory_store.print(user_id=user_id)
learning_machine.session_context_store.print(session_id=session_id)
learning_machine.entity_memory_store.search(query="...", limit=10)
learning_machine.learned_knowledge_store.print(query="...")
```
---
## Known Issues
1. **Design docs missing**: The `specs/learning-machine/` directory referenced in the main CLAUDE.md doesn't exist.
2. **HITL mode**: Not implemented in any store - warnings are logged but no actual human-in-the-loop functionality.
3. **PROPOSE mode**: Only supported in LearnedKnowledgeStore.
4. **Model in cookbooks**: Uses `gpt-5.2` which may not be available to all users.
5. **Claude model version matters**: Use `claude-sonnet-4-5` or newer. Older model IDs (e.g., `claude-sonnet-4-20250514`) don't support structured outputs and extraction will fail. See `06_quick_tests/04_claude_model.py`.
6. **Lazy initialization**: `agent.learning_machine` now lazily initializes, so it works even before the agent runs. There is no need to call `print_response()` first.
7. **upsert_learning doesn't update learning_type**: The `upsert_learning` method in postgres.py (and likely other DB adapters) only updates `content`, `metadata`, and `updated_at` on conflict - it does NOT update `learning_type`. This means if you rename a learning type (e.g., `memories` → `user_memory`), existing rows won't be found. **Future work:** Fix `upsert_learning` to update `learning_type` on conflict.
---
## Custom Stores
Create custom learning stores by inheriting from the `LearningStore` protocol:
```python
from dataclasses import dataclass, field
from typing import Any, Callable, Dict, List, Optional
from agno.learn.stores.protocol import LearningStore
@dataclass
class ProjectContextStore(LearningStore):
"""Custom store for project-specific context."""
# Pass custom context at construction time (pattern choice, not required by protocol)
context: Dict[str, Any] = field(default_factory=dict)
@property
def learning_type(self) -> str:
return "project_context"
@property
def schema(self) -> Any:
return dict
def recall(self, **kwargs) -> Optional[Any]:
project_id = self.context.get("project_id")
# Retrieve from your storage...
async def arecall(self, **kwargs) -> Optional[Any]:
return self.recall(**kwargs)
def process(self, messages: List[Any], **kwargs) -> None:
# Extract and save relevant context
pass
async def aprocess(self, messages: List[Any], **kwargs) -> None:
self.process(messages, **kwargs)
def build_context(self, data: Any) -> str:
if not data:
return ""
return f"<project_context>\n{data}\n</project_context>"
def get_tools(self, **kwargs) -> List[Callable]:
return []
async def aget_tools(self, **kwargs) -> List[Callable]:
return []
@property
def was_updated(self) -> bool:
return False
# Plug into LearningMachine
learning = LearningMachine(
custom_stores={
"project": ProjectContextStore(
context={"project_id": "my-project"},
),
},
)
```
See `08_custom_stores/` for complete examples.
---
## Debugging
Enable debug output:
```python
import os
os.environ["AGNO_DEBUG"] = "true"
```
Or in the store:
```python
store = UserProfileStore(config=..., debug_mode=True)
```
cookbook/08_learning/CLEANUP_PROMPT.md
-54
View File
@@ -1,54 +0,0 @@
Goal: Clean up and standardize `cookbook/08_learning` so all Python examples pass the structure checker and use current model conventions.
Context files (read these first):
- `AGENTS.md` — Project conventions, virtual environments, testing workflow
- `cookbook/STYLE_GUIDE.md` — Python file structure rules (canonical reference)
Environment:
- Python: `.venvs/demo/bin/python`
- API keys: loaded via `direnv allow`
- Database: `./cookbook/scripts/run_pgvector.sh` (needed for learning examples)
Execution requirements:
1. Run the structure checker and fix all violations:
```
.venvs/demo/bin/python cookbook/scripts/check_cookbook_pattern.py --base-dir cookbook/08_learning --recursive
```
Common fixes:
- Add module docstring if missing
- Add `if __name__ == "__main__":` gate if missing
- Add "Create" and "Run" section banners if missing
- Ensure "Create" section appears before "Run" section
- Remove emoji characters
- Ensure section banners use `# ---------------------------------------------------------------------------` style (75 dashes)
2. Ensure imports are between the module docstring and the first section banner.
No imports should appear inside sections.
3. Migrate OpenAI model references. Only change files that use OpenAI models.
Do NOT modify files using other providers (Gemini, Claude, Groq, Ollama, etc.).
| Before | After |
|--------|-------|
| `from agno.models.openai import OpenAIChat` | `from agno.models.openai import OpenAIResponses` |
| `OpenAIChat(id="gpt-4o")` | `OpenAIResponses(id="gpt-5.2")` |
| `OpenAIChat(id="gpt-4o-mini")` | `OpenAIResponses(id="gpt-5.2-mini")` |
| Any other `OpenAIChat(` usage | `OpenAIResponses(` with updated model ID |
If a file imports both `OpenAIChat` and another provider, only change the OpenAI parts.
4. Make only minimal, behavior-preserving edits.
Do not change agent logic, tool configurations, prompts, or runtime behavior.
Validation commands (must all pass before finishing):
- `.venvs/demo/bin/python cookbook/scripts/check_cookbook_pattern.py --base-dir cookbook/08_learning --recursive`
Final response format:
1. Summary of changes made (structural fixes, model migrations).
2. Validation command output showing zero violations.
3. Files changed:
| File | Changes |
|------|---------|
| `path/to/file.py` | OpenAIChat -> OpenAIResponses, added main gate |
cookbook/09_evals/CLEANUP_PROMPT.md
-53
View File
@@ -1,53 +0,0 @@
Goal: Clean up and standardize `cookbook/09_evals` so all Python examples pass the structure checker and use current model conventions.
Context files (read these first):
- `AGENTS.md` — Project conventions, virtual environments, testing workflow
- `cookbook/STYLE_GUIDE.md` — Python file structure rules (canonical reference)
Environment:
- Python: `.venvs/demo/bin/python`
- API keys: loaded via `direnv allow`
Execution requirements:
1. Run the structure checker and fix all violations:
```
.venvs/demo/bin/python cookbook/scripts/check_cookbook_pattern.py --base-dir cookbook/09_evals --recursive
```
Common fixes:
- Add module docstring if missing
- Add `if __name__ == "__main__":` gate if missing
- Add "Create" and "Run" section banners if missing
- Ensure "Create" section appears before "Run" section
- Remove emoji characters
- Ensure section banners use `# ---------------------------------------------------------------------------` style (75 dashes)
2. Ensure imports are between the module docstring and the first section banner.
No imports should appear inside sections.
3. Migrate OpenAI model references. Only change files that use OpenAI models.
Do NOT modify files using other providers (Gemini, Claude, Groq, Ollama, etc.).
| Before | After |
|--------|-------|
| `from agno.models.openai import OpenAIChat` | `from agno.models.openai import OpenAIResponses` |
| `OpenAIChat(id="gpt-4o")` | `OpenAIResponses(id="gpt-5.2")` |
| `OpenAIChat(id="gpt-4o-mini")` | `OpenAIResponses(id="gpt-5.2-mini")` |
| Any other `OpenAIChat(` usage | `OpenAIResponses(` with updated model ID |
If a file imports both `OpenAIChat` and another provider, only change the OpenAI parts.
4. Make only minimal, behavior-preserving edits.
Do not change agent logic, tool configurations, prompts, or runtime behavior.
Validation commands (must all pass before finishing):
- `.venvs/demo/bin/python cookbook/scripts/check_cookbook_pattern.py --base-dir cookbook/09_evals --recursive`
Final response format:
1. Summary of changes made (structural fixes, model migrations).
2. Validation command output showing zero violations.
3. Files changed:
| File | Changes |
|------|---------|
| `path/to/file.py` | OpenAIChat -> OpenAIResponses, added main gate |
cookbook/10_reasoning/CLEANUP_PROMPT.md
-53
View File
@@ -1,53 +0,0 @@
Goal: Clean up and standardize `cookbook/10_reasoning` so all Python examples pass the structure checker and use current model conventions.
Context files (read these first):
- `AGENTS.md` — Project conventions, virtual environments, testing workflow
- `cookbook/STYLE_GUIDE.md` — Python file structure rules (canonical reference)
Environment:
- Python: `.venvs/demo/bin/python`
- API keys: loaded via `direnv allow`
Execution requirements:
1. Run the structure checker and fix all violations:
```
.venvs/demo/bin/python cookbook/scripts/check_cookbook_pattern.py --base-dir cookbook/10_reasoning --recursive
```
Common fixes:
- Add module docstring if missing
- Add `if __name__ == "__main__":` gate if missing
- Add "Create" and "Run" section banners if missing
- Ensure "Create" section appears before "Run" section
- Remove emoji characters
- Ensure section banners use `# ---------------------------------------------------------------------------` style (75 dashes)
2. Ensure imports are between the module docstring and the first section banner.
No imports should appear inside sections.
3. Migrate OpenAI model references. Only change files that use OpenAI models.
Do NOT modify files using other providers (Gemini, Claude, Groq, Ollama, etc.).
| Before | After |
|--------|-------|
| `from agno.models.openai import OpenAIChat` | `from agno.models.openai import OpenAIResponses` |
| `OpenAIChat(id="gpt-4o")` | `OpenAIResponses(id="gpt-5.2")` |
| `OpenAIChat(id="gpt-4o-mini")` | `OpenAIResponses(id="gpt-5.2-mini")` |
| Any other `OpenAIChat(` usage | `OpenAIResponses(` with updated model ID |
If a file imports both `OpenAIChat` and another provider, only change the OpenAI parts.
4. Make only minimal, behavior-preserving edits.
Do not change agent logic, tool configurations, prompts, or runtime behavior.
Validation commands (must all pass before finishing):
- `.venvs/demo/bin/python cookbook/scripts/check_cookbook_pattern.py --base-dir cookbook/10_reasoning --recursive`
Final response format:
1. Summary of changes made (structural fixes, model migrations).
2. Validation command output showing zero violations.
3. Files changed:
| File | Changes |
|------|---------|
| `path/to/file.py` | OpenAIChat -> OpenAIResponses, added main gate |
cookbook/11_memory/CLAUDE.md
-80
View File
@@ -1,80 +0,0 @@
# CLAUDE.md - Memory Cookbook
Instructions for Claude Code when testing the memory cookbooks.
---
## Overview
This folder contains **memory management** examples - how agents remember user information, share memories, and use memory tools.
**Total Examples:** 21
---
## Quick Reference
**Test Environment:**
```bash
.venvs/demo/bin/python
./cookbook/scripts/run_pgvector.sh # For database-backed memory
```
**Run a cookbook:**
```bash
.venvs/demo/bin/python cookbook/80_memory/01_agent_with_memory.py
```
---
## Folder Structure
| File/Folder | Description |
|:------------|:------------|
| `01_agent_with_memory.py` | Basic memory manager |
| `02_agentic_memory.py` | Agentic memory updates |
| `03_agents_share_memory.py` | Shared memory between agents |
| `04_custom_memory_manager.py` | Custom memory implementation |
| `05_multi_user_multi_session_chat.py` | Multi-tenant memory |
| `06_multi_user_multi_session_chat_concurrent.py` | Concurrent access |
| `07_share_memory_and_history_between_agents.py` | Memory + history sharing |
| `08_memory_tools.py` | Memory as tools |
| `memory_manager/` | MemoryManager patterns |
| `optimize_memories/` | Memory optimization strategies |
---
## Key Patterns
### Basic Memory
Agent remembers facts about users across sessions.
### Agentic Memory
Agent decides when to save memories based on context.
### Shared Memory
Multiple agents access the same memory store.
### Memory Tools
Explicit tools for memory operations.
---
## Testing Order
1. `01_agent_with_memory.py` - Basic pattern
2. `02_agentic_memory.py` - Automatic updates
3. `03_agents_share_memory.py` - Multi-agent
---
## API Keys Required
- `OPENAI_API_KEY` - Most examples
---
## Dependencies
- PostgreSQL with PgVector for persistent memory
- SQLite available for local testing
cookbook/11_memory/CLEANUP_PROMPT.md
-54
View File
@@ -1,54 +0,0 @@
Goal: Clean up and standardize `cookbook/11_memory` so all Python examples pass the structure checker and use current model conventions.
Context files (read these first):
- `AGENTS.md` — Project conventions, virtual environments, testing workflow
- `cookbook/STYLE_GUIDE.md` — Python file structure rules (canonical reference)
Environment:
- Python: `.venvs/demo/bin/python`
- API keys: loaded via `direnv allow`
- Database: `./cookbook/scripts/run_pgvector.sh` (needed for memory examples)
Execution requirements:
1. Run the structure checker and fix all violations:
```
.venvs/demo/bin/python cookbook/scripts/check_cookbook_pattern.py --base-dir cookbook/11_memory --recursive
```
Common fixes:
- Add module docstring if missing
- Add `if __name__ == "__main__":` gate if missing
- Add "Create" and "Run" section banners if missing
- Ensure "Create" section appears before "Run" section
- Remove emoji characters
- Ensure section banners use `# ---------------------------------------------------------------------------` style (75 dashes)
2. Ensure imports are between the module docstring and the first section banner.
No imports should appear inside sections.
3. Migrate OpenAI model references. Only change files that use OpenAI models.
Do NOT modify files using other providers (Gemini, Claude, Groq, Ollama, etc.).
| Before | After |
|--------|-------|
| `from agno.models.openai import OpenAIChat` | `from agno.models.openai import OpenAIResponses` |
| `OpenAIChat(id="gpt-4o")` | `OpenAIResponses(id="gpt-5.2")` |
| `OpenAIChat(id="gpt-4o-mini")` | `OpenAIResponses(id="gpt-5.2-mini")` |
| Any other `OpenAIChat(` usage | `OpenAIResponses(` with updated model ID |
If a file imports both `OpenAIChat` and another provider, only change the OpenAI parts.
4. Make only minimal, behavior-preserving edits.
Do not change agent logic, tool configurations, prompts, or runtime behavior.
Validation commands (must all pass before finishing):
- `.venvs/demo/bin/python cookbook/scripts/check_cookbook_pattern.py --base-dir cookbook/11_memory --recursive`
Final response format:
1. Summary of changes made (structural fixes, model migrations).
2. Validation command output showing zero violations.
3. Files changed:
| File | Changes |
|------|---------|
| `path/to/file.py` | OpenAIChat -> OpenAIResponses, added main gate |
cookbook/90_models/CLEANUP_PROMPT.md
-44
View File
@@ -1,44 +0,0 @@
Goal: Clean up and standardize `cookbook/90_models` so all Python examples pass the structure checker.
Context files (read these first):
- `AGENTS.md` — Project conventions, virtual environments, testing workflow
- `cookbook/STYLE_GUIDE.md` — Python file structure rules (canonical reference)
Environment:
- Python: `.venvs/demo/bin/python`
- API keys: loaded via `direnv allow`
Note: This directory contains provider-specific model examples. Do NOT migrate model classes
or model IDs — the specific model usage in each file is intentional.
Execution requirements:
1. Run the structure checker and fix all violations:
```
.venvs/demo/bin/python cookbook/scripts/check_cookbook_pattern.py --base-dir cookbook/90_models --recursive
```
Common fixes:
- Add module docstring if missing
- Add `if __name__ == "__main__":` gate if missing
- Add "Create" and "Run" section banners if missing
- Ensure "Create" section appears before "Run" section
- Remove emoji characters
- Ensure section banners use `# ---------------------------------------------------------------------------` style (75 dashes)
2. Ensure imports are between the module docstring and the first section banner.
No imports should appear inside sections.
3. Make only minimal, behavior-preserving edits.
Do not change agent logic, tool configurations, model choices, prompts, or runtime behavior.
Validation commands (must all pass before finishing):
- `.venvs/demo/bin/python cookbook/scripts/check_cookbook_pattern.py --base-dir cookbook/90_models --recursive`
Final response format:
1. Summary of changes made (structural fixes).
2. Validation command output showing zero violations.
3. Files changed:
| File | Changes |
|------|---------|
| `path/to/file.py` | Added main gate, fixed banner style |
cookbook/91_tools/CLEANUP_PROMPT.md
-44
View File
@@ -1,44 +0,0 @@
Goal: Clean up and standardize `cookbook/91_tools` so all Python examples pass the structure checker.
Context files (read these first):
- `AGENTS.md` — Project conventions, virtual environments, testing workflow
- `cookbook/STYLE_GUIDE.md` — Python file structure rules (canonical reference)
Environment:
- Python: `.venvs/demo/bin/python`
- API keys: loaded via `direnv allow`
Note: This directory contains tool-specific examples. Do NOT migrate model classes
or model IDs — the specific model usage in each file is intentional.
Execution requirements:
1. Run the structure checker and fix all violations:
```
.venvs/demo/bin/python cookbook/scripts/check_cookbook_pattern.py --base-dir cookbook/91_tools --recursive
```
Common fixes:
- Add module docstring if missing
- Add `if __name__ == "__main__":` gate if missing
- Add "Create" and "Run" section banners if missing
- Ensure "Create" section appears before "Run" section
- Remove emoji characters
- Ensure section banners use `# ---------------------------------------------------------------------------` style (75 dashes)
2. Ensure imports are between the module docstring and the first section banner.
No imports should appear inside sections.
3. Make only minimal, behavior-preserving edits.
Do not change agent logic, tool configurations, model choices, prompts, or runtime behavior.
Validation commands (must all pass before finishing):
- `.venvs/demo/bin/python cookbook/scripts/check_cookbook_pattern.py --base-dir cookbook/91_tools --recursive`
Final response format:
1. Summary of changes made (structural fixes).
2. Validation command output showing zero violations.
3. Files changed:
| File | Changes |
|------|---------|
| `path/to/file.py` | Added main gate, fixed banner style |
cookbook/92_integrations/CLAUDE.md
-88
View File
@@ -1,88 +0,0 @@
# CLAUDE.md - Integrations Cookbook
Instructions for Claude Code when testing the integrations cookbooks.
---
## Overview
This folder contains **third-party integration** examples - observability platforms, memory services, and messaging platforms.
**Total Examples:** 40
**Organization:** By integration type
---
## Quick Reference
**Test Environment:**
```bash
.venvs/demo/bin/python
```
**Run a cookbook:**
```bash
.venvs/demo/bin/python cookbook/92_integrations/observability/langfuse_via_openinference.py
```
---
## Folder Structure
| Folder | Description |
|:-------|:------------|
| `a2a/` | Agent-to-agent protocol |
| `discord/` | Discord bot integration |
| `memory/` | External memory services |
| `observability/` | Tracing platforms |
| `rag/` | Third-party RAG integrations |
| `surrealdb/` | SurrealDB memory integration |
---
## Observability Platforms
| Platform | File |
|:---------|:-----|
| AgentOps | `agent_ops.py` |
| Arize Phoenix | `arize_phoenix_*.py` |
| Langfuse | `langfuse_*.py` |
| LangSmith | `langsmith_*.py` |
| Langtrace | `langtrace_op.py` |
| LangWatch | `langwatch_op.py` |
| Logfire | `logfire_*.py` |
| Opik | `opik_*.py` |
| Traceloop | `traceloop_op.py` |
| Weave | `weave_op.py` |
| Maxim | `maxim_ops.py` |
| Atla | `atla_op.py` |
---
## Memory Services
| Service | File |
|:--------|:-----|
| Mem0 | `mem0_integration.py` |
| Memori | `memori_integration.py` |
| Zep | `zep_integration.py` |
---
## Messaging
| Platform | Files |
|:---------|:------|
| Discord | `basic.py`, `agent_with_media.py`, `agent_with_user_memory.py` |
---
## Testing Priorities
### High Priority
- `observability/langfuse_via_openinference.py` - Popular choice
- `discord/basic.py` - Discord integration
### Notes
- Most observability integrations require API keys
- Discord requires bot token and permissions
cookbook/92_integrations/CLEANUP_PROMPT.md
-44
View File
@@ -1,44 +0,0 @@
Goal: Clean up and standardize `cookbook/92_integrations` so all Python examples pass the structure checker.
Context files (read these first):
- `AGENTS.md` — Project conventions, virtual environments, testing workflow
- `cookbook/STYLE_GUIDE.md` — Python file structure rules (canonical reference)
Environment:
- Python: `.venvs/demo/bin/python`
- API keys: loaded via `direnv allow`
Note: This directory contains integration-specific examples. Do NOT migrate model classes
or model IDs — the specific model usage in each file is intentional.
Execution requirements:
1. Run the structure checker and fix all violations:
```
.venvs/demo/bin/python cookbook/scripts/check_cookbook_pattern.py --base-dir cookbook/92_integrations --recursive
```
Common fixes:
- Add module docstring if missing
- Add `if __name__ == "__main__":` gate if missing
- Add "Create" and "Run" section banners if missing
- Ensure "Create" section appears before "Run" section
- Remove emoji characters
- Ensure section banners use `# ---------------------------------------------------------------------------` style (75 dashes)
2. Ensure imports are between the module docstring and the first section banner.
No imports should appear inside sections.
3. Make only minimal, behavior-preserving edits.
Do not change agent logic, tool configurations, model choices, prompts, or runtime behavior.
Validation commands (must all pass before finishing):
- `.venvs/demo/bin/python cookbook/scripts/check_cookbook_pattern.py --base-dir cookbook/92_integrations --recursive`
Final response format:
1. Summary of changes made (structural fixes).
2. Validation command output showing zero violations.
3. Files changed:
| File | Changes |
|------|---------|
| `path/to/file.py` | Added main gate, fixed banner style |
cookbook/93_components/CLEANUP_PROMPT.md
-44
View File
@@ -1,44 +0,0 @@
Goal: Clean up and standardize `cookbook/93_components` so all Python examples pass the structure checker.
Context files (read these first):
- `AGENTS.md` — Project conventions, virtual environments, testing workflow
- `cookbook/STYLE_GUIDE.md` — Python file structure rules (canonical reference)
Environment:
- Python: `.venvs/demo/bin/python`
- API keys: loaded via `direnv allow`
Note: This directory contains component-specific examples. Do NOT migrate model classes
or model IDs — the specific model usage in each file is intentional.
Execution requirements:
1. Run the structure checker and fix all violations:
```
.venvs/demo/bin/python cookbook/scripts/check_cookbook_pattern.py --base-dir cookbook/93_components --recursive
```
Common fixes:
- Add module docstring if missing
- Add `if __name__ == "__main__":` gate if missing
- Add "Create" and "Run" section banners if missing
- Ensure "Create" section appears before "Run" section
- Remove emoji characters
- Ensure section banners use `# ---------------------------------------------------------------------------` style (75 dashes)
2. Ensure imports are between the module docstring and the first section banner.
No imports should appear inside sections.
3. Make only minimal, behavior-preserving edits.
Do not change agent logic, tool configurations, model choices, prompts, or runtime behavior.
Validation commands (must all pass before finishing):
- `.venvs/demo/bin/python cookbook/scripts/check_cookbook_pattern.py --base-dir cookbook/93_components --recursive`
Final response format:
1. Summary of changes made (structural fixes).
2. Validation command output showing zero violations.
3. Files changed:
| File | Changes |
|------|---------|
| `path/to/file.py` | Added main gate, fixed banner style |
cookbook/STYLE_GUIDE.md
-14
View File
@@ -50,17 +50,3 @@ example_agent = Agent(...)
if __name__ == "__main__":
example_agent.print_response("...", stream=True)
```
## Validation
Run structure checks:
```bash
.venvs/demo/bin/python cookbook/scripts/check_cookbook_pattern.py --base-dir cookbook/00_quickstart
```
Run metadata checks:
```bash
.venvs/demo/bin/python cookbook/scripts/audit_cookbook_metadata.py --scope direct
```
cookbook/scripts/audit_cookbook_metadata.py
-228
View File
@@ -1,228 +0,0 @@
#!/usr/bin/env python3
"""Audit cookbook folders for required metadata files.
Checks cookbook folders that contain runnable Python examples and verifies:
- README.md
- TEST_LOG.md
Usage examples:
python3 cookbook/scripts/audit_cookbook_metadata.py
python3 cookbook/scripts/audit_cookbook_metadata.py --scope recursive --fail-on-missing
python3 cookbook/scripts/audit_cookbook_metadata.py --output-format json --write-json .context/cookbook-metadata.json
"""
from __future__ import annotations
import argparse
import json
import os
import sys
from dataclasses import asdict, dataclass
from pathlib import Path
from typing import Iterable
REQUIRED_FILES = ("README.md", "TEST_LOG.md")
SKIP_DIR_NAMES = {"__pycache__", ".git", ".context"}
SKIP_PATH_PREFIXES = {"scripts"}
@dataclass
class FolderAudit:
path: str
has_readme: bool
has_test_log: bool
missing_files: list[str]
def is_example_python_file(path: Path) -> bool:
return path.is_file() and path.suffix == ".py" and path.name != "__init__.py"
def should_skip(path: Path, base_dir: Path) -> bool:
rel_parts = path.relative_to(base_dir).parts
if any(part in SKIP_DIR_NAMES for part in rel_parts):
return True
if rel_parts and rel_parts[0] in SKIP_PATH_PREFIXES:
return True
return False
def has_examples(path: Path, scope: str) -> bool:
if scope == "direct":
return any(is_example_python_file(child) for child in path.iterdir())
if scope == "recursive":
return any(is_example_python_file(child) for child in path.rglob("*.py"))
raise ValueError(f"Unsupported scope: {scope}")
def iter_candidate_dirs(base_dir: Path) -> Iterable[Path]:
for root, dirnames, _ in os.walk(base_dir):
path = Path(root)
if should_skip(path, base_dir):
dirnames[:] = []
continue
dirnames[:] = [
d
for d in dirnames
if d not in SKIP_DIR_NAMES and not should_skip(path / d, base_dir)
]
yield path
def audit(base_dir: Path, scope: str) -> list[FolderAudit]:
audits: list[FolderAudit] = []
for folder in iter_candidate_dirs(base_dir):
if folder == base_dir:
continue
if not has_examples(folder, scope=scope):
continue
has_readme = (folder / "README.md").exists()
has_test_log = (folder / "TEST_LOG.md").exists()
missing = [name for name in REQUIRED_FILES if not (folder / name).exists()]
audits.append(
FolderAudit(
path=folder.as_posix(),
has_readme=has_readme,
has_test_log=has_test_log,
missing_files=missing,
)
)
audits.sort(key=lambda row: row.path)
return audits
def build_summary(rows: list[FolderAudit]) -> dict[str, int]:
total = len(rows)
missing_any = sum(1 for row in rows if row.missing_files)
missing_readme = sum(1 for row in rows if not row.has_readme)
missing_test_log = sum(1 for row in rows if not row.has_test_log)
compliant = total - missing_any
return {
"total_folders": total,
"compliant_folders": compliant,
"missing_any": missing_any,
"missing_readme": missing_readme,
"missing_test_log": missing_test_log,
}
def render_text(
rows: list[FolderAudit], summary: dict[str, int], base_dir: Path
) -> str:
lines: list[str] = []
lines.append(f"Cookbook metadata audit for {base_dir.as_posix()}")
lines.append("")
lines.append(
"Summary: "
f"total={summary['total_folders']} "
f"compliant={summary['compliant_folders']} "
f"missing_any={summary['missing_any']} "
f"missing_readme={summary['missing_readme']} "
f"missing_test_log={summary['missing_test_log']}"
)
lines.append("")
lines.append("Folder results:")
for row in rows:
status = (
"OK" if not row.missing_files else "MISSING " + ",".join(row.missing_files)
)
lines.append(f"- [{status}] {row.path}")
return "\n".join(lines)
def parse_args() -> argparse.Namespace:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument(
"--base-dir",
default="cookbook",
help="Directory to audit (default: cookbook).",
)
parser.add_argument(
"--scope",
choices=["direct", "recursive"],
default="direct",
help="direct: folder has .py files directly. recursive: folder has .py files in any descendants.",
)
parser.add_argument(
"--min-depth",
type=int,
default=1,
help="Minimum folder depth under base-dir to include (default: 1).",
)
parser.add_argument(
"--max-depth",
type=int,
default=2,
help="Maximum folder depth under base-dir to include (default: 2). Use -1 for no limit.",
)
parser.add_argument(
"--output-format",
choices=["text", "json"],
default="text",
help="Render output as plain text or JSON (default: text).",
)
parser.add_argument(
"--write-json",
default=None,
help="Optional path to write JSON output.",
)
parser.add_argument(
"--fail-on-missing",
action="store_true",
help="Exit 1 when any folder is missing required files.",
)
return parser.parse_args()
def main() -> int:
args = parse_args()
base_dir = Path(args.base_dir).resolve()
if args.min_depth < 1:
print("Error: --min-depth must be >= 1", file=sys.stderr)
return 2
if args.max_depth != -1 and args.max_depth < args.min_depth:
print(
"Error: --max-depth must be >= --min-depth, or -1",
file=sys.stderr,
)
return 2
raw_rows = audit(base_dir=base_dir, scope=args.scope)
rows: list[FolderAudit] = []
for row in raw_rows:
folder = Path(row.path)
depth = len(folder.relative_to(base_dir).parts)
if depth < args.min_depth:
continue
if args.max_depth != -1 and depth > args.max_depth:
continue
rows.append(row)
summary = build_summary(rows)
payload = {
"base_dir": base_dir.as_posix(),
"scope": args.scope,
"min_depth": args.min_depth,
"max_depth": args.max_depth,
"required_files": list(REQUIRED_FILES),
"summary": summary,
"folders": [asdict(row) for row in rows],
}
if args.output_format == "json":
print(json.dumps(payload, indent=2))
else:
print(render_text(rows, summary=summary, base_dir=base_dir))
if args.write_json:
output_path = Path(args.write_json)
output_path.parent.mkdir(parents=True, exist_ok=True)
output_path.write_text(json.dumps(payload, indent=2) + "\n", encoding="utf-8")
if args.fail_on_missing and summary["missing_any"] > 0:
return 1
return 0
if __name__ == "__main__":
raise SystemExit(main())
cookbook/templates/README.template.md
-41
View File
@@ -1,41 +0,0 @@
# <Cookbook Name>
Short description of what this cookbook demonstrates.
## What It Covers
- Pattern 1
- Pattern 2
- Pattern 3
## Files
| File | Purpose |
|------|---------|
| `example.py` | What this example does |
## Prerequisites
- Python environment: `.venvs/demo/bin/python`
- Required environment variables: `<VAR_1>`, `<VAR_2>`
- Required services (if any): `<service>`
## Run
```bash
.venvs/demo/bin/python cookbook/<folder>/<example.py>
```
If the example is package-based, run it as a module:
```bash
.venvs/demo/bin/python -m cookbook.<folder>.<module_name>
```
## Expected Outcome
Describe what success looks like when this cookbook runs.
## Test Log
Update and maintain `TEST_LOG.md` in this folder after each run.
cookbook/templates/TEST_LOG.template.md
-20
View File
@@ -1,20 +0,0 @@
# TEST_LOG.md - <Cookbook Name>
Test results for `cookbook/<folder>/`.
**Test Date:** YYYY-MM-DD
**Environment:** `.venvs/demo/bin/python`
**Model(s):** `<model-id>`
**Services:** `<db/service info if relevant>`
---
### <filename.py>
**Status:** PASS/FAIL
**Description:** What the test does and what was observed.
**Result:** Summary of success/failure.
---