agno-agi/agno: Build, run, and manage agent platforms. - agno

mirror of https://github.com/agno-agi/agno.git synced 2026-10-12 02:10:54 +00:00

T

f3aa7be6af fix: create a separate approval record for each approval-required tool call in a run (#10812 )

## Summary

Each approval-required (`@approval`) tool call in a run now gets its own
approval record. This applies to agents and teams.

### The issue

When a run paused a second time for a new `@approval` tool call, no
approval record was created for that call. The next continue then ran it
using the approval resolved for the first call.

Example with `pay_invoice` decorated with `@approval` +
`@tool(requires_confirmation=True)`:

1. The run pauses on `pay_invoice(INV-1)`, and one pending record is
created.
2. The admin approves it, and the run is continued by `run_id` with no
requirements (the documented database-resolution flow).
3. INV-1 is paid. The model calls `pay_invoice(INV-2)`, and the run
pauses again.
4. No pending record exists for INV-2, so there is nothing for the admin
to approve.
5. Continuing the run pays INV-2 anyway. The run has one approval
record, and it names INV-1 only.

There were two causes, both in `agno/run/approval.py`:

- **No record for the second call.** `create_approval_from_pause`
skipped creating a record if any required tool in `run_response.tools`
already had an `approval_id`. That list grows across every pause in a
run, so the earlier call (already approved and run) made the check
return the old id.
- **The old record was reused.** The new call had no `approval_id`, so
`_group_tools_by_approval` used the run-level fallback. That fallback
returned the run's latest record, which was the approved INV-1 record,
and the new call was confirmed with it.

### The fix

- **`create_approval_from_pause` / `acreate_approval_from_pause`:** a
record is skipped only when every required tool already has an
`approval_id`. Tools without one get a new record.
`_build_approval_dict` builds the record from the current pause only:
`tool_name`, `tool_args` and `context.tool_names` come from the new
approval calls, and `requirements` lists those calls plus any plain HITL
calls paused in the same turn. The second record names INV-2, not INV-1.
A repeated pause hook on the same pause still creates no duplicate.
- **`_group_tools_by_approval` / `_agroup_tools_by_approval`:** a record
found by a run-level or member-run lookup is not paired with a tool call
it doesn't name in its `requirements`. Such a call is left without a
record, so the continue is blocked. Records that don't list their tool
calls behave as before.
- **`check_and_apply_approval_resolution` /
`acheck_and_apply_approval_resolution`:** only
`approval_type="required"` tools are gated. Audit tools never get a
required record, so they no longer block the continue.
`metadata["approval"]` is now the record of the call this continue runs,
not the oldest record in the run.
- **Agent and team continue with requirements:** a requirement no longer
replaces a tool call that already has a result. After a pause resolved
from the approvals table, the requirement can hold an out-of-date copy
of the call (confirmed, no result), and resending `run.requirements` ran
the call a second time.

### Agent and team coverage

| Scenario | Before | After |
|---|---|---|
| Agent with an `@approval` tool | Second call ran under the first
call's approval | Separate record and admin decision per call |
| Team with an `@approval` tool on the team leader | Same bug (the team
pause hook uses the same `create_approval_from_pause`) | Separate record
and admin decision per call |
| Team with an `@approval` tool on a member agent | Each pause got its
own record, but the second also named the first call and showed its
arguments. If the second record was deleted, the first call's approval
ran the second call | Second record names only the second call; a
deleted record blocks the continue |

The sync, async and streaming paths all go through the shared functions
above.

Related: #10600 handles a different route to the approval gate
(continues that send tool or approval fields from the client). This PR
covers the continue with no requirements, where the call picked up the
old record inside `approval.py`.

## Reproduction

<details>
<summary>Reproduction code (AgentOS app)</summary>

```python
"""
Approval Reuse Across Pauses (AgentOS repro)
============================================

A second @approval tool call in the same run must get its own approval record.
Before the fix, the second call had no record, and continuing the run executed
it under the approval an admin resolved for the first call.

Components served:
- invoice-agent:          agent on OpenAIResponses (needs OPENAI_API_KEY)
- invoice-team:           team with the tool on the team leader, OpenAIResponses
- invoice-agent-scripted: agent, deterministic model (no API key needed)
- invoice-team-scripted:  team-level tool, deterministic model

The real-model components run with parallel tool calls off, so every
pay_invoice call is its own pause. The scripted model pays every invoice named
in the user message (e.g. INV-1, INV-2), one tool call per model turn.

Run the server:  python <file>.py
Run the client:  python <file>.py --demo [--scripted]
"""

import argparse
import json
import os
import re
from typing import Any, AsyncIterator, Iterator, List

import httpx
from agno.agent import Agent
from agno.approval import approval
from agno.db.sqlite import SqliteDb
from agno.metrics import MessageMetrics
from agno.models.base import Model
from agno.models.openai import OpenAIResponses
from agno.models.response import ModelResponse, ModelResponseEvent
from agno.os import AgentOS
from agno.team import Team
from agno.tools import tool

BASE_URL = "http://localhost:7777"
DB_FILE = "tmp/approval_repro.db"
PAYMENTS: List[str] = []


# ---------------------------------------------------------------------------
# Scripted Model: one pay_invoice call per turn for each invoice in the request
# ---------------------------------------------------------------------------
class InvoiceScriptedModel(Model):
    """Stateless, so it is safe to share across runs and sessions.

    Reads the latest user message for invoice numbers and counts the tool
    results that followed it: with n results, it pays invoice n + 1, or answers
    once every invoice is paid.
    """

    def __init__(self):
        super().__init__(id="invoice-scripted", name="invoice-scripted", provider="test")

    def _next(self, messages: List[Any]) -> ModelResponse:
        last_user = max((i for i, m in enumerate(messages) if m.role == "user"), default=-1)
        user_text = str(messages[last_user].content) if last_user >= 0 else ""
        invoices = re.findall(r"\b[A-Z]+-\d+\b", user_text) or ["INV-1", "INV-2"]
        answered = sum(1 for m in messages[last_user + 1 :] if m.role == "tool")

        if answered < len(invoices):
            r = ModelResponse(role="assistant")
            args = {"invoice": invoices[answered], "payee": "ACME", "amount": "100.00"}
            r.tool_calls = [
                {
                    "id": f"call_{answered + 1}",
                    "type": "function",
                    "function": {"name": "pay_invoice", "arguments": json.dumps(args)},
                }
            ]
        else:
            r = ModelResponse(content=f"Paid {', '.join(invoices)}.", role="assistant")
            r.event = ModelResponseEvent.assistant_response.value
        r.response_usage = MessageMetrics(input_tokens=10, output_tokens=5, total_tokens=15)
        return r

    def invoke(self, *a, **k):
        return self._next(k["messages"])

    async def ainvoke(self, *a, **k):
        return self._next(k["messages"])

    def invoke_stream(self, *a, **k) -> Iterator[ModelResponse]:
        yield self._next(k["messages"])

    async def ainvoke_stream(self, *a, **k) -> AsyncIterator[ModelResponse]:
        yield self._next(k["messages"])

    def _parse_provider_response(self, response: Any, **k) -> ModelResponse:
        return response if isinstance(response, ModelResponse) else ModelResponse()

    def _parse_provider_response_delta(self, response: Any) -> ModelResponse:
        return response if isinstance(response, ModelResponse) else ModelResponse()


# ---------------------------------------------------------------------------
# Approval-gated tool
# ---------------------------------------------------------------------------
@approval
@tool(requires_confirmation=True)
def pay_invoice(invoice: str, payee: str, amount: str) -> str:
    """Pay a vendor invoice. This moves money and cannot be undone.

    Args:
        invoice (str): Invoice number
        payee (str): Payee account
        amount (str): Amount to pay
    """
    PAYMENTS.append(invoice)
    print(f"[ledger] PAID {invoice} to {payee} ({amount}). All payments: {PAYMENTS}")
    return f"PAID {invoice}"


# ---------------------------------------------------------------------------
# Create AgentOS
# ---------------------------------------------------------------------------
os.makedirs("tmp", exist_ok=True)
db = SqliteDb(id="approval-repro-db", db_file=DB_FILE, approvals_table="approvals")

LLM_INSTRUCTIONS = [
    "You pay vendor invoices with the pay_invoice tool.",
    "Pay EVERY invoice the user lists. Call pay_invoice for one invoice, and as soon as its result comes back, "
    "call pay_invoice for the next unpaid invoice.",
    "Do not reply to the user or ask for confirmation until every listed invoice has been paid.",
    "Use payee ACME and amount 100.00 unless the user says otherwise.",
]

agents = [
    Agent(
        id="invoice-agent",
        name="Invoice Agent",
        model=OpenAIResponses(id="gpt-5.6-luna", parallel_tool_calls=False),
        tools=[pay_invoice],
        instructions=LLM_INSTRUCTIONS,
        db=db,
    ),
    Agent(
        id="invoice-agent-scripted",
        name="Invoice Agent (scripted)",
        model=InvoiceScriptedModel(),
        tools=[pay_invoice],
        db=db,
    ),
]
teams = [
    Team(
        id="invoice-team",
        name="Invoice Team",
        model=OpenAIResponses(id="gpt-5.6-luna", parallel_tool_calls=False),
        members=[
            Agent(
                id="helper",
                name="Helper",
                role="Answers general questions",
                model=OpenAIResponses(id="gpt-5.6-luna"),
            )
        ],
        tools=[pay_invoice],
        instructions=LLM_INSTRUCTIONS + ["Call pay_invoice yourself; do not delegate payments."],
        db=db,
    ),
    Team(
        id="invoice-team-scripted",
        name="Invoice Team (scripted)",
        model=InvoiceScriptedModel(),
        members=[Agent(id="helper-scripted", name="Helper", model=InvoiceScriptedModel())],
        tools=[pay_invoice],
        db=db,
    ),
]

agent_os = AgentOS(id="approval-reuse-os", db=db, agents=agents, teams=teams)
app = agent_os.get_app()


# ---------------------------------------------------------------------------
# Demo client: the documented "database resolution" flow over HTTP
# ---------------------------------------------------------------------------
def describe_approvals(client: httpx.Client, run_id: str) -> List[str]:
    out = []
    resp = client.get("/approvals", params={"run_id": run_id, "approval_type": "required"})
    for a in resp.json().get("data", []):
        calls = [
            (
                (r.get("tool_execution") or {}).get("tool_call_id"),
                ((r.get("tool_execution") or {}).get("tool_args") or {}).get("invoice"),
            )
            for r in a.get("requirements") or []
        ]
        out.append(f"{a['id'][:8]} {a['status']} (by {a.get('resolved_by')}) for {calls}")
    return out


def describe_tools(run: dict) -> list:
    return [
        (t.get("tool_call_id"), (t.get("tool_args") or {}).get("invoice"), t.get("confirmed"))
        for t in run.get("tools") or []
        if t.get("tool_name") == "pay_invoice"
    ]


def run_demo(kind: str, component_id: str) -> None:
    print(f"\n=== {kind}/{component_id} ===")
    with httpx.Client(base_url=BASE_URL, timeout=120.0) as client:
        resp = client.post(
            f"/{kind}/{component_id}/runs",
            data={"message": "Pay invoices INV-1 and INV-2 to ACME.", "stream": "false"},
        )
        resp.raise_for_status()
        run = resp.json()
        run_id, session_id = run["run_id"], run["session_id"]
        print(f"run: {run['status']} tools={describe_tools(run)}")

        for step in range(1, 5):
            print(f"  approvals: {describe_approvals(client, run_id)}")
            if run.get("status") != "PAUSED":
                break
            pending = client.get(
                "/approvals", params={"run_id": run_id, "status": "pending", "approval_type": "required"}
            ).json()["data"]
            for a in pending:
                client.post(f"/approvals/{a['id']}/resolve", json={"status": "approved", "resolved_by": "admin"})
            print(f"  resolved {len(pending)} pending approval(s)")
            resp = client.post(
                f"/{kind}/{component_id}/runs/{run_id}/continue",
                data={"tools": "", "session_id": session_id, "stream": "false"},
            )
            if resp.status_code >= 400:
                print(f"continue {step}: HTTP {resp.status_code} {resp.text[:160]}")
                break
            run = resp.json()
            print(f"continue {step}: {run['status']} tools={describe_tools(run)}")

        records = client.get("/approvals", params={"run_id": run_id, "approval_type": "required"}).json()
        executed = [t for t in describe_tools(run) if t[2]]
        print(
            f"pay_invoice calls executed: {len(executed)}; required approval records: {records['meta']['total_count']}"
        )


# ---------------------------------------------------------------------------
# Run AgentOS
# ---------------------------------------------------------------------------
if __name__ == "__main__":
    parser = argparse.ArgumentParser(description=__doc__)
    parser.add_argument("--demo", action="store_true", help="Run the HTTP client against a server on port 7777.")
    parser.add_argument("--scripted", action="store_true", help="With --demo, drive the scripted components.")
    args = parser.parse_args()

    if args.demo:
        suffix = "-scripted" if args.scripted else ""
        run_demo("agents", f"invoice-agent{suffix}")
        run_demo("teams", f"invoice-team{suffix}")
    else:
        agent_os.serve(app=app, port=7777)
```

</details>

## Type of change

- [x] Bug fix
- [ ] New feature
- [ ] Breaking change
- [ ] Improvement
- [ ] Model update
- [ ] Other:

---

## Checklist

- [x] Code complies with style guidelines
- [x] Ran format/validation scripts (`./scripts/format.sh` and
`./scripts/validate.sh`)
- [x] Self-review completed
- [x] Documentation updated (comments, docstrings)
- [ ] Examples and guides: Relevant cookbook examples have been included
or updated (if applicable)
- [ ] Tested in clean environment
- [x] Tests added/updated (if applicable)

### Duplicate and AI-Generated PR Check

- [x] I have searched existing [open pull
requests](https://github.com/agno-agi/agno/pulls) and confirmed that no
other PR already addresses this issue
- [ ] If a similar PR exists, I have explained below why this PR is a
better approach
- [ ] Check if this PR was entirely AI-generated (by Copilot, Claude
Code, Cursor, etc.)

---

## Additional Notes

**Tests added**

- `tests/unit/run/test_approval.py`: record creation on a later pause
(new record for the new call only, with that call's requirements and
args), no duplicate record from a repeated pause hook, and the fallback
refusing a record that names a different call, including when the tool's
own record is missing. Sync and async.
- `tests/unit/run/test_approval_multi_pause.py`: two calls one after the
other in one run produce two records and two admin decisions, and the
second call doesn't run while its record is pending. Covers an agent and
a team-leader tool, sync and async, with a scripted model.
- `tests/integration/os/test_approval_multi_pause.py`: the AgentOS HTTP
flow (`GET /approvals`, `POST /approvals/{id}/resolve`, `POST
/{agents|teams}/{id}/runs/{run_id}/continue` with empty tools) for the
agent and the team. It checks that the second call gets its own pending
record, isn't run while that record is pending, and doesn't run if
rejected. Uses a scripted model, so no API key is needed.
-
`tests/integration/agent/human_in_the_loop/test_approval_multi_pause.py`
and
`tests/integration/teams/human_in_the_loop/test_team_approval_multi_pause.py`:
the same checks with a real model (`OpenAIResponses`,
`parallel_tool_calls=False`). Skipped without `OPENAI_API_KEY`.

All new tests fail on `main` and pass with this change. The existing
run, agent, team and os unit suites pass. `validate.sh` reports mypy
errors only in files this PR doesn't touch, and the same errors appear
on `main`.

**Behaviour change:** a continue with no requirements now blocks if a
required tool call is paired with a record that names other tool calls.
Before, that call ran under the other record.

**Not in this PR** (same behaviour on `main`, to be fixed separately):
one approval can still pay a call twice when a run is forked and the
original is continued too, or when two continues for the same run arrive
at once. A workflow continue with no payload doesn't read the approval
record (see #9494).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Harsh <[email protected]>
Co-authored-by: Claude Opus 5.5 (1M context) <[email protected]>

2026-10-06 11:30:49 +05:30

.github

feat: make MCPConfig default and lifecycle tools opt-in (#10285 )

2026-10-01 13:35:13 +05:30

cookbook

[cookbook] Fix DynamoDb import path in the DynamoDB cookbook README (#10788 )

2026-10-05 12:54:13 +05:30

libs

fix: create a separate approval record for each approval-required tool call in a run (#10812 )

2026-10-06 11:30:49 +05:30

scripts

feat: v3.0 (#8210 )

2026-08-24 18:19:42 +05:30

.cursorrules

chore: update model guidance to gpt-5.5 (#8383 )

2026-06-13 18:53:25 +01:00

.editorconfig

fix .editorconfig

2024-02-12 21:05:49 +00:00

.gitignore

feat: v3.0 (#8210 )

2026-08-24 18:19:42 +05:30

AGENTS.md

chore: make AGENTS.md a symlink to CLAUDE.md (#9813 )

2026-09-04 14:56:39 +05:30

CLAUDE.md

chore: make AGENTS.md a symlink to CLAUDE.md (#9813 )

2026-09-04 14:56:39 +05:30

CODE_OF_CONDUCT.md

feat: V2.0.0 (#4226 )

2025-09-09 15:39:09 +02:00

CODEOWNERS

update codeowners (#1945 )

2025-01-30 20:50:59 +05:30

CONTRIBUTING.md

fix: CONTRIBUTING paths, agno_infra license note, and docstring accuracy (#9722 )

2026-09-16 18:42:47 +05:30

LICENSE

chore: update LICENSE copyright with correct attribution (#6255 )

2026-01-31 10:25:39 +00:00

README.md

chore: update

2026-07-14 22:55:34 +01:00

README.md

Build, run, and manage agent platforms.

Introduction

Agno is a framework and runtime for agent platforms. Build agents, run them as a service, manage your platform using a web UI.

Build your agent platform using the Agno SDK.
Run your agent platform using the AgentOS runtime.
Manage everything using the AgentOS UI.

Agno allows you to own your agent stack. Maintain control of your data, memory, and security posture (JWT-based RBAC), and turn your agent platform into a learning loop with simulations and usage data.

Get started

Hand this prompt to your coding agent (Claude Code, Cursor, Codex):

Help me set up my agent platform.

Clone https://github.com/agno-agi/agentos-railway into a folder called
agent-platform, cd in, read the README, and follow the get started guide.

Your coding agent will set up your agent platform and run it locally using Docker, giving you a REST API for serving your agents, a Postgres database for storing your data and traces, an MCP server, and a control plane.

Deploying somewhere else? Use the same prompt but point it to a different repo. The starter templates are identical except for the deploy scripts: swap agentos-railway for agentos-docker, agentos-aws, agentos-gcp, agentos-azure, agentos-fly, agentos-render, agentos-modal, or agentos-helm.

Prefer to code by hand?

Features

Production API. 50+ endpoints with SSE and websockets to build a product on top.
Storage. Store sessions, memory, knowledge, and traces in your own database.
100+ integrations. Connect to GitHub, Slack, Postgres, and more using pre-built toolkits.
Context Providers. Access live data from Slack, Drive, wikis, MCP, and custom sources.
Human approval. Pause runs for user confirmation. Block tools that require admin approval.
Observability. Monitor with OpenTelemetry tracing, run history, and audit logs.
Security. Get JWT-based RBAC and multi-user, multi-tenant isolation out of the box.
Interfaces. Expose your agents via Slack, Telegram, WhatsApp, Discord, AG-UI, A2A.
Scheduling. Cron-based scheduling and background jobs with no external infrastructure.
Deploy anywhere. Run on any cloud platform that runs containers. Docker, Railway, AWS, GCP.

Use Agno with your coding agent

Two options:

Recommended: Add Agno docs as an MCP server. Add docs.agno.com/mcp to your favourite coding agent.
Add Agno docs as an indexed source. In Cursor: Settings → Indexing & Docs → Add https://docs.agno.com/llms-full.txt. Also works in VSCode, Windsurf, and similar tools.

Read the full guide here.

Community

(https://x.com/AgnoAgi): follow for releases and demos
Newsletter: monthly updates on what's shipping

Contributing

See the contributing guide.

License

Agno is distributed under the Apache-2.0 license.

Telemetry

Agno sends a telemetry event per agent run so we know which model providers to prioritize. Prompts, messages, and outputs are never sent. Disable by setting AGNO_TELEMETRY=false.

↑ Back to top