Files
5baeb410d6 feat: v2.7 — service accounts, MCP interface v2, agnoctl, and eval suites (#8747)
## Summary

The v2.7 release branch: **one backend for every frontend**. A running
AgentOS becomes operable by any MCP client with one command:

```bash
uvx agno connect        # at 2.7 stable; during the alpha: uvx agnoctl connect
```

The branch is released continuously as alphas — **agno
`2.7.0a1`–`2.7.0a5`** and **agnoctl `0.1.0a1`–`0.1.0a5`** (versions in
lockstep) via OIDC trusted publishing.

This PR consolidates and supersedes #8742 (service accounts + MCP
interface v2) and #8743 (agnoctl), and cherry-picks #8745 (@kausmeows).
Larger increments landed on the branch through individually reviewed
PRs, each with its full write-up: #8752 and #8760 (security hardening),
#8755 (MCP trace attribution), #8759 (pyproject extras), #8761 (eval
suite runner), #8740 (community cookbook), #8770 (agnoctl templates +
chat-app connect, RBAC-shaped token payloads, ultra-review fixes), #8777
(service-account `user_id` ownership column).

## What's in v2.7

### 1. Service accounts (PAT auth)

Machine identities authenticated by opaque `agno_pat_...` tokens (GitHub
PAT model): SHA-256-only storage, `sa:<name>` principals, conservative
default scopes (run + read + `config:read`), 90-day default expiry,
`POST/GET/DELETE /service-accounts`, TTL'd verification cache with
instant local revocation, uniform-401/fail-closed/throttled
verification.

- PAT scopes are first-party ACL data, enforced in **every** deployment
mode — including `os_security_key` mode, on `/mcp` tools, and on
WebSockets.
- `sa:` principals are **self-scoped**: a PAT reads and writes only the
data it created, regardless of `user_isolation`; cross-user access
requires an explicitly minted `agent_os:admin` token (#8760 D1).
On-behalf-of writes are coerced to the `sa:<name>` principal.
- Minting requires a real credential (`OS_SECURITY_KEY` or JWT) —
anonymous mint on an open instance returns 401, so no durable credential
can be created before an operator enables auth (#8752 S4).
- Scope payloads follow the RBAC governance shape (#8770): responses
return parsed scope objects (`{raw, namespace, sub_namespace,
permission, value}`), create requests take `{scope, effect}` objects
only (bare strings and `effect="deny"` are 422s). Storage keeps raw
strings; only the API payload shape changed, so the UI builds one scopes
integration ahead of Governance.
- `user_id` ownership column (#8777), stamped at mint from the
authenticated principal — `user_id` = ownership (what 3.0 user isolation
will key on), `created_by` = audit, `NULL` = workspace-level machine
account. Schema-only for now; added pre-GA because the adapters fail
closed on missing columns, so post-GA it would have cost every
deployment a migration.

### 2. MCP interface v2

The 19-tool REST-mirror surface becomes an **8-tool operator surface**:
`get_agentos_config`, `run_agent`, `run_team`, `run_workflow`,
`continue_run`, `cancel_run`, `get_sessions`, `get_session_runs`.

- Run results are **trimmed by default**; `result_mode="full"` and
`get_session_runs(run_id=...)` restore full detail (transcript, events,
metrics). Progress notifications during runs; HITL pause →
`continue_run`/`cancel_run` lifecycle, with the admin-approval
continuation gate shared with REST (`run_continuation_blocked_reason`).
- One implementation for REST and MCP via the shared `agno/os/services/`
layer; every tool call is scope-gated by mapping it onto its
REST-equivalent route, so per-resource scopes and admin bypass behave
identically on both surfaces.
- **Request isolation matching REST**: components resolve through the
shared `resolve_*` helpers with a `create_fresh` deep copy (no
shared-singleton state across concurrent runs), DB-registry components
resolve and enumerate, the config roster is filtered to the caller's
accessible resources, and a fresh session is minted per call when
`session_id` is omitted (no cross-request history leaks).
- MCP-initiated runs are correctly attributed in traces: the run span
starts its own root trace instead of nesting under FastMCP's
identity-less protocol span (#8755).
- Ultra-review fixes (#8770): trimmed session history keeps
`workflow_id` on workflow runs, and `continue_run`/`cancel_run` fail
closed with a clean message (instead of an `AttributeError`) for scoped
callers on remote components — a real remote ownership contract is
SDK-543.

### 3. agnoctl — the Agno CLI

New distribution at `libs/agnoctl` (PyPI `agnoctl`, command `agno` +
alias `agnoctl`), zero dependency on the `agno` framework. Core `agno`
declares `[project.scripts] agno = "agnoctl.main:app"` plus a dependency
on agnoctl, which is what makes `uvx agno connect` work at stable.

- **`agno connect`**: discover the AgentOS (ports 7777/7778/7779/8000,
or `--url`/`AGENTOS_URL`) → mint per-client PATs → write client configs
→ verify with a real MCP handshake. Clients: Claude Code, Codex, Cursor,
**Claude Desktop** (stdio→remote bridge via `mcp-remote`; token rides an
env var, not argv), and the chat apps **Claude** (`claude-ai`) and
**ChatGPT** — instructions-only with a distinct non-failing `manual`
status, since connectors are added in the app's own UI with OAuth, not
bearer tokens (#8770).
- **Deployed-OS awareness** (#8770): when the discovered AgentOS is on a
public HTTPS URL without a token gate, connect auto-appends the
Claude/ChatGPT connector setup steps after the coding-agent results (the
templates' deploy scripts set `AGENTOS_URL`, so this is the default
experience on a deploy box); reachability is judged on the parsed
hostname (loopback/RFC1918/link-local are out), an explicit `--clients`
still scopes the run exactly, and the connect header + `--json` report
where the target URL came from (`url_source`: flag/env/default).
- **Credential guardrails**: refuses to send a bearer credential over
plaintext HTTP to a non-loopback host (`--allow-http` to opt in), atomic
0600-from-birth config writes, the PAT never appears on any process
argv, rotated tokens print a restart reminder for the affected client.
- `agno tokens create/list/revoke` (revoke confirms interactively),
`agno create` (scaffolds from the `agentos-<provider>` template repos —
all five: docker/aws/fly/gcp/railway, #8770), `agno up/down/restart`
(compose), `agno status`. `--json` everywhere, deterministic exit codes,
truthful outcomes, branded home screen.

### 4. One auth layer

AgentOS installs a single `AuthMiddleware` on the parent app whenever
any credential source is configured (JWT, `OS_SECURITY_KEY`, or a
service-account verifier). Starlette runs parent-app middleware before
dispatching to mounts, so that one instance covers REST, the mounted
`/mcp` app, WebSockets, and every interface that does not authenticate
its own requests — the mounted MCP app carries no auth code of its own.

- `JWTMiddleware` is now an alias of `AuthMiddleware` (JWT is one
credential source among three). It refuses to construct with no
credential source, and `authorization=True` with no JWT verification
source **fails closed at construction** instead of silently serving an
open instance.
- Interfaces that verify their own webhook signatures (Slack, Telegram,
WhatsApp) declare `authenticates_own_requests = True` and are excluded;
everything else — including A2A and AGUI — sits behind the auth layer
(#8752 S1).
- The WebSocket path has full parity: PATs prefix-dispatch before JWT
handling, verified identity + scopes attach to the connection context,
and the RBAC/attribution gates fire exactly as on REST.

### 5. Security hardening

The auth surface went through six review rounds (in-branch adversarial
review, an independent external review by Codex + Opus, #8752, a PAT/JWT
hardening pass, #8760, and a multi-agent ultra review of this PR whose
two verified findings are fixed in #8770). Highlights beyond the items
above:

- **A2A/AGUI authorization + identity** (#8760 B1/B2): interfaces now
declare their own RBAC scope mappings (`Interface.get_scope_mappings()`,
merged against the actual mount prefix), closing a read→execute
escalation where a `config:read` token could run agents via
`message:send`. All A2A/AGUI run sites resolve identity server-side
through a shared `resolve_run_user_id()` — client-supplied
`X-User-ID`/`metadata.userId`/`forwardedProps.user_id` is honored only
for anonymous callers and never for reserved principals, closing
run/session/memory attribution spoofing and a session IDOR. A regression
test asserts every mounted `/a2a` route resolves to a non-empty
required-scope set.
- **Trace IDOR fixed** (#8752 S2, pre-dates this branch): `GET
/traces/{trace_id}` now enforces `user_isolation` ownership, including
the `run_id`/`span_id` paths and the RemoteDb branch.
- **JWTs cannot impersonate reserved principals**: tokens whose subject
claims `sa:*` or `__scheduler__` are rejected on HTTP and WebSocket.
- `validate=False` + `authorization=True` no longer fails open on
undecodable tokens (#8752 S3, pre-dates this branch); resource-type
classification is anchored to the first path segment; the internal
scheduler token keeps its strict scope check.

### 6. Eval suite runner (#8761)

`agno.eval` gains a suite layer: a frozen `Case` (one input to one agent
+ optional judge/reliability checks), a sequential single-event-loop
runner (`run_cases`/`arun_cases`) that does zero console I/O and returns
a `SuiteResult` whose `to_dict()` is a stable CI/JSON contract, and an
argparse `cli()`/`acli()` built purely on the runner's hooks. Replaces
the ~430-line runner previously copy-pasted per template. Also:
`show_spinner` flag on the four existing eval classes, and sync-DB eval
logging moved off the event loop.

### 7. Packaging, toolchain, and release engineering

- **Core deps 14 → 10**: dropped `gitpython`, `python-dotenv`, `typer`
(zero imports in `libs/agno`); `python-multipart` moved into the extras
that serve FastAPI routes; `fastapi[standard]` unbundled into `fastapi`
+ `python-multipart` + `uvicorn` + `websockets` — `agno[os]` no longer
drags `sentry-sdk`/`fastapi-cloud-cli`. `fastmcp` folded into
`agno[mcp]` so AgentOS can serve `/mcp` without a separate install (a3).
- **Extras tidy** (#8759): alphabetized groups, aggregate gaps closed,
underscore→hyphen key normalization (free PEP 503 aliases), and a real
fix — `agno[embedders]` was unresolvable (dangling `agno[vllm]`
reference). Verified by diffing the resolved closure of all 129 extras:
zero removals.
- **Python floor raised to 3.9** for both packages (agno's old `>=3.7`
was already unsatisfiable via its own pydantic pin — honest metadata,
not a support cut).
- **release.yml**: OIDC trusted publishing (no long-lived API tokens),
per-package "already on PyPI?" gates, explicit job permissions,
concurrency guard, agno `needs` agnoctl.
- **`format.sh` / `validate.sh` are real gates**: honest non-zero exit
codes, per-target sections; mypy on `libs/agno` is clean for the first
time (0 errors in 903 files) with the toolchain pinned at latest (mypy
2.1.0, ruff 0.15.20, pytest 9.1.1). `dev_setup.sh`/`demo_setup.sh`
rebuilt on one pattern (fresh 3.12 venv, workspace agnoctl editable next
to `agno[dev]`/`agno[demo]`); `agno[dev]` now collects and passes the
entire first-party unit tree (4,767 tests) in a fresh venv; `agno_infra`
is out of the core dev flow.

### 8. Cookbooks

Service accounts (`cookbook/05_agent_os/middleware/`), MCP demo updates
(`cookbook/05_agent_os/mcp_demo/`), eval suites
(`cookbook/09_evals/suite/`), and a community BGPT MCP example (#8740,
`cookbook/91_tools/mcp/bgpt.py`).

## Breaking changes (vs 2.6)

Intentional and documented in the design docs:

1. **MCP surface 19 → 8 tools.** Memory/session-write tools removed;
`MCPServerConfig(include_tags={"memory"})` fails validation loudly. Run
results trimmed by default (`result_mode="full"` restores the old
payload).
2. **`authorization=True` now protects `/a2a/*` and `/agui`** —
authentication, per-route RBAC scopes, and server-side identity.
Deployments that ran A2A/AGUI with auth enabled and relied on those
routes being open now need credentials. Custom `BaseInterface`
subclasses that verify their own inbound webhooks must set
`authenticates_own_requests = True` (default is fail-closed).
3. **A2A `tasks:get` and `tasks:cancel` require `contextId`** (400
instead of a deep 500; the a2a-sdk returns it in every `message:send`
response). Client-supplied user ids on A2A/AGUI are honored only for
anonymous callers.
4. **Packaging**: code relying on agno transitively installing
`gitpython`/`typer`/`python-dotenv`, or on `fastapi[standard]`'s bundled
extras, must declare those deps itself. `requires-python >= 3.9`.

Alpha-tester notes (a1–a3 → a4): rotate PATs minted before `config:read`
joined the default scopes (`agno connect --rotate`); `AGNO_OS_URL` is
now `AGENTOS_URL` (renamed after 0.1.0a1). (a4 → a5): an existing
`agno_service_accounts` table trips the fail-closed schema check after
the `user_id` column — drop the table and re-mint (`agno connect
--rotate`); the service-accounts API now returns scope objects and
rejects bare-string scopes on create (422).

## Review map

| Area | Where |
|------|-------|
| Service accounts | `agno/os/routers/service_accounts/`,
`agno/os/service_accounts.py`, `service_accounts` table in `agno/db/*` |
| Auth layer, scopes, identity | `agno/os/middleware/`,
`agno/os/auth.py`, `agno/os/scopes.py` |
| MCP interface | `agno/os/mcp.py`, `agno/os/mcp_results.py`,
`agno/os/services/` |
| Interface RBAC (A2A/AGUI) | `agno/os/interfaces/base.py`,
`agno/os/interfaces/a2a/`, `agno/os/interfaces/agui/` |
| CLI | `libs/agnoctl/` |
| Eval suites | `agno/eval/`, `cookbook/09_evals/suite/` |
| Packaging & release | `libs/agno/pyproject.toml`,
`libs/agnoctl/pyproject.toml`, `.github/workflows/release.yml`,
`scripts/` |

The merged sub-PRs (#8752, #8755, #8759, #8760, #8761, #8770, #8777)
each carry a full standalone write-up, including per-finding provenance
(what pre-dates this branch vs what v2.7 introduced) and test evidence.

## Type of change

- [x] New feature
- [x] Bug fix
- [x] Breaking change

---

## Checklist

- [x] Code complies with style guidelines
- [x] Ran format/validation scripts — `./scripts/format.sh` and
`./scripts/validate.sh` are now hard gates and pass clean (mypy: 0
errors, 903 files)
- [x] Self-review completed — six review rounds over the auth surface
(in-branch adversarial, external Codex + Opus, #8752, PAT/JWT pass,
#8760, multi-agent ultra review); the ultra review's two verified
findings are fixed in #8770
- [x] Documentation updated (comments, docstrings,
`libs/agnoctl/README.md`, design docs in the private specs repo)
- [x] Examples and guides: service-accounts, MCP demo, and eval-suite
cookbooks
- [x] Tested in clean environment — fresh-venv dev setup passes the full
first-party unit tree; `agno[os]` closure and multipart upload verified
in a clean venv; auth matrix validated over real uvicorn; MCP fixes
reproduced against a live FastMCP server
- [x] Tests added/updated — os suite (unit + integration) + eval unit:
**2,298 passed** (the two former known failures — a stale
JWKS-precedence test and a numpy-dependent serialization test — are
repaired on the branch); agnoctl: **153 passed**, now also run in CI

### Duplicate and AI-Generated PR Check

- [x] I have searched existing [open pull requests](../../pulls) — #8742
and #8743 are consolidated here and closed; #8745 is cherry-picked;
#8752/#8755/#8759/#8760/#8761/#8740/#8770/#8777 merged into this branch
- [x] If a similar PR exists, I have explained below why this PR is a
better approach
- [x] Check if this PR was entirely AI-generated (by Copilot, Claude
Code, Cursor, etc.) — built with Claude Code, human-directed

---

## Additional Notes

**Release status**

- agno `2.7.0a1`–`a5` + agnoctl `0.1.0a1`–`a5` are live on PyPI (OIDC
trusted publishing; the old agnoctl `0.1.0rc*` pre-releases were
removed, so resolvers prefer the alphas).
- During the alpha the working commands are `uvx agnoctl connect` and
`uvx "agno==2.7.0a5" connect`; bare `uvx agno connect` unlocks at 2.7
stable.

**CI status**

- **`test.yml` (the PR gate) is green.** agnoctl now has its own CI job
(ruff + mypy + its 153-test suite — the package previously published
four alphas with zero CI); the style checks and `test_setup.sh` install
the workspace agnoctl editable, so a version bump can no longer race the
PyPI publish and redden checks.
- **Root causes fixed on the branch**: `agno[tests]` had become
unresolvable via #8759's aggregate additions (`infinity_client` pins
`httpx<0.28.0` against a2a-sdk's `>=0.28.1` floor; `brave-search` 0.2.0
ships metadata uv cannot parse) — both are excluded from the aggregates
again, in the litellm/mistral style. `test_setup.sh` was rebuilt to fail
fast (a failed install previously surfaced as 194 collection errors),
and `requirements.txt` is regenerated from the trimmed core (closes the
tracked follow-up).
- **The "Main Validation" model matrix runs per-push on this branch and
is green on the code side** (all 22 code-side jobs; the remaining reds
are environmental — test-deepinfra is a provider-account issue being
fixed account-side, and the live-external-API tests flake on the
third-party services they call). Getting there took fixing every failure
class it surfaced: two real product bugs (DalleTools sent the removed
`style` parameter — every image generation 400'd; fastmcp 3.4.3's new
Host/Origin guard would have 421'd every deployed `/mcp` host), stale
tests (retired `gemini-2.0-flash` pins, multi-db MCP calls without
`db_id`, an empty-keys JWKS file, a GC-timing-dependent coroutine-leak
assert), harness rot (system-test containers now install `agno[os]`
instead of a hand-copied dep list; the health gate covers all five
services; the dev-setup job verifies agnoctl, not the
deliberately-dropped agno-infra), and structural flake handling — the
repo's `@pytest.mark.flaky` convention was a silent no-op (the plugin
was never a dependency; two conflicting mark dialects existed), now
standardized on pytest-rerunfailures with a workflow-level `--reruns 1`
blanket retry for live-model jobs, pytest-timeout bounding the Gemini
concurrency suite (a TLS wedge previously hung the job for hours), and
provider-availability failures (billing/quota/brownouts) skipping with
named reasons instead of failing.
- Known-and-documented: the google-genai shared-client TLS corruption is
`xfail(strict=False)` on eight stress tests; the adapter-level fix
(thread-local sync client, per-loop async client) is queued as its own
change.

**Remaining before stable** (tracked, deliberately not in this PR)

- Port the Windows setup scripts (`dev_setup.bat`/`.ps1`,
`demo_setup.bat`/`.ps1`) to the rebuilt flow.
- Fix the chronic model-matrix failures on `main` (Gemini
SSL/structured-output, DeepInfra tool-use, live-external-API tests) so
Main Validation can gate again.
- Drop agno-infra's `agno` console script (deferred to stable;
co-installed pip envs have last-install-wins on `bin/agno` during the
alpha — uvx is unaffected).
- `create_fresh=True` as the default resolution mode is a v3.0 change
(SDK-534).

**Known, documented limitations**: WS workflow `reconnect` under
`user_isolation=False` replays buffered events to any connection (the
documented single-tenant contract of that mode); `authorization=True`
also gates the A2A agent card, with no anonymous-discovery escape hatch
yet; per-resource scopes on a custom-prefix A2A mount degrade to the
global run scope (fail-closed).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Fable 5 <[email protected]>
Co-authored-by: kausmeows <[email protected]>
Co-authored-by: Monalisha Mishra <[email protected]>
Co-authored-by: Monalisha Mishra <[email protected]>
Co-authored-by: Harsh <[email protected]>
Co-authored-by: Yash Pratap Solanky <[email protected]>
2026-07-07 14:38:12 +05:30

87 lines
2.4 KiB
Bash
Executable File

#!/bin/bash
############################################################################
#
# Agno Validate
#
# ruff check + mypy for agno and agnoctl, ruff + pattern check for
# cookbook. Exits non-zero if anything fails — safe to gate on.
# agno-infra is not part of the dev venv; it validates in its own CI job.
#
# Usage: ./scripts/validate.sh
#
############################################################################
CURR_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
REPO_ROOT="$(dirname "${CURR_DIR}")"
# Colors
ORANGE='\033[38;5;208m'
GREEN='\033[32m'
RED='\033[31m'
DIM='\033[2m'
BOLD='\033[1m'
NC='\033[0m'
# Fall back to the dev venv tools when not already on PATH
if ! command -v ruff &> /dev/null && [[ -x "${REPO_ROOT}/.venv/bin/ruff" ]]; then
export PATH="${REPO_ROOT}/.venv/bin:${PATH}"
fi
for tool in ruff mypy; do
if ! command -v "${tool}" &> /dev/null; then
echo " ${tool} not found. Run ./scripts/dev_setup.sh first."
exit 1
fi
done
echo ""
echo -e " ${BOLD}Agno Validate${NC}"
FAILURES=()
CURRENT_TARGET=""
START=${SECONDS}
target() {
CURRENT_TARGET="$1"
echo ""
echo -e " ${ORANGE}${CURRENT_TARGET}${NC}"
}
run_step() {
local label="$1"
shift
local output
if output="$("$@" 2>&1)"; then
local summary
summary="$(printf '%s' "${output}" | tail -n 1)"
echo -e " ${GREEN}✓${NC} ${label}${summary:+ ${DIM}— ${summary}${NC}}"
else
echo -e " ${RED}✗ ${label}${NC}"
printf '%s\n' "${output}" | sed 's/^/ /'
FAILURES+=("${CURRENT_TARGET}: ${label}")
fi
}
target "agno"
run_step "ruff check" ruff check "${REPO_ROOT}/libs/agno"
run_step "mypy" mypy "${REPO_ROOT}/libs/agno" --config-file "${REPO_ROOT}/libs/agno/pyproject.toml"
target "agnoctl"
run_step "ruff check" ruff check "${REPO_ROOT}/libs/agnoctl"
run_step "mypy" mypy "${REPO_ROOT}/libs/agnoctl/agnoctl" --config-file "${REPO_ROOT}/libs/agnoctl/pyproject.toml"
target "cookbook"
run_step "ruff check" ruff check "${REPO_ROOT}/cookbook"
run_step "pattern check" python3 "${REPO_ROOT}/cookbook/scripts/check_cookbook_pattern.py" --base-dir "${REPO_ROOT}/cookbook/00_quickstart"
ELAPSED=$((SECONDS - START))
echo ""
if [[ ${#FAILURES[@]} -eq 0 ]]; then
echo -e " ${BOLD}Validated in ${ELAPSED}s.${NC}"
echo ""
else
echo -e " ${BOLD}${RED}Failed:${NC} ${FAILURES[*]}"
echo ""
exit 1
fi