Significant-Gravitas/AutoGPT

0 Fork 0

mirror of https://github.com/Significant-Gravitas/AutoGPT.git synced 2026-11-09 17:53:56 -10:00

Files

T

b7b75494f4 feat(backend/api): External API v2 (#12206 )

Adds v2 of the platform's public API at `/api/v2`: REST endpoints for a
user's graphs, library agents, folders, presets and webhook triggers,
runs and their human-in-the-loop reviews, schedules, workspace files,
credits, credentials and marketplace submissions, plus an MCP server
exposing AutoPilot's tools, all callable with an API key or an OAuth
token and gated by per-resource scopes.

- Resolves #11678
- Partly based on #12284

### Why

- v1 can create and execute a graph, execute a block, read the
marketplace and manage integration credentials, and nothing else.
- An app signing users in through OAuth (#11617) cannot list their
agents, run a saved preset, set up a webhook trigger, follow, stop or
share a run, answer a review, schedule a run, or read their files and
spend through v1, so it cannot use the platform as its backend.

### What

New surface, in `backend.api.external.v2` (too much to review whole;
read `mcp_server.py`, `mcp_calls.py`, `mcp_tenancy.py` and
`library/presets.py`, and scan `models.py`):
- Resources: `/graphs`, `/library/agents`, `/library/folders`,
`/library/presets`, `/runs`, `/runs/reviews`, `/schedules`, `/files`,
`/credits`, `/integrations/credentials`, `/marketplace`, `/blocks`,
`/search`, `/me`
- MCP server at `/api/v2/mcp`, with its RFC 9728 metadata at
`/.well-known/oauth-protected-resource/api/v2/mcp`: AutoPilot tools that
opt in through `allow_external_use`; `tools/list` shows only the tools
the caller's scopes allow; a refused or failed call returns an `isError`
result
- 13 new `APIKeyPermission` scopes, one per resource and verb
(`schema.prisma`, migration `20261008140000_add_v2_api_key_permissions`;
`READ_STORE` was already on `dev`), each described on the OAuth consent
screen and in the API key dialog
- v1 stays at `/external-api/v1`, deprecated: it stops working on
2027-02-01, after its last day of service on 2027-01-31
- v2 guides under `docs/platform/integrating/`, and the v2 OpenAPI spec
published to GitBook from `master` (`docs-gitbook-openapi.yml`)

Who can call what:
- Every route resolves its credential to one organization and team; a
row in another organization is a 404, over REST and MCP alike
(`mcp_tenancy.py` checks every id argument of every exposed tool before
it runs).
- A banned account's API key or OAuth token is refused on every path:
`X-API-Key`, an API key sent as a Bearer token (how MCP clients send
one), an OAuth token, and the MCP server's own token check.
- A graph someone else owns, a colleague's or a marketplace agent's, is
read as its shared copy, as on the internal routes: no owner ids,
credential references or secret inputs.
- A colleague's run or graph in the caller's organization is readable
but not writable: `DELETE /runs/{id}`, `POST /runs/{id}/stop`, `PUT
/graphs/{id}` and `PUT /graphs/{id}/versions/active` answer 403 before
acting.
- An account without an active plan gets a 402 (`payment_required`) from
every route that starts work, stores files or calls out, and from every
MCP tool call. Reads, billing, deletes, stopping a run, sharing and
organising stay open, as on the internal API since #15676.
- MCP tool calls run outside the action approval gate: an MCP session is
an automation's, so nobody is asked to approve a card, and the
credential's scopes are the authorization, as in REST. A run an MCP call
starts is tagged an API run, as v1's are, so its reviews come back
through `/runs/reviews` rather than a chat that was never saved.

Runs, presets and triggers:
- `POST /library/agents/{id}/runs` and `POST
/library/presets/{id}/execute` take an `Idempotency-Key`: a repeat
returns the run the first request started, a repeat while that one is in
flight is a 409, and the key reused on another agent, preset or body is
a 422.
- Both answer 422 for an input the graph does not have, or a required
one omitted or `null`, with one `details.errors` entry per input (for a
preset, unless it is a trigger, whose inputs are its block's settings);
`POST /schedules` does the same, and schedules the active version when
the request names none.
- Presets: list, get, create, update, delete and execute. A preset must
name a graph in the key's organization. One created for a graph that a
single hired expert installed belongs to that expert, as on the internal
routes, and so do triggers and schedules. Executing one runs for the
expert it belongs to (409 while that expert is paused) and lands in the
preset's organization.
- Webhook triggers: `POST /library/presets/setup-trigger` registers the
webhook and returns `webhook_url`, where a hand-configured service sends
its events. New trigger `inputs` or `credentials_inputs` re-register the
webhook, and whichever a PATCH leaves out keeps its saved value.
Deleting a trigger removes its webhook once nothing else uses it.
- A trigger's secret settings (a generic webhook's `secret_token`, All
Quiet's `signing_secret`) are returned only when the trigger is created.
Later reads leave them out of `inputs` and list them in `hidden_inputs`,
and an update that leaves them out keeps them.
- An active trigger starts runs, so arming one needs `RUN_AGENT` as well
as `WRITE_LIBRARY`: `setup-trigger`; a PATCH that sets `is_active` to
true, or gives a trigger that stays active new inputs or credentials
(which re-register its webhook); and creating an active preset of an
agent with a webhook trigger. Over MCP, `update_preset` needs it for
`is_active: true`, and for new `inputs` unless they come with
`is_active: false`, since the tool can't see the graph before it runs.
- Over MCP: `list_presets`, `update_preset`, `delete_preset`,
`list_agent_triggers` and `setup_agent_webhook_trigger`, scoped to the
key's organization as above.

Everything else:
- `POST /schedules` creates a schedule for the `graph_id` in its body.
- `POST /files/upload` writes to the workspace `GET /files` lists, and
returns the file's id and a `workspace://` URI. `GET /files` lists the
user's files, not Otto's working files or skill packages, as the Files
page does.
- Every response carries `X-RateLimit-Limit`, `-Remaining` and `-Reset`,
and a 429 adds `Retry-After` (none when Redis is unreachable). An API
key is counted per client and key head before it is hashed, and a
request is verified once; a limiter that fails open logs only its own
name, never the bucket.
- A v2 request body is capped at 10 MB, MCP included; an upload gets its
file cap plus 1 MB of framing, and a marketplace media upload past 10 MB
gets a 413 as soon as it passes the cap.
- `POST /runs/{id}/reviews` takes `auto_approve_future`, and answers 404
for an unknown review and 409 for a run that is not awaiting review.
- One spelling per concept: `inputs`, `credentials_inputs`, amounts in
`_cents`, durations in `_seconds`, times as ISO datetimes; one
`ErrorResponse` body on every error. MCP results and 500s carry no
internal exception text.
- `POST /graphs` and `PUT /graphs/{id}` validate a version's credentials
only when it is saved active. Activating a version moves the graph's
webhook presets onto it only if the credential also has `RUN_AGENT`.
- The served `openapi.json` lists only the declared `servers`
(`root_path_in_servers=False`), not the mount path FastAPI would put
first.

Changes to existing code:
> [!NOTE]
> For an overview of all changes to the internal API, check out
[`autogpt_platform/frontend/src/app/api/openapi.json`](https://github.com/Significant-Gravitas/AutoGPT/pull/12206/changes#diff-a1bb1783d6023599f9aaba5414bfd0b49637b61dad8da09c7bed1b2ee01ba852)

- The external middleware resolves every credential in one
`resolve_auth_info`, verified at most once per request, and refuses a
banned owner's on each form; `require_api_key` and
`require_access_token` are gone.
- The internal upload route and the internal review route call the
services v2 uses: `workspace/service.py` and
`graph_executions/review/service.py`. The review service carries
#15564's step-card handling and #15062's chat-queue wake.
- `workspace/service.py`'s size-capped reader is public
(`read_within_size_limit`), and refuses before reading when the
multipart parser already counted the size.
- The copilot's `run_agent` checks inputs with `unknown_inputs` and
`missing_inputs` (`data/graph.py`), which v2 shares; a required input
sent as `null` now counts as missing there too.
- `db.update_preset` stores a preset's inputs and credentials in the
same rows, and now rewrites them whenever both are given, either of them
possibly `{}`. Before, it refused new inputs beside empty credentials,
which the v2 PATCH and the copilot's `update_preset` tool both send for
a preset that needs none, and it silently kept the old inputs when every
input was cleared. `update_triggered_preset` prunes a webhook it
registered for an update that then fails, unless it is the one the
trigger already had.
- The copilot's preset and trigger tools scope their listings, and the
library entry a trigger setup creates, to an External API caller's
organization; in a chat they behave as before.
- `UserTransaction` is renamed `UserCreditTransactionAdminView` (admin
credit routes and the admin spending page).
- The Redis fixed-window rate limiter is extracted from
`search/rate_limit.py` into `api/utils/rate_limit.py`, which v2 shares;
#15703's public store-search limiter sits beside it. Its Redis round
trip is bounded at 0.25 s and fails open past it, because every v2
request waits on several of these checks.
- The store cache's helpers are public (`get_cached_store_agents` and
the rest), for v1 and v2.
- New setting `TRUSTED_PROXY_COUNT` (default 3), commented in
`backend/.env.default`.
- The API key dialog lists permissions in three groups, "API v2", "API
v1 and v2" and "API v1 only (stops working 2027-02-01)", each with the
consent screen's description, and a key's info dialog marks its v1-only
permissions. The descriptions and each permission's API version live in
`lib/api-key-scopes.ts`, which the consent page reads too; both are
typed `Record<APIKeyPermission, ...>`, so a new permission fails tsc
until it is described and classified.
- `oauth-tool` offers every permission, v2's included, read from the
enum.
- Library data layer: `list_library_agents` takes `published` and
`favorite` filters, and matches `published` by the graph's live,
approved listing; `get_library_agent` fetches the marketplace listing
with its active version and creator profile in one query;
`add_store_agent_to_library` drops unneeded queries.
- HITL data layer: `get_reviews` with optional filters;
`get_pending_reviews_for_user` and `get_pending_reviews_for_execution`
wrap it; `graph_runs_only` leaves out AutoPilot chat reviews.
- `get_graph_all_versions` takes `include_subgraphs` and `offset`;
`get_active_graph_version` returns a graph's active version number, and
`get_graph(version=None)` keeps returning the newest.
- `delete_store_submission_or_raise` raises why a submission couldn't be
deleted; `delete_store_submission` wraps it and keeps answering true or
false.
- `GraphSettings.from_graph()` and
`update_library_agent_version_and_settings` are removed (replaced by
`GraphSettings()` and `update_agent_version_in_library`).
- `PATCH /api/store/profile` accepts partial updates; `Profile` is
replaced by `ProfileUpdateRequest`. `PATCH /library/agents/{id}` with
`graph_version` points an entry only at the caller's own version in the
key's organization, or at another user's version approved in the
marketplace.
- The consent page's Deny button does nothing until the app's info has
loaded, which happens only once the backend has accepted the
`redirect_uri`; the rest of that fix reached `dev` with #15057. Two
tests pin the app-info endpoint's `redirect_uri` check.
- `add_auth_responses_to_openapi` works with any auth scheme; exception
handler setup is extracted from `rest_api` into
`api/utils/exceptions.py:add_exception_handlers`; `export-api-schema`
exports the external APIs' OpenAPI schemas too.
- `# pragma: allowlist secret` on existing lines the secret scanner
flagged; fast `pre-commit` checks run before slow ones.

### How

- v2 is its own FastAPI app, mounted on the REST app at `/api/v2` beside
the internal routes; v1 stays under `/external-api`
(`backend.api.external.fastapi_app`). Each has its own `/docs`, and
neither serves metrics, which the REST process serves on its own port.
`rest_api_test.py` keeps v2 and the internal API apart: no internal
route may start with `/api/v<n>`, and every v2 path sent to the host app
must reach the v2 mount rather than an internal route with a path
parameter.
- Every v2 route depends on `require_permission(<scope>)`, which
resolves the API key or OAuth token to a `TenantContext` (user,
organization, team). A route that needs a plan depends on
`require_paid_permission(<scope>)` instead, which adds the internal
API's `enforce_payment_paywall`. v2's own `paywall_routes_test.py` fails
on a route that neither checks nor sits on its open list.
- An MCP call is held to the credential before the tool runs
(`mcp_calls.py`): unknown arguments are refused, an argument that does
more than the tool's own permission needs the matching one
(`external_permissions`), ids are checked against the organization, and
a tool that bills the platform runs the paywall and usage-cap checks.
- The MCP server is served through `MCPMount`, which the platform app's
lifespan runs (a mounted app's lifespan never runs); it answers 503
before startup.
- The preset routes call the internal API's trigger helpers
(`library/triggers.py`), so webhook registration and cleanup are one
implementation. The hidden trigger settings are read from the trigger
blocks' own inputs marked `secret`, so a new trigger block is covered
without a list to maintain.
- The anonymous rate-limit bucket keys on the `X-Forwarded-For` entry
our own proxies appended, `TRUSTED_PROXY_COUNT` (default 3: Cloudflare,
then Google's load balancer) places from the right; `0` uses the socket
peer.
- Idempotency keys live in Redis, held per user, organization and team,
where the run lands. A key that can't be claimed is a retryable 503; an
in-flight key holds for 24 hours, so a request that dies after its run
started never lets a retry start a second one.
- `contract_shape_test.py` checks the published OpenAPI document, so a
later model cannot reintroduce a retired spelling or an amount without
its unit.

### Known gaps

- `pause_schedule` and `resume_schedule` are not exposed over MCP
pending a decision on exposing them under `WRITE_SCHEDULE`.
- MCP OAuth authorization-server discovery (RFC 8414) is not published:
without dynamic client registration, and with a token endpoint that
takes JSON rather than a form, it would advertise a flow a conformant
client cannot complete.
- `/search` still counts the user's library agents in other
organizations in `total_count`; it needs the organization in the shared
hybrid-search query.
- One valid key presented from many addresses still costs a hash per
request, up to 300 a minute per address.
- The internal API and v1 have no request body cap; this PR caps v2
only.
- v2's requests are counted under the REST API's metrics
(`autogpt_rest_api_*`, handlers `/api/v2/...`), not the external API's,
so the dashboard's External API panels show v1 only until a separate
infra change adds v2's handlers to them. That change also routes
`/api/v2` on `dev-api.agpt.co`, which routes only `/external-api` today.

### Verified

On `7b1b7cf192` I ran `backend/api/external/`, the library, review,
search and store suites, all of `backend/copilot/tools/`,
`architecture_test.py`, both paywall route tests and `test_block.py`:
8071 passed, 84 skipped. What did not pass is this machine's, not this
branch's: the store skill suites refuse a database that already holds
listings (62 errors at setup), the browser tests need Chromium (5),
`test_ensure_content_embedding_blocks` fails on clean `dev` here, and
`skill_file_routes_test.py` passes alone (9 of 9). pyright is clean over
the backend, and the frontend type-checks and its auth page tests pass
(45). The commits after it (two test-only fixes, the rate-limit warning,
the file listing, the shared graph copy and the MCP run's tagging) were
checked with `backend/api/external/`, `copilot/gate/`, the graph routes,
`run_agent`'s tests and the executor's. Each new guard turned its test
red when removed: the ban check on each of the four credential paths,
the hidden trigger settings on read and on update, the `RUN_AGENT`
requirement over REST and MCP, the rate-limit warning naming its bucket,
Otto's files in the listing, a non-owner's graph read whole, and an MCP
session made interactive, which parks an approval. None of this ran
against a live stack.

On `20e9247c22`, which moves v2 to `/api/v2`, groups the dialog's
permissions and moves v1's end date: `backend/api/external/`, both
paywall route tests, `rest_api_test.py`, `public_metrics_test.py` and
`architecture_test.py` (472 passed), `test_block.py` (3714 passed, 84
skipped), and the API key, consent and profile frontend suites (135
passed), with tsc clean. The two route guards each went red on a
mutation (an internal `/api/v3` route; an internal
`/api/{x}/library/agents`, which answered `GET /api/v2/library/agents`),
as did the dialog tests and the tsc check on the version map. A REST
server started from this tree served `/api/v2/openapi.json` and
`/api/v2/docs`, answered `/api/v2/me` and `/api/v2/mcp/` with 401s
pointing at the new discovery document, served that document, still
served v1 at `/external-api/v1`, and answered 404 at `/external-api/v2`.

On `f4db212b6b`, which fixes the preset update:
`backend/api/features/library`, `backend/api/external/`, the copilot's
preset tool tests, the webhook ingress tests and `architecture_test.py`
(680 passed), and `test_block.py` (3714 passed, 84 skipped). The two
PATCH tests drive the real `db.update_preset` and went red with the old
guard; the prune went red when removed and when it pruned the trigger's
own webhook; and the tests added for a listing's and an update's
withheld trigger secrets and a trigger's run each went red when that
code was reverted.

On `61dd619efb`, which merges `dev` at `ea588f7375` (#15057's redirect
checks and client-id encoding, #16175's invitation-only membership): the
consent page conflicted, and keeps the shared descriptions module, the
Deny guard, `dev`'s `encodeURIComponent` on the app-info request and
`dev`'s test file. The internal spec equals a fresh export.
`backend/api/external/`, the library, OAuth, orgs, paywall, `rest_api`
and public-metrics tests, the copilot's preset tool tests and
`architecture_test.py` passed (860), as did `test_block.py` (3714
passed, 84 skipped) and the consent, setup-wizard, proxy and API key
frontend suites (324), with tsc clean.

On `76ea8cf9e9` through `0a63dac8cf`, which close the remaining review
findings, each fix has a test that goes red without it, through the real
route, or the real `db.update_preset` where the bug sat between layers:
credentials-only and clear-all PATCHes; a limiter whose Redis never
answers; an idempotency key reused under another team; expert
attribution on presets, triggers and schedules; a key holding only
`WRITE_LIBRARY` refused when it arms an active trigger by PATCH or
creates an active trigger preset, and the MCP equivalent; a failed
attach pruning its webhook; an MCP tool failure returning no internal
text; and a team-pinned key stopping once its membership ends. On
`0a63dac8cf`, `backend/api/external/`, the library suites, the copilot's
preset and trigger tool tests, both paywall route tests and
`architecture_test.py` passed (703), as did `test_block.py` (3714
passed, 84 skipped).

### Agents and large language models used

- Claude Code with Claude Opus 5.5 (implementation, merges, tests, docs)
- T3 Code delegated reviewers for #15376's security review rounds:
Claude Mythos 5.1 and Claude Fable 5.1 (Claude), GPT Daybreak Blue
(Codex)

### Checklist 📋

#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
- [x] The `backend/api/external/` package and the library, graphs,
review, workspace and copilot tool suites, on this head
- [x] Every v2 route's scope and tenant check (`permissions_test.py`,
`tenancy_test.py`), and every MCP tool's id arguments
(`mcp_tenancy_test.py`)
- [x] The published OpenAPI contract (`contract_test.py`,
`contract_shape_test.py`)

#### For configuration changes:

- [x] `.env.default` is updated or already compatible with my changes
- [x] `docker-compose.yml` is updated or already compatible with my
changes
- [x] I have included a list of my configuration changes in the PR
description (under **Changes**)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
Co-authored-by: Nicholas Tindle <[email protected]>
Co-authored-by: Nicholas Tindle <[email protected]>

2026-10-26 00:14:28 +00:00

3.5 KiB

Raw Permalink Blame History

API docs on GitBook

How the AutoGPT Platform API docs at agpt.co/docs/platform/api-and-integrations/ are built, and what has to be set up in GitBook for them.

What comes from where

Guides are Markdown in docs/platform/integrating/, listed in docs/platform/SUMMARY.md under "API & Integrations". They reach GitBook with every other page through docs-gitbook-sync.yml (master to the gitbook branch).
The API reference is generated by GitBook from an OpenAPI spec stored in the GitBook organization as autogpt-api-v2. The builtin:openapi entry under "API reference" in SUMMARY.md names that slug and generates one page per operation (grouping: by-operation), plus a models page and a spec download link.
.github/workflows/docs-gitbook-openapi.yml republishes the spec from master whenever backend/api/external/ changes, so the reference follows the same commit as the guides.
The spec's own metadata shapes the reference: tag titles (x-page-title) name the sections, servers feed GitBook's "Test it" panel and every operation page, and x-enable-proxy sends "Test it" requests through GitBook's proxy because the API doesn't allow the docs site's origin. These live in backend/api/external/v2/app.py.

One-time setup

In GitBook, create an API token for an organization admin, and store it as the repository secret GITBOOK_TOKEN.
Store the organization ID (the segment after /o/ in GitBook app URLs) as the repository variable GITBOOK_ORGANIZATION_ID.
Publish the spec once by hand before the release that first ships the builtin:openapi entry to master, so the autogpt-api-v2 spec exists when GitBook imports the new SUMMARY.md. The workflow can't do this first publish: GitHub offers workflow_dispatch only for a workflow already on the default branch, and this one reaches master in that same release. From autogpt_platform/backend on the release branch, run the workflow's own two steps:
```
poetry run export-api-schema --api v2 --pretty --output /tmp/autogpt-api-v2.json
GITBOOK_TOKEN=... npx --yes @gitbook/[email protected] openapi publish \
  --spec autogpt-api-v2 --organization "$GITBOOK_ORGANIZATION_ID" \
  /tmp/autogpt-api-v2.json
```
After that release, every push to master that changes backend/api/external/ republishes it, and the workflow can be run by hand from master.
Make the spec public in GitBook (OpenAPI → autogpt-api-v2 → visibility). "Test it" only appears for a spec with a public URL.

Site settings worth keeping on

These are GitBook site settings, not files in this repository:

Page actions, including "Copy page", "View as Markdown", "Open in ChatGPT / Claude" and the MCP actions. Turning page actions off also disables the docs MCP server at agpt.co/docs/~gitbook/mcp, which the "Build with AI coding agents" page tells people to use.
AI assistant, if the plan allows it. The .md version of every page links its ?ask= endpoint for agents.

llms.txt, llms-full.txt and the .md page versions are automatic. Each guide's description: frontmatter becomes its line in llms.txt, so keep one on every page.

Keeping the guides true

Every request and response in the guides was copied from a running instance, and the Quickstart's scripts run as written. When the API changes, run the changed examples against an instance built from the branch before merging; the Quickstart scripts take AUTOGPT_API_URL and AUTOGPT_API_KEY and exit non-zero unless the sample agent answers Hello, Ada!.

3.5 KiB Raw Permalink Blame History