mirror of
https://github.com/agno-agi/agno.git
synced 2026-10-12 02:10:54 +00:00
83dd1329bc
## Summary
Adds **conversation compaction** — keeping a long session inside the
context window by folding old turns into a summary, while the originals
stay recoverable.
### The problem
A long session has no good options in Agno today:
| Existing mechanism | Scope | Why it doesn't solve this |
|---|---|---|
| `num_history_runs` / `num_history_messages` | History window | Drops
old turns entirely — lossy, no summary |
| `CompressionManager` | Individual **tool results**, inside the model
loop | Per-run only. Never touches conversation turns |
| `ResultStore` (offload) | Oversized **tool results** → AgentFS |
Tool-call-scoped by construction. No message equivalent |
| `SessionSummaryManager` | Session summary → system prompt |
*Additive*: you get the summary **plus** all raw history. Fires every
run, no threshold. Shrinks nothing |
### The approach: derived views over an append-only transcript
Compaction never rewrites stored messages. A **`CompactionRecord`** —
summary, boundary anchor, elision watermark — is written to
`agno_compactions`, and the model payload is **derived fresh per call**
from the canonical messages plus that record.
```
CANONICAL RECORD (one row) DERIVED (per call, discarded)
runs[].messages + first_kept_message_id → [system] [summary] [kept tail]
(never mutated) elision_watermark_id
summary · archived_messages
```
Three properties fall out: nothing is destroyed (drop the record and the
full conversation returns), concurrent readers are safe, and an anchor
that no longer resolves **fails open** to the full list rather than
cutting in the wrong place.
By default (`searchable=True`) the agent also gets a **grep over the
archived transcript**, so a summary that dropped a detail is still
recoverable mid-conversation.
---
## DX
**One flag**
```python
agent = Agent(
model=OpenAIResponses(id="gpt-5.6-luna"),
db=SqliteDb(db_file="tmp/compaction.db"),
add_history_to_context=True,
compaction=True,
)
```
**Tuned — every field a plain scalar**
```python
from agno.compaction import Compaction
compaction=Compaction(
model=OpenAIResponses(id="gpt-5-mini"), # a cheap model does the summarizing
compact_at_tokens=100_000,
uncompacted_runs=5,
)
```
**Folding on demand**
```python
result = agent.compact(session_id="s1") # and acompact()
if result.compacted:
print(result.record.tokens_before, "->", result.record.tokens_after)
else:
print(result.status.value, result.message)
```
Only the size trigger is bypassed — a caller asking to compact has
supplied the judgement `compact_at_tokens` exists to make. Every other
guard still applies, and **declining is reported, not raised**: a
summary costs a few hundred tokens whatever it replaces, so folding a
smaller span would leave the context *bigger*. `CompactionResult`
carries a status and a message written for display.
**Searchable archive**
```python
compaction=Compaction(uncompacted_runs=5, searchable=True)
```
The agent gains one tool, `search_compacted_history(pattern,
context_lines)` — a **regex grep** returning numbered lines with
context, `grep -n -C` style, scoped to this session by construction.
Observed in a live run, unprompted:
```
search_compacted_history(pattern=Pinned build hash|Rotation window|b7f2ae91c4|47 days, context_lines=2)
```
**Inspecting what happened**
```python
run = agent.run("...")
print(run.compaction)
# CompactionRecord(messages_compacted=6, tokens_before=17786, tokens_after=5537, archived=True)
```
**Streaming events**
```python
async for chunk in agent.arun(q, stream=True, stream_events=True):
if chunk.event == RunEvent.compaction_completed.value:
print(chunk.messages_compacted, chunk.tokens_before, chunk.tokens_after)
```
### Full surface
| kwarg | default | meaning |
|---|---|---|
| `model` | agent's model | model used to summarize |
| `compact_at_tokens` | `150_000` | fold when the context exceeds this;
`None` disables the automatic trigger |
| `uncompacted_runs` | `5` | recent runs kept verbatim |
| `uncompacted_tokens` | `None` | recent history kept verbatim, bounded
by size instead of turns; mutually exclusive with `uncompacted_runs` |
| `archive` | `True` | store the folded transcript alongside the record
|
| `searchable` | `True` | give the agent grep over the archive; off when
`archive=False` leaves nothing to search |
| `elide_tool_results` | `True` | render old tool results as a
placeholder |
| `min_fold_ratio` | `2.0` | skip a fold not meaningfully larger than
the tail it keeps |
| `summary_budget_tokens` | `2_000` | soft length target given to the
summarizer |
| `on_context_overflow` | `False` | also fold reactively when the
provider rejects a request as too long; bare `compaction=True` turns
this on (and leaves `compact_at_tokens` unset) |
| `instructions` | `None` | custom summarization prompt |
`bool | object` matches every existing Agno manager, so nothing new is
learned.
**The tail is measured in runs, not messages.** A run is one turn, so a
run-based tail never
cuts through the middle of one — which is what the pair-safe boundary
walk wants anyway. A message
count was also invisible to the planner-window guard below, which
reasons in runs, so it could
strand a boundary anchor with no warning.
**Size is the only automatic trigger.** Run and message counts were
considered and dropped: twenty short exchanges and twenty research turns
differ by orders of magnitude, so counting them fires on conversations
far too small to fold and stays quiet on ones that overflow.
`agent.compact()` covers the "fold at a seam I choose" case that a count
was standing in for.
---
## Design notes
**Where it plugs in.** `get_run_messages()` / `aget_run_messages()`,
right after the history block — one seam shared by all four run paths,
so none of them needed changing.
**Anchors are message ids, not indexes.** History is rebuilt from stored
runs every run, so a positional boundary means something different each
time and the cut silently crawls forward. An id resolves to the same
message forever, or not at all.
**Three tiers, increasing cost.** Elision replaces old tool results with
a placeholder (zero inference, reclaims the bulk of a tool-heavy
transcript); folding summarizes old turns (one LLM call); archiving
stores the originals (one write, buys recoverability). Independent
knobs, not one mode.
**Cut safety.** `choose_boundary` never splits a tool batch — in either
direction — and never anchors on a message that won't survive to the
next run (`temporary`, `add_to_agent_memory=False`). Returns `None`
rather than cutting unsafely.
**Storage.** `agno_compactions`, one row per fold, written once and
never updated. Records are immutable facts about single runs, so
concurrent writers can't collide and a resumed run can resolve the fold
it actually saw. Optional-table pattern following `agno_tool_results`:
base raises `NotImplementedError`, SQLite and PostgreSQL implement,
other backends degrade to summary-only with a warning.
### Bounding the kept tail by size
`uncompacted_runs` says how many turns survive a fold. It does not say
how large they are — and
compaction only folds what sits in FRONT of the tail, so a verbose turn
landing inside the tail
is one it can never bring back down. Five huge runs produce an unbounded
tail that no amount of
folding reaches.
`uncompacted_tokens` bounds the tail's size instead. On five detailed
turns of ~4,700 tokens each:
```
uncompacted_runs=2 fold 14,178 tail 9,452 ratio 1.50 -> declined
uncompacted_tokens=2000 fold 18,929 tail 4,701 ratio 4.03 -> folds
```
Both kept "the recent conversation". The run count kept two enormous
turns, leaving too little
in front of them to pay for a summary — so nothing happened and all
23,630 tokens stayed.
The cut is still pair-safe: it snaps to a turn boundary rather than
severing a tool call from
its result, so the tail can come out somewhat larger than the budget. It
is a target, not a hard
cap — and once the tail holds a single turn, no smaller budget can
shrink it further.
The two are **mutually exclusive and raise** if both are set. They
describe the same tail in
different units, and silently honouring one of two values the user
deliberately set is the kind
of surprise that costs an afternoon.
### The history window compaction reads
`num_history_runs` defaults to `3`; `uncompacted_runs` defaults to `5`.
Left alone, those two independent defaults collide and
**`compaction=True` could never fold** — the tail is larger than the
whole window, so nothing is ever in front of it.
Worse, a fold's anchor is a message id but `num_history_runs` is a
sliding count. A few runs after a fold the anchor falls outside the
window, stops resolving, and the view fails open — so the model loses
the folded turns *and* the summary that replaced them, silently.
So the planner reads its **own bounded window** rather than the replay
window. Bounded, not unlimited: capped so a 10,000-run session costs no
more than a short one. An explicit `num_history_runs` is respected
as-is, except where it sits at or below `uncompacted_runs` — a
configuration that cannot express a working compaction — where it is
raised to the minimum that keeps the anchor findable, **with a
warning**. Silently ignoring a number the user set is worse than the
misconfiguration it works around.
```
compaction=True (both defaulted) planner window widened
num_history_runs=50 50, respected exactly
num_history_runs=3, uncompacted_runs=5 6, and warns
```
`num_history_runs` continues to govern what the model replays. Only the
planner's read is widened.
---
## How a fold is decided
Two gates, and both have to pass.
**Gate 1 — the trigger: "is there enough here?"** `compact_at_tokens`,
measured against the view about to be sent — system text, tool schemas
and the history that survives the active record. Passing it only means
*look closer*.
The obvious cheaper signal is wrong here. `RunMetrics.input_tokens` is
billing telemetry: it sums *every* model call in a run, so a tool loop
that re-sends a 20k context four times reports 80k, and a 25k threshold
fires on a request that never exceeded 21k. It also absorbs background
and compaction-model calls, so folding inflates the number that decides
whether to fold again. Per-call sizes are not recoverable either —
`ModelMetrics` entries merge by provider+id before compaction sees them.
So the trigger estimates locally instead, via
`agno.utils.tokens.count_tokens` rather than `Model.count_tokens` (a
network call on some providers). ~150ms on a 400-message transcript,
once per run, against a summarizer call costing seconds. It is also the
same unit the ratio guard below already uses, so both gates now measure
the same quantity.
**Gate 2 — the ratio: "is it worth it?"** This is the one that decides:
```
ratio = tokens being folded ÷ tokens in the kept tail
```
`min_fold_ratio` defaults to `2.0` — the fold has to be at least twice
the tail.
A summary is never free: it costs a few hundred tokens whatever it
replaces. Folding 20,000 tokens to save 19,500 is obviously worth it;
folding 600 to save 100 is not, and it discards the prompt-cache prefix
to do it. A ratio catches that at any scale, where a fixed size
threshold would not.
From a live session (`compact_at_tokens=4_000`, `uncompacted_runs=1`, an
agent instructed to answer at length):
```
Q1 fold 9,691 tail 8,265 ratio 1.17 skipped - barely more than is kept
Q2 FOLD 24,965 → 8,620 tokens
Q3 fold 7,009 tail 12,075 ratio 0.58 skipped - little new since the fold
Q4 fold 19,084 tail 10,962 ratio 1.74 skipped - just under the bar
manual compact() 59,045 → 14,173 tokens
```
Q3 is the clearest case: right after a fold, the summary plus the tail
*is* most of the context, so there is little in front of the tail left
to reclaim. Q4 shows the bar doing its job on a near miss — and note
that Q3's total (19,084) becomes Q4's foldable span exactly, once one
more turn pushes a new tail in front of it.
**Tuning.** Continuing the conversation is usually the real answer: the
fold grows with every
turn while the tail stays roughly fixed, so the ratio climbs on its own.
Shrinking the tail helps
only while it still holds more than one turn — the cut is pair-safe, so
no setting can cut inside
a turn, and a decline names that setting only when changing it could
actually move the boundary.
`min_fold_ratio=1.5` is the other lever.
A declined fold says why, with the numbers:
```
This fold would replace 1234 tokens against a 1234-token tail (ratio 1.00, needs 2.0),
so the context would not shrink. Continue the conversation, or lower uncompacted_runs=2
or min_fold_ratio to fold sooner.
```
**Known limit.** The ratio measures the *input*, and cannot know how
large the summary will be until it exists. On a span that is mostly raw
data — a table, a log dump — the prompt requires identifiers be
reproduced verbatim, so the summary can come back nearly the size of
what it replaced. Observed one fold saving only 20 tokens for this
reason. Catching that needs a post-hoc check; see Additional Notes.
### Mental model
```
← fold grows forever ──────┤ window is fixed at N turns ├→
(everything ever said) (but its SIZE varies wildly,
because turns vary in length)
ratio = fold ÷ window want ≥ 2.0
```
### Provider interaction worth flagging
`OpenAIResponses` chains via `previous_response_id`, and the server then
replays the **whole** pre-fold conversation — so the savings would be
imaginary and the agent would never consult the archive. The view strips
that key from assistant copies. Only that key: reasoning items survive,
because a `function_call` without its paired reasoning item is a hard
400.
**Only `OpenAIResponses` has been audited for this.** Other providers
with server-side conversation state may need the same treatment.
### Composing with `offload_tool_results`
Supported. Offload envelopes are never elided, and when one is folded
away its `result_id` is carried forward in the summary so `read_result`
still resolves the payload. Envelopes are also excluded from the
fold-ratio calculation, since they're pinned cost that folding can't
reclaim.
---
## AgentOS route
```
POST /agents/{agent_id}/sessions/{session_id}/compact
```
Returns **200 with a status** whether or not it folded, because
declining is a normal outcome rather than a failure — raising would make
"folding would not help here" indistinguishable from an error and force
every caller to catch it.
```json
{
"status": "compacted",
"message": "Compacted 18 messages (15864 -> 3981 tokens).",
"compacted": true,
"record": { "messages_compacted": 18, "tokens_before": 15864, "tokens_after": 3981 }
}
```
| status | meaning |
|---|---|
| `compacted` | the fold happened; `record` carries the counts |
| `not_worth_it` | the span is too small to pay for the summary
replacing it |
| `nothing_to_fold` | the kept tail covers the whole conversation |
| `already_compacted` | a previous fold already covers everything
foldable |
| `no_history` | the session has no stored history yet |
| `not_enabled` | compaction is not configured on this agent |
| `summary_failed` | the summarizer returned nothing |
A missing agent is a 404 — that is a real error. Consumed by the AgentOS
chat UI as a "Compact now" action (separate repo, not in this diff).
---
## Type of change
- [x] New feature
- [ ] Bug fix
- [ ] Breaking change
- [ ] Improvement
- [ ] Model update
- [ ] Other:
---
## Checklist
- [x] Code complies with style guidelines
- [x] Ran format/validation scripts (`./scripts/format.sh` and
`./scripts/validate.sh`)
- [x] Self-review completed
- [x] Documentation updated (comments, docstrings)
- [x] Examples and guides: cookbook examples included
- [x] Tested in clean environment
- [x] Tests added/updated
### Duplicate and AI-Generated PR Check
- [x] I have searched existing open pull requests
- [x] If a similar PR exists, I have explained below why this PR is a
better approach
- [x] Check if this PR was entirely AI-generated (by Copilot, Claude
Code, Cursor, etc.)
---
## Relationship to #9834 and #9291
**#9834** (context compaction for agents and teams) is the more complete
implementation, and this PR **adopts its core architecture**: derived
views (`_view.py`), pair-safe durable cut selection (`_cut.py`),
local-only token estimation, and its structured summarization prompt.
Credit where due — those are better than what this PR had.
What this PR keeps that #9834 does not have:
- **A searchable archive.** #9834's summary *is* the history; a detail
it dropped is gone. Here the folded transcript is stored and greppable,
so the summary works as an index over ground truth.
- **Records in a table** rather than on the session, giving
`continue_run` correctness and multi-writer safety.
- **Manual folding** — `agent.compact()` plus an AgentOS route, with
declines reported as statuses rather than raised.
What #9834 still has that this does not: background folds, an in-run
loop seam, and Teams.
**#9291** is superseded by both. It rewrites stored history in place,
has no handling for provider-side conversation state (so its savings are
partly illusory on `OpenAIResponses`), and its own PR body notes the DX
is under review across three companion PRs.
---
## Testing
**83 unit tests** in `libs/agno/tests/unit/compaction/` plus **4 route
tests** in `libs/agno/tests/unit/os/routers/`: trigger matrix, boundary
safety (property test over every cut point), grep behaviour, archive
round-trip and cross-session isolation, provider-state detachment, both
fold guards, `continue_run` resolution, the planner-window fix and its
warning, the manual-compaction result contract, the trigger measuring a
tool loop's context rather than its billing total, and the size-bounded
tail with its mutual exclusion.
Cookbooks, run end-to-end against a live model:
- `cookbook/02_agents/03_context_management/compaction/` — basic,
thresholds, searchable archive, streaming events, async, manual, with
tools, Anthropic, Anthropic + thinking
- `cookbook/05_agent_os/21_compaction/` — an AgentOS serving both an
automatic and a manual-only agent, plus a REST script driving a fold
over HTTP
---
## Additional Notes
Deliberate follow-ups, not gaps:
- **Team parity.** `Team` mirrors every compression flag but has none of
this wired yet — the largest remaining gap.
- **Background folds and the in-run seam.** The summarizer call
currently blocks the turn, and compaction only fires at run start, so a
single long tool-calling run can still overflow. These are coupled: a
fold started in the background needs a loop-top to be adopted at.
- **`compress_tool_results` deprecation.** The existing surface still
runs in parallel. Nothing here changes or breaks it — it also still
covers in-run tool results that compaction cannot reach.
- **A post-hoc size check.** The ratio bounds the input but cannot
guarantee the outcome; verifying a fold actually shrank the context
needs a check after the summary exists.
---------
Co-authored-by: Anurag <[email protected]>
Co-authored-by: Harsh <[email protected]>
Co-authored-by: Harsh Sinha <[email protected]>
Co-authored-by: Claude Opus 5.5 (1M context) <[email protected]>
Co-authored-by: Yash Pratap Solanky <[email protected]>