mirror of
https://github.com/agno-agi/agno.git
synced 2026-10-11 22:09:58 +00:00
feat: auto compaction with a searchable archive (#9873)
## Summary
Adds **conversation compaction** — keeping a long session inside the
context window by folding old turns into a summary, while the originals
stay recoverable.
### The problem
A long session has no good options in Agno today:
| Existing mechanism | Scope | Why it doesn't solve this |
|---|---|---|
| `num_history_runs` / `num_history_messages` | History window | Drops
old turns entirely — lossy, no summary |
| `CompressionManager` | Individual **tool results**, inside the model
loop | Per-run only. Never touches conversation turns |
| `ResultStore` (offload) | Oversized **tool results** → AgentFS |
Tool-call-scoped by construction. No message equivalent |
| `SessionSummaryManager` | Session summary → system prompt |
*Additive*: you get the summary **plus** all raw history. Fires every
run, no threshold. Shrinks nothing |
### The approach: derived views over an append-only transcript
Compaction never rewrites stored messages. A **`CompactionRecord`** —
summary, boundary anchor, elision watermark — is written to
`agno_compactions`, and the model payload is **derived fresh per call**
from the canonical messages plus that record.
```
CANONICAL RECORD (one row) DERIVED (per call, discarded)
runs[].messages + first_kept_message_id → [system] [summary] [kept tail]
(never mutated) elision_watermark_id
summary · archived_messages
```
Three properties fall out: nothing is destroyed (drop the record and the
full conversation returns), concurrent readers are safe, and an anchor
that no longer resolves **fails open** to the full list rather than
cutting in the wrong place.
By default (`searchable=True`) the agent also gets a **grep over the
archived transcript**, so a summary that dropped a detail is still
recoverable mid-conversation.
---
## DX
**One flag**
```python
agent = Agent(
model=OpenAIResponses(id="gpt-5.6-luna"),
db=SqliteDb(db_file="tmp/compaction.db"),
add_history_to_context=True,
compaction=True,
)
```
**Tuned — every field a plain scalar**
```python
from agno.compaction import Compaction
compaction=Compaction(
model=OpenAIResponses(id="gpt-5-mini"), # a cheap model does the summarizing
compact_at_tokens=100_000,
uncompacted_runs=5,
)
```
**Folding on demand**
```python
result = agent.compact(session_id="s1") # and acompact()
if result.compacted:
print(result.record.tokens_before, "->", result.record.tokens_after)
else:
print(result.status.value, result.message)
```
Only the size trigger is bypassed — a caller asking to compact has
supplied the judgement `compact_at_tokens` exists to make. Every other
guard still applies, and **declining is reported, not raised**: a
summary costs a few hundred tokens whatever it replaces, so folding a
smaller span would leave the context *bigger*. `CompactionResult`
carries a status and a message written for display.
**Searchable archive**
```python
compaction=Compaction(uncompacted_runs=5, searchable=True)
```
The agent gains one tool, `search_compacted_history(pattern,
context_lines)` — a **regex grep** returning numbered lines with
context, `grep -n -C` style, scoped to this session by construction.
Observed in a live run, unprompted:
```
search_compacted_history(pattern=Pinned build hash|Rotation window|b7f2ae91c4|47 days, context_lines=2)
```
**Inspecting what happened**
```python
run = agent.run("...")
print(run.compaction)
# CompactionRecord(messages_compacted=6, tokens_before=17786, tokens_after=5537, archived=True)
```
**Streaming events**
```python
async for chunk in agent.arun(q, stream=True, stream_events=True):
if chunk.event == RunEvent.compaction_completed.value:
print(chunk.messages_compacted, chunk.tokens_before, chunk.tokens_after)
```
### Full surface
| kwarg | default | meaning |
|---|---|---|
| `model` | agent's model | model used to summarize |
| `compact_at_tokens` | `150_000` | fold when the context exceeds this;
`None` disables the automatic trigger |
| `uncompacted_runs` | `5` | recent runs kept verbatim |
| `uncompacted_tokens` | `None` | recent history kept verbatim, bounded
by size instead of turns; mutually exclusive with `uncompacted_runs` |
| `archive` | `True` | store the folded transcript alongside the record
|
| `searchable` | `True` | give the agent grep over the archive; off when
`archive=False` leaves nothing to search |
| `elide_tool_results` | `True` | render old tool results as a
placeholder |
| `min_fold_ratio` | `2.0` | skip a fold not meaningfully larger than
the tail it keeps |
| `summary_budget_tokens` | `2_000` | soft length target given to the
summarizer |
| `on_context_overflow` | `False` | also fold reactively when the
provider rejects a request as too long; bare `compaction=True` turns
this on (and leaves `compact_at_tokens` unset) |
| `instructions` | `None` | custom summarization prompt |
`bool | object` matches every existing Agno manager, so nothing new is
learned.
**The tail is measured in runs, not messages.** A run is one turn, so a
run-based tail never
cuts through the middle of one — which is what the pair-safe boundary
walk wants anyway. A message
count was also invisible to the planner-window guard below, which
reasons in runs, so it could
strand a boundary anchor with no warning.
**Size is the only automatic trigger.** Run and message counts were
considered and dropped: twenty short exchanges and twenty research turns
differ by orders of magnitude, so counting them fires on conversations
far too small to fold and stays quiet on ones that overflow.
`agent.compact()` covers the "fold at a seam I choose" case that a count
was standing in for.
---
## Design notes
**Where it plugs in.** `get_run_messages()` / `aget_run_messages()`,
right after the history block — one seam shared by all four run paths,
so none of them needed changing.
**Anchors are message ids, not indexes.** History is rebuilt from stored
runs every run, so a positional boundary means something different each
time and the cut silently crawls forward. An id resolves to the same
message forever, or not at all.
**Three tiers, increasing cost.** Elision replaces old tool results with
a placeholder (zero inference, reclaims the bulk of a tool-heavy
transcript); folding summarizes old turns (one LLM call); archiving
stores the originals (one write, buys recoverability). Independent
knobs, not one mode.
**Cut safety.** `choose_boundary` never splits a tool batch — in either
direction — and never anchors on a message that won't survive to the
next run (`temporary`, `add_to_agent_memory=False`). Returns `None`
rather than cutting unsafely.
**Storage.** `agno_compactions`, one row per fold, written once and
never updated. Records are immutable facts about single runs, so
concurrent writers can't collide and a resumed run can resolve the fold
it actually saw. Optional-table pattern following `agno_tool_results`:
base raises `NotImplementedError`, SQLite and PostgreSQL implement,
other backends degrade to summary-only with a warning.
### Bounding the kept tail by size
`uncompacted_runs` says how many turns survive a fold. It does not say
how large they are — and
compaction only folds what sits in FRONT of the tail, so a verbose turn
landing inside the tail
is one it can never bring back down. Five huge runs produce an unbounded
tail that no amount of
folding reaches.
`uncompacted_tokens` bounds the tail's size instead. On five detailed
turns of ~4,700 tokens each:
```
uncompacted_runs=2 fold 14,178 tail 9,452 ratio 1.50 -> declined
uncompacted_tokens=2000 fold 18,929 tail 4,701 ratio 4.03 -> folds
```
Both kept "the recent conversation". The run count kept two enormous
turns, leaving too little
in front of them to pay for a summary — so nothing happened and all
23,630 tokens stayed.
The cut is still pair-safe: it snaps to a turn boundary rather than
severing a tool call from
its result, so the tail can come out somewhat larger than the budget. It
is a target, not a hard
cap — and once the tail holds a single turn, no smaller budget can
shrink it further.
The two are **mutually exclusive and raise** if both are set. They
describe the same tail in
different units, and silently honouring one of two values the user
deliberately set is the kind
of surprise that costs an afternoon.
### The history window compaction reads
`num_history_runs` defaults to `3`; `uncompacted_runs` defaults to `5`.
Left alone, those two independent defaults collide and
**`compaction=True` could never fold** — the tail is larger than the
whole window, so nothing is ever in front of it.
Worse, a fold's anchor is a message id but `num_history_runs` is a
sliding count. A few runs after a fold the anchor falls outside the
window, stops resolving, and the view fails open — so the model loses
the folded turns *and* the summary that replaced them, silently.
So the planner reads its **own bounded window** rather than the replay
window. Bounded, not unlimited: capped so a 10,000-run session costs no
more than a short one. An explicit `num_history_runs` is respected
as-is, except where it sits at or below `uncompacted_runs` — a
configuration that cannot express a working compaction — where it is
raised to the minimum that keeps the anchor findable, **with a
warning**. Silently ignoring a number the user set is worse than the
misconfiguration it works around.
```
compaction=True (both defaulted) planner window widened
num_history_runs=50 50, respected exactly
num_history_runs=3, uncompacted_runs=5 6, and warns
```
`num_history_runs` continues to govern what the model replays. Only the
planner's read is widened.
---
## How a fold is decided
Two gates, and both have to pass.
**Gate 1 — the trigger: "is there enough here?"** `compact_at_tokens`,
measured against the view about to be sent — system text, tool schemas
and the history that survives the active record. Passing it only means
*look closer*.
The obvious cheaper signal is wrong here. `RunMetrics.input_tokens` is
billing telemetry: it sums *every* model call in a run, so a tool loop
that re-sends a 20k context four times reports 80k, and a 25k threshold
fires on a request that never exceeded 21k. It also absorbs background
and compaction-model calls, so folding inflates the number that decides
whether to fold again. Per-call sizes are not recoverable either —
`ModelMetrics` entries merge by provider+id before compaction sees them.
So the trigger estimates locally instead, via
`agno.utils.tokens.count_tokens` rather than `Model.count_tokens` (a
network call on some providers). ~150ms on a 400-message transcript,
once per run, against a summarizer call costing seconds. It is also the
same unit the ratio guard below already uses, so both gates now measure
the same quantity.
**Gate 2 — the ratio: "is it worth it?"** This is the one that decides:
```
ratio = tokens being folded ÷ tokens in the kept tail
```
`min_fold_ratio` defaults to `2.0` — the fold has to be at least twice
the tail.
A summary is never free: it costs a few hundred tokens whatever it
replaces. Folding 20,000 tokens to save 19,500 is obviously worth it;
folding 600 to save 100 is not, and it discards the prompt-cache prefix
to do it. A ratio catches that at any scale, where a fixed size
threshold would not.
From a live session (`compact_at_tokens=4_000`, `uncompacted_runs=1`, an
agent instructed to answer at length):
```
Q1 fold 9,691 tail 8,265 ratio 1.17 skipped - barely more than is kept
Q2 FOLD 24,965 → 8,620 tokens
Q3 fold 7,009 tail 12,075 ratio 0.58 skipped - little new since the fold
Q4 fold 19,084 tail 10,962 ratio 1.74 skipped - just under the bar
manual compact() 59,045 → 14,173 tokens
```
Q3 is the clearest case: right after a fold, the summary plus the tail
*is* most of the context, so there is little in front of the tail left
to reclaim. Q4 shows the bar doing its job on a near miss — and note
that Q3's total (19,084) becomes Q4's foldable span exactly, once one
more turn pushes a new tail in front of it.
**Tuning.** Continuing the conversation is usually the real answer: the
fold grows with every
turn while the tail stays roughly fixed, so the ratio climbs on its own.
Shrinking the tail helps
only while it still holds more than one turn — the cut is pair-safe, so
no setting can cut inside
a turn, and a decline names that setting only when changing it could
actually move the boundary.
`min_fold_ratio=1.5` is the other lever.
A declined fold says why, with the numbers:
```
This fold would replace 1234 tokens against a 1234-token tail (ratio 1.00, needs 2.0),
so the context would not shrink. Continue the conversation, or lower uncompacted_runs=2
or min_fold_ratio to fold sooner.
```
**Known limit.** The ratio measures the *input*, and cannot know how
large the summary will be until it exists. On a span that is mostly raw
data — a table, a log dump — the prompt requires identifiers be
reproduced verbatim, so the summary can come back nearly the size of
what it replaced. Observed one fold saving only 20 tokens for this
reason. Catching that needs a post-hoc check; see Additional Notes.
### Mental model
```
← fold grows forever ──────┤ window is fixed at N turns ├→
(everything ever said) (but its SIZE varies wildly,
because turns vary in length)
ratio = fold ÷ window want ≥ 2.0
```
### Provider interaction worth flagging
`OpenAIResponses` chains via `previous_response_id`, and the server then
replays the **whole** pre-fold conversation — so the savings would be
imaginary and the agent would never consult the archive. The view strips
that key from assistant copies. Only that key: reasoning items survive,
because a `function_call` without its paired reasoning item is a hard
400.
**Only `OpenAIResponses` has been audited for this.** Other providers
with server-side conversation state may need the same treatment.
### Composing with `offload_tool_results`
Supported. Offload envelopes are never elided, and when one is folded
away its `result_id` is carried forward in the summary so `read_result`
still resolves the payload. Envelopes are also excluded from the
fold-ratio calculation, since they're pinned cost that folding can't
reclaim.
---
## AgentOS route
```
POST /agents/{agent_id}/sessions/{session_id}/compact
```
Returns **200 with a status** whether or not it folded, because
declining is a normal outcome rather than a failure — raising would make
"folding would not help here" indistinguishable from an error and force
every caller to catch it.
```json
{
"status": "compacted",
"message": "Compacted 18 messages (15864 -> 3981 tokens).",
"compacted": true,
"record": { "messages_compacted": 18, "tokens_before": 15864, "tokens_after": 3981 }
}
```
| status | meaning |
|---|---|
| `compacted` | the fold happened; `record` carries the counts |
| `not_worth_it` | the span is too small to pay for the summary
replacing it |
| `nothing_to_fold` | the kept tail covers the whole conversation |
| `already_compacted` | a previous fold already covers everything
foldable |
| `no_history` | the session has no stored history yet |
| `not_enabled` | compaction is not configured on this agent |
| `summary_failed` | the summarizer returned nothing |
A missing agent is a 404 — that is a real error. Consumed by the AgentOS
chat UI as a "Compact now" action (separate repo, not in this diff).
---
## Type of change
- [x] New feature
- [ ] Bug fix
- [ ] Breaking change
- [ ] Improvement
- [ ] Model update
- [ ] Other:
---
## Checklist
- [x] Code complies with style guidelines
- [x] Ran format/validation scripts (`./scripts/format.sh` and
`./scripts/validate.sh`)
- [x] Self-review completed
- [x] Documentation updated (comments, docstrings)
- [x] Examples and guides: cookbook examples included
- [x] Tested in clean environment
- [x] Tests added/updated
### Duplicate and AI-Generated PR Check
- [x] I have searched existing open pull requests
- [x] If a similar PR exists, I have explained below why this PR is a
better approach
- [x] Check if this PR was entirely AI-generated (by Copilot, Claude
Code, Cursor, etc.)
---
## Relationship to #9834 and #9291
**#9834** (context compaction for agents and teams) is the more complete
implementation, and this PR **adopts its core architecture**: derived
views (`_view.py`), pair-safe durable cut selection (`_cut.py`),
local-only token estimation, and its structured summarization prompt.
Credit where due — those are better than what this PR had.
What this PR keeps that #9834 does not have:
- **A searchable archive.** #9834's summary *is* the history; a detail
it dropped is gone. Here the folded transcript is stored and greppable,
so the summary works as an index over ground truth.
- **Records in a table** rather than on the session, giving
`continue_run` correctness and multi-writer safety.
- **Manual folding** — `agent.compact()` plus an AgentOS route, with
declines reported as statuses rather than raised.
What #9834 still has that this does not: background folds, an in-run
loop seam, and Teams.
**#9291** is superseded by both. It rewrites stored history in place,
has no handling for provider-side conversation state (so its savings are
partly illusory on `OpenAIResponses`), and its own PR body notes the DX
is under review across three companion PRs.
---
## Testing
**83 unit tests** in `libs/agno/tests/unit/compaction/` plus **4 route
tests** in `libs/agno/tests/unit/os/routers/`: trigger matrix, boundary
safety (property test over every cut point), grep behaviour, archive
round-trip and cross-session isolation, provider-state detachment, both
fold guards, `continue_run` resolution, the planner-window fix and its
warning, the manual-compaction result contract, the trigger measuring a
tool loop's context rather than its billing total, and the size-bounded
tail with its mutual exclusion.
Cookbooks, run end-to-end against a live model:
- `cookbook/02_agents/03_context_management/compaction/` — basic,
thresholds, searchable archive, streaming events, async, manual, with
tools, Anthropic, Anthropic + thinking
- `cookbook/05_agent_os/21_compaction/` — an AgentOS serving both an
automatic and a manual-only agent, plus a REST script driving a fold
over HTTP
---
## Additional Notes
Deliberate follow-ups, not gaps:
- **Team parity.** `Team` mirrors every compression flag but has none of
this wired yet — the largest remaining gap.
- **Background folds and the in-run seam.** The summarizer call
currently blocks the turn, and compaction only fires at run start, so a
single long tool-calling run can still overflow. These are coupled: a
fold started in the background needs a loop-top to be adopted at.
- **`compress_tool_results` deprecation.** The existing surface still
runs in parallel. Nothing here changes or breaks it — it also still
covers in-run tool results that compaction cannot reach.
- **A post-hoc size check.** The ratio bounds the input but cannot
guarantee the outcome; verifying a fold actually shrank the context
needs a check after the summary exists.
---------
Co-authored-by: Anurag <[email protected]>
Co-authored-by: Harsh <[email protected]>
Co-authored-by: Harsh Sinha <[email protected]>
Co-authored-by: Claude Opus 5.5 (1M context) <[email protected]>
Co-authored-by: Yash Pratap Solanky <[email protected]>
This commit is contained in:
Kaustubh
co-authored by
Anurag
Harsh
Harsh Sinha
Claude Opus 5.5
Yash Pratap Solanky
Anurag
Harsh
Harsh Sinha
Claude Opus 5.5
Yash Pratap Solanky
parent
98e3751e52
commit
83dd1329bc
@@ -7,7 +7,7 @@ on:
|
||||
branches:
|
||||
- "*release*" # Run on release-* format
|
||||
- "release/*" # Run on release/1.2.3 branch format
|
||||
- "feat/v3.1"
|
||||
- "feat/agno-compaction"
|
||||
workflow_dispatch: # Allows manual triggering
|
||||
inputs:
|
||||
branch:
|
||||
@@ -806,6 +806,7 @@ jobs:
|
||||
./libs/agno/tests/integration/teams/test_session_state.py \
|
||||
./libs/agno/tests/integration/teams/test_storage_and_memory.py \
|
||||
./libs/agno/tests/integration/teams/human_in_the_loop/test_team_tool_hitl.py \
|
||||
./libs/agno/tests/integration/teams/human_in_the_loop/test_team_approval_multi_pause.py \
|
||||
./libs/agno/tests/integration/teams/human_in_the_loop/test_team_background_stream_continue.py \
|
||||
./libs/agno/tests/integration/teams/test_filter_tool_calls.py \
|
||||
./libs/agno/tests/integration/teams/test_multimodal.py
|
||||
|
||||
@@ -3,6 +3,19 @@
|
||||
Examples for instructions, system messages, introduction messages, and context shaping.
|
||||
|
||||
## Files
|
||||
- `compaction/compaction.py` - Keep a long session in the context window with `compaction=True`.
|
||||
- `compaction/compaction_thresholds.py` - Tune when compaction fires and how much it keeps.
|
||||
- `compaction/compaction_manual.py` - Fold now with `agent.compact()`, and read the status when it declines.
|
||||
- `compaction/compaction_token_counter.py` - Choose how tokens are counted: the local default, the model's own count with `use_model_token_count=True`, or any tokenizer such as Hugging Face.
|
||||
- `compaction/compaction_async_token_counter.py` - Count tokens on the async path: an async `token_counter` awaited in `arun`, here the model's count plus a safety margin.
|
||||
- `compaction/compaction_tail_budget.py` - Bound the kept tail by size with `uncompacted_tokens`, for agents whose turns vary in length.
|
||||
- `compaction/compaction_threshold_room.py` - Size `compact_at_tokens` so a fold leaves room: too small folds every run (and warns), sized well folds every few runs.
|
||||
- `compaction/compaction_searchable_archive.py` - Let the agent search history that was compacted away.
|
||||
- `compaction/compaction_events.py` - Stream `CompactionStarted` / `CompactionCompleted` and show what was reclaimed.
|
||||
- `compaction/compaction_async.py` - Compaction on the async path.
|
||||
- `compaction/compaction_with_tools.py` - Folding a transcript that contains tool calls, without splitting a call from its result.
|
||||
- `compaction/compaction_anthropic.py` - The same folding on Claude, which carries history in the request rather than by id.
|
||||
- `compaction/compaction_anthropic_thinking.py` - Extended thinking plus tool calls, the hardest shape for a cut to respect.
|
||||
- `few_shot_learning.py` - Demonstrates few-shot learning with example messages.
|
||||
- `filter_tool_calls_from_history.py` - Filter tool calls from conversation history.
|
||||
- `instructions.py` - Set agent instructions.
|
||||
|
||||
@@ -1,10 +1,110 @@
|
||||
# Test Log -- 03_context_management
|
||||
|
||||
> Note: entries below predate the removal of `compact_at_runs` / `compact_at_messages`.
|
||||
> Compaction now triggers on `compact_at_tokens` only, or on an explicit `agent.compact()`.
|
||||
> The configurations quoted here no longer construct; the observations still stand.
|
||||
|
||||
|
||||
**Tested:** 2026-02-13
|
||||
**Environment:** .venvs/demo/bin/python, pgvector: running
|
||||
|
||||
---
|
||||
|
||||
### compaction/compaction_with_tools.py
|
||||
|
||||
**Status:** PASS
|
||||
**Description:** Folding a transcript containing tool calls. Four calculator calls across five
|
||||
turns, then a question that spans them.
|
||||
**Result:** 12 messages folded, 1014 -> 471 tokens. All 4 assistant tool-call turns and 4 tool
|
||||
results still stored (23 messages total).
|
||||
|
||||
---
|
||||
|
||||
### compaction/compaction_anthropic.py
|
||||
|
||||
**Status:** PASS
|
||||
**Description:** The same folding on `claude-sonnet-4-5`. Anthropic carries history in the
|
||||
request rather than by id, so there is no server-side chain for compaction to sever.
|
||||
**Result:** 6 messages folded, 24602 -> 7732 tokens (69% smaller).
|
||||
|
||||
---
|
||||
|
||||
### compaction/compaction_anthropic_thinking.py
|
||||
|
||||
**Status:** PASS
|
||||
**Description:** Extended thinking (`budget_tokens=1024`) plus calculator tools - an assistant
|
||||
turn is then a thinking block, a tool call, and its result, all of which must stay together.
|
||||
**Result:** 18 messages folded, 7858 -> 1621 tokens (79% smaller). 15 thinking blocks preserved
|
||||
in storage; no signature or pairing errors.
|
||||
|
||||
---
|
||||
|
||||
### compaction.py
|
||||
|
||||
**Status:** PASS
|
||||
**Tier:** untagged
|
||||
**Description:** Compaction with `compaction=Compaction(compact_at_runs=5, uncompacted_runs=2)` over an
|
||||
8-turn session. Verified the compaction fired, the summary replaced the older turns, and the agent
|
||||
still answered "remind me what my budget was" correctly from a turn that had been compacted away.
|
||||
**Result:** Completed successfully. 6 messages replaced by the summary, all 16 still stored in the
|
||||
session, archived at `0001.md`.
|
||||
|
||||
---
|
||||
|
||||
### compaction_thresholds.py
|
||||
|
||||
**Status:** PASS
|
||||
**Tier:** untagged
|
||||
**Description:** Token-based threshold (`compact_at_tokens=100_000`) with a cheaper summarization
|
||||
model. Two short runs stayed well under the threshold.
|
||||
**Result:** Completed successfully. 0 compactions, as expected for a short session.
|
||||
|
||||
---
|
||||
|
||||
### compaction_searchable_archive.py
|
||||
|
||||
**Status:** PASS
|
||||
**Tier:** untagged
|
||||
**Description:** `search_compacted_messages=True` with `compacted_token_budget=150` and manual folding, on a fresh
|
||||
session per run. Shares a 30-row parts list, folds it away - no 150-token summary can carry 30 rows -
|
||||
then asks for the lot number of part 23 with `print_response`, so the tool call is visible.
|
||||
**Result:** 6 of 6 live runs on `gpt-5.6-luna` folded, called `search_compacted_history` (e.g.
|
||||
`pattern=part 23|23|lot`, shown in the Tool Calls panel) and answered with the correct lot number,
|
||||
82137. The earlier version planted three short values a summary could keep, so the final question
|
||||
was often answered from the summary with no search, and reused a fixed session id across runs.
|
||||
|
||||
---
|
||||
|
||||
### compaction_events.py
|
||||
|
||||
**Status:** PASS
|
||||
**Tier:** untagged
|
||||
**Description:** Streams `CompactionStarted` / `CompactionCompleted` over six long-answer turns and
|
||||
prints the token reduction each compaction achieved.
|
||||
**Result:** Completed successfully. 3 compactions, reducing context by 44.6%, 55.4% and 40.7%
|
||||
(for example 16787 -> 9296 tokens), each archived to its own file.
|
||||
|
||||
---
|
||||
|
||||
### compaction_async.py
|
||||
|
||||
**Status:** PASS
|
||||
**Tier:** untagged
|
||||
**Description:** Compaction on the async path via `aprint_response`.
|
||||
**Result:** Completed successfully. 1 compaction.
|
||||
|
||||
---
|
||||
|
||||
### compaction_local_archive.py
|
||||
|
||||
**Status:** PASS
|
||||
**Tier:** untagged
|
||||
**Description:** `fs=LocalFileSystem(root="tmp/compaction_archive")` writes the archive to disk as
|
||||
markdown instead of into the database.
|
||||
**Result:** Completed successfully. Wrote a readable `0001.md` under a per-session directory.
|
||||
|
||||
---
|
||||
|
||||
### few_shot_learning.py
|
||||
|
||||
**Status:** PASS
|
||||
@@ -58,3 +158,27 @@
|
||||
**Result:** Completed successfully in 11s.
|
||||
|
||||
---
|
||||
|
||||
### compaction/compaction_token_counter.py
|
||||
|
||||
**Status:** PASS
|
||||
**Description:** Compares three token counters on the same two messages, then runs an agent with `Compaction(compact_at_tokens=1_500, uncompacted_runs=1, use_model_token_count=True)` on `gpt-5.6-luna` over five detailed questions.
|
||||
**Result:** Every threshold check counted with the model's own `count_tokens` (4995, 8768, 13104, 12119 tokens) with no fallback. The fold happened on the fourth run: `Folded 4 messages (13072 -> 5404 tokens)`. Earlier runs declined because the fold was still under 2x the one-run tail (the fold ratio guard was then on by default; it is now opt-in via `enforce_min_fold_ratio`). `compact_at_tokens=1_500` sits below the size after a fold, so compaction warns that the next run will fold again - expected for a threshold this low.
|
||||
|
||||
---
|
||||
|
||||
### compaction/compaction_async_token_counter.py
|
||||
|
||||
**Status:** PASS
|
||||
**Description:** Runs an agent with `arun`, `num_history_runs=10` and `Compaction(compact_at_tokens=2_800, uncompacted_runs=1, token_counter=count_with_margin)`, where `count_with_margin` is an async function returning the model's `acount_tokens` plus 10%, on `gpt-5.6-luna`. Each of five turns shares a fixed ~700-token document and asks for a one-word reply, on a fresh session per run.
|
||||
**Result:** Deterministic across two runs - every size check awaited the async counter with the same counts (716 -> 787, 1430 -> 1573, 2144 -> 2358, 2858 -> 3143). The fifth run crossed the threshold and folded both times: `Folded 6 messages (2816 -> 1070 tokens)` and `(2816 -> 1147 tokens)`; only the summary's length varies. Completed in 18s.
|
||||
|
||||
---
|
||||
|
||||
### compaction/compaction_threshold_room.py
|
||||
|
||||
**Status:** PASS
|
||||
**Description:** The same six ~700-token turns twice on `gpt-5.6-luna`, with `uncompacted_runs=1` and `compacted_token_budget=300`: first at `compact_at_tokens=1_200`, then at `3_000`.
|
||||
**Result:** At 1,200 every run folded from run 3 (846-1,010 tokens left after each fold) and compaction warned on each fold that the threshold leaves too little room after the system prompt, tools and summary. At 3,000 the first fold came on run 6 (979 tokens left), with no warning. Completed in 55s.
|
||||
|
||||
---
|
||||
|
||||
@@ -0,0 +1,87 @@
|
||||
"""
|
||||
Compaction
|
||||
=============================
|
||||
|
||||
Keeps a long session inside the context window: older messages are replaced by a
|
||||
generated summary and the recent turns are kept verbatim. Only the messages sent
|
||||
to the model are shortened - the session still stores every message, which the run
|
||||
at the bottom demonstrates.
|
||||
|
||||
`compaction=True` is the whole setup. It folds when the provider rejects a request
|
||||
as too long, and not before. A proactive threshold would be a guess about a number
|
||||
nobody can look up - no provider exposes its context window, and the same model id
|
||||
differs across deployments - so the rejection is the one signal that is always right.
|
||||
|
||||
Pass a `Compaction` object to opt into a proactive threshold instead:
|
||||
|
||||
compaction=Compaction(compact_at_tokens=100_000)
|
||||
|
||||
Either way `agent.compact()` folds on demand, which is what this example uses: a
|
||||
short demo reaches neither the provider's limit nor a threshold.
|
||||
"""
|
||||
|
||||
from agno.agent import Agent
|
||||
from agno.compaction import Compaction
|
||||
from agno.db.postgres import PostgresDb
|
||||
from agno.models.openai import OpenAIResponses
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Create Database
|
||||
# ---------------------------------------------------------------------------
|
||||
db_url = "postgresql+psycopg://ai:ai@localhost:5532/ai"
|
||||
db = PostgresDb(db_url=db_url)
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Create Agent
|
||||
# ---------------------------------------------------------------------------
|
||||
agent = Agent(
|
||||
model=OpenAIResponses(id="gpt-5.6-luna"),
|
||||
db=db,
|
||||
session_id="compaction_demo",
|
||||
add_history_to_context=True,
|
||||
# `compaction=True` would fold only on a provider rejection. uncompacted_runs is lowered so the fold
|
||||
# below has history in front of the tail to work with at demo scale.
|
||||
compaction=Compaction(uncompacted_runs=2),
|
||||
)
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Run Agent
|
||||
# ---------------------------------------------------------------------------
|
||||
if __name__ == "__main__":
|
||||
questions = [
|
||||
"I am planning a trip to Japan in April. Remember that my budget is 4000 dollars.",
|
||||
"What are the best cities to see cherry blossoms?",
|
||||
"How many days should I spend in Kyoto?",
|
||||
"What is a reasonable daily food budget there?",
|
||||
"Do I need a rail pass?",
|
||||
"What about getting a pocket wifi?",
|
||||
"Which airport should I fly into?",
|
||||
]
|
||||
|
||||
for question in questions:
|
||||
agent.print_response(question)
|
||||
|
||||
# Fold now, rather than waiting for the context to reach compact_at_tokens.
|
||||
# Returns a CompactionResult: `compacted` says whether a fold happened, and
|
||||
# `message` explains the outcome either way - declining is a normal answer,
|
||||
# since a summary cannot pay for itself on a short span.
|
||||
result = agent.compact(session_id="compaction_demo")
|
||||
if result.compacted:
|
||||
r = result.record
|
||||
print(
|
||||
f"\n[compacted {r.messages_compacted} messages: "
|
||||
f"{r.tokens_before} -> {r.tokens_after} tokens, archived={r.archived}]"
|
||||
)
|
||||
else:
|
||||
# Not an error: a summary cannot pay for itself on a short span, so the
|
||||
# server declines and says why.
|
||||
print(f"\n[{result.status.value}] {result.message}")
|
||||
|
||||
# The next run sends the summary in place of the folded turns.
|
||||
agent.print_response("Remind me what my budget was.")
|
||||
|
||||
# The summary shortens the request, never the record: every message the
|
||||
# session stored is still there.
|
||||
session = agent.get_session(session_id="compaction_demo")
|
||||
stored = sum(len(run.messages or []) for run in session.runs or [])
|
||||
print(f"\nMessages still stored in the session: {stored}")
|
||||
@@ -0,0 +1,65 @@
|
||||
"""
|
||||
Compaction With Anthropic
|
||||
=============================
|
||||
|
||||
Compaction is provider-agnostic: it rewrites the message list, not the request
|
||||
format, so the same folding works on Claude as on OpenAI.
|
||||
|
||||
One difference is worth knowing. OpenAI's Responses API can continue a
|
||||
conversation by id (`previous_response_id`), which means the server replays
|
||||
history the request never sent - so compaction has to sever that chain, or the
|
||||
saving is imaginary. Anthropic has no such mechanism: every request carries its
|
||||
own history, so what compaction sends is exactly what the model sees.
|
||||
|
||||
Prerequisites: ANTHROPIC_API_KEY, and a Postgres running on localhost:5532
|
||||
Run: .venvs/demo/bin/python cookbook/02_agents/03_context_management/compaction/compaction_anthropic.py
|
||||
"""
|
||||
|
||||
from agno.agent import Agent
|
||||
from agno.compaction import Compaction
|
||||
from agno.db.postgres import PostgresDb
|
||||
from agno.models.anthropic import Claude
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Create Database
|
||||
# ---------------------------------------------------------------------------
|
||||
db_url = "postgresql+psycopg://ai:ai@localhost:5532/ai"
|
||||
db = PostgresDb(db_url=db_url)
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Create Agent
|
||||
# ---------------------------------------------------------------------------
|
||||
# Low token threshold so a short demo trips the automatic path; the default is 150k.
|
||||
compaction = Compaction(compact_at_tokens=3_000, uncompacted_runs=1)
|
||||
|
||||
agent = Agent(
|
||||
model=Claude(id="claude-sonnet-4-5"),
|
||||
db=db,
|
||||
session_id="compaction_anthropic",
|
||||
add_history_to_context=True,
|
||||
compaction=compaction,
|
||||
instructions=[
|
||||
"Answer thoroughly - long answers grow the context, which is the point here."
|
||||
],
|
||||
)
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Run Agent
|
||||
# ---------------------------------------------------------------------------
|
||||
if __name__ == "__main__":
|
||||
for question in [
|
||||
"Explain how B-tree indexes work, in detail.",
|
||||
"Explain hash indexes and how they differ, in detail.",
|
||||
"Explain covering indexes and index-only scans, in detail.",
|
||||
"Explain when an index hurts more than it helps, in detail.",
|
||||
"Summarize the trade-offs we covered.",
|
||||
]:
|
||||
agent.print_response(question)
|
||||
run = agent.get_last_run_output(session_id="compaction_anthropic")
|
||||
if run is not None and run.compaction is not None:
|
||||
r = run.compaction
|
||||
print(
|
||||
f"\n[compacted {r.messages_compacted} messages: {r.tokens_before} -> {r.tokens_after} tokens]"
|
||||
)
|
||||
|
||||
print(f"\nCompactions: {compaction.stats.compactions}")
|
||||
@@ -0,0 +1,83 @@
|
||||
"""
|
||||
Compaction With Thinking And Tools
|
||||
=============================
|
||||
|
||||
Extended thinking plus tool calls is the hardest shape for compaction, because
|
||||
an assistant turn is then three things at once: a thinking block, a tool call,
|
||||
and the result that answers it. All three have to stay together.
|
||||
|
||||
How each provider carries thinking decides how much can go wrong:
|
||||
|
||||
- Anthropic keeps the thinking block *in the message*, as reasoning_content
|
||||
with a signature. Compaction copies messages without touching those fields, so
|
||||
a kept turn arrives intact and a folded one takes its thinking with it.
|
||||
|
||||
- OpenAI's Responses API keeps reasoning items *on the server*, reachable by
|
||||
previous_response_id. Compaction severs that chain on purpose - otherwise the
|
||||
server replays the history the fold just removed - so the reasoning item has
|
||||
to travel in the request instead, or the API rejects the function_call it
|
||||
belongs to.
|
||||
|
||||
Either way the rule is the same: a tool batch is never split by the cut.
|
||||
|
||||
Prerequisites: ANTHROPIC_API_KEY, and a Postgres running on localhost:5532
|
||||
Run: .venvs/demo/bin/python cookbook/02_agents/03_context_management/compaction/compaction_anthropic_thinking.py
|
||||
"""
|
||||
|
||||
from agno.agent import Agent
|
||||
from agno.compaction import Compaction
|
||||
from agno.db.postgres import PostgresDb
|
||||
from agno.models.anthropic import Claude
|
||||
from agno.tools.calculator import CalculatorTools
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Create Database
|
||||
# ---------------------------------------------------------------------------
|
||||
db_url = "postgresql+psycopg://ai:ai@localhost:5532/ai"
|
||||
db = PostgresDb(db_url=db_url)
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Create Agent
|
||||
# ---------------------------------------------------------------------------
|
||||
# Low token threshold so a short demo trips the automatic path; the default is 150k.
|
||||
compaction = Compaction(compact_at_tokens=3_000, uncompacted_runs=1)
|
||||
|
||||
agent = Agent(
|
||||
model=Claude(
|
||||
id="claude-sonnet-4-5", thinking={"type": "enabled", "budget_tokens": 1024}
|
||||
),
|
||||
db=db,
|
||||
session_id="compaction_anthropic_thinking",
|
||||
tools=[CalculatorTools()],
|
||||
add_history_to_context=True,
|
||||
compaction=compaction,
|
||||
instructions=[
|
||||
"Use the calculator for arithmetic rather than doing it in your head.",
|
||||
"Then answer at length - long answers grow the context, which is the point here.",
|
||||
],
|
||||
)
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Run Agent
|
||||
# ---------------------------------------------------------------------------
|
||||
if __name__ == "__main__":
|
||||
for question in [
|
||||
"Compute 8123 * 4471, then explain multiplication algorithms in detail.",
|
||||
"Compute 99991 / 7, then explain long division and remainders in detail.",
|
||||
"Compute the factorial of 12, then explain combinatorial growth in detail.",
|
||||
"Compute 2**40, then explain binary growth and storage sizes in detail.",
|
||||
"Which result so far was the largest, and why?",
|
||||
]:
|
||||
agent.print_response(question)
|
||||
run = agent.get_last_run_output(session_id="compaction_anthropic_thinking")
|
||||
if run is not None and run.compaction is not None:
|
||||
r = run.compaction
|
||||
print(
|
||||
f"\n[compacted {r.messages_compacted} messages: {r.tokens_before} -> {r.tokens_after} tokens]"
|
||||
)
|
||||
|
||||
session = agent.get_session(session_id="compaction_anthropic_thinking")
|
||||
messages = [m for run in (session.runs or []) for m in (run.messages or [])]
|
||||
thinking = sum(1 for m in messages if m.reasoning_content)
|
||||
print(f"\nStored messages: {len(messages)} | with thinking: {thinking}")
|
||||
print(f"Compactions: {compaction.stats.compactions}")
|
||||
@@ -0,0 +1,60 @@
|
||||
"""
|
||||
Async Compaction
|
||||
=============================
|
||||
|
||||
Compaction works the same way on the async path - the summary is generated with
|
||||
`aresponse`, so a fold never blocks the event loop, and the folded messages are archived
|
||||
in the agent's database alongside the session.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
|
||||
from agno.agent import Agent
|
||||
from agno.compaction import Compaction
|
||||
from agno.db.postgres import PostgresDb
|
||||
from agno.models.openai import OpenAIResponses
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Create Database
|
||||
# ---------------------------------------------------------------------------
|
||||
db_url = "postgresql+psycopg://ai:ai@localhost:5532/ai"
|
||||
db = PostgresDb(db_url=db_url)
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Create Agent
|
||||
# ---------------------------------------------------------------------------
|
||||
# Keep only the latest turn verbatim, so a short demo has something in front of it to fold.
|
||||
compaction = Compaction(uncompacted_runs=1)
|
||||
|
||||
agent = Agent(
|
||||
model=OpenAIResponses(id="gpt-5.6-luna"),
|
||||
db=db,
|
||||
session_id="compaction_async",
|
||||
add_history_to_context=True,
|
||||
compaction=compaction,
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Run Agent
|
||||
# ---------------------------------------------------------------------------
|
||||
async def main() -> None:
|
||||
questions = [
|
||||
"Explain the CAP theorem in detail, with worked examples.",
|
||||
"Explain partition tolerance in depth and why it is not optional.",
|
||||
"Compare AP and CP databases in detail, with named systems.",
|
||||
"Explain consistency models in detail: linearizable to eventual.",
|
||||
"Which model would you pick for a payments ledger, and why?",
|
||||
]
|
||||
for question in questions:
|
||||
await agent.aprint_response(question)
|
||||
|
||||
# The async counterpart of agent.compact(), returning the same CompactionResult.
|
||||
result = await agent.acompact(session_id=agent.session_id)
|
||||
print(f"\n[{result.status.value}] {result.message}")
|
||||
|
||||
print(f"Compactions: {compaction.stats.compactions}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
asyncio.run(main())
|
||||
@@ -0,0 +1,72 @@
|
||||
"""
|
||||
Async Token Counting
|
||||
=============================
|
||||
|
||||
In async runs, `token_counter` can be an async function: it is awaited, so a network call never
|
||||
blocks the event loop. Here it is the model's own count plus a 10% safety margin, so the fold
|
||||
happens a little before the limit. Without a margin, `use_model_token_count=True` does the same.
|
||||
|
||||
Each turn shares a fixed ~700-token document, so the fold lands on the fifth run every time.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
from uuid import uuid4
|
||||
|
||||
from agno.agent import Agent
|
||||
from agno.compaction import Compaction
|
||||
from agno.db.sqlite import SqliteDb
|
||||
from agno.models.openai import OpenAIResponses
|
||||
|
||||
model = OpenAIResponses(id="gpt-5.6-luna")
|
||||
|
||||
|
||||
async def count_with_margin(messages, tools):
|
||||
exact = await model.acount_tokens(messages, tools)
|
||||
print(f" counted {int(exact * 1.1)} tokens (model's count {exact} + 10%)")
|
||||
return int(exact * 1.1)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Create Agent
|
||||
# ---------------------------------------------------------------------------
|
||||
agent = Agent(
|
||||
model=model,
|
||||
db=SqliteDb(db_file="tmp/compaction_async_token_counter.db"),
|
||||
session_id=f"compaction_async_token_counter_{uuid4().hex[:8]}",
|
||||
add_history_to_context=True,
|
||||
compaction=Compaction(
|
||||
# Four ~700-token turns plus the margin cross this; three do not.
|
||||
compact_at_tokens=2_800,
|
||||
uncompacted_runs=1,
|
||||
token_counter=count_with_margin,
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
def release_notes(version: int) -> str:
|
||||
"""Forty changes for one release - the same size for every version."""
|
||||
lines = [f"Release notes for version 2.{version}.0"]
|
||||
for change in range(1, 41):
|
||||
lines.append(
|
||||
f"- change {version}.{change}: ticket OPS-{1000 + 37 * version + change}, rollout {5 + change}%"
|
||||
)
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Run Agent
|
||||
# ---------------------------------------------------------------------------
|
||||
async def main() -> None:
|
||||
for version in range(1, 6):
|
||||
print(f"\nRun {version}")
|
||||
run = await agent.arun(
|
||||
f"File these release notes. Reply with just: noted.\n\n{release_notes(version)}"
|
||||
)
|
||||
if run.compaction is not None:
|
||||
print(
|
||||
f" Folded {run.compaction.messages_compacted} messages ({run.compaction.tokens_before} -> {run.compaction.tokens_after} tokens)"
|
||||
)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
asyncio.run(main())
|
||||
@@ -0,0 +1,86 @@
|
||||
"""
|
||||
Compaction Events
|
||||
=============================
|
||||
|
||||
Compaction emits `CompactionStarted` and `CompactionCompleted` on a streaming
|
||||
run, so a UI can show what happened to the context instead of silently losing
|
||||
turns.
|
||||
|
||||
`CompactionCompleted` carries how many messages were replaced, the token count
|
||||
before and after, and where the originals were archived.
|
||||
|
||||
Note these arrive during message assembly, before the model is called - so they
|
||||
land near the start of a run, ahead of the first `ModelRequestStarted`.
|
||||
|
||||
The questions below ask for long answers on purpose, so the conversation crosses
|
||||
`compact_at_tokens` within a few turns.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
|
||||
from agno.agent import Agent
|
||||
from agno.compaction import Compaction
|
||||
from agno.db.postgres import PostgresDb
|
||||
from agno.models.openai import OpenAIResponses
|
||||
from agno.run.agent import RunEvent
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Create Database
|
||||
# ---------------------------------------------------------------------------
|
||||
db_url = "postgresql+psycopg://ai:ai@localhost:5532/ai"
|
||||
db = PostgresDb(db_url=db_url)
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Create Agent
|
||||
# ---------------------------------------------------------------------------
|
||||
agent = Agent(
|
||||
model=OpenAIResponses(id="gpt-5.6-luna"),
|
||||
db=db,
|
||||
session_id="compaction_events",
|
||||
add_history_to_context=True,
|
||||
# A low token threshold so a short demo trips the AUTOMATIC path - the manual
|
||||
# agent.compact() folds just as well but emits no run events, and events are
|
||||
# what this example is about.
|
||||
compaction=Compaction(compact_at_tokens=2_000, uncompacted_runs=1),
|
||||
)
|
||||
|
||||
|
||||
async def main():
|
||||
questions = [
|
||||
"Explain message queues in detail, with delivery guarantees and examples.",
|
||||
"Compare queues and topics in depth, covering fan-out and consumer groups.",
|
||||
"Explain at-least-once, at-most-once and exactly-once delivery in detail.",
|
||||
"Explain in detail how to make a consumer idempotent, with examples.",
|
||||
"Explain dead letter queues in detail: causes, handling and replay.",
|
||||
"Explain backpressure strategies in detail, with trade-offs.",
|
||||
]
|
||||
|
||||
for question in questions:
|
||||
print(f"\n--- {question}")
|
||||
|
||||
async for chunk in agent.arun(question, stream=True, stream_events=True):
|
||||
if chunk.event == RunEvent.compaction_started.value:
|
||||
print("[CompactionStarted] context is being compacted")
|
||||
|
||||
elif chunk.event == RunEvent.compaction_completed.value:
|
||||
print(
|
||||
f"[CompactionCompleted] replaced {chunk.messages_compacted} messages"
|
||||
)
|
||||
if chunk.tokens_before and chunk.tokens_after:
|
||||
saved = chunk.tokens_before - chunk.tokens_after
|
||||
reduction = (1 - chunk.tokens_after / chunk.tokens_before) * 100
|
||||
print(
|
||||
f" Tokens: {chunk.tokens_before} -> {chunk.tokens_after} (saved {saved}, {reduction:.1f}%)"
|
||||
)
|
||||
if chunk.archived:
|
||||
print(" Originals archived and searchable")
|
||||
|
||||
elif chunk.event == RunEvent.run_completed.value:
|
||||
print("[RunCompleted]")
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Run Agent
|
||||
# ---------------------------------------------------------------------------
|
||||
if __name__ == "__main__":
|
||||
asyncio.run(main())
|
||||
@@ -0,0 +1,76 @@
|
||||
"""
|
||||
Manual Compaction
|
||||
=============================
|
||||
|
||||
`agent.compact()` folds the conversation now, rather than waiting for it to reach
|
||||
`compact_at_tokens`. Useful at a natural seam: the end of a topic, or before handing
|
||||
the agent a long task.
|
||||
|
||||
Compaction can legitimately decline - there may be nothing in front of the kept tail
|
||||
to fold yet - so this returns a result rather than raising, and the status says which
|
||||
case you hit:
|
||||
|
||||
- `compacted` - the fold happened; `record` carries the token counts
|
||||
- `not_worth_it` - with `enforce_min_fold_ratio=True`: the span is too small to pay for its summary
|
||||
- `nothing_to_fold` - the kept tail covers the whole conversation
|
||||
- `already_compacted` - a previous fold already covers everything foldable
|
||||
- `no_history` - the session has no stored history yet
|
||||
- `not_enabled` - compaction is not configured on this agent
|
||||
- `summary_failed` - the summarizer returned nothing
|
||||
|
||||
The same result is what the AgentOS route returns as JSON:
|
||||
|
||||
POST /agents/{agent_id}/sessions/{session_id}/compact
|
||||
"""
|
||||
|
||||
from agno.agent import Agent
|
||||
from agno.compaction import Compaction
|
||||
from agno.db.postgres import PostgresDb
|
||||
from agno.models.openai import OpenAIResponses
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Create Database
|
||||
# ---------------------------------------------------------------------------
|
||||
db_url = "postgresql+psycopg://ai:ai@localhost:5532/ai"
|
||||
db = PostgresDb(db_url=db_url)
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Create Agent
|
||||
# ---------------------------------------------------------------------------
|
||||
agent = Agent(
|
||||
model=OpenAIResponses(id="gpt-5.6-luna"),
|
||||
db=db,
|
||||
session_id="compaction_manual",
|
||||
add_history_to_context=True,
|
||||
# No automatic trigger at all: this session folds only when asked to, keeping the
|
||||
# latest turn verbatim.
|
||||
compaction=Compaction(compact_at_tokens=None, uncompacted_runs=1),
|
||||
)
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Run Agent
|
||||
# ---------------------------------------------------------------------------
|
||||
if __name__ == "__main__":
|
||||
for question in [
|
||||
"Explain database indexing in detail, with worked examples.",
|
||||
"Explain B-tree indexes in depth and when they are the right choice.",
|
||||
"Explain hash indexes and how they differ, in detail.",
|
||||
"Explain covering indexes and index-only scans, in detail.",
|
||||
]:
|
||||
agent.print_response(question)
|
||||
|
||||
# Fold at a seam of your choosing.
|
||||
result = agent.compact(session_id="compaction_manual")
|
||||
print(f"\nstatus : {result.status.value}")
|
||||
print(f"message: {result.message}")
|
||||
|
||||
if result.compacted:
|
||||
print(f"tokens : {result.record.tokens_before} -> {result.record.tokens_after}")
|
||||
else:
|
||||
# Not an error - compaction decided folding would not help here.
|
||||
print("History was left unchanged.")
|
||||
|
||||
# Calling again immediately declines: the previous fold already covers
|
||||
# everything foldable, and nothing new has been said since.
|
||||
again = agent.compact(session_id="compaction_manual")
|
||||
print(f"\nsecond call: {again.status.value} - {again.message}")
|
||||
@@ -0,0 +1,97 @@
|
||||
"""
|
||||
Compaction With A Searchable Archive
|
||||
=============================
|
||||
|
||||
`search_compacted_messages=True` gives the agent read-only search over its own stored
|
||||
history, which changes what a summary is for. Normally a summary replaces the
|
||||
conversation, so any detail it left out is gone. Here the originals are still
|
||||
stored, so the summary works as an index and the agent can go read the rest.
|
||||
|
||||
The flow is summary-first, archive-as-fallback. When a summary is enough the
|
||||
agent answers from it. When the question needs a detail the summary could not
|
||||
keep, the agent calls `search_compacted_history` and reads it from the archive.
|
||||
|
||||
This example makes that fallback happen every time. It shares a 30-row parts
|
||||
list, folds it away under a deliberately small summary budget - no summary of
|
||||
that size can carry 30 rows - and then asks for a single row. The run prints
|
||||
the search the agent made, so you can see where the answer came from.
|
||||
|
||||
`search_compacted_messages` is on by default whenever `store_compacted_messages` is;
|
||||
it is set here to make the example explicit.
|
||||
"""
|
||||
|
||||
from uuid import uuid4
|
||||
|
||||
from agno.agent import Agent
|
||||
from agno.compaction import Compaction
|
||||
from agno.db.postgres import PostgresDb
|
||||
from agno.models.openai import OpenAIResponses
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Compaction
|
||||
# ---------------------------------------------------------------------------
|
||||
compaction = Compaction(
|
||||
search_compacted_messages=True,
|
||||
# Fold only when this example says so, so every run folds at the same point.
|
||||
compact_at_tokens=None,
|
||||
# Keep only the most recent turn verbatim; everything before it is folded.
|
||||
uncompacted_runs=1,
|
||||
# A summary this small cannot carry a 30-row table, so the rows have to be
|
||||
# looked up in the archive - which is what this example demonstrates.
|
||||
compacted_token_budget=150,
|
||||
)
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Create Database
|
||||
# ---------------------------------------------------------------------------
|
||||
db_url = "postgresql+psycopg://ai:ai@localhost:5532/ai"
|
||||
db = PostgresDb(db_url=db_url)
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Create Agent
|
||||
# ---------------------------------------------------------------------------
|
||||
agent = Agent(
|
||||
model=OpenAIResponses(id="gpt-5.6-luna"),
|
||||
db=db,
|
||||
# A fresh session per run, so an earlier run's history cannot change this one.
|
||||
session_id=f"compaction_searchable_{uuid4().hex[:8]}",
|
||||
add_history_to_context=True,
|
||||
compaction=compaction,
|
||||
)
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Run Agent
|
||||
# ---------------------------------------------------------------------------
|
||||
if __name__ == "__main__":
|
||||
# 1. Share a detailed list. Values a model cannot guess, so a correct answer
|
||||
# can only have come from the archive.
|
||||
parts = "\n".join(
|
||||
f"part {i}: {['valve', 'gasket', 'bearing', 'seal', 'flange'][i % 5]}, "
|
||||
f"supplier {['Acme', 'Globex', 'Initech', 'Umbrella', 'Hooli'][i % 5]}, "
|
||||
f"lot {i * 7919 % 100000:05d}"
|
||||
for i in range(1, 31)
|
||||
)
|
||||
agent.print_response(f"Here is our parts list for later. Just say noted.\n{parts}")
|
||||
|
||||
# 2. A turn after it, so there is history in front of the kept tail to fold.
|
||||
agent.print_response("In one sentence: what is a bill of materials?")
|
||||
|
||||
# 3. Fold now. The parts list leaves the context; only a short summary and the
|
||||
# archived original remain.
|
||||
result = agent.compact(session_id=agent.session_id)
|
||||
print(f"\n[{result.status.value}] {result.message}")
|
||||
|
||||
# 4. Ask for one row. The summary cannot have it, so the agent searches - the
|
||||
# search_compacted_history call shows up in the output below.
|
||||
agent.print_response(
|
||||
"What is the lot number of part 23 in the parts list I shared?"
|
||||
)
|
||||
|
||||
run = agent.get_last_run_output()
|
||||
searches = [
|
||||
tool.tool_args.get("pattern")
|
||||
for tool in (run.tools if run else None) or []
|
||||
if tool.tool_name == "search_compacted_history"
|
||||
]
|
||||
print(f"\nArchive searches: {searches}")
|
||||
print(f"Expected lot number: {23 * 7919 % 100000:05d}")
|
||||
@@ -0,0 +1,94 @@
|
||||
"""
|
||||
Bounding The Kept Tail By Size
|
||||
=============================
|
||||
|
||||
`uncompacted_runs` says how many turns survive a fold. It does not say how large they
|
||||
are - and compaction only folds what sits in FRONT of the kept tail, so a verbose
|
||||
turn landing inside the tail is one it can never bring back down.
|
||||
|
||||
`uncompacted_tokens` bounds the tail's size instead. On detailed turns of roughly 4,700
|
||||
tokens each, with `compact_at_tokens=8_000`:
|
||||
|
||||
uncompacted_runs=2 asks for two turns -> ~9,400 tokens, more than the threshold itself
|
||||
uncompacted_tokens=2000 keeps one turn -> ~4,700 tokens, well under it
|
||||
|
||||
Both mean "keep the recent conversation", but two turns this size could never be folded
|
||||
back under the threshold - with everything in front of them folded, the context would
|
||||
still be over. So compaction overrides the run count and keeps one turn, logging that it
|
||||
did. A run count is a setting the model's answer lengths can overrule; a token budget says
|
||||
the size you actually want, and with it how much room each fold leaves under the threshold.
|
||||
|
||||
With `enforce_min_fold_ratio=True` (off by default), a large tail also blocks the fold
|
||||
itself: the fold must be at least `min_fold_ratio` times the tail, and two big turns
|
||||
leave too little in front of them to pass.
|
||||
|
||||
The two are mutually exclusive - they describe the same tail in different units, so
|
||||
setting both raises rather than silently picking a winner.
|
||||
|
||||
The cut still snaps to a turn boundary, so a tool call is never severed from its
|
||||
result. That means the tail can come out somewhat larger than the budget: it is a
|
||||
target, not a hard cap.
|
||||
|
||||
Reach for `uncompacted_tokens` when turns vary a lot in length, which is most
|
||||
tool-calling agents. `uncompacted_runs` is simpler and fine when turns are uniform.
|
||||
|
||||
Prerequisites: OPENAI_API_KEY, and a local Postgres (./cookbook/scripts/run_pgvector.sh)
|
||||
Run: .venvs/demo/bin/python cookbook/02_agents/03_context_management/compaction/compaction_tail_budget.py
|
||||
"""
|
||||
|
||||
from agno.agent import Agent
|
||||
from agno.compaction import Compaction
|
||||
from agno.db.postgres import PostgresDb
|
||||
from agno.models.openai import OpenAIResponses
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Create Database
|
||||
# ---------------------------------------------------------------------------
|
||||
db_url = "postgresql+psycopg://ai:ai@localhost:5532/ai"
|
||||
db = PostgresDb(db_url=db_url)
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Create Agent
|
||||
# ---------------------------------------------------------------------------
|
||||
compaction = Compaction(
|
||||
compact_at_tokens=8_000,
|
||||
# The tail is bounded by size, not by turn count.
|
||||
uncompacted_tokens=2_000,
|
||||
)
|
||||
|
||||
agent = Agent(
|
||||
model=OpenAIResponses(id="gpt-5.6-luna"),
|
||||
db=db,
|
||||
session_id="compaction_tail_budget",
|
||||
add_history_to_context=True,
|
||||
compaction=compaction,
|
||||
markdown=True,
|
||||
)
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Run Agent
|
||||
# ---------------------------------------------------------------------------
|
||||
if __name__ == "__main__":
|
||||
# Every turn is long - the shape a run count cannot bound, because however many turns
|
||||
# it keeps, their size is up to the model.
|
||||
questions = [
|
||||
"Explain how B-tree indexes work, in full detail, with worked examples.",
|
||||
"Explain hash indexes and how they differ, in full detail.",
|
||||
"Explain covering indexes and index-only scans, in full detail.",
|
||||
"Explain when an index hurts more than it helps, in full detail.",
|
||||
"Explain index maintenance and fragmentation, in full detail.",
|
||||
]
|
||||
|
||||
for question in questions:
|
||||
agent.print_response(question)
|
||||
run = agent.get_last_run_output(session_id="compaction_tail_budget")
|
||||
if run is not None and run.compaction is not None:
|
||||
r = run.compaction
|
||||
print(
|
||||
f"\n[compacted {r.messages_compacted} messages: {r.tokens_before} -> {r.tokens_after} tokens]"
|
||||
)
|
||||
|
||||
# Compaction shortens the request, never the record.
|
||||
session = agent.get_session(session_id="compaction_tail_budget")
|
||||
stored = sum(len(run.messages or []) for run in session.runs or [])
|
||||
print(f"\nMessages still stored in the session: {stored}")
|
||||
@@ -0,0 +1,71 @@
|
||||
"""
|
||||
Sizing compact_at_tokens
|
||||
=============================
|
||||
|
||||
After a fold, the request still carries the system prompt and tool definitions, the summary,
|
||||
and the latest turn - compaction never folds the turn the model is answering. If
|
||||
`compact_at_tokens` is not much bigger than that, a fold lands just under it, the next turn
|
||||
crosses it again, and every run pays for another summary. Compaction warns when this happens.
|
||||
|
||||
Rule of thumb: set `compact_at_tokens` to at least the system prompt and tools, plus the
|
||||
summary, plus about two of your largest turns. At real thresholds (100k+) this only matters
|
||||
when single turns are huge - a 60k-token tool result against a 100k threshold, say.
|
||||
|
||||
This runs the same conversation twice - six turns of ~700 tokens each - first with a
|
||||
threshold barely above one turn, then with one sized by the rule of thumb.
|
||||
"""
|
||||
|
||||
from uuid import uuid4
|
||||
|
||||
from agno.agent import Agent
|
||||
from agno.compaction import Compaction
|
||||
from agno.db.sqlite import SqliteDb
|
||||
from agno.models.openai import OpenAIResponses
|
||||
|
||||
db = SqliteDb(db_file="tmp/compaction_threshold_room.db")
|
||||
|
||||
|
||||
def release_notes(version: int) -> str:
|
||||
"""Forty changes for one release - about 700 tokens, the same for every version."""
|
||||
lines = [f"Release notes for version 2.{version}.0"]
|
||||
for change in range(1, 41):
|
||||
lines.append(
|
||||
f"- change {version}.{change}: ticket OPS-{1000 + 37 * version + change}, rollout {5 + change}%"
|
||||
)
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
def run(label: str, compact_at_tokens: int) -> None:
|
||||
agent = Agent(
|
||||
model=OpenAIResponses(id="gpt-5.6-luna"),
|
||||
db=db,
|
||||
session_id=f"compaction_threshold_room_{uuid4().hex[:8]}",
|
||||
add_history_to_context=True,
|
||||
compaction=Compaction(
|
||||
compact_at_tokens=compact_at_tokens,
|
||||
uncompacted_runs=1,
|
||||
compacted_token_budget=300,
|
||||
),
|
||||
)
|
||||
print(f"\n{label}: compact_at_tokens={compact_at_tokens}")
|
||||
for version in range(1, 7):
|
||||
result = agent.run(
|
||||
f"File these release notes. Reply with just: noted.\n\n{release_notes(version)}"
|
||||
)
|
||||
if result.compaction is not None:
|
||||
print(
|
||||
f" run {version}: folded -> {result.compaction.tokens_after} tokens left"
|
||||
)
|
||||
else:
|
||||
print(f" run {version}: no fold")
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Run Agent
|
||||
# ---------------------------------------------------------------------------
|
||||
if __name__ == "__main__":
|
||||
# One ~700-token turn plus a 300-token summary is already ~1,000: every run folds, and
|
||||
# compaction warns that the threshold leaves no room.
|
||||
run("Too small", compact_at_tokens=1_200)
|
||||
# The summary plus about two turns, with room to spare: a fold every few runs, no warning.
|
||||
run("Sized by the rule of thumb", compact_at_tokens=3_000)
|
||||
@@ -0,0 +1,86 @@
|
||||
"""
|
||||
Compaction Thresholds
|
||||
=============================
|
||||
|
||||
Tuning when compaction fires and how much it keeps.
|
||||
|
||||
Size is the only automatic trigger. `compact_at_tokens` is measured from what the
|
||||
provider reported for the last run, so it costs nothing to evaluate. A run or message
|
||||
count is deliberately not offered: twenty short exchanges and twenty research turns
|
||||
differ by orders of magnitude, so counting them fires on conversations far too small
|
||||
to fold and stays quiet on ones that overflow.
|
||||
|
||||
Set `compact_at_tokens=None` to turn the automatic trigger off entirely and fold only
|
||||
when you call `agent.compact()`.
|
||||
|
||||
The kept tail is set one of two ways, and they cannot both be used:
|
||||
|
||||
- `uncompacted_runs` keeps whole turns. Simple, but a run count says how many turns
|
||||
survive, not how large they are - a few verbose turns produce a tail compaction
|
||||
cannot bring back down, because it only folds what sits in FRONT of the tail.
|
||||
- `uncompacted_tokens` bounds the tail's size instead. The cut still snaps to a turn
|
||||
boundary rather than severing a tool call from its result, so the tail can come out
|
||||
somewhat larger than asked - it is a budget, not a hard cap.
|
||||
|
||||
Reach for `uncompacted_tokens` when turns vary a lot in length, which is most
|
||||
tool-calling agents.
|
||||
|
||||
By default every fold the threshold asks for happens. `enforce_min_fold_ratio=True`
|
||||
skips one that would not shrink the context: the folded span must be at least
|
||||
`min_fold_ratio` (2.0) times the kept tail. A summary costs a few hundred tokens
|
||||
whatever it replaces, so folding a span not much bigger than the tail can leave the
|
||||
context larger. A skipped fold is retried as the conversation grows.
|
||||
|
||||
A cheaper model can do the summarizing, which is usually the right call: the
|
||||
work is mechanical and the main model never sees the transcript being condensed.
|
||||
"""
|
||||
|
||||
from agno.agent import Agent
|
||||
from agno.compaction import Compaction
|
||||
from agno.db.postgres import PostgresDb
|
||||
from agno.models.openai import OpenAIResponses
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Compaction
|
||||
# ---------------------------------------------------------------------------
|
||||
compaction = Compaction(
|
||||
# A small model is enough to summarize a transcript.
|
||||
model=OpenAIResponses(id="gpt-5-mini"),
|
||||
compact_at_tokens=1_000,
|
||||
# Keep the last turn verbatim; everything older folds into the summary.
|
||||
uncompacted_runs=1,
|
||||
# Skip folds too small to shrink the context (off by default):
|
||||
# enforce_min_fold_ratio=True,
|
||||
)
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Create Database
|
||||
# ---------------------------------------------------------------------------
|
||||
db_url = "postgresql+psycopg://ai:ai@localhost:5532/ai"
|
||||
db = PostgresDb(db_url=db_url)
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Create Agent
|
||||
# ---------------------------------------------------------------------------
|
||||
agent = Agent(
|
||||
model=OpenAIResponses(id="gpt-5.6-luna"),
|
||||
db=db,
|
||||
session_id="compaction_thresholds",
|
||||
add_history_to_context=True,
|
||||
compaction=compaction,
|
||||
)
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Run Agent
|
||||
# ---------------------------------------------------------------------------
|
||||
if __name__ == "__main__":
|
||||
for question in [
|
||||
"Explain how a database index works, in detail.",
|
||||
"Now compare that to a hash index, in detail.",
|
||||
"Explain covering indexes and index-only scans, in detail.",
|
||||
"Explain when an index hurts more than it helps, in detail.",
|
||||
]:
|
||||
agent.print_response(question)
|
||||
|
||||
print(f"\nCompactions so far: {compaction.stats.compactions}")
|
||||
print(f"Messages compacted: {compaction.stats.messages_compacted}")
|
||||
@@ -0,0 +1,110 @@
|
||||
"""
|
||||
Compaction Token Counter
|
||||
========================
|
||||
|
||||
`compact_at_tokens` is a number of tokens, so something has to count them. By default
|
||||
compaction uses a local tiktoken estimate: in-process, a few milliseconds, and a few percent
|
||||
off for models that do not use OpenAI's tokenizer.
|
||||
|
||||
Three ways to count:
|
||||
|
||||
- Nothing (the default): the local tiktoken estimate.
|
||||
- `use_model_token_count=True`: the model the request goes to counts it - `count_tokens` in
|
||||
sync runs, `acount_tokens` in async ones. Exact, but for Claude, OpenAI Responses, Gemini and
|
||||
Bedrock it is a network call on every size check (roughly 350-700 ms).
|
||||
- `token_counter=`: any function that takes `(messages, tools)` and returns an int, such as a
|
||||
Hugging Face tokenizer for an open-weight model - wrapped in a small function, as below.
|
||||
|
||||
The count decides when `compact_at_tokens` is reached and sizes the request during overflow
|
||||
recovery. If counting fails, compaction logs a warning and falls back to the local estimate,
|
||||
so it can never fail a run.
|
||||
"""
|
||||
|
||||
import json
|
||||
|
||||
from agno.agent import Agent
|
||||
from agno.compaction import Compaction
|
||||
from agno.db.sqlite import SqliteDb
|
||||
from agno.models.message import Message
|
||||
from agno.models.openai import OpenAIResponses
|
||||
from agno.utils.tokens import count_tokens
|
||||
from tokenizers import Tokenizer
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# A Hugging Face tokenizer as a token counter
|
||||
# ---------------------------------------------------------------------------
|
||||
# The tokenizer an open-weight model was trained with counts its tokens exactly. Any
|
||||
# tokenizer works the same way: turn the messages (and tool definitions) into text and
|
||||
# count the tokens.
|
||||
llama3 = Tokenizer.from_pretrained("Xenova/llama-3-tokenizer")
|
||||
|
||||
|
||||
def llama3_counter(messages, tools):
|
||||
text = "\n".join(message.get_content_string() for message in messages)
|
||||
if tools:
|
||||
# Tools arrive as Function objects; count the schema the model is sent, not their repr.
|
||||
schemas = [
|
||||
tool.to_dict() if hasattr(tool, "to_dict") else tool for tool in tools
|
||||
]
|
||||
text += "\n" + json.dumps(schemas)
|
||||
return len(llama3.encode(text).ids)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Create Agent
|
||||
# ---------------------------------------------------------------------------
|
||||
model = OpenAIResponses(id="gpt-5.6-luna")
|
||||
|
||||
agent = Agent(
|
||||
model=model,
|
||||
db=SqliteDb(db_file="tmp/compaction_token_counter.db"),
|
||||
session_id="compaction_token_counter",
|
||||
add_history_to_context=True,
|
||||
compaction=Compaction(
|
||||
# Low enough that this short demo reaches it; the default is 150k.
|
||||
compact_at_tokens=1_500,
|
||||
uncompacted_runs=1,
|
||||
# The model counts its own requests. Or count with your own function instead:
|
||||
# token_counter=llama3_counter
|
||||
use_model_token_count=True,
|
||||
),
|
||||
)
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Run Agent
|
||||
# ---------------------------------------------------------------------------
|
||||
if __name__ == "__main__":
|
||||
# 1. Three counters, one conversation. They disagree, and by how much depends on the
|
||||
# text - which is why the choice is yours.
|
||||
sample = [
|
||||
Message(
|
||||
role="user",
|
||||
content="Summarize the 2023 revenue: Q1 1.2M, Q2 1.45M, Q3 1.61M, Q4 2.07M.",
|
||||
),
|
||||
Message(
|
||||
role="assistant",
|
||||
content="Revenue grew every quarter, from 1.2M to 2.07M, totalling 6.33M.",
|
||||
),
|
||||
]
|
||||
print("Counting the same two messages:")
|
||||
print(f" local estimate (default) : {count_tokens(sample)}")
|
||||
print(f" OpenAI count_tokens : {model.count_tokens(sample)}")
|
||||
print(f" Llama 3 tokenizer : {llama3_counter(sample, None)}")
|
||||
|
||||
# 2. A conversation that crosses compact_at_tokens, measured by the provider.
|
||||
questions = [
|
||||
"Explain how a database index works, in detail.",
|
||||
"Now compare that to a hash index, in detail.",
|
||||
"Explain covering indexes and index-only scans, in detail.",
|
||||
"Explain when an index hurts more than it helps, in detail.",
|
||||
"Explain partial and expression indexes, in detail.",
|
||||
]
|
||||
for question in questions:
|
||||
run = agent.run(question)
|
||||
if run.compaction is not None:
|
||||
print(
|
||||
f"\nFolded {run.compaction.messages_compacted} messages "
|
||||
f"({run.compaction.tokens_before} -> {run.compaction.tokens_after} tokens)"
|
||||
)
|
||||
else:
|
||||
print(f"\nNo fold yet after: {question}")
|
||||
@@ -0,0 +1,81 @@
|
||||
"""
|
||||
Compaction With Tools
|
||||
=============================
|
||||
|
||||
A tool call is stored as three messages, not one:
|
||||
|
||||
assistant tool_calls=[{id: call_1, ...}] the request
|
||||
tool tool_call_id=call_1 the result - usually the bulk
|
||||
assistant "the answer" what the model made of it
|
||||
|
||||
Compaction has to respect that grouping. The cut never lands between an
|
||||
assistant's tool_calls and the tool message answering it: providers reject a
|
||||
function_call whose result is missing, so a boundary that would split a batch is
|
||||
moved to keep the batch whole.
|
||||
|
||||
Prerequisites: OPENAI_API_KEY, and a Postgres running on localhost:5532
|
||||
Run: .venvs/demo/bin/python cookbook/02_agents/03_context_management/compaction/compaction_with_tools.py
|
||||
"""
|
||||
|
||||
from agno.agent import Agent
|
||||
from agno.compaction import Compaction
|
||||
from agno.db.postgres import PostgresDb
|
||||
from agno.models.openai import OpenAIResponses
|
||||
from agno.tools.calculator import CalculatorTools
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Create Database
|
||||
# ---------------------------------------------------------------------------
|
||||
db_url = "postgresql+psycopg://ai:ai@localhost:5532/ai"
|
||||
db = PostgresDb(db_url=db_url)
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Create Agent
|
||||
# ---------------------------------------------------------------------------
|
||||
compaction = Compaction(
|
||||
# Low enough that a short demo trips the automatic path; the default is 150k.
|
||||
# Tool-calling turns are short - a calculation and a paragraph, not an essay -
|
||||
# so this sits well under what a prose-heavy agent would need.
|
||||
compact_at_tokens=2_000,
|
||||
uncompacted_runs=2,
|
||||
)
|
||||
|
||||
agent = Agent(
|
||||
model=OpenAIResponses(id="gpt-5.4"),
|
||||
db=db,
|
||||
session_id="compaction_with_tools",
|
||||
tools=[CalculatorTools()],
|
||||
add_history_to_context=True,
|
||||
compaction=compaction,
|
||||
instructions=[
|
||||
"Use the calculator for arithmetic rather than doing it in your head."
|
||||
],
|
||||
)
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Run Agent
|
||||
# ---------------------------------------------------------------------------
|
||||
if __name__ == "__main__":
|
||||
for question in [
|
||||
"Compute 8123 * 4471 and explain what the magnitude means.",
|
||||
"Now compute 99991 / 7 and explain long division.",
|
||||
"Compute the factorial of 12 and explain why factorials grow so fast.",
|
||||
"Compute 2**40 and explain binary growth.",
|
||||
"Which of the results so far was the largest?",
|
||||
]:
|
||||
agent.print_response(question)
|
||||
run = agent.get_last_run_output(session_id="compaction_with_tools")
|
||||
if run is not None and run.compaction is not None:
|
||||
r = run.compaction
|
||||
print(
|
||||
f"\n[compacted {r.messages_compacted} messages: {r.tokens_before} -> {r.tokens_after} tokens]"
|
||||
)
|
||||
|
||||
# Every tool call and result is still stored, however much the request shrank.
|
||||
session = agent.get_session(session_id="compaction_with_tools")
|
||||
messages = [m for run in (session.runs or []) for m in (run.messages or [])]
|
||||
print(f"\nStored messages: {len(messages)}")
|
||||
print(
|
||||
f" assistant turns with tool calls: {sum(1 for m in messages if m.tool_calls)}"
|
||||
)
|
||||
print(f" tool results: {sum(1 for m in messages if m.role == 'tool')}")
|
||||
@@ -0,0 +1,67 @@
|
||||
# 28_compaction
|
||||
|
||||
Conversation compaction served on AgentOS, with the fold visible in the chat UI.
|
||||
|
||||
Compaction keeps a long session inside the model's context window: once the
|
||||
conversation crosses a threshold, older turns are folded into a summary and the
|
||||
recent ones are kept verbatim. The stored transcript is never rewritten, so the
|
||||
session still holds every message - only what is sent to the model gets shorter.
|
||||
|
||||
Two events stream to the UI while a fold runs:
|
||||
|
||||
| Event | Carries |
|
||||
|---|---|
|
||||
| `CompactionStarted` | run and session ids |
|
||||
| `CompactionCompleted` | `messages_compacted`, `tokens_before`, `tokens_after`, `archived` |
|
||||
|
||||
Agno OS renders these in the Behind the Scenes panel of a chat, so a long
|
||||
conversation shows what happened to its context instead of silently losing turns.
|
||||
|
||||
Compaction also folds on demand, without waiting for the threshold:
|
||||
|
||||
```
|
||||
POST /agents/{agent_id}/sessions/{session_id}/compact
|
||||
```
|
||||
|
||||
For a session that exists, the route answers 200 with a `status`, because declining is a
|
||||
normal outcome (a session that does not exist is a 404):
|
||||
a summary costs a few hundred tokens whatever it replaces, so folding a smaller span
|
||||
would leave the context bigger. Branch on `compacted`, and show `message` verbatim.
|
||||
|
||||
| status | meaning |
|
||||
|---|---|
|
||||
| `compacted` | the fold happened; `record` carries the token counts |
|
||||
| `not_worth_it` | with `enforce_min_fold_ratio=True`: the span is too small to pay for its summary |
|
||||
| `nothing_to_fold` | the kept tail covers the whole conversation |
|
||||
| `already_compacted` | a previous fold already covers everything foldable |
|
||||
| `no_history` | the session exists but has no stored history yet |
|
||||
| `not_enabled` | compaction is not configured on this agent |
|
||||
| `summary_failed` | the summarizer returned nothing |
|
||||
|
||||
## Files
|
||||
- `compaction_os.py` - An AgentOS serving two agents:
|
||||
- **Research Agent** (`compaction-agent`) folds automatically at `compact_at_tokens`.
|
||||
- **Manual Compaction Agent** (`manual-compaction-agent`) has `compact_at_tokens=None`,
|
||||
so it folds only when the API asks.
|
||||
- `rest_api_compaction.py` - Drive a fold over HTTP against the manual agent.
|
||||
|
||||
Two agents rather than one because a threshold and a manual call race each other: with
|
||||
the automatic trigger on, the server usually folds first and a manual call then reports
|
||||
`already_compacted` - work it did not do.
|
||||
|
||||
Sizing note: these agents are told to answer at length, so one turn costs roughly 10k
|
||||
tokens. `compact_at_tokens` has to sit above a single turn or it trips on turn one and
|
||||
re-evaluates every turn, mostly to decline.
|
||||
|
||||
## Prerequisites
|
||||
- Load environment variables with `direnv allow` (including `OPENAI_API_KEY`).
|
||||
- Create the demo environment with `./scripts/demo_setup.sh`, then run cookbooks with `.venvs/demo/bin/python`.
|
||||
|
||||
## Run
|
||||
- `.venvs/demo/bin/python cookbook/05_agent_os/28_compaction/compaction_os.py`
|
||||
- Open the chat UI, pick the Research Agent, and ask several questions in one session.
|
||||
- After the fourth turn the Behind the Scenes panel shows "Context compacted".
|
||||
|
||||
With that server running, drive a fold over the API instead:
|
||||
- `.venvs/demo/bin/python cookbook/05_agent_os/28_compaction/rest_api_compaction.py`
|
||||
- It compacts twice at the end: the second call reports `already_compacted`.
|
||||
@@ -0,0 +1,24 @@
|
||||
# Test Log -- 28_compaction
|
||||
|
||||
> Note: entries below predate the removal of `compact_at_runs` / `compact_at_messages`.
|
||||
> Compaction now triggers on `compact_at_tokens` only, or on an explicit `agent.compact()`.
|
||||
> The configurations quoted here no longer construct; the observations still stand.
|
||||
|
||||
|
||||
**Tested:** 2026-09-03
|
||||
**Environment:** .venvs/demo/bin/python
|
||||
|
||||
---
|
||||
|
||||
### compaction_os.py
|
||||
|
||||
**Status:** PASS
|
||||
|
||||
**Description:** Builds an AgentOS serving one agent with
|
||||
`Compaction(compact_at_runs=4, uncompacted_runs=2)` and a cheaper summarization
|
||||
model. Verified the FastAPI app constructs and the agent resolves its Compaction
|
||||
config with the expected thresholds.
|
||||
|
||||
**Result:** App built successfully; compaction resolved with runs=4, keep=2.
|
||||
|
||||
---
|
||||
@@ -0,0 +1,114 @@
|
||||
"""
|
||||
Compaction on AgentOS
|
||||
=====================
|
||||
|
||||
Serve an agent whose context is compacted as the conversation grows, and watch it
|
||||
happen from the chat UI.
|
||||
|
||||
Compaction folds older turns into a summary once the conversation crosses a
|
||||
threshold, keeping recent turns verbatim. The stored transcript is never
|
||||
rewritten - only what is sent to the model gets shorter - so the session still
|
||||
holds every message.
|
||||
|
||||
Two events stream to the UI while it runs:
|
||||
CompactionStarted - a fold has begun
|
||||
CompactionCompleted - messages_compacted, tokens_before, tokens_after
|
||||
|
||||
Agno OS renders these in the Behind the Scenes panel of a chat, so a long
|
||||
conversation shows "Context compacted - 6 messages folded, 17.8k -> 5.5k tokens"
|
||||
rather than silently losing turns.
|
||||
|
||||
Compaction also folds on demand, without waiting for the threshold:
|
||||
|
||||
POST /agents/{agent_id}/sessions/{session_id}/compact
|
||||
|
||||
See rest_api_compaction.py in this folder for that flow end to end.
|
||||
|
||||
Prerequisites: OPENAI_API_KEY
|
||||
Run: .venvs/demo/bin/python cookbook/05_agent_os/28_compaction/compaction_os.py
|
||||
Try: open the chat UI, then ask several long questions in one session
|
||||
"""
|
||||
|
||||
from agno.agent import Agent
|
||||
from agno.compaction import Compaction
|
||||
from agno.db.postgres import PostgresDb
|
||||
from agno.models.openai import OpenAIResponses
|
||||
from agno.os import AgentOS
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Create Database
|
||||
# ---------------------------------------------------------------------------
|
||||
db = PostgresDb(
|
||||
id="compaction-db", db_url="postgresql+psycopg://ai:ai@localhost:5532/ai"
|
||||
)
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Create Agent
|
||||
# ---------------------------------------------------------------------------
|
||||
# The default (fold at 150k tokens, keep the last 5 runs) suits a long-lived
|
||||
# session. These are lowered so a handful of turns in the UI is enough to see a fold.
|
||||
research_agent = Agent(
|
||||
id="compaction-agent",
|
||||
name="Research Agent",
|
||||
model=OpenAIResponses(id="gpt-5.6-luna"),
|
||||
db=db,
|
||||
add_history_to_context=True,
|
||||
compaction=Compaction(
|
||||
# A cheaper model is enough to write the summary.
|
||||
model=OpenAIResponses(id="gpt-5.4"),
|
||||
# Low enough that a chat session in the UI trips it, but above what a single
|
||||
# turn of this agent costs: it is instructed to answer at length, and one
|
||||
# question-plus-answer runs to ~10k tokens. A threshold under that is crossed
|
||||
# on turn one and re-evaluates every turn, mostly to decline.
|
||||
compact_at_tokens=25_000,
|
||||
# Keep the latest turn verbatim; everything older folds into the summary.
|
||||
uncompacted_runs=1,
|
||||
search_compacted_messages=True,
|
||||
),
|
||||
markdown=True,
|
||||
instructions=[
|
||||
"Answer thoroughly and at length - long answers make the context grow,",
|
||||
"which is what this example is demonstrating.",
|
||||
],
|
||||
)
|
||||
|
||||
# A second agent with the automatic trigger switched off, so folds happen only when
|
||||
# something asks for one. rest_api_compaction.py drives this agent: with a threshold
|
||||
# in play, the server would usually fold first and a manual call would just report
|
||||
# work it did not do.
|
||||
manual_agent = Agent(
|
||||
id="manual-compaction-agent",
|
||||
name="Manual Compaction Agent",
|
||||
model=OpenAIResponses(id="gpt-5.6-luna"),
|
||||
db=db,
|
||||
add_history_to_context=True,
|
||||
compaction=Compaction(
|
||||
model=OpenAIResponses(id="gpt-5.4"),
|
||||
# No automatic trigger: this session folds only via POST .../compact.
|
||||
compact_at_tokens=None,
|
||||
uncompacted_runs=1,
|
||||
search_compacted_messages=True,
|
||||
),
|
||||
markdown=True,
|
||||
instructions=[
|
||||
"Answer thoroughly and at length - long answers make the context grow,",
|
||||
"which is what this example is demonstrating.",
|
||||
],
|
||||
)
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Create AgentOS
|
||||
# ---------------------------------------------------------------------------
|
||||
agent_os = AgentOS(
|
||||
id="compaction-os",
|
||||
description="An AgentOS showing conversation compaction in the chat UI",
|
||||
db=db,
|
||||
agents=[research_agent, manual_agent],
|
||||
)
|
||||
app = agent_os.get_app()
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Run AgentOS
|
||||
# ---------------------------------------------------------------------------
|
||||
if __name__ == "__main__":
|
||||
agent_os.serve(app="compaction_os:app", reload=True)
|
||||
@@ -0,0 +1,157 @@
|
||||
"""
|
||||
Compact a Session over REST
|
||||
===========================
|
||||
|
||||
Drive compaction from the AgentOS API rather than from Python: hold a conversation
|
||||
over /runs, fold it with /compact, and confirm the stored transcript is untouched.
|
||||
|
||||
POST /agents/{agent_id}/sessions/{session_id}/compact
|
||||
|
||||
For a session that exists, the route answers 200 with a status, because declining to
|
||||
compact is a normal outcome rather than a failure (a session that does not exist is a
|
||||
404, as on the other session routes). A summary costs a few hundred tokens whatever it
|
||||
replaces, so folding a span smaller than that would leave the context BIGGER - which
|
||||
is why the server declines instead of obeying:
|
||||
|
||||
compacted the fold happened; "record" carries the token counts
|
||||
not_worth_it with enforce_min_fold_ratio=True: the span is too small to pay for its summary
|
||||
nothing_to_fold the kept tail covers the whole conversation
|
||||
already_compacted a previous fold already covers everything foldable
|
||||
no_history the session exists but has no stored history yet
|
||||
not_enabled compaction is not configured on this agent
|
||||
summary_failed the summarizer returned nothing
|
||||
|
||||
A UI branches on "compacted" and shows "message" verbatim.
|
||||
|
||||
Prerequisites: compaction_os.py running on http://localhost:7777
|
||||
Run: .venvs/demo/bin/python cookbook/05_agent_os/28_compaction/rest_api_compaction.py
|
||||
The script compacts twice at the end: the second call reports already_compacted.
|
||||
"""
|
||||
|
||||
import os
|
||||
from typing import Any
|
||||
from uuid import uuid4
|
||||
|
||||
import httpx
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Configuration
|
||||
# ---------------------------------------------------------------------------
|
||||
BASE_URL = os.getenv("AGENT_OS_BASE_URL", "http://localhost:7777")
|
||||
# The agent with no automatic trigger, so every fold here is one this script asked
|
||||
# for. Point this at "compaction-agent" instead to watch the two paths interleave.
|
||||
AGENT_ID = "manual-compaction-agent"
|
||||
|
||||
# Enough long answers that there is a real span to fold in front of the kept tail.
|
||||
# Asking for detail is what makes them long.
|
||||
QUESTIONS = [
|
||||
"Explain database indexing in detail, with worked examples.",
|
||||
"Explain B-tree indexes in depth and when they are the right choice.",
|
||||
"Explain hash indexes and how they differ, in detail.",
|
||||
"Explain covering indexes and index-only scans, in detail.",
|
||||
"Explain partial and filtered indexes in detail, with examples.",
|
||||
"Explain when an index hurts more than it helps, in detail.",
|
||||
]
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# API Helpers
|
||||
# ---------------------------------------------------------------------------
|
||||
def ask(client: httpx.Client, session_id: str, message: str) -> None:
|
||||
"""One non-streaming turn."""
|
||||
response = client.post(
|
||||
f"/agents/{AGENT_ID}/runs",
|
||||
data={"message": message, "session_id": session_id, "stream": "false"},
|
||||
)
|
||||
response.raise_for_status()
|
||||
|
||||
|
||||
def compact(client: httpx.Client, session_id: str) -> dict[str, Any]:
|
||||
"""Fold this session's history now, and report what the server decided."""
|
||||
response = client.post(f"/agents/{AGENT_ID}/sessions/{session_id}/compact")
|
||||
if response.status_code == 404:
|
||||
# Not a decline: there is no such session to fold.
|
||||
return {"status": "404", "message": response.json().get("detail", "Not found")}
|
||||
response.raise_for_status()
|
||||
return response.json()
|
||||
|
||||
|
||||
def show(result: dict[str, Any]) -> None:
|
||||
print(f" status : {result['status']}")
|
||||
print(f" message: {result['message']}")
|
||||
record = result.get("record")
|
||||
if record:
|
||||
print(
|
||||
f" tokens : {record['tokens_before']} -> {record['tokens_after']} "
|
||||
f"({record['messages_compacted']} messages folded)"
|
||||
)
|
||||
|
||||
|
||||
def stored_turn_count(client: httpx.Client, session_id: str) -> int:
|
||||
"""User and assistant messages the session still holds, whatever compaction did.
|
||||
|
||||
chat_history skips system and tool roles, so this is a count of conversation turns
|
||||
rather than of every stored message - enough to show the transcript is intact.
|
||||
"""
|
||||
response = client.get(f"/sessions/{session_id}", params={"type": "agent"})
|
||||
if response.status_code == 404:
|
||||
# The session is only created by the first run, so before that there is
|
||||
# nothing to count rather than an error.
|
||||
return 0
|
||||
response.raise_for_status()
|
||||
return len(response.json().get("chat_history") or [])
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Run
|
||||
# ---------------------------------------------------------------------------
|
||||
def main() -> None:
|
||||
session_id = f"compaction-rest-{uuid4().hex[:8]}"
|
||||
|
||||
with httpx.Client(base_url=BASE_URL, timeout=300.0) as client:
|
||||
print(f"Session: {session_id}\n")
|
||||
|
||||
# 1. The session is only created by the first run, so compacting it now is a 404 -
|
||||
# an error, unlike the declines below, which are normal outcomes.
|
||||
print("Compacting a session that does not exist yet:")
|
||||
show(compact(client, session_id))
|
||||
|
||||
# 2. Build up a conversation worth folding. The questions all run first, so
|
||||
# there is one sizeable span to fold rather than a small fold per turn.
|
||||
print("\nAsking questions...")
|
||||
for question in QUESTIONS:
|
||||
print(f" - {question}")
|
||||
ask(client, session_id, question)
|
||||
|
||||
before = stored_turn_count(client, session_id)
|
||||
|
||||
# 3. Fold it. This agent has compact_at_tokens=None, so nothing folds unless
|
||||
# asked - which makes the result below entirely this call's doing.
|
||||
print("\nCompacting:")
|
||||
result = compact(client, session_id)
|
||||
show(result)
|
||||
|
||||
# 4. Compaction shortens the request, never the record - but that is only
|
||||
# worth asserting when a fold actually happened.
|
||||
after = stored_turn_count(client, session_id)
|
||||
print(f"\nStored turns: {before} before, {after} after")
|
||||
if not result["compacted"]:
|
||||
if result["status"] == "already_compacted":
|
||||
print(" The server had already folded this session on its own.")
|
||||
else:
|
||||
print(" Nothing was folded, so there is nothing to compare.")
|
||||
print(" Ask more questions, or lower uncompacted_runs on the server.")
|
||||
return
|
||||
if before == after:
|
||||
print(
|
||||
" The transcript is unchanged - only what is sent to the model got shorter."
|
||||
)
|
||||
|
||||
# 5. Immediately again: the previous fold already covers everything
|
||||
# foldable, and nothing new has been said since.
|
||||
print("\nCompacting again straight away:")
|
||||
show(compact(client, session_id))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -57,6 +57,7 @@ surface.
|
||||
| [24_showcase](./24_showcase/) | Run the secure, traced capstone with RAG, web and finance research, a Team, and a real evaluation. |
|
||||
| [25_agentos_tools](./25_agentos_tools/) | Answer platform ops questions (usage, latency, tool statistics) with an agent using AgentOSTools. |
|
||||
| [26_authorization](./26_authorization/) | Replace the built-in scope check with managed roles, a user directory, an IdP, ReBAC, or your own provider. |
|
||||
| [28_compaction](./28_compaction/) | Keep long sessions inside the context window, and fold a session on demand over the `/compact` route. |
|
||||
|
||||
## Canonical ports
|
||||
|
||||
@@ -104,6 +105,7 @@ surface.
|
||||
| `24_showcase` | `OPENAI_API_KEY`, `ANTHROPIC_API_KEY`, `OS_SECURITY_KEY` | `./cookbook/scripts/run_pgvector.sh`, internet access, and tracing |
|
||||
| `25_agentos_tools` | `OPENAI_API_KEY` | Local SQLite with tracing enabled |
|
||||
| `26_authorization` | None for the local examples | Throwaway SQLite; `agno[os]`, and `agno[fga]` only to swap in a real OpenFGA |
|
||||
| `28_compaction` | `OPENAI_API_KEY` | `./cookbook/scripts/run_pgvector.sh`; AgentOS on 7777 for the REST script |
|
||||
|
||||
Run cookbook files with `.venvs/demo/bin/python`. Development checks use
|
||||
`.venv`.
|
||||
|
||||
@@ -197,6 +197,70 @@ def set_compression_manager(agent: Agent) -> None:
|
||||
agent.compress_tool_results = True
|
||||
|
||||
|
||||
def disable_compaction_without_records(agent: Agent) -> None:
|
||||
"""Turn compaction off, with a warning, when the db cannot store compaction records.
|
||||
|
||||
A fold lasts only as long as its record: without one, every run starts from the full history
|
||||
and pays for another summary of all of it. Only SqliteDb and PostgresDb store records. Checked
|
||||
when the agent is built, and again at run start for a db assigned afterwards.
|
||||
"""
|
||||
if not agent.compaction or agent.db is None:
|
||||
return
|
||||
from agno.compaction.archive import stores_compaction_records
|
||||
|
||||
if not stores_compaction_records(agent.db):
|
||||
log_warning(
|
||||
f"Compaction is only supported with SqliteDb and PostgresDb; {type(agent.db).__name__} cannot "
|
||||
"store compaction records, so compaction is turned off. Use SqliteDb or PostgresDb to enable it."
|
||||
)
|
||||
agent.compaction = None
|
||||
|
||||
|
||||
def set_compaction(agent: Agent) -> None:
|
||||
"""Resolve ``agent.compaction`` into the Compaction the run uses.
|
||||
|
||||
``True`` folds only when the provider rejects a request as too long. A proactive threshold
|
||||
is a guess about a number nobody can look up - no provider exposes its context window, and
|
||||
the same model id has different limits across deployments - so 150k is wrong for a 32k
|
||||
model and pointless for a 1M one. The rejection is the one signal that is always right, at
|
||||
the cost of a single failed request before the first fold.
|
||||
|
||||
Pass a ``Compaction`` object to opt into the proactive threshold, which defaults to 150k
|
||||
there because someone configuring it has a size in mind.
|
||||
|
||||
The model defaults to the agent's either way, so a bare ``compaction=True`` is still the
|
||||
cheapest correct configuration.
|
||||
"""
|
||||
from agno.compaction.compaction import Compaction
|
||||
|
||||
if agent.compaction is True:
|
||||
agent.compaction = Compaction(compact_at_tokens=None, on_context_overflow=True)
|
||||
elif agent.compaction is False:
|
||||
agent.compaction = None
|
||||
|
||||
disable_compaction_without_records(agent)
|
||||
|
||||
if isinstance(agent.compaction, Compaction) and agent.compaction.model is None:
|
||||
agent.compaction.model = agent.model
|
||||
|
||||
if isinstance(agent.compaction, Compaction):
|
||||
# Compaction folds the history a run replays. Without replayed history a size threshold
|
||||
# has nothing to measure and a fold is never sent. Overflow recovery still acts within a
|
||||
# run, so a bare compaction=True - which has no threshold - is not a mistake here.
|
||||
if not agent.add_history_to_context and agent.compaction.compact_at_tokens is not None:
|
||||
log_warning(
|
||||
"compaction is set but add_history_to_context is False, so no history is replayed: "
|
||||
"compact_at_tokens never fires and a fold is never sent. Set add_history_to_context=True."
|
||||
)
|
||||
# Both put a summary of the same history into the context, so the model reads it twice.
|
||||
if agent.add_session_summary_to_context:
|
||||
log_warning(
|
||||
"compaction and session summaries are both enabled, so the context carries two summaries "
|
||||
"of the same history. Compaction already replaces old turns with its own summary; "
|
||||
"consider add_session_summary_to_context=False."
|
||||
)
|
||||
|
||||
|
||||
def set_result_store(agent: Agent) -> None:
|
||||
"""Resolve ``agent.offload_tool_results`` into the store the run uses.
|
||||
|
||||
@@ -403,6 +467,11 @@ def get_models(agent: Agent) -> None:
|
||||
if agent.compression_manager is not None and agent.compression_manager.model is None:
|
||||
agent.compression_manager.model = agent.model
|
||||
|
||||
from agno.compaction.compaction import Compaction as _Compaction
|
||||
|
||||
if isinstance(agent.compaction, _Compaction) and agent.compaction.model is None:
|
||||
agent.compaction.model = agent.model
|
||||
|
||||
|
||||
def initialize_agent(agent: Agent, debug_mode: Optional[bool] = None) -> None:
|
||||
set_default_model(agent)
|
||||
@@ -418,6 +487,8 @@ def initialize_agent(agent: Agent, debug_mode: Optional[bool] = None) -> None:
|
||||
set_session_summary_manager(agent)
|
||||
if agent.compress_tool_results or agent.compression_manager is not None:
|
||||
set_compression_manager(agent)
|
||||
if agent.compaction is not None:
|
||||
set_compaction(agent)
|
||||
# Resolved when a setting is present or when a store exists.
|
||||
if agent.offload_tool_results or agent._result_store is not None:
|
||||
set_result_store(agent)
|
||||
|
||||
@@ -4,7 +4,7 @@ from __future__ import annotations
|
||||
|
||||
import re
|
||||
import string
|
||||
from collections import ChainMap
|
||||
from collections import ChainMap, Counter
|
||||
from typing import (
|
||||
TYPE_CHECKING,
|
||||
Any,
|
||||
@@ -12,6 +12,7 @@ from typing import (
|
||||
List,
|
||||
Optional,
|
||||
Sequence,
|
||||
Tuple,
|
||||
Type,
|
||||
Union,
|
||||
)
|
||||
@@ -20,10 +21,12 @@ from pydantic import BaseModel
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from agno.agent.agent import Agent
|
||||
from agno.compaction import Compaction
|
||||
|
||||
from agno.agent._utils import convert_dependencies_to_string, convert_documents_to_string
|
||||
from agno.filters import FilterExpr
|
||||
from agno.media import Audio, File, Image, Video
|
||||
from agno.metrics import RunMetrics
|
||||
from agno.models.message import Message, MessageReferences
|
||||
from agno.models.response import ModelResponse
|
||||
from agno.run import RunContext
|
||||
@@ -39,13 +42,787 @@ from agno.utils.agent import (
|
||||
execute_system_message,
|
||||
)
|
||||
from agno.utils.common import is_typed_dict
|
||||
from agno.utils.log import log_debug, log_info, log_warning
|
||||
from agno.utils.knowledge import get_run_response_kwarg, get_user_id_kwarg
|
||||
from agno.utils.log import log_debug, log_warning
|
||||
from agno.utils.message import copy_history_message, filter_tool_calls, get_text_from_message, render_instructions
|
||||
from agno.utils.prompts import get_json_output_prompt, get_response_model_format_prompt
|
||||
from agno.utils.timer import Timer
|
||||
|
||||
|
||||
def agent_compaction(agent: "Agent") -> Optional["Compaction"]:
|
||||
"""The agent's Compaction, or None when compaction is off.
|
||||
|
||||
set_compaction turns compaction=True into a Compaction at init, so anything else left on the
|
||||
field means compaction is off.
|
||||
"""
|
||||
from agno.compaction import Compaction
|
||||
|
||||
return agent.compaction if isinstance(agent.compaction, Compaction) else None
|
||||
|
||||
|
||||
def _stored_compaction(agent: "Agent", session: AgentSession, up_to_run_id: Optional[str] = None) -> Optional[Any]:
|
||||
"""The compaction in force for this run.
|
||||
|
||||
Read from the record table rather than from a value on the session: records are immutable
|
||||
facts about single runs, so two containers writing different runs never clobber one another,
|
||||
and a resumed run can resolve the fold it actually saw.
|
||||
"""
|
||||
from agno.compaction.types import CompactionRecord
|
||||
|
||||
compaction = agent_compaction(agent)
|
||||
if compaction is None:
|
||||
return None
|
||||
archive = compaction.archive_for(session.session_id, agent.db)
|
||||
if archive is None:
|
||||
return None
|
||||
row = archive.latest(up_to_run_id)
|
||||
if not row:
|
||||
return None
|
||||
try:
|
||||
return CompactionRecord.from_dict(row)
|
||||
except Exception as e: # noqa: BLE001
|
||||
log_warning(f"Ignoring unreadable compaction record: {e}")
|
||||
return None
|
||||
|
||||
|
||||
def _compaction_as_of(run_response: Optional[RunOutput]) -> Optional[str]:
|
||||
"""The run whose fold this run must inherit, if it is resuming one.
|
||||
|
||||
A fresh run returns None and takes the latest. A fork or regeneration returns the run it
|
||||
branched from, so it rebuilds the context that run actually had rather than one summarizing
|
||||
its own future.
|
||||
"""
|
||||
if run_response is None:
|
||||
return None
|
||||
return getattr(run_response, "forked_from_run_id", None) or getattr(run_response, "regenerated_from", None)
|
||||
|
||||
|
||||
def _estimated_context_tokens(
|
||||
agent: "Agent", messages: List[Message], tools: Optional[List[Any]] = None
|
||||
) -> Optional[int]:
|
||||
"""Local estimate for the model-bound context this compaction decision is about.
|
||||
|
||||
RunMetrics.input_tokens is billing telemetry. It accumulates every model call in a tool loop,
|
||||
plus background and compaction model calls, so it can be much larger than any single request
|
||||
the provider had to fit. ``compact_at_tokens`` is a context-size threshold, so use the current
|
||||
message view instead.
|
||||
|
||||
Counted by the model or ``Compaction.token_counter`` when one is chosen, and locally otherwise.
|
||||
"""
|
||||
from agno.compaction._tokens import count_request
|
||||
from agno.utils.tokens import count_tokens
|
||||
|
||||
counted = count_request(_sync_counter(agent), messages, tools)
|
||||
if counted is not None:
|
||||
return counted
|
||||
model_id = getattr(agent.model, "id", None) or "gpt-4o"
|
||||
try:
|
||||
return count_tokens(messages, tools=tools, model_id=model_id)
|
||||
except Exception as e: # noqa: BLE001 - token estimation must never fail a run
|
||||
log_warning(f"Could not estimate tokens for compaction: {e}")
|
||||
return None
|
||||
|
||||
|
||||
def _compaction_inputs(
|
||||
agent: "Agent", messages: Optional[List[Message]] = None, tools: Optional[List[Any]] = None
|
||||
) -> Dict[str, Any]:
|
||||
return {
|
||||
"context_tokens": _estimated_context_tokens(agent, messages, tools) if messages is not None else None,
|
||||
"model": agent.model,
|
||||
}
|
||||
|
||||
|
||||
async def _acompaction_inputs(
|
||||
agent: "Agent", messages: Optional[List[Message]] = None, tools: Optional[List[Any]] = None
|
||||
) -> Dict[str, Any]:
|
||||
"""``_compaction_inputs`` for async runs, counting without blocking the event loop."""
|
||||
if messages is None:
|
||||
return _compaction_inputs(agent, messages, tools)
|
||||
counted = await _acount(agent, messages, tools)
|
||||
return {
|
||||
"context_tokens": counted if counted is not None else _estimated_context_tokens(agent, messages, tools),
|
||||
"model": agent.model,
|
||||
}
|
||||
|
||||
|
||||
def _sync_counter(agent: "Agent") -> Any:
|
||||
"""What counts a request in a sync run: the model's own count_tokens when use_model_token_count
|
||||
is set, else the token_counter, else nothing - the local estimate."""
|
||||
compaction = agent.compaction
|
||||
if getattr(compaction, "use_model_token_count", False) and agent.model is not None:
|
||||
return agent.model.count_tokens
|
||||
return getattr(compaction, "token_counter", None)
|
||||
|
||||
|
||||
async def _acount(agent: "Agent", messages: List[Message], tools: Optional[List[Any]]) -> Optional[int]:
|
||||
"""A request's count in an async run, or None to use the local estimate.
|
||||
|
||||
The model's own count is awaited through acount_tokens. An async token_counter is awaited too;
|
||||
a sync one is usually a network call, so it runs in a worker thread rather than blocking the
|
||||
event loop.
|
||||
"""
|
||||
from agno.compaction._tokens import acount_request
|
||||
|
||||
compaction = agent.compaction
|
||||
if getattr(compaction, "use_model_token_count", False) and agent.model is not None:
|
||||
try:
|
||||
return int(await agent.model.acount_tokens(messages, tools))
|
||||
except Exception as e: # noqa: BLE001 - counting is never worth failing a run
|
||||
log_warning(f"Compaction: the model's token count failed, using the local estimate instead: {e}")
|
||||
return None
|
||||
return await acount_request(getattr(compaction, "token_counter", None), messages, tools)
|
||||
|
||||
|
||||
def _request_tokens(agent: "Agent", messages: List[Message], tools: Optional[List[Any]]) -> int:
|
||||
"""A request's size for overflow recovery, counted the way the trigger counts it, so before and
|
||||
after are comparable."""
|
||||
from agno.compaction._tokens import count_request, estimate_tokens
|
||||
|
||||
counted = count_request(_sync_counter(agent), messages, tools)
|
||||
return counted if counted is not None else estimate_tokens(messages, tools)
|
||||
|
||||
|
||||
async def _arequest_tokens(agent: "Agent", messages: List[Message], tools: Optional[List[Any]]) -> int:
|
||||
"""``_request_tokens`` for async runs."""
|
||||
from agno.compaction._tokens import estimate_tokens
|
||||
|
||||
counted = await _acount(agent, messages, tools)
|
||||
return counted if counted is not None else estimate_tokens(messages, tools)
|
||||
|
||||
|
||||
def _log_compaction(record: Any) -> None:
|
||||
"""Report what one fold achieved. Sizes were measured when the record was built."""
|
||||
detail = f"Compacted {record.messages_compacted} messages"
|
||||
if record.tokens_before and record.tokens_after:
|
||||
saved = record.tokens_before - record.tokens_after
|
||||
detail += f" ({record.tokens_before} -> {record.tokens_after} tokens, saved {saved})"
|
||||
if record.archived:
|
||||
detail += ", originals archived"
|
||||
log_info(detail)
|
||||
|
||||
|
||||
def _compaction_events(run_response: Optional[RunOutput], record: Any, started: bool = False) -> List[Any]:
|
||||
"""The events one compaction raises, when there is a run to attach them to."""
|
||||
if run_response is None:
|
||||
return []
|
||||
from agno.utils.events import create_compaction_completed_event, create_compaction_started_event
|
||||
|
||||
if started:
|
||||
return [create_compaction_started_event(from_run_response=run_response)]
|
||||
return [
|
||||
create_compaction_completed_event(
|
||||
from_run_response=run_response,
|
||||
messages_compacted=record.messages_compacted,
|
||||
tokens_before=record.tokens_before,
|
||||
tokens_after=record.tokens_after,
|
||||
archived=record.archived,
|
||||
)
|
||||
]
|
||||
|
||||
|
||||
# Headroom over compact_at_tokens for the planner's own window. The planner must see enough
|
||||
# history to find a foldable span BEFORE the trigger fires, plus the tail it will keep; a window
|
||||
# sized exactly at the trigger would leave nothing in front of the tail to fold.
|
||||
_PLANNER_WINDOW_MAX_RUNS = 500
|
||||
|
||||
|
||||
def _compaction_history_runs(agent: "Agent") -> Optional[int]:
|
||||
"""How many runs the compaction planner may read, or None for no extra limit.
|
||||
|
||||
num_history_runs governs how much history a RUN replays and defaults to 3. Compaction folds
|
||||
what sits in FRONT of the kept tail, so a 3-run window leaves it nothing to fold and it never
|
||||
fires - and worse, an anchor outside that window cannot resolve, which drops the summary
|
||||
silently along with the turns it replaced.
|
||||
|
||||
So the planner reads its own window. Bounded, not unlimited: capped so a long session costs
|
||||
no more than a short one. Never narrower than num_history_runs, though.
|
||||
"""
|
||||
if agent_compaction(agent) is None:
|
||||
return agent.num_history_runs
|
||||
return max(agent.num_history_runs or 0, _PLANNER_WINDOW_MAX_RUNS)
|
||||
|
||||
|
||||
def _history_for_run(
|
||||
agent: "Agent", session: AgentSession, run_response: Optional[RunOutput], skip_role: Optional[str]
|
||||
) -> Tuple[List[Message], Any, Optional[Counter[str]]]:
|
||||
"""The history a run works from, the fold in force for it, and the ids of the messages a run
|
||||
replays before the first fold - or None when it replays all of ``history``.
|
||||
|
||||
With a compact_at_tokens threshold, compaction owns the window: everything since the last fold
|
||||
is replayed, so the trigger, the fold and its metrics measure what the model is sent. Without
|
||||
one, num_history_runs and num_history_messages still bound the replay before the first fold,
|
||||
since nothing else would stop it growing. The ids are counted - a fork copies its source run's
|
||||
messages with the same ids, so one id can stand for more than one message.
|
||||
|
||||
Once a fold exists its anchor has to be in what is read, however old it is - the view replays
|
||||
the summary and everything from the anchor onward, and an anchor that is not found drops the
|
||||
summary along with the turns it replaced. What sits in front of it is already folded.
|
||||
"""
|
||||
|
||||
def fetch(last_n_runs: Optional[int], limit: Optional[int]) -> List[Message]:
|
||||
return session.get_messages(
|
||||
last_n_runs=last_n_runs,
|
||||
limit=limit,
|
||||
skip_roles=[skip_role] if skip_role else None,
|
||||
agent_id=agent.id if agent.team_id is not None else None,
|
||||
)
|
||||
|
||||
compaction = agent_compaction(agent)
|
||||
if compaction is None:
|
||||
return fetch(agent.num_history_runs, agent.num_history_messages), None, None
|
||||
|
||||
history = fetch(_compaction_history_runs(agent), None)
|
||||
record = _stored_compaction(agent, session, _compaction_as_of(run_response))
|
||||
anchor = record.first_kept_message_id if record is not None and record.summary else None
|
||||
if anchor is not None and not any(m.id == anchor for m in history):
|
||||
everything = fetch(None, None)
|
||||
start = next((i for i, m in enumerate(everything) if m.id == anchor), None)
|
||||
if start is not None:
|
||||
history = everything[start:]
|
||||
|
||||
if compaction.compact_at_tokens is not None:
|
||||
# A threshold is the limit, so compaction owns the window: everything since the last fold
|
||||
# is replayed. Capping it at num_history_runs meant a threshold above a few turns never
|
||||
# fired, older turns fell out unsummarized, and the fold measured a list never sent.
|
||||
return history, record, None
|
||||
# No threshold - compaction=True, or manual folds only - so nothing would stop the replay
|
||||
# growing until the provider rejects a request. Keep num_history_runs until the first fold.
|
||||
replay = fetch(agent.num_history_runs, agent.num_history_messages)
|
||||
return history, record, Counter(m.id for m in replay if m.id is not None)
|
||||
|
||||
|
||||
def _replayed_view(
|
||||
compaction: Any, history: List[Message], record: Any, replay_ids: Optional[Counter[str]]
|
||||
) -> List[Message]:
|
||||
"""What a run sends from ``history``.
|
||||
|
||||
Once a fold exists, the summary and everything from its anchor onward - the anchor has to be
|
||||
replayed, or the summary is dropped along with the turns it replaced. Before that, the replay
|
||||
window when there is one, else all of ``history``.
|
||||
"""
|
||||
if record is not None and record.summary and record.first_kept_message_id:
|
||||
if any(m.id == record.first_kept_message_id for m in history):
|
||||
return compaction.apply_record(history, record)
|
||||
if replay_ids is None:
|
||||
return history
|
||||
# The window is the newest runs, so match it from the end, each id as many times as the window
|
||||
# holds it. A plain membership test would also let in a fork's source run from outside the
|
||||
# window, since the fork's copies share its message ids, and send those turns twice.
|
||||
remaining = Counter(replay_ids)
|
||||
kept: List[Message] = []
|
||||
for message in reversed(history):
|
||||
if message.id is not None and remaining[message.id] > 0:
|
||||
remaining[message.id] -= 1
|
||||
kept.append(message)
|
||||
kept.reverse()
|
||||
return kept
|
||||
|
||||
|
||||
def _history_for_compaction(agent: "Agent", session: AgentSession) -> List[Message]:
|
||||
"""The history a manual compaction folds.
|
||||
|
||||
Deliberately not limited by ``num_history_runs`` or ``num_history_messages``: those govern how
|
||||
much history a RUN replays, and applying them here would let them silently cap what compaction
|
||||
can ever see - the same starvation the automatic path guards against.
|
||||
"""
|
||||
skip_role = agent.system_message_role if agent.system_message_role not in ["user", "assistant", "tool"] else None
|
||||
return session.get_messages(
|
||||
skip_roles=[skip_role] if skip_role else None,
|
||||
agent_id=agent.id if agent.team_id is not None else None,
|
||||
)
|
||||
|
||||
|
||||
def _compaction_result(new_record: Any, metrics: Optional[RunMetrics] = None) -> Any:
|
||||
from agno.compaction.types import CompactionResult, CompactionStatus
|
||||
|
||||
return CompactionResult(
|
||||
status=CompactionStatus.COMPACTED,
|
||||
message=(
|
||||
f"Compacted {new_record.messages_compacted} messages "
|
||||
f"({new_record.tokens_before} -> {new_record.tokens_after} tokens)."
|
||||
),
|
||||
record=new_record,
|
||||
metrics=metrics,
|
||||
)
|
||||
|
||||
|
||||
def _last_sent(agent: "Agent", session: AgentSession, compaction: Any) -> List[Message]:
|
||||
"""The history the model was last sent, which is what a fold's "before" size measures.
|
||||
|
||||
A manual fold folds everything the planner reads, but without a threshold the model was only
|
||||
sent the num_history_runs window - recording the planner's read would overstate what the fold
|
||||
reclaimed from the context.
|
||||
"""
|
||||
skip_role = agent.system_message_role if agent.system_message_role not in ["user", "assistant", "tool"] else None
|
||||
history, record, replay_ids = _history_for_run(agent, session, None, skip_role)
|
||||
return _replayed_view(compaction, history, record, replay_ids)
|
||||
|
||||
|
||||
def compact_now(agent: "Agent", session: AgentSession, history: List[Message]) -> Any:
|
||||
"""Fold ``history`` now, without waiting for the size trigger.
|
||||
|
||||
The explicit counterpart to the automatic path: same boundary, same guards, same archive.
|
||||
Only ``compact_at_tokens`` is bypassed - a caller asking to compact has supplied the
|
||||
judgement that threshold exists to make.
|
||||
|
||||
Every other guard still applies, and a decline is reported rather than raised. The ratio
|
||||
guard in particular is not a preference: a summary costs a few hundred tokens whatever it
|
||||
replaces, so folding a smaller span leaves the context BIGGER while spending a model call
|
||||
and discarding the prompt-cache prefix. Declining is the correct outcome, and the returned
|
||||
status says so in terms a UI can show.
|
||||
"""
|
||||
from agno.compaction.types import CompactionResult, CompactionStatus
|
||||
|
||||
compaction = agent_compaction(agent)
|
||||
if compaction is None:
|
||||
return CompactionResult(status=CompactionStatus.NOT_ENABLED, message="Compaction is not enabled on this agent.")
|
||||
if not history:
|
||||
return CompactionResult(status=CompactionStatus.NO_HISTORY, message="This session has no stored history yet.")
|
||||
|
||||
record = _stored_compaction(agent, session)
|
||||
boundary, status, reason = compaction.plan_with_reason(history, record)
|
||||
if boundary is None:
|
||||
return CompactionResult(status=status, message=reason)
|
||||
|
||||
log_info("Compacting conversation history")
|
||||
# No run to add the summarizer's usage to, so it is collected here and returned.
|
||||
run_metrics = RunMetrics()
|
||||
new_record = compaction.compact(
|
||||
history,
|
||||
session_id=session.session_id,
|
||||
db=agent.db,
|
||||
user_id=session.user_id,
|
||||
previous=record,
|
||||
run_metrics=run_metrics,
|
||||
sent_messages=_last_sent(agent, session, compaction),
|
||||
)
|
||||
metrics = run_metrics if run_metrics.details else None
|
||||
if new_record is None:
|
||||
return CompactionResult(
|
||||
status=CompactionStatus.SUMMARY_FAILED,
|
||||
message="The summarizer returned nothing, so history was left unchanged.",
|
||||
metrics=metrics,
|
||||
)
|
||||
_log_compaction(new_record)
|
||||
return _compaction_result(new_record, metrics)
|
||||
|
||||
|
||||
async def acompact_now(agent: "Agent", session: AgentSession, history: List[Message]) -> Any:
|
||||
from agno.compaction.types import CompactionResult, CompactionStatus
|
||||
|
||||
compaction = agent_compaction(agent)
|
||||
if compaction is None:
|
||||
return CompactionResult(status=CompactionStatus.NOT_ENABLED, message="Compaction is not enabled on this agent.")
|
||||
if not history:
|
||||
return CompactionResult(status=CompactionStatus.NO_HISTORY, message="This session has no stored history yet.")
|
||||
|
||||
record = _stored_compaction(agent, session)
|
||||
boundary, status, reason = compaction.plan_with_reason(history, record)
|
||||
if boundary is None:
|
||||
return CompactionResult(status=status, message=reason)
|
||||
|
||||
log_info("Compacting conversation history")
|
||||
# No run to add the summarizer's usage to, so it is collected here and returned.
|
||||
run_metrics = RunMetrics()
|
||||
new_record = await compaction.acompact(
|
||||
history,
|
||||
session_id=session.session_id,
|
||||
db=agent.db,
|
||||
user_id=session.user_id,
|
||||
previous=record,
|
||||
run_metrics=run_metrics,
|
||||
sent_messages=_last_sent(agent, session, compaction),
|
||||
)
|
||||
metrics = run_metrics if run_metrics.details else None
|
||||
if new_record is None:
|
||||
return CompactionResult(
|
||||
status=CompactionStatus.SUMMARY_FAILED,
|
||||
message="The summarizer returned nothing, so history was left unchanged.",
|
||||
metrics=metrics,
|
||||
)
|
||||
_log_compaction(new_record)
|
||||
return _compaction_result(new_record, metrics)
|
||||
|
||||
|
||||
def _add_to_session_metrics(agent: "Agent", session: AgentSession, metrics: RunMetrics) -> None:
|
||||
"""Add a manual compaction's summarizer usage to the session's metrics.
|
||||
|
||||
A fold inside a run reaches them through the run's metrics. A manual one has no run, so
|
||||
without this its model call would be missing from the session's totals and from the metrics
|
||||
AgentOS aggregates from them.
|
||||
"""
|
||||
from agno.agent._storage import get_session_metrics_internal
|
||||
|
||||
session_metrics = get_session_metrics_internal(agent, session)
|
||||
session_metrics.accumulate_from_run(metrics)
|
||||
if session.session_data is None:
|
||||
session.session_data = {}
|
||||
session.session_data["session_metrics"] = session_metrics.to_dict()
|
||||
|
||||
|
||||
def compact_session(agent: "Agent", session_id: Optional[str] = None, user_id: Optional[str] = None) -> Any:
|
||||
from agno.agent import _session
|
||||
from agno.compaction.types import CompactionResult, CompactionStatus
|
||||
|
||||
session = _session.get_session(agent, session_id=session_id, user_id=user_id)
|
||||
if session is None or not isinstance(session, AgentSession):
|
||||
return CompactionResult(status=CompactionStatus.NO_HISTORY, message="No such session.")
|
||||
result = compact_now(agent, session, _history_for_compaction(agent, session))
|
||||
if result.metrics is not None:
|
||||
_add_to_session_metrics(agent, session, result.metrics)
|
||||
try:
|
||||
_session.save_session(agent, session)
|
||||
except Exception as e: # noqa: BLE001 - the fold is already stored; only the totals are lost
|
||||
log_warning(f"Could not add the compaction's usage to the session metrics: {e}")
|
||||
return result
|
||||
|
||||
|
||||
async def acompact_session(agent: "Agent", session_id: Optional[str] = None, user_id: Optional[str] = None) -> Any:
|
||||
from agno.agent import _session
|
||||
from agno.compaction.types import CompactionResult, CompactionStatus
|
||||
|
||||
session = await _session.aget_session(agent, session_id=session_id, user_id=user_id)
|
||||
if session is None or not isinstance(session, AgentSession):
|
||||
return CompactionResult(status=CompactionStatus.NO_HISTORY, message="No such session.")
|
||||
result = await acompact_now(agent, session, _history_for_compaction(agent, session))
|
||||
if result.metrics is not None:
|
||||
_add_to_session_metrics(agent, session, result.metrics)
|
||||
try:
|
||||
await _session.asave_session(agent, session)
|
||||
except Exception as e: # noqa: BLE001 - the fold is already stored; only the totals are lost
|
||||
log_warning(f"Could not add the compaction's usage to the session metrics: {e}")
|
||||
return result
|
||||
|
||||
|
||||
def _plan_overflow_fold(agent: "Agent", run_messages: Any) -> Optional[Tuple[Any, Any, List[Message]]]:
|
||||
"""Where to cut after a context-window rejection: the Compaction, the variant that folds, and
|
||||
the message list the model call holds. None when there is nothing to do or no safe cut.
|
||||
|
||||
Shared by the sync and async recovery, which differ only in how the summarizer is called.
|
||||
"""
|
||||
from dataclasses import replace
|
||||
|
||||
from agno.compaction._cut import leading_system_count
|
||||
from agno.compaction._tokens import estimate_tokens
|
||||
|
||||
compaction = agent_compaction(agent)
|
||||
if compaction is None or not getattr(compaction, "on_context_overflow", False):
|
||||
return None
|
||||
|
||||
messages = getattr(run_messages, "messages", None)
|
||||
if not messages:
|
||||
return None
|
||||
|
||||
lead = leading_system_count(messages)
|
||||
# Keep the configured tail when it works. Only when folding in front of it reclaims too
|
||||
# little to be worth retrying - an oversized turn sitting INSIDE the tail, which no cut in
|
||||
# front of it can reach - is the tail given up, one run at a time. The request has already
|
||||
# been rejected, so a smaller tail beats no answer, but the setting is still the default.
|
||||
folder, chosen_keep = None, compaction.uncompacted_runs
|
||||
# A token tail is sized, not counted, so there is no run count to give up - only a run-count
|
||||
# tail is shrunk. Varying uncompacted_runs alongside uncompacted_tokens is not a valid config.
|
||||
if compaction.uncompacted_tokens is not None:
|
||||
tails: Sequence[Optional[int]] = [compaction.uncompacted_runs]
|
||||
else:
|
||||
tails = range(compaction.uncompacted_runs or 1, 0, -1)
|
||||
for keep in tails:
|
||||
if keep == compaction.uncompacted_runs:
|
||||
candidate = compaction
|
||||
else:
|
||||
candidate = replace(compaction, uncompacted_runs=keep, stats=compaction.stats)
|
||||
boundary = candidate.boundary_for(messages, min_index=lead, overhead_tokens=estimate_tokens(messages[:lead]))
|
||||
if boundary is None or boundary <= lead:
|
||||
continue
|
||||
folder, chosen_keep = candidate, keep
|
||||
# Stop as soon as the fold is large enough to be worth a summarizer call, rather than
|
||||
# shrinking the tail further than the rejection requires.
|
||||
if estimate_tokens(messages[lead:boundary]) >= estimate_tokens(messages[boundary:]):
|
||||
break
|
||||
if folder is not None and chosen_keep != compaction.uncompacted_runs:
|
||||
log_info(
|
||||
f"Compaction: keeping {chosen_keep} run(s) instead of {compaction.uncompacted_runs} - "
|
||||
f"the request was rejected as too long, and the configured tail leaves too little "
|
||||
f"in front of it to fold."
|
||||
)
|
||||
if folder is None:
|
||||
log_warning(
|
||||
"Compaction: the request exceeded the model's context window and there is no safe "
|
||||
"cut left to make - the most recent turn alone is too large to send. Shorten what "
|
||||
"it produces, or use a model with a larger context window."
|
||||
)
|
||||
return None
|
||||
return compaction, folder, messages
|
||||
|
||||
|
||||
def _finish_overflow_fold(
|
||||
messages: List[Message],
|
||||
record: Any,
|
||||
compacted: List[Message],
|
||||
before: int,
|
||||
after: int,
|
||||
run_response: Optional[RunOutput],
|
||||
) -> bool:
|
||||
"""Apply a fold made after a rejection to the list the model call holds. True if it shrank."""
|
||||
if after >= before:
|
||||
# A summary has a floor cost, so a fold that reclaims nothing leaves the request no
|
||||
# more sendable than it was. Retrying an identical payload just fails twice.
|
||||
log_warning(
|
||||
f"Compaction: folding after a context-window rejection did not shrink the request "
|
||||
f"({before} -> {after} tokens), so it is not worth retrying."
|
||||
)
|
||||
return False
|
||||
|
||||
# Mutate the list in place rather than rebinding it. The caller has already passed this
|
||||
# exact list object into the model call's kwargs, so a new list would leave the retry
|
||||
# sending the payload that was just rejected.
|
||||
messages[:] = compacted
|
||||
if run_response is not None:
|
||||
run_response.compaction = record
|
||||
log_info(
|
||||
f"Compaction: request exceeded the context window, folded {record.messages_compacted} "
|
||||
f"messages ({before} -> {after} tokens) and retrying once."
|
||||
)
|
||||
return True
|
||||
|
||||
|
||||
def _overflow_fold_kwargs(
|
||||
agent: "Agent", session: AgentSession, run_response: Optional[RunOutput], before: int, messages: List[Message]
|
||||
) -> Dict[str, Any]:
|
||||
from agno.compaction._cut import leading_system_count
|
||||
from agno.compaction._tokens import estimate_tokens
|
||||
|
||||
return dict(
|
||||
session_id=session.session_id,
|
||||
db=agent.db,
|
||||
user_id=session.user_id,
|
||||
previous=_stored_compaction(agent, session),
|
||||
run_id=run_response.run_id if run_response is not None else None,
|
||||
run_metrics=run_response.metrics if run_response is not None else None,
|
||||
tokens_before=before,
|
||||
# The leading system messages are part of the list being folded; counting them keeps the
|
||||
# cut the same as the one the overflow planner chose.
|
||||
overhead_tokens=estimate_tokens(messages[: leading_system_count(messages)]),
|
||||
)
|
||||
|
||||
|
||||
def _recompact_after_overflow(
|
||||
agent: "Agent",
|
||||
session: AgentSession,
|
||||
run_messages: Any,
|
||||
run_response: Optional[RunOutput] = None,
|
||||
tools: Optional[List[Any]] = None,
|
||||
) -> bool:
|
||||
"""Fold harder after the provider rejected a request as too long. True if the payload shrank.
|
||||
|
||||
Nobody can know a model's context window ahead of time - no provider exposes it, and the
|
||||
same model id has different limits across deployments - so a threshold set in advance is
|
||||
always a guess. The rejection is the one authoritative signal that the guess was wrong, and
|
||||
this is the only path that can act on it.
|
||||
|
||||
Folds against the messages actually sent, so it reaches spans the run-start pass could not:
|
||||
the current turn's input, and anything a tool loop appended since. The pair-safe boundary
|
||||
still applies - an unsendable payload is no improvement on a too-long one - but the fold
|
||||
ratio does not: the request has already failed, so a fold that merely helps beats the run
|
||||
dying.
|
||||
"""
|
||||
from dataclasses import replace
|
||||
|
||||
plan = _plan_overflow_fold(agent, run_messages)
|
||||
if plan is None:
|
||||
return False
|
||||
compaction, folder, messages = plan
|
||||
before = _request_tokens(agent, messages, tools)
|
||||
# min_fold_ratio is the run-start question - is this fold worth paying for. Here the request
|
||||
# has already been rejected, so any fold that shrinks it is worth making.
|
||||
record = replace(folder, min_fold_ratio=0, stats=compaction.stats).compact(
|
||||
messages, **_overflow_fold_kwargs(agent, session, run_response, before, messages)
|
||||
)
|
||||
if record is None:
|
||||
return False
|
||||
compacted = compaction.apply_record(messages, record)
|
||||
after = _request_tokens(agent, compacted, tools)
|
||||
return _finish_overflow_fold(messages, record, compacted, before, after, run_response)
|
||||
|
||||
|
||||
async def _arecompact_after_overflow(
|
||||
agent: "Agent",
|
||||
session: AgentSession,
|
||||
run_messages: Any,
|
||||
run_response: Optional[RunOutput] = None,
|
||||
tools: Optional[List[Any]] = None,
|
||||
) -> bool:
|
||||
"""Async variant of :func:`_recompact_after_overflow`. The summarizer is awaited, so a recovery
|
||||
on the async path does not block the event loop for the length of a model call."""
|
||||
from dataclasses import replace
|
||||
|
||||
plan = _plan_overflow_fold(agent, run_messages)
|
||||
if plan is None:
|
||||
return False
|
||||
compaction, folder, messages = plan
|
||||
before = await _arequest_tokens(agent, messages, tools)
|
||||
record = await replace(folder, min_fold_ratio=0, stats=compaction.stats).acompact(
|
||||
messages, **_overflow_fold_kwargs(agent, session, run_response, before, messages)
|
||||
)
|
||||
if record is None:
|
||||
return False
|
||||
compacted = compaction.apply_record(messages, record)
|
||||
after = await _arequest_tokens(agent, compacted, tools)
|
||||
return _finish_overflow_fold(messages, record, compacted, before, after, run_response)
|
||||
|
||||
|
||||
def apply_compaction(
|
||||
agent: "Agent",
|
||||
session: AgentSession,
|
||||
history: List[Message],
|
||||
record: Any,
|
||||
run_response: Optional[RunOutput] = None,
|
||||
events: Optional[List[Any]] = None,
|
||||
context_prefix: Optional[List[Message]] = None,
|
||||
tools: Optional[List[Any]] = None,
|
||||
replay_ids: Optional[Counter[str]] = None,
|
||||
) -> List[Message]:
|
||||
"""Replace the head of ``history`` with a summary once it grows too long.
|
||||
|
||||
Returns the list to send to the model. ``history`` itself is not mutated,
|
||||
and what the session persists is never touched: compaction shortens the
|
||||
request, not the record.
|
||||
"""
|
||||
compaction = agent_compaction(agent)
|
||||
if compaction is None or not history:
|
||||
return history
|
||||
|
||||
# A stored compaction is replayed rather than recomputed, so the summary is
|
||||
# paid for once and the prompt prefix stays stable between runs. The trigger
|
||||
# measures this - what is actually sent - not the planner's wider read.
|
||||
in_context = _replayed_view(compaction, history, record, replay_ids)
|
||||
|
||||
prefix = context_prefix or []
|
||||
# Size is the only automatic trigger. Without a threshold nothing would read a count, so none
|
||||
# is taken - whatever does the counting, and the model's own count is a network call.
|
||||
if compaction.compact_at_tokens is None:
|
||||
return in_context
|
||||
inputs = _compaction_inputs(agent, prefix + in_context, tools)
|
||||
if not compaction.should_compact(in_context, **inputs):
|
||||
return in_context
|
||||
|
||||
# Announce only once the guards have passed. should_compact cannot see the
|
||||
# pair-safe boundary or the size floor, so announcing on it alone reports
|
||||
# compactions that then never happen.
|
||||
# The system prompt and tools share the threshold with whatever the fold keeps.
|
||||
from agno.compaction._tokens import estimate_tokens
|
||||
|
||||
overhead = estimate_tokens(prefix, tools) if (prefix or tools) else 0
|
||||
if compaction.plan(history, record, overhead) is None:
|
||||
return in_context
|
||||
|
||||
log_info("Auto-compacting conversation history")
|
||||
if events is not None:
|
||||
events.extend(_compaction_events(run_response, None, started=True))
|
||||
|
||||
# Compact against the FULL history, so the boundary the record stores is an
|
||||
# absolute index into it. A boundary measured on the already-compacted list
|
||||
# would advance by only one message per run, so the context would never
|
||||
# actually shrink and every later run would compact again.
|
||||
new_record = compaction.compact(
|
||||
history,
|
||||
session_id=session.session_id,
|
||||
db=agent.db,
|
||||
user_id=session.user_id,
|
||||
previous=record,
|
||||
run_metrics=run_response.metrics if run_response is not None else None,
|
||||
tokens_before=inputs["context_tokens"],
|
||||
run_id=run_response.run_id if run_response is not None else None,
|
||||
context_prefix=prefix,
|
||||
overhead_tokens=overhead,
|
||||
# Once a fold exists the model is sent the summary and the anchor onward, not every turn the
|
||||
# planner reads, so that is what "before" measures.
|
||||
sent_messages=in_context,
|
||||
)
|
||||
if new_record is None:
|
||||
return in_context
|
||||
|
||||
compacted = compaction.apply_record(history, new_record)
|
||||
# Measure before storing, so the persisted record carries the real sizes.
|
||||
_log_compaction(new_record)
|
||||
# Surface it on the run, so `run.compaction` reports what happened here.
|
||||
if run_response is not None:
|
||||
run_response.compaction = new_record
|
||||
if events is not None:
|
||||
events.extend(_compaction_events(run_response, new_record))
|
||||
return compacted
|
||||
|
||||
|
||||
async def aapply_compaction(
|
||||
agent: "Agent",
|
||||
session: AgentSession,
|
||||
history: List[Message],
|
||||
record: Any,
|
||||
run_response: Optional[RunOutput] = None,
|
||||
events: Optional[List[Any]] = None,
|
||||
context_prefix: Optional[List[Message]] = None,
|
||||
tools: Optional[List[Any]] = None,
|
||||
replay_ids: Optional[Counter[str]] = None,
|
||||
) -> List[Message]:
|
||||
compaction = agent_compaction(agent)
|
||||
if compaction is None or not history:
|
||||
return history
|
||||
|
||||
in_context = _replayed_view(compaction, history, record, replay_ids)
|
||||
|
||||
prefix = context_prefix or []
|
||||
# Size is the only automatic trigger. Without a threshold nothing would read a count, so none
|
||||
# is taken - whatever does the counting, and the model's own count is a network call.
|
||||
if compaction.compact_at_tokens is None:
|
||||
return in_context
|
||||
inputs = await _acompaction_inputs(agent, prefix + in_context, tools)
|
||||
if not compaction.should_compact(in_context, **inputs):
|
||||
return in_context
|
||||
|
||||
# Announce only once the guards have passed. should_compact cannot see the
|
||||
# pair-safe boundary or the size floor, so announcing on it alone reports
|
||||
# compactions that then never happen.
|
||||
# The system prompt and tools share the threshold with whatever the fold keeps.
|
||||
from agno.compaction._tokens import estimate_tokens
|
||||
|
||||
overhead = estimate_tokens(prefix, tools) if (prefix or tools) else 0
|
||||
if compaction.plan(history, record, overhead) is None:
|
||||
return in_context
|
||||
|
||||
log_info("Auto-compacting conversation history")
|
||||
if events is not None:
|
||||
events.extend(_compaction_events(run_response, None, started=True))
|
||||
|
||||
# Compact against the FULL history so the stored boundary is absolute -
|
||||
# see the sync path for why a relative boundary never shrinks the context.
|
||||
new_record = await compaction.acompact(
|
||||
history,
|
||||
session_id=session.session_id,
|
||||
db=agent.db,
|
||||
user_id=session.user_id,
|
||||
previous=record,
|
||||
run_metrics=run_response.metrics if run_response is not None else None,
|
||||
tokens_before=inputs["context_tokens"],
|
||||
run_id=run_response.run_id if run_response is not None else None,
|
||||
context_prefix=prefix,
|
||||
overhead_tokens=overhead,
|
||||
# Once a fold exists the model is sent the summary and the anchor onward, not every turn the
|
||||
# planner reads, so that is what "before" measures.
|
||||
sent_messages=in_context,
|
||||
)
|
||||
if new_record is None:
|
||||
return in_context
|
||||
|
||||
compacted = compaction.apply_record(history, new_record)
|
||||
# Measure before storing, so the persisted record carries the real sizes.
|
||||
_log_compaction(new_record)
|
||||
# Surface it on the run, so `run.compaction` reports what happened here.
|
||||
if run_response is not None:
|
||||
run_response.compaction = new_record
|
||||
if events is not None:
|
||||
events.extend(_compaction_events(run_response, new_record))
|
||||
return compacted
|
||||
|
||||
|
||||
def _get_resolved_knowledge(agent: "Agent", run_context: Optional[RunContext] = None) -> Any:
|
||||
"""Get the resolved knowledge, preferring run_context over agent.knowledge."""
|
||||
from agno.utils.callables import get_resolved_knowledge
|
||||
@@ -1193,12 +1970,7 @@ def get_run_messages(
|
||||
agent.system_message_role if agent.system_message_role not in ["user", "assistant", "tool"] else None
|
||||
)
|
||||
|
||||
history: List[Message] = session.get_messages(
|
||||
last_n_runs=agent.num_history_runs,
|
||||
limit=agent.num_history_messages,
|
||||
skip_roles=[skip_role] if skip_role else None,
|
||||
agent_id=agent.id if agent.team_id is not None else None,
|
||||
)
|
||||
history, stored_record, replay_ids = _history_for_run(agent, session, run_response, skip_role)
|
||||
|
||||
if len(history) > 0:
|
||||
history_copy = [copy_history_message(msg) for msg in history]
|
||||
@@ -1207,6 +1979,20 @@ def get_run_messages(
|
||||
if agent.max_tool_calls_from_history is not None:
|
||||
filter_tool_calls(history_copy, agent.max_tool_calls_from_history)
|
||||
|
||||
# Replace the older part of the history with a summary once it has
|
||||
# grown past the configured threshold.
|
||||
history_copy = apply_compaction(
|
||||
agent,
|
||||
session,
|
||||
history_copy,
|
||||
stored_record,
|
||||
run_response,
|
||||
events=run_messages.events,
|
||||
context_prefix=run_messages.messages,
|
||||
tools=tools,
|
||||
replay_ids=replay_ids,
|
||||
)
|
||||
|
||||
log_debug(f"Adding {len(history_copy)} messages from history")
|
||||
|
||||
run_messages.messages += history_copy
|
||||
@@ -1399,12 +2185,7 @@ async def aget_run_messages(
|
||||
agent.system_message_role if agent.system_message_role not in ["user", "assistant", "tool"] else None
|
||||
)
|
||||
|
||||
history: List[Message] = session.get_messages(
|
||||
last_n_runs=agent.num_history_runs,
|
||||
limit=agent.num_history_messages,
|
||||
skip_roles=[skip_role] if skip_role else None,
|
||||
agent_id=agent.id if agent.team_id is not None else None,
|
||||
)
|
||||
history, stored_record, replay_ids = _history_for_run(agent, session, run_response, skip_role)
|
||||
|
||||
if len(history) > 0:
|
||||
history_copy = [copy_history_message(msg) for msg in history]
|
||||
@@ -1413,6 +2194,20 @@ async def aget_run_messages(
|
||||
if agent.max_tool_calls_from_history is not None:
|
||||
filter_tool_calls(history_copy, agent.max_tool_calls_from_history)
|
||||
|
||||
# Replace the older part of the history with a summary once it has
|
||||
# grown past the configured threshold.
|
||||
history_copy = await aapply_compaction(
|
||||
agent,
|
||||
session,
|
||||
history_copy,
|
||||
stored_record,
|
||||
run_response,
|
||||
events=run_messages.events,
|
||||
context_prefix=run_messages.messages,
|
||||
tools=tools,
|
||||
replay_ids=replay_ids,
|
||||
)
|
||||
|
||||
log_debug(f"Adding {len(history_copy)} messages from history")
|
||||
|
||||
run_messages.messages += history_copy
|
||||
@@ -1572,11 +2367,8 @@ def _build_continue_run_messages(
|
||||
agent.system_message_role if agent.system_message_role not in ["user", "assistant", "tool"] else None
|
||||
)
|
||||
|
||||
history: List[Message] = continue_history_session(session, run_response).get_messages(
|
||||
last_n_runs=agent.num_history_runs,
|
||||
limit=agent.num_history_messages,
|
||||
skip_roles=[skip_role] if skip_role else None,
|
||||
agent_id=agent.id if agent.team_id is not None else None,
|
||||
history, stored_record, replay_ids = _history_for_run(
|
||||
agent, continue_history_session(session, run_response), run_response, skip_role
|
||||
)
|
||||
|
||||
if len(history) > 0:
|
||||
@@ -1586,6 +2378,11 @@ def _build_continue_run_messages(
|
||||
if agent.max_tool_calls_from_history is not None:
|
||||
filter_tool_calls(history_copy, agent.max_tool_calls_from_history)
|
||||
|
||||
# Replay the stored fold, as a fresh run does. No new fold is planned here: the run
|
||||
# checked this same history before it paused, and its own messages are never folded.
|
||||
if agent_compaction(agent) is not None:
|
||||
history_copy = _replayed_view(agent.compaction, history_copy, stored_record, replay_ids)
|
||||
|
||||
log_debug(f"Adding {len(history_copy)} messages from history")
|
||||
run_messages.messages += history_copy
|
||||
|
||||
|
||||
@@ -1037,6 +1037,8 @@ def handle_model_response_stream(
|
||||
session_state: Optional[Dict[str, Any]] = None,
|
||||
run_context: Optional[RunContext] = None,
|
||||
) -> Iterator[RunOutputEvent]:
|
||||
from agno.agent._messages import _recompact_after_overflow
|
||||
|
||||
agent.model = cast(Model, agent.model)
|
||||
|
||||
reasoning_state = {
|
||||
@@ -1059,6 +1061,7 @@ def handle_model_response_stream(
|
||||
for model_response_event in call_model_stream_with_fallback(
|
||||
agent.model,
|
||||
agent.fallback_config,
|
||||
recover_from_overflow=lambda: _recompact_after_overflow(agent, session, run_messages, run_response, tools),
|
||||
messages=run_messages.messages,
|
||||
response_format=response_format,
|
||||
tools=tools,
|
||||
@@ -1198,6 +1201,8 @@ async def ahandle_model_response_stream(
|
||||
session_state: Optional[Dict[str, Any]] = None,
|
||||
run_context: Optional[RunContext] = None,
|
||||
) -> AsyncIterator[RunOutputEvent]:
|
||||
from agno.agent._messages import _arecompact_after_overflow
|
||||
|
||||
agent.model = cast(Model, agent.model)
|
||||
|
||||
reasoning_state = {
|
||||
@@ -1220,6 +1225,7 @@ async def ahandle_model_response_stream(
|
||||
model_response_stream = acall_model_stream_with_fallback(
|
||||
agent.model,
|
||||
agent.fallback_config,
|
||||
recover_from_overflow=lambda: _arecompact_after_overflow(agent, session, run_messages, run_response, tools),
|
||||
messages=run_messages.messages,
|
||||
response_format=response_format,
|
||||
tools=tools,
|
||||
|
||||
@@ -401,7 +401,7 @@ def _run(
|
||||
"""
|
||||
from agno.agent._hooks import execute_post_hooks, execute_pre_hooks
|
||||
from agno.agent._init import disconnect_connectable_tools
|
||||
from agno.agent._messages import get_run_messages
|
||||
from agno.agent._messages import _recompact_after_overflow, get_run_messages
|
||||
from agno.agent._response import (
|
||||
convert_response_to_structured_format,
|
||||
generate_followups,
|
||||
@@ -554,6 +554,9 @@ def _run(
|
||||
model_response: ModelResponse = call_model_with_fallback(
|
||||
agent.model,
|
||||
agent.fallback_config,
|
||||
recover_from_overflow=lambda: _recompact_after_overflow(
|
||||
agent, agent_session, run_messages, run_response, _tools
|
||||
),
|
||||
messages=run_messages.messages,
|
||||
tools=_tools,
|
||||
tool_choice=agent.tool_choice,
|
||||
@@ -926,6 +929,17 @@ def _run_stream(
|
||||
if len(run_messages.messages) == 0:
|
||||
log_error("No messages to be sent to the model.")
|
||||
|
||||
# Events raised while assembling messages (compaction). Assembly
|
||||
# is not a generator, so they are collected there and emitted here.
|
||||
if stream_events:
|
||||
for assembly_event in run_messages.events:
|
||||
yield handle_event( # type: ignore
|
||||
assembly_event,
|
||||
run_response,
|
||||
events_to_skip=agent.events_to_skip, # type: ignore
|
||||
store_events=agent.store_events,
|
||||
)
|
||||
|
||||
# 7. Start memory creation in background thread
|
||||
from agno.agent import _managers
|
||||
|
||||
@@ -1533,7 +1547,7 @@ async def _arun(
|
||||
"""
|
||||
from agno.agent._hooks import aexecute_post_hooks, aexecute_pre_hooks
|
||||
from agno.agent._init import disconnect_connectable_tools, disconnect_mcp_tools
|
||||
from agno.agent._messages import aget_run_messages
|
||||
from agno.agent._messages import _arecompact_after_overflow, aget_run_messages
|
||||
from agno.agent._response import (
|
||||
agenerate_followups,
|
||||
agenerate_response_with_output_model,
|
||||
@@ -1656,7 +1670,6 @@ async def _arun(
|
||||
)
|
||||
if len(run_messages.messages) == 0:
|
||||
log_error("No messages to be sent to the model.")
|
||||
|
||||
# 7. Start memory creation as a background task (runs concurrently with the main execution)
|
||||
from agno.agent import _managers
|
||||
|
||||
@@ -1692,6 +1705,9 @@ async def _arun(
|
||||
model_response: ModelResponse = await acall_model_with_fallback(
|
||||
agent.model,
|
||||
agent.fallback_config,
|
||||
recover_from_overflow=lambda: _arecompact_after_overflow(
|
||||
agent, agent_session, run_messages, run_response, _tools
|
||||
),
|
||||
messages=run_messages.messages,
|
||||
tools=_tools,
|
||||
tool_choice=agent.tool_choice,
|
||||
@@ -2419,6 +2435,17 @@ async def _arun_stream(
|
||||
if len(run_messages.messages) == 0:
|
||||
log_error("No messages to be sent to the model.")
|
||||
|
||||
# Events raised while assembling messages (compaction). Assembly
|
||||
# is not a generator, so they are collected there and emitted here.
|
||||
if stream_events:
|
||||
for assembly_event in run_messages.events:
|
||||
yield handle_event( # type: ignore
|
||||
assembly_event,
|
||||
run_response,
|
||||
events_to_skip=agent.events_to_skip, # type: ignore
|
||||
store_events=agent.store_events,
|
||||
)
|
||||
|
||||
# 7. Start memory creation as a background task (runs concurrently with the main execution)
|
||||
from agno.agent import _managers
|
||||
|
||||
@@ -3795,6 +3822,7 @@ def _continue_run(
|
||||
# Register run for cancellation tracking
|
||||
from agno.agent._hooks import execute_post_hooks
|
||||
from agno.agent._init import disconnect_connectable_tools
|
||||
from agno.agent._messages import _recompact_after_overflow
|
||||
from agno.agent._response import (
|
||||
convert_response_to_structured_format,
|
||||
generate_followups,
|
||||
@@ -3824,6 +3852,9 @@ def _continue_run(
|
||||
model_response: ModelResponse = call_model_with_fallback(
|
||||
agent.model,
|
||||
agent.fallback_config,
|
||||
recover_from_overflow=lambda: _recompact_after_overflow(
|
||||
agent, session, run_messages, run_response, tools
|
||||
),
|
||||
messages=run_messages.messages,
|
||||
response_format=response_format,
|
||||
tools=tools,
|
||||
@@ -4820,7 +4851,7 @@ async def _acontinue_run(
|
||||
"""
|
||||
from agno.agent._hooks import aexecute_post_hooks
|
||||
from agno.agent._init import disconnect_connectable_tools, disconnect_mcp_tools
|
||||
from agno.agent._messages import aget_continue_run_messages
|
||||
from agno.agent._messages import _arecompact_after_overflow, aget_continue_run_messages
|
||||
from agno.agent._response import (
|
||||
agenerate_followups,
|
||||
agenerate_response_with_output_model,
|
||||
@@ -5063,6 +5094,9 @@ async def _acontinue_run(
|
||||
model_response: ModelResponse = await acall_model_with_fallback(
|
||||
agent.model,
|
||||
agent.fallback_config,
|
||||
recover_from_overflow=lambda: _arecompact_after_overflow(
|
||||
agent, agent_session, run_messages, run_response, _tools
|
||||
),
|
||||
messages=run_messages.messages,
|
||||
response_format=response_format,
|
||||
tools=_tools,
|
||||
|
||||
@@ -1203,6 +1203,10 @@ def to_dict(agent: Agent) -> Dict[str, Any]:
|
||||
# TODO: implement compression manager serialization
|
||||
# if agent.compression_manager is not None:
|
||||
# config["compression_manager"] = agent.compression_manager.to_dict()
|
||||
# TODO: implement compaction serialization - until then an agent saved with compaction
|
||||
# (True or a Compaction) loads back with it off.
|
||||
# if agent.compaction is not None:
|
||||
# config["compaction"] = agent.compaction if isinstance(agent.compaction, bool) else agent.compaction.to_dict()
|
||||
|
||||
# --- Callable factory settings ---
|
||||
if not agent.cache_callables:
|
||||
@@ -1423,6 +1427,12 @@ def from_dict(
|
||||
# from agno.compression.manager import CompressionManager
|
||||
# config["compression_manager"] = CompressionManager.from_dict(config["compression_manager"])
|
||||
|
||||
# --- Handle Compaction reconstruction ---
|
||||
# TODO: implement compaction deserialization
|
||||
# if "compaction" in config and isinstance(config["compaction"], dict):
|
||||
# from agno.compaction import Compaction
|
||||
# config["compaction"] = Compaction.from_dict(config["compaction"])
|
||||
|
||||
# --- Handle Learning reconstruction ---
|
||||
# A named machine is stored as a reference and resolved from the registry;
|
||||
# any other dict is an inline machine config and is rebuilt here.
|
||||
|
||||
@@ -236,6 +236,16 @@ def get_tools(
|
||||
_raise_if_async_tools_in_list(resolved_tools)
|
||||
agent_tools.extend(resolved_tools)
|
||||
|
||||
# Read-only search over history this session compacted away. Scoped to
|
||||
# this session's archive namespace, so one session can never read another's.
|
||||
from agno.agent._messages import agent_compaction
|
||||
|
||||
compaction = agent_compaction(agent)
|
||||
if compaction is not None and compaction.search_compacted_messages:
|
||||
archive_tools = compaction.tools_for(session.session_id, agent.db)
|
||||
if archive_tools:
|
||||
agent_tools.extend(archive_tools)
|
||||
|
||||
_append_filesystem_tools(agent, agent_tools)
|
||||
|
||||
# Add tools for accessing memory
|
||||
@@ -373,6 +383,16 @@ async def aget_tools(
|
||||
# Add the tool (MCP tools that passed checks, or any non-MCP tool)
|
||||
agent_tools.append(tool)
|
||||
|
||||
# Read-only search over history this session compacted away. Scoped to
|
||||
# this session's archive namespace, so one session can never read another's.
|
||||
from agno.agent._messages import agent_compaction
|
||||
|
||||
compaction = agent_compaction(agent)
|
||||
if compaction is not None and compaction.search_compacted_messages:
|
||||
archive_tools = compaction.tools_for(session.session_id, agent.db)
|
||||
if archive_tools:
|
||||
agent_tools.extend(archive_tools)
|
||||
|
||||
_append_filesystem_tools(agent, agent_tools)
|
||||
|
||||
# Add tools for accessing memory
|
||||
|
||||
@@ -33,6 +33,7 @@ from agno.agent import (
|
||||
_tools,
|
||||
_utils,
|
||||
)
|
||||
from agno.compaction.compaction import Compaction
|
||||
from agno.agent.followup import FollowupConfig, resolve_followup_settings
|
||||
from agno.compression.manager import CompressionManager
|
||||
from agno.db.base import AsyncBaseDb, BaseDb, ComponentType, UserMemory
|
||||
@@ -42,6 +43,7 @@ from agno.guardrails import BaseGuardrail
|
||||
from agno.knowledge.protocol import KnowledgeProtocol
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from agno.compaction.types import CompactionResult
|
||||
from agno.fs import FileSystem
|
||||
from agno.learn.machine import LearningMachine
|
||||
from agno.tools.component import ComponentTool
|
||||
@@ -363,6 +365,13 @@ class Agent:
|
||||
# Metadata stored with this agent
|
||||
metadata: Optional[Dict[str, Any]] = None
|
||||
|
||||
# --- Compaction ---
|
||||
# Keep a long session inside the context window: when the conversation
|
||||
# crosses a threshold, older messages are archived and replaced by a
|
||||
# summary. True uses the defaults; a Compaction sets the thresholds, what
|
||||
# is kept verbatim, and whether the agent can search the archive.
|
||||
compaction: Optional[Union[bool, "Compaction"]] = None
|
||||
|
||||
# --- Context Compression ---
|
||||
# If True, compress tool call results to save context
|
||||
compress_tool_results: bool = False
|
||||
@@ -425,6 +434,7 @@ class Agent:
|
||||
enable_session_summaries: bool = False,
|
||||
add_session_summary_to_context: Optional[bool] = None,
|
||||
session_summary_manager: Optional[SessionSummaryManager] = None,
|
||||
compaction: Optional[Union[bool, Compaction]] = None,
|
||||
compress_tool_results: bool = False,
|
||||
compression_manager: Optional[CompressionManager] = None,
|
||||
offload_tool_results: Optional[Union[bool, "ResultStore"]] = None,
|
||||
@@ -554,6 +564,9 @@ class Agent:
|
||||
|
||||
self.add_session_summary_to_context = add_session_summary_to_context
|
||||
|
||||
# Compaction settings
|
||||
self.compaction = compaction
|
||||
|
||||
# Context compression settings
|
||||
self.compress_tool_results = compress_tool_results
|
||||
self.compression_manager = compression_manager
|
||||
@@ -720,6 +733,7 @@ class Agent:
|
||||
self._callable_knowledge_cache: Dict[str, Any] = {}
|
||||
|
||||
_init.get_models(self)
|
||||
_init.disable_compaction_without_records(self)
|
||||
|
||||
# ---------------------------------------------------------------
|
||||
# Properties
|
||||
@@ -1101,6 +1115,23 @@ class Agent:
|
||||
async def asave_session(self, session: Union[AgentSession, TeamSession, WorkflowSession]) -> None:
|
||||
return await _session.asave_session(self, session=session)
|
||||
|
||||
def compact(self, session_id: Optional[str] = None, user_id: Optional[str] = None) -> "CompactionResult":
|
||||
"""Compact this session's history now, without waiting for the size trigger.
|
||||
|
||||
For folding at a moment you choose - the end of a topic, before a long task - rather
|
||||
than when the context happens to cross a threshold.
|
||||
|
||||
Returns a CompactionResult carrying a status and a human-readable message. A fold can
|
||||
legitimately decline: if the span is too small to pay for the summary replacing it,
|
||||
compacting would leave the context bigger, so it is reported rather than performed.
|
||||
Check ``result.compacted``, or show ``result.message``. ``result.metrics`` carries the
|
||||
summarizer's token usage under ``compaction_model``, or None when no summary was made.
|
||||
"""
|
||||
return _messages.compact_session(self, session_id=session_id, user_id=user_id)
|
||||
|
||||
async def acompact(self, session_id: Optional[str] = None, user_id: Optional[str] = None) -> "CompactionResult":
|
||||
return await _messages.acompact_session(self, session_id=session_id, user_id=user_id)
|
||||
|
||||
def rename(self, name: str, session_id: Optional[str] = None) -> None:
|
||||
return _session.rename(self, name=name, session_id=session_id)
|
||||
|
||||
|
||||
@@ -0,0 +1,14 @@
|
||||
"""Conversation compaction: replace old history with a summary over an archive."""
|
||||
|
||||
from agno.compaction.archive import CompactionArchive
|
||||
from agno.compaction.compaction import Compaction
|
||||
from agno.compaction.types import CompactionRecord, CompactionResult, CompactionStats, CompactionStatus
|
||||
|
||||
__all__ = [
|
||||
"Compaction",
|
||||
"CompactionArchive",
|
||||
"CompactionRecord",
|
||||
"CompactionResult",
|
||||
"CompactionStats",
|
||||
"CompactionStatus",
|
||||
]
|
||||
@@ -0,0 +1,156 @@
|
||||
"""Boundary selection: pair-safe, anchor-durable cut points.
|
||||
|
||||
All functions here are pure over an in-memory message list. Callers map chosen indices onto
|
||||
stored-run coordinates.
|
||||
"""
|
||||
|
||||
from typing import List, Optional
|
||||
|
||||
from agno.compaction._tokens import estimate_message_tokens
|
||||
from agno.compaction.prompts import SUMMARY_PREFIX
|
||||
from agno.models.message import Message
|
||||
|
||||
_LEADING_ROLES = ("system", "developer")
|
||||
|
||||
|
||||
def leading_system_count(messages: List[Message]) -> int:
|
||||
"""Number of leading system/developer messages — the block every view keeps verbatim."""
|
||||
count = 0
|
||||
for message in messages:
|
||||
if message.role in _LEADING_ROLES:
|
||||
count += 1
|
||||
else:
|
||||
break
|
||||
return count
|
||||
|
||||
|
||||
def is_injected_compaction_message(message: Message) -> bool:
|
||||
"""The summary/notice pair injected at view build. Never a boundary candidate, never counted
|
||||
as foldable: it is regenerated with fresh message ids each pass, so anchoring on it would fail
|
||||
permanently on the next build."""
|
||||
if not message.from_history or not isinstance(message.content, str):
|
||||
return False
|
||||
return message.content.startswith(SUMMARY_PREFIX)
|
||||
|
||||
|
||||
def is_offload_envelope(message: Message) -> bool:
|
||||
"""A stored-result envelope from offload_tool_results. Never folded away: the
|
||||
result_id must survive verbatim so the model can read the payload back."""
|
||||
return message.role == "tool" and isinstance(message.content, str) and message.content.startswith('<result id="')
|
||||
|
||||
|
||||
def keep_tail_start(messages: List[Message], keep_tokens: int, *, start: int = 0) -> int:
|
||||
"""Index of the first message of a kept tail of ~keep_tokens, walking backward from the end.
|
||||
|
||||
Injected summary/notice messages are not counted (they are regenerated per view, not kept
|
||||
content). Returns start when the whole span fits.
|
||||
"""
|
||||
accumulated = 0
|
||||
index = len(messages)
|
||||
while index > start:
|
||||
candidate = messages[index - 1]
|
||||
if not is_injected_compaction_message(candidate):
|
||||
accumulated += estimate_message_tokens(candidate)
|
||||
if accumulated > keep_tokens:
|
||||
break
|
||||
index -= 1
|
||||
return index
|
||||
|
||||
|
||||
def _owning_batch_head(messages: List[Message], index: int) -> Optional[int]:
|
||||
"""For a tool-role message, the index of the assistant message owning its tool_call_id."""
|
||||
tool_call_id = messages[index].tool_call_id
|
||||
if not tool_call_id:
|
||||
return None
|
||||
for assistant_index in range(index - 1, -1, -1):
|
||||
for tool_call in messages[assistant_index].tool_calls or []:
|
||||
call_id = tool_call.get("id") if isinstance(tool_call, dict) else getattr(tool_call, "id", None)
|
||||
if call_id == tool_call_id:
|
||||
return assistant_index
|
||||
return None
|
||||
|
||||
|
||||
def _is_durable_anchor(message: Message, *, allow_tool_batch_heads: bool = True) -> bool:
|
||||
"""Anchor durability: the boundary message must survive in the stored transcript.
|
||||
|
||||
temporary messages are removed mid-run; add_to_agent_memory=False messages are never
|
||||
persisted; when tool messages are scrubbed from storage, the assistant batch heads that own
|
||||
them are deleted too and cannot anchor.
|
||||
"""
|
||||
if message.temporary or not message.add_to_agent_memory:
|
||||
return False
|
||||
if is_injected_compaction_message(message):
|
||||
return False
|
||||
if not allow_tool_batch_heads and message.role == "assistant" and message.tool_calls:
|
||||
return False
|
||||
return True
|
||||
|
||||
|
||||
def choose_boundary(
|
||||
messages: List[Message],
|
||||
keep_tokens: Optional[int] = None,
|
||||
*,
|
||||
keep_from_index: Optional[int] = None,
|
||||
min_index: int = 0,
|
||||
allow_tool_batch_heads: bool = True,
|
||||
) -> Optional[int]:
|
||||
"""Choose the boundary: the index of the first message kept verbatim.
|
||||
|
||||
The kept tail accumulates ~keep_tokens walking backward; the boundary then snaps toward
|
||||
min_index to a pair-safe, durable user/assistant message — a tool result never starts the
|
||||
tail, and an orphan-threatened tool batch moves whole into it. Returns None when no valid
|
||||
boundary exists at or after min_index (the pass must abort rather than cut unsafely).
|
||||
|
||||
min_index enforces monotonicity against the previous record's resolved boundary, and — for
|
||||
in-run passes — keeps the cut inside the current run's own messages.
|
||||
"""
|
||||
lead = leading_system_count(messages)
|
||||
floor_index = max(min_index, lead)
|
||||
# A caller that already knows where the tail starts - "the last N runs" is an index, not a
|
||||
# size - passes it directly. Converting such a request into a token budget and walking
|
||||
# backward to rediscover the same index is only indirection.
|
||||
if keep_from_index is not None:
|
||||
candidate = max(keep_from_index, floor_index)
|
||||
else:
|
||||
candidate = keep_tail_start(messages, keep_tokens or 0, start=floor_index)
|
||||
if candidate <= floor_index:
|
||||
return None
|
||||
# An oversized newest message can push the walk past the end; a tail of at least one message
|
||||
# always survives.
|
||||
candidate = min(candidate, len(messages) - 1)
|
||||
|
||||
index = candidate
|
||||
while index > floor_index:
|
||||
message = messages[index]
|
||||
if message.role == "tool":
|
||||
# Move the whole batch into the kept tail.
|
||||
head = _owning_batch_head(messages, index)
|
||||
index = head if head is not None and head >= floor_index else index - 1
|
||||
continue
|
||||
if message.role in _LEADING_ROLES or not _is_durable_anchor(
|
||||
message, allow_tool_batch_heads=allow_tool_batch_heads
|
||||
):
|
||||
index -= 1
|
||||
continue
|
||||
# A tool result anywhere in the kept tail whose owning assistant falls before the
|
||||
# boundary would be orphaned; move the boundary to that batch head instead.
|
||||
orphan_head = _earliest_orphan_head(messages, index)
|
||||
if orphan_head is not None:
|
||||
index = orphan_head
|
||||
continue
|
||||
return index
|
||||
return None
|
||||
|
||||
|
||||
def _earliest_orphan_head(messages: List[Message], boundary: int) -> Optional[int]:
|
||||
"""The earliest batch head before boundary owning a tool message in messages[boundary:], or
|
||||
None when the tail is pair-safe. Tool messages with no recorded head are ignored — the
|
||||
canonical list itself carries that defect and a view must not be held to a higher bar."""
|
||||
earliest: Optional[int] = None
|
||||
for index in range(boundary, len(messages)):
|
||||
if messages[index].role != "tool":
|
||||
continue
|
||||
head = _owning_batch_head(messages, index)
|
||||
if head is not None and head < boundary and (earliest is None or head < earliest):
|
||||
earliest = head
|
||||
return earliest
|
||||
@@ -0,0 +1,106 @@
|
||||
"""Token accounting for compaction.
|
||||
|
||||
Estimates here are always computed locally (tiktoken or a character heuristic) - never via the
|
||||
provider ``count_tokens`` overrides, several of which are a network call per invocation. A user
|
||||
who wants a different count supplies it as ``Compaction.token_counter``; ``count_request`` runs it
|
||||
and falls back to these estimates when it is absent or fails.
|
||||
"""
|
||||
|
||||
import inspect
|
||||
from typing import Any, Awaitable, Callable, List, Optional, Sequence, Union
|
||||
|
||||
from agno.models.message import Message
|
||||
from agno.utils.log import log_warning
|
||||
|
||||
# Counts the tokens of a request: its messages and, when given, its tool definitions. It may be
|
||||
# async, for agents run with arun.
|
||||
TokenCounter = Callable[[List[Message], Optional[List[Any]]], Union[int, Awaitable[int]]]
|
||||
|
||||
# An async counter can be awaited only where there is an event loop to await it on.
|
||||
ASYNC_COUNTER_IN_SYNC_RUN = (
|
||||
"token_counter is async, so it can only count in async runs (agent.arun). For agent.run, pass a "
|
||||
"sync function, or set use_model_token_count=True to count with the model in both."
|
||||
)
|
||||
|
||||
|
||||
def count_request(
|
||||
counter: Optional[TokenCounter], messages: List[Message], tools: Optional[Sequence[Any]] = None
|
||||
) -> Optional[int]:
|
||||
"""``counter``'s count of a request in a sync run, or None when there is no counter or it failed.
|
||||
|
||||
A counter is often a network call or third-party code, and counting is never worth failing
|
||||
a run over, so any error leaves the caller on the local estimate. An async counter is the one
|
||||
exception: a sync run has nothing to await it with, which is a configuration error rather than
|
||||
a failed count, so it raises instead of quietly ignoring what was configured.
|
||||
"""
|
||||
if counter is None:
|
||||
return None
|
||||
if is_async_callable(counter):
|
||||
raise TypeError(ASYNC_COUNTER_IN_SYNC_RUN)
|
||||
try:
|
||||
counted = counter(list(messages), list(tools) if tools else None)
|
||||
except Exception as e: # noqa: BLE001
|
||||
log_warning(f"Compaction: token_counter failed, using the local estimate instead: {e}")
|
||||
return None
|
||||
if inspect.isawaitable(counted):
|
||||
# A sync function that hands back a coroutine - a lambda around an async call, say. Nothing
|
||||
# here can await it, so close it rather than leave it to warn "never awaited".
|
||||
if inspect.iscoroutine(counted):
|
||||
counted.close()
|
||||
log_warning(
|
||||
f"Compaction: token_counter returned a coroutine, using the local estimate instead. {ASYNC_COUNTER_IN_SYNC_RUN}"
|
||||
)
|
||||
return None
|
||||
return _as_count(counted)
|
||||
|
||||
|
||||
async def acount_request(
|
||||
counter: Optional[TokenCounter], messages: List[Message], tools: Optional[Sequence[Any]] = None
|
||||
) -> Optional[int]:
|
||||
"""``count_request`` for async runs. An async counter is awaited where the run is; a sync one runs
|
||||
in a worker thread, since it is usually a network call and the event loop must not wait on it."""
|
||||
if counter is None:
|
||||
return None
|
||||
if not is_async_callable(counter):
|
||||
import asyncio
|
||||
|
||||
return await asyncio.to_thread(count_request, counter, messages, tools)
|
||||
try:
|
||||
counted = await counter(list(messages), list(tools) if tools else None) # type: ignore[misc]
|
||||
except Exception as e: # noqa: BLE001
|
||||
log_warning(f"Compaction: token_counter failed, using the local estimate instead: {e}")
|
||||
return None
|
||||
return _as_count(counted)
|
||||
|
||||
|
||||
def _as_count(counted: Any) -> Optional[int]:
|
||||
try:
|
||||
return int(counted)
|
||||
except (TypeError, ValueError) as e:
|
||||
log_warning(f"Compaction: token_counter returned {counted!r}, using the local estimate instead: {e}")
|
||||
return None
|
||||
|
||||
|
||||
def is_async_callable(obj: Any) -> bool:
|
||||
"""Whether calling ``obj`` returns a coroutine: an async function or method, a partial of one, or
|
||||
an object whose __call__ is async."""
|
||||
return inspect.iscoroutinefunction(obj) or inspect.iscoroutinefunction(getattr(obj, "__call__", None))
|
||||
|
||||
|
||||
def estimate_tokens(messages: List[Message], tools: Optional[Sequence[Any]] = None) -> int:
|
||||
"""Local token estimate for a list of messages. Never makes a network call.
|
||||
|
||||
Pass ``tools`` when the estimate stands for a whole request. Tool schemas ride along with
|
||||
every call and are a fixed cost compaction cannot fold, so a request-sized estimate that
|
||||
leaves them out understates what the provider has to fit. Comparisons BETWEEN slices of one
|
||||
message list should leave them out: the same fixed cost on both sides cancels.
|
||||
"""
|
||||
from agno.utils.tokens import count_tokens
|
||||
|
||||
return count_tokens(list(messages), tools=list(tools) if tools else None)
|
||||
|
||||
|
||||
def estimate_message_tokens(message: Message) -> int:
|
||||
from agno.utils.tokens import count_tokens
|
||||
|
||||
return count_tokens([message])
|
||||
@@ -0,0 +1,112 @@
|
||||
"""Derived model views: what the provider receives when compaction is active.
|
||||
|
||||
A view is built per provider call from the canonical message list plus the active record, then
|
||||
discarded — never stored, never appended to, never re-read. Canonical messages are untouched:
|
||||
every transformation lands on a shallow copy with the changed attribute rebound.
|
||||
"""
|
||||
|
||||
import re
|
||||
from typing import List, Optional
|
||||
|
||||
from agno.compaction._cut import is_injected_compaction_message, is_offload_envelope, leading_system_count
|
||||
from agno.compaction.prompts import SUMMARY_PREFIX
|
||||
from agno.compaction.types import CompactionRecord
|
||||
from agno.models.message import Message
|
||||
|
||||
_RESULT_ID_PATTERN = re.compile(r'<result id="([^"]+)"')
|
||||
_MAX_SURVIVING_IDS = 100
|
||||
# provider_data keys a provider continues a conversation from instead of the messages sent:
|
||||
# OpenAI Responses chains on response_id, GeminiInteractions on interaction_id. Left on a kept
|
||||
# assistant message, either makes the provider send only what follows it, so the server replays
|
||||
# the folded turns and the summary in front of it is never sent.
|
||||
_CHAINING_KEYS = ("response_id", "interaction_id")
|
||||
|
||||
|
||||
def surviving_result_ids(folded: List[Message]) -> List[str]:
|
||||
"""Result ids from offload envelopes that this fold is about to remove.
|
||||
|
||||
An envelope's id is the only handle the model has on a stored payload, so folding one away
|
||||
silently orphans it. Pinning envelopes in the kept tail was the other option, but a single
|
||||
early envelope then caps the boundary forever and compaction stops working entirely. Carrying
|
||||
the ids forward keeps the payloads reachable at a bounded, constant cost.
|
||||
"""
|
||||
ids: List[str] = []
|
||||
for message in folded:
|
||||
if not is_offload_envelope(message):
|
||||
continue
|
||||
content = message.content if isinstance(message.content, str) else ""
|
||||
match = _RESULT_ID_PATTERN.search(content)
|
||||
if match and match.group(1) not in ids:
|
||||
ids.append(match.group(1))
|
||||
if len(ids) >= _MAX_SURVIVING_IDS:
|
||||
break
|
||||
return ids
|
||||
|
||||
|
||||
def summary_message(
|
||||
record: CompactionRecord, suffix: Optional[str] = None, result_ids: Optional[List[str]] = None
|
||||
) -> Message:
|
||||
content = SUMMARY_PREFIX + (record.summary or "")
|
||||
if result_ids:
|
||||
content += (
|
||||
"\n\nStored tool results from the folded conversation remain readable with "
|
||||
f"read_result(id): {', '.join(result_ids)}"
|
||||
)
|
||||
if suffix:
|
||||
content += suffix
|
||||
return Message(role="user", content=content, from_history=True)
|
||||
|
||||
|
||||
def _find_index(messages: List[Message], message_id: str, *, start: int = 0) -> Optional[int]:
|
||||
for index in range(start, len(messages)):
|
||||
if messages[index].id == message_id:
|
||||
return index
|
||||
return None
|
||||
|
||||
|
||||
def build_view(
|
||||
messages: List[Message],
|
||||
record: Optional[CompactionRecord],
|
||||
*,
|
||||
strip_provider_chaining: bool = False,
|
||||
summary_suffix: Optional[str] = None,
|
||||
) -> List[Message]:
|
||||
"""Derive the provider payload for one call.
|
||||
|
||||
Leading system/developer messages pass verbatim. When the record's boundary resolves in this
|
||||
list, the summary is injected and everything before the boundary is omitted — a
|
||||
summary is never injected unless its cut applies, so an unresolvable boundary fails open to
|
||||
the list as given. With strip_provider_chaining, assistant copies drop the chaining keys from
|
||||
provider_data (reasoning items and other payload survive: a function_call without its paired
|
||||
reasoning item is a provider error) so server-side chaining cannot silently rebuild the full
|
||||
history behind the view's back.
|
||||
"""
|
||||
lead = leading_system_count(messages)
|
||||
|
||||
boundary_index: Optional[int] = None
|
||||
if record is not None and record.summary and record.first_kept_message_id:
|
||||
boundary_index = _find_index(messages, record.first_kept_message_id, start=lead)
|
||||
|
||||
view: List[Message] = list(messages[:lead])
|
||||
if boundary_index is not None and record is not None:
|
||||
view.append(summary_message(record, summary_suffix, surviving_result_ids(messages[lead:boundary_index])))
|
||||
body_start = boundary_index
|
||||
else:
|
||||
body_start = lead
|
||||
|
||||
for index in range(body_start, len(messages)):
|
||||
message = messages[index]
|
||||
# When this build injects the pair itself, drop any previously injected pair so a summary
|
||||
# never appears twice in one view.
|
||||
if boundary_index is not None and is_injected_compaction_message(message):
|
||||
continue
|
||||
if (
|
||||
strip_provider_chaining
|
||||
and message.role == "assistant"
|
||||
and message.provider_data is not None
|
||||
and any(key in message.provider_data for key in _CHAINING_KEYS)
|
||||
):
|
||||
trimmed = {key: value for key, value in message.provider_data.items() if key not in _CHAINING_KEYS}
|
||||
message = message.model_copy(update={"provider_data": trimmed or None})
|
||||
view.append(message)
|
||||
return view
|
||||
@@ -0,0 +1,186 @@
|
||||
"""Durable storage for the messages a compaction replaced.
|
||||
|
||||
The archive is what makes compaction non-lossy. A summary alone is a guess about what mattered;
|
||||
with the originals still readable the summary becomes an index over ground truth, and a detail it
|
||||
dropped can still be recovered - by a developer reading the row, or by the agent itself when
|
||||
``search_compacted_messages`` is on.
|
||||
|
||||
Records live in the ``agno_compactions`` table, one row per fold, written once and never updated.
|
||||
Rows rather than files because a fold is a fact about a run: two containers writing different runs
|
||||
never collide, retention is an ordinary DELETE, and every database Agno supports can store a row -
|
||||
where the filesystem backend is implemented for SQLite and PostgreSQL only.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Any, Dict, List, Optional
|
||||
from uuid import uuid4
|
||||
|
||||
from agno.models.message import Message
|
||||
from agno.utils.log import log_debug, log_warning
|
||||
|
||||
# Tool results dominate the byte count of a transcript and are the least useful part to read back
|
||||
# verbatim. Clip each one so a single enormous result cannot bloat one row.
|
||||
MAX_ARCHIVED_TOOL_RESULT_CHARS = 20_000
|
||||
|
||||
|
||||
def render_message(message: Message) -> str:
|
||||
"""Render one message as a markdown block. Tool results are clipped to MAX_ARCHIVED_TOOL_RESULT_CHARS."""
|
||||
heading = message.role
|
||||
if message.tool_name:
|
||||
heading = f"{message.role} ({message.tool_name})"
|
||||
parts: List[str] = [f"## {heading}"]
|
||||
|
||||
content = message.get_content_string()
|
||||
if content:
|
||||
parts.append(_clip(content, MAX_ARCHIVED_TOOL_RESULT_CHARS) if message.role == "tool" else content)
|
||||
|
||||
for tool_call in message.tool_calls or []:
|
||||
function = tool_call.get("function") if isinstance(tool_call, dict) else None
|
||||
if isinstance(function, dict):
|
||||
parts.append(f"**calls** `{function.get('name')}`: {function.get('arguments')}")
|
||||
|
||||
return "\n\n".join(parts)
|
||||
|
||||
|
||||
def render_messages(messages: List[Message]) -> str:
|
||||
"""Render messages as readable markdown.
|
||||
|
||||
Written for a human, or an agent searching the archive, so the role is a heading and tool calls
|
||||
name the tool: a search for a tool name or a phrase from an old answer should land on the turn
|
||||
that produced it.
|
||||
"""
|
||||
return "\n\n".join(render_message(message) for message in messages)
|
||||
|
||||
|
||||
def _clip(text: str, limit: int) -> str:
|
||||
if len(text) <= limit:
|
||||
return text
|
||||
return text[:limit] + f"\n... [clipped, {len(text) - limit} more characters]"
|
||||
|
||||
|
||||
# The most alternatives one search may carry.
|
||||
MAX_SEARCH_TERMS = 20
|
||||
|
||||
|
||||
def search_terms(pattern: str) -> List[str]:
|
||||
"""The literal terms of a search: the pattern split on "|"."""
|
||||
return [term.strip() for term in pattern.split("|") if term.strip()][:MAX_SEARCH_TERMS]
|
||||
|
||||
|
||||
def stores_compaction_records(db: Any) -> bool:
|
||||
"""Whether ``db`` can store compaction records - SqliteDb and PostgresDb.
|
||||
|
||||
The sync BaseDb declares the record methods as stubs that raise, so only a db that overrides
|
||||
them stores rows; async dbs do not declare them at all.
|
||||
"""
|
||||
method = getattr(type(db), "upsert_compaction", None)
|
||||
if method is None:
|
||||
return False
|
||||
from agno.db.base import BaseDb
|
||||
|
||||
return method is not getattr(BaseDb, "upsert_compaction", None)
|
||||
|
||||
|
||||
class CompactionArchive:
|
||||
"""Reads and writes one session's compaction records."""
|
||||
|
||||
def __init__(self, db: Any, session_id: str, user_id: Optional[str] = None) -> None:
|
||||
self.db = db
|
||||
self.session_id = session_id
|
||||
self.user_id = user_id
|
||||
|
||||
def _row(self, record: Any, messages: List[Message]) -> Dict[str, Any]:
|
||||
return {
|
||||
"compaction_id": record.id or uuid4().hex,
|
||||
"session_id": self.session_id,
|
||||
"run_id": record.run_id,
|
||||
"user_id": self.user_id,
|
||||
"first_kept_message_id": record.first_kept_message_id,
|
||||
"summary": record.summary,
|
||||
"archived_messages": render_messages(messages) if messages else None,
|
||||
"messages_compacted": record.messages_compacted,
|
||||
"tokens_before": record.tokens_before,
|
||||
"tokens_after": record.tokens_after,
|
||||
"created_at": record.created_at,
|
||||
}
|
||||
|
||||
def _method(self, name: str) -> Any:
|
||||
"""The db's implementation of one record method.
|
||||
|
||||
Only the sync BaseDb declares the record contract, as stubs that raise. A db without the
|
||||
method at all - every async db - raises the same way, so it takes the same path.
|
||||
"""
|
||||
method = getattr(self.db, name, None)
|
||||
if method is None:
|
||||
raise NotImplementedError(name)
|
||||
return method
|
||||
|
||||
def write(self, record: Any, messages: List[Message]) -> bool:
|
||||
"""Persist one record. A failure here loses recoverability, never the run."""
|
||||
try:
|
||||
self._method("upsert_compaction")(self._row(record, messages))
|
||||
log_debug(f"Stored compaction {record.id} with {len(messages)} archived messages")
|
||||
return True
|
||||
except NotImplementedError:
|
||||
log_warning(
|
||||
"This database does not implement compaction records; the summary still applies "
|
||||
"but the folded messages are not recoverable."
|
||||
)
|
||||
except Exception as e: # noqa: BLE001 - persistence must never fail a run
|
||||
log_warning(f"Could not store compaction record: {e}")
|
||||
return False
|
||||
|
||||
def latest(self, up_to_run_id: Optional[str] = None) -> Optional[Dict[str, Any]]:
|
||||
"""The record in force, as of ``up_to_run_id`` when one is given.
|
||||
|
||||
Resuming or forking an earlier run must use the fold *that run saw*. A later record
|
||||
summarizes turns the earlier run never had, and its anchor sits ahead of where that run
|
||||
was, so applying it would show the resumed run a summary of its own future.
|
||||
"""
|
||||
try:
|
||||
rows = self._method("get_compactions_for_session")(self.session_id)
|
||||
except NotImplementedError:
|
||||
return None
|
||||
except Exception as e: # noqa: BLE001
|
||||
log_warning(f"Could not read compaction records: {e}")
|
||||
return None
|
||||
if not rows:
|
||||
return None
|
||||
if up_to_run_id is None:
|
||||
return rows[0]
|
||||
# Rows are newest first, so the first at or before the target run is the one it saw.
|
||||
seen_target = False
|
||||
for row in rows:
|
||||
if row.get("run_id") == up_to_run_id:
|
||||
seen_target = True
|
||||
if seen_target:
|
||||
return row
|
||||
return None
|
||||
|
||||
def search(self, query: str, limit: int = 10) -> List[Dict[str, Any]]:
|
||||
"""Candidate rows for a search: every fold containing any of its terms, newest first.
|
||||
|
||||
All terms go to SQL as one substring filter. LIKE wildcards in a term can only widen the
|
||||
match, and the caller matches exactly afterwards, so no fold that holds a term is dropped -
|
||||
however old it is.
|
||||
"""
|
||||
terms = search_terms(query)
|
||||
if not terms:
|
||||
return []
|
||||
try:
|
||||
return self._method("search_compactions")(self.session_id, terms, limit)
|
||||
except NotImplementedError:
|
||||
return []
|
||||
except Exception as e: # noqa: BLE001
|
||||
log_warning(f"Could not search compaction records: {e}")
|
||||
return []
|
||||
|
||||
|
||||
__all__ = [
|
||||
"CompactionArchive",
|
||||
"render_message",
|
||||
"render_messages",
|
||||
"search_terms",
|
||||
"MAX_ARCHIVED_TOOL_RESULT_CHARS",
|
||||
]
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,88 @@
|
||||
"""Prompt constants for conversation compaction."""
|
||||
|
||||
# Appended in place of what was cut when a summary over its budget is shortened to fit.
|
||||
SUMMARY_CUT_NOTE = "(The rest of this summary was cut to fit its length budget.)"
|
||||
SUMMARY_CUT_NOTE_SEARCHABLE = (
|
||||
"(The rest of this summary was cut to fit its length budget - search the stored messages for it.)"
|
||||
)
|
||||
|
||||
# The length rule in the summarization prompt. With a budget the model is given the number three
|
||||
# ways, since it cannot count tokens but can estimate words and characters. Without one it is asked
|
||||
# to stay compact, which leaves the length to the model and the summary uncut.
|
||||
LENGTH_RULE_BUDGET = (
|
||||
"- Hard length budget: {budget_tokens} tokens (roughly {budget_words} words, {budget_characters} characters).\n"
|
||||
" Compress prose before dropping facts."
|
||||
)
|
||||
LENGTH_RULE_COMPACT = (
|
||||
"- Keep the summary compact: compress prose before dropping facts, and do not pad a section to\n"
|
||||
" make it look complete. An empty section is one line."
|
||||
)
|
||||
|
||||
# Marker line that opens the summary message injected into model input.
|
||||
# Fixed so injected summaries are identifiable (and skippable) across builds.
|
||||
SUMMARY_PREFIX = "Summary of earlier conversation (compacted):\n\n"
|
||||
|
||||
DEFAULT_COMPACTION_PROMPT = """You maintain the running summary of a long conversation between a user and an AI agent. The
|
||||
conversation exceeds the model's context window, so everything older than a recent tail is folded into
|
||||
the summary you produce. Your summary is the ONLY memory of the folded conversation: anything you omit
|
||||
is lost to the agent.
|
||||
|
||||
You are given the previous summary (if one exists) and a transcript segment to fold into it. Produce an
|
||||
updated summary with exactly these sections, in this order:
|
||||
|
||||
## Goal
|
||||
## In progress / next steps
|
||||
## Critical context
|
||||
## Constraints & preferences
|
||||
## Key decisions & facts
|
||||
## Errors & fixes
|
||||
## Completed
|
||||
|
||||
The order is by importance, most important first. A summary that runs too long may be cut from the
|
||||
end, so what the agent needs to carry on comes first and finished history last.
|
||||
|
||||
Section guidance:
|
||||
- Goal: what the user is ultimately trying to achieve. Rarely changes.
|
||||
- In progress / next steps: what is underway and what comes next.
|
||||
- Critical context: exact file paths, identifiers, result_ids, URLs, error messages, and numbers -
|
||||
verbatim. Never paraphrase these.
|
||||
- Constraints & preferences: standing user constraints and instructions that remain in force - coding
|
||||
rules, limits, style and process preferences. These are easy to lose; keep every one that has not been
|
||||
explicitly lifted.
|
||||
- Key decisions & facts: decisions made and why, plus durable facts established in conversation.
|
||||
- Errors & fixes: errors encountered and how they were resolved (or that they remain open).
|
||||
- Completed: finished work, stated compactly.
|
||||
|
||||
Rules:
|
||||
- Preserve everything from the previous summary unless the new segment supersedes it.
|
||||
- Move items that the new segment shows are finished into Completed.
|
||||
- Do not continue the conversation, answer questions, or add commentary.
|
||||
- Output only the summary, starting at "## Goal".
|
||||
{length_rule}
|
||||
|
||||
Durable state (offloaded tool results, defined variables, files on disk) outlives this summary and is
|
||||
listed separately for the agent; you do not need to reproduce its contents, only reference identifiers
|
||||
in Critical context when they matter."""
|
||||
|
||||
|
||||
# Appended to the summarization prompt when the folded messages are archived and the agent can
|
||||
# read them back. A summary cannot be complete, and the agent reading it cannot tell what is
|
||||
# missing unless the summary says so - which is what turns "answer from the summary" into
|
||||
# "check the archive first" on exactly the questions that need it.
|
||||
ARCHIVE_AWARE_PROMPT = """
|
||||
|
||||
The full transcript of the folded segment remains available to the agent.
|
||||
Directly under the Goal section, add one line beginning "Not covered here:" naming the kinds of detail
|
||||
a reader would have to look up - for example bulk tabular data, long tool output, or full error text.
|
||||
Omit the line only if the summary genuinely preserves every specific in the segment."""
|
||||
|
||||
# Appended to the injected summary message when the archive is readable by the agent. States the
|
||||
# rule rather than suggesting it: a model told to "search if needed" will usually judge the
|
||||
# summary sufficient and answer from it, including for the exact values a summary is least
|
||||
# likely to have kept.
|
||||
ARCHIVE_LOOKUP_INSTRUCTION = """
|
||||
|
||||
The full text of the folded conversation is stored and searchable.
|
||||
Before answering any question about it that calls for an exact value - an identifier, figure,
|
||||
name, quote, command, or error message - search it rather than relying on this
|
||||
summary. Say you do not know only after looking."""
|
||||
@@ -0,0 +1,159 @@
|
||||
"""Result types for conversation compaction."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass, field
|
||||
from enum import Enum
|
||||
from time import time
|
||||
from typing import TYPE_CHECKING, Any, Dict, Optional
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from agno.metrics import RunMetrics
|
||||
|
||||
# The namespace prefix every compaction archive lives under. Sibling to the
|
||||
# "tool-results" prefix result offloading uses, so one database backs both
|
||||
# without either being able to read the other's files.
|
||||
ARCHIVE_NAMESPACE_PREFIX = "history"
|
||||
|
||||
|
||||
class CompactionStatus(str, Enum):
|
||||
"""Why a compaction attempt ended the way it did.
|
||||
|
||||
A manual compaction can decline for reasons that are not failures, so a bare None cannot
|
||||
tell a caller what happened. These are the outcomes an API or UI needs to distinguish.
|
||||
"""
|
||||
|
||||
# The fold happened.
|
||||
COMPACTED = "compacted"
|
||||
# Compaction is not configured on this agent.
|
||||
NOT_ENABLED = "not_enabled"
|
||||
# The session has no stored history yet.
|
||||
NO_HISTORY = "no_history"
|
||||
# uncompacted_runs covers the whole conversation, so there is no
|
||||
# history in front of the kept tail to fold.
|
||||
NOTHING_TO_FOLD = "nothing_to_fold"
|
||||
# A previous fold already covers everything up to the only safe cut point.
|
||||
ALREADY_COMPACTED = "already_compacted"
|
||||
# Folding this span would cost more in summary than it reclaims. Only with
|
||||
# enforce_min_fold_ratio=True.
|
||||
NOT_WORTH_IT = "not_worth_it"
|
||||
# The summarizer returned nothing.
|
||||
SUMMARY_FAILED = "summary_failed"
|
||||
|
||||
|
||||
@dataclass
|
||||
class CompactionResult:
|
||||
"""The outcome of an explicit ``agent.compact()`` call.
|
||||
|
||||
Returned instead of a bare record so a caller - an API route, a UI - can tell a decline
|
||||
apart from a failure, and show the reason without parsing a log line.
|
||||
"""
|
||||
|
||||
status: CompactionStatus
|
||||
message: str
|
||||
record: Optional["CompactionRecord"] = None
|
||||
# The summarizer's token usage, under compaction_model. A manual compaction has no run to
|
||||
# add it to, so it is reported here instead. None when no summarizer call was made.
|
||||
metrics: Optional["RunMetrics"] = None
|
||||
|
||||
@property
|
||||
def compacted(self) -> bool:
|
||||
return self.status == CompactionStatus.COMPACTED
|
||||
|
||||
def to_dict(self) -> Dict[str, Any]:
|
||||
return {
|
||||
"status": self.status.value,
|
||||
"message": self.message,
|
||||
"compacted": self.compacted,
|
||||
"record": self.record.to_dict() if self.record is not None else None,
|
||||
"metrics": self.metrics.to_dict() if self.metrics is not None else None,
|
||||
}
|
||||
|
||||
|
||||
@dataclass
|
||||
class CompactionRecord:
|
||||
"""What one compaction did: the summary, and where the originals went.
|
||||
|
||||
Persisted as a row in ``agno_compactions`` so a later run replays the fold rather than paying
|
||||
for it again. ``archived`` is False when the transcript could not be stored - a database that
|
||||
does not implement the optional contract, or a write that failed. The summary still stands; it
|
||||
is simply no longer recoverable in full, which is why the two are separate.
|
||||
"""
|
||||
|
||||
# Number of history messages this compaction replaced.
|
||||
messages_compacted: int
|
||||
# The generated summary that stands in for them.
|
||||
summary: str
|
||||
# Id of the first message kept verbatim - the boundary anchor.
|
||||
#
|
||||
# An id, not an index: history is rebuilt from stored runs on every run, so a positional
|
||||
# boundary means something different each time the list grows and the cut silently crawls.
|
||||
# An id resolves to the same message forever, or fails to resolve at all - in which case the
|
||||
# view falls open to the full list rather than cutting in the wrong place.
|
||||
first_kept_message_id: Optional[str] = None
|
||||
# Row id of this record, and the run that produced it. Together they make the record a fact
|
||||
# about one run rather than a value shared by the session.
|
||||
id: Optional[str] = None
|
||||
run_id: Optional[str] = None
|
||||
# Whether the folded transcript was stored alongside this record.
|
||||
archived: bool = False
|
||||
tokens_before: Optional[int] = None
|
||||
tokens_after: Optional[int] = None
|
||||
# Stamped at construction: the row requires it, and a record without a time cannot be
|
||||
# ordered in the chain that resolves which fold is in force.
|
||||
created_at: int = field(default_factory=lambda: int(time()))
|
||||
|
||||
def to_dict(self) -> Dict[str, Any]:
|
||||
_dict = {
|
||||
"messages_compacted": self.messages_compacted,
|
||||
"summary": self.summary,
|
||||
"first_kept_message_id": self.first_kept_message_id,
|
||||
"id": self.id,
|
||||
"run_id": self.run_id,
|
||||
"archived": self.archived,
|
||||
"tokens_before": self.tokens_before,
|
||||
"tokens_after": self.tokens_after,
|
||||
"created_at": self.created_at,
|
||||
}
|
||||
return {k: v for k, v in _dict.items() if v is not None}
|
||||
|
||||
@classmethod
|
||||
def from_dict(cls, data: Dict[str, Any]) -> "CompactionRecord":
|
||||
return cls(
|
||||
messages_compacted=data.get("messages_compacted", 0),
|
||||
summary=data.get("summary", ""),
|
||||
first_kept_message_id=data.get("first_kept_message_id"),
|
||||
id=data.get("id") or data.get("compaction_id"),
|
||||
run_id=data.get("run_id"),
|
||||
archived=bool(data.get("archived") or data.get("archived_messages")),
|
||||
tokens_before=data.get("tokens_before"),
|
||||
tokens_after=data.get("tokens_after"),
|
||||
created_at=data.get("created_at") or int(time()),
|
||||
)
|
||||
|
||||
|
||||
@dataclass
|
||||
class CompactionStats:
|
||||
"""Running totals for one agent, rendered after a run."""
|
||||
|
||||
compactions: int = 0
|
||||
messages_compacted: int = 0
|
||||
tokens_before: int = 0
|
||||
tokens_after: int = 0
|
||||
extra: Dict[str, Any] = field(default_factory=dict)
|
||||
|
||||
def record(self, record: CompactionRecord) -> None:
|
||||
self.compactions += 1
|
||||
self.messages_compacted += record.messages_compacted
|
||||
self.tokens_before += record.tokens_before or 0
|
||||
self.tokens_after += record.tokens_after or 0
|
||||
|
||||
def clear(self) -> None:
|
||||
self.compactions = 0
|
||||
self.messages_compacted = 0
|
||||
self.tokens_before = 0
|
||||
self.tokens_after = 0
|
||||
self.extra.clear()
|
||||
|
||||
|
||||
__all__ = ["ARCHIVE_NAMESPACE_PREFIX", "CompactionRecord", "CompactionStats"]
|
||||
@@ -3,7 +3,7 @@ import threading
|
||||
from abc import ABC, abstractmethod
|
||||
from datetime import date, datetime
|
||||
from enum import Enum
|
||||
from typing import TYPE_CHECKING, Any, Dict, List, Literal, Optional, Set, Tuple, Union
|
||||
from typing import TYPE_CHECKING, Any, Dict, List, Literal, Optional, Sequence, Set, Tuple, Union
|
||||
from uuid import uuid4
|
||||
|
||||
if TYPE_CHECKING:
|
||||
@@ -355,6 +355,7 @@ class BaseDb(ABC):
|
||||
self.schedule_runs_table_name = schedule_runs_table or "agno_schedule_runs"
|
||||
self.job_table_name = job_table or "agno_jobs"
|
||||
self.tool_results_table_name = "agno_tool_results"
|
||||
self.compactions_table_name = "agno_compactions"
|
||||
self.approvals_table_name = approvals_table or "agno_approvals"
|
||||
self.auth_tokens_table_name = auth_tokens_table or "agno_auth_tokens"
|
||||
self.service_accounts_table_name = service_accounts_table or "agno_service_accounts"
|
||||
@@ -1907,6 +1908,40 @@ class BaseDb(ABC):
|
||||
"""Rows whose expires_at has passed."""
|
||||
raise NotImplementedError
|
||||
|
||||
# --- Compactions (Optional) ---
|
||||
# Optional: the record table for conversation compaction (agno_compactions).
|
||||
# A record is an immutable fact - one fold, produced by one run - so rows are
|
||||
# written once and never updated. The fold in force for a run is a query over
|
||||
# them, not a stored value, which is what keeps concurrent writers from
|
||||
# clobbering one another.
|
||||
#
|
||||
# ``archived_messages`` carries the folded transcript verbatim. A backend that
|
||||
# cannot store a large text column may leave it null: the summary still stands,
|
||||
# and only recoverability is lost.
|
||||
|
||||
def upsert_compaction(self, row: Dict[str, Any]) -> None:
|
||||
"""Insert one compaction record."""
|
||||
raise NotImplementedError
|
||||
|
||||
def get_compaction(self, compaction_id: str) -> Optional[Dict[str, Any]]:
|
||||
"""Get one compaction record by id."""
|
||||
raise NotImplementedError
|
||||
|
||||
def get_compactions_for_session(self, session_id: str, limit: Optional[int] = None) -> List[Dict[str, Any]]:
|
||||
"""The session's compaction records, newest first."""
|
||||
raise NotImplementedError
|
||||
|
||||
def delete_compactions_for_session(self, session_id: str) -> int:
|
||||
"""Delete a session's records. Returns the number deleted."""
|
||||
raise NotImplementedError
|
||||
|
||||
def search_compactions(
|
||||
self, session_id: str, query: Union[str, Sequence[str]], limit: int = 10
|
||||
) -> List[Dict[str, Any]]:
|
||||
"""Records in this session whose archived transcript contains ``query`` - or, given several
|
||||
terms, any one of them - newest first."""
|
||||
raise NotImplementedError
|
||||
|
||||
# --- Approvals (Optional) ---
|
||||
# These methods are optional. Override in subclasses to enable approval persistence.
|
||||
|
||||
@@ -2331,6 +2366,7 @@ class AsyncBaseDb(ABC):
|
||||
self.schedule_runs_table_name = schedule_runs_table or "agno_schedule_runs"
|
||||
self.job_table_name = job_table or "agno_jobs"
|
||||
self.tool_results_table_name = "agno_tool_results"
|
||||
self.compactions_table_name = "agno_compactions"
|
||||
self.approvals_table_name = approvals_table or "agno_approvals"
|
||||
self.auth_tokens_table_name = auth_tokens_table or "agno_auth_tokens"
|
||||
self.service_accounts_table_name = service_accounts_table or "agno_service_accounts"
|
||||
|
||||
@@ -765,6 +765,14 @@ class PostgresDb(BaseDb):
|
||||
)
|
||||
return self.service_accounts_table
|
||||
|
||||
if table_type == "compactions":
|
||||
self.compactions_table = self._get_or_create_table(
|
||||
table_name=self.compactions_table_name,
|
||||
table_type="compactions",
|
||||
create_table_if_not_found=create_table_if_not_found,
|
||||
)
|
||||
return self.compactions_table
|
||||
|
||||
if table_type in MCP_OAUTH_TABLE_NAME_ATTRS:
|
||||
return self._get_or_create_table(
|
||||
table_name=getattr(self, MCP_OAUTH_TABLE_NAME_ATTRS[table_type]),
|
||||
@@ -1289,6 +1297,7 @@ class PostgresDb(BaseDb):
|
||||
if table is None:
|
||||
return False
|
||||
runs_table = self._get_table(table_type="runs")
|
||||
compactions_table = self._get_table(table_type="compactions")
|
||||
|
||||
with self.Session() as sess, sess.begin():
|
||||
delete_stmt = table.delete().where(table.c.session_id == session_id)
|
||||
@@ -1303,6 +1312,11 @@ class PostgresDb(BaseDb):
|
||||
# Also delete the runs belonging to the session
|
||||
if runs_table is not None:
|
||||
sess.execute(runs_table.delete().where(runs_table.c.session_id == session_id))
|
||||
# And its compaction records. They hold the folded transcript verbatim, and are
|
||||
# found by session id - left behind, a session recreated under the same id would
|
||||
# inherit the old fold and offer the agent a search over the deleted conversation.
|
||||
if compactions_table is not None:
|
||||
sess.execute(compactions_table.delete().where(compactions_table.c.session_id == session_id))
|
||||
|
||||
log_debug(f"Successfully deleted session with session_id: {session_id} in table {table.name}")
|
||||
|
||||
@@ -1332,6 +1346,7 @@ class PostgresDb(BaseDb):
|
||||
if table is None:
|
||||
return
|
||||
runs_table = self._get_table(table_type="runs")
|
||||
compactions_table = self._get_table(table_type="compactions")
|
||||
|
||||
with self.Session() as sess, sess.begin():
|
||||
# The ids a user_id-scoped delete is allowed to touch. The
|
||||
@@ -1358,6 +1373,13 @@ class PostgresDb(BaseDb):
|
||||
runs_delete_stmt = runs_delete_stmt.where(runs_table.c.user_id == user_id)
|
||||
sess.execute(runs_delete_stmt)
|
||||
|
||||
# And their compaction records. They hold the folded transcript verbatim, and are
|
||||
# found by session id - left behind, a session recreated under the same id would
|
||||
# inherit the old fold and offer the agent a search over the deleted conversation.
|
||||
# Scoped like the tool-result cascade: only sessions this delete was allowed to remove.
|
||||
if compactions_table is not None:
|
||||
sess.execute(compactions_table.delete().where(compactions_table.c.session_id.in_(cascade_ids)))
|
||||
|
||||
log_debug(f"Successfully deleted {result.rowcount} sessions")
|
||||
|
||||
# Cascade offloaded tool results after the session delete commits.
|
||||
@@ -1495,6 +1517,73 @@ class PostgresDb(BaseDb):
|
||||
with self.Session() as sess:
|
||||
return [dict(row._mapping) for row in sess.execute(stmt).fetchall()]
|
||||
|
||||
# --- Compactions ---
|
||||
|
||||
def upsert_compaction(self, row: Dict[str, Any]) -> None:
|
||||
"""Insert one compaction record.
|
||||
|
||||
Records are immutable facts about a single fold, so a conflicting id means
|
||||
the same record is being written twice; the insert is a no-op rather than
|
||||
an update.
|
||||
"""
|
||||
table = self._get_table(table_type="compactions", create_table_if_not_found=True)
|
||||
if table is None:
|
||||
raise ValueError(f"Could not create table: {self.compactions_table_name}")
|
||||
from sqlalchemy.dialects.postgresql import insert as _insert
|
||||
|
||||
stmt = _insert(table).values(**row).on_conflict_do_nothing(index_elements=["compaction_id"])
|
||||
with self.Session() as sess, sess.begin():
|
||||
sess.execute(stmt)
|
||||
|
||||
def get_compaction(self, compaction_id: str) -> Optional[Dict[str, Any]]:
|
||||
table = self._get_table(table_type="compactions")
|
||||
if table is None:
|
||||
return None
|
||||
with self.Session() as sess:
|
||||
row = sess.execute(select(table).where(table.c.compaction_id == compaction_id)).fetchone()
|
||||
return dict(row._mapping) if row is not None else None
|
||||
|
||||
def get_compactions_for_session(self, session_id: str, limit: Optional[int] = None) -> List[Dict[str, Any]]:
|
||||
table = self._get_table(table_type="compactions")
|
||||
if table is None:
|
||||
return []
|
||||
stmt = select(table).where(table.c.session_id == session_id).order_by(table.c.created_at.desc())
|
||||
if limit is not None:
|
||||
stmt = stmt.limit(limit)
|
||||
with self.Session() as sess:
|
||||
return [dict(r._mapping) for r in sess.execute(stmt).fetchall()]
|
||||
|
||||
def delete_compactions_for_session(self, session_id: str) -> int:
|
||||
table = self._get_table(table_type="compactions")
|
||||
if table is None:
|
||||
return 0
|
||||
with self.Session() as sess, sess.begin():
|
||||
result = sess.execute(table.delete().where(table.c.session_id == session_id))
|
||||
return result.rowcount or 0
|
||||
|
||||
def search_compactions(
|
||||
self, session_id: str, query: Union[str, Sequence[str]], limit: int = 10
|
||||
) -> List[Dict[str, Any]]:
|
||||
"""Records whose archived transcript contains ``query`` - or any of several terms - newest first.
|
||||
|
||||
A substring match, scoped to one session so a search can never reach another conversation's
|
||||
history. Several terms are one query, so every fold matching any of them is a candidate.
|
||||
"""
|
||||
table = self._get_table(table_type="compactions")
|
||||
terms = [query] if isinstance(query, str) else list(query)
|
||||
terms = [term for term in terms if term]
|
||||
if table is None or not terms:
|
||||
return []
|
||||
stmt = (
|
||||
select(table)
|
||||
.where(table.c.session_id == session_id)
|
||||
.where(or_(*(table.c.archived_messages.ilike(f"%{term}%") for term in terms)))
|
||||
.order_by(table.c.created_at.desc())
|
||||
.limit(limit)
|
||||
)
|
||||
with self.Session() as sess:
|
||||
return [dict(r._mapping) for r in sess.execute(stmt).fetchall()]
|
||||
|
||||
def get_session(
|
||||
self,
|
||||
session_id: str,
|
||||
|
||||
@@ -482,6 +482,31 @@ SERVICE_ACCOUNT_TABLE_SCHEMA = {
|
||||
}
|
||||
|
||||
|
||||
COMPACTIONS_TABLE_SCHEMA = {
|
||||
"compaction_id": {"type": String, "primary_key": True, "nullable": False},
|
||||
"session_id": {"type": String, "nullable": False},
|
||||
# The run that produced this fold. A record is an immutable fact about one
|
||||
# run, so resuming that run resolves the fold it actually saw.
|
||||
"run_id": {"type": String, "nullable": True},
|
||||
"user_id": {"type": String, "nullable": True},
|
||||
# Anchors are message ids, never indexes: history is rebuilt from stored runs
|
||||
# each run, so a position means something different every time.
|
||||
"first_kept_message_id": {"type": String, "nullable": True},
|
||||
"summary": {"type": String, "nullable": True},
|
||||
# The folded transcript, verbatim. Null when the backend or the caller opted
|
||||
# out: the summary still stands, only recoverability is lost.
|
||||
"archived_messages": {"type": String, "nullable": True},
|
||||
"messages_compacted": {"type": BigInteger, "nullable": True},
|
||||
"tokens_before": {"type": BigInteger, "nullable": True},
|
||||
"tokens_after": {"type": BigInteger, "nullable": True},
|
||||
"created_at": {"type": BigInteger, "nullable": False},
|
||||
"__composite_indexes__": [
|
||||
# Resolving the fold in force, and session cleanup.
|
||||
{"name": "compactions_session_created_at", "columns": ["session_id", "created_at"]},
|
||||
],
|
||||
}
|
||||
|
||||
|
||||
def get_table_schema_definition(
|
||||
table_type: str,
|
||||
traces_table_name: str = "agno_traces",
|
||||
@@ -526,6 +551,7 @@ def get_table_schema_definition(
|
||||
"schedules": SCHEDULE_TABLE_SCHEMA,
|
||||
"jobs": JOBS_TABLE_SCHEMA,
|
||||
"tool_results": TOOL_RESULTS_TABLE_SCHEMA,
|
||||
"compactions": COMPACTIONS_TABLE_SCHEMA,
|
||||
"approvals": APPROVAL_TABLE_SCHEMA,
|
||||
"auth_tokens": AUTH_TOKEN_TABLE_SCHEMA,
|
||||
"service_accounts": SERVICE_ACCOUNT_TABLE_SCHEMA,
|
||||
|
||||
@@ -420,6 +420,31 @@ TOOL_RESULTS_TABLE_SCHEMA = {
|
||||
}
|
||||
|
||||
|
||||
COMPACTIONS_TABLE_SCHEMA = {
|
||||
"compaction_id": {"type": String, "primary_key": True, "nullable": False},
|
||||
"session_id": {"type": String, "nullable": False},
|
||||
# The run that produced this fold. A record is an immutable fact about one
|
||||
# run, so resuming that run resolves the fold it actually saw.
|
||||
"run_id": {"type": String, "nullable": True},
|
||||
"user_id": {"type": String, "nullable": True},
|
||||
# Anchors are message ids, never indexes: history is rebuilt from stored runs
|
||||
# each run, so a position means something different every time.
|
||||
"first_kept_message_id": {"type": String, "nullable": True},
|
||||
"summary": {"type": String, "nullable": True},
|
||||
# The folded transcript, verbatim. Null when the backend or the caller opted
|
||||
# out: the summary still stands, only recoverability is lost.
|
||||
"archived_messages": {"type": String, "nullable": True},
|
||||
"messages_compacted": {"type": BigInteger, "nullable": True},
|
||||
"tokens_before": {"type": BigInteger, "nullable": True},
|
||||
"tokens_after": {"type": BigInteger, "nullable": True},
|
||||
"created_at": {"type": BigInteger, "nullable": False},
|
||||
"__composite_indexes__": [
|
||||
# Resolving the fold in force, and session cleanup.
|
||||
{"name": "compactions_session_created_at", "columns": ["session_id", "created_at"]},
|
||||
],
|
||||
}
|
||||
|
||||
|
||||
def get_table_schema_definition(
|
||||
table_type: str,
|
||||
traces_table_name: str = "agno_traces",
|
||||
@@ -462,6 +487,7 @@ def get_table_schema_definition(
|
||||
"learnings": LEARNINGS_TABLE_SCHEMA,
|
||||
"schedules": SCHEDULE_TABLE_SCHEMA,
|
||||
"tool_results": TOOL_RESULTS_TABLE_SCHEMA,
|
||||
"compactions": COMPACTIONS_TABLE_SCHEMA,
|
||||
"approvals": APPROVAL_TABLE_SCHEMA,
|
||||
"auth_tokens": AUTH_TOKEN_TABLE_SCHEMA,
|
||||
"service_accounts": SERVICE_ACCOUNT_TABLE_SCHEMA,
|
||||
|
||||
@@ -772,6 +772,14 @@ class SqliteDb(BaseDb):
|
||||
)
|
||||
return self.service_accounts_table
|
||||
|
||||
elif table_type == "compactions":
|
||||
self.compactions_table = self._get_or_create_table(
|
||||
table_name=self.compactions_table_name,
|
||||
table_type="compactions",
|
||||
create_table_if_not_found=create_table_if_not_found,
|
||||
)
|
||||
return self.compactions_table
|
||||
|
||||
elif table_type in AUTHZ_TABLE_NAME_ATTRS:
|
||||
return self._get_or_create_table(
|
||||
table_name=getattr(self, AUTHZ_TABLE_NAME_ATTRS[table_type]),
|
||||
@@ -1302,6 +1310,7 @@ class SqliteDb(BaseDb):
|
||||
if table is None:
|
||||
return False
|
||||
runs_table = self._get_table(table_type="runs")
|
||||
compactions_table = self._get_table(table_type="compactions")
|
||||
|
||||
with self.Session() as sess, sess.begin():
|
||||
delete_stmt = table.delete().where(table.c.session_id == session_id)
|
||||
@@ -1315,6 +1324,11 @@ class SqliteDb(BaseDb):
|
||||
# Also delete the runs belonging to the session
|
||||
if runs_table is not None:
|
||||
sess.execute(runs_table.delete().where(runs_table.c.session_id == session_id))
|
||||
# And its compaction records. They hold the folded transcript verbatim, and are
|
||||
# found by session id - left behind, a session recreated under the same id would
|
||||
# inherit the old fold and offer the agent a search over the deleted conversation.
|
||||
if compactions_table is not None:
|
||||
sess.execute(compactions_table.delete().where(compactions_table.c.session_id == session_id))
|
||||
|
||||
log_debug(f"Successfully deleted session with session_id: {session_id}")
|
||||
|
||||
@@ -1344,6 +1358,7 @@ class SqliteDb(BaseDb):
|
||||
if table is None:
|
||||
return
|
||||
runs_table = self._get_table(table_type="runs")
|
||||
compactions_table = self._get_table(table_type="compactions")
|
||||
|
||||
with self.Session() as sess, sess.begin():
|
||||
# The ids a user_id-scoped delete is allowed to touch. The
|
||||
@@ -1370,6 +1385,13 @@ class SqliteDb(BaseDb):
|
||||
runs_delete_stmt = runs_delete_stmt.where(runs_table.c.user_id == user_id)
|
||||
sess.execute(runs_delete_stmt)
|
||||
|
||||
# And their compaction records. They hold the folded transcript verbatim, and are
|
||||
# found by session id - left behind, a session recreated under the same id would
|
||||
# inherit the old fold and offer the agent a search over the deleted conversation.
|
||||
# Scoped like the tool-result cascade: only sessions this delete was allowed to remove.
|
||||
if compactions_table is not None:
|
||||
sess.execute(compactions_table.delete().where(compactions_table.c.session_id.in_(cascade_ids)))
|
||||
|
||||
log_debug(f"Successfully deleted {result.rowcount} sessions")
|
||||
|
||||
# Cascade offloaded tool results after the session delete commits.
|
||||
@@ -1510,6 +1532,73 @@ class SqliteDb(BaseDb):
|
||||
with self.Session() as sess:
|
||||
return [dict(row._mapping) for row in sess.execute(stmt).fetchall()]
|
||||
|
||||
# --- Compactions ---
|
||||
|
||||
def upsert_compaction(self, row: Dict[str, Any]) -> None:
|
||||
"""Insert one compaction record.
|
||||
|
||||
Records are immutable facts about a single fold, so a conflicting id means
|
||||
the same record is being written twice; the insert is a no-op rather than
|
||||
an update.
|
||||
"""
|
||||
table = self._get_table(table_type="compactions", create_table_if_not_found=True)
|
||||
if table is None:
|
||||
raise ValueError(f"Could not create table: {self.compactions_table_name}")
|
||||
from sqlalchemy.dialects.sqlite import insert as _insert
|
||||
|
||||
stmt = _insert(table).values(**row).on_conflict_do_nothing(index_elements=["compaction_id"])
|
||||
with self.Session() as sess, sess.begin():
|
||||
sess.execute(stmt)
|
||||
|
||||
def get_compaction(self, compaction_id: str) -> Optional[Dict[str, Any]]:
|
||||
table = self._get_table(table_type="compactions")
|
||||
if table is None:
|
||||
return None
|
||||
with self.Session() as sess:
|
||||
row = sess.execute(select(table).where(table.c.compaction_id == compaction_id)).fetchone()
|
||||
return dict(row._mapping) if row is not None else None
|
||||
|
||||
def get_compactions_for_session(self, session_id: str, limit: Optional[int] = None) -> List[Dict[str, Any]]:
|
||||
table = self._get_table(table_type="compactions")
|
||||
if table is None:
|
||||
return []
|
||||
stmt = select(table).where(table.c.session_id == session_id).order_by(table.c.created_at.desc())
|
||||
if limit is not None:
|
||||
stmt = stmt.limit(limit)
|
||||
with self.Session() as sess:
|
||||
return [dict(r._mapping) for r in sess.execute(stmt).fetchall()]
|
||||
|
||||
def delete_compactions_for_session(self, session_id: str) -> int:
|
||||
table = self._get_table(table_type="compactions")
|
||||
if table is None:
|
||||
return 0
|
||||
with self.Session() as sess, sess.begin():
|
||||
result = sess.execute(table.delete().where(table.c.session_id == session_id))
|
||||
return result.rowcount or 0
|
||||
|
||||
def search_compactions(
|
||||
self, session_id: str, query: Union[str, Sequence[str]], limit: int = 10
|
||||
) -> List[Dict[str, Any]]:
|
||||
"""Records whose archived transcript contains ``query`` - or any of several terms - newest first.
|
||||
|
||||
A substring match, scoped to one session so a search can never reach another conversation's
|
||||
history. Several terms are one query, so every fold matching any of them is a candidate.
|
||||
"""
|
||||
table = self._get_table(table_type="compactions")
|
||||
terms = [query] if isinstance(query, str) else list(query)
|
||||
terms = [term for term in terms if term]
|
||||
if table is None or not terms:
|
||||
return []
|
||||
stmt = (
|
||||
select(table)
|
||||
.where(table.c.session_id == session_id)
|
||||
.where(or_(*(table.c.archived_messages.ilike(f"%{term}%") for term in terms)))
|
||||
.order_by(table.c.created_at.desc())
|
||||
.limit(limit)
|
||||
)
|
||||
with self.Session() as sess:
|
||||
return [dict(r._mapping) for r in sess.execute(stmt).fetchall()]
|
||||
|
||||
def get_session(
|
||||
self,
|
||||
session_id: str,
|
||||
|
||||
@@ -145,6 +145,13 @@ class ModelProviderError(AgnoError):
|
||||
"prompt is too long",
|
||||
"prompt too long",
|
||||
"exceeds the model",
|
||||
# Gemini reports the count rather than the limit and never says "context window", in
|
||||
# more than one wording. "token count exceeds" matches the short forms ("input token count
|
||||
# exceeds the maximum"); the long form puts the count between those words ("The input token
|
||||
# count (1050000) exceeds the maximum number of tokens allowed"), so it needs the second
|
||||
# pattern. "token count" alone is not enough - it also appears in billing messages.
|
||||
"token count exceeds",
|
||||
"exceeds the maximum number of tokens",
|
||||
]
|
||||
|
||||
def __init__(
|
||||
|
||||
@@ -18,6 +18,7 @@ class ModelType(str, Enum):
|
||||
SESSION_SUMMARY_MODEL = "session_summary_model"
|
||||
LEARNING_MODEL = "learning_model"
|
||||
COMPRESSION_MODEL = "compression_model"
|
||||
COMPACTION_MODEL = "compaction_model"
|
||||
FOLLOWUP_MODEL = "followup_model"
|
||||
|
||||
|
||||
|
||||
@@ -3,7 +3,8 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass, field
|
||||
from typing import Any, AsyncIterator, Callable, Iterator, List, Optional, Union
|
||||
from inspect import isawaitable
|
||||
from typing import Any, AsyncIterator, Awaitable, Callable, Iterator, List, Optional, Union
|
||||
|
||||
from agno.exceptions import ContextWindowExceededError, ModelProviderError, ModelRateLimitError
|
||||
from agno.models.base import Model
|
||||
@@ -73,6 +74,50 @@ class FallbackConfig:
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _recovered_from_overflow(recover_from_overflow: Callable[[], bool]) -> bool:
|
||||
"""Run the overflow hook, reading any failure as "not recovered".
|
||||
|
||||
Recovery is an extra step in front of the normal handling, not a replacement for it. The hook
|
||||
runs inside the handler for the provider's error, so an exception from it would surface in
|
||||
place of that error and skip the fallback chain entirely.
|
||||
"""
|
||||
try:
|
||||
return bool(recover_from_overflow())
|
||||
except Exception as e: # noqa: BLE001 - recovery must never mask the provider's error
|
||||
log_warning(f"Could not shrink the request after a context-window error: {e}")
|
||||
return False
|
||||
|
||||
|
||||
async def _arecovered_from_overflow(recover_from_overflow: Callable[[], Union[bool, Awaitable[bool]]]) -> bool:
|
||||
"""Async variant of :func:`_recovered_from_overflow`.
|
||||
|
||||
Awaits a hook that returns an awaitable. Recovery calls the summarizer model, and run
|
||||
synchronously on the async path that call would block the event loop - every other request
|
||||
on the server waiting for it. A plain sync hook still works.
|
||||
"""
|
||||
try:
|
||||
result = recover_from_overflow()
|
||||
if isawaitable(result):
|
||||
result = await result
|
||||
return bool(result)
|
||||
except Exception as e: # noqa: BLE001 - recovery must never mask the provider's error
|
||||
log_warning(f"Could not shrink the request after a context-window error: {e}")
|
||||
return False
|
||||
|
||||
|
||||
def _is_context_overflow(error: Exception) -> bool:
|
||||
"""Whether this error means the request was too long, after classification.
|
||||
|
||||
Providers report it inconsistently, so the already-classified type is checked first and a
|
||||
generic ModelProviderError is classified as a fallback.
|
||||
"""
|
||||
if isinstance(error, ContextWindowExceededError):
|
||||
return True
|
||||
if isinstance(error, ModelProviderError):
|
||||
return isinstance(ModelProviderError.classify(error), ContextWindowExceededError)
|
||||
return False
|
||||
|
||||
|
||||
def get_fallback_models(fallback_config: Optional[FallbackConfig], error: Exception) -> Optional[List[Model]]:
|
||||
"""Return the appropriate fallback list for the given error.
|
||||
|
||||
@@ -158,6 +203,7 @@ def _sync_appended_messages(
|
||||
def call_model_with_fallback(
|
||||
model: Model,
|
||||
fallback_config: Optional[FallbackConfig],
|
||||
recover_from_overflow: Optional[Callable[[], bool]] = None,
|
||||
**kwargs: Any,
|
||||
) -> ModelResponse:
|
||||
"""Call the primary model, falling back on failure.
|
||||
@@ -167,9 +213,23 @@ def call_model_with_fallback(
|
||||
try:
|
||||
return model.response(**kwargs)
|
||||
except ModelProviderError as primary_error:
|
||||
# Shrinking the request and retrying the same model is cheaper than switching to a
|
||||
# larger one, and is the only response that works when no larger model is configured.
|
||||
# The fallback chain still runs if the retry fails, so the two compose.
|
||||
if (
|
||||
recover_from_overflow is not None
|
||||
and _is_context_overflow(primary_error)
|
||||
and _recovered_from_overflow(recover_from_overflow)
|
||||
):
|
||||
try:
|
||||
return model.response(**kwargs)
|
||||
except ModelProviderError as retry_error:
|
||||
primary_error = retry_error
|
||||
fallbacks = get_fallback_models(fallback_config, primary_error)
|
||||
if not fallbacks:
|
||||
raise
|
||||
# Not a bare raise: that re-raises the first error, and after a failed retry the
|
||||
# caller should see the retry's, which describes the request as it was last sent.
|
||||
raise primary_error
|
||||
log_warning(f"Primary model '{model.id}' failed. Trying fallback models...: {primary_error}")
|
||||
return _try_fallback_models(
|
||||
fallbacks,
|
||||
@@ -185,15 +245,27 @@ def call_model_with_fallback(
|
||||
async def acall_model_with_fallback(
|
||||
model: Model,
|
||||
fallback_config: Optional[FallbackConfig],
|
||||
recover_from_overflow: Optional[Callable[[], Union[bool, Awaitable[bool]]]] = None,
|
||||
**kwargs: Any,
|
||||
) -> ModelResponse:
|
||||
"""Async variant of call_model_with_fallback."""
|
||||
try:
|
||||
return await model.aresponse(**kwargs)
|
||||
except ModelProviderError as primary_error:
|
||||
if (
|
||||
recover_from_overflow is not None
|
||||
and _is_context_overflow(primary_error)
|
||||
and await _arecovered_from_overflow(recover_from_overflow)
|
||||
):
|
||||
try:
|
||||
return await model.aresponse(**kwargs)
|
||||
except ModelProviderError as retry_error:
|
||||
primary_error = retry_error
|
||||
fallbacks = get_fallback_models(fallback_config, primary_error)
|
||||
if not fallbacks:
|
||||
raise
|
||||
# Not a bare raise: that re-raises the first error, and after a failed retry the
|
||||
# caller should see the retry's, which describes the request as it was last sent.
|
||||
raise primary_error
|
||||
log_warning(f"Primary model '{model.id}' failed. Trying fallback models...: {primary_error}")
|
||||
return await _atry_fallback_models(
|
||||
fallbacks,
|
||||
@@ -214,15 +286,33 @@ async def acall_model_with_fallback(
|
||||
def call_model_stream_with_fallback(
|
||||
model: Model,
|
||||
fallback_config: Optional[FallbackConfig],
|
||||
recover_from_overflow: Optional[Callable[[], bool]] = None,
|
||||
**kwargs: Any,
|
||||
) -> Iterator[StreamEvent]:
|
||||
"""Call the primary model stream, falling back on failure."""
|
||||
"""Call the primary model stream, falling back on failure.
|
||||
|
||||
``recover_from_overflow`` shrinks the payload and retries the same model before any fallback,
|
||||
exactly as the non-streaming path does. A rejection for length arrives before the first
|
||||
chunk, so nothing has been yielded yet and the retry is safe to start from scratch.
|
||||
"""
|
||||
try:
|
||||
yield from model.response_stream(**kwargs)
|
||||
except ModelProviderError as primary_error:
|
||||
if (
|
||||
recover_from_overflow is not None
|
||||
and _is_context_overflow(primary_error)
|
||||
and _recovered_from_overflow(recover_from_overflow)
|
||||
):
|
||||
try:
|
||||
yield from model.response_stream(**kwargs)
|
||||
return
|
||||
except ModelProviderError as retry_error:
|
||||
primary_error = retry_error
|
||||
fallbacks = get_fallback_models(fallback_config, primary_error)
|
||||
if not fallbacks:
|
||||
raise
|
||||
# Not a bare raise: that re-raises the first error, and after a failed retry the
|
||||
# caller should see the retry's, which describes the request as it was last sent.
|
||||
raise primary_error
|
||||
log_warning(f"Primary model '{model.id}' failed. Trying fallback models...: {primary_error}")
|
||||
yield ModelResponse(event=ModelResponseEvent.fallback_model_activated.value)
|
||||
yield from _try_fallback_models_stream(
|
||||
@@ -238,6 +328,7 @@ def call_model_stream_with_fallback(
|
||||
async def acall_model_stream_with_fallback(
|
||||
model: Model,
|
||||
fallback_config: Optional[FallbackConfig],
|
||||
recover_from_overflow: Optional[Callable[[], Union[bool, Awaitable[bool]]]] = None,
|
||||
**kwargs: Any,
|
||||
) -> AsyncIterator[StreamEvent]:
|
||||
"""Async variant of call_model_stream_with_fallback."""
|
||||
@@ -245,9 +336,22 @@ async def acall_model_stream_with_fallback(
|
||||
async for event in model.aresponse_stream(**kwargs):
|
||||
yield event
|
||||
except ModelProviderError as primary_error:
|
||||
if (
|
||||
recover_from_overflow is not None
|
||||
and _is_context_overflow(primary_error)
|
||||
and await _arecovered_from_overflow(recover_from_overflow)
|
||||
):
|
||||
try:
|
||||
async for event in model.aresponse_stream(**kwargs):
|
||||
yield event
|
||||
return
|
||||
except ModelProviderError as retry_error:
|
||||
primary_error = retry_error
|
||||
fallbacks = get_fallback_models(fallback_config, primary_error)
|
||||
if not fallbacks:
|
||||
raise
|
||||
# Not a bare raise: that re-raises the first error, and after a failed retry the
|
||||
# caller should see the retry's, which describes the request as it was last sent.
|
||||
raise primary_error
|
||||
log_warning(f"Primary model '{model.id}' failed. Trying fallback models...: {primary_error}")
|
||||
yield ModelResponse(event=ModelResponseEvent.fallback_model_activated.value)
|
||||
async for event in _atry_fallback_models_stream(
|
||||
|
||||
@@ -353,6 +353,7 @@ class GeminiInteractions(Model):
|
||||
|
||||
The v2.x SDK uses a step_list format:
|
||||
- UserInputStep: {"type": "user_input", "content": [{"type": "text", "text": "..."}, ...]}
|
||||
- ModelOutputStep: {"type": "model_output", "content": [{"type": "text", "text": "..."}]}
|
||||
- FunctionCallStep: {"type": "function_call", "id": "...", "name": "...", "arguments": {...}}
|
||||
- FunctionResultStep: {"type": "function_result", "call_id": "...", "result": "...", "name": "..."}
|
||||
|
||||
@@ -411,8 +412,12 @@ class GeminiInteractions(Model):
|
||||
else:
|
||||
log_warning("Skipping user message with no usable content (all media may have failed to load)")
|
||||
|
||||
# Assistant messages with tool calls become FunctionCallSteps
|
||||
# Assistant text becomes a ModelOutputStep and tool calls become FunctionCallSteps.
|
||||
# Without the text, an unchained request (store=False, or a compacted history) would
|
||||
# show the model the user's turns but none of its own replies.
|
||||
elif message.role == "assistant":
|
||||
if message.content and isinstance(message.content, str):
|
||||
steps.append({"type": "model_output", "content": [{"type": "text", "text": message.content}]})
|
||||
if message.tool_calls:
|
||||
for tool_call in message.tool_calls:
|
||||
func = tool_call.get("function", {})
|
||||
|
||||
@@ -712,7 +712,12 @@ class OpenAIResponses(Model):
|
||||
fc_id_to_call_id = self._build_fc_id_to_call_id_map(messages)
|
||||
|
||||
for message in messages_to_format:
|
||||
# Without a response to chain from, replay reasoning before the assistant's text or function calls.
|
||||
# Without a response to chain from, replay reasoning before the assistant's text or
|
||||
# function calls. The chain is absent when the model is configured without it, and
|
||||
# also when it was severed at request time - compaction drops response_id from the
|
||||
# assistant copies it sends, so a fold leaves a reasoning model's function_call with
|
||||
# no server-held reasoning item. Either way the item has to travel with the call, or
|
||||
# the API rejects the pair outright.
|
||||
replayed_reasoning = False
|
||||
if (
|
||||
previous_response_id is None
|
||||
@@ -812,6 +817,10 @@ class OpenAIResponses(Model):
|
||||
output_schema: Optional[Union[Dict, Type[BaseModel]]] = None,
|
||||
) -> int:
|
||||
try:
|
||||
# A count covers the whole context. Stored response ids make formatting keep only what
|
||||
# follows the last response - right for a request that chains on it, but this call
|
||||
# carries no previous_response_id, so the input would come out empty.
|
||||
messages = self._without_response_ids(messages)
|
||||
formatted_input = self._format_messages(messages, compress_tool_results=True, tools=tools)
|
||||
formatted_tools = self._format_tool_params(messages, tools) if tools is not None else None
|
||||
|
||||
@@ -834,6 +843,10 @@ class OpenAIResponses(Model):
|
||||
) -> int:
|
||||
"""Async version of count_tokens using the async client."""
|
||||
try:
|
||||
# A count covers the whole context. Stored response ids make formatting keep only what
|
||||
# follows the last response - right for a request that chains on it, but this call
|
||||
# carries no previous_response_id, so the input would come out empty.
|
||||
messages = self._without_response_ids(messages)
|
||||
formatted_input = self._format_messages(messages, compress_tool_results=True, tools=tools)
|
||||
formatted_tools = self._format_tool_params(messages, tools) if tools else None
|
||||
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
import asyncio
|
||||
import contextlib
|
||||
import json
|
||||
from typing import TYPE_CHECKING, Any, AsyncGenerator, List, Literal, Optional, Union, cast
|
||||
from typing import TYPE_CHECKING, Any, AsyncGenerator, Dict, List, Literal, Optional, Union, cast
|
||||
from uuid import uuid4
|
||||
|
||||
from fastapi import (
|
||||
@@ -54,6 +54,7 @@ from agno.os.job_queue import (
|
||||
)
|
||||
from agno.os.middleware.user_scope import (
|
||||
SESSION_ID_REQUIRED,
|
||||
SESSION_NOT_FOUND,
|
||||
assert_session_matches_component,
|
||||
assert_session_writable,
|
||||
caller_is_admin,
|
||||
@@ -1829,6 +1830,118 @@ def get_agent_router(
|
||||
|
||||
return {"session_id": new_session_id, "forked_from_session_id": session_id}
|
||||
|
||||
@router.post(
|
||||
"/agents/{agent_id}/sessions/{session_id}/compact",
|
||||
tags=["Agents"],
|
||||
operation_id="compact_agent_session",
|
||||
summary="Compact Agent Session",
|
||||
description=(
|
||||
"Fold this session's older history into a summary now, without waiting for the "
|
||||
"context to reach ``compact_at_tokens``. The stored transcript is never modified - "
|
||||
"compaction shortens what is sent to the model, not the record.\n\n"
|
||||
"A compaction can legitimately decline, which is reported rather than raised. Check "
|
||||
"``compacted``, and show ``message`` to the user:\n"
|
||||
"- ``compacted`` - the fold happened; ``record`` carries the token counts\n"
|
||||
"- ``not_worth_it`` - with ``enforce_min_fold_ratio``, the span is too small to pay for "
|
||||
"the summary replacing it, so folding would leave the context bigger\n"
|
||||
"- ``nothing_to_fold`` - the kept tail covers the whole conversation\n"
|
||||
"- ``already_compacted`` - a previous fold already covers everything up to the only "
|
||||
"safe cut point\n"
|
||||
"- ``no_history`` - the session exists but has no stored history yet\n"
|
||||
"- ``not_enabled`` - compaction is not configured on this agent\n"
|
||||
"- ``summary_failed`` - the summarizer returned nothing"
|
||||
),
|
||||
responses={
|
||||
200: {
|
||||
"description": "Compaction attempted; see status for the outcome",
|
||||
"content": {
|
||||
"application/json": {
|
||||
"examples": {
|
||||
"compacted": {
|
||||
"summary": "The fold happened",
|
||||
"value": {
|
||||
"status": "compacted",
|
||||
"message": "Compacted 18 messages (15864 -> 3981 tokens).",
|
||||
"compacted": True,
|
||||
"record": {
|
||||
"messages_compacted": 18,
|
||||
"tokens_before": 15864,
|
||||
"tokens_after": 3981,
|
||||
},
|
||||
},
|
||||
},
|
||||
"declined": {
|
||||
"summary": "Declined - folding would not help",
|
||||
"value": {
|
||||
"status": "not_worth_it",
|
||||
"message": (
|
||||
"This fold would replace 1200 tokens against a 900-token tail "
|
||||
"(ratio 1.33, needs 2.0), so the context would not shrink. "
|
||||
"Continue the conversation, or lower uncompacted_runs or "
|
||||
"min_fold_ratio, or set enforce_min_fold_ratio=False, to fold "
|
||||
"sooner."
|
||||
),
|
||||
"compacted": False,
|
||||
"record": None,
|
||||
},
|
||||
},
|
||||
}
|
||||
}
|
||||
},
|
||||
},
|
||||
404: {"description": "Agent or session not found", "model": NotFoundResponse},
|
||||
},
|
||||
dependencies=[Depends(require_resource_access("agents", "run", "agent_id"))],
|
||||
)
|
||||
async def compact_agent_session(
|
||||
agent_id: str,
|
||||
session_id: str,
|
||||
request: Request,
|
||||
user_id: Optional[str] = None,
|
||||
) -> Dict[str, Any]:
|
||||
if hasattr(request.state, "user_id") and request.state.user_id is not None:
|
||||
user_id = request.state.user_id
|
||||
|
||||
try:
|
||||
agent = get_agent_by_id(
|
||||
agent_id=agent_id,
|
||||
agents=os.agents,
|
||||
db=os.db,
|
||||
registry=os.registry,
|
||||
create_fresh=True,
|
||||
user_id=get_scoped_user_id(request),
|
||||
strict=False,
|
||||
published_only=False,
|
||||
)
|
||||
except HTTPException:
|
||||
raise
|
||||
except Exception as e:
|
||||
log_error(f"Error resolving agent '{agent_id}': {e}")
|
||||
raise HTTPException(status_code=500, detail="Internal server error")
|
||||
if agent is None:
|
||||
raise HTTPException(status_code=404, detail="Agent not found")
|
||||
|
||||
# Scope the session read to the caller, so one user cannot compact another's session.
|
||||
scoped_user_id = get_scoped_user_id(request)
|
||||
effective_user_id = scoped_user_id or user_id
|
||||
|
||||
# The session must belong to this agent: the per-resource gate authorised agent_id, not
|
||||
# whichever session id the client named.
|
||||
await verify_run_belongs_to_component(
|
||||
request,
|
||||
getattr(agent, "db", None) or os.db,
|
||||
component_type="agents",
|
||||
component_id=agent_id,
|
||||
session_id=session_id,
|
||||
)
|
||||
# A missing session is a 404, as on the other session routes. no_history is kept for a
|
||||
# session that exists but has nothing to fold yet.
|
||||
if await agent.aget_session(session_id=session_id, user_id=effective_user_id) is None: # type: ignore[union-attr]
|
||||
raise HTTPException(status_code=404, detail=SESSION_NOT_FOUND)
|
||||
|
||||
result = await agent.acompact(session_id=session_id, user_id=effective_user_id) # type: ignore[union-attr]
|
||||
return result.to_dict()
|
||||
|
||||
@router.get(
|
||||
"/agents",
|
||||
response_model=List[AgentResponse],
|
||||
|
||||
@@ -23,6 +23,7 @@ from agno.utils.media import (
|
||||
)
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from agno.compaction.types import CompactionRecord
|
||||
from agno.session.summary import SessionSummary
|
||||
|
||||
|
||||
@@ -188,6 +189,9 @@ class RunEvent(str, Enum):
|
||||
compression_started = "CompressionStarted"
|
||||
compression_completed = "CompressionCompleted"
|
||||
|
||||
compaction_started = "CompactionStarted"
|
||||
compaction_completed = "CompactionCompleted"
|
||||
|
||||
followups_started = "FollowupsStarted"
|
||||
followups_completed = "FollowupsCompleted"
|
||||
|
||||
@@ -499,6 +503,24 @@ class CompressionCompletedEvent(BaseAgentRunEvent):
|
||||
compressed_size: Optional[int] = None
|
||||
|
||||
|
||||
@dataclass
|
||||
class CompactionStartedEvent(BaseAgentRunEvent):
|
||||
"""Event sent when conversation compaction is about to start"""
|
||||
|
||||
event: str = RunEvent.compaction_started.value
|
||||
|
||||
|
||||
@dataclass
|
||||
class CompactionCompletedEvent(BaseAgentRunEvent):
|
||||
"""Event sent when conversation compaction has completed"""
|
||||
|
||||
event: str = RunEvent.compaction_completed.value
|
||||
messages_compacted: Optional[int] = None
|
||||
tokens_before: Optional[int] = None
|
||||
tokens_after: Optional[int] = None
|
||||
archived: Optional[bool] = None
|
||||
|
||||
|
||||
@dataclass
|
||||
class FollowupsStartedEvent(BaseAgentRunEvent):
|
||||
event: str = RunEvent.followups_started.value
|
||||
@@ -555,6 +577,8 @@ RunOutputEvent = Union[
|
||||
ModelRequestCompletedEvent,
|
||||
CompressionStartedEvent,
|
||||
CompressionCompletedEvent,
|
||||
CompactionStartedEvent,
|
||||
CompactionCompletedEvent,
|
||||
FollowupsStartedEvent,
|
||||
FollowupsCompletedEvent,
|
||||
CustomEvent,
|
||||
@@ -600,6 +624,8 @@ RUN_EVENT_TYPE_REGISTRY = {
|
||||
RunEvent.model_request_completed.value: ModelRequestCompletedEvent,
|
||||
RunEvent.compression_started.value: CompressionStartedEvent,
|
||||
RunEvent.compression_completed.value: CompressionCompletedEvent,
|
||||
RunEvent.compaction_started.value: CompactionStartedEvent,
|
||||
RunEvent.compaction_completed.value: CompactionCompletedEvent,
|
||||
RunEvent.followups_started.value: FollowupsStartedEvent,
|
||||
RunEvent.followups_completed.value: FollowupsCompletedEvent,
|
||||
RunEvent.custom_event.value: CustomEvent,
|
||||
@@ -642,6 +668,8 @@ class RunOutput:
|
||||
model_provider: Optional[str] = None
|
||||
messages: Optional[List[Message]] = None
|
||||
metrics: Optional[RunMetrics] = None
|
||||
# What compaction did to this run's context, when it ran.
|
||||
compaction: Optional["CompactionRecord"] = None
|
||||
additional_input: Optional[List[Message]] = None
|
||||
|
||||
tools: Optional[List[ToolExecution]] = None
|
||||
@@ -731,6 +759,7 @@ class RunOutput:
|
||||
_HAND_SERIALIZED_FIELDS = (
|
||||
"messages",
|
||||
"metrics",
|
||||
"compaction",
|
||||
"tools",
|
||||
"metadata",
|
||||
"images",
|
||||
@@ -765,6 +794,9 @@ class RunOutput:
|
||||
if self.metrics is not None:
|
||||
_dict["metrics"] = self.metrics.to_dict() if isinstance(self.metrics, RunMetrics) else self.metrics
|
||||
|
||||
if self.compaction is not None:
|
||||
_dict["compaction"] = self.compaction.to_dict() if hasattr(self.compaction, "to_dict") else self.compaction
|
||||
|
||||
if self.events is not None:
|
||||
_dict["events"] = [e.to_dict() for e in self.events]
|
||||
|
||||
@@ -926,6 +958,12 @@ class RunOutput:
|
||||
if metrics:
|
||||
metrics = RunMetrics.from_dict(metrics)
|
||||
|
||||
compaction = data.pop("compaction", None)
|
||||
if compaction:
|
||||
from agno.compaction.types import CompactionRecord
|
||||
|
||||
compaction = CompactionRecord.from_dict(compaction)
|
||||
|
||||
additional_input = data.pop("additional_input", None)
|
||||
|
||||
if additional_input is not None:
|
||||
@@ -955,6 +993,7 @@ class RunOutput:
|
||||
return cls(
|
||||
messages=messages,
|
||||
metrics=metrics,
|
||||
compaction=compaction,
|
||||
citations=citations,
|
||||
tools=tools,
|
||||
images=images,
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
from dataclasses import dataclass, field
|
||||
from typing import List, Optional
|
||||
from typing import Any, List, Optional
|
||||
|
||||
from agno.models.message import Message
|
||||
|
||||
@@ -13,12 +13,16 @@ class RunMessages:
|
||||
system_message: The system message for this run
|
||||
user_message: The user message for this run
|
||||
extra_messages: Extra messages added after the system and user messages
|
||||
events: Events raised while assembling these messages, for the caller
|
||||
to emit. Assembly is not a generator, so anything worth streaming
|
||||
(compaction, for one) is collected here instead of yielded.
|
||||
"""
|
||||
|
||||
messages: List[Message] = field(default_factory=list)
|
||||
system_message: Optional[Message] = None
|
||||
user_message: Optional[Message] = None
|
||||
extra_messages: Optional[List[Message]] = None
|
||||
events: List[Any] = field(default_factory=list)
|
||||
|
||||
def get_input_messages(self) -> List[Message]:
|
||||
"""Get the input messages for the model."""
|
||||
|
||||
@@ -5,6 +5,8 @@ from agno.models.message import Citations
|
||||
from agno.models.response import ToolExecution
|
||||
from agno.reasoning.step import ReasoningStep
|
||||
from agno.run.agent import (
|
||||
CompactionCompletedEvent,
|
||||
CompactionStartedEvent,
|
||||
CompressionCompletedEvent,
|
||||
CompressionStartedEvent,
|
||||
FollowupsCompletedEvent,
|
||||
@@ -971,6 +973,36 @@ def create_compression_completed_event(
|
||||
)
|
||||
|
||||
|
||||
def create_compaction_started_event(
|
||||
from_run_response: RunOutput,
|
||||
) -> CompactionStartedEvent:
|
||||
return CompactionStartedEvent(
|
||||
session_id=from_run_response.session_id,
|
||||
agent_id=from_run_response.agent_id, # type: ignore
|
||||
agent_name=from_run_response.agent_name, # type: ignore
|
||||
run_id=from_run_response.run_id,
|
||||
)
|
||||
|
||||
|
||||
def create_compaction_completed_event(
|
||||
from_run_response: RunOutput,
|
||||
messages_compacted: Optional[int] = None,
|
||||
tokens_before: Optional[int] = None,
|
||||
tokens_after: Optional[int] = None,
|
||||
archived: Optional[bool] = None,
|
||||
) -> CompactionCompletedEvent:
|
||||
return CompactionCompletedEvent(
|
||||
session_id=from_run_response.session_id,
|
||||
agent_id=from_run_response.agent_id, # type: ignore
|
||||
agent_name=from_run_response.agent_name, # type: ignore
|
||||
run_id=from_run_response.run_id,
|
||||
messages_compacted=messages_compacted,
|
||||
tokens_before=tokens_before,
|
||||
tokens_after=tokens_after,
|
||||
archived=archived,
|
||||
)
|
||||
|
||||
|
||||
def create_team_compression_started_event(
|
||||
from_run_response: TeamRunOutput,
|
||||
) -> TeamCompressionStartedEvent:
|
||||
|
||||
@@ -292,6 +292,9 @@ def test_the_approval_gate_lets_a_run_continue_on_a_database_with_no_approvals_t
|
||||
|
||||
def test_error_responses_do_not_reflect_an_origin_without_an_allow_list():
|
||||
middleware = AuthMiddleware.__new__(AuthMiddleware)
|
||||
# Built without __init__, so the shared origin policy it would have set is absent: the
|
||||
# per-call allow list is what this test exercises.
|
||||
middleware.cors_origin_policy = None
|
||||
resp = AuthMiddleware._create_error_response(
|
||||
middleware, 401, "no", origin="https://evil.example", cors_allowed_origins=None
|
||||
)
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -143,12 +143,26 @@ class TestBuildInput:
|
||||
assert len(steps) == 1
|
||||
assert steps[0]["type"] == "user_input"
|
||||
|
||||
def test_assistant_message_without_tool_calls_skipped(self):
|
||||
"""Assistant messages without tool calls produce no steps (text responses are not re-sent)."""
|
||||
def test_assistant_text_becomes_model_output(self):
|
||||
"""An unchained request (store=False, or a compacted history) must carry the model's own
|
||||
replies, or it sees the user's turns with none of its answers."""
|
||||
model = self._make_model()
|
||||
messages = [Message(role="assistant", content="I can help")]
|
||||
steps = model._build_input(messages)
|
||||
assert len(steps) == 0
|
||||
assert steps == [{"type": "model_output", "content": [{"type": "text", "text": "I can help"}]}]
|
||||
|
||||
def test_chained_request_does_not_resend_the_prior_reply(self):
|
||||
"""With a previous_interaction_id the server holds the prior turns, so only what follows
|
||||
the last chained reply is sent."""
|
||||
model = self._make_model()
|
||||
messages = [
|
||||
Message(role="user", content="q0"),
|
||||
Message(role="assistant", content="a0", provider_data={"interaction_id": "int_0"}),
|
||||
Message(role="user", content="q1"),
|
||||
]
|
||||
kwargs = model._get_request_kwargs(messages)
|
||||
assert kwargs["previous_interaction_id"] == "int_0"
|
||||
assert [step["type"] for step in kwargs["input"]] == ["user_input"]
|
||||
|
||||
def test_tool_call_message(self):
|
||||
model = self._make_model()
|
||||
|
||||
@@ -291,3 +291,34 @@ async def test_missing_previous_response_is_not_retried_without_a_chain(stale_ch
|
||||
await _invoke(model, "sync", [Message(role="user", content="hi")])
|
||||
|
||||
assert len(requests) == 1
|
||||
|
||||
|
||||
@pytest.mark.parametrize("mode", ["sync", "async"])
|
||||
async def test_count_tokens_counts_the_whole_history_despite_stored_response_ids(mode):
|
||||
"""A count call carries no previous_response_id, so it must not drop what a chained request
|
||||
would leave to the server - otherwise the input is empty and the API rejects it."""
|
||||
requests = []
|
||||
|
||||
def handle(request):
|
||||
requests.append((request.url.path, json.loads(request.content)))
|
||||
return httpx.Response(200, json={"object": "response.input_tokens", "input_tokens": 123})
|
||||
|
||||
transport = httpx.MockTransport(handle)
|
||||
messages = [
|
||||
Message(role="user", content="first question"),
|
||||
Message(role="assistant", content="first answer", provider_data={"response_id": "resp_1"}),
|
||||
Message(role="user", content="second question"),
|
||||
]
|
||||
with OpenAI(api_key="test", http_client=httpx.Client(transport=transport)) as client:
|
||||
async with AsyncOpenAI(api_key="test", http_client=httpx.AsyncClient(transport=transport)) as async_client:
|
||||
model = OpenAIResponses(id="gpt-5.6-luna", client=client, async_client=async_client)
|
||||
if mode == "sync":
|
||||
counted = model.count_tokens(messages)
|
||||
else:
|
||||
counted = await model.acount_tokens(messages)
|
||||
|
||||
assert counted == 123
|
||||
path, payload = requests[0]
|
||||
assert path.endswith("/responses/input_tokens")
|
||||
assert "previous_response_id" not in payload
|
||||
assert [item["content"] for item in payload["input"]] == ["first question", "first answer", "second question"]
|
||||
|
||||
@@ -4,6 +4,51 @@ import pytest
|
||||
|
||||
from agno.exceptions import ContextWindowExceededError, ModelProviderError
|
||||
|
||||
# =============================================================================
|
||||
# Gemini context window patterns
|
||||
# =============================================================================
|
||||
|
||||
|
||||
class TestGeminiContextWindowPatterns:
|
||||
"""Gemini reports the count rather than the limit and never says "context window".
|
||||
|
||||
Its wording matched none of the original patterns, so an oversized Gemini request was
|
||||
never classified - and anything keyed on ContextWindowExceededError, including compaction's
|
||||
reactive fold, silently did nothing.
|
||||
"""
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"error_message",
|
||||
[
|
||||
"The input token count (1050000) exceeds the maximum number of tokens allowed (1048575).",
|
||||
"400 INVALID_ARGUMENT. Request contains an invalid argument: input token count exceeds the maximum",
|
||||
"Token count exceeds the maximum allowed for this model.",
|
||||
],
|
||||
ids=[
|
||||
"gemini_input_token_count_with_limit",
|
||||
"gemini_invalid_argument_wrapper",
|
||||
"gemini_capitalized_short_form",
|
||||
],
|
||||
)
|
||||
def test_classify_gemini_context_window_errors(self, error_message: str):
|
||||
"""Gemini context window errors should be classified as ContextWindowExceededError."""
|
||||
error = ModelProviderError(message=error_message, status_code=400)
|
||||
classified = ModelProviderError.classify(error)
|
||||
|
||||
assert isinstance(classified, ContextWindowExceededError)
|
||||
assert classified.message == error_message
|
||||
|
||||
def test_token_count_alone_does_not_match(self):
|
||||
""" "token count" on its own appears in billing and usage messages.
|
||||
|
||||
Both halves are required, or an unrelated error would be treated as an oversized
|
||||
request and trigger a fold that cannot help.
|
||||
"""
|
||||
error = ModelProviderError(message="Billing: your token count for this month is 1.2M", status_code=400)
|
||||
|
||||
assert not isinstance(ModelProviderError.classify(error), ContextWindowExceededError)
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# Anthropic context window patterns
|
||||
# =============================================================================
|
||||
|
||||
@@ -0,0 +1,98 @@
|
||||
"""The compaction route's contract: a decline is a 200 with a status, a missing session is a 404."""
|
||||
|
||||
import tempfile
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
from fastapi import FastAPI
|
||||
from fastapi.testclient import TestClient
|
||||
|
||||
from agno.agent import Agent
|
||||
from agno.compaction import Compaction
|
||||
from agno.db.sqlite import SqliteDb
|
||||
from agno.os import AgentOS
|
||||
from agno.os.routers.agents.router import get_agent_router
|
||||
from agno.session import AgentSession
|
||||
|
||||
|
||||
def _client(authorization: bool = False) -> TestClient:
|
||||
db = SqliteDb(db_file=str(Path(tempfile.mkdtemp()) / "compaction.db"))
|
||||
db.upsert_session(AgentSession(session_id="s1", agent_id="a1", runs=[]))
|
||||
db.upsert_session(AgentSession(session_id="s2", agent_id="a2", runs=[]))
|
||||
agents = [Agent(id=agent_id, db=db, compaction=Compaction()) for agent_id in ("a1", "a2")]
|
||||
app = FastAPI()
|
||||
if authorization:
|
||||
|
||||
@app.middleware("http")
|
||||
async def enable_authorization(request, call_next):
|
||||
request.state.authorization_enabled = True
|
||||
return await call_next(request)
|
||||
|
||||
app.include_router(get_agent_router(AgentOS(agents=agents)))
|
||||
return TestClient(app)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def client() -> TestClient:
|
||||
return _client()
|
||||
|
||||
|
||||
def test_route_is_registered():
|
||||
db = SqliteDb(db_file=str(Path(tempfile.mkdtemp()) / "compaction.db"))
|
||||
agent_os = AgentOS(agents=[Agent(id="a1", db=db, compaction=Compaction())])
|
||||
|
||||
paths = {(route.path, tuple(sorted(route.methods))) for route in get_agent_router(agent_os).routes}
|
||||
|
||||
assert ("/agents/{agent_id}/sessions/{session_id}/compact", ("POST",)) in paths
|
||||
|
||||
|
||||
def test_decline_is_a_200_with_a_reason(client: TestClient):
|
||||
"""Declining to fold is a normal outcome, so a UI must not have to treat it as an error.
|
||||
|
||||
Raising would make "folding would not help here" indistinguishable from a failure.
|
||||
"""
|
||||
response = client.post("/agents/a1/sessions/s1/compact")
|
||||
|
||||
assert response.status_code == 200
|
||||
body = response.json()
|
||||
assert body["compacted"] is False
|
||||
assert body["status"] == "no_history"
|
||||
assert body["message"] # always something displayable
|
||||
assert body["record"] is None
|
||||
|
||||
|
||||
def test_unknown_session_is_a_404(client: TestClient):
|
||||
"""A session that does not exist is a 404, as on the other session routes - not no_history,
|
||||
which would read as "exists, nothing to fold yet"."""
|
||||
response = client.post("/agents/a1/sessions/does-not-exist/compact")
|
||||
|
||||
assert response.status_code == 404
|
||||
|
||||
|
||||
def test_another_agents_session_is_a_404(monkeypatch):
|
||||
"""The resource gate authorised a1, so a1's route must not fold a2's session."""
|
||||
|
||||
async def may_run_a1_only(request, resource_id, resource_type, action):
|
||||
return resource_id == "a1"
|
||||
|
||||
monkeypatch.setattr("agno.os.auth.acheck_resource_access", may_run_a1_only)
|
||||
client = _client(authorization=True)
|
||||
|
||||
assert client.post("/agents/a1/sessions/s1/compact").status_code == 200
|
||||
response = client.post("/agents/a1/sessions/s2/compact")
|
||||
|
||||
assert response.status_code == 404
|
||||
|
||||
|
||||
def test_unknown_agent_is_a_404(client: TestClient):
|
||||
"""A missing agent IS an error - it means the caller asked for something that does not exist."""
|
||||
response = client.post("/agents/nope/sessions/s1/compact")
|
||||
|
||||
assert response.status_code == 404
|
||||
|
||||
|
||||
def test_response_shape_is_stable(client: TestClient):
|
||||
"""These keys are the contract the FE renders; adding is safe, renaming is not."""
|
||||
body = client.post("/agents/a1/sessions/s1/compact").json()
|
||||
|
||||
assert set(body) == {"status", "message", "compacted", "record", "metrics"}
|
||||
@@ -775,3 +775,193 @@ class TestEdgeCases:
|
||||
"""FallbackConfig callback defaults to None."""
|
||||
config = FallbackConfig()
|
||||
assert config.callback is None
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# Overflow recovery must never mask the provider's error
|
||||
# =============================================================================
|
||||
|
||||
|
||||
class TestFailingOverflowRecovery:
|
||||
"""The recovery hook runs inside the handler for the provider's error. If it raises, the
|
||||
provider's error and the fallback chain must still run as if recovery had not been tried."""
|
||||
|
||||
@staticmethod
|
||||
def _broken_hook() -> bool:
|
||||
raise ValueError("bad config")
|
||||
|
||||
def test_the_providers_error_is_raised_not_the_hooks(self):
|
||||
primary = _make_model("primary")
|
||||
with patch.object(primary, "response", side_effect=ContextWindowExceededError("too long")):
|
||||
with pytest.raises(ContextWindowExceededError):
|
||||
call_model_with_fallback(primary, None, recover_from_overflow=self._broken_hook, messages=[])
|
||||
|
||||
def test_the_fallback_chain_still_runs(self):
|
||||
primary = _make_model("primary")
|
||||
fallback = _make_model("fallback")
|
||||
config = FallbackConfig(on_context_overflow=[fallback])
|
||||
with patch.object(primary, "response", side_effect=ContextWindowExceededError("too long")):
|
||||
with patch.object(fallback, "response", return_value=ModelResponse(content="fallback-ok")):
|
||||
result = call_model_with_fallback(primary, config, recover_from_overflow=self._broken_hook, messages=[])
|
||||
assert result.content == "fallback-ok"
|
||||
|
||||
def test_streaming_and_async_paths_behave_the_same(self):
|
||||
import asyncio
|
||||
|
||||
primary = _make_model("primary")
|
||||
error = ContextWindowExceededError("too long")
|
||||
|
||||
def stream(**kwargs):
|
||||
raise error
|
||||
yield # pragma: no cover
|
||||
|
||||
async def aresponse(**kwargs):
|
||||
raise error
|
||||
|
||||
async def astream(**kwargs):
|
||||
raise error
|
||||
yield # pragma: no cover
|
||||
|
||||
async def consume_async_stream():
|
||||
async for _ in acall_model_stream_with_fallback(
|
||||
primary, None, recover_from_overflow=self._broken_hook, messages=[]
|
||||
):
|
||||
pass
|
||||
|
||||
with patch.object(primary, "response_stream", side_effect=stream):
|
||||
with pytest.raises(ContextWindowExceededError):
|
||||
list(
|
||||
call_model_stream_with_fallback(primary, None, recover_from_overflow=self._broken_hook, messages=[])
|
||||
)
|
||||
with patch.object(primary, "aresponse", side_effect=aresponse):
|
||||
with pytest.raises(ContextWindowExceededError):
|
||||
asyncio.run(
|
||||
acall_model_with_fallback(primary, None, recover_from_overflow=self._broken_hook, messages=[])
|
||||
)
|
||||
with patch.object(primary, "aresponse_stream", side_effect=astream):
|
||||
with pytest.raises(ContextWindowExceededError):
|
||||
asyncio.run(consume_async_stream())
|
||||
|
||||
|
||||
class TestAsyncOverflowHook:
|
||||
"""The async wrappers await a hook that returns an awaitable, so recovery - a summarizer call -
|
||||
never blocks the event loop. A sync hook keeps working."""
|
||||
|
||||
def test_an_async_hook_is_awaited_and_the_call_retried(self):
|
||||
import asyncio
|
||||
|
||||
primary = _make_model("primary")
|
||||
recovered = []
|
||||
|
||||
async def hook():
|
||||
await asyncio.sleep(0)
|
||||
recovered.append(True)
|
||||
return True
|
||||
|
||||
responses = [ContextWindowExceededError("too long"), ModelResponse(content="ok")]
|
||||
|
||||
async def aresponse(**kwargs):
|
||||
outcome = responses.pop(0)
|
||||
if isinstance(outcome, Exception):
|
||||
raise outcome
|
||||
return outcome
|
||||
|
||||
with patch.object(primary, "aresponse", side_effect=aresponse):
|
||||
result = asyncio.run(acall_model_with_fallback(primary, None, recover_from_overflow=hook, messages=[]))
|
||||
|
||||
assert result.content == "ok"
|
||||
assert recovered == [True]
|
||||
|
||||
def test_a_sync_hook_still_works_on_the_async_path(self):
|
||||
import asyncio
|
||||
|
||||
primary = _make_model("primary")
|
||||
responses = [ContextWindowExceededError("too long"), ModelResponse(content="ok")]
|
||||
|
||||
async def aresponse(**kwargs):
|
||||
outcome = responses.pop(0)
|
||||
if isinstance(outcome, Exception):
|
||||
raise outcome
|
||||
return outcome
|
||||
|
||||
with patch.object(primary, "aresponse", side_effect=aresponse):
|
||||
result = asyncio.run(
|
||||
acall_model_with_fallback(primary, None, recover_from_overflow=lambda: True, messages=[])
|
||||
)
|
||||
|
||||
assert result.content == "ok"
|
||||
|
||||
def test_a_failing_async_hook_does_not_mask_the_providers_error(self):
|
||||
import asyncio
|
||||
|
||||
primary = _make_model("primary")
|
||||
|
||||
async def broken():
|
||||
raise ValueError("bad config")
|
||||
|
||||
with patch.object(primary, "aresponse", side_effect=ContextWindowExceededError("too long")):
|
||||
with pytest.raises(ContextWindowExceededError):
|
||||
asyncio.run(acall_model_with_fallback(primary, None, recover_from_overflow=broken, messages=[]))
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# A failed retry after overflow recovery raises the retry's error
|
||||
# =============================================================================
|
||||
|
||||
|
||||
class TestFailedOverflowRetry:
|
||||
"""After recovery shrinks the request and the retry still overflows, the caller must see the
|
||||
retry's error - it describes the request as last sent, not the one before the fold."""
|
||||
|
||||
first = ContextWindowExceededError("input is 300000 tokens")
|
||||
second = ContextWindowExceededError("input is 210000 tokens")
|
||||
|
||||
def test_sync(self):
|
||||
primary = _make_model("primary")
|
||||
with patch.object(primary, "response", side_effect=[self.first, self.second]):
|
||||
with pytest.raises(ContextWindowExceededError) as raised:
|
||||
call_model_with_fallback(primary, None, recover_from_overflow=lambda: True, messages=[])
|
||||
assert raised.value is self.second
|
||||
|
||||
def test_async(self):
|
||||
import asyncio
|
||||
|
||||
primary = _make_model("primary")
|
||||
with patch.object(primary, "aresponse", AsyncMock(side_effect=[self.first, self.second])):
|
||||
with pytest.raises(ContextWindowExceededError) as raised:
|
||||
asyncio.run(acall_model_with_fallback(primary, None, recover_from_overflow=lambda: True, messages=[]))
|
||||
assert raised.value is self.second
|
||||
|
||||
def test_stream(self):
|
||||
primary = _make_model("primary")
|
||||
errors = iter([self.first, self.second])
|
||||
|
||||
def stream(**kwargs):
|
||||
raise next(errors)
|
||||
yield # pragma: no cover
|
||||
|
||||
with patch.object(primary, "response_stream", side_effect=stream):
|
||||
with pytest.raises(ContextWindowExceededError) as raised:
|
||||
list(call_model_stream_with_fallback(primary, None, recover_from_overflow=lambda: True, messages=[]))
|
||||
assert raised.value is self.second
|
||||
|
||||
def test_async_stream(self):
|
||||
import asyncio
|
||||
|
||||
primary = _make_model("primary")
|
||||
errors = iter([self.first, self.second])
|
||||
|
||||
async def astream(**kwargs):
|
||||
raise next(errors)
|
||||
yield # pragma: no cover
|
||||
|
||||
async def consume():
|
||||
async for _ in acall_model_stream_with_fallback(
|
||||
primary, None, recover_from_overflow=lambda: True, messages=[]
|
||||
):
|
||||
pass
|
||||
|
||||
with patch.object(primary, "aresponse_stream", side_effect=astream):
|
||||
with pytest.raises(ContextWindowExceededError) as raised:
|
||||
asyncio.run(consume())
|
||||
assert raised.value is self.second
|
||||
|
||||
Reference in New Issue
Block a user