When the streaming tool-selection decision returned no tool calls and empty
content on every attempt, the retry loop exhausted with lastErr still nil, so
the final fmt.Errorf("...: %w", maxRetries, lastErr) rendered the literal
"%!w(<nil>)" — hiding the real cause.
This reliably bites image turns with reasoning models (e.g. gemma-4-e2b via
LocalAI): the model streams reasoning but its reasoning exhausts the output
budget before any visible content, finishing with finish_reason=length and no
content. The empty-content path then retried three identical truncations and
reported %!w(<nil>).
Capture finish_reason from StreamEventDone and, in the empty-content branch:
- finish_reason=length: fail fast with an actionable error (retrying
truncates identically) instead of looping into a nil wrap.
- otherwise: keep retrying, but always record a real lastErr so the final
error is never a nil wrap.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
usableTools returned an error if ANY mcpSession's ListTools/ListPrompts failed,
so a single session that is momentarily unavailable (mid-reconnect, or being
closed/rebuilt by a config reload — surfaced by the go-sdk as "client is closing:
hanging GET: failed to reconnect") aborted the whole turn. This is what broke
/learn in dante-desktop. Skip the unhealthy session for this turn and log a
warning; its tools return once it is healthy again.
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Completes parity with OpenAIClient so wiz's per-agent-type LLM
factory (which passes temperature) can switch client
implementations without losing the override.
Gives LocalAIClient parity with OpenAIClient's reasoning_effort
passthrough so callers can switch between the two client
implementations without losing that lever — needed because
LocalAIClient is the client that actually parses LocalAI/vLLM's
"reasoning" response field (OpenAIClient only knows go-openai's
deprecated "reasoning_content" key, so it silently drops it).
The no-tool reply the model produces right before a park gate was only
recorded inside the fragment; embedders had no way to read it at park
time. nib worked around this by capturing the reasoning callback, but
toolSelection fires that callback with resp.ReasoningContent (reasoning
tokens, empty for most models), not the reply text — so parked replies
(e.g. the answer to a mid-run injected user message) were silently
dropped by the UI.
WithOnPark now passes the assistant reply text that preceded the park:
the no-tool reply at the agents-still-running gate, or "" at the
sink-state gate where the reply is produced after the loop.
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Introduce an opt-in seam so embedders can execute spawned sub-agents
out-of-process (e.g. dispatch to a remote worker) while cogito keeps
owning all lifecycle bookkeeping: registration, status, done, callbacks,
completion injection, and detach. A nil dispatcher preserves the existing
in-process behavior exactly; ErrDispatchFallback lets a dispatcher defer
a given spawn back to the in-process path.
New API: AgentRunSpec, AgentEvent, AgentDispatcher, ErrDispatchFallback,
WithAgentDispatcher. runAgent branches on the dispatcher; spawn builds an
AgentRunSpec (resolved persona/tools/model) and an optional Emit stream.
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add OpenAIOptions.ReasoningEffort, attached to every chat-completion request as
the OpenAI-standard "reasoning_effort" field (Ask, CreateChatCompletion,
CreateChatCompletionStream). Mirrors the existing Metadata passthrough.
Why: Metadata ({"enable_thinking":"false"}) only disables thinking when the
model's chat template honors that flag. Reasoning models whose GGUF template has
no enable_thinking toggle (e.g. LFM2.5) ignore it and keep reasoning — but they
DO honor reasoning_effort:"none". This gives callers the portable lever to
silence reasoning regardless of the template. Unset = field omitted (no behavior
change).
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add an optional metadata object that is attached verbatim to every
chat-completion request as the OpenAI "metadata" field. Backends such as
LocalAI use it for per-request flags, e.g. {"enable_thinking": "false"}
to disable a reasoning model's thinking.
- OpenAIOptions.Metadata + OpenAIClient.metadata, applied on all call
paths: Ask, CreateChatCompletion, CreateChatCompletionStream.
- AgentDefinition.Metadata for per-agent-type overrides; resolveLLM now
passes it to the agent LLM factory and fires the factory on a
metadata-only override (previously only a model override did).
- WithAgentLLMFactory signature gains a metadata map (breaking change to
the factory type).
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(agent): expose AgentState.Background for spawned background agents
Add an exported Background flag on AgentState, set true for background
spawns (spawn_agent background=true) and false for foreground ones. This
lets embedders tell unattended background work apart from a foreground
sub-agent whose result is consumed inline — e.g. to auto-notify on
completion only for background agents.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* feat: add per-run token usage accumulator (CumulativeUsage)
Adds a counting LLM decorator and a CumulativeUsage field on Status so a
full ExecuteTools run's token usage can be summed and exposed. Preserves
StreamingLLM so wrapping does not disable streaming.
* refactor: harden counting stream forwarder and document usage limits
Buffer the forwarded stream channel and make the send context-aware to
match the client convention and prevent goroutine leaks. Document that
streaming-path usage is unpopulated by the bundled clients today.
* feat: expose cumulative token usage on the ExecuteTools result
Wraps the run LLM in a counting decorator and stamps the summed usage onto
the returned fragment's Status.CumulativeUsage via a deferred named return,
covering all exit paths. Each sub-agent run reports its own total.
---------
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Add an exported Background flag on AgentState, set true for background
spawns (spawn_agent background=true) and false for foreground ones. This
lets embedders tell unattended background work apart from a foreground
sub-agent whose result is consumed inline — e.g. to auto-notify on
completion only for background agents.
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* fix(agent): keep detached/background sub-agents alive after parent turn ends
A sub-agent's context was a child of the parent turn context (r.ctx), so
when an embedder cancelled its per-turn context right after ExecuteTools
returned — the normal end-of-turn cleanup — a just-detached (or
background) agent was cancelled too, defeating the whole point of
detaching.
Parent the sub-agent context off context.WithoutCancel(r.ctx): it keeps
the context's values but no longer propagates the parent's cancellation.
Foreground cancellation is unaffected because the foreground select
already watches r.ctx.Done() and calls cancel() explicitly; detached and
background agents now run to completion independently and remain
cancellable via AgentState.Cancel.
Add TestDetachedAgentSurvivesParentCancel, which fails on the old
behavior (agent cancelled after parent cancel) and passes now.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* chore: gofmt cancel_inflight_test.go under Go 1.25
Go 1.25's gofmt expands a single-line function body that earlier versions
left inline, so the repo-wide `go fmt ./...` CI check failed on this file
(inherited from #52). No behavior change.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
ExecuteTools' retry loops slept on a non-cancellable
time.Sleep(attempts+1 seconds) backoff and re-issued the LLM call even
after the caller's context was cancelled. A Ctrl+C therefore took up to
several seconds to take effect because an in-flight/retrying call kept
running through the backoff before noticing cancellation.
Add a backoffOrCancel(ctx, attempt) helper that waits the backoff but
returns ctx.Err() immediately when the context is done, and check
ctx.Err() at the top of each retry loop. The interrupt now propagates in
~0.3s.
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* feat(agent): add AgentID to SessionState for sub-agent tool routing
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(agent): route sub-agent tool calls through parent callback + MCPs with AgentID
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(agent): add AgentDefinition type and WithAgentDefinitions option
* feat(agent): apply agent definition (prompt/tools/limits) on spawn
* feat(clients): add temperature support to OpenAI client
* feat(agent): per-agent model+temperature via WithAgentLLMFactory
* feat(agent): per-agent injection channel and AgentManager.Inject
* feat(agent): add unified send_agent_message resume/inject tool
* feat(agent): detachable foreground spawns + AgentManager.Detach
Register every foreground sub-agent and run it in a goroutine so an
embedder can promote it to the background. spawn_agent now selects on
agent.done (completed -> return result like the old synchronous path),
agent.detach (promoted -> return the ID immediately, goroutine keeps
running), or ctx.Done. Extract the shared goroutine body into runAgent
and add derefFragment so foreground and background share lifecycle
bookkeeping. Stamp withAgentIDStamp(agentID) with the real registry ID
on BOTH foreground and background registered agents so sub-agent tool
calls carry the correct AgentID. Add AgentState.detach and
AgentManager.Detach (errors on unknown / non-detachable agent;
non-blocking send).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(agent): add WithAgentSpawnCallback and AgentState.Type for running-agent signal
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* test(agent): cover definitions, AgentID approval, and spawn callback in Sub-Agent Spawning suite
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(mcp): handle non-text tool result content without panicking
An MCP tool result is a slice of mcp.Content, which may contain image,
audio, resource-link or embedded-resource blocks in addition to text.
mcpTool.Execute asserted every block to *mcp.TextContent unconditionally,
so any tool returning media (e.g. osmmcp get_map_image, NASA image tools)
triggered:
panic: interface conversion: mcp.Content is *mcp.ImageContent, not *mcp.TextContent
The panic was on an agent goroutine with no recover, taking the whole
host process down (mudler/LocalAI#10101).
Extract a contentToString helper that type-switches over every mcp.Content
variant: text is concatenated verbatim (preserving prior behavior), media
and resource blocks are summarized with a descriptive marker, and unknown
types are logged instead of crashing.
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* fix(tools): never send consecutive assistant messages to the backend
cogito's tool loop appends an assistant "reasoning" message (toolSelection
no-tool / no-tool-selected paths) on top of a fragment that may already end
with an assistant message. The resulting decision request then ends with two
assistant messages in a row, which llama.cpp via LocalAI rejects:
500 InvalidArgument: Cannot have 2 or more assistant messages at the end of the list
Because decisionWithStreaming retries the identical request, all attempts hit
the same error and the whole reviewer/tool flow fails (the Test E2E
"refine a content with a search tool" spec, and any agent feeding an
assistant-terminated fragment into a tool loop).
Add mergeConsecutiveAssistantMessages, applied alongside normalizeSystemMessages
at the single boundary where both decision() and decisionWithStreaming() build
the request. It collapses any run of consecutive assistant messages into one,
concatenating content and preserving tool calls, so the list can never end with
two assistant messages.
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* test(e2e): compare tool-call arguments by JSON value, not raw bytes
The "should select a tool" spec re-marshaled the parsed arguments map to
compact JSON and compared it byte-for-byte against the tool-call message's
Arguments via HaveExactElements. The message preserves the model's raw
arguments string, whose whitespace differs ({"city": "San Francisco"} vs
{"city":"San Francisco"}), so the assertion failed deterministically
(all 5 flake attempts) even though the tool and arguments were correct.
Assert length, type and name explicitly and match the arguments with
MatchJSON, which compares JSON semantically and ignores insignificant
whitespace and key ordering.
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
---------
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Embedders running their own background goroutines (which cogito's
AgentManager knows nothing about) previously had no way to keep the
ExecuteTools loop alive while that work was pending: once the LLM
selected no tool / a sink state, the loop returned.
WithPendingWork(fn func() bool) lets such an embedder express "I still
have work in flight". Both park gates in ExecuteTools — the no-tool path
(~tools.go:1343) and the sink-state path (~tools.go:1451) — now broaden
their condition so the loop blocks on the message-injection channel when
EITHER cogito's own AgentManager has running agents OR the embedder's
predicate returns true. The embedder wakes the loop by injecting a
message via WithMessageInjectionChan.
Because the park branch blocks on the injection channel, a
pendingWork-only caller (no agent spawning) would otherwise hit a
nil-channel block releasable only by ctx. The channel is now
auto-created (same buffer size of 16) when pendingWork is set, mirroring
the existing agent-spawning setup.
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
CI's fmt gate (`go fmt ./...` must be a no-op) and the golangci-lint
job (gosimple default linter) were red on main, independent of any
feature work:
- gofmt: clients/localai_client.go, prompt/prompt.go, stream.go were
not gofmt-clean.
- gosimple S1011 (autoimprove.go): a loop appending each message is
replaced with a single variadic append.
No behaviour change. Verified: `go fmt ./...` no-op, `go vet ./...`
clean, `go build ./...` ok, and golangci-lint v1.64.8 (the version CI
pins) exits 0.
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
When a background sub-agent finishes, cogito injects a fixed prose
notification ("Background agent <id> has completed.\nTask: …\nResult: …")
into the parent loop. Host applications that drive their own UI want to
control exactly what the parent LLM sees on wake — a clean structured
summary, or a marker they can intercept and render themselves — rather
than leaving the model to re-parse prose.
This adds WithAgentCompletionFormatter(func(*AgentState) string), threaded
through to the spawn goroutine's injection site. The message-building is
extracted into a pure formatAgentCompletion helper: nil formatter keeps
the existing default prose (backwards compatible), a set formatter fully
controls the injected content (an explicit "" is honoured verbatim).
Tested:
- formatAgentCompletion: default completed/failed prose + custom override
+ explicit-empty honoured (pure unit tests)
- spawnAgentRunner background path: the formatter's output is exactly the
message injected on the channel (end-to-end via an inline mock LLM)
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
mcpToolsFromTransport now accepts an optional filter that's invoked
once per (session, tool) pair during the initial tool-discovery pass;
tools for which the filter returns false are dropped from the agent's
discovered set and the LLM never sees them.
Use case: per-user enable/disable of remote MCP server tools (notary's
customize tab persists per-user toggles in user_mcp_server_tools and
plugs them into the filter at chat time). The filter keys on the
*mcp.ClientSession pointer so each registered server can have its own
enable map without cross-contamination.
A nil filter is the default and is equivalent to "always allow", so
existing WithMCPs callers see no behavior change.
The first pass only walked properties + items. JSON Schema 2020-12
puts schema nodes in many other places — composition keywords
(oneOf/anyOf/allOf), tuple items, additionalProperties / patternProperties,
$defs / definitions, contains, not, if/then/else, propertyNames,
prefixItems. Any of those can carry a nullable Go-typed field that
gets emitted as "type": ["null", "X"] by gomcp v1.4+, and any
unflattened type-array silently fails the langchaingo unmarshal,
dropping the tool.
Refactor into coerceSchema(node) so a single visitor covers every
location uniformly, then recurses. Coverage by tests:
- properties (object), items (single + tuple), additionalProperties
- oneOf, anyOf, allOf members
- $defs / definitions named schemas
Test asserts deep-walk completeness via containsTypeArray helper
(no JSON-array type left anywhere). When the schema shape is
representable in langchaingo's Definition (no oneOf, no tuple items),
also asserts the round-trip into Definition succeeds.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
So callers (e.g. notary) can run the same workaround in their
tool-discovery tests and assert their MCP servers stay importable
into the agent's tool list. Same body as the unexported version.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
modelcontextprotocol/go-sdk v1.4+ emits "type": ["null", "X"] for
nullable fields like Go []string slices (even with omitempty). Our
downstream langchaingo/jsonschema.Definition.Type is a single string,
so the property unmarshal fails on the type-array, the surrounding
catch silently `continue`s, and the entire tool drops out of the
agent's tool list.
Effect in prod: notary registered `search`, `get_timeframe`,
`create_page`, and `update_page` MCP tools (all four take []string
inputs) but the agent never saw them — chat fell back to list_pages
+ read_page only, references panel went empty.
Add coerceNullableTypes: walk Properties before unmarshal and rewrite
"type": ["null", "X"] → "type": "X" (first non-null member, fallback
to first member). Recurses into nested object properties + array
items so deeper schemas are normalised too.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds a metadata map and SetMetadata method on LocalAIClient. When set, the
map is serialized as a top-level "metadata" object in the LocalAI request
body alongside the existing optional "grammar" field. This lets callers
override per-model defaults on a single call (e.g.
{"enable_thinking": "true"}) for both unary and streaming completions.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat: add sub-agents support
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* add example
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
---------
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* feat: support streaming mode
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* feat: support streaming mode for tool calls
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* feat: add decision with streaming
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
---------
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
The code was assigning to 'messages' but then using 'intentionMessages'
throughout the rest of the function. This caused a golang-ci lint error.
Closes#42
* feat: add automatic conversation compaction based on token threshold
This commit adds automatic conversation compaction to prevent context overflow
during long-running tool execution sessions.
Key changes:
- Added LLMUsage struct to track token usage from LLM responses
- Modified LLM interface to return token usage alongside Fragment
- Added WithCompactionThreshold option to set token count threshold
- Added WithCompactionKeepMessages option to configure recent messages to keep
- Added compaction logic in ExecuteTools after LLM calls
- Added helper functions: compactFragment, checkAndCompact, estimateTokens
- Added PromptConversationCompaction for generating conversation summaries
- Updated OpenAI and LocalAI clients to return token usage
- Updated mock client for testing
When compactionThreshold is set (> 0), the conversation will be automatically
compacted when estimated token count exceeds the threshold. The compaction
generates a summary of the conversation history using an LLM call while
preserving recent messages.
Signed-off-by: Autonomous Coding Agent <agent@autonomous>
* fix: use actual usage tokens from LLM response for compaction
- Store LastUsage in Status struct from LLM responses
- checkAndCompact now uses actual TotalTokens from LLM response
- Removed estimateTokens function (no longer needed)
- Fallback estimate only used on first iteration when no usage data available
* fix: capture usage tokens after sink state LLM call for compaction
The sink state handling was not capturing usage tokens from the LLM response,
which meant the compaction check would use the rough estimate instead of
actual usage tokens. This change ensures LastUsage is stored after the
llm.Ask call in the hasSinkState block, allowing proper token-based compaction.
* fix: move compaction check to beginning of tool loop
- Removed compaction check after max iterations (not needed)
- Removed compaction check after sink state (not needed)
- Added compaction check at beginning of tool loop (after totalIterations++)
- Uses actual usage tokens from LLM response
* fix: update Ask to return usage tokens from LocalAIClient
* fix: set LastUsage in Ask function return fragment
This addresses reviewer feedback that Ask() should automatically update
the Fragment's LastUsage, not have callers do it. The OpenAIClient and
LocalAIClient Ask functions now set Status.LastUsage before returning.
* refactor: Ask() updates Fragment.Status.LastUsage directly
Instead of returning LLMUsage from Ask(), the LLM clients now update
the Fragment's Status.LastUsage directly. This simplifies the interface
and ensures usage is always tracked in the fragment.
Changes:
- LLM.Ask() now returns (Fragment, error) instead of (Fragment, LLMUsage, error)
- Clients (openai_client.go, localai_client.go) set LastUsage on the returned fragment
- Mock client also updated to set usage in Status
- All callers updated to use new 2-value return signature
This addresses reviewer feedback on PR #41.
* Apply suggestion from @mudler
* Apply suggestion from @mudler
* Apply suggestion from @mudler
* Apply suggestion from @mudler
* Apply suggestions from code review
* Apply suggestion from @mudler
* test: add mocked tests for compaction functionality
- Add DefaultPrompts() function to prompt package for tests
- Export CompactFragment and CheckAndCompact functions for testing
- Add comprehensive unit tests for compaction logic using mocks
- Remove duplicate Ginkgo compaction tests that have import issues
* test: add compaction tests to tools_test.go suite
Add Ginkgo tests for compaction functionality within the existing
tools_test.go suite. Tests cover:
- No compaction when threshold is disabled (0)
- No compaction when tokens below threshold
- Compaction when token threshold is exceeded
- Parent fragment preservation after compaction
- Status preservation after compaction
- Rough token estimate usage when LastUsage is not set
This addresses the reviewer's request to keep tests consistent with
other ginkgo tests in tools_test.go.
* chore: run go fmt and add compaction docs to README
* Make CompactFragment and CheckAndCompact private
Per reviewer request:
- Changed CompactFragment to compactFragment (private)
- Changed CheckAndCompact to checkAndCompact (private)
- Removed tools_compaction_test.go (tests should be in tools_test.go)
The compaction functionality is still available internally via
ExecuteTools with WithCompactionThreshold option.
* chore: remove exported functions from README, keep them private
* chore: verify all changes applied - build passes
* chore: verify build and vet pass
---------
Signed-off-by: Autonomous Coding Agent <agent@autonomous>
Co-authored-by: Ettore Di Giacinto <mudler@users.noreply.github.com>
Ettore Di Giacinto