mirror of
https://github.com/codewhale-hq/Codewhale.git
synced 2026-11-07 17:09:13 -10:00
fix(tui): carry the resolved context window with its source through planning, compaction and clients [skip ci]
The effective context window (provider facts, operator configuration or
fallback) and its source are now one ContextWindowResolution value kept
beside, never inside, the provider's RouteLimits. Turn planning, the
Engine, compaction (parent, headless, runtime-thread and worker), tool
output sizing, output caps, child agents, the context inspector, the model
picker and the client's alternate-model rebind all read it. Before, an
exact custom model with no context metadata planned compaction against
1,048,576 tokens while its route resolved the 128,000 fallback.
Op::SetModel clears the installed window only when the model or limits
change, so the same-route syncs hosts send on load, reset and re-sync keep
it. Alternate-model rebinding derives its window and limits from the
client's frozen declarations and captured overrides rather than a later
mutable configuration. Accepted-route restoration in the session
projection keeps the paired window across the model canonicalization.
Also: a checkpoint or reset save with no pending Work update now reports an
unavailable persistence actor instead of dropping the request silently.
Refs #7133 (CW-7640), #7095 (CW-7553). Frozen 2,192 tracked inputs on
adcf3f5856 plus 42 dirty source paths, unchanged across the run: 113 of 113
focused existing cases passed (the 94 planner, route install, projection,
picker, budget, tool-output and pressure cases, 18 persistence/backlog/
lifecycle cases, and the two existing SetModel cases), including the
alternate-declared-model frozen-limit case that failed on the previous
candidate. rustfmt, sync-changelog --check and git diff --check pass.
Not established: the original Windows report is not reproduced; a restored
Auto session has no persisted window receipt and uses the fallback until
its next turn selects a route; the out-of-process protocol and provider
rollback paths are reviewed, not exercised end to end; no real-provider,
hosted CI or platform proof.
This commit is contained in:
1 parent
adcf3f5856
commit
da026a2436
44 files changed
+812
-356
No files matched your search
@@ -123,6 +123,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
||||
|
||||
### Fixed
|
||||
|
||||
- **One context window from planning to compaction (#7133):** the window a route resolves (from provider facts, your configuration or a fallback) and where it came from now travel together. Turn planning, automatic compaction, tool-output sizing, output caps, child agents, the context inspector and the model picker all use the same window. Previously a custom model with no context metadata was sent with a 128K fallback while compaction planned against 1M. A model switch or re-sync that keeps the same model and limits keeps that window. A restored Auto session uses the fallback window until its next turn picks a route. The original Windows report has not been reproduced.
|
||||
- **Session saves no longer fail silently (#7095):** if the session store has stopped, a checkpoint or reset save now reports the failure in the status line instead of being dropped with only a log line.
|
||||
- **Saves keep your recovery checkpoint (#7095):** a finished turn is saved even when a newer in-progress checkpoint arrives at the same time, and a `/clear` or `/new` reset no longer drops its own snapshot when saves queue up. If the reset snapshot cannot be built or written, the previous recovery checkpoint is kept and the save failure is shown in the transcript and status line instead. Retrying an interrupted session deletion finishes it without removing the shared checkpoint an older Codewhale may still be writing. These fixes do not yet reproduce the original report.
|
||||
- **Context inspector shows the real compaction trigger (#7133):** `/context` uses the current route's effective threshold, including when it is zero. When automatic compaction is off, it says so, and the Work rail points to manual `/compact`.
|
||||
- **`doctor --probe-local` changes nothing (#7115):** when a configured route rules out local Ollama, the check skips credential inspection. Otherwise it reads credentials without migrating the legacy secret store; `exec` keeps its normal resolver.
|
||||
|
||||
@@ -123,6 +123,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
||||
|
||||
### Fixed
|
||||
|
||||
- **One context window from planning to compaction (#7133):** the window a route resolves (from provider facts, your configuration or a fallback) and where it came from now travel together. Turn planning, automatic compaction, tool-output sizing, output caps, child agents, the context inspector and the model picker all use the same window. Previously a custom model with no context metadata was sent with a 128K fallback while compaction planned against 1M. A model switch or re-sync that keeps the same model and limits keeps that window. A restored Auto session uses the fallback window until its next turn picks a route. The original Windows report has not been reproduced.
|
||||
- **Session saves no longer fail silently (#7095):** if the session store has stopped, a checkpoint or reset save now reports the failure in the status line instead of being dropped with only a log line.
|
||||
- **Saves keep your recovery checkpoint (#7095):** a finished turn is saved even when a newer in-progress checkpoint arrives at the same time, and a `/clear` or `/new` reset no longer drops its own snapshot when saves queue up. If the reset snapshot cannot be built or written, the previous recovery checkpoint is kept and the save failure is shown in the transcript and status line instead. Retrying an interrupted session deletion finishes it without removing the shared checkpoint an older Codewhale may still be writing. These fixes do not yet reproduce the original report.
|
||||
- **Context inspector shows the real compaction trigger (#7133):** `/context` uses the current route's effective threshold, including when it is zero. When automatic compaction is off, it says so, and the Work rail points to manual `/compact`.
|
||||
- **`doctor --probe-local` changes nothing (#7115):** when a configured route rules out local Ollama, the check skips credential inspection. Otherwise it reads credentials without migrating the legacy secret store; `exec` keeps its normal resolver.
|
||||
|
||||
@@ -241,11 +241,14 @@ pub struct CodewhaleClient {
|
||||
billing_surface: Option<String>,
|
||||
billing_mode: crate::cost_status::RouteBillingMode,
|
||||
configured_models: Arc<Vec<codewhale_config::catalog::configured::ConfiguredModel>>,
|
||||
context_window_override: Option<u32>,
|
||||
model_context_windows: Option<std::collections::BTreeMap<String, u32>>,
|
||||
/// Non-secret limits frozen from the same resolved candidate as the
|
||||
/// endpoint and wire model. Auxiliary calls carry only this client, so
|
||||
/// they must not reconstruct output caps with `None` and discard a custom
|
||||
/// route's context window.
|
||||
route_limits: Option<RouteLimits>,
|
||||
context_window: Option<crate::route_runtime::ContextWindowResolution>,
|
||||
route_facts: Option<Arc<codewhale_config::catalog::CatalogOffering>>,
|
||||
/// Verified ChatGPT subject captured with Codewhale's own selected grant.
|
||||
pub(super) codex_account_id: Option<String>,
|
||||
@@ -555,7 +558,10 @@ impl Clone for CodewhaleClient {
|
||||
billing_surface: self.billing_surface.clone(),
|
||||
billing_mode: self.billing_mode,
|
||||
configured_models: Arc::clone(&self.configured_models),
|
||||
context_window_override: self.context_window_override,
|
||||
model_context_windows: self.model_context_windows.clone(),
|
||||
route_limits: self.route_limits,
|
||||
context_window: self.context_window,
|
||||
route_facts: self.route_facts.clone(),
|
||||
codex_account_id: self.codex_account_id.clone(),
|
||||
chatgpt_reasoning_api: self.chatgpt_reasoning_api.clone(),
|
||||
@@ -1452,15 +1458,18 @@ impl CodewhaleClient {
|
||||
let route =
|
||||
crate::route_runtime::resolve_runtime_route_for_identity(config, &identity, None)
|
||||
.map_err(anyhow::Error::msg)?;
|
||||
return Self::from_candidate(&route.config, &route.candidate);
|
||||
return Self::from_candidate(&route.config, &route.candidate)
|
||||
.map(|client| client.with_context_window(route.context_window));
|
||||
}
|
||||
let route_limits = crate::route_runtime::resolve_runtime_route_for_identity(
|
||||
let resolved = crate::route_runtime::resolve_runtime_route_for_identity(
|
||||
config,
|
||||
&identity,
|
||||
Some(&default_model),
|
||||
)
|
||||
.ok()
|
||||
.and_then(|route| crate::route_budget::known_route_limits(route.candidate.limits()));
|
||||
.ok();
|
||||
let route_limits = resolved
|
||||
.as_ref()
|
||||
.and_then(|route| crate::route_budget::known_route_limits(route.candidate.limits()));
|
||||
Self::from_parts(
|
||||
config.active_route_base_url(),
|
||||
default_model,
|
||||
@@ -1468,6 +1477,10 @@ impl CodewhaleClient {
|
||||
route_limits,
|
||||
config,
|
||||
)
|
||||
.map(|mut client| {
|
||||
client.context_window = resolved.map(|route| route.context_window);
|
||||
client
|
||||
})
|
||||
}
|
||||
|
||||
/// Construct only the model-list probe before a route has a concrete model.
|
||||
@@ -1527,6 +1540,10 @@ impl CodewhaleClient {
|
||||
.active_provider_identity()
|
||||
.map_err(anyhow::Error::msg)?;
|
||||
let api_provider = admitted_identity.provider;
|
||||
let context_window_override = config.context_window_for_provider_config(&admitted_identity);
|
||||
let model_context_windows = config
|
||||
.model_context_windows_for(&admitted_identity)
|
||||
.cloned();
|
||||
anyhow::ensure!(
|
||||
api_provider != ProviderKind::Antigravity,
|
||||
codewhale_config::LEGACY_ANTIGRAVITY_TOMBSTONE_MESSAGE
|
||||
@@ -1824,7 +1841,10 @@ impl CodewhaleClient {
|
||||
billing_surface,
|
||||
billing_mode,
|
||||
configured_models: Arc::new(config.custom_models.clone().unwrap_or_default()),
|
||||
context_window_override,
|
||||
model_context_windows,
|
||||
route_limits,
|
||||
context_window: None,
|
||||
route_facts,
|
||||
codex_account_id,
|
||||
chatgpt_reasoning_api,
|
||||
@@ -2187,10 +2207,37 @@ impl CodewhaleClient {
|
||||
return Ok(None);
|
||||
}
|
||||
let candidate = self.resolve_model_route(model)?;
|
||||
let candidate_limits = crate::route_budget::known_route_limits(candidate.limits());
|
||||
let resolved = crate::route_runtime::resolve_declared_model_candidate_with_wire_override(
|
||||
self.api_provider,
|
||||
self.admitted_identity.key.as_str(),
|
||||
model,
|
||||
&self.base_url,
|
||||
self.context_window_override,
|
||||
self.model_context_windows.as_ref(),
|
||||
&self.configured_models,
|
||||
(self.api_provider == ProviderKind::Custom).then_some(self.wire_format),
|
||||
)
|
||||
.ok()
|
||||
.filter(|route| {
|
||||
route.candidate.wire_model_id() == candidate.wire_model_id()
|
||||
&& route.candidate.protocol() == candidate.protocol()
|
||||
&& route.candidate.endpoint().base_url == candidate.endpoint().base_url
|
||||
&& route.candidate.endpoint().endpoint_key == candidate.endpoint().endpoint_key
|
||||
});
|
||||
let candidate_limits = resolved.as_ref().map_or_else(
|
||||
|| crate::route_budget::known_route_limits(candidate.limits()),
|
||||
|route| crate::route_budget::known_route_limits(route.candidate.limits()),
|
||||
);
|
||||
let context_window = resolved.map(|route| route.context_window).or_else(|| {
|
||||
(candidate.wire_model_id().as_str() == self.default_model
|
||||
&& candidate_limits == self.route_limits)
|
||||
.then_some(self.context_window)
|
||||
.flatten()
|
||||
});
|
||||
if candidate.protocol() == self.wire_format
|
||||
&& candidate.wire_model_id().as_str() == self.default_model
|
||||
&& candidate_limits == self.route_limits
|
||||
&& context_window == self.context_window
|
||||
{
|
||||
return Ok(None);
|
||||
}
|
||||
@@ -2203,6 +2250,7 @@ impl CodewhaleClient {
|
||||
let mut rebound = self.clone();
|
||||
rebound.default_model = candidate.wire_model_id().as_str().to_string();
|
||||
rebound.route_limits = candidate_limits;
|
||||
rebound.context_window = context_window;
|
||||
rebound.route_facts = rebound.catalog_facts_for_model(&rebound.default_model);
|
||||
return Ok(Some(rebound));
|
||||
}
|
||||
@@ -2216,7 +2264,11 @@ impl CodewhaleClient {
|
||||
)
|
||||
})?;
|
||||
let mut rebound = Self::from_candidate(config, &candidate)?;
|
||||
rebound.route_limits = candidate_limits;
|
||||
rebound.context_window = context_window;
|
||||
rebound.configured_models = Arc::clone(&self.configured_models);
|
||||
rebound.context_window_override = self.context_window_override;
|
||||
rebound.model_context_windows = self.model_context_windows.clone();
|
||||
rebound.route_facts = rebound.catalog_facts_for_model(&rebound.default_model);
|
||||
Ok(Some(rebound))
|
||||
}
|
||||
@@ -3120,12 +3172,13 @@ impl CodewhaleClient {
|
||||
match self.wire_format {
|
||||
WireFormat::ChatCompletions => {
|
||||
let chat_shape_provider = self.chat_shape_provider(&request.model);
|
||||
let mut wire = chat::build_chat_wire_body(
|
||||
let mut wire = chat::build_chat_wire_body_with_window(
|
||||
&request,
|
||||
chat_shape_provider,
|
||||
&self.base_url,
|
||||
stream,
|
||||
request_route_limits,
|
||||
self.context_window_for_model(&request.model),
|
||||
)?;
|
||||
if let Some(effort) = published_wire_effort.as_deref() {
|
||||
if wire.body.get("reasoning_effort").is_some() {
|
||||
@@ -3268,6 +3321,28 @@ impl CodewhaleClient {
|
||||
self.route_limits
|
||||
}
|
||||
|
||||
pub(crate) fn with_context_window(
|
||||
mut self,
|
||||
context_window: crate::route_runtime::ContextWindowResolution,
|
||||
) -> Self {
|
||||
self.context_window = Some(context_window);
|
||||
self
|
||||
}
|
||||
|
||||
fn context_window_for_model(&self, model: &str) -> Option<u32> {
|
||||
self.context_window_resolution_for_model(model)
|
||||
.map(|window| window.tokens)
|
||||
}
|
||||
|
||||
pub(crate) fn context_window_resolution_for_model(
|
||||
&self,
|
||||
model: &str,
|
||||
) -> Option<crate::route_runtime::ContextWindowResolution> {
|
||||
(model.trim() == self.default_model)
|
||||
.then_some(self.context_window)
|
||||
.flatten()
|
||||
}
|
||||
|
||||
/// Output cap for a request dispatched by this exact client route.
|
||||
#[must_use]
|
||||
pub fn effective_max_output_tokens(&self, requested_model: &str) -> u32 {
|
||||
@@ -3295,10 +3370,11 @@ impl CodewhaleClient {
|
||||
max_tokens: u32,
|
||||
input_tokens: usize,
|
||||
) -> Result<u32> {
|
||||
crate::route_budget::request_output_cap_for_input(
|
||||
crate::route_budget::request_output_cap_for_input_with_window(
|
||||
self.api_provider,
|
||||
model,
|
||||
route_limits,
|
||||
self.context_window_for_model(model),
|
||||
max_tokens,
|
||||
u64::try_from(input_tokens).unwrap_or(u64::MAX),
|
||||
)
|
||||
@@ -3318,10 +3394,11 @@ impl CodewhaleClient {
|
||||
reasoning_effort: Option<&str>,
|
||||
) -> u32 {
|
||||
let wire_model = self.wire_model_for_route(requested_model);
|
||||
crate::route_budget::effective_max_output_tokens_for_turn(
|
||||
crate::route_budget::effective_max_output_tokens_for_turn_with_window(
|
||||
self.api_provider,
|
||||
&wire_model,
|
||||
route_limits,
|
||||
self.context_window_for_model(requested_model),
|
||||
None,
|
||||
reasoning_effort,
|
||||
)
|
||||
@@ -4823,6 +4900,10 @@ impl LlmClient for CodewhaleClient {
|
||||
CodewhaleClient::route_limits(self)
|
||||
}
|
||||
|
||||
fn effective_context_window(&self, model: &str) -> Option<u32> {
|
||||
self.context_window_for_model(model)
|
||||
}
|
||||
|
||||
fn effective_max_output_tokens(&self, requested_model: &str) -> u32 {
|
||||
CodewhaleClient::effective_max_output_tokens(self, requested_model)
|
||||
}
|
||||
|
||||
@@ -1178,17 +1178,30 @@ pub(crate) struct ChatWireBody {
|
||||
/// `stream` selects the streaming shape (`stream` + `stream_options`) and, to
|
||||
/// preserve historical behavior exactly, also gates the thinking-mode replay
|
||||
/// sanitizer — the blocking path has never run it.
|
||||
#[cfg(test)]
|
||||
pub(crate) fn build_chat_wire_body(
|
||||
request: &MessageRequest,
|
||||
provider: ProviderKind,
|
||||
base_url: &str,
|
||||
stream: bool,
|
||||
route_limits: Option<RouteLimits>,
|
||||
) -> Result<ChatWireBody> {
|
||||
build_chat_wire_body_with_window(request, provider, base_url, stream, route_limits, None)
|
||||
}
|
||||
|
||||
pub(crate) fn build_chat_wire_body_with_window(
|
||||
request: &MessageRequest,
|
||||
provider: ProviderKind,
|
||||
base_url: &str,
|
||||
stream: bool,
|
||||
route_limits: Option<RouteLimits>,
|
||||
effective_window: Option<u32>,
|
||||
) -> Result<ChatWireBody> {
|
||||
let messages = PromptBuilder::for_request(request).build_for_provider_and_route(
|
||||
provider,
|
||||
base_url,
|
||||
route_limits,
|
||||
effective_window,
|
||||
);
|
||||
let model = {
|
||||
let wire = wire_model_for_provider_route(provider, base_url, &request.model);
|
||||
@@ -1807,7 +1820,7 @@ pub(super) fn build_chat_messages_for_request_and_provider_and_route(
|
||||
provider: ProviderKind,
|
||||
base_url: &str,
|
||||
) -> Vec<Value> {
|
||||
PromptBuilder::for_request(request).build_for_provider_and_route(provider, base_url, None)
|
||||
PromptBuilder::for_request(request).build_for_provider_and_route(provider, base_url, None, None)
|
||||
}
|
||||
|
||||
pub(crate) fn inspect_prompt_for_request(request: &MessageRequest) -> PromptInspection {
|
||||
@@ -1853,15 +1866,20 @@ impl<'a> PromptBuilder<'a> {
|
||||
provider: ProviderKind,
|
||||
base_url: &str,
|
||||
route_limits: Option<RouteLimits>,
|
||||
effective_window: Option<u32>,
|
||||
) -> Vec<Value> {
|
||||
let mut messages = build_chat_messages_with_reasoning(
|
||||
self.system,
|
||||
self.messages,
|
||||
crate::route_budget::route_inline_char_budget_for_route(
|
||||
provider,
|
||||
self.model,
|
||||
route_limits,
|
||||
),
|
||||
crate::route_budget::route_inline_char_budget(Some(effective_window.unwrap_or_else(
|
||||
|| {
|
||||
crate::route_budget::route_context_window_tokens(
|
||||
provider,
|
||||
self.model,
|
||||
route_limits,
|
||||
)
|
||||
},
|
||||
))),
|
||||
should_replay_reasoning_content_for_provider_on_route(
|
||||
provider,
|
||||
base_url,
|
||||
|
||||
@@ -266,12 +266,8 @@ pub(crate) fn show_single_setting(app: &App, key: &str) -> ConfigReply {
|
||||
"{} (effective {} from {})",
|
||||
config_context_window_override(app)
|
||||
.map_or_else(|| "not set".to_string(), |tokens| tokens.to_string()),
|
||||
crate::route_budget::route_context_window_tokens(
|
||||
app.api_provider,
|
||||
app.effective_model_for_budget(),
|
||||
app.active_route_limits,
|
||||
),
|
||||
app.active_context_window_source.display_label(),
|
||||
app.effective_context_window(),
|
||||
app.effective_context_window_source().display_label(),
|
||||
)),
|
||||
"stream_chunk_timeout_secs" => Some(app.stream_chunk_timeout_secs.to_string()),
|
||||
"posture_bar" => Some(app.posture_bar.as_setting().to_string()),
|
||||
@@ -770,12 +766,8 @@ pub(crate) fn config_editability_audit(
|
||||
"effective_context_window",
|
||||
format!(
|
||||
"{} ({})",
|
||||
crate::route_budget::route_context_window_tokens(
|
||||
app.api_provider,
|
||||
app.effective_model_for_budget(),
|
||||
app.active_route_limits,
|
||||
),
|
||||
app.active_context_window_source.display_label(),
|
||||
app.effective_context_window(),
|
||||
app.effective_context_window_source().display_label(),
|
||||
),
|
||||
"runtime",
|
||||
"/config context_window",
|
||||
|
||||
@@ -87,17 +87,13 @@ fn project_status(app: &App) -> ConfigStatusView {
|
||||
let metrics = crate::tui::session_metrics::snapshot_from_app(app);
|
||||
let outputs =
|
||||
crate::tool_output_receipts::tool_output_status(&app.api_messages, &app.session_artifacts);
|
||||
let context_window = crate::route_budget::route_context_window_tokens(
|
||||
app.api_provider,
|
||||
app.effective_model_for_budget(),
|
||||
app.active_route_limits,
|
||||
);
|
||||
let context_window = app.effective_context_window();
|
||||
let context_used = crate::compaction::estimate_input_tokens_conservative(
|
||||
&app.api_messages,
|
||||
app.system_prompt.as_ref(),
|
||||
)
|
||||
.max(codewhale_util::utils::estimate_message_chars(&app.api_messages) / 4);
|
||||
let context_source = match app.active_context_window_source {
|
||||
let context_source = match app.effective_context_window_source() {
|
||||
crate::route_runtime::ContextWindowSource::Configured => StatusContextSource::Configured,
|
||||
crate::route_runtime::ContextWindowSource::UserDeclared => {
|
||||
StatusContextSource::UserDeclared
|
||||
|
||||
@@ -59,11 +59,7 @@ impl CommandDebugDiagnosticsContext for DebugDiagnosticsAdapter<'_> {
|
||||
|
||||
fn token_projection(&self) -> DebugTokenProjection {
|
||||
let app = self.host.app.borrow();
|
||||
let window = crate::route_budget::route_context_window_tokens(
|
||||
app.api_provider,
|
||||
app.effective_model_for_budget(),
|
||||
app.active_route_limits,
|
||||
);
|
||||
let window = app.effective_context_window();
|
||||
let estimated = crate::compaction::estimate_input_tokens_conservative(
|
||||
&app.api_messages,
|
||||
app.system_prompt.as_ref(),
|
||||
|
||||
@@ -177,6 +177,11 @@ pub fn model(app: &mut App, model_name: Option<&str>) -> CommandResult {
|
||||
let model_changed = !app.auto_model || app.model != "auto";
|
||||
app.set_model_selection("auto".to_string());
|
||||
app.active_route_limits = app.context_window_override_limits();
|
||||
app.active_context_window = app.configured_context_window_for(&app.model);
|
||||
app.active_context_window_source = app
|
||||
.active_context_window
|
||||
.map(|window| window.source)
|
||||
.unwrap_or(crate::route_runtime::ContextWindowSource::Fallback);
|
||||
app.update_model_compaction_budget();
|
||||
if model_changed {
|
||||
app.clear_model_scoped_telemetry();
|
||||
@@ -280,10 +285,11 @@ pub fn model(app: &mut App, model_name: Option<&str>) -> CommandResult {
|
||||
app.set_active_route_resolution(
|
||||
resolution.candidate.endpoint().base_url.clone(),
|
||||
resolution.candidate.limits(),
|
||||
resolution.context_window.source,
|
||||
resolution.context_window,
|
||||
);
|
||||
} else {
|
||||
app.active_route_limits = app.context_window_override_limits();
|
||||
app.active_context_window = app.configured_context_window_for(&app.model);
|
||||
app.active_context_window_source = app
|
||||
.configured_context_window_for(&app.model)
|
||||
.map(|resolution| resolution.source)
|
||||
|
||||
@@ -1968,17 +1968,23 @@ async fn create_summary(
|
||||
// compaction-only ceiling that can be consumed by hidden reasoning.
|
||||
let cost_route = client.effective_route_envelope(&config.model, chrono::Utc::now());
|
||||
let ceiling = client.effective_max_output_tokens(&cost_route.model);
|
||||
let summary_floor = crate::route_budget::route_output_reservation(
|
||||
let summary_floor = crate::route_budget::route_output_reservation_with_window(
|
||||
cost_route.provider,
|
||||
&cost_route.model,
|
||||
client.route_limits(),
|
||||
client
|
||||
.effective_context_window(&cost_route.model)
|
||||
.or(config.effective_context_window),
|
||||
)
|
||||
.min(ceiling)
|
||||
/ 2;
|
||||
let max_tokens = match crate::route_budget::request_output_cap_for_input(
|
||||
let max_tokens = match crate::route_budget::request_output_cap_for_input_with_window(
|
||||
cost_route.provider,
|
||||
&cost_route.model,
|
||||
client.route_limits(),
|
||||
client
|
||||
.effective_context_window(&cost_route.model)
|
||||
.or(config.effective_context_window),
|
||||
ceiling,
|
||||
u64::try_from(calibrate_token_estimate(
|
||||
estimate_request_input_tokens(&request_messages, system_prompt, tools),
|
||||
|
||||
@@ -23,7 +23,6 @@ use crate::compaction::{
|
||||
estimate_text_tokens_conservative,
|
||||
};
|
||||
use crate::prompts::CORE_EXECUTION_PROFILE_PROMPT;
|
||||
use crate::route_budget::route_context_window_tokens;
|
||||
use crate::tui::app::App;
|
||||
use codewhale_config::AppMode;
|
||||
use codewhale_models::{CacheControl, ContentBlock, Message, SystemPrompt, Tool};
|
||||
@@ -262,15 +261,9 @@ impl ReportBuilder {
|
||||
}
|
||||
|
||||
pub fn build_context_report(app: &App) -> PromptSourceMap {
|
||||
// The host still stores the rung apart from the number; pair them against
|
||||
// the same route limits the pressure meter reads.
|
||||
let context_window = crate::route_runtime::ContextWindowResolution {
|
||||
tokens: route_context_window_tokens(
|
||||
app.api_provider,
|
||||
app.effective_model_for_budget(),
|
||||
app.active_route_limits,
|
||||
),
|
||||
source: app.active_context_window_source,
|
||||
tokens: app.effective_context_window(),
|
||||
source: app.effective_context_window_source(),
|
||||
};
|
||||
let mut builder = base_source_entries(
|
||||
&app.model,
|
||||
@@ -401,19 +394,22 @@ pub fn build_headless_context_report(config: &Config, workspace: &Path) -> Promp
|
||||
let budgets = crate::project_context::InstructionBudgets::resolve_or_default(
|
||||
&settings.budget,
|
||||
context_window.instruction_budget_tokens(),
|
||||
Some(crate::route_budget::effective_max_output_tokens_for_turn(
|
||||
provider,
|
||||
&model,
|
||||
limits,
|
||||
None,
|
||||
crate::core::engine::turn_loop::resolve_auto_effort(
|
||||
config.reasoning_effort(),
|
||||
Some(
|
||||
crate::route_budget::effective_max_output_tokens_for_turn_with_window(
|
||||
provider,
|
||||
&config.active_route_base_url(),
|
||||
&model,
|
||||
)
|
||||
.as_deref(),
|
||||
) as usize),
|
||||
limits,
|
||||
Some(context_window.tokens),
|
||||
None,
|
||||
crate::core::engine::turn_loop::resolve_auto_effort(
|
||||
config.reasoning_effort(),
|
||||
provider,
|
||||
&config.active_route_base_url(),
|
||||
&model,
|
||||
)
|
||||
.as_deref(),
|
||||
) as usize,
|
||||
),
|
||||
);
|
||||
let assembly = crate::project_context::assemble_instruction_block(&mut raw, &[], &budgets);
|
||||
let mut builder = base_source_entries(
|
||||
|
||||
@@ -168,6 +168,7 @@ fn assert_one_pressure_number(
|
||||
app.model = route.model.clone();
|
||||
app.active_route_limits = crate::route_budget::known_route_limits(route.candidate.limits());
|
||||
app.active_context_window_source = route.context_window.source;
|
||||
app.active_context_window = Some(route.context_window);
|
||||
// Install the transcript the way the host applies an engine session
|
||||
// projection (`apply_engine_session_projection`): the meter's per-message
|
||||
// cache is dropped before the rewritten history lands, so a compaction
|
||||
|
||||
@@ -1101,6 +1101,7 @@ pub struct Engine {
|
||||
/// it; a changed table cannot be blessed by reinterpreting those strings.
|
||||
api_provider_identity: Option<codewhale_runtime::config::ProviderIdentity>,
|
||||
active_route_limits: Option<codewhale_config::route::RouteLimits>,
|
||||
active_context_window: Option<crate::route_runtime::ContextWindowResolution>,
|
||||
active_route_capabilities: codewhale_config::route::RouteCapabilities,
|
||||
/// The endpoint the current client was built from: base URL, endpoint
|
||||
/// key, and wire protocol. Part of the route identity the input-bill
|
||||
@@ -1975,6 +1976,7 @@ impl Engine {
|
||||
/// No identity guessing or config re-resolution is allowed at this
|
||||
/// boundary: the descriptor is the single authority for the turn.
|
||||
fn install_validated_runtime_route(&mut self, route: ValidatedRuntimeRoute) {
|
||||
let context_window = route.context_window;
|
||||
let provider = route.identity.provider;
|
||||
let identity = route.identity;
|
||||
let model = route.model;
|
||||
@@ -1996,6 +1998,10 @@ impl Engine {
|
||||
self.api_provider_identity = Some(identity);
|
||||
self.api_config = api_config;
|
||||
self.active_route_limits = limits;
|
||||
if self.active_context_window != Some(context_window) {
|
||||
self.session.latest_parent_input_tokens = None;
|
||||
}
|
||||
self.active_context_window = Some(context_window);
|
||||
self.active_route_capabilities = capabilities;
|
||||
self.api_key_env_only_recovery = Self::env_only_api_key_recovery_hint(&self.api_config);
|
||||
self.codewhale_client = Some(client.clone());
|
||||
@@ -2021,6 +2027,7 @@ impl Engine {
|
||||
return Ok(());
|
||||
}
|
||||
|
||||
let context_window = route.context_window;
|
||||
let preflighted_client = route.take_preflighted_client();
|
||||
let provider = route.identity.provider;
|
||||
let identity = route.identity;
|
||||
@@ -2030,7 +2037,8 @@ impl Engine {
|
||||
let api_config = *route.config;
|
||||
let concrete_client = preflighted_client
|
||||
.map(Ok)
|
||||
.unwrap_or_else(|| CodewhaleClient::from_candidate(&api_config, &route.candidate));
|
||||
.unwrap_or_else(|| CodewhaleClient::from_candidate(&api_config, &route.candidate))
|
||||
.map(|client| client.with_context_window(context_window));
|
||||
|
||||
let endpoint = route.candidate.endpoint().clone();
|
||||
self.forget_input_bill_if_route_changes(
|
||||
@@ -2045,6 +2053,10 @@ impl Engine {
|
||||
self.api_provider_identity = Some(identity);
|
||||
self.api_config = api_config;
|
||||
self.active_route_limits = limits;
|
||||
if self.active_context_window != Some(context_window) {
|
||||
self.session.latest_parent_input_tokens = None;
|
||||
}
|
||||
self.active_context_window = Some(context_window);
|
||||
self.active_route_capabilities = capabilities;
|
||||
self.api_key_env_only_recovery = Self::env_only_api_key_recovery_hint(&self.api_config);
|
||||
match concrete_client {
|
||||
@@ -2584,6 +2596,7 @@ impl Engine {
|
||||
api_provider,
|
||||
api_provider_identity,
|
||||
active_route_limits,
|
||||
active_context_window: None,
|
||||
active_route_capabilities: codewhale_config::route::RouteCapabilities::default(),
|
||||
active_route_endpoint: None,
|
||||
learned_image_caps: HashMap::new(),
|
||||
@@ -4185,6 +4198,9 @@ impl Engine {
|
||||
);
|
||||
self.session.auto_model = model.trim().eq_ignore_ascii_case("auto");
|
||||
let switched = self.session.model != model;
|
||||
if switched || self.active_route_limits != route_limits {
|
||||
self.active_context_window = None;
|
||||
}
|
||||
self.session.model = model;
|
||||
self.config.model.clone_from(&self.session.model);
|
||||
self.active_route_limits = route_limits;
|
||||
@@ -4445,10 +4461,11 @@ impl Engine {
|
||||
}
|
||||
Op::GetContextBudget { tx } => {
|
||||
let input_tokens = self.estimated_input_tokens() as u64;
|
||||
let budget = route_context_budget_for_route(
|
||||
let budget = crate::route_budget::route_context_budget_with_window(
|
||||
self.api_provider,
|
||||
&self.session.model,
|
||||
self.active_route_limits,
|
||||
self.active_context_window.map(|window| window.tokens),
|
||||
usize::try_from(input_tokens).unwrap_or(usize::MAX),
|
||||
);
|
||||
let snapshot = budget.map(|budget| SessionContextBudget {
|
||||
@@ -6772,20 +6789,23 @@ impl Engine {
|
||||
translation_enabled,
|
||||
verbosity.clone(),
|
||||
);
|
||||
prompt_context.context_window = self.active_context_window;
|
||||
let planned_effort = turn_loop::resolve_auto_effort(
|
||||
reasoning_effort.as_deref(),
|
||||
effective_provider,
|
||||
&route_api_config.active_route_base_url(),
|
||||
&model,
|
||||
);
|
||||
prompt_context.planned_output_tokens =
|
||||
Some(crate::route_budget::effective_max_output_tokens_for_turn(
|
||||
prompt_context.planned_output_tokens = Some(
|
||||
crate::route_budget::effective_max_output_tokens_for_turn_with_window(
|
||||
effective_provider,
|
||||
&model,
|
||||
route_limits,
|
||||
self.active_context_window.map(|window| window.tokens),
|
||||
max_output_tokens,
|
||||
planned_effort.as_deref(),
|
||||
));
|
||||
),
|
||||
);
|
||||
if max_output_tokens.is_some()
|
||||
&& let Some(client) = self.model_client.as_ref()
|
||||
{
|
||||
@@ -7946,10 +7966,15 @@ impl Engine {
|
||||
let mut context = child
|
||||
.authority
|
||||
.context()
|
||||
.with_route_context_window(crate::route_budget::route_context_window_tokens(
|
||||
route.provider,
|
||||
&route.model,
|
||||
route.limits,
|
||||
.with_route_context_window(route.context_window().map_or_else(
|
||||
|| {
|
||||
crate::route_budget::route_context_window_tokens(
|
||||
route.provider,
|
||||
&route.model,
|
||||
route.limits,
|
||||
)
|
||||
},
|
||||
|window| window.tokens,
|
||||
))
|
||||
.with_session_objects(crate::rlm::session::SessionObjectSnapshot::new(
|
||||
self.session.id.clone(),
|
||||
@@ -7993,10 +8018,15 @@ impl Engine {
|
||||
authority.auto_approve,
|
||||
)
|
||||
.with_state_namespace(self.session.id.clone())
|
||||
.with_route_context_window(crate::route_budget::route_context_window_tokens(
|
||||
route.provider,
|
||||
&route.model,
|
||||
route.limits,
|
||||
.with_route_context_window(route.context_window().map_or_else(
|
||||
|| {
|
||||
crate::route_budget::route_context_window_tokens(
|
||||
route.provider,
|
||||
&route.model,
|
||||
route.limits,
|
||||
)
|
||||
},
|
||||
|window| window.tokens,
|
||||
))
|
||||
.with_features(self.config.features.clone())
|
||||
.with_shell_manager(self.shell_manager.clone())
|
||||
@@ -9684,10 +9714,15 @@ impl Engine {
|
||||
locale_tag: &self.config.locale_tag,
|
||||
translation_enabled: context.translation_enabled,
|
||||
model_id: &context.model,
|
||||
context_window_override: Some(crate::route_budget::route_context_window_tokens(
|
||||
context.provider,
|
||||
&context.model,
|
||||
context.route_limits,
|
||||
context_window_override: Some(context.context_window.map_or_else(
|
||||
|| {
|
||||
crate::route_budget::route_context_window_tokens(
|
||||
context.provider,
|
||||
&context.model,
|
||||
context.route_limits,
|
||||
)
|
||||
},
|
||||
|window| window.tokens,
|
||||
)),
|
||||
verbosity: context.verbosity.as_deref(),
|
||||
recovery_hint: recovery_hint.as_deref(),
|
||||
@@ -9706,14 +9741,18 @@ impl Engine {
|
||||
) -> crate::project_context::InstructionBudgets {
|
||||
crate::project_context::InstructionBudgets::resolve_or_default(
|
||||
&self.instruction_settings.budget,
|
||||
crate::route_runtime::resolve_context_window(
|
||||
context.provider,
|
||||
&context.model,
|
||||
context.route_limits,
|
||||
None,
|
||||
None,
|
||||
)
|
||||
.instruction_budget_tokens(),
|
||||
context
|
||||
.context_window
|
||||
.unwrap_or_else(|| {
|
||||
crate::route_runtime::resolve_context_window(
|
||||
context.provider,
|
||||
&context.model,
|
||||
context.route_limits,
|
||||
None,
|
||||
None,
|
||||
)
|
||||
})
|
||||
.instruction_budget_tokens(),
|
||||
context.planned_output_tokens.map(|tokens| tokens as usize),
|
||||
)
|
||||
}
|
||||
@@ -9843,20 +9882,23 @@ impl Engine {
|
||||
self.config.translation_enabled,
|
||||
self.config.verbosity.clone(),
|
||||
);
|
||||
context.context_window = self.active_context_window;
|
||||
let effort = turn_loop::resolve_auto_effort(
|
||||
self.session.reasoning_effort.as_deref(),
|
||||
self.api_provider,
|
||||
&self.api_config.active_route_base_url(),
|
||||
&self.config.model,
|
||||
);
|
||||
context.planned_output_tokens =
|
||||
Some(crate::route_budget::effective_max_output_tokens_for_turn(
|
||||
context.planned_output_tokens = Some(
|
||||
crate::route_budget::effective_max_output_tokens_for_turn_with_window(
|
||||
self.api_provider,
|
||||
&self.config.model,
|
||||
self.active_route_limits,
|
||||
self.active_context_window.map(|window| window.tokens),
|
||||
None,
|
||||
effort.as_deref(),
|
||||
));
|
||||
),
|
||||
);
|
||||
context
|
||||
}
|
||||
}
|
||||
@@ -10576,6 +10618,7 @@ pub(crate) struct NextTurnPromptContext {
|
||||
pub(crate) provider: ProviderKind,
|
||||
pub(crate) model: String,
|
||||
pub(crate) route_limits: Option<codewhale_config::route::RouteLimits>,
|
||||
pub(crate) context_window: Option<crate::route_runtime::ContextWindowResolution>,
|
||||
pub(crate) planned_output_tokens: Option<u32>,
|
||||
pub(crate) mode: AppMode,
|
||||
pub(crate) goal_objective: Option<String>,
|
||||
@@ -10589,6 +10632,7 @@ impl PartialEq for NextTurnPromptContext {
|
||||
self.provider == other.provider
|
||||
&& self.model == other.model
|
||||
&& self.route_limits == other.route_limits
|
||||
&& self.context_window == other.context_window
|
||||
&& self.mode == other.mode
|
||||
&& self.goal_objective == other.goal_objective
|
||||
&& self.goal_token_budget == other.goal_token_budget
|
||||
@@ -10612,7 +10656,7 @@ pub(crate) fn explicit_prompt_context_change_reason(
|
||||
if pinned.model != next.model {
|
||||
fields.push("model");
|
||||
}
|
||||
if pinned.route_limits != next.route_limits {
|
||||
if pinned.route_limits != next.route_limits || pinned.context_window != next.context_window {
|
||||
fields.push("route");
|
||||
}
|
||||
if pinned.mode != next.mode {
|
||||
@@ -10660,6 +10704,7 @@ impl NextTurnPromptContext {
|
||||
)),
|
||||
model,
|
||||
route_limits,
|
||||
context_window: None,
|
||||
mode,
|
||||
goal_objective: (goal_status == GoalStatus::Active)
|
||||
.then(|| normalized_goal_objective(goal_objective.as_deref()))
|
||||
@@ -10841,6 +10886,12 @@ pub(crate) struct TurnRouteContext {
|
||||
}
|
||||
|
||||
impl TurnRouteContext {
|
||||
fn context_window(&self) -> Option<crate::route_runtime::ContextWindowResolution> {
|
||||
self.client
|
||||
.as_ref()
|
||||
.and_then(|client| client.context_window_resolution_for_model(&self.model))
|
||||
}
|
||||
|
||||
pub(crate) fn capability_profile(&self) -> crate::model_profile::CapabilityProfile {
|
||||
crate::model_profile::resolved_capability_profile_for_route(
|
||||
self.provider,
|
||||
@@ -10959,6 +11010,7 @@ use crate::compaction::estimate_input_tokens_conservative;
|
||||
#[cfg(test)]
|
||||
pub(crate) use context::compact_tool_result_for_context;
|
||||
pub(crate) use context::compact_tool_result_for_route;
|
||||
pub(crate) use context::compact_tool_result_for_route_with_window;
|
||||
/// Public so external hosts/wrappers can reuse the engine's input-budget math
|
||||
/// (see `context_input_budget_for_route`'s doc) instead of re-deriving it.
|
||||
pub use context::context_input_budget_for_route;
|
||||
|
||||
@@ -209,10 +209,11 @@ impl Engine {
|
||||
}
|
||||
let input_tokens =
|
||||
self.active_input_tokens_with_current_text(current_text, system_prompt, messages);
|
||||
let budget = route_context_budget_for_route(
|
||||
let budget = crate::route_budget::route_context_budget_with_window(
|
||||
prompt_context.provider,
|
||||
&prompt_context.model,
|
||||
prompt_context.route_limits,
|
||||
prompt_context.context_window.map(|window| window.tokens),
|
||||
input_tokens,
|
||||
)?;
|
||||
context_pressure_message(budget.usage_percent()).map(|warning| format!(
|
||||
@@ -878,12 +879,14 @@ impl Engine {
|
||||
.await;
|
||||
return false;
|
||||
}
|
||||
let Some(target_budget) = context_input_budget_for_route(
|
||||
let Some(target_budget) = crate::route_budget::route_context_budget_with_window(
|
||||
self.api_provider,
|
||||
&self.session.model,
|
||||
self.active_route_limits,
|
||||
self.active_context_window.map(|window| window.tokens),
|
||||
0,
|
||||
) else {
|
||||
)
|
||||
.and_then(|budget| usize::try_from(budget.available_input_tokens).ok()) else {
|
||||
return false;
|
||||
};
|
||||
if let Some(message) = turn.compaction_thrash_message() {
|
||||
|
||||
@@ -609,6 +609,25 @@ pub(crate) fn compact_tool_result_for_route(
|
||||
tool_result_context_view(provider, model, route_limits, tool_name, output).text
|
||||
}
|
||||
|
||||
pub(crate) fn compact_tool_result_for_route_with_window(
|
||||
provider: ProviderKind,
|
||||
model: &str,
|
||||
route_limits: Option<RouteLimits>,
|
||||
effective_window: Option<u32>,
|
||||
tool_name: &str,
|
||||
output: &ToolResult,
|
||||
) -> String {
|
||||
tool_result_context_view_with_window(
|
||||
provider,
|
||||
model,
|
||||
route_limits,
|
||||
effective_window,
|
||||
tool_name,
|
||||
output,
|
||||
)
|
||||
.text
|
||||
}
|
||||
|
||||
/// The model's view of one tool result (#6508).
|
||||
///
|
||||
/// There is one size authority: [`crate::route_budget::route_inline_char_budget`].
|
||||
@@ -628,6 +647,17 @@ pub(crate) fn tool_result_context_view(
|
||||
route_limits: Option<RouteLimits>,
|
||||
tool_name: &str,
|
||||
output: &ToolResult,
|
||||
) -> ToolResultContextView {
|
||||
tool_result_context_view_with_window(provider, model, route_limits, None, tool_name, output)
|
||||
}
|
||||
|
||||
pub(crate) fn tool_result_context_view_with_window(
|
||||
provider: ProviderKind,
|
||||
model: &str,
|
||||
route_limits: Option<RouteLimits>,
|
||||
effective_window: Option<u32>,
|
||||
tool_name: &str,
|
||||
output: &ToolResult,
|
||||
) -> ToolResultContextView {
|
||||
let raw = output.content.trim();
|
||||
if raw.is_empty() {
|
||||
@@ -662,7 +692,9 @@ pub(crate) fn tool_result_context_view(
|
||||
}
|
||||
|
||||
let budget =
|
||||
crate::route_budget::route_inline_char_budget_for_route(provider, model, route_limits);
|
||||
crate::route_budget::route_inline_char_budget(Some(effective_window.unwrap_or_else(
|
||||
|| crate::route_budget::route_context_window_tokens(provider, model, route_limits),
|
||||
)));
|
||||
let recovery = recovery_ref(metadata);
|
||||
let recovery_path = recovery.as_ref().map(|recovery| recovery.path);
|
||||
let retrieval_ref = recovery.as_ref().and_then(|recovery| recovery.artifact_id);
|
||||
|
||||
@@ -380,13 +380,15 @@ impl Engine {
|
||||
&base_url,
|
||||
&model,
|
||||
);
|
||||
let planned_output_tokens = crate::route_budget::effective_max_output_tokens_for_turn(
|
||||
provider,
|
||||
&model,
|
||||
limits,
|
||||
None,
|
||||
effective_reasoning_effort.as_deref(),
|
||||
);
|
||||
let planned_output_tokens =
|
||||
crate::route_budget::effective_max_output_tokens_for_turn_with_window(
|
||||
provider,
|
||||
&model,
|
||||
limits,
|
||||
Some(route.context_window.tokens),
|
||||
None,
|
||||
effective_reasoning_effort.as_deref(),
|
||||
);
|
||||
let (system_prompt, preview_raw, preview_assembly) = if self.session.system_prompt_override
|
||||
|| self.session.pinned_prompt_context.as_ref() == Some(&prompt_context)
|
||||
{
|
||||
|
||||
@@ -1192,7 +1192,7 @@ async fn captured_child_output_caps_preserve_utf8_error_kinds_metadata_and_raw_b
|
||||
))),
|
||||
ProviderKind::Deepseek,
|
||||
DEFAULT_TEXT_MODEL,
|
||||
None,
|
||||
(None, None),
|
||||
"origin-session",
|
||||
("default-cap", "read"),
|
||||
default_child.child_tool_result_token_cap(),
|
||||
@@ -1245,7 +1245,7 @@ async fn captured_child_output_caps_preserve_utf8_error_kinds_metadata_and_raw_b
|
||||
Err(original),
|
||||
ProviderKind::Deepseek,
|
||||
DEFAULT_TEXT_MODEL,
|
||||
None,
|
||||
(None, None),
|
||||
"origin-session",
|
||||
(id, "read"),
|
||||
child.child_tool_result_token_cap(),
|
||||
@@ -1310,7 +1310,7 @@ async fn captured_child_output_caps_preserve_utf8_error_kinds_metadata_and_raw_b
|
||||
)),
|
||||
ProviderKind::Deepseek,
|
||||
DEFAULT_TEXT_MODEL,
|
||||
None,
|
||||
(None, None),
|
||||
"origin-session",
|
||||
("save-failed-cap", "read"),
|
||||
child.child_tool_result_token_cap(),
|
||||
@@ -1338,7 +1338,7 @@ async fn captured_child_output_caps_preserve_utf8_error_kinds_metadata_and_raw_b
|
||||
Err(ToolError::permission_denied(raw.clone())),
|
||||
ProviderKind::Deepseek,
|
||||
DEFAULT_TEXT_MODEL,
|
||||
None,
|
||||
(None, None),
|
||||
"origin-session",
|
||||
("ordinary", "read"),
|
||||
None,
|
||||
|
||||
@@ -216,7 +216,9 @@ fn custom_route_endpoint_change_forgets_the_previous_bill() {
|
||||
.expect("resolve lm-studio")
|
||||
.validate()
|
||||
.expect("preflight lm-studio");
|
||||
let context_window = route.context_window;
|
||||
engine.install_validated_runtime_route(route);
|
||||
assert_eq!(engine.active_context_window, Some(context_window));
|
||||
};
|
||||
install(&mut engine, &config);
|
||||
engine.session.latest_parent_input_tokens = Some(150_000);
|
||||
|
||||
@@ -229,7 +229,7 @@ pub(super) async fn preserve_tool_output_before_fanout(
|
||||
result: Result<RichToolResult, ToolError>,
|
||||
provider: ProviderKind,
|
||||
model: &str,
|
||||
route_limits: Option<codewhale_config::route::RouteLimits>,
|
||||
route_budget: (Option<codewhale_config::route::RouteLimits>, Option<u32>),
|
||||
session_id: &str,
|
||||
tool_call: (&str, &str),
|
||||
child_output_cap: Option<std::num::NonZeroU32>,
|
||||
@@ -360,10 +360,11 @@ pub(super) async fn preserve_tool_output_before_fanout(
|
||||
"path": path.display().to_string(),
|
||||
}));
|
||||
}
|
||||
if super::context::tool_result_context_view(
|
||||
if super::context::tool_result_context_view_with_window(
|
||||
provider,
|
||||
&model,
|
||||
route_limits,
|
||||
route_budget.0,
|
||||
route_budget.1,
|
||||
&tool_name,
|
||||
&rich.result,
|
||||
)
|
||||
@@ -2916,6 +2917,7 @@ impl Engine {
|
||||
let provider = self.api_provider;
|
||||
let model = self.session.model.clone();
|
||||
let route_limits = self.active_route_limits;
|
||||
let route_window = self.active_context_window.map(|window| window.tokens);
|
||||
let child_output_cap = self.child_tool_result_token_cap();
|
||||
let started_at = Instant::now();
|
||||
let shell_permits = shell_permits.clone();
|
||||
@@ -2978,7 +2980,7 @@ impl Engine {
|
||||
result,
|
||||
provider,
|
||||
&model,
|
||||
route_limits,
|
||||
(route_limits, route_window),
|
||||
&session_id,
|
||||
(&plan.id, &plan.name),
|
||||
child_output_cap,
|
||||
@@ -3646,7 +3648,10 @@ impl Engine {
|
||||
result,
|
||||
self.api_provider,
|
||||
&self.session.model,
|
||||
self.active_route_limits,
|
||||
(
|
||||
self.active_route_limits,
|
||||
self.active_context_window.map(|window| window.tokens),
|
||||
),
|
||||
&self.session.id,
|
||||
(&tool_id, &tool_name),
|
||||
self.child_tool_result_token_cap(),
|
||||
@@ -4516,13 +4521,15 @@ impl Engine {
|
||||
"status": terminal_status.as_str(),
|
||||
"success": output.success,
|
||||
}));
|
||||
let output_for_context = compact_tool_result_for_route(
|
||||
self.api_provider,
|
||||
&self.session.model,
|
||||
self.active_route_limits,
|
||||
&outcome.name,
|
||||
&output,
|
||||
);
|
||||
let output_for_context =
|
||||
super::context::compact_tool_result_for_route_with_window(
|
||||
self.api_provider,
|
||||
&self.session.model,
|
||||
self.active_route_limits,
|
||||
self.active_context_window.map(|window| window.tokens),
|
||||
&outcome.name,
|
||||
&output,
|
||||
);
|
||||
let tool_was_executed = output
|
||||
.metadata
|
||||
.as_ref()
|
||||
|
||||
@@ -195,13 +195,15 @@ impl Engine {
|
||||
&self.api_config.active_route_base_url(),
|
||||
&self.config.model,
|
||||
);
|
||||
let mut output_tokens = crate::route_budget::effective_max_output_tokens_for_turn(
|
||||
self.api_provider,
|
||||
&self.session.model,
|
||||
self.active_route_limits,
|
||||
turn.max_output_tokens,
|
||||
guard_effort.as_deref(),
|
||||
);
|
||||
let mut output_tokens =
|
||||
crate::route_budget::effective_max_output_tokens_for_turn_with_window(
|
||||
self.api_provider,
|
||||
&self.session.model,
|
||||
self.active_route_limits,
|
||||
self.active_context_window.map(|window| window.tokens),
|
||||
turn.max_output_tokens,
|
||||
guard_effort.as_deref(),
|
||||
);
|
||||
if turn.max_output_tokens.is_some() {
|
||||
output_tokens =
|
||||
output_tokens.min(client.effective_max_output_tokens(&self.session.model));
|
||||
@@ -256,17 +258,19 @@ impl Engine {
|
||||
)
|
||||
.and_then(|tokens| usize::try_from(tokens).ok())
|
||||
.unwrap_or(0);
|
||||
if let Some(budget) = route_context_budget_for_route(
|
||||
if let Some(budget) = crate::route_budget::route_context_budget_with_window(
|
||||
self.api_provider,
|
||||
&self.session.model,
|
||||
self.active_route_limits,
|
||||
self.active_context_window.map(|window| window.tokens),
|
||||
estimated_input,
|
||||
) {
|
||||
let input_budget = usize::try_from(
|
||||
crate::route_budget::route_admission_input_ceiling(
|
||||
crate::route_budget::route_admission_input_ceiling_with_window(
|
||||
self.api_provider,
|
||||
&self.session.model,
|
||||
self.active_route_limits,
|
||||
self.active_context_window.map(|window| window.tokens),
|
||||
)
|
||||
.unwrap_or(budget.input_budget_ceiling),
|
||||
)
|
||||
@@ -291,10 +295,11 @@ impl Engine {
|
||||
resolved_route_window_tokens = budget.window_tokens,
|
||||
resolved_model_output_ceiling_tokens = ?output_ceiling.clamp_tokens(),
|
||||
resolved_model_output_ceiling_source = output_ceiling.as_str(),
|
||||
effective_request_output_cap_tokens = crate::route_budget::effective_max_output_tokens_for_turn(
|
||||
effective_request_output_cap_tokens = crate::route_budget::effective_max_output_tokens_for_turn_with_window(
|
||||
self.api_provider,
|
||||
&self.session.model,
|
||||
self.active_route_limits,
|
||||
self.active_context_window.map(|window| window.tokens),
|
||||
turn.max_output_tokens,
|
||||
effective_reasoning_effort.as_deref(),
|
||||
),
|
||||
@@ -520,10 +525,11 @@ impl Engine {
|
||||
}
|
||||
messages
|
||||
},
|
||||
max_tokens: crate::route_budget::effective_max_output_tokens_for_turn(
|
||||
max_tokens: crate::route_budget::effective_max_output_tokens_for_turn_with_window(
|
||||
self.api_provider,
|
||||
&self.session.model,
|
||||
self.active_route_limits,
|
||||
self.active_context_window.map(|window| window.tokens),
|
||||
turn.max_output_tokens,
|
||||
effective_reasoning_effort.as_deref(),
|
||||
),
|
||||
@@ -597,10 +603,11 @@ impl Engine {
|
||||
let request_input = turn
|
||||
.calibrated_input_tokens(request_input)
|
||||
.max(estimated_input);
|
||||
let output_cap = crate::route_budget::request_output_cap_for_input(
|
||||
let output_cap = crate::route_budget::request_output_cap_for_input_with_window(
|
||||
self.api_provider,
|
||||
&self.session.model,
|
||||
self.active_route_limits,
|
||||
self.active_context_window.map(|window| window.tokens),
|
||||
request.max_tokens,
|
||||
u64::try_from(request_input).unwrap_or(u64::MAX),
|
||||
)
|
||||
@@ -610,10 +617,11 @@ impl Engine {
|
||||
});
|
||||
let Some(output_cap) = output_cap else {
|
||||
progress.reasoning_only_nudge = request_nudge;
|
||||
let input_budget = crate::route_budget::route_admission_input_ceiling(
|
||||
let input_budget = crate::route_budget::route_admission_input_ceiling_with_window(
|
||||
self.api_provider,
|
||||
&self.session.model,
|
||||
self.active_route_limits,
|
||||
self.active_context_window.map(|window| window.tokens),
|
||||
)
|
||||
.map_or(0, |ceiling| usize::try_from(ceiling).unwrap_or(usize::MAX));
|
||||
return self
|
||||
@@ -820,13 +828,15 @@ impl Engine {
|
||||
&self.api_config.active_route_base_url(),
|
||||
&self.config.model,
|
||||
);
|
||||
let mut planned_output_tokens = crate::route_budget::effective_max_output_tokens_for_turn(
|
||||
self.api_provider,
|
||||
&self.session.model,
|
||||
self.active_route_limits,
|
||||
turn.max_output_tokens,
|
||||
reasoning_effort.as_deref(),
|
||||
);
|
||||
let mut planned_output_tokens =
|
||||
crate::route_budget::effective_max_output_tokens_for_turn_with_window(
|
||||
self.api_provider,
|
||||
&self.session.model,
|
||||
self.active_route_limits,
|
||||
self.active_context_window.map(|window| window.tokens),
|
||||
turn.max_output_tokens,
|
||||
reasoning_effort.as_deref(),
|
||||
);
|
||||
if turn.max_output_tokens.is_some() {
|
||||
planned_output_tokens =
|
||||
planned_output_tokens.min(client.effective_max_output_tokens(&self.session.model));
|
||||
@@ -844,18 +854,20 @@ impl Engine {
|
||||
self.session.latest_parent_input_tokens,
|
||||
)
|
||||
.unwrap_or(0);
|
||||
if route_context_budget_for_route(
|
||||
if crate::route_budget::route_context_budget_with_window(
|
||||
self.api_provider,
|
||||
&self.session.model,
|
||||
self.active_route_limits,
|
||||
self.active_context_window.map(|window| window.tokens),
|
||||
usize::try_from(estimated).unwrap_or(usize::MAX),
|
||||
)
|
||||
.is_some_and(|budget| {
|
||||
estimated
|
||||
> crate::route_budget::route_admission_input_ceiling(
|
||||
> crate::route_budget::route_admission_input_ceiling_with_window(
|
||||
self.api_provider,
|
||||
&self.session.model,
|
||||
self.active_route_limits,
|
||||
self.active_context_window.map(|window| window.tokens),
|
||||
)
|
||||
.unwrap_or(budget.input_budget_ceiling)
|
||||
}) {
|
||||
@@ -864,10 +876,11 @@ impl Engine {
|
||||
let mut request = prepare_primary_turn_request(PrimaryTurnRequest {
|
||||
model: self.session.model.clone(),
|
||||
messages: self.messages_with_turn_metadata(),
|
||||
max_tokens: crate::route_budget::effective_max_output_tokens_for_turn(
|
||||
max_tokens: crate::route_budget::effective_max_output_tokens_for_turn_with_window(
|
||||
self.api_provider,
|
||||
&self.session.model,
|
||||
self.active_route_limits,
|
||||
self.active_context_window.map(|window| window.tokens),
|
||||
turn.max_output_tokens,
|
||||
reasoning_effort.as_deref(),
|
||||
),
|
||||
@@ -876,10 +889,11 @@ impl Engine {
|
||||
tool_choice: None,
|
||||
reasoning_effort,
|
||||
});
|
||||
let Some(output_cap) = crate::route_budget::request_output_cap_for_input(
|
||||
let Some(output_cap) = crate::route_budget::request_output_cap_for_input_with_window(
|
||||
self.api_provider,
|
||||
&self.session.model,
|
||||
self.active_route_limits,
|
||||
self.active_context_window.map(|window| window.tokens),
|
||||
request.max_tokens,
|
||||
estimated,
|
||||
) else {
|
||||
|
||||
@@ -30,12 +30,18 @@ pub trait ModelClient: Send + Sync {
|
||||
fn route_limits(&self) -> Option<codewhale_config::route::RouteLimits> {
|
||||
None
|
||||
}
|
||||
fn effective_context_window(&self, _model: &str) -> Option<u32> {
|
||||
None
|
||||
}
|
||||
fn effective_max_output_tokens(&self, requested_model: &str) -> u32 {
|
||||
let route = self.effective_route_envelope(requested_model, chrono::Utc::now());
|
||||
crate::route_budget::effective_max_output_tokens_for_route(
|
||||
crate::route_budget::effective_max_output_tokens_for_turn_with_window(
|
||||
route.provider,
|
||||
&route.model,
|
||||
self.route_limits(),
|
||||
self.effective_context_window(requested_model),
|
||||
None,
|
||||
None,
|
||||
)
|
||||
}
|
||||
fn effective_route_envelope(
|
||||
@@ -96,6 +102,10 @@ where
|
||||
LlmClient::route_limits(self)
|
||||
}
|
||||
|
||||
fn effective_context_window(&self, model: &str) -> Option<u32> {
|
||||
LlmClient::effective_context_window(self, model)
|
||||
}
|
||||
|
||||
fn effective_max_output_tokens(&self, requested_model: &str) -> u32 {
|
||||
LlmClient::effective_max_output_tokens(self, requested_model)
|
||||
}
|
||||
|
||||
@@ -569,6 +569,7 @@ pub(crate) async fn run_exec_agent(
|
||||
);
|
||||
let active_route_limits =
|
||||
crate::route_budget::known_route_limits(validated_route.candidate.limits());
|
||||
let effective_context_window = validated_route.context_window;
|
||||
let max_subagents = if max_subagents
|
||||
== config.max_subagents_for_provider(
|
||||
&config
|
||||
@@ -609,15 +610,12 @@ pub(crate) async fn run_exec_agent(
|
||||
let compaction = CompactionConfig {
|
||||
enabled: auto_compact_enabled,
|
||||
model: effective_model.clone(),
|
||||
effective_context_window: Some(crate::route_budget::route_context_window_tokens(
|
||||
effective_provider,
|
||||
&effective_model,
|
||||
active_route_limits,
|
||||
)),
|
||||
token_threshold: crate::route_budget::compaction_threshold_for_route_at_percent(
|
||||
effective_context_window: Some(validated_route.context_window.tokens),
|
||||
token_threshold: crate::route_budget::compaction_threshold_for_route_at_percent_with_window(
|
||||
effective_provider,
|
||||
&effective_model,
|
||||
active_route_limits,
|
||||
Some(validated_route.context_window.tokens),
|
||||
settings.auto_compact_threshold_percent,
|
||||
),
|
||||
summary_instructions: execution_config.compaction_summary_instructions(),
|
||||
@@ -987,7 +985,7 @@ pub(crate) async fn run_exec_agent(
|
||||
let mut latest_session_id = loaded_session_id;
|
||||
let mut latest_messages: Arc<Vec<Message>> = Arc::new(Vec::new());
|
||||
let mut latest_system_prompt: Option<SystemPrompt> = None;
|
||||
let mut latest_model = effective_model;
|
||||
let mut latest_model = effective_model.clone();
|
||||
let mut latest_workspace = workspace.clone();
|
||||
let mut tool_starts: HashMap<String, (Instant, String)> = HashMap::new();
|
||||
let mut turn_usage_seq: u32 = 0;
|
||||
@@ -1570,10 +1568,14 @@ pub(crate) async fn run_exec_agent(
|
||||
// wrong ceiling is visible in the receipt rather than
|
||||
// requiring packet capture.
|
||||
let codewhale_max_output_tokens =
|
||||
crate::route_budget::effective_max_output_tokens_for_route(
|
||||
crate::route_budget::effective_max_output_tokens_for_turn_with_window(
|
||||
effective_provider,
|
||||
&latest_model,
|
||||
active_route_limits,
|
||||
(latest_model == effective_model)
|
||||
.then_some(effective_context_window.tokens),
|
||||
None,
|
||||
None,
|
||||
);
|
||||
let codewhale_max_output_tokens_source =
|
||||
crate::route_budget::output_ceiling_source(
|
||||
|
||||
@@ -118,13 +118,20 @@ pub trait LlmClient: Send + Sync {
|
||||
None
|
||||
}
|
||||
|
||||
fn effective_context_window(&self, _model: &str) -> Option<u32> {
|
||||
None
|
||||
}
|
||||
|
||||
/// Output cap for a request sent through this exact client route.
|
||||
fn effective_max_output_tokens(&self, requested_model: &str) -> u32 {
|
||||
let route = self.effective_route_envelope(requested_model, chrono::Utc::now());
|
||||
crate::route_budget::effective_max_output_tokens_for_route(
|
||||
crate::route_budget::effective_max_output_tokens_for_turn_with_window(
|
||||
route.provider,
|
||||
&route.model,
|
||||
self.route_limits(),
|
||||
self.effective_context_window(requested_model),
|
||||
None,
|
||||
None,
|
||||
)
|
||||
}
|
||||
|
||||
|
||||
+111
-23
@@ -37,11 +37,30 @@ pub(crate) fn effective_max_output_tokens_for_turn(
|
||||
route_limits: Option<RouteLimits>,
|
||||
allowance: Option<std::num::NonZeroU32>,
|
||||
reasoning_effort: Option<&str>,
|
||||
) -> u32 {
|
||||
effective_max_output_tokens_for_turn_with_window(
|
||||
provider,
|
||||
model,
|
||||
route_limits,
|
||||
None,
|
||||
allowance,
|
||||
reasoning_effort,
|
||||
)
|
||||
}
|
||||
|
||||
pub(crate) fn effective_max_output_tokens_for_turn_with_window(
|
||||
provider: ProviderKind,
|
||||
model: &str,
|
||||
route_limits: Option<RouteLimits>,
|
||||
effective_window: Option<u32>,
|
||||
allowance: Option<std::num::NonZeroU32>,
|
||||
reasoning_effort: Option<&str>,
|
||||
) -> u32 {
|
||||
let ceiling = effective_max_output_tokens_for_route_with_cap(
|
||||
provider,
|
||||
model,
|
||||
route_limits,
|
||||
effective_window,
|
||||
automatic_max_output_tokens(provider, reasoning_effort),
|
||||
);
|
||||
allowance.map_or(ceiling, |allowance| ceiling.min(allowance.get()))
|
||||
@@ -347,6 +366,7 @@ pub(crate) fn effective_max_output_tokens_for_route(
|
||||
provider,
|
||||
model,
|
||||
route_limits,
|
||||
None,
|
||||
API_MAX_OUTPUT_TOKENS,
|
||||
)
|
||||
}
|
||||
@@ -355,9 +375,12 @@ fn effective_max_output_tokens_for_route_with_cap(
|
||||
provider: ProviderKind,
|
||||
model: &str,
|
||||
route_limits: Option<RouteLimits>,
|
||||
effective_window: Option<u32>,
|
||||
automatic_cap: u32,
|
||||
) -> u32 {
|
||||
let window = route_context_window_tokens(provider, model, route_limits);
|
||||
let window = effective_window
|
||||
.filter(|tokens| *tokens > 0)
|
||||
.unwrap_or_else(|| route_context_window_tokens(provider, model, route_limits));
|
||||
let compatibility_source = output_ceiling_source(provider, model);
|
||||
let compatibility_cap = compatibility_source.clamp_tokens();
|
||||
let route_cap = route_output_limit_tokens(route_limits);
|
||||
@@ -482,11 +505,21 @@ pub(crate) fn review_visible_text_reserve_tokens(model: &str, allowance: u32) ->
|
||||
}
|
||||
|
||||
/// Output reservation used by the internal input budget for a route.
|
||||
#[cfg(test)]
|
||||
#[must_use]
|
||||
pub(crate) fn route_output_reservation(
|
||||
provider: ProviderKind,
|
||||
model: &str,
|
||||
route_limits: Option<RouteLimits>,
|
||||
) -> u32 {
|
||||
route_output_reservation_with_window(provider, model, route_limits, None)
|
||||
}
|
||||
|
||||
pub(crate) fn route_output_reservation_with_window(
|
||||
provider: ProviderKind,
|
||||
model: &str,
|
||||
route_limits: Option<RouteLimits>,
|
||||
effective_window: Option<u32>,
|
||||
) -> u32 {
|
||||
// Use exactly the value that can reach the wire on every window size,
|
||||
// except on a route whose published output limit is more than half its
|
||||
@@ -498,39 +531,43 @@ pub(crate) fn route_output_reservation(
|
||||
// disagree with the actual request. Reasoning effort is a request control,
|
||||
// not separately metered non-wire output, so it does not justify a second
|
||||
// hidden context reservation.
|
||||
let ceiling = effective_max_output_tokens_for_route(provider, model, route_limits);
|
||||
if shares_context_window(provider, model, route_limits) {
|
||||
ceiling.min(
|
||||
(route_context_window_tokens(provider, model, route_limits) / 4)
|
||||
.clamp(1, UNCATALOGUED_COMPAT_MAX_OUTPUT_TOKENS),
|
||||
)
|
||||
let window = effective_window
|
||||
.filter(|tokens| *tokens > 0)
|
||||
.unwrap_or_else(|| route_context_window_tokens(provider, model, route_limits));
|
||||
let ceiling = effective_max_output_tokens_for_route_with_cap(
|
||||
provider,
|
||||
model,
|
||||
route_limits,
|
||||
Some(window),
|
||||
API_MAX_OUTPUT_TOKENS,
|
||||
);
|
||||
if shares_context_window(route_limits, window) {
|
||||
ceiling.min((window / 4).clamp(1, UNCATALOGUED_COMPAT_MAX_OUTPUT_TOKENS))
|
||||
} else {
|
||||
ceiling
|
||||
}
|
||||
}
|
||||
|
||||
fn shares_context_window(
|
||||
provider: ProviderKind,
|
||||
model: &str,
|
||||
route_limits: Option<RouteLimits>,
|
||||
) -> bool {
|
||||
fn shares_context_window(route_limits: Option<RouteLimits>, window: u32) -> bool {
|
||||
route_limits
|
||||
.and_then(|limits| limits.context_tokens)
|
||||
.is_some_and(|tokens| (1..=u64::from(u32::MAX)).contains(&tokens))
|
||||
&& explicit_max_output_tokens_override().is_none()
|
||||
&& route_output_limit_tokens(route_limits)
|
||||
.is_some_and(|cap| cap > route_context_window_tokens(provider, model, route_limits) / 2)
|
||||
&& route_output_limit_tokens(route_limits).is_some_and(|cap| cap > window / 2)
|
||||
}
|
||||
|
||||
#[must_use]
|
||||
pub(crate) fn route_admission_input_ceiling(
|
||||
pub(crate) fn route_admission_input_ceiling_with_window(
|
||||
provider: ProviderKind,
|
||||
model: &str,
|
||||
route_limits: Option<RouteLimits>,
|
||||
effective_window: Option<u32>,
|
||||
) -> Option<u64> {
|
||||
shares_context_window(provider, model, route_limits).then(|| {
|
||||
let window = effective_window
|
||||
.filter(|tokens| *tokens > 0)
|
||||
.unwrap_or_else(|| route_context_window_tokens(provider, model, route_limits));
|
||||
shares_context_window(route_limits, window).then(|| {
|
||||
ContextBudget::new_with_input_limit(
|
||||
u64::from(route_context_window_tokens(provider, model, route_limits)),
|
||||
u64::from(window),
|
||||
0,
|
||||
MIN_REQUEST_OUTPUT_TOKENS,
|
||||
route_input_limit_tokens(route_limits).map(u64::from),
|
||||
@@ -547,10 +584,31 @@ pub(crate) fn request_output_cap_for_input(
|
||||
ceiling: u32,
|
||||
estimated_input_tokens: u64,
|
||||
) -> Option<u32> {
|
||||
if !shares_context_window(provider, model, route_limits) {
|
||||
request_output_cap_for_input_with_window(
|
||||
provider,
|
||||
model,
|
||||
route_limits,
|
||||
None,
|
||||
ceiling,
|
||||
estimated_input_tokens,
|
||||
)
|
||||
}
|
||||
|
||||
pub(crate) fn request_output_cap_for_input_with_window(
|
||||
provider: ProviderKind,
|
||||
model: &str,
|
||||
route_limits: Option<RouteLimits>,
|
||||
effective_window: Option<u32>,
|
||||
ceiling: u32,
|
||||
estimated_input_tokens: u64,
|
||||
) -> Option<u32> {
|
||||
let window = effective_window
|
||||
.filter(|tokens| *tokens > 0)
|
||||
.unwrap_or_else(|| route_context_window_tokens(provider, model, route_limits));
|
||||
if !shares_context_window(route_limits, window) {
|
||||
return Some(ceiling);
|
||||
}
|
||||
let remaining = u64::from(route_context_window_tokens(provider, model, route_limits))
|
||||
let remaining = u64::from(window)
|
||||
.saturating_sub(estimated_input_tokens)
|
||||
.saturating_sub(codewhale_runtime::context_budget::CONTEXT_HEADROOM_TOKENS);
|
||||
u32::try_from(remaining.min(u64::from(ceiling)))
|
||||
@@ -565,8 +623,21 @@ pub(crate) fn route_context_budget(
|
||||
route_limits: Option<RouteLimits>,
|
||||
input_tokens: usize,
|
||||
) -> Option<ContextBudget> {
|
||||
let window = route_context_window_tokens(provider, model, route_limits);
|
||||
let output_cap = route_output_reservation(provider, model, route_limits);
|
||||
route_context_budget_with_window(provider, model, route_limits, None, input_tokens)
|
||||
}
|
||||
|
||||
pub(crate) fn route_context_budget_with_window(
|
||||
provider: ProviderKind,
|
||||
model: &str,
|
||||
route_limits: Option<RouteLimits>,
|
||||
effective_window: Option<u32>,
|
||||
input_tokens: usize,
|
||||
) -> Option<ContextBudget> {
|
||||
let window = effective_window
|
||||
.filter(|tokens| *tokens > 0)
|
||||
.unwrap_or_else(|| route_context_window_tokens(provider, model, route_limits));
|
||||
let output_cap =
|
||||
route_output_reservation_with_window(provider, model, route_limits, Some(window));
|
||||
Some(ContextBudget::new_with_input_limit(
|
||||
u64::from(window),
|
||||
u64::try_from(input_tokens).ok()?,
|
||||
@@ -575,6 +646,7 @@ pub(crate) fn route_context_budget(
|
||||
))
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
#[must_use]
|
||||
pub(crate) fn compaction_threshold_for_route_at_percent(
|
||||
provider: ProviderKind,
|
||||
@@ -582,7 +654,23 @@ pub(crate) fn compaction_threshold_for_route_at_percent(
|
||||
route_limits: Option<RouteLimits>,
|
||||
percent: f64,
|
||||
) -> usize {
|
||||
route_context_budget(provider, model, route_limits, 0)
|
||||
compaction_threshold_for_route_at_percent_with_window(
|
||||
provider,
|
||||
model,
|
||||
route_limits,
|
||||
None,
|
||||
percent,
|
||||
)
|
||||
}
|
||||
|
||||
pub(crate) fn compaction_threshold_for_route_at_percent_with_window(
|
||||
provider: ProviderKind,
|
||||
model: &str,
|
||||
route_limits: Option<RouteLimits>,
|
||||
effective_window: Option<u32>,
|
||||
percent: f64,
|
||||
) -> usize {
|
||||
route_context_budget_with_window(provider, model, route_limits, effective_window, 0)
|
||||
.and_then(|budget| {
|
||||
usize::try_from(budget.compaction_trigger_for_percent(percent.clamp(10.0, 100.0))).ok()
|
||||
})
|
||||
|
||||
@@ -277,14 +277,16 @@ impl ResolvedRuntimeRoute {
|
||||
self.config.verify_provider_identity(&self.identity)?;
|
||||
if self.preflighted_client.is_none() {
|
||||
self.preflighted_client = Some(Box::new(
|
||||
CodewhaleClient::from_candidate(&self.config, &self.candidate).map_err(|err| {
|
||||
format_provider_route_preflight_error(
|
||||
self.identity.key.as_str(),
|
||||
&self.model,
|
||||
&err,
|
||||
RouteErrorSurface::Interactive,
|
||||
)
|
||||
})?,
|
||||
CodewhaleClient::from_candidate(&self.config, &self.candidate)
|
||||
.map_err(|err| {
|
||||
format_provider_route_preflight_error(
|
||||
self.identity.key.as_str(),
|
||||
&self.model,
|
||||
&err,
|
||||
RouteErrorSurface::Interactive,
|
||||
)
|
||||
})?
|
||||
.with_context_window(self.context_window),
|
||||
));
|
||||
}
|
||||
Ok(self)
|
||||
@@ -312,7 +314,8 @@ impl ResolvedRuntimeRoute {
|
||||
)
|
||||
})?
|
||||
}
|
||||
};
|
||||
}
|
||||
.with_context_window(self.context_window);
|
||||
if client.admitted_provider_identity() != &self.identity {
|
||||
return Err("preflighted client does not match captured provider identity".into());
|
||||
}
|
||||
@@ -1242,6 +1245,18 @@ mod tests {
|
||||
assert_eq!(unknown.candidate.limits().context_tokens, None);
|
||||
assert_eq!(unknown.candidate.limits().output_tokens, None);
|
||||
assert_eq!(unknown.context_window.source, ContextWindowSource::Fallback);
|
||||
let budget = crate::route_budget::route_context_budget_with_window(
|
||||
provider,
|
||||
model,
|
||||
crate::route_budget::known_route_limits(unknown.candidate.limits()),
|
||||
Some(unknown.context_window.tokens),
|
||||
0,
|
||||
)
|
||||
.unwrap();
|
||||
assert_eq!(
|
||||
budget.window_tokens,
|
||||
u64::from(unknown.context_window.tokens)
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -40,8 +40,7 @@ use crate::cost_status::{
|
||||
};
|
||||
use crate::provider_catalog_live::ProviderLivePricingQuote;
|
||||
use crate::route_budget::{
|
||||
auto_compact_default_for_route, compaction_threshold_for_route_at_percent, known_route_limits,
|
||||
route_context_window_tokens,
|
||||
auto_compact_default_for_route, known_route_limits, route_context_window_tokens,
|
||||
};
|
||||
use crate::route_runtime::{ResolvedRuntimeRoute, resolve_runtime_route_for_identity};
|
||||
use crate::tools::plan::new_shared_plan_state;
|
||||
|
||||
@@ -2355,11 +2355,12 @@ pub(super) fn runtime_compaction_config(
|
||||
config: &Config,
|
||||
provider: ProviderKind,
|
||||
model: &str,
|
||||
route_limits: Option<codewhale_config::route::RouteLimits>,
|
||||
route_budget: (Option<codewhale_config::route::RouteLimits>, Option<u32>),
|
||||
auto_compact: bool,
|
||||
auto_compact_explicit: bool,
|
||||
threshold_percent: f64,
|
||||
) -> CompactionConfig {
|
||||
let (route_limits, effective_window) = route_budget;
|
||||
CompactionConfig {
|
||||
enabled: if auto_compact_explicit {
|
||||
auto_compact
|
||||
@@ -2367,13 +2368,17 @@ pub(super) fn runtime_compaction_config(
|
||||
auto_compact_default_for_route(provider, model, route_limits)
|
||||
},
|
||||
model: model.to_string(),
|
||||
token_threshold: compaction_threshold_for_route_at_percent(
|
||||
token_threshold: crate::route_budget::compaction_threshold_for_route_at_percent_with_window(
|
||||
provider,
|
||||
model,
|
||||
route_limits,
|
||||
effective_window,
|
||||
threshold_percent,
|
||||
),
|
||||
effective_context_window: Some(route_context_window_tokens(provider, model, route_limits)),
|
||||
effective_context_window: Some(
|
||||
effective_window
|
||||
.unwrap_or_else(|| route_context_window_tokens(provider, model, route_limits)),
|
||||
),
|
||||
summary_instructions: config.compaction_summary_instructions(),
|
||||
retained_user_message_tokens: config.compaction_retained_user_message_tokens(),
|
||||
..Default::default()
|
||||
@@ -3602,7 +3607,7 @@ impl RuntimeThreadManager {
|
||||
&route.config,
|
||||
provider,
|
||||
&route.model,
|
||||
route_limits,
|
||||
(route_limits, Some(route.context_window.tokens)),
|
||||
settings.auto_compact,
|
||||
codewhale_runtime::settings::Settings::auto_compact_explicitly_configured(),
|
||||
settings.auto_compact_threshold_percent,
|
||||
@@ -11997,10 +12002,11 @@ impl RuntimeThreadManager {
|
||||
let model = route.model.clone();
|
||||
let route_limits = known_route_limits(route.candidate.limits());
|
||||
let max_output_tokens = req.max_output_tokens.and_then(|requested| {
|
||||
std::num::NonZeroU32::new(crate::route_budget::effective_max_output_tokens_for_turn(
|
||||
std::num::NonZeroU32::new(crate::route_budget::effective_max_output_tokens_for_turn_with_window(
|
||||
provider,
|
||||
&model,
|
||||
route_limits,
|
||||
Some(route.context_window.tokens),
|
||||
Some(requested),
|
||||
reasoning_effort.as_deref(),
|
||||
))
|
||||
@@ -12010,7 +12016,7 @@ impl RuntimeThreadManager {
|
||||
&route.config,
|
||||
provider,
|
||||
&model,
|
||||
route_limits,
|
||||
(route_limits, Some(route.context_window.tokens)),
|
||||
settings.auto_compact,
|
||||
codewhale_runtime::settings::Settings::auto_compact_explicitly_configured(),
|
||||
settings.auto_compact_threshold_percent,
|
||||
@@ -12546,7 +12552,7 @@ impl RuntimeThreadManager {
|
||||
&route.config,
|
||||
route_provider,
|
||||
&route_model,
|
||||
route_limits,
|
||||
(route_limits, Some(route.context_window.tokens)),
|
||||
settings.auto_compact,
|
||||
codewhale_runtime::settings::Settings::auto_compact_explicitly_configured(),
|
||||
settings.auto_compact_threshold_percent,
|
||||
@@ -12818,6 +12824,7 @@ impl RuntimeThreadManager {
|
||||
// route limits, compaction budget, and restored session all agree.
|
||||
let base_config = self.read_config().clone();
|
||||
let route = self.resolved_route_for_thread(&base_config, &thread)?;
|
||||
let context_window = route.context_window;
|
||||
let provider = route.identity.provider;
|
||||
let route_identity = route.identity;
|
||||
let route_model = route.model;
|
||||
@@ -12832,7 +12839,7 @@ impl RuntimeThreadManager {
|
||||
&cfg,
|
||||
provider,
|
||||
&route_model,
|
||||
route_limits,
|
||||
(route_limits, Some(context_window.tokens)),
|
||||
settings.auto_compact,
|
||||
codewhale_runtime::settings::Settings::auto_compact_explicitly_configured(),
|
||||
settings.auto_compact_threshold_percent,
|
||||
|
||||
@@ -2720,7 +2720,7 @@ fn runtime_compaction_uses_provider_route_context() {
|
||||
&Config::default(),
|
||||
ProviderKind::OpenaiCodex,
|
||||
"gpt-5.5",
|
||||
Some(limits),
|
||||
(Some(limits), None),
|
||||
false,
|
||||
false,
|
||||
80.0,
|
||||
|
||||
+97
-29
@@ -1487,6 +1487,7 @@ pub(crate) struct PendingProviderSwitch {
|
||||
pub previous_route_limits: Option<RouteLimits>,
|
||||
pub previous_route_base_url: String,
|
||||
pub previous_context_window_source: crate::route_runtime::ContextWindowSource,
|
||||
pub previous_context_window: Option<crate::route_runtime::ContextWindowResolution>,
|
||||
pub previous_context_window_override: Option<u32>,
|
||||
pub previous_config: Config,
|
||||
pub previous_onboarding: OnboardingState,
|
||||
@@ -1710,6 +1711,12 @@ pub struct App {
|
||||
pub auto_model: bool,
|
||||
/// Last concrete model chosen while `auto_model` is active.
|
||||
pub last_effective_model: Option<String>,
|
||||
pub(crate) last_effective_budget: Option<(
|
||||
ProviderKind,
|
||||
String,
|
||||
Option<RouteLimits>,
|
||||
crate::route_runtime::ContextWindowResolution,
|
||||
)>,
|
||||
/// Provider that actually served the latest auto-routed turn.
|
||||
pub last_effective_provider: Option<ProviderKind>,
|
||||
/// Exact non-secret identity for the provider that served the latest Auto
|
||||
@@ -1769,6 +1776,7 @@ pub struct App {
|
||||
/// Provenance for `active_route_limits`' effective context window. This
|
||||
/// is an operator-facing receipt, not a claim about provider billing.
|
||||
pub active_context_window_source: crate::route_runtime::ContextWindowSource,
|
||||
pub(crate) active_context_window: Option<crate::route_runtime::ContextWindowResolution>,
|
||||
/// User-configured provider context-window override for the active route.
|
||||
pub active_context_window_override: Option<u32>,
|
||||
/// `[providers.<id>.model_context_windows]` for the active provider
|
||||
@@ -3222,6 +3230,7 @@ impl App {
|
||||
self.pending_auto_route_receipt = None;
|
||||
self.active_turn = None;
|
||||
self.last_effective_model = None;
|
||||
self.last_effective_budget = None;
|
||||
self.last_effective_provider = None;
|
||||
self.last_effective_provider_identity = None;
|
||||
self.last_auto_route_receipt = None;
|
||||
@@ -3244,6 +3253,7 @@ impl App {
|
||||
self.last_effective_reasoning_effort = None;
|
||||
if self.auto_model {
|
||||
self.last_effective_model = None;
|
||||
self.last_effective_budget = None;
|
||||
self.last_effective_provider = None;
|
||||
self.last_effective_provider_identity = None;
|
||||
self.last_auto_route_receipt = None;
|
||||
@@ -7045,19 +7055,19 @@ impl App {
|
||||
}
|
||||
|
||||
pub fn update_model_compaction_budget(&mut self) {
|
||||
let model = self.effective_model_for_budget().to_string();
|
||||
self.compact_threshold = crate::route_budget::compaction_threshold_for_route_at_percent(
|
||||
self.api_provider,
|
||||
&model,
|
||||
self.active_route_limits,
|
||||
self.auto_compact_threshold_percent,
|
||||
);
|
||||
if !self.auto_compact_user_configured {
|
||||
self.auto_compact = crate::route_budget::auto_compact_default_for_route(
|
||||
self.api_provider,
|
||||
let (provider, model, limits, window) = self.effective_budget_route();
|
||||
let model = model.to_string();
|
||||
self.compact_threshold =
|
||||
crate::route_budget::compaction_threshold_for_route_at_percent_with_window(
|
||||
provider,
|
||||
&model,
|
||||
self.active_route_limits,
|
||||
limits,
|
||||
window.map(|window| window.tokens),
|
||||
self.auto_compact_threshold_percent,
|
||||
);
|
||||
if !self.auto_compact_user_configured {
|
||||
self.auto_compact =
|
||||
crate::route_budget::auto_compact_default_for_route(provider, &model, limits);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -7071,11 +7081,12 @@ impl App {
|
||||
&mut self,
|
||||
base_url: impl Into<String>,
|
||||
limits: RouteLimits,
|
||||
context_window_source: crate::route_runtime::ContextWindowSource,
|
||||
context_window: crate::route_runtime::ContextWindowResolution,
|
||||
) {
|
||||
self.active_route_base_url = base_url.into();
|
||||
self.set_active_route_limits(limits);
|
||||
self.active_context_window_source = context_window_source;
|
||||
self.active_context_window_source = context_window.source;
|
||||
self.active_context_window = Some(context_window);
|
||||
}
|
||||
|
||||
/// Refresh the operator-configured windows for the active provider
|
||||
@@ -7087,7 +7098,8 @@ impl App {
|
||||
) {
|
||||
self.active_context_window_override = config.context_window_for_provider_config(identity);
|
||||
self.active_model_context_windows = config.model_context_windows_for(identity).cloned();
|
||||
if let Some(resolution) = self.configured_context_window_for(&self.model.clone()) {
|
||||
self.active_context_window = self.configured_context_window_for(&self.model.clone());
|
||||
if let Some(resolution) = self.active_context_window {
|
||||
self.active_context_window_source = resolution.source;
|
||||
}
|
||||
if self.active_route_limits.is_none() {
|
||||
@@ -7130,13 +7142,32 @@ impl App {
|
||||
|
||||
pub fn set_model_selection(&mut self, model: String) {
|
||||
let auto_model = model.trim().eq_ignore_ascii_case("auto");
|
||||
let model_changed = self.model != model || self.auto_model != auto_model;
|
||||
self.model = if auto_model {
|
||||
"auto".to_string()
|
||||
} else {
|
||||
model
|
||||
};
|
||||
self.auto_model = auto_model;
|
||||
if model_changed {
|
||||
self.active_route_limits = self.context_window_override_limits();
|
||||
self.active_context_window = self.configured_context_window_for(&self.model);
|
||||
self.active_context_window_source = self.active_context_window.map_or_else(
|
||||
|| {
|
||||
crate::route_runtime::resolve_context_window(
|
||||
self.api_provider,
|
||||
&self.model,
|
||||
self.active_route_limits,
|
||||
None,
|
||||
None,
|
||||
)
|
||||
.source
|
||||
},
|
||||
|window| window.source,
|
||||
);
|
||||
}
|
||||
self.last_effective_model = None;
|
||||
self.last_effective_budget = None;
|
||||
self.last_effective_provider = None;
|
||||
self.last_effective_provider_identity = None;
|
||||
self.last_auto_route_receipt = None;
|
||||
@@ -7298,6 +7329,44 @@ impl App {
|
||||
&self.model
|
||||
}
|
||||
|
||||
pub(crate) fn effective_context_window(&self) -> u32 {
|
||||
let (provider, model, limits, window) = self.effective_budget_route();
|
||||
window.map_or_else(
|
||||
|| crate::route_budget::route_context_window_tokens(provider, model, limits),
|
||||
|window| window.tokens,
|
||||
)
|
||||
}
|
||||
|
||||
pub(crate) fn effective_budget_route(
|
||||
&self,
|
||||
) -> (
|
||||
ProviderKind,
|
||||
&str,
|
||||
Option<RouteLimits>,
|
||||
Option<crate::route_runtime::ContextWindowResolution>,
|
||||
) {
|
||||
if self.auto_model
|
||||
&& let Some((provider, model, limits, window)) = self.last_effective_budget.as_ref()
|
||||
&& self.last_effective_model.as_deref() == Some(model.as_str())
|
||||
{
|
||||
return (*provider, model, *limits, Some(*window));
|
||||
}
|
||||
(
|
||||
self.api_provider,
|
||||
self.effective_model_for_budget(),
|
||||
self.active_route_limits,
|
||||
self.active_context_window,
|
||||
)
|
||||
}
|
||||
|
||||
pub(crate) fn effective_context_window_source(
|
||||
&self,
|
||||
) -> crate::route_runtime::ContextWindowSource {
|
||||
self.effective_budget_route()
|
||||
.3
|
||||
.map_or(self.active_context_window_source, |window| window.source)
|
||||
}
|
||||
|
||||
pub fn model_display_label(&self) -> String {
|
||||
if self.auto_model {
|
||||
if let Some(effective) = self.last_effective_model.as_deref()
|
||||
@@ -7627,11 +7696,8 @@ impl App {
|
||||
}
|
||||
|
||||
pub fn compaction_config(&self) -> CompactionConfig {
|
||||
let mut config = self.compaction_config_for_route(
|
||||
self.api_provider,
|
||||
self.effective_model_for_budget(),
|
||||
self.active_route_limits,
|
||||
);
|
||||
let (provider, model, limits, window) = self.effective_budget_route();
|
||||
let mut config = self.compaction_config_for_route(provider, model, limits, window);
|
||||
// These cached fields are the active-route compatibility authority and
|
||||
// are updated together by `update_model_compaction_budget`. Commands
|
||||
// and embedders may also adjust them directly between route updates.
|
||||
@@ -7651,6 +7717,7 @@ impl App {
|
||||
provider: ProviderKind,
|
||||
model: &str,
|
||||
route_limits: Option<RouteLimits>,
|
||||
context_window: Option<crate::route_runtime::ContextWindowResolution>,
|
||||
) -> CompactionConfig {
|
||||
CompactionConfig {
|
||||
enabled: if self.auto_compact_user_configured {
|
||||
@@ -7658,17 +7725,18 @@ impl App {
|
||||
} else {
|
||||
crate::route_budget::auto_compact_default_for_route(provider, model, route_limits)
|
||||
},
|
||||
token_threshold: crate::route_budget::compaction_threshold_for_route_at_percent(
|
||||
provider,
|
||||
model,
|
||||
route_limits,
|
||||
self.auto_compact_threshold_percent,
|
||||
),
|
||||
token_threshold:
|
||||
crate::route_budget::compaction_threshold_for_route_at_percent_with_window(
|
||||
provider,
|
||||
model,
|
||||
route_limits,
|
||||
context_window.map(|window| window.tokens),
|
||||
self.auto_compact_threshold_percent,
|
||||
),
|
||||
model: model.to_string(),
|
||||
effective_context_window: Some(crate::route_budget::route_context_window_tokens(
|
||||
provider,
|
||||
model,
|
||||
route_limits,
|
||||
effective_context_window: Some(context_window.map_or_else(
|
||||
|| crate::route_budget::route_context_window_tokens(provider, model, route_limits),
|
||||
|window| window.tokens,
|
||||
)),
|
||||
summary_instructions: self.compaction_summary_instructions.clone(),
|
||||
retained_user_message_tokens: self.compaction_retained_user_message_tokens,
|
||||
|
||||
@@ -372,44 +372,51 @@ impl App {
|
||||
.as_ref()
|
||||
.and_then(|identity| config.model_context_windows_for(identity).cloned());
|
||||
let configured_route_base_url = effective_auth_config.active_route_base_url();
|
||||
let (active_route_limits, active_route_base_url, active_context_window_source) =
|
||||
if auto_model {
|
||||
(
|
||||
active_context_window_override.map(|window| RouteLimits {
|
||||
context_tokens: Some(u64::from(window)),
|
||||
..RouteLimits::default()
|
||||
}),
|
||||
let (
|
||||
active_route_limits,
|
||||
active_route_base_url,
|
||||
active_context_window_source,
|
||||
active_context_window,
|
||||
) = if auto_model {
|
||||
(
|
||||
active_context_window_override.map(|window| RouteLimits {
|
||||
context_tokens: Some(u64::from(window)),
|
||||
..RouteLimits::default()
|
||||
}),
|
||||
configured_route_base_url,
|
||||
if active_context_window_override.is_some() {
|
||||
crate::route_runtime::ContextWindowSource::Configured
|
||||
} else {
|
||||
crate::route_runtime::ContextWindowSource::Fallback
|
||||
},
|
||||
None,
|
||||
)
|
||||
} else {
|
||||
provider_identity
|
||||
.as_ref()
|
||||
.ok_or_else(|| "provider identity unavailable".to_string())
|
||||
.and_then(|identity| {
|
||||
crate::route_runtime::resolve_runtime_route_for_identity(
|
||||
&effective_auth_config,
|
||||
identity,
|
||||
Some(&model),
|
||||
)
|
||||
})
|
||||
.map(|resolution| {
|
||||
(
|
||||
crate::route_budget::known_route_limits(resolution.candidate.limits()),
|
||||
resolution.candidate.endpoint().base_url.clone(),
|
||||
resolution.context_window.source,
|
||||
Some(resolution.context_window),
|
||||
)
|
||||
})
|
||||
.unwrap_or((
|
||||
None,
|
||||
configured_route_base_url,
|
||||
if active_context_window_override.is_some() {
|
||||
crate::route_runtime::ContextWindowSource::Configured
|
||||
} else {
|
||||
crate::route_runtime::ContextWindowSource::Fallback
|
||||
},
|
||||
)
|
||||
} else {
|
||||
provider_identity
|
||||
.as_ref()
|
||||
.ok_or_else(|| "provider identity unavailable".to_string())
|
||||
.and_then(|identity| {
|
||||
crate::route_runtime::resolve_runtime_route_for_identity(
|
||||
&effective_auth_config,
|
||||
identity,
|
||||
Some(&model),
|
||||
)
|
||||
})
|
||||
.map(|resolution| {
|
||||
(
|
||||
crate::route_budget::known_route_limits(resolution.candidate.limits()),
|
||||
resolution.candidate.endpoint().base_url.clone(),
|
||||
resolution.context_window.source,
|
||||
)
|
||||
})
|
||||
.unwrap_or((
|
||||
None,
|
||||
configured_route_base_url,
|
||||
crate::route_runtime::ContextWindowSource::Fallback,
|
||||
))
|
||||
};
|
||||
crate::route_runtime::ContextWindowSource::Fallback,
|
||||
None,
|
||||
))
|
||||
};
|
||||
let reasoning_effort_explicit = config.fleet_operator_reasoning_applied
|
||||
|| settings.reasoning_effort.is_some()
|
||||
|| config.reasoning_effort_is_explicit();
|
||||
@@ -429,12 +436,14 @@ impl App {
|
||||
} else {
|
||||
model.as_str()
|
||||
};
|
||||
let compact_threshold = crate::route_budget::compaction_threshold_for_route_at_percent(
|
||||
provider,
|
||||
threshold_model,
|
||||
active_route_limits,
|
||||
auto_compact_threshold_percent,
|
||||
);
|
||||
let compact_threshold =
|
||||
crate::route_budget::compaction_threshold_for_route_at_percent_with_window(
|
||||
provider,
|
||||
threshold_model,
|
||||
active_route_limits,
|
||||
active_context_window.map(|window| window.tokens),
|
||||
auto_compact_threshold_percent,
|
||||
);
|
||||
let auto_compact = if auto_compact_user_configured {
|
||||
settings_auto_compact
|
||||
} else {
|
||||
@@ -844,6 +853,7 @@ impl App {
|
||||
pinned_models: settings.pinned_models.clone(),
|
||||
auto_model,
|
||||
last_effective_model: None,
|
||||
last_effective_budget: None,
|
||||
last_effective_provider: None,
|
||||
last_effective_provider_identity: None,
|
||||
last_auto_route_receipt: None,
|
||||
@@ -860,6 +870,7 @@ impl App {
|
||||
active_route_limits,
|
||||
active_route_base_url,
|
||||
active_context_window_source,
|
||||
active_context_window,
|
||||
active_context_window_override,
|
||||
active_model_context_windows,
|
||||
pending_provider_switch: None,
|
||||
|
||||
@@ -267,11 +267,7 @@ pub fn build_context_inspector_text(app: &App, locale: Locale) -> String {
|
||||
}
|
||||
|
||||
fn context_usage(app: &App) -> (usize, u32, f64) {
|
||||
let max = crate::route_budget::route_context_window_tokens(
|
||||
app.api_provider,
|
||||
app.effective_model_for_budget(),
|
||||
app.active_route_limits,
|
||||
);
|
||||
let max = app.effective_context_window();
|
||||
// The meter must show the SAME pressure signal the auto-compaction trigger
|
||||
// decides on (compaction::estimate_input_tokens_for_pressure, the
|
||||
// non-inflated estimate with framing overhead). The old conservative
|
||||
|
||||
@@ -369,20 +369,26 @@ pub fn init_actor(handle: PersistActorHandle) {
|
||||
/// already shut down — the drop is logged instead of discarded silently, so
|
||||
/// lost session/work-graph state is diagnosable after the fact.
|
||||
pub fn persist(request: PersistRequest) {
|
||||
let _ = persist_checked(request);
|
||||
}
|
||||
|
||||
pub(crate) fn persist_checked(request: PersistRequest) -> Result<(), String> {
|
||||
let label = request_label(&request);
|
||||
if try_persist(request) {
|
||||
return;
|
||||
return Ok(());
|
||||
}
|
||||
if ACTOR_TX.get().is_some() {
|
||||
tracing::warn!(
|
||||
request = label,
|
||||
"persistence request dropped: actor channel is closed (shutdown already happened)"
|
||||
);
|
||||
Err("persistence actor is unavailable".to_string())
|
||||
} else {
|
||||
tracing::debug!(
|
||||
request = label,
|
||||
"persistence request dropped: actor not initialised yet"
|
||||
);
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -353,12 +353,8 @@ impl SetupRuntimeFacts {
|
||||
.map_or(identity.key.as_str(), |row| row.label)
|
||||
})
|
||||
.to_string();
|
||||
let context_window = crate::route_budget::route_context_window_tokens(
|
||||
app.api_provider,
|
||||
&app.model,
|
||||
app.active_route_limits,
|
||||
);
|
||||
let context_window_source = app.active_context_window_source.display_label();
|
||||
let context_window = app.effective_context_window();
|
||||
let context_window_source = app.effective_context_window_source().display_label();
|
||||
let provider =
|
||||
format!("{provider_name} · context {context_window} ({context_window_source})");
|
||||
let auth = readiness.label().into_owned();
|
||||
|
||||
@@ -76,6 +76,7 @@ pub(crate) fn test_app_with_options(options: crate::tui::app::TuiOptions) -> cra
|
||||
app.auto_model = false;
|
||||
app.last_effective_model = None;
|
||||
app.active_route_limits = None;
|
||||
app.active_context_window = None;
|
||||
app.active_context_window_override = None;
|
||||
// Fixtures replace `app.workspace` freely. Do not retain `App::new`'s real
|
||||
// process cwd as a second discovery root: parallel tests and a large
|
||||
|
||||
@@ -807,10 +807,12 @@ fn active_turn_has_running_tool(app: &App) -> bool {
|
||||
/// with a 240-character preview, and that copy is what `SyncSession` sends
|
||||
/// back to the engine.
|
||||
fn tool_result_content_for_api_message(app: &App, name: &str, output: &ToolResult) -> String {
|
||||
crate::core::engine::compact_tool_result_for_route(
|
||||
app.api_provider,
|
||||
&app.model,
|
||||
app.active_route_limits,
|
||||
let (provider, model, limits, window) = app.effective_budget_route();
|
||||
crate::core::engine::compact_tool_result_for_route_with_window(
|
||||
provider,
|
||||
model,
|
||||
limits,
|
||||
window.map(|window| window.tokens),
|
||||
name,
|
||||
output,
|
||||
)
|
||||
@@ -830,6 +832,7 @@ pub(crate) struct UserDispatchOutcome {
|
||||
effective_reasoning_effort: EffectiveReasoningEffort,
|
||||
auto_selection: Option<crate::model_routing::AutoRouteSelection>,
|
||||
route_limits: codewhale_config::route::RouteLimits,
|
||||
context_window: crate::route_runtime::ContextWindowResolution,
|
||||
}
|
||||
|
||||
/// Tell the operator that an explicit "make this my default" request did not
|
||||
|
||||
@@ -995,6 +995,7 @@ pub(crate) async fn apply_model_picker_choice(
|
||||
let model_changed = model != previous_model || app.auto_model != model_is_auto;
|
||||
let mut resolved_model = model.clone();
|
||||
let mut route_base_url = config.active_route_base_url();
|
||||
let mut resolved_budget = None;
|
||||
if !model_is_auto {
|
||||
match crate::route_runtime::resolve_runtime_route_for_identity(
|
||||
config,
|
||||
@@ -1005,12 +1006,8 @@ pub(crate) async fn apply_model_picker_choice(
|
||||
resolved_model = resolution.candidate.wire_model_id().as_str().to_string();
|
||||
route_base_url = resolution.candidate.endpoint().base_url.clone();
|
||||
if model_changed {
|
||||
app.set_active_context_window_override(config, ¤t_identity);
|
||||
app.set_active_route_resolution(
|
||||
route_base_url.clone(),
|
||||
resolution.candidate.limits(),
|
||||
resolution.context_window.source,
|
||||
);
|
||||
resolved_budget =
|
||||
Some((resolution.candidate.limits(), resolution.context_window));
|
||||
}
|
||||
}
|
||||
Err(reason) => {
|
||||
@@ -1019,14 +1016,6 @@ pub(crate) async fn apply_model_picker_choice(
|
||||
return;
|
||||
}
|
||||
}
|
||||
} else if model_changed {
|
||||
app.set_active_context_window_override(config, ¤t_identity);
|
||||
app.active_route_limits = app.context_window_override_limits();
|
||||
app.active_route_base_url = route_base_url.clone();
|
||||
app.active_context_window_source = app
|
||||
.configured_context_window_for(&app.model)
|
||||
.map(|resolution| resolution.source)
|
||||
.unwrap_or(crate::route_runtime::ContextWindowSource::Fallback);
|
||||
}
|
||||
|
||||
let effective_effort = if model_is_auto {
|
||||
@@ -1043,6 +1032,17 @@ pub(crate) async fn apply_model_picker_choice(
|
||||
|
||||
if model_changed {
|
||||
app.set_model_selection(resolved_model.clone());
|
||||
app.set_active_context_window_override(config, ¤t_identity);
|
||||
if let Some((limits, context_window)) = resolved_budget {
|
||||
app.set_active_route_resolution(route_base_url.clone(), limits, context_window);
|
||||
} else {
|
||||
app.active_route_limits = app.context_window_override_limits();
|
||||
app.active_route_base_url = route_base_url.clone();
|
||||
app.active_context_window_source = app.active_context_window.map_or(
|
||||
crate::route_runtime::ContextWindowSource::Fallback,
|
||||
|window| window.source,
|
||||
);
|
||||
}
|
||||
let provider_identity = app.provider_identity_for_persistence().to_string();
|
||||
app.provider_models
|
||||
.insert(provider_identity.clone(), resolved_model.clone());
|
||||
@@ -1242,7 +1242,7 @@ pub(crate) async fn apply_provider_fallback_switch(
|
||||
let resolved_endpoint = resolved_route.candidate.endpoint().base_url.clone();
|
||||
let next_config = resolved_route.config;
|
||||
let new_model = resolved_route.model;
|
||||
let context_window_source = resolved_route.context_window.source;
|
||||
let context_window = resolved_route.context_window;
|
||||
|
||||
if let Err(err) = CodewhaleClient::from_candidate(&next_config, &resolved_route.candidate) {
|
||||
app.set_provider_identity_record(previous_identity);
|
||||
@@ -1273,7 +1273,7 @@ pub(crate) async fn apply_provider_fallback_switch(
|
||||
app.set_active_route_resolution(
|
||||
new_base_url.clone(),
|
||||
resolved_route.candidate.limits(),
|
||||
context_window_source,
|
||||
context_window,
|
||||
);
|
||||
app.update_model_compaction_budget();
|
||||
if cache_scope_changed {
|
||||
@@ -3051,7 +3051,7 @@ fn apply_validated_profile_config(
|
||||
app.set_active_route_resolution(
|
||||
route.candidate.endpoint().base_url.clone(),
|
||||
route.candidate.limits(),
|
||||
route.context_window.source,
|
||||
route.context_window,
|
||||
);
|
||||
app.update_model_compaction_budget();
|
||||
app.session.last_prompt_tokens = None;
|
||||
|
||||
@@ -435,13 +435,9 @@ pub(crate) fn maybe_warn_context_pressure_for_config(
|
||||
app.context_pressure_warning_dismissed = None;
|
||||
return;
|
||||
}
|
||||
let max = config.effective_context_window.unwrap_or_else(|| {
|
||||
crate::route_budget::route_context_window_tokens(
|
||||
app.api_provider,
|
||||
app.effective_model_for_budget(),
|
||||
app.active_route_limits,
|
||||
)
|
||||
});
|
||||
let max = config
|
||||
.effective_context_window
|
||||
.unwrap_or_else(|| app.effective_context_window());
|
||||
let Some((used, max, percent)) = context_usage_snapshot_for_window(app, max) else {
|
||||
return;
|
||||
};
|
||||
@@ -477,7 +473,7 @@ pub(crate) fn maybe_warn_context_pressure_for_config(
|
||||
|
||||
// #5239: the meter drives real budgets off this window, so an unverified
|
||||
// one must say so next to the numbers that depend on it.
|
||||
let window_note = if app.active_context_window_source.is_verified() {
|
||||
let window_note = if app.effective_context_window_source().is_verified() {
|
||||
""
|
||||
} else {
|
||||
", unverified window"
|
||||
|
||||
@@ -997,6 +997,7 @@ pub(crate) async fn spawned_dispatch_inner(
|
||||
batch: Some(initial_routed_usage.clone()),
|
||||
};
|
||||
let route_limits = turn_route.candidate.limits();
|
||||
let context_window = turn_route.context_window;
|
||||
let op = Op::SendMessage(TurnSpec {
|
||||
profile_constitution: None,
|
||||
max_output_tokens: None,
|
||||
@@ -1042,6 +1043,7 @@ pub(crate) async fn spawned_dispatch_inner(
|
||||
effective_reasoning_effort: effective_reasoning_receipt,
|
||||
auto_selection,
|
||||
route_limits,
|
||||
context_window,
|
||||
};
|
||||
Box::new(move |app, current_engine, config| {
|
||||
// Admission stays serialized by this flag until its callback retires,
|
||||
@@ -1113,6 +1115,21 @@ pub(crate) fn build_dispatch_success_closure(
|
||||
app.clear_receipt();
|
||||
app.tool_evidence.clear();
|
||||
|
||||
if prepare.auto_model {
|
||||
app.last_effective_model = Some(outcome.effective_model.clone());
|
||||
app.last_effective_budget = Some((
|
||||
outcome.effective_provider,
|
||||
outcome.effective_model.clone(),
|
||||
crate::route_budget::known_route_limits(outcome.route_limits),
|
||||
outcome.context_window,
|
||||
));
|
||||
} else {
|
||||
app.set_active_route_limits(outcome.route_limits);
|
||||
app.active_context_window = Some(outcome.context_window);
|
||||
app.active_context_window_source = outcome.context_window.source;
|
||||
}
|
||||
app.update_model_compaction_budget();
|
||||
|
||||
app.system_prompt = Some(build_app_system_prompt_with_goal(
|
||||
app,
|
||||
config,
|
||||
@@ -1147,8 +1164,6 @@ pub(crate) fn build_dispatch_success_closure(
|
||||
app.last_effective_provider = None;
|
||||
app.last_effective_provider_identity = None;
|
||||
app.last_auto_route_receipt = None;
|
||||
app.set_active_route_limits(outcome.route_limits);
|
||||
app.update_model_compaction_budget();
|
||||
}
|
||||
app.pending_auto_route_receipt = outcome
|
||||
.auto_selection
|
||||
|
||||
@@ -164,7 +164,23 @@ pub(super) fn apply_engine_session_projection(
|
||||
if app.auto_model {
|
||||
app.last_effective_model = Some(model);
|
||||
} else {
|
||||
let accepted_route =
|
||||
app.pending_turn_route
|
||||
.as_ref()
|
||||
.is_some_and(|(provider, accepted_model, auto)| {
|
||||
!auto && *provider == app.api_provider && accepted_model == &model
|
||||
});
|
||||
let budget = accepted_route.then_some((
|
||||
app.active_route_limits,
|
||||
app.active_context_window,
|
||||
app.active_context_window_source,
|
||||
));
|
||||
app.set_model_selection(model);
|
||||
if let Some((limits, window, source)) = budget {
|
||||
app.active_route_limits = limits;
|
||||
app.active_context_window = window;
|
||||
app.active_context_window_source = source;
|
||||
}
|
||||
}
|
||||
app.update_model_compaction_budget();
|
||||
if app.workspace != workspace {
|
||||
|
||||
@@ -949,15 +949,18 @@ pub(crate) async fn build_preview_request_inputs(
|
||||
app.translation_enabled,
|
||||
app.verbosity.clone(),
|
||||
);
|
||||
prompt_context.context_window = Some(planned.route.context_window);
|
||||
let effort = planned.effective_reasoning_effort.as_deref();
|
||||
prompt_context.planned_output_tokens =
|
||||
Some(crate::route_budget::effective_max_output_tokens_for_turn(
|
||||
prompt_context.planned_output_tokens = Some(
|
||||
crate::route_budget::effective_max_output_tokens_for_turn_with_window(
|
||||
planned.route.identity.provider,
|
||||
&planned.route.model,
|
||||
crate::route_budget::known_route_limits(planned.route.candidate.limits()),
|
||||
Some(planned.route.context_window.tokens),
|
||||
None,
|
||||
effort,
|
||||
));
|
||||
),
|
||||
);
|
||||
posture(
|
||||
Some(Box::new(PreviewNextTurn {
|
||||
content,
|
||||
@@ -1173,21 +1176,21 @@ pub(crate) fn build_app_system_prompt_with_goal(
|
||||
let budgets = crate::project_context::InstructionBudgets::resolve_or_default(
|
||||
&settings.budget,
|
||||
crate::route_runtime::ContextWindowResolution {
|
||||
tokens: crate::route_budget::route_context_window_tokens(
|
||||
app.api_provider,
|
||||
&app.model,
|
||||
app.active_route_limits,
|
||||
),
|
||||
source: app.active_context_window_source,
|
||||
tokens: app.effective_context_window(),
|
||||
source: app.effective_context_window_source(),
|
||||
}
|
||||
.instruction_budget_tokens(),
|
||||
Some(crate::route_budget::effective_max_output_tokens_for_turn(
|
||||
app.api_provider,
|
||||
&app.model,
|
||||
app.active_route_limits,
|
||||
None,
|
||||
effort.as_deref(),
|
||||
) as usize),
|
||||
Some({
|
||||
let (provider, model, limits, window) = app.effective_budget_route();
|
||||
crate::route_budget::effective_max_output_tokens_for_turn_with_window(
|
||||
provider,
|
||||
model,
|
||||
limits,
|
||||
window.map(|window| window.tokens),
|
||||
None,
|
||||
effort.as_deref(),
|
||||
) as usize
|
||||
}),
|
||||
);
|
||||
prompts::system_prompt_with_instruction_assembly_for_host(
|
||||
&app.workspace,
|
||||
@@ -1201,11 +1204,7 @@ pub(crate) fn build_app_system_prompt_with_goal(
|
||||
locale_tag: app.ui_locale.tag(),
|
||||
translation_enabled: app.translation_enabled,
|
||||
model_id: &app.model,
|
||||
context_window_override: Some(crate::route_budget::route_context_window_tokens(
|
||||
app.api_provider,
|
||||
&app.model,
|
||||
app.active_route_limits,
|
||||
)),
|
||||
context_window_override: Some(app.effective_context_window()),
|
||||
verbosity: app.verbosity.as_deref(),
|
||||
recovery_hint: recovery_hint.as_deref(),
|
||||
skills_config: app.skills_config.clone(),
|
||||
@@ -2349,11 +2348,7 @@ pub(crate) fn estimated_context_tokens(app: &App) -> Option<i64> {
|
||||
}
|
||||
|
||||
pub(crate) fn context_usage_snapshot(app: &App) -> Option<(i64, u32, f64)> {
|
||||
let max = crate::route_budget::route_context_window_tokens(
|
||||
app.api_provider,
|
||||
app.effective_model_for_budget(),
|
||||
app.active_route_limits,
|
||||
);
|
||||
let max = app.effective_context_window();
|
||||
context_usage_snapshot_for_window(app, max)
|
||||
}
|
||||
|
||||
|
||||
@@ -508,6 +508,7 @@ pub(crate) fn rollback_provider_after_auth_failure(
|
||||
previous_route_limits,
|
||||
previous_route_base_url,
|
||||
previous_context_window_source,
|
||||
previous_context_window,
|
||||
previous_context_window_override,
|
||||
previous_config,
|
||||
previous_onboarding,
|
||||
@@ -541,6 +542,7 @@ pub(crate) fn rollback_provider_after_auth_failure(
|
||||
app.active_route_limits = previous_route_limits;
|
||||
app.active_route_base_url = previous_route_base_url;
|
||||
app.active_context_window_source = previous_context_window_source;
|
||||
app.active_context_window = previous_context_window;
|
||||
app.update_model_compaction_budget();
|
||||
app.clear_model_scoped_telemetry();
|
||||
app.offline_mode = false;
|
||||
@@ -594,6 +596,7 @@ pub(crate) fn compaction_for_validated_route(
|
||||
route.identity.provider,
|
||||
&route.model,
|
||||
crate::route_budget::known_route_limits(route.candidate.limits()),
|
||||
Some(route.context_window),
|
||||
);
|
||||
config.image_input = route.candidate.capabilities().image_input;
|
||||
config
|
||||
@@ -718,6 +721,7 @@ async fn switch_provider_inner(
|
||||
previous_route_limits: app.active_route_limits,
|
||||
previous_route_base_url: app.active_route_base_url.clone(),
|
||||
previous_context_window_source: app.active_context_window_source,
|
||||
previous_context_window: app.active_context_window,
|
||||
previous_context_window_override: app.active_context_window_override,
|
||||
previous_config: previous_config.clone(),
|
||||
previous_onboarding: app.onboarding,
|
||||
@@ -825,14 +829,14 @@ async fn switch_provider_inner(
|
||||
let target_identity = target_identity_record.key.to_string();
|
||||
let new_base_url = validated_route.candidate.endpoint().base_url.clone();
|
||||
let route_limits = validated_route.candidate.limits();
|
||||
let context_window_source = validated_route.context_window.source;
|
||||
let context_window = validated_route.context_window;
|
||||
let new_model = validated_route.model.clone();
|
||||
let apply_route = || {
|
||||
*config = *validated_route.config;
|
||||
app.set_provider_identity_record(target_identity_record.clone());
|
||||
app.set_model_selection(new_model.clone());
|
||||
app.set_active_context_window_override(config, &target_identity_record);
|
||||
app.set_active_route_resolution(new_base_url.clone(), route_limits, context_window_source);
|
||||
app.set_active_route_resolution(new_base_url.clone(), route_limits, context_window);
|
||||
};
|
||||
let applied = if let Some((endpoint, ticket)) = probe {
|
||||
crate::provider_catalog_live::with_current_route_ticket(
|
||||
|
||||
@@ -158,8 +158,7 @@ pub(crate) fn persist_with_pending_work_boundary(
|
||||
.as_ref()
|
||||
.is_some_and(|work| work.has_pending_publish());
|
||||
if !has_pending {
|
||||
persistence_actor::persist(request);
|
||||
return Ok(());
|
||||
return persistence_actor::persist_checked(request);
|
||||
}
|
||||
if !persistence_actor::try_persist(request) {
|
||||
return Err("persistence actor is unavailable".to_string());
|
||||
@@ -1345,6 +1344,7 @@ pub(crate) fn restore_loaded_session_provider(
|
||||
app.set_active_context_window_override(config, &identity);
|
||||
app.active_route_limits = app.context_window_override_limits();
|
||||
app.active_route_base_url = config.active_route_base_url();
|
||||
app.active_context_window = app.configured_context_window_for(&app.model);
|
||||
app.active_context_window_source = app
|
||||
.configured_context_window_for(&app.model)
|
||||
.map(|resolution| resolution.source)
|
||||
@@ -1370,6 +1370,7 @@ pub(crate) fn resolve_loaded_session_route(app: &mut App, config: &Config) {
|
||||
if app.auto_model {
|
||||
app.active_route_limits = app.context_window_override_limits();
|
||||
app.active_route_base_url = config.active_route_base_url();
|
||||
app.active_context_window = app.configured_context_window_for(&app.model);
|
||||
app.active_context_window_source = app
|
||||
.configured_context_window_for(&app.model)
|
||||
.map(|resolution| resolution.source)
|
||||
@@ -1386,12 +1387,13 @@ pub(crate) fn resolve_loaded_session_route(app: &mut App, config: &Config) {
|
||||
app.set_active_route_resolution(
|
||||
resolution.candidate.endpoint().base_url.clone(),
|
||||
resolution.candidate.limits(),
|
||||
resolution.context_window.source,
|
||||
resolution.context_window,
|
||||
);
|
||||
}
|
||||
Err(_) => {
|
||||
app.active_route_limits = app.context_window_override_limits();
|
||||
app.active_route_base_url = config.active_route_base_url();
|
||||
app.active_context_window = app.configured_context_window_for(&app.model);
|
||||
app.active_context_window_source = app
|
||||
.configured_context_window_for(&app.model)
|
||||
.map(|resolution| resolution.source)
|
||||
|
||||
@@ -12875,8 +12875,12 @@ async fn paused_dispatch_at_compaction_threshold_enqueues_one_atomic_send() {
|
||||
app.auto_compact_threshold_percent = 10.0;
|
||||
app.api_messages =
|
||||
std::sync::Arc::new(vec![text_message("assistant", &"context ".repeat(240_000))]);
|
||||
let planned_compaction =
|
||||
app.compaction_config_for_route(app.api_provider, &app.model, app.active_route_limits);
|
||||
let planned_compaction = app.compaction_config_for_route(
|
||||
app.api_provider,
|
||||
&app.model,
|
||||
app.active_route_limits,
|
||||
app.active_context_window,
|
||||
);
|
||||
let fixture_used =
|
||||
estimated_context_tokens(&app).expect("fixture context estimate should be available");
|
||||
assert!(
|
||||
@@ -13523,8 +13527,12 @@ fn auto_routed_turn_compaction_uses_selected_route_not_stale_app_route() {
|
||||
.expect("preflight auto-selected route");
|
||||
let route_limits = crate::route_budget::known_route_limits(route.candidate.limits());
|
||||
|
||||
let compaction =
|
||||
app.compaction_config_for_route(route.identity.provider, &route.model, route_limits);
|
||||
let compaction = app.compaction_config_for_route(
|
||||
route.identity.provider,
|
||||
&route.model,
|
||||
route_limits,
|
||||
Some(route.context_window),
|
||||
);
|
||||
|
||||
assert_eq!(compaction.model, "vendor/model-b");
|
||||
assert_eq!(compaction.effective_context_window, Some(196_000));
|
||||
@@ -33847,6 +33855,14 @@ fn stale_session_projection_cannot_rewind_a_host_launch() {
|
||||
}
|
||||
app.runtime_turn_id = None;
|
||||
app.is_loading = false;
|
||||
let accepted_window = crate::route_runtime::ContextWindowResolution {
|
||||
tokens: 128_000,
|
||||
source: crate::route_runtime::ContextWindowSource::Fallback,
|
||||
};
|
||||
app.active_route_limits = None;
|
||||
app.active_context_window = Some(accepted_window);
|
||||
app.active_context_window_source = accepted_window.source;
|
||||
app.pending_turn_route = Some((app.api_provider, "canonical-model".into(), false));
|
||||
assert!(super::event_loop::apply_engine_session_projection(
|
||||
&mut app,
|
||||
&Config::default(),
|
||||
@@ -33858,7 +33874,7 @@ fn stale_session_projection_cannot_rewind_a_host_launch() {
|
||||
]),
|
||||
system_prompt: None,
|
||||
instruction_assembly: None,
|
||||
model,
|
||||
model: "canonical-model".into(),
|
||||
workspace,
|
||||
}
|
||||
));
|
||||
@@ -33868,6 +33884,8 @@ fn stale_session_projection_cannot_rewind_a_host_launch() {
|
||||
vec![text_message("user", "expected engine transcript")]
|
||||
);
|
||||
assert_eq!(app.input, "typing remains responsive");
|
||||
assert_eq!(app.active_context_window, Some(accepted_window));
|
||||
assert_eq!(app.effective_context_window(), 128_000);
|
||||
}
|
||||
|
||||
#[test]
|
||||
|
||||
@@ -2386,12 +2386,8 @@ impl ConfigView {
|
||||
key: "effective_context_window".to_string(),
|
||||
value: format!(
|
||||
"{} tokens · {}",
|
||||
crate::route_budget::route_context_window_tokens(
|
||||
app.api_provider,
|
||||
app.effective_model_for_budget(),
|
||||
app.active_route_limits,
|
||||
),
|
||||
app.active_context_window_source.display_label()
|
||||
app.effective_context_window(),
|
||||
app.effective_context_window_source().display_label()
|
||||
),
|
||||
editable: false,
|
||||
scope: ConfigScope::Session,
|
||||
|
||||
@@ -249,19 +249,16 @@ pub(crate) async fn plan_turn_route(
|
||||
turn_route_limits,
|
||||
)
|
||||
},
|
||||
token_threshold: crate::route_budget::compaction_threshold_for_route_at_percent(
|
||||
token_threshold: crate::route_budget::compaction_threshold_for_route_at_percent_with_window(
|
||||
turn_route.identity.provider,
|
||||
&turn_route.model,
|
||||
turn_route_limits,
|
||||
Some(turn_route.context_window.tokens),
|
||||
request.auto_compact_threshold_percent,
|
||||
),
|
||||
model: turn_route.model.clone(),
|
||||
image_input: turn_route.candidate.capabilities().image_input,
|
||||
effective_context_window: Some(crate::route_budget::route_context_window_tokens(
|
||||
turn_route.identity.provider,
|
||||
&turn_route.model,
|
||||
turn_route_limits,
|
||||
)),
|
||||
effective_context_window: Some(turn_route.context_window.tokens),
|
||||
summary_instructions: request.route_config.compaction_summary_instructions(),
|
||||
retained_user_message_tokens: request
|
||||
.route_config
|
||||
|
||||
Reference in new issue
Block a user