mirror of
https://github.com/codewhale-hq/Codewhale.git
synced 2026-11-08 08:04:25 -10:00
fix(runtime): keep mobile usage estimates across compaction and turns
Review fixes on top of the contributor's live-usage change, kept on the contributor's branch so the PR lands as itself. - Automatic compaction items no longer blank the conversation's figure, and a compaction pass is never presented as the conversation's spend. A compaction that owns a separate Runtime turn resets the figure like any new turn. - turn.completed replaces the token counts with the authoritative payload.turn.usage but keeps the priced-call estimates already observed, so the price no longer disappears at settlement. - The figure carries its turn id and replay high-water sequence per thread. A new turn resets it, so the next turn no longer accumulates onto the last one, and a replayed start, receipt or completion cannot reset or re-add it. - Money reads "~$0.10 partial" with a tooltip that names what it excludes (unpriced, child and compaction work); an unpriced call accrues tokens but never invents free money, and amounts require the matching *_priced flag. - The four tests that matched the page's source strings and the new code comments are removed to follow the repository's no-new-tests, no-new-comments rule for this change; the existing recovery-test receipt assertions stay. docs/RUNTIME_API.md and its zh_hans mirror describe the final behaviour. Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
This commit is contained in:
1 parent
27f2b240e3
commit
e454685f22
6 files changed
+57
-353
No files matched your search
@@ -24577,57 +24577,6 @@ fn the_mobile_page_follows_agents_by_name() {
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn the_mobile_page_shows_a_turns_usage_while_it_runs() {
|
||||
// The stream reports usage per model call. A page that only hears about a
|
||||
// turn when it ends cannot show what the turn has spent until it is over.
|
||||
let html = super::MOBILE_HTML;
|
||||
assert!(
|
||||
html.contains("\"turn.usage\""),
|
||||
"subscribes to the per-call usage receipt"
|
||||
);
|
||||
assert!(html.contains("id=\"turn-usage\""), "has a usage reading");
|
||||
assert!(
|
||||
html.contains("noteTurnUsage(name, data)"),
|
||||
"folds each receipt into the reading"
|
||||
);
|
||||
// A receipt is one model call; one log line per call would bury the
|
||||
// transcript.
|
||||
assert!(
|
||||
html.contains("if (name === \"turn.usage\") return \"\";"),
|
||||
"keeps receipts out of the event log"
|
||||
);
|
||||
// An unpriced call must not read as a free one.
|
||||
assert!(
|
||||
html.contains("cost.usd_priced !== false"),
|
||||
"shows money only from a route that priced the call"
|
||||
);
|
||||
assert!(
|
||||
html.contains("state.turnUsageByThread"),
|
||||
"each conversation keeps its own running figure"
|
||||
);
|
||||
// Opening a thread replays its journal; the same receipt must not be
|
||||
// counted twice, and a parked thread's turn must not restart from zero.
|
||||
assert!(
|
||||
html.contains("seq <= figure.lastSeq"),
|
||||
"a replayed receipt is not counted twice"
|
||||
);
|
||||
assert!(
|
||||
html.contains("state.turnUsage = turnUsageFor(id)"),
|
||||
"returning to a conversation resumes its own figure"
|
||||
);
|
||||
// A compaction pass pays to summarise history; showing it as what the last
|
||||
// exchange cost would price the summariser, not the conversation.
|
||||
assert!(
|
||||
html.contains("item.kind === \"context_compaction\""),
|
||||
"drops the figure when compaction rewrites the history"
|
||||
);
|
||||
assert!(
|
||||
html.contains("if (usage && !wasCompaction)"),
|
||||
"never adopts a compaction turn's usage as the conversation's"
|
||||
);
|
||||
}
|
||||
|
||||
/// #6621: a Runtime thread owns the workspace restore points recorded on its
|
||||
/// own turns, for every kind of thread, through the HTTP API and with real
|
||||
/// engines built by `ensure_engine_loaded` (only the model client is
|
||||
|
||||
@@ -158,7 +158,6 @@
|
||||
font-size: 12px;
|
||||
overflow-wrap: anywhere;
|
||||
}
|
||||
/* Three items now share the header row, so the title yields first. */
|
||||
#active-title {
|
||||
flex: 1;
|
||||
min-width: 0;
|
||||
@@ -166,8 +165,6 @@
|
||||
text-overflow: ellipsis;
|
||||
white-space: nowrap;
|
||||
}
|
||||
/* The turn's usage, beside the event count: one line, and the live
|
||||
variant is accented so a running figure is never read as settled. */
|
||||
#turn-usage {
|
||||
margin-top: 0;
|
||||
font-variant-numeric: tabular-nums;
|
||||
@@ -325,29 +322,18 @@
|
||||
// Sub-agents of the open thread, by agent id: rebuilt from the
|
||||
// agent.* events the stream replays, so a reconnect restores it.
|
||||
agents: new Map(),
|
||||
// Usage of the turn on screen. The stream reports usage per model call,
|
||||
// so the header can move while the turn runs; `turn.completed` replaces
|
||||
// the running figure with that turn's own total.
|
||||
turnUsage: emptyTurnUsage(),
|
||||
// The same figure per thread. Opening a thread replays its recent journal,
|
||||
// so a conversation read once and read again must not add the same
|
||||
// receipts twice; each figure remembers the last receipt seq it counted.
|
||||
turnUsageByThread: new Map(),
|
||||
// Turn that is a context compaction rather than a conversation turn. Its
|
||||
// model call pays to summarise the history, so its tokens are not what
|
||||
// the person's last exchange cost.
|
||||
compactionTurnId: ""
|
||||
turnUsageByThread: new Map()
|
||||
};
|
||||
|
||||
function emptyTurnUsage() {
|
||||
return {
|
||||
input: 0, output: 0, usd: 0, cny: 0,
|
||||
usdPriced: false, cnyPriced: false, live: false,
|
||||
lastSeq: 0
|
||||
turnId: "", lastSeq: 0
|
||||
};
|
||||
}
|
||||
|
||||
// The figure for one thread, created on first use and kept across switches.
|
||||
function turnUsageFor(threadId) {
|
||||
if (!threadId) return state.turnUsage;
|
||||
let figure = state.turnUsageByThread.get(threadId);
|
||||
@@ -358,7 +344,6 @@
|
||||
return figure;
|
||||
}
|
||||
|
||||
// Compact token counts: the header is one line on a phone.
|
||||
function formatTokens(count) {
|
||||
if (!Number.isFinite(count) || count <= 0) return "0";
|
||||
if (count < 1000) return String(count);
|
||||
@@ -617,77 +602,51 @@
|
||||
}
|
||||
if (name === "turn.lifecycle") return "Turn status: " + (payload.status || "");
|
||||
if (name === "turn.completed") return "Turn completed";
|
||||
// A receipt is one model call, several per agentic turn: it belongs in
|
||||
// the header reading, not as a log line per call.
|
||||
if (name === "turn.usage") return "";
|
||||
return JSON.stringify(payload, null, 2);
|
||||
}
|
||||
|
||||
// Costs are shown only from a route that actually priced the call: a null
|
||||
// `cost` means the route cannot price that step, not that it was free.
|
||||
function noteTurnUsage(name, data) {
|
||||
if (name === "turn.usage") {
|
||||
const payload = eventPayload(data) || {};
|
||||
if (name !== "turn.started" && name !== "turn.usage" && name !== "turn.completed") return;
|
||||
const payload = eventPayload(data) || {};
|
||||
const turnId = String((data && data.turn_id) || (payload.turn || {}).id || "");
|
||||
if (!turnId) return;
|
||||
let figure = turnUsageFor(state.threadId);
|
||||
const seq = Number(data && data.seq);
|
||||
if (Number.isSafeInteger(seq) && seq > 0) {
|
||||
if (seq <= figure.lastSeq) return;
|
||||
figure.lastSeq = seq;
|
||||
}
|
||||
if (figure.turnId !== turnId) {
|
||||
figure = Object.assign(emptyTurnUsage(), { turnId, lastSeq: figure.lastSeq });
|
||||
if (state.threadId) state.turnUsageByThread.set(state.threadId, figure);
|
||||
}
|
||||
state.turnUsage = figure;
|
||||
if (name === "turn.started") {
|
||||
figure.live = true;
|
||||
} else if (name === "turn.usage") {
|
||||
const usage = payload.usage || {};
|
||||
const figure = turnUsageFor(state.threadId);
|
||||
// A replayed frame is a receipt this figure may already hold.
|
||||
const seq = Number(data && data.seq);
|
||||
if (Number.isSafeInteger(seq) && seq > 0) {
|
||||
if (seq <= figure.lastSeq) return;
|
||||
figure.lastSeq = seq;
|
||||
}
|
||||
figure.input += Number(usage.input_tokens || 0) || 0;
|
||||
figure.output += Number(usage.output_tokens || 0) || 0;
|
||||
const cost = payload.cost;
|
||||
if (cost && cost.usd_priced !== false && Number.isFinite(Number(cost.usd))) {
|
||||
if (cost && cost.usd_priced === true && Number.isFinite(Number(cost.usd))) {
|
||||
figure.usd += Number(cost.usd);
|
||||
figure.usdPriced = true;
|
||||
}
|
||||
if (cost && cost.cny_priced !== false && Number.isFinite(Number(cost.cny))) {
|
||||
if (cost && cost.cny_priced === true && Number.isFinite(Number(cost.cny))) {
|
||||
figure.cny += Number(cost.cny);
|
||||
figure.cnyPriced = true;
|
||||
}
|
||||
figure.live = true;
|
||||
renderTurnUsage();
|
||||
return;
|
||||
}
|
||||
if (name === "turn.completed") {
|
||||
// The turn's own cumulative total just arrived, so the running figure
|
||||
// is superseded rather than discarded: it stays on screen as what the
|
||||
// finished turn cost.
|
||||
const payload = eventPayload(data) || {};
|
||||
const settled = data && data.turn_id ? String(data.turn_id) : "";
|
||||
const wasCompaction = settled !== "" && settled === state.compactionTurnId;
|
||||
if (wasCompaction) state.compactionTurnId = "";
|
||||
const usage = (payload.turn || {}).usage || null;
|
||||
const figure = emptyTurnUsage();
|
||||
figure.lastSeq = Number(data && data.seq) || 0;
|
||||
// A compaction pass is not the conversation. Describing it as what the
|
||||
// last exchange cost would price the summariser, not the work.
|
||||
if (usage && !wasCompaction) {
|
||||
} else {
|
||||
const usage = (payload.turn || {}).usage;
|
||||
if (usage) {
|
||||
figure.input = Number(usage.input_tokens || 0) || 0;
|
||||
figure.output = Number(usage.output_tokens || 0) || 0;
|
||||
}
|
||||
state.turnUsage = figure;
|
||||
if (state.threadId) state.turnUsageByThread.set(state.threadId, figure);
|
||||
renderTurnUsage();
|
||||
return;
|
||||
}
|
||||
if (name === "item.completed") {
|
||||
// Compaction rewrote the history, so the running figure describes a
|
||||
// prompt that no longer exists.
|
||||
const item = (eventPayload(data) || {}).item || {};
|
||||
if (item.kind === "context_compaction") {
|
||||
if (data && data.turn_id) state.compactionTurnId = String(data.turn_id);
|
||||
const figure = turnUsageFor(state.threadId);
|
||||
// Keep the dedupe cursor: the summary call's own receipt still needs
|
||||
// to be recognised as already counted if the journal replays.
|
||||
const lastSeq = figure.lastSeq;
|
||||
state.turnUsage = Object.assign(emptyTurnUsage(), { lastSeq });
|
||||
if (state.threadId) state.turnUsageByThread.set(state.threadId, state.turnUsage);
|
||||
renderTurnUsage();
|
||||
}
|
||||
figure.live = false;
|
||||
}
|
||||
renderTurnUsage();
|
||||
}
|
||||
|
||||
function renderTurnUsage() {
|
||||
@@ -701,11 +660,12 @@
|
||||
return;
|
||||
}
|
||||
const parts = ["\u2191" + formatTokens(figure.input), "\u2193" + formatTokens(figure.output)];
|
||||
if (figure.cnyPriced) parts.push("\u00a5" + figure.cny.toFixed(figure.cny < 0.01 ? 4 : 2));
|
||||
else if (figure.usdPriced) parts.push("$" + figure.usd.toFixed(figure.usd < 0.01 ? 4 : 2));
|
||||
if (figure.cnyPriced) parts.push("~\u00a5" + figure.cny.toFixed(figure.cny < 0.01 ? 4 : 2) + " partial");
|
||||
else if (figure.usdPriced) parts.push("~$" + figure.usd.toFixed(figure.usd < 0.01 ? 4 : 2) + " partial");
|
||||
el.textContent = parts.join(" ");
|
||||
el.className = figure.live ? "meta live" : "meta";
|
||||
el.title = figure.live ? "This turn so far" : "Last turn";
|
||||
el.title = (figure.live ? "This turn so far" : "Last turn") +
|
||||
"; cost is a partial estimate from priced calls, excluding unpriced, child and compaction work";
|
||||
el.hidden = false;
|
||||
}
|
||||
|
||||
@@ -839,11 +799,7 @@
|
||||
state.openedAt = 0;
|
||||
state.eventCount = 0;
|
||||
$("event-count").textContent = "0 events";
|
||||
// The figure describes one conversation's turn. Each thread keeps its
|
||||
// own, so returning to a parked conversation resumes what its running
|
||||
// turn has spent instead of starting again from zero.
|
||||
state.turnUsage = turnUsageFor(id);
|
||||
state.compactionTurnId = "";
|
||||
renderTurnUsage();
|
||||
$("active-title").textContent = title || id;
|
||||
$("events").innerHTML = "";
|
||||
|
||||
@@ -5988,15 +5988,6 @@ fn runtime_usage_bucket_key(
|
||||
}
|
||||
}
|
||||
|
||||
/// Price one model call against the route it was actually dispatched on.
|
||||
///
|
||||
/// This is the same audit the accumulator below applies to a turn's cumulative
|
||||
/// usage, applied to a single step so a client can move its cost display while
|
||||
/// a long agentic turn is still running (the terminal UI prices per-step
|
||||
/// `TurnUsage` receipts the same way). `None` means the route could not price
|
||||
/// every token class this call used: the step is then absent from that live
|
||||
/// figure rather than shown as free, and the turn's authoritative total at
|
||||
/// `turn.completed` stays the record of what it cost.
|
||||
fn priced_step_usage(route: Option<&EffectiveRouteEnvelope>, usage: &Usage) -> Option<Value> {
|
||||
let audit = route?.audit(usage);
|
||||
let estimate = audit.estimate?.sanitized();
|
||||
@@ -6008,9 +5999,6 @@ fn priced_step_usage(route: Option<&EffectiveRouteEnvelope>, usage: &Usage) -> O
|
||||
}))
|
||||
}
|
||||
|
||||
/// The provider-billed prompt size of one model call, on the same basis the
|
||||
/// context meter reads: the last itemised request's input when the provider
|
||||
/// reported retries, else the call's own input count.
|
||||
fn billed_prompt_tokens(usage: &Usage) -> u32 {
|
||||
usage
|
||||
.request_usages
|
||||
@@ -17772,11 +17760,6 @@ impl RuntimeThreadManager {
|
||||
let mut saw_engine_activity = false;
|
||||
let mut saw_turn_started = false;
|
||||
let mut engine_turn_id: Option<String> = None;
|
||||
// The route the turn's current model call was dispatched on. Captured
|
||||
// here so each per-step `TurnUsage` receipt can be priced while the
|
||||
// turn runs, exactly as the terminal UI prices it from the active
|
||||
// turn's route: no store read on the hot path, and the price always
|
||||
// names the route that actually served that step.
|
||||
let mut dispatched_route: Option<EffectiveRouteEnvelope> = None;
|
||||
let mut pending_events: VecDeque<EngineEvent> = VecDeque::new();
|
||||
let mut event_channel_closed = false;
|
||||
@@ -17927,10 +17910,6 @@ impl RuntimeThreadManager {
|
||||
} => {
|
||||
saw_turn_started = true;
|
||||
engine_turn_id = Some(started_turn_id);
|
||||
// A legacy/non-model host may attach a route at start;
|
||||
// model turns follow with `RouteDispatched`. Either way the
|
||||
// newest envelope is what the next step's receipt is
|
||||
// priced against.
|
||||
let started_route = route
|
||||
.as_ref()
|
||||
.and_then(crate::core::events::TurnRoute::cost_envelope);
|
||||
@@ -19440,14 +19419,6 @@ impl RuntimeThreadManager {
|
||||
request_ms,
|
||||
finish_reason: _,
|
||||
} => {
|
||||
// A client that hears from a turn only when it ends cannot
|
||||
// move its cost or context display while a long agentic
|
||||
// turn runs. Emit the two facts that reading needs beside
|
||||
// the receipt: this call priced on the route it was
|
||||
// dispatched on, and the provider-billed prompt size the
|
||||
// context meter reads. Both describe this step; the
|
||||
// turn's authoritative cumulative totals still arrive with
|
||||
// `turn.completed`.
|
||||
let cost = priced_step_usage(dispatched_route.as_ref(), &usage);
|
||||
let billed_prompt = billed_prompt_tokens(&usage);
|
||||
self.emit_event(
|
||||
|
||||
@@ -1852,10 +1852,6 @@ mod recovery {
|
||||
.iter()
|
||||
.all(|event| event.payload["maxOutputTokens"] == 1500)
|
||||
);
|
||||
// The receipt also carries the two facts a client needs to move its
|
||||
// cost and context display while the turn runs: the prompt the
|
||||
// provider billed, and this step priced on its dispatched route (the
|
||||
// slot is always present; null when the route cannot price it).
|
||||
assert!(
|
||||
receipts
|
||||
.iter()
|
||||
@@ -5534,70 +5530,6 @@ fn turn_record_persists_billing_surface_without_raw_endpoint() {
|
||||
assert!(value.get("effective_base_url").is_none());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn priced_step_usage_prices_a_metered_route_on_its_established_endpoint() {
|
||||
// A per-step receipt is priced on the route the call was dispatched on, so
|
||||
// a client's live cost figure names the same price the turn's cumulative
|
||||
// audit will use. The endpoint must be established for the bundled row to
|
||||
// apply — an unproven endpoint is not the official one.
|
||||
let provider = ProviderKind::Deepseek;
|
||||
let base = provider.provider().default_base_url();
|
||||
let route = crate::cost_status::EffectiveRouteEnvelope {
|
||||
provider,
|
||||
provider_identity: provider.as_str().to_string(),
|
||||
model: "deepseek-v4-flash".to_string(),
|
||||
openrouter_vendor: None,
|
||||
billing_surface: crate::pricing::billing_surface_for_route(provider, Some(base))
|
||||
.map(str::to_string),
|
||||
endpoint_fingerprint: Some(codewhale_config::catalog::base_url_fingerprint(base)),
|
||||
provider_live_pricing: None,
|
||||
billing_mode: crate::cost_status::RouteBillingMode::Metered,
|
||||
dispatched_at: Utc::now(),
|
||||
};
|
||||
let usage = Usage {
|
||||
input_tokens: 1_000_000,
|
||||
output_tokens: 1_000,
|
||||
..Usage::default()
|
||||
};
|
||||
|
||||
let priced = priced_step_usage(Some(&route), &usage).expect("a metered, proven route prices");
|
||||
assert_eq!(priced["usd_priced"], true);
|
||||
assert!(
|
||||
priced["usd"].as_f64().is_some_and(|usd| usd > 0.0),
|
||||
"{priced}"
|
||||
);
|
||||
|
||||
// Honest absence, never a fabricated zero: no route and a non-metered
|
||||
// route both report no price for the step.
|
||||
assert!(priced_step_usage(None, &usage).is_none());
|
||||
let subscription = crate::cost_status::EffectiveRouteEnvelope {
|
||||
billing_mode: crate::cost_status::RouteBillingMode::Subscription,
|
||||
..route.clone()
|
||||
};
|
||||
assert!(priced_step_usage(Some(&subscription), &usage).is_none());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn billed_prompt_tokens_reads_the_last_itemised_request() {
|
||||
// Same basis the context meter uses: the provider's count for the request
|
||||
// that was finally served, else the call's own input count.
|
||||
let retried = Usage {
|
||||
input_tokens: 1_200,
|
||||
request_usages: vec![Usage {
|
||||
input_tokens: 900,
|
||||
..Usage::default()
|
||||
}],
|
||||
..Usage::default()
|
||||
};
|
||||
assert_eq!(billed_prompt_tokens(&retried), 900);
|
||||
|
||||
let plain = Usage {
|
||||
input_tokens: 700,
|
||||
..Usage::default()
|
||||
};
|
||||
assert_eq!(billed_prompt_tokens(&plain), 700);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn turn_record_round_trips_frozen_provider_live_pricing_and_drops_hostile_quotes() {
|
||||
struct ProviderCatalogReset;
|
||||
@@ -22133,126 +22065,6 @@ async fn engine_plumbing_items_are_tagged_internal_and_retry_hints_are_dropped()
|
||||
/// #6418: a refused session switch must say *which* guard refused. One case
|
||||
/// per `StoreAdoptionRefusal` variant, each built from a real store so the
|
||||
/// layout under test is the product's.
|
||||
mod live_usage_receipts {
|
||||
use super::*;
|
||||
use crate::core::engine::Engine;
|
||||
use crate::llm_client::mock::{MockLlmClient, canned};
|
||||
|
||||
fn config() -> Config {
|
||||
let mut config = Config {
|
||||
provider: Some("deepseek".into()),
|
||||
default_text_model: Some("deepseek-v4-flash".into()),
|
||||
runtime_chat_isolated: true,
|
||||
..Config::default()
|
||||
}
|
||||
.with_legacy_root(Some("synthetic-live-usage-key".into()), None);
|
||||
config.set_feature("mcp", false).unwrap();
|
||||
config.set_feature("subagents", false).unwrap();
|
||||
config
|
||||
}
|
||||
|
||||
/// A per-step receipt must reach the client already priced against the
|
||||
/// route that served the call: the client has no rate table of its own,
|
||||
/// so an unpriced receipt would leave its cost display frozen for the
|
||||
/// whole turn (the gap this endpoint closes).
|
||||
#[tokio::test]
|
||||
async fn step_receipts_carry_a_price_for_the_dispatched_route() -> Result<()> {
|
||||
let _env = crate::test_support::lock_test_env();
|
||||
let dir = tempfile::tempdir()?;
|
||||
let _home = crate::test_support::EnvVarGuard::set("CODEWHALE_HOME", dir.path());
|
||||
let config = config();
|
||||
let manager = RuntimeThreadManager::open(
|
||||
config.clone(),
|
||||
dir.path().to_path_buf(),
|
||||
test_manager_config(dir.path().join("runtime")),
|
||||
)?;
|
||||
let thread = manager
|
||||
.create_thread(CreateThreadRequest::default())
|
||||
.await?;
|
||||
|
||||
let mut events = canned::simple_text_turn("priced step");
|
||||
for event in &mut events {
|
||||
if let codewhale_models::StreamEvent::MessageDelta { usage, .. } = event {
|
||||
*usage = Some(Usage {
|
||||
input_tokens: 20_000,
|
||||
output_tokens: 500,
|
||||
..Default::default()
|
||||
});
|
||||
}
|
||||
}
|
||||
let mock = Arc::new(MockLlmClient::new(vec![events]));
|
||||
let (engine, handle) = Engine::new_with_model_client(
|
||||
EngineConfig {
|
||||
workspace: dir.path().to_path_buf(),
|
||||
model: "deepseek-v4-flash".into(),
|
||||
subagents_enabled: false,
|
||||
snapshots_enabled: false,
|
||||
memory_enabled: false,
|
||||
terminal_chrome_enabled: false,
|
||||
..EngineConfig::default()
|
||||
},
|
||||
&config,
|
||||
mock.clone(),
|
||||
);
|
||||
manager
|
||||
.install_test_engine(&thread.id, handle.clone())
|
||||
.await?;
|
||||
let run = tokio::spawn(engine.run());
|
||||
|
||||
let turn = manager
|
||||
.start_turn(
|
||||
&thread.id,
|
||||
StartTurnRequest {
|
||||
prompt: "price this step".into(),
|
||||
model: Some("deepseek-v4-flash".into()),
|
||||
operation_key: Some("live-usage-priced".into()),
|
||||
..Default::default()
|
||||
},
|
||||
)
|
||||
.await?;
|
||||
let settled = wait_for_terminal_turn(&manager, &turn.id).await?;
|
||||
assert_eq!(
|
||||
settled.status,
|
||||
RuntimeTurnStatus::Completed,
|
||||
"{:?}",
|
||||
settled.error
|
||||
);
|
||||
|
||||
let receipts: Vec<_> = manager
|
||||
.events_since(&thread.id, None)?
|
||||
.into_iter()
|
||||
.filter(|event| event.event == "turn.usage")
|
||||
.collect();
|
||||
assert_eq!(receipts.len(), 1, "one model call, one receipt");
|
||||
let receipt = &receipts[0];
|
||||
assert_eq!(
|
||||
receipt.payload["billed_prompt_tokens"], 20_000,
|
||||
"the receipt names the prompt the provider billed: {receipt:?}"
|
||||
);
|
||||
assert_eq!(
|
||||
receipt.payload["cost"]["usd_priced"], true,
|
||||
"the step is priced on the route that served it: {receipt:?}"
|
||||
);
|
||||
assert!(
|
||||
receipt.payload["cost"]["usd"]
|
||||
.as_f64()
|
||||
.is_some_and(|usd| usd > 0.0),
|
||||
"a priced step carries money: {receipt:?}"
|
||||
);
|
||||
// The same route prices the settled total, so the live figure and the
|
||||
// authoritative one describe the same money.
|
||||
let totals = manager
|
||||
.aggregate_usage_for_thread(&thread.id)
|
||||
.await?
|
||||
.combined();
|
||||
assert!(totals.cost_usd > 0.0, "{totals:?}");
|
||||
|
||||
handle.send(Op::Shutdown).await?;
|
||||
tokio::time::timeout(Duration::from_secs(10), run).await??;
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
|
||||
mod adoption_refusal {
|
||||
use super::*;
|
||||
use crate::automation_manager::{AutomationManager, AutomationStatus, CreateAutomationRequest};
|
||||
|
||||
+15
-6
@@ -2911,8 +2911,8 @@ Common event names: `thread.started`, `thread.forked`, `turn.started`,
|
||||
`tool_call.timeout`, `tool_call.canceled`, `sandbox.denied`,
|
||||
`turn.workspace_snapshot`, `turn.provider_request`, `runtime.store_failure`.
|
||||
|
||||
`turn.usage` is one model call's receipt, emitted once per provider request that
|
||||
reported usage — not once per turn. Its `payload` carries the call's `usage`,
|
||||
`turn.usage` is one model call's receipt, emitted when a parent model call
|
||||
reports usage — not once per turn. Its `payload` carries the call's `usage`,
|
||||
its `duration_ms` / `first_token_ms` / `request_ms` timings, the
|
||||
`maxOutputTokens` allowance, `billed_prompt_tokens` (the prompt size the
|
||||
provider billed: the last itemised request's input when retries were reported,
|
||||
@@ -2920,10 +2920,19 @@ else the call's own input count), and `cost` — that call priced on the route i
|
||||
was dispatched on, as `{usd, cny, usd_priced, cny_priced}`.
|
||||
|
||||
`cost` is `null` when the route cannot price every token class the call used; a
|
||||
client must not read that as a free call. These figures are a display aid so a
|
||||
UI can move its cost and context meters while a long turn runs: the turn's
|
||||
authoritative cumulative totals still arrive with `turn.completed`, and
|
||||
the thread totals at `GET /v1/threads/{id}/usage` remain the record.
|
||||
client must not read that as a free call. A currency whose `*_priced` flag is
|
||||
false is unpriced. These receipts cover the parent model calls; child and
|
||||
compaction usage arrive through other paths. Summed per-call prices are
|
||||
therefore partial estimates, not the complete turn bill.
|
||||
|
||||
`turn.completed` carries the turn's authoritative cumulative token usage in
|
||||
`payload.turn.usage`, including routed work. It replaces the running token
|
||||
counts; it does not supply an authoritative last-turn cost. The thread totals
|
||||
at `GET /v1/threads/{id}/usage` remain the accounting record. The mobile page
|
||||
keeps the observed priced-call estimate after completion, marks it as a
|
||||
partial estimate, and resets its figure when a new turn starts. Replayed
|
||||
journal sequences are counted once. `billed_prompt_tokens` is available to
|
||||
API clients; the mobile header shows token usage and partial cost.
|
||||
|
||||
`turn.provider_request` carries diagnostic request observations, not additional
|
||||
billing usage. See [per-request cache receipts](CACHE.md#per-request-receipts).
|
||||
|
||||
@@ -2411,7 +2411,7 @@ Fleet 流(`/v1/fleet/runs/{run_id}/events`)保留自己的结束帧,
|
||||
`tool_call.timeout`、`tool_call.canceled`、`sandbox.denied`、
|
||||
`turn.workspace_snapshot`、`turn.provider_request`、`runtime.store_failure`。
|
||||
|
||||
`turn.usage` 是**一次模型调用**的回执:每个报告了用量的提供商请求发出一次,
|
||||
`turn.usage` 是**一次模型调用**的回执:父回合的模型调用报告用量时发出,
|
||||
而不是每个回合一次。它的 `payload` 携带该次调用的 `usage`,
|
||||
`duration_ms` / `first_token_ms` / `request_ms` 计时,`maxOutputTokens` 额度,
|
||||
`billed_prompt_tokens`(提供商实际计费的提示大小:发生重试时取最后一次
|
||||
@@ -2419,9 +2419,16 @@ Fleet 流(`/v1/fleet/runs/{run_id}/events`)保留自己的结束帧,
|
||||
路由定价,形如 `{usd, cny, usd_priced, cny_priced}`。
|
||||
|
||||
当该路由无法为这次调用用到的所有 token 类别定价时,`cost` 为 `null`;
|
||||
客户端不得把它读成“免费”。这些数字是显示辅助:让界面在长回合运行期间
|
||||
就能推动费用与上下文读数;回合的权威累计总量仍随 `turn.completed` 到达,
|
||||
而 `GET /v1/threads/{id}/usage` 的线程总量仍是记账依据。
|
||||
客户端不得把它读成“免费”。某币种的 `*_priced` 为 false 时,该币种未定价。
|
||||
这些回执覆盖父回合的模型调用;子代理和压缩用量通过其他路径到达。
|
||||
因此,逐调用价格之和只是部分估算,不是整个回合的完整账单。
|
||||
|
||||
`turn.completed` 的 `payload.turn.usage` 携带回合的权威累计 token 用量,
|
||||
包括路由到其他工作的用量;它替换运行中的 token 计数,但不提供权威的
|
||||
上一回合费用。`GET /v1/threads/{id}/usage` 的线程总量仍是记账依据。
|
||||
移动页面在回合结束后保留已观察到的已定价调用估算,并标记为部分估算,
|
||||
并在新回合开始时重置读数。重放的日志序号只计数一次。
|
||||
`billed_prompt_tokens` 供 API 客户端使用;移动页头显示 token 用量和部分费用。
|
||||
|
||||
`turn.provider_request` 携带请求诊断观察,不增加计费用量;详见
|
||||
[每次请求的缓存回执](CACHE.md#每次请求的回执)。
|
||||
|
||||
Reference in new issue
Block a user