mirror of
https://github.com/codewhale-hq/Codewhale.git
synced 2026-11-07 17:09:13 -10:00
pricing: a models.dev row's context tier fails closed instead of base-pricing
A live models.dev row can publish context tiers (`tiers` or the legacy
`context_over_200k`). The catalog audit path priced every turn on such a
row at its base rate, so a turn above the first tier was undercharged
silently. `refused_catalog_pricing_audit` (was
`invalid_catalog_pricing_audit`) now also returns
`UnpricedReason::UnrepresentedTier` ("pricing tier unavailable") when the
turn's total prompt input (`Usage::input_tokens`, cache read and write
included) is at or above a context tier's size, and for any tier kind
other than "context". It runs at both catalog call sites: the
provider/catalog route and the OpenRouter bundled-only path.
This is the first consumer of `ModelsDevCost::context_tiers()`, which
already folds `tiers` and `context_over_200k` into one sorted list and had
no caller until now. No new reason, string or locale key: the existing
reason and its 15 translations are reused.
Frozen quotes are untouched because they cannot carry tiers: provider-live
costs are built from the four base fields (client.rs, two sites),
cloud-facts patches replace the whole cost block (catalog_patch.rs), and
custom_models accepts only the four keys (catalog/configured.rs). Tier
rates themselves are still not applied; the 0.10.2 Known limits line now
says so, and crates/tui/CHANGELOG.md is regenerated by
scripts/sync-changelog.sh.
Not model-visible: cost audits never reach a request, so no session event
and no KV-cache effect. Pure computation, so #6149 does not apply.
Evidence:
- cargo test -p codewhale-tui --lib --locked -- pricing:: cost_status::
test result: ok. 130 passed; 0 failed (86 pricing::, 44 cost_status::),
including malformed_catalog_row_has_an_explicit_runtime_reason and
models_dev_live_rates_price_the_published_estimate (live refresh path,
untiered row, still priced)
- cargo fmt --all -- --check: clean
- scripts/sync-changelog.sh --check: up to date
- the tier edge itself is not exercised by a test (no new tests without
founder request); the bundled catalog has no tiered rows
Refs #6396
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01BsXVvReYca2UxzimQtj7Mj
This commit is contained in:
3 files changed
+28
-16
No files matched your search
+5
-3
@@ -421,9 +421,11 @@ launcher guidance.
|
||||
([#6915](https://github.com/codewhale-hq/Codewhale/issues/6915)).
|
||||
- Idle CPU has not been measured on FreeBSD or Linux for this release
|
||||
([#6728](https://github.com/codewhale-hq/Codewhale/issues/6728)).
|
||||
- Models.dev prices with context tiers: a turn above a model's first tier can
|
||||
be priced at its base rate. The bundled catalog has no tiered rows, so this
|
||||
can only happen after a live models.dev refresh
|
||||
- Models.dev prices with context tiers are not applied. A turn at or above a
|
||||
model's first context tier reads as unpriced ("pricing tier unavailable")
|
||||
and is counted as missing from the cost total, instead of being priced at
|
||||
its base rate. The bundled catalog has no tiered rows, so this appears only
|
||||
after a live models.dev refresh
|
||||
([#6396](https://github.com/codewhale-hq/Codewhale/issues/6396)).
|
||||
- Shell commands run by delegated agents appear in `/jobs` and the work dock
|
||||
with their owner, not in the agent's own detail view; named terminal
|
||||
|
||||
@@ -421,9 +421,11 @@ launcher guidance.
|
||||
([#6915](https://github.com/codewhale-hq/Codewhale/issues/6915)).
|
||||
- Idle CPU has not been measured on FreeBSD or Linux for this release
|
||||
([#6728](https://github.com/codewhale-hq/Codewhale/issues/6728)).
|
||||
- Models.dev prices with context tiers: a turn above a model's first tier can
|
||||
be priced at its base rate. The bundled catalog has no tiered rows, so this
|
||||
can only happen after a live models.dev refresh
|
||||
- Models.dev prices with context tiers are not applied. A turn at or above a
|
||||
model's first context tier reads as unpriced ("pricing tier unavailable")
|
||||
and is counted as missing from the cost total, instead of being priced at
|
||||
its base rate. The bundled catalog has no tiered rows, so this appears only
|
||||
after a live models.dev refresh
|
||||
([#6396](https://github.com/codewhale-hq/Codewhale/issues/6396)).
|
||||
- Shell commands run by delegated agents appear in `/jobs` and the work dock
|
||||
with their owner, not in the agent's own detail view; named terminal
|
||||
|
||||
+18
-10
@@ -1634,7 +1634,10 @@ fn audit_turn_cost_for_provider_on_endpoint_for_identity_at(
|
||||
VerifiedOffering::Absent => None,
|
||||
};
|
||||
|
||||
if let Some(audit) = offering.as_ref().and_then(invalid_catalog_pricing_audit) {
|
||||
if let Some(audit) = offering
|
||||
.as_ref()
|
||||
.and_then(|offering| refused_catalog_pricing_audit(offering, usage))
|
||||
{
|
||||
return audit.with_live_defect(live_defect);
|
||||
}
|
||||
|
||||
@@ -1692,17 +1695,22 @@ fn audit_turn_cost_for_provider_on_endpoint_for_identity_at(
|
||||
}
|
||||
}
|
||||
|
||||
/// Convert malformed catalog numerics into an explicit runtime audit reason.
|
||||
/// Convert malformed catalog numerics, or a context tier this turn reached,
|
||||
/// into an explicit runtime audit reason.
|
||||
/// Keeping this distinct from the ordinary `None` projection prevents a bad
|
||||
/// published row from becoming indistinguishable from an absent price.
|
||||
fn invalid_catalog_pricing_audit(
|
||||
fn refused_catalog_pricing_audit(
|
||||
offering: &codewhale_config::catalog::CatalogOffering,
|
||||
usage: &Usage,
|
||||
) -> Option<TurnCostAudit> {
|
||||
offering
|
||||
.cost
|
||||
.as_ref()
|
||||
.is_some_and(|cost| !codewhale_config::pricing::catalog_cost_is_valid(cost))
|
||||
.then(|| TurnCostAudit::unpriced(UnpricedReason::InvalidPricingRow))
|
||||
let cost = offering.cost.as_ref()?;
|
||||
if !codewhale_config::pricing::catalog_cost_is_valid(cost) {
|
||||
return Some(TurnCostAudit::unpriced(UnpricedReason::InvalidPricingRow));
|
||||
}
|
||||
cost.context_tiers()
|
||||
.iter()
|
||||
.any(|tier| tier.tier.kind != "context" || u64::from(usage.input_tokens) >= tier.tier.size)
|
||||
.then(|| TurnCostAudit::unpriced(UnpricedReason::UnrepresentedTier))
|
||||
}
|
||||
|
||||
/// Outcome of checking a catalog row's pricing provenance against the route.
|
||||
@@ -2109,7 +2117,7 @@ fn audit_openrouter_immutable_pricing(
|
||||
ProviderKind::Openrouter,
|
||||
&canonical_model,
|
||||
) {
|
||||
if let Some(audit) = invalid_catalog_pricing_audit(&offering) {
|
||||
if let Some(audit) = refused_catalog_pricing_audit(&offering, usage) {
|
||||
return audit;
|
||||
}
|
||||
if let Some(pricing) = effective_offering_pricing(
|
||||
@@ -2665,7 +2673,7 @@ mod tests {
|
||||
..Default::default()
|
||||
};
|
||||
|
||||
let audit = invalid_catalog_pricing_audit(&offering)
|
||||
let audit = refused_catalog_pricing_audit(&offering, &Usage::default())
|
||||
.expect("malformed row must become an explicit failed-closed audit");
|
||||
assert!(!audit.is_priced());
|
||||
assert_eq!(
|
||||
|
||||
Reference in new issue
Block a user