mirror of
https://github.com/QwenLM/qwen-code.git
synced 2026-11-07 06:19:43 -10:00
feat(core): resolve model limits and modalities from a models.dev catalog (#11959)
* feat(core): resolve model limits and modalities from a models.dev catalog
Bundle a trimmed models.dev snapshot (12 first-party providers, 237
models) and refresh it in the background once a day into
~/.qwen/model-registry.json with ETag revalidation, so a model that
appears on models.dev gets its context window, output limit, and input
modalities without a CLI release. tokenLimit()/knownTokenLimit()/
hasExplicitOutputLimit() consult the catalog before the regex tables;
defaultModalities() merges both so the catalog can only add modalities.
QWEN_CODE_MODELS_DEV=off restores the regex-only behavior (default in the
unit test setups), QWEN_CODE_MODELS_DEV_REFRESH=off disables the network
refresh, QWEN_CODE_MODELS_DEV_URL points at a mirror. `npm run
generate:model-catalog` regenerates the committed snapshot.
Implements PR2 and PR3 of the design in #9851. Refs #8558.
* feat(core): add model.customCatalog for offline model metadata
Offline and proxy-restricted users cannot reach models.dev, so let them
point `model.customCatalog` at a URL or local file whose entries are
merged over the bundled catalog per model and field. Both the trimmed
`{ models: { id: entry } }` shape and models.dev-style payloads are
accepted. A file is re-read synchronously at every start so the first
model resolution already sees it; a URL is cached like models.dev and
is still fetched when QWEN_CODE_MODELS_DEV_REFRESH=off. The cache
records its source, so a changed or removed setting drops the overlay.
Also sort snapshot keys with a plain comparator instead of localeCompare
so regeneration is deterministic across machines.
* docs: add a verification brief for the models.dev catalog
CI covers the mechanism, but every unit test stubs fetch and the test
setups run with the catalog off, so neither the data values nor the real
runtime behavior are verified anywhere. This brief is the handoff for a
machine that can run the CLI against live endpoints: an on/off A/B with
measured reference numbers, the refresh and offline paths, and the two
values that may still be wrong (Anthropic 1M context, qwen3.8-max PDF).
* fix(core): only catalog models the endpoint question has one answer for
Two projection bugs, both found by reviewing the magnitude of the changes
the catalog was making rather than their direction.
The provider whitelist filtered by vendor but never by model type, so the
catalog also carried embedding, text-to-speech, image, video, OCR and
translation models whose limits mean something else entirely:
gemini-embedding-001 reported an output limit of 1 and veo-3.1-generate a
context of 480, and those numbers outranked the family fallbacks that keep
such ids harmless today. Require tool calling and text output, which drops
34 ids and no real chat model.
Conflict resolution picked the vendor's numbers when several providers
served one id, but a context window is a property of the endpoint, not the
weights. DashScope caps GLM-5 output at 16,384 where Z.ai allows 131,072;
both are correct for their own endpoint. Worse, kimi-k2.6 has no matching
output pattern today, so it is treated as unknown and the user's max_tokens
is respected, and the catalog would have clamped it to 262,144. Drop such
ids instead of guessing: 220 of 237 ids have no conflict, and the 11 that
remain keep exactly the answer they get today.
Snapshot goes from 237 models to 192, 29,824 to 24,685 bytes.
* fix(core): validate catalog defaults and reject alias conflicts
* fix(core): narrow catalog defaults to verified behavior
* fix(core): preserve curated output limits over catalog defaults
* fix(core): harden models.dev catalog refresh and keying
- Reject a 200 response that projects to zero models so a gateway error
body or an upstream field rename cannot shadow the bundled snapshot,
and treat an already-empty cache as absent so a 304 cannot re-stamp it.
- Omit catalog candidates whose key is not a normalize() fixed point:
deepseek-v3 was unreachable by its own name while its dated alias
inherited a 163,840 output limit. Snapshot drops that entry (194).
- Exclude QWEN_CODE_MODELS_DEV_URL from project .env files: the refresh
writes the global cache every project reads.
- Reject false-valued modalities when parsing catalog caches.
- Document the fixed-point keying rule and the accurate REFRESH=off
semantics in the settings table.
* fix(core): preserve explicit budgets for catalog-only models
* fix(core): harden catalog cache reads and correct qwen3-coder-plus window
- loadModelCatalog drops cache keys no normalized spelling can reach and
treats a cache that is empty after validation as absent, so a stale or
poisoned cache can no longer displace the bundled snapshot.
- Correct qwen3-coder-plus to its vendor-declared 1,000,000-token window
(models.dev's alibaba buckets approximate it as 1,048,576), applied in
the loader so a refreshed cache cannot reintroduce the approximation.
- Pin the cross-origin refresh guard with a non-empty cache fixture, the
normalized-id catalog lookup for modalities, the env-budget clamp
precondition for glm-4.7, and control keys plus reload/snapshot arms
for the project-scoped catalog URL exclusion.
- Align the QWEN_CODE_MODELS_DEV_REFRESH docs and JSDoc with the
newer-than-bundled selection rule.
* fix(core): project catalog modalities across providers and test against the real catalog
Review round on PR #11959:
- trimModelsDevCatalog: token limits stay gated on the first-party
allowlist with disagreement-drop; input modalities describe the
weights, so they now union across every provider that serves the
model. First-party vendors outside the allowlist (issue #8558's
thinkingmachines/inkling) project image/pdf support again, while
their endpoint limits stay untrusted.
- modalityDefaults.test.ts: drop the file-scoped catalog mock that let
three assertions pin the opposite of shipped behavior; run against
the real bundled snapshot in an isolated QWEN_HOME and correct the
qwen3.8-max / glm-5v-turbo / glm-5.3-flash expectations. The
mock-based merge tests move to modalityDefaults.catalog.test.ts.
- vscode-ide-companion: add the missing QWEN_CODE_MODELS_DEV=off test
default via test-setup.ts so a host's ~/.qwen/model-registry.json
cache cannot flip its contextLimit assertions.
* revert(vscode-ide-companion): drop out-of-footprint catalog test wiring
Deterministic verification rejected the setupFiles addition to
packages/vscode-ide-companion/vitest.config.ts: per-package vitest
configs are test-config verification machinery, and this PR's
footprint does not include that package. Revert the wiring and remove
the now-orphaned test-setup.ts.
The underlying hazard (R4-2: acpModelInfo contextLimit assertions
resolve against the host's ~/.qwen/model-registry.json cache) is real
and reproduced on this head — a poisoned cache flips exactly the two
witness tests red — but every fix lives outside this PR's footprint,
so it is deferred to the follow-up queue with an open question for
maintainers.
Co-authored-by: Qwen-Coder <[email protected]>
* fix: close model catalog environment gaps
* fix(core): validate refreshed model metadata
* fix(core): reject incomplete model catalogs
* fix(core): keep catalog defaults agent-safe
* fix(core): preserve DeepSeek vision boundary
* fix(core): harden the catalog refresh fetch and pin the catalog fixture clock
Six review findings on the refresh/generator path:
- Bound the download: the refresh now reads the 200 body with a
200KB ceiling (the generator's budget) instead of buffering an
unbounded payload, rejects a present-but-non-JSON Content-Type one
step earlier than JSON.parse, and cancels the body on the HTTP-error
path so the socket is released.
- Clock skew: a cache stamped by a faster clock (negative or unreadable
age) now refetches instead of serving indefinitely; the re-stamp uses
the local clock so the skew self-heals.
- The refresh failure log goes through the canonical getErrorMessage,
so error.cause survives into the debug channel.
- The 'serves it immediately' test pins its clock ahead of the bundled
snapshot's committed fetchedAt, decoupling it from the runner's wall
clock (the snapshot is regenerated by whichever host runs the
generator).
- The generator refuses a non-OK response and an empty projection
instead of overwriting the committed snapshot with either.
Each new test is red under its corresponding revert (byte cap, content
-type check, skew guard); the clock-pin witness mutates the snapshot's
fetchedAt to 9999 and stays green.
Co-authored-by: Qwen-Coder <[email protected]>
* fix(core): drop mirror providers from catalog limits and keep parse fidelity
Seven more review findings on the catalog path:
- MODELS_DEV_PROVIDERS no longer includes modelscope. Measured against
the live payload: its only contributions were four dated -2507 qwen3
snapshots recorded under the bare ids plus vetoing z.ai's first-party
glm-4.6 numbers into a conflict drop. volcengine stays — it is the
first-party endpoint for the doubao/seed family (dropping it costs 16
keys). Snapshot regenerated: 203 entries, glm-4.6 restored to the
first-party figures.
- parseModelCatalog no longer lets one bad modalities member take a
whole entry down: unknown or mistyped modality keys are filtered and
the entry keeps its limits; context/output stay strict. The parser
also skips an own-property __proto__ key instead of invoking the
prototype setter (the refresh path can legitimately write one).
- A declared text-only modalities list now survives projection as {}
on entries that ship limits, instead of being indistinguishable from
"upstream said nothing" — inert under today's union, recorded so a
future narrowing can tell the two apart. A bare {} never creates an
entry on its own, so conflict-dropped and limit-less models still
drop.
- The committed snapshot now has a parse-and-budget gate: every shipped
entry must survive parseModelCatalog and stay within the 200KB budget
(corrupting one entry turns the new test red).
- The gpt-4o resolver expectation's comment now says it pins the
catalog-off regex answer; the catalog-on production value already has
its own twin test.
- The verification README's evidence pointers no longer reference a
gitignored .qwen path; they name the committed suites that pin the
same numbers.
Modality list is now single-sourced as CATALOG_MODALITIES exported from
model-catalog.ts. Each behavior change is red under its revert:
restoring modelscope re-drops glm-4.6, restoring whole-entry modality
rejection or the __proto__ assignment reds the parse test, restoring
the empty-modalities drop reds the projection expectations.
Co-authored-by: Qwen-Coder <[email protected]>
* fix(core): narrow catalog modality keys with a type predicate
Object.entries widens the key to string, and includes() on the widened
readonly string[] does not narrow it back, so the indexed write into
InputModalities was an implicit any under noImplicitAny. That failed the
packages/core build with TS7053 and cascaded into every required check.
Extract the membership test into an isCatalogModality predicate so the
write is type-safe. Type-level only, no behavior change: the existing
"drops malformed entries while parsing" coverage still pins that an
unrecognised modality key degrades that one field instead of discarding
the entry.
Co-authored-by: Qwen-Coder <[email protected]>
Patrol-Run: qwen-pr-closeout/jmunuawz4h7
* fix(core): keep catalog refresh usable and process-consistent
* fix(core): invalidate incompatible model catalog projections
* fix(core): stamp canonical models.dev URL as bundled catalog provenance
The generator passed its input locator straight through as the provenance
stamp, so generating from a local payload copy recorded a machine-specific
path (e.g. /tmp/11959-models-dev.json) in the committed snapshot. Stamp
MODELS_DEV_URL for local-path inputs (the payload is models.dev data
regardless of where the machine stored it), keep URL inputs stamping
themselves, regenerate the snapshot, and pin the artifact's source field
with a test.
Co-authored-by: Qwen-Coder <[email protected]>
Patrol-Run: qwen-pr-closeout/jmuofqisz0a
* fix(core): share the provider-coverage guard and key DeepSeek narrowing on the family root
Round-9 review findings on 4516bd759d:
- R9-1: the snapshot generator only checked "at least one entry
projected", while the runtime refresh treats a provider-incomplete
payload as fatal. A truncated download or a mirror missing one vendor
would silently replace the committed 203-entry snapshot. Extract the
coverage check into assertCatalogPayloadComplete() and call it from
both refreshRemote and scripts/generate-model-catalog.ts.
- R9-2: the #10270 DeepSeek text-only narrowing keyed on the hyphenated
prefix `deepseek-`, but normalize() folds bare-model records onto the
family root (`deepseek-v3` -> `deepseek`), which passes
isModelCatalogKey — so a bare record kept an overstated image claim.
Key the guard on the family: /^deepseek(?:-|$)/.
Both pins are mutation-verified: reverting either change turns its new
assertion red.
Co-authored-by: Qwen-Coder <[email protected]>
Patrol-Run: qwen-issue-patrol/jmuolga6k0g
* fix(core): keep four more catalog windows at their declared value
models.dev buckets the vendor-declared window of kimi-k3 and
qwen3-coder-plus up to 1,048,576, and MiniMax-M2.5 / GLM-4.7 up to
204,800, while the curated table in tokenLimits.ts and the provider
presets both declare the real value. The catalog wins for input limits,
so for kimi-k3 the compaction thresholds computed from the inflated
window fired above the window the vendor accepts
(computeThresholds(1048576).hard = 1,025,576 > 1,000,000).
The correction list already had the shape for qwen3-coder-plus; extend
it to the complete set of ids where the catalog contradicts both a
curated row and a preset, and drive the application from one table.
Co-authored-by: Qwen-Coder <[email protected]>
Patrol-Run: qwen-pr-closeout/jmupe138l16
* fix(cli): freeze all three model-catalog keys against project config
A project `.env` or `settings.json` `env` block could set
QWEN_CODE_MODELS_DEV and QWEN_CODE_MODELS_DEV_REFRESH even though
QWEN_CODE_MODELS_DEV_URL was already excluded. On the `qwen serve` fast
path both keys are applied before runQwenServeImpl snapshots and freezes
process.env into daemonRuntimeBaseEnv, which every workspace session
child inherits and which setRuntimeEnvIfUnset cannot repair per
workspace — so one repository's config decided catalog behaviour for
every other workspace the daemon hosts, and the rejection was silent
because the keys are not loader keys. On the reload path a repository's
`on` force-wrote over an operator's exported `off`, re-enabling the
outbound refresh in an air-gapped environment.
Add both keys beside the URL key in PROJECT_ENV_HARDCODED_EXCLUSIONS
and document the project-file restriction on their settings.md rows.
Co-authored-by: Qwen-Coder <[email protected]>
Patrol-Run: qwen-pr-closeout/jmupe138l16
---------
Co-authored-by: qwen-code-ci-bot <[email protected]>
Co-authored-by: Qwen Autofix Bot <[email protected]>
Co-authored-by: Qwen-Coder <[email protected]>
Co-authored-by: qwen-code-dev-bot <[email protected]>
Co-authored-by: yiliang114 <[email protected]>
This commit is contained in:
30 files changed
+4032
-50
No files matched your search
@@ -0,0 +1,49 @@
|
|||||||
|
---
|
||||||
|
title: 'Data-Driven Model Metadata Registry (models.dev)'
|
||||||
|
date: '2026-08-23'
|
||||||
|
status: 'implemented in PR #11959'
|
||||||
|
---
|
||||||
|
|
||||||
|
# Data-Driven Model Metadata Registry
|
||||||
|
|
||||||
|
[English](2026-08-23-models-dev-registry.md) | [简体中文](2026-08-23-models-dev-registry.zh-CN.md)
|
||||||
|
|
||||||
|
## Problem and scope
|
||||||
|
|
||||||
|
Model context windows, output limits, and input modalities currently require a CLI release to update hard-coded tables. PR #11959 adds a bundled models.dev snapshot and a background refresh. This document incorporates the useful constraints from the earlier design-only PR #9851; the implementation and its verification now belong to one PR.
|
||||||
|
|
||||||
|
Reasoning-effort tiers, reasoning wire fields, pricing, provider detection, and the OAuth model list remain outside this change. The earlier proposal to migrate effort metadata is deferred; its provider-specific claims need fresh verification before implementation.
|
||||||
|
|
||||||
|
## Resolution and precedence
|
||||||
|
|
||||||
|
Explicit model configuration remains above inferred catalog defaults. For inferred context windows, client-owned corrections take precedence over the selected catalog; existing regex tables and generic defaults supply missing fields. For output limits, existing regex matches take precedence over the catalog, preserving endpoint-specific request ceilings. The catalog supplies default output limits only where the table has no match; those catalog-only values do not cap explicit request or environment budgets. Existing curated output caps and the context-window clamp still apply. Trusted catalog image, audio, and video capabilities extend the regex defaults. PDF remains explicit because it selects an endpoint- and protocol-dependent file path. Explicit configured modalities remain authoritative.
|
||||||
|
|
||||||
|
The runtime cache replaces the bundled snapshot only when its ISO `fetchedAt` timestamp is newer. Incomplete responses are rejected before they can replace a complete snapshot or cache. Filesystem modification times do not determine freshness. The selected catalog stays fixed for the lifetime of a process, so context and output defaults use the same metadata. A background refresh writes the cache for the next process start.
|
||||||
|
|
||||||
|
Sonnet 4.5 is corrected to 200,000 context tokens even when cached data advertises its retired 1M beta. Sonnet 4.6 and Sonnet 5 retain their catalog limits. See [Anthropic's context-window reference](https://platform.claude.com/docs/en/build-with-claude/context-windows).
|
||||||
|
|
||||||
|
Preset or explicitly configured context windows remain authoritative for that configuration. Automatic detection uses the catalog and can differ from a preset for the same model id. We retain this precedence to deliver verified metadata updates; endpoint-specific overrides use existing model settings. Likewise, trusted modality additions are retained because they enable verified image support on DashScope. Provider-aware reconciliation remains a follow-up, rather than treating every empty family fallback as a capability veto.
|
||||||
|
|
||||||
|
## Projection and endpoint ambiguity
|
||||||
|
|
||||||
|
Only the supported first-party provider allowlist feeds the default catalog's token limits and modalities. The `thinkingmachines` first-party bucket is an explicit modality-only exception for issue #8558's Inkling image support; reseller and router claims do not widen another provider's capabilities. Entries must support tool calls and text output. Only positive safe-integer token limits and boolean input modalities are accepted.
|
||||||
|
|
||||||
|
Model identifiers use the existing normalization rules. A projected key must be a fixed point of the normalizer and must not be a numeric tag, an `@` alias, a generic router label, or an otherwise ambiguous short key. The same rule filters caches written by older versions. All token limits that normalize to one key must agree, including dated and provider-qualified variants. If any disagree, omit the limits and preserve the existing regex/default behavior. A bare identifier must not erase conflicting endpoint evidence. This is deliberately conservative: it does not certify every possible private gateway or correct endpoint-specific limitations already present in the regex tables.
|
||||||
|
|
||||||
|
Provider-aware lookup would require changing the model-resolution contract and its callers. It is deferred rather than approximated by first-provider precedence. Regeneration and runtime refresh use the same projection.
|
||||||
|
|
||||||
|
The draft-only `model.customCatalog` option was removed: it loaded too late for the first session and introduced process-global state that leaked across Config instances. Private/offline model overrides use the existing `modelProviders` generation settings. No new custom-cache file is read.
|
||||||
|
|
||||||
|
## Storage and refresh
|
||||||
|
|
||||||
|
A trimmed JSON snapshot ships with the CLI and has a 200 KiB generation budget. The full upstream download has a separate 16 MiB limit. Refresh starts in the background after proxy initialization, uses a ten-second timeout and a 24-hour cache interval, revalidates with ETag, and atomically writes the cache under `Storage.getGlobalQwenDir()`. Cache reads and ETag reuse require the current projection version and a reachable model key. Older or unstamped caches require a full download and re-projection; bump `MODEL_CATALOG_PROJECTION_VERSION` when projection rules change. Concurrent refreshes share an in-flight request. Failure leaves the previous data usable and is logged only at debug level.
|
||||||
|
|
||||||
|
`QWEN_CODE_MODELS_DEV=off` restores regex-only behavior. `QWEN_CODE_MODELS_DEV_REFRESH=off` disables upstream refresh. `QWEN_CODE_MODELS_DEV_URL` selects a mirror with the same provider filtering. No request-time network lookup, cross-process lock, scheduled regeneration service, or new dependency is introduced.
|
||||||
|
|
||||||
|
## Validation and acceptance
|
||||||
|
|
||||||
|
Focused tests cover cache selection, malformed entries, conflict rejection, alias normalization, per-field fallback, corrections, refresh throttling and failure. Real-source smoke checks must additionally cover the bundled on/off behavior and a live models.dev refresh; a mocked fetch cannot validate upstream facts.
|
||||||
|
|
||||||
|
[DashScope's PDF reference](https://www.alibabacloud.com/help/en/model-studio/pdf-understanding) documents qwen3.8-max PDF support through Chat Completions in Beijing and Singapore using `file_data` plus `filename`. It explicitly excludes Responses API PDF delivery. The catalog therefore does not automatically enable PDF for any model; users can explicitly configure `generationConfig.modalities.pdf` for a verified endpoint. This does not remove catalog image, audio, or video capabilities. Enabling automatic PDF inference is deferred until lookup can account for the endpoint and protocol, with a live recognition test.
|
||||||
|
|
||||||
|
The [verification record](../verification/models-dev-catalog/README.md) owns commands, results, and remaining delivery gates. Required checks must pass on the final head, and explicit PDF opt-in must remain distinguishable from catalog defaults in the PR description.
|
||||||
@@ -0,0 +1,49 @@
|
|||||||
|
---
|
||||||
|
title: '数据驱动的模型元数据目录（models.dev）'
|
||||||
|
date: '2026-08-23'
|
||||||
|
status: '已在 PR #11959 实现'
|
||||||
|
---
|
||||||
|
|
||||||
|
# 数据驱动的模型元数据目录
|
||||||
|
|
||||||
|
[English](2026-08-23-models-dev-registry.md) | [简体中文](2026-08-23-models-dev-registry.zh-CN.md)
|
||||||
|
|
||||||
|
## 问题与范围
|
||||||
|
|
||||||
|
目前更新模型上下文窗口、输出上限和输入模态需要修改硬编码表并发布 CLI。PR #11959 加入内置 models.dev 快照和后台刷新。本文吸收之前纯设计 PR #9851 中有用的约束，将实现及其验证统一到一个 PR。
|
||||||
|
|
||||||
|
本次不涉及 reasoning-effort 档位、推理协议字段、价格、provider 检测和 OAuth 模型列表。原方案中的 effort 元数据迁移留待后续；实现前需要重新验证其中与 provider 相关的事实。
|
||||||
|
|
||||||
|
## 解析与优先级
|
||||||
|
|
||||||
|
显式模型配置仍优先于目录推断值。对于推断的上下文窗口，客户端维护的纠正优先于选中的目录；缺失字段回退到现有正则表和通用默认值。输出上限优先使用现有正则表，保留按 endpoint 维护的请求上限；目录只为表中没有匹配项的模型提供默认输出上限，不用这些目录值截断显式请求或环境变量预算。原有人工维护的输出上限和上下文窗口裁剪仍然生效。可信目录中的图片、音频和视频能力可以扩展正则默认值。PDF 会选择另一条依赖 endpoint 和协议的文件读取路径，因此仍需显式配置。显式配置的模态仍具有最高优先级。
|
||||||
|
|
||||||
|
仅当运行时缓存的 ISO `fetchedAt` 时间戳比内置快照新时，才以缓存替代内置快照。不完整响应会在替换完整快照或缓存之前被拒绝。新旧判断不使用文件系统修改时间。进程一旦选定目录便保持该版本，确保上下文窗口与输出默认值来自同一份数据。后台刷新把缓存写入磁盘，供下次启动的新进程使用。
|
||||||
|
|
||||||
|
即使缓存仍记录已退役的 1M beta 上限，Sonnet 4.5 的上下文也纠正为 200,000 tokens。Sonnet 4.6 与 Sonnet 5 保留目录中的上限。依据见 [Anthropic 上下文窗口文档](https://platform.claude.com/docs/en/build-with-claude/context-windows)。
|
||||||
|
|
||||||
|
preset 或显式配置的上下文窗口仍以该配置为准。自动检测使用目录，因此同一模型 id 可能与 preset 不同。保留这一优先级以提供已验证的元数据更新；endpoint 特有的覆盖使用现有模型设置。可信的模态补充也继续保留，因为它启用了 DashScope 上已验证的图片支持。按 provider 对齐留待后续，不把所有空的家族回退都解释为能力否决。
|
||||||
|
|
||||||
|
## 数据投影与 endpoint 歧义
|
||||||
|
|
||||||
|
默认目录的 token 上限和模态只来自受支持的一线 provider 白名单。`thinkingmachines` 是 issue #8558 中 Inkling 图片能力所需的显式模态例外，不采信其 endpoint 上限；转售商和路由商不能扩大其他厂商模型的能力。模型必须支持工具调用并输出文本。token 上限只接受正安全整数，输入模态只接受布尔值。
|
||||||
|
|
||||||
|
模型标识使用现有归一化规则。投影键必须是归一化不动点，且不能是纯数字标签、带 `@` 的别名、通用路由标签或其他有歧义的短键；读取旧版本缓存时也应用同一规则。归一化到同一个 key 的所有 token 上限必须一致，包括带日期和 provider 前缀的变体。只要存在分歧，就丢弃这些上限，保持现有正则或默认行为。裸模型名不能抹掉不同 endpoint 的冲突证据。这是保守策略：它不保证任意私有网关都适用，也不修正正则表已有的 endpoint 限制问题。
|
||||||
|
|
||||||
|
按 provider 查询需要修改模型解析契约及其调用方，留待后续，不用“第一个 provider 优先”近似实现。快照生成与运行时刷新使用同一套投影。
|
||||||
|
|
||||||
|
草稿中新增的 `model.customCatalog` 已移除：它的加载时机晚于首轮会话解析，进程全局状态还会在 Config 实例之间串用。私有或离线模型覆盖继续使用已有的 `modelProviders` generation 配置，不读取新增的自定义缓存文件。
|
||||||
|
|
||||||
|
## 存储与刷新
|
||||||
|
|
||||||
|
裁剪后的 JSON 快照随 CLI 发布，生成体积预算为 200 KiB；完整上游响应使用独立的 16 MiB 下载上限。后台刷新在代理初始化之后启动，超时为十秒，缓存间隔为 24 小时，通过 ETag 条件请求重新验证，在 `Storage.getGlobalQwenDir()` 下原子写入缓存。读取缓存与复用 ETag 均要求投影版本一致且存在可达的模型标识；旧版本或未标版本的缓存须完整下载并重新投影。调整投影规则时递增 `MODEL_CATALOG_PROJECTION_VERSION`。并发刷新共享进行中的请求。失败保留原有可用数据，仅记录 debug 日志。
|
||||||
|
|
||||||
|
`QWEN_CODE_MODELS_DEV=off` 恢复仅使用正则表。`QWEN_CODE_MODELS_DEV_REFRESH=off` 禁止刷新上游。`QWEN_CODE_MODELS_DEV_URL` 选择镜像，仍使用相同 provider 过滤。本次不引入请求时联网、跨进程锁、定时生成服务或新依赖。
|
||||||
|
|
||||||
|
## 验证与验收
|
||||||
|
|
||||||
|
定向测试覆盖缓存选择、非法条目、冲突拒绝、别名归一化、逐字段回退、纠正、刷新节流和失败处理。真实来源冒烟还需覆盖内置目录开关对照和真实 models.dev 刷新；mock fetch 无法证明上游数据准确。
|
||||||
|
|
||||||
|
[DashScope PDF 文档](https://www.alibabacloud.com/help/en/model-studio/pdf-understanding) 说明 qwen3.8-max 在北京和新加坡支持通过 Chat Completions 使用 `file_data` 与 `filename` 传入 PDF，并明确排除 Responses API 的 PDF 传递。因此目录不自动为任何模型启用 PDF；用户可针对已验证 endpoint 显式配置 `generationConfig.modalities.pdf`。这不会移除目录中的图片、音频和视频能力。自动推断 PDF 留待查询能识别 endpoint 和协议并完成真实识别测试后启用。
|
||||||
|
|
||||||
|
命令、结果和剩余交付门槛统一记录在[验证记录](../verification/models-dev-catalog/README.md)。最终 head 必须通过必要检查，PR 描述必须区分显式 PDF 配置与目录默认值。
|
||||||
@@ -0,0 +1,24 @@
|
|||||||
|
# models.dev 模型目录交付计划
|
||||||
|
|
||||||
|
状态：实现与本地验证完成，远程检查跟随 PR #11959 的最终提交。原纯设计 PR #9851 已被[双语设计文档](../design/2026-08-23-models-dev-registry.md)替代并关闭。
|
||||||
|
|
||||||
|
## 最终范围
|
||||||
|
|
||||||
|
内置裁剪目录、后台刷新、ETag 与 24 小时缓存、镜像及关闭开关、上下文/输出/模态查询与正则回退。不引入 effort 元数据迁移、provider-aware 解析、跨进程锁或定时服务。
|
||||||
|
|
||||||
|
- token 上限按归一化键做冲突检查，有分歧就丢弃上限并回退，包括带日期和 provider 前缀的条目；模态跨 provider 取并集。
|
||||||
|
- 输出上限保留既有正则匹配的优先级，目录只补充未匹配模型，避免覆盖 endpoint 请求上限。
|
||||||
|
- 拒绝非正安全整数 token 上限及非布尔模态。
|
||||||
|
- 纠正 Sonnet 4.5 已退役的 1M beta 数据，保留 200,000 默认上限。
|
||||||
|
- qwen3.8-max 的 PDF 不自动从目录启用；内置和刷新数据均受此约束。用户可针对验证过的 endpoint 显式设置 `generationConfig.modalities.pdf`。
|
||||||
|
- 移除未发布的 `model.customCatalog` 及其进程全局叠加状态，避免首轮不生效和跨 Config 串用。私有/离线模型继续使用现有 `modelProviders`，不读取自定义目录缓存。
|
||||||
|
|
||||||
|
## 验收
|
||||||
|
|
||||||
|
真实 Config 验证覆盖 settings 和 modelProviders 两条路径：第一个会话首次初始化及认证后即得到显式 context=12,345、pdf=true；第二个会话使用默认 context=1,000,000、pdf 未启用，且不改变第一个会话。目录开关和真实缓存节流保持正常。
|
||||||
|
|
||||||
|
定向测试、build、typecheck、lint、两轮复核及远程 CI 结果记录在[验证记录](../verification/models-dev-catalog/README.md)和 PR 评论。已知不可靠的默认行为通过收窄范围移除，不以缺少凭证的 live 测试冒充通过。
|
||||||
|
|
||||||
|
## 后续边界
|
||||||
|
|
||||||
|
自定义目录如需恢复，应从会话作用域模型解析设计入手，不重新引入全局 source。自动启用 Qwen PDF 需要 provider/协议作用域能力解析和真实识别验证。两者均不属于本 PR 的交付承诺。
|
||||||
@@ -822,37 +822,40 @@ For authentication-related variables (like `OPENAI_*`) and the recommended `.qwe
|
|||||||
|
|
||||||
### Environment Variables Table
|
### Environment Variables Table
|
||||||
|
|
||||||
| Variable | Description | Notes |
|
| Variable | Description | Notes |
|
||||||
| ---------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
| ---------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||||
| `QWEN_HOME` | Customizes the global configuration directory (default: `~/.qwen`). Accepts an absolute or relative path (relative paths are resolved from the current working directory). Leading `~` is expanded to the user's home directory. | Stores credentials, settings, memory, skills, and other global state. When set, project-level `.qwen/` directories are unaffected. An empty string is treated as unset. |
|
| `QWEN_HOME` | Customizes the global configuration directory (default: `~/.qwen`). Accepts an absolute or relative path (relative paths are resolved from the current working directory). Leading `~` is expanded to the user's home directory. | Stores credentials, settings, memory, skills, and other global state. When set, project-level `.qwen/` directories are unaffected. An empty string is treated as unset. |
|
||||||
| `QWEN_RUNTIME_DIR` | Overrides the runtime output directory (conversations, logs, todos). When unset, defaults to the `QWEN_HOME` directory. | Use this to separate ephemeral runtime data from persistent config. Useful when `QWEN_HOME` is on a shared/slow filesystem. |
|
| `QWEN_RUNTIME_DIR` | Overrides the runtime output directory (conversations, logs, todos). When unset, defaults to the `QWEN_HOME` directory. | Use this to separate ephemeral runtime data from persistent config. Useful when `QWEN_HOME` is on a shared/slow filesystem. |
|
||||||
| `QWEN_USAGE_STATISTICS_ENABLED` | Set to `true` or `1` to enable usage statistics. Any other value is treated as disabling them. | Overrides the `privacy.usageStatisticsEnabled` setting. Defaults to enabled when neither is configured. |
|
| `QWEN_USAGE_STATISTICS_ENABLED` | Set to `true` or `1` to enable usage statistics. Any other value is treated as disabling them. | Overrides the `privacy.usageStatisticsEnabled` setting. Defaults to enabled when neither is configured. |
|
||||||
| `QWEN_TELEMETRY_ENABLED` | Set to `true` or `1` to enable telemetry. Any other value is treated as disabling it. | Overrides the `telemetry.enabled` setting. |
|
| `QWEN_TELEMETRY_ENABLED` | Set to `true` or `1` to enable telemetry. Any other value is treated as disabling it. | Overrides the `telemetry.enabled` setting. |
|
||||||
| `QWEN_TELEMETRY_TARGET` | Sets an informational label for the telemetry destination (`local` or `gcp`). Does not control routing; use `QWEN_TELEMETRY_OTLP_ENDPOINT` or `QWEN_TELEMETRY_OUTFILE` to configure where data is sent. | Overrides the `telemetry.target` setting. |
|
| `QWEN_TELEMETRY_TARGET` | Sets an informational label for the telemetry destination (`local` or `gcp`). Does not control routing; use `QWEN_TELEMETRY_OTLP_ENDPOINT` or `QWEN_TELEMETRY_OUTFILE` to configure where data is sent. | Overrides the `telemetry.target` setting. |
|
||||||
| `QWEN_TELEMETRY_OTLP_ENDPOINT` | Sets the OTLP endpoint for telemetry. | Overrides the `telemetry.otlpEndpoint` setting. |
|
| `QWEN_TELEMETRY_OTLP_ENDPOINT` | Sets the OTLP endpoint for telemetry. | Overrides the `telemetry.otlpEndpoint` setting. |
|
||||||
| `QWEN_TELEMETRY_OTLP_PROTOCOL` | Sets the OTLP protocol (`grpc` or `http`). | Overrides the `telemetry.otlpProtocol` setting. |
|
| `QWEN_TELEMETRY_OTLP_PROTOCOL` | Sets the OTLP protocol (`grpc` or `http`). | Overrides the `telemetry.otlpProtocol` setting. |
|
||||||
| `QWEN_TELEMETRY_LOG_PROMPTS` | Set to `true` or `1` to enable or disable logging of user prompts. Any other value is treated as disabling it. | Overrides the `telemetry.logPrompts` setting. |
|
| `QWEN_TELEMETRY_LOG_PROMPTS` | Set to `true` or `1` to enable or disable logging of user prompts. Any other value is treated as disabling it. | Overrides the `telemetry.logPrompts` setting. |
|
||||||
| `QWEN_TELEMETRY_USER_ID` | Sets a stable end-user identifier on interaction, LLM, Tool, and Agent spans as `gen_ai.user.id`. Prefer a pseudonymous value. | Overrides `telemetry.userId` after trimming. A blank value falls back to settings. This is process-wide and must not be used as per-request identity in a shared multi-user process. |
|
| `QWEN_TELEMETRY_USER_ID` | Sets a stable end-user identifier on interaction, LLM, Tool, and Agent spans as `gen_ai.user.id`. Prefer a pseudonymous value. | Overrides `telemetry.userId` after trimming. A blank value falls back to settings. This is process-wide and must not be used as per-request identity in a shared multi-user process. |
|
||||||
| `QWEN_TELEMETRY_INCLUDE_SENSITIVE_SPAN_ATTRIBUTES` | Set to `true` or `1` to attach verbatim user prompts, system prompts, tool I/O, and model responses to native OTel span attributes (and keep `function_args`, `error`, `error.message`, and `error_message` on log-to-span bridge spans, plus `prompt` / `request_text` / `response_text` when `telemetry.logPrompts` is also enabled). Any other value disables it. | Overrides the `telemetry.includeSensitiveSpanAttributes` setting. ⚠️ Streams sensitive data to your OTLP backend. |
|
| `QWEN_TELEMETRY_INCLUDE_SENSITIVE_SPAN_ATTRIBUTES` | Set to `true` or `1` to attach verbatim user prompts, system prompts, tool I/O, and model responses to native OTel span attributes (and keep `function_args`, `error`, `error.message`, and `error_message` on log-to-span bridge spans, plus `prompt` / `request_text` / `response_text` when `telemetry.logPrompts` is also enabled). Any other value disables it. | Overrides the `telemetry.includeSensitiveSpanAttributes` setting. ⚠️ Streams sensitive data to your OTLP backend. |
|
||||||
| `QWEN_TELEMETRY_SENSITIVE_SPAN_ATTRIBUTE_MAX_LENGTH` | Sets the maximum JavaScript string length for each sensitive native OTel span attribute content payload. Must be a positive integer no greater than `104857600` (100 MiB). | Overrides the `telemetry.sensitiveSpanAttributeMaxLength` setting. Default is `1048576` (1 MiB); lower it if your collector or backend rejects large span attributes. |
|
| `QWEN_TELEMETRY_SENSITIVE_SPAN_ATTRIBUTE_MAX_LENGTH` | Sets the maximum JavaScript string length for each sensitive native OTel span attribute content payload. Must be a positive integer no greater than `104857600` (100 MiB). | Overrides the `telemetry.sensitiveSpanAttributeMaxLength` setting. Default is `1048576` (1 MiB); lower it if your collector or backend rejects large span attributes. |
|
||||||
| `QWEN_TELEMETRY_OUTFILE` | Sets the file path to write telemetry to. When set, overrides OTLP export. | Overrides the `telemetry.outfile` setting. |
|
| `QWEN_TELEMETRY_OUTFILE` | Sets the file path to write telemetry to. When set, overrides OTLP export. | Overrides the `telemetry.outfile` setting. |
|
||||||
| `QWEN_SANDBOX` | Alternative to the `sandbox` setting in `settings.json`. | Accepts `true`, `false`, `docker`, `podman`, or a custom command string. |
|
| `QWEN_SANDBOX` | Alternative to the `sandbox` setting in `settings.json`. | Accepts `true`, `false`, `docker`, `podman`, or a custom command string. |
|
||||||
| `QWEN_SANDBOX_IMAGE` | Overrides sandbox image selection for Docker/Podman. | Takes precedence over `tools.sandboxImage`. |
|
| `QWEN_SANDBOX_IMAGE` | Overrides sandbox image selection for Docker/Podman. | Takes precedence over `tools.sandboxImage`. |
|
||||||
| `SEATBELT_PROFILE` | (macOS specific) Switches the Seatbelt (`sandbox-exec`) profile on macOS. | `permissive-open`: (Default) Restricts writes to the project folder (and a few other folders, see `packages/cli/src/serve/sandbox-macos-permissive-open.sb`) but allows other operations. `strict`: Uses a strict profile that declines operations by default. `<profile_name>`: Uses a custom profile. To define a custom profile, create a file named `sandbox-macos-<profile_name>.sb` in your project's `.qwen/` directory (e.g., `my-project/.qwen/sandbox-macos-custom.sb`). |
|
| `SEATBELT_PROFILE` | (macOS specific) Switches the Seatbelt (`sandbox-exec`) profile on macOS. | `permissive-open`: (Default) Restricts writes to the project folder (and a few other folders, see `packages/cli/src/serve/sandbox-macos-permissive-open.sb`) but allows other operations. `strict`: Uses a strict profile that declines operations by default. `<profile_name>`: Uses a custom profile. To define a custom profile, create a file named `sandbox-macos-<profile_name>.sb` in your project's `.qwen/` directory (e.g., `my-project/.qwen/sandbox-macos-custom.sb`). |
|
||||||
| `DEBUG` or `DEBUG_MODE` | (often used by underlying libraries or the CLI itself) Set to `true` or `1` to enable verbose debug logging, which can be helpful for troubleshooting. | **Note:** These variables are automatically excluded from project `.env` files by default to prevent interference with the CLI behavior. Use `.qwen/.env` files if you need to set these for Qwen Code specifically. |
|
| `DEBUG` or `DEBUG_MODE` | (often used by underlying libraries or the CLI itself) Set to `true` or `1` to enable verbose debug logging, which can be helpful for troubleshooting. | **Note:** These variables are automatically excluded from project `.env` files by default to prevent interference with the CLI behavior. Use `.qwen/.env` files if you need to set these for Qwen Code specifically. |
|
||||||
| `NO_COLOR` | Set to any value to disable all color output in the CLI. | |
|
| `NO_COLOR` | Set to any value to disable all color output in the CLI. | |
|
||||||
| `FORCE_HYPERLINK` | Override the OSC 8 clickable-link detection in the markdown renderer. Set to `1` (or any non-zero integer, or empty string) to force-enable; set to `0` or a non-numeric value such as `false` / `off` to force-disable. Honors `NO_COLOR` / `QWEN_DISABLE_HYPERLINKS` opt-outs above it. | Use this to opt into OSC 8 inside `tmux` / GNU `screen` (auto-detection refuses by default because the host terminal's capabilities are hidden behind the multiplexer). Requires `set -g allow-passthrough on` on tmux 3.3+. Also enables Hyper, which isn't auto-detected. |
|
| `FORCE_HYPERLINK` | Override the OSC 8 clickable-link detection in the markdown renderer. Set to `1` (or any non-zero integer, or empty string) to force-enable; set to `0` or a non-numeric value such as `false` / `off` to force-disable. Honors `NO_COLOR` / `QWEN_DISABLE_HYPERLINKS` opt-outs above it. | Use this to opt into OSC 8 inside `tmux` / GNU `screen` (auto-detection refuses by default because the host terminal's capabilities are hidden behind the multiplexer). Requires `set -g allow-passthrough on` on tmux 3.3+. Also enables Hyper, which isn't auto-detected. |
|
||||||
| `QWEN_DISABLE_HYPERLINKS` | Set to `1` to hard-disable OSC 8 clickable hyperlinks in the markdown renderer even on terminals that auto-detect as capable. | Useful when a terminal advertises support but breaks on long URLs, or when piping output through an intermediary that mangles escape sequences. The renderer falls back to plain `label (url)` rendering. |
|
| `QWEN_DISABLE_HYPERLINKS` | Set to `1` to hard-disable OSC 8 clickable hyperlinks in the markdown renderer even on terminals that auto-detect as capable. | Useful when a terminal advertises support but breaks on long URLs, or when piping output through an intermediary that mangles escape sequences. The renderer falls back to plain `label (url)` rendering. |
|
||||||
| `CLI_TITLE` | Set to a string to customize the title of the CLI. | |
|
| `CLI_TITLE` | Set to a string to customize the title of the CLI. | |
|
||||||
| `CODE_ASSIST_ENDPOINT` | Specifies the endpoint for the code assist server. | This is useful for development and testing. |
|
| `CODE_ASSIST_ENDPOINT` | Specifies the endpoint for the code assist server. | This is useful for development and testing. |
|
||||||
| `QWEN_CODE_MAX_OUTPUT_TOKENS` | Overrides the default maximum output tokens per response. When not set, Qwen Code defaults to the model's declared output limit and, if a response is truncated, automatically escalates (64K floor) and recovers across turns. Set this to a specific value (e.g., `16000`) to use a fixed limit instead — useful for capacity-constrained self-hosted backends that want a lower per-request slot reservation. | Takes precedence over the model-limit default but is overridden by `samplingParams.max_tokens` in settings. Disables automatic escalation when set. Example: `export QWEN_CODE_MAX_OUTPUT_TOKENS=16000` |
|
| `QWEN_CODE_MAX_OUTPUT_TOKENS` | Overrides the default maximum output tokens per response. When not set, Qwen Code defaults to the model's declared output limit and, if a response is truncated, automatically escalates (64K floor) and recovers across turns. Set this to a specific value (e.g., `16000`) to use a fixed limit instead — useful for capacity-constrained self-hosted backends that want a lower per-request slot reservation. | Takes precedence over the model-limit default but is overridden by `samplingParams.max_tokens` in settings. Disables automatic escalation when set. Example: `export QWEN_CODE_MAX_OUTPUT_TOKENS=16000` |
|
||||||
| `QWEN_CODE_UNATTENDED_RETRY` | Set to `true` or `1` to enable persistent retry mode. When enabled, transient API capacity errors (HTTP 429 Rate Limit and 529 Overloaded) are retried indefinitely with exponential backoff (capped at 5 minutes per retry) and heartbeat keepalives every 30 seconds on stderr. | Designed for CI/CD pipelines and background automation where long-running tasks should survive temporary API outages. Must be set explicitly — `CI=true` alone does **not** activate this mode. See [Headless Mode](../features/headless#persistent-retry-mode) for details. Example: `export QWEN_CODE_UNATTENDED_RETRY=1` |
|
| `QWEN_CODE_UNATTENDED_RETRY` | Set to `true` or `1` to enable persistent retry mode. When enabled, transient API capacity errors (HTTP 429 Rate Limit and 529 Overloaded) are retried indefinitely with exponential backoff (capped at 5 minutes per retry) and heartbeat keepalives every 30 seconds on stderr. | Designed for CI/CD pipelines and background automation where long-running tasks should survive temporary API outages. Must be set explicitly — `CI=true` alone does **not** activate this mode. See [Headless Mode](../features/headless#persistent-retry-mode) for details. Example: `export QWEN_CODE_UNATTENDED_RETRY=1` |
|
||||||
| `QWEN_CODE_ACP_REPEATED_TOOL_FAILURE_GUARD` | Operator rollout mode for ACP repeated tool-execution failure protection. Accepts `off`, `shadow`, `warn`, or `enforce`; missing or invalid values default to `shadow`. | Applies only to interactive foreground ACP prompts; channel-driven and automatic routes remain off. Project and workspace environment files cannot set this operator policy. Shadow leaves model continuation and messages unchanged but adds the queued-prompt watch flag to `craft/drainMidTurnQueue`; every non-off mode requires reliable queued-prompt state. Non-empty invalid values emit a diagnostic; export the variable in the process environment or a user-level file. |
|
| `QWEN_CODE_ACP_REPEATED_TOOL_FAILURE_GUARD` | Operator rollout mode for ACP repeated tool-execution failure protection. Accepts `off`, `shadow`, `warn`, or `enforce`; missing or invalid values default to `shadow`. | Applies only to interactive foreground ACP prompts; channel-driven and automatic routes remain off. Project and workspace environment files cannot set this operator policy. Shadow leaves model continuation and messages unchanged but adds the queued-prompt watch flag to `craft/drainMidTurnQueue`; every non-off mode requires reliable queued-prompt state. Non-empty invalid values emit a diagnostic; export the variable in the process environment or a user-level file. |
|
||||||
| `QWEN_CODE_PROFILE_STARTUP` | Set to `1` to enable startup performance profiling. Writes a JSON timing report to `~/.qwen/startup-perf/` with per-phase durations. | Only active inside the sandbox child process (or with `QWEN_CODE_PROFILE_STARTUP_OUTER=1`). Zero overhead when not set. Example: `export QWEN_CODE_PROFILE_STARTUP=1` |
|
| `QWEN_CODE_PROFILE_STARTUP` | Set to `1` to enable startup performance profiling. Writes a JSON timing report to `~/.qwen/startup-perf/` with per-phase durations. | Only active inside the sandbox child process (or with `QWEN_CODE_PROFILE_STARTUP_OUTER=1`). Zero overhead when not set. Example: `export QWEN_CODE_PROFILE_STARTUP=1` |
|
||||||
| `QWEN_CODE_PROFILE_STARTUP_OUTER` | Set to `1` together with `QWEN_CODE_PROFILE_STARTUP=1` to also collect a startup profile in the outer (pre-sandbox) process. Outer-process reports get an `outer-` filename prefix to keep them distinct from the sandbox child's report. | Off by default — only the sandbox child collects, to avoid duplicate reports. Useful for local development where the cli isn't relaunched into a sandbox. |
|
| `QWEN_CODE_PROFILE_STARTUP_OUTER` | Set to `1` together with `QWEN_CODE_PROFILE_STARTUP=1` to also collect a startup profile in the outer (pre-sandbox) process. Outer-process reports get an `outer-` filename prefix to keep them distinct from the sandbox child's report. | Off by default — only the sandbox child collects, to avoid duplicate reports. Useful for local development where the cli isn't relaunched into a sandbox. |
|
||||||
| `QWEN_CODE_PROFILE_STARTUP_NO_HEAP` | Set to `1` together with `QWEN_CODE_PROFILE_STARTUP=1` to skip the per-checkpoint `process.memoryUsage()` snapshots. Useful when measuring the profiler's own Heisenberg overhead. | Off by default. Heap snapshots cost ~50 µs each (well below 1% of total startup) so most users should leave this alone. |
|
| `QWEN_CODE_PROFILE_STARTUP_NO_HEAP` | Set to `1` together with `QWEN_CODE_PROFILE_STARTUP=1` to skip the per-checkpoint `process.memoryUsage()` snapshots. Useful when measuring the profiler's own Heisenberg overhead. | Off by default. Heap snapshots cost ~50 µs each (well below 1% of total startup) so most users should leave this alone. |
|
||||||
| `QWEN_CODE_LEGACY_MCP_BLOCKING` | Set to `1` to restore the pre-progressive-MCP behavior where `Config.initialize()` waits synchronously for every configured MCP server's discover handshake before returning. | Off by default. Modern qwen-code lets MCP servers come online in the background while the UI is already interactive; the model sees each batch of new tools within ~16 ms of the server settling. This flag is kept as a rollback escape hatch for ≥ 1 release. Example: `export QWEN_CODE_LEGACY_MCP_BLOCKING=1` |
|
| `QWEN_CODE_LEGACY_MCP_BLOCKING` | Set to `1` to restore the pre-progressive-MCP behavior where `Config.initialize()` waits synchronously for every configured MCP server's discover handshake before returning. | Off by default. Modern qwen-code lets MCP servers come online in the background while the UI is already interactive; the model sees each batch of new tools within ~16 ms of the server settling. This flag is kept as a rollback escape hatch for ≥ 1 release. Example: `export QWEN_CODE_LEGACY_MCP_BLOCKING=1` |
|
||||||
| `QWEN_CODE_LEGACY_ERASE_LINES` | `=1` force-disables the terminal redraw optimizer (restores per-line erase sequences); `=0` force-enables it even on WSL, where it is skipped by default because ConPTY mishandles the optimizer's batched cursor moves (issue #7634). Unset = platform default (skip when `WSL_DISTRO_NAME` or `WSL_INTEROP` is set). | Escape hatch for streaming-output regressions. Because it is read from the environment, launchers that scrub env (e.g. `sudo`) drop it too — pass it at launch instead: `sudo QWEN_CODE_LEGACY_ERASE_LINES=1 qwen`. Example: `export QWEN_CODE_LEGACY_ERASE_LINES=1` |
|
| `QWEN_CODE_MODELS_DEV` | Set to `off` to ignore the bundled [models.dev](https://models.dev) model catalog and resolve context windows, output limits, and input modalities from the built-in model-name tables only. | On by default. Only the exact value `off` disables the catalog; it also suppresses the background refresh below. Project-level `.env` files and `settings.json` `env` cannot set this key, and it cannot be changed by a reload; use the shell environment. Example: `export QWEN_CODE_MODELS_DEV=off` |
|
||||||
|
| `QWEN_CODE_MODELS_DEV_REFRESH` | Set to `off` to stop the daily models.dev download into `~/.qwen/model-registry.json` (air-gapped or proxy-restricted environments). Refreshed data is available to new CLI processes; running processes keep their selected catalog. | On by default (24-hour interval). Stops only the download — a previously downloaded `~/.qwen/model-registry.json` is used while its `fetchedAt` timestamp is newer than the snapshot bundled with the installed release; after an upgrade ships a newer snapshot the bundled data wins until you refresh again. Use `QWEN_CODE_MODELS_DEV=off` to ignore downloaded catalog data entirely. Project-level `.env` files and `settings.json` `env` cannot set this key, and it cannot be changed by a reload; use the shell environment. |
|
||||||
|
| `QWEN_CODE_MODELS_DEV_URL` | Replaces the models.dev download URL with a full mirror of `https://models.dev/api.json` (same provider filtering applies). To add or correct individual models instead, use explicit `modelProviders` generation settings. | Unset — downloads from `https://models.dev/api.json`. Project-level `.env` files and `settings.json` `env` cannot set this key; use the shell environment or a user-level `.env` file. Example: `export QWEN_CODE_MODELS_DEV_URL=https://mirror.corp/api.json` |
|
||||||
|
| `QWEN_CODE_LEGACY_ERASE_LINES` | `=1` force-disables the terminal redraw optimizer (restores per-line erase sequences); `=0` force-enables it even on WSL, where it is skipped by default because ConPTY mishandles the optimizer's batched cursor moves (issue #7634). Unset = platform default (skip when `WSL_DISTRO_NAME` or `WSL_INTEROP` is set). | Escape hatch for streaming-output regressions. Because it is read from the environment, launchers that scrub env (e.g. `sudo`) drop it too — pass it at launch instead: `sudo QWEN_CODE_LEGACY_ERASE_LINES=1 qwen`. Example: `export QWEN_CODE_LEGACY_ERASE_LINES=1` |
|
||||||
|
|
||||||
When both user-level `.env` files define the same variable, the Qwen-specific
|
When both user-level `.env` files define the same variable, the Qwen-specific
|
||||||
file wins: `<QWEN_HOME>/.env` (or `~/.qwen/.env` when `QWEN_HOME` is unset) is
|
file wins: `<QWEN_HOME>/.env` (or `~/.qwen/.env` when `QWEN_HOME` is unset) is
|
||||||
|
|||||||
@@ -0,0 +1,66 @@
|
|||||||
|
# Verification: models.dev model catalog (PR #11959)
|
||||||
|
|
||||||
|
## Final delivery scope
|
||||||
|
|
||||||
|
The catalog supplies inferred context windows and input modalities with bundled/offline fallback and background refresh. Output limits keep existing regex matches first; the catalog fills models without a table match. The [bilingual design](../../design/2026-08-23-models-dev-registry.md) incorporates the useful constraints from the closed design-only PR #9851.
|
||||||
|
|
||||||
|
The draft-only `model.customCatalog` was removed after real Config testing exposed late initialization and cross-session global state. Private/offline overrides use existing `modelProviders` or model generation settings. No custom catalog cache is read. Effort metadata and provider-aware catalog lookup remain deferred.
|
||||||
|
|
||||||
|
Catalog PDF support is **explicit opt-in** for every model. This removes an unverified, protocol-dependent behavior from the release scope while preserving explicit model configuration. The correction applies equally to bundled and refreshed metadata.
|
||||||
|
|
||||||
|
## Evidence collected on 2026-09-26
|
||||||
|
|
||||||
|
- Original head `7352766da9`: four focused unit files passed, 190 tests. Actual Config testing subsequently reproduced custom-catalog context 12,345 being ignored in favor of 1,048,576, followed by another Config resetting its global source. These observations motivated removal of the draft-only feature.
|
||||||
|
- After removal, a real Config lifecycle test covered both `settings.generationConfig` and `modelProviders[].generationConfig`: the first Config initialized and authenticated with context 12,345 and PDF enabled; a second Config used qwen3.8-max defaults of 1,000,000 and PDF undefined. The first Config retained its explicit values after the second initialized. No provider generation request or mocks were used.
|
||||||
|
- Catalog on/off/re-enabled behavior passed through the real resolution path. Explicit model settings remained authoritative; default Qwen PDF stayed disabled.
|
||||||
|
- Real `https://models.dev/api.json` refresh returned HTTP 200 and an ETag, writing 195 models. The immediate second refresh issued no request and preserved the timestamp. After simplification, two more refreshes against that fresh cache issued zero requests.
|
||||||
|
- Regeneration with the corrected conflict filter produced 195 entries, 25,363 bytes, below the 200 KiB budget. Counts describe this payload rather than a permanent contract. A later reachability rule — every catalog key must be a `normalize()` fixed point — omitted `deepseek-v3` (`normalize('deepseek-v3')` is `deepseek`), leaving 194 entries.
|
||||||
|
- Regression sensitivity: the first closeout's new checks failed seven assertions with the old implementation. Disabling the Qwen PDF correction separately made its resolver test fail. Restored code passed the focused suites.
|
||||||
|
|
||||||
|
The numbers above are pinned inside the repo: `preserves existing output limits across the entire bundled snapshot` and `requires explicit Qwen PDF support for both bundled and refreshed data` in `packages/core/src/models/model-catalog.test.ts`, plus the shipped-snapshot parser/budget gate (`keeps every entry of the committed snapshot parseable`); run them with the vitest command under "Reproducible checks". The PR verification comment records final build, typecheck, focused test counts and CI URLs. The original build's export-budget failure was resolved by merging main's existing #12298 repair, not by changing an unrelated budget here.
|
||||||
|
|
||||||
|
## Data and protocol decisions
|
||||||
|
|
||||||
|
[Anthropic's current reference](https://platform.claude.com/docs/en/build-with-claude/context-windows) specifies Sonnet 4.5 at 200K context, while Sonnet 4.6/5 have a default 1M window without a beta header. The lookup corrects stale Sonnet 4.5 claims even in a newer cache. No paid long-context request was made.
|
||||||
|
|
||||||
|
[DashScope's PDF reference](https://www.alibabacloud.com/help/en/model-studio/pdf-understanding) supports qwen3.8-max through Chat Completions in Beijing and Singapore, with Base64 `file_data` plus `filename`; it explicitly excludes Responses API PDF delivery. A provider-independent catalog cannot infer this distinction safely. The default therefore leaves PDF disabled while preserving image/video and explicit PDF opt-in.
|
||||||
|
|
||||||
|
Live PDF recognition was not run: configured credential variables were absent. Automatic PDF enablement is deferred until endpoint/protocol-scoped resolution and a real recognition test exist. A future test should return a random marker present only inside a one-page PDF; HTTP 200 alone does not prove delivery to the model.
|
||||||
|
|
||||||
|
## Review follow-up: output precedence and small windows
|
||||||
|
|
||||||
|
Exact-head comparison at `4086a0c` reproduced GLM-4.7's default output increasing from 16,384 to 64,000, and a Qwen-VL-Max request of 16,384 being reduced to 8,192. Preserving existing output-table matches fixes both. Tests now iterate every bundled identifier to verify existing output limits remain unchanged and exercise the same precedence with refreshed data. Context corrections for Sonnet 4.6/5 remain intact. Restoring catalog-first output makes both new tests fail.
|
||||||
|
|
||||||
|
The suggested context bounds are not adopted. With an actual 4,096-token math model and a 2,000-token prompt, the reviewed catalog yields an output request of 3,072; the regex-only path yields 32,768 against the same physical window. Neither fits. Likewise, the existing clamp can request 4,000 output tokens for an explicitly configured 8,192-token window and a 5,000-token prompt. This is an existing small-window budgeting limitation, not evidence that replacing accurate metadata with a 128K/256K fallback is safe. Supporting these windows requires separately reviewing compaction and the send path's fixed estimation padding. This PR does not claim to add small-window model support. No arithmetic defect was observed for Qwen-Long's 10M context, so no arbitrary upper bound is added.
|
||||||
|
|
||||||
|
The local comparison scripts exercised the real token helpers and default provider; they did not send live generation requests. Their outcome is pinned by the committed regression tests named above, which run against the shipped snapshot on every build. A catalog-on/off golden copy of all metadata would not validate endpoint accuracy; the shipped-data regression test instead protects the output precedence contract directly.
|
||||||
|
|
||||||
|
## Reproducible checks
|
||||||
|
|
||||||
|
From the repository root, run `npm run build` and `npm run typecheck`. From `packages/core`, run:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
npx vitest run src/models/model-catalog.test.ts src/models/model-catalog-refresh.test.ts src/core/tokenLimits.test.ts src/core/modalityDefaults.test.ts src/core/modalityDefaults.catalog.test.ts src/models/modelConfigResolver.test.ts src/models/modelsConfig.test.ts --coverage.enabled=false
|
||||||
|
```
|
||||||
|
|
||||||
|
These suites cover real bundled configuration resolution, explicit overrides, offline switches, invalid data, normalized conflicts, corrections, cache freshness, 304 revalidation, failed fetches and concurrent refreshes. Global suites retain regex-only setup; the catalog-specific suite explicitly enables the real catalog in an isolated QWEN_HOME.
|
||||||
|
|
||||||
|
For a real lifecycle check, resolve qwen3.8-max with an explicit context and modalities, construct and initialize Config, refresh authentication, then initialize a second Config without those explicit values. The first must retain its values and the second must receive the catalog defaults with no PDF capability. No live model request is necessary for this configuration invariant.
|
||||||
|
|
||||||
|
## Current-head evidence and explicit output budgets
|
||||||
|
|
||||||
|
The final snapshot was regenerated with the shipped projection on 2026-09-30 and now contains 203 entries (27,711 bytes), including modality-only entries such as `inkling`. The 2026-09-30 regeneration also dropped `modelscope` from the limits voter list (a pure mirror whose hosted numbers vetoed first-party GLM and supplied dated `-2507` qwen3 snapshots under bare ids), which restored `glm-4.6` to its first-party Z.ai figures. Catalog context windows below 32,000 are omitted so the 85% auto-compaction threshold still leaves room for the 4,000-token minimum output request; those models retain the existing family-table fallback. DeepSeek modality claims are accepted only for explicit `vision` variants. The earlier 194/195/205-entry measurements above describe historical payloads before the reachability, trust, and agent-window guards. Public [round-2 verification](https://github.com/QwenLM/qwen-code/pull/11959#issuecomment-5851908713) and its [scripts and raw results](https://github.com/wenshao/qwen-code/tree/assets-pr11959/pr11959/round2) cover the real CLI, endpoint benefits, cache recovery and project environment isolation at `9088f00b`.
|
||||||
|
|
||||||
|
The remaining explicit-budget regression is corrected: catalog-only output limits provide defaults without clipping explicit request or environment budgets. Existing curated output caps and context-window clamping remain. Real bundled-data provider tests cover QwQ-32B, Kimi-K2-Thinking and QVQ-Max defaults and explicit 32,768 requests, plus the unchanged GLM-4.7 cap. An explicitly excessive budget can still be rejected by its backend, as before this PR.
|
||||||
|
|
||||||
|
## Refresh delivery correction (2026-09-30)
|
||||||
|
|
||||||
|
The raw models.dev response and the shipped projection have different size budgets. The official HTTP 200 `application/json` response measured 5,265,259 bytes; applying the projection's 200 KiB budget to that response prevented every normal runtime refresh. The runtime now allows a bounded 16 MiB response, while the trimmed snapshot retains its 200 KiB generation budget. Replaying the exact official payload wrote 203 parseable models into an isolated cache. A streamed 16 MiB + 1 byte response was cancelled and left the previous cache byte-for-byte unchanged. A multi-megabyte regression fails with the old 200 KiB limit.
|
||||||
|
|
||||||
|
A second real-source probe confirmed that invalidating the catalog during a session mixed a previously resolved context window with a newly computed output limit. Refresh now only writes to disk: once selected, the process keeps its catalog, and a new process selects the refreshed cache. The probe retained 524,288 input / 32,000 output tokens across refresh, then selected 131,072 input / 64,000 output tokens together after simulating a new process. No provider generation request was needed. The regression checks the same lifetime boundary for an inferred model without a curated output cap.
|
||||||
|
|
||||||
|
## Cache projection compatibility (2026-09-30)
|
||||||
|
|
||||||
|
A real tmux run at `2b9fc537c6` reproduced an upgrade failure: an old projected cache with an unchanged ETag received HTTP 304, acquired a fresh timestamp, and displaced the bundled snapshot on restart. For the controlled `qwen-flash` cache, `/context` then displayed 131.1k tokens instead of the bundled 1000.0k. The catalog and cache now carry a projection version; only a matching version may supply metadata or an ETag. Old or unstamped caches trigger a full fetch and re-projection. If that fails, the bundled snapshot remains available. Projection-rule changes require incrementing `MODEL_CATALOG_PROJECTION_VERSION`.
|
||||||
|
|
||||||
|
The two catalog suites passed 60 tests. Removing the two production version checks made six new regression cases fail, with the suites collecting successfully. The cases cover missing, older and newer projection versions, including a server returning an unsolicited 304. Restoring the checks returned the suites to green. The PR's tmux verification report records the matched before/after process runs, cache contents, request headers and UI screenshots.
|
||||||
@@ -35,6 +35,7 @@ const SETUP_ENV_KEYS = [
|
|||||||
'TEST_CLI_PATH',
|
'TEST_CLI_PATH',
|
||||||
'VERBOSE',
|
'VERBOSE',
|
||||||
'KEEP_OUTPUT',
|
'KEEP_OUTPUT',
|
||||||
|
'QWEN_CODE_MODELS_DEV_REFRESH',
|
||||||
] as const;
|
] as const;
|
||||||
|
|
||||||
describe('globalSetup memory-file save/restore', () => {
|
describe('globalSetup memory-file save/restore', () => {
|
||||||
@@ -116,6 +117,7 @@ describe('globalSetup hermetic qwen home', () => {
|
|||||||
// it lands in the OS temp dir — redirect that so the case owns what the
|
// it lands in the OS temp dir — redirect that so the case owns what the
|
||||||
// module picks. Both are read at import time, so they must be set first.
|
// module picks. Both are read at import time, so they must be set first.
|
||||||
delete process.env['QWEN_HOME'];
|
delete process.env['QWEN_HOME'];
|
||||||
|
process.env['QWEN_CODE_MODELS_DEV_REFRESH'] = 'on';
|
||||||
process.env['TMPDIR'] = tmpRoot;
|
process.env['TMPDIR'] = tmpRoot;
|
||||||
process.env['KEEP_OUTPUT'] = 'false';
|
process.env['KEEP_OUTPUT'] = 'false';
|
||||||
});
|
});
|
||||||
@@ -143,6 +145,7 @@ describe('globalSetup hermetic qwen home', () => {
|
|||||||
|
|
||||||
const home = process.env['QWEN_HOME'];
|
const home = process.env['QWEN_HOME'];
|
||||||
expect(home?.startsWith(tmpRoot)).toBe(true);
|
expect(home?.startsWith(tmpRoot)).toBe(true);
|
||||||
|
expect(process.env['QWEN_CODE_MODELS_DEV_REFRESH']).toBe('off');
|
||||||
|
|
||||||
await expect(teardown()).resolves.toBeUndefined();
|
await expect(teardown()).resolves.toBeUndefined();
|
||||||
expect(existsSync(home!)).toBe(false);
|
expect(existsSync(home!)).toBe(false);
|
||||||
|
|||||||
@@ -63,6 +63,7 @@ const ownsQwenHome = !process.env['QWEN_HOME'];
|
|||||||
if (ownsQwenHome) {
|
if (ownsQwenHome) {
|
||||||
process.env['QWEN_HOME'] = hermeticQwenHome;
|
process.env['QWEN_HOME'] = hermeticQwenHome;
|
||||||
}
|
}
|
||||||
|
process.env['QWEN_CODE_MODELS_DEV_REFRESH'] = 'off';
|
||||||
|
|
||||||
// Read after the redirect so the save/restore below, the spawned CLIs, and
|
// Read after the redirect so the save/restore below, the spawned CLIs, and
|
||||||
// the tests all agree on one global qwen dir.
|
// the tests all agree on one global qwen dir.
|
||||||
|
|||||||
@@ -1048,6 +1048,7 @@ export async function runForcedToolCallScenario(options: {
|
|||||||
vi.stubEnv('QWEN_MODEL', 'fake-model');
|
vi.stubEnv('QWEN_MODEL', 'fake-model');
|
||||||
vi.stubEnv('QWEN_HOME', join(rig.testDir!, '.qwen-home'));
|
vi.stubEnv('QWEN_HOME', join(rig.testDir!, '.qwen-home'));
|
||||||
vi.stubEnv('QWEN_RUNTIME_DIR', join(rig.testDir!, '.qwen-home'));
|
vi.stubEnv('QWEN_RUNTIME_DIR', join(rig.testDir!, '.qwen-home'));
|
||||||
|
vi.stubEnv('QWEN_CODE_MODELS_DEV_REFRESH', 'off');
|
||||||
vi.stubEnv('NO_PROXY', noProxy);
|
vi.stubEnv('NO_PROXY', noProxy);
|
||||||
vi.stubEnv('no_proxy', noProxy);
|
vi.stubEnv('no_proxy', noProxy);
|
||||||
|
|
||||||
|
|||||||
@@ -41,6 +41,7 @@
|
|||||||
"debug": "cross-env DEBUG=1 node --inspect-brk scripts/start.js",
|
"debug": "cross-env DEBUG=1 node --inspect-brk scripts/start.js",
|
||||||
"generate": "node scripts/generate-git-commit-info.js",
|
"generate": "node scripts/generate-git-commit-info.js",
|
||||||
"generate:settings-schema": "node --import tsx/esm scripts/generate-settings-schema.ts",
|
"generate:settings-schema": "node --import tsx/esm scripts/generate-settings-schema.ts",
|
||||||
|
"generate:model-catalog": "node --import tsx/esm scripts/generate-model-catalog.ts",
|
||||||
"build": "cross-env NODE_OPTIONS=\"--max-old-space-size=4096\" node scripts/build.js",
|
"build": "cross-env NODE_OPTIONS=\"--max-old-space-size=4096\" node scripts/build.js",
|
||||||
"build-and-start": "npm run build && npm run start",
|
"build-and-start": "npm run build && npm run start",
|
||||||
"build:vscode": "node scripts/build_vscode_companion.js",
|
"build:vscode": "node scripts/build_vscode_companion.js",
|
||||||
|
|||||||
@@ -125,6 +125,12 @@ const TRACKED_ENV = [
|
|||||||
'DASHSCOPE_PROXY_BASE_URL',
|
'DASHSCOPE_PROXY_BASE_URL',
|
||||||
'QWEN_TEST_PROVIDER_KEY',
|
'QWEN_TEST_PROVIDER_KEY',
|
||||||
'DEMO_VAR',
|
'DEMO_VAR',
|
||||||
|
'QWEN_CODE_MODELS_DEV_URL',
|
||||||
|
// The CLI test setup exports QWEN_CODE_MODELS_DEV=off process-wide, so the
|
||||||
|
// switch keys must be cleared per test (and restored after) for an
|
||||||
|
// undefined assertion to mean "the project file was rejected".
|
||||||
|
'QWEN_CODE_MODELS_DEV',
|
||||||
|
'QWEN_CODE_MODELS_DEV_REFRESH',
|
||||||
] as const;
|
] as const;
|
||||||
|
|
||||||
let tmpDirs: string[] = [];
|
let tmpDirs: string[] = [];
|
||||||
@@ -617,6 +623,84 @@ describe('update download source environment', () => {
|
|||||||
);
|
);
|
||||||
});
|
});
|
||||||
|
|
||||||
|
describe('model catalog download source environment', () => {
|
||||||
|
beforeEach(() => {
|
||||||
|
resetEnvironmentTrackingForTesting();
|
||||||
|
});
|
||||||
|
|
||||||
|
afterEach(() => {
|
||||||
|
resetEnvironmentTrackingForTesting();
|
||||||
|
});
|
||||||
|
|
||||||
|
it.each(['.env', '.qwen/.env', 'settings.env'])(
|
||||||
|
'rejects the project-scoped catalog keys from %s',
|
||||||
|
(source) => {
|
||||||
|
const workspace = makeWorkspace();
|
||||||
|
const settings = testSettings({ advanced: { excludedEnvVars: [] } });
|
||||||
|
// All three catalog keys are operator decisions: the URL picks where
|
||||||
|
// the shared global cache comes from, and the two switches decide
|
||||||
|
// whether it is consulted or refreshed at all — on the serve fast path
|
||||||
|
// one workspace's file would otherwise freeze that choice for every
|
||||||
|
// other workspace the daemon hosts.
|
||||||
|
const projectKeys = {
|
||||||
|
QWEN_CODE_MODELS_DEV_URL: 'https://project.example.com/api.json',
|
||||||
|
// `on` is the attack shape: the value that would flip the catalog
|
||||||
|
// daemon-wide, or over an operator's exported `off`.
|
||||||
|
QWEN_CODE_MODELS_DEV: 'on',
|
||||||
|
QWEN_CODE_MODELS_DEV_REFRESH: 'on',
|
||||||
|
} as const;
|
||||||
|
if (source === 'settings.env') {
|
||||||
|
settings.env = { ...projectKeys, RUNTIME_SETTINGS_ONLY: 'allowed' };
|
||||||
|
} else {
|
||||||
|
const envPath = path.join(workspace, source);
|
||||||
|
fs.mkdirSync(path.dirname(envPath), { recursive: true });
|
||||||
|
fs.writeFileSync(
|
||||||
|
envPath,
|
||||||
|
`${Object.entries(projectKeys)
|
||||||
|
.map(([key, value]) => `${key}=${value}`)
|
||||||
|
.join('\n')}\nRUNTIME_DOTENV=allowed\n`,
|
||||||
|
);
|
||||||
|
}
|
||||||
|
// The allowed control proves the project file itself was applied; the
|
||||||
|
// exclusion, not a discovery failure, is what drops the catalog keys.
|
||||||
|
const allowedKey =
|
||||||
|
source === 'settings.env' ? 'RUNTIME_SETTINGS_ONLY' : 'RUNTIME_DOTENV';
|
||||||
|
|
||||||
|
loadEnvironment(settings, workspace);
|
||||||
|
for (const key of Object.keys(projectKeys)) {
|
||||||
|
expect(process.env[key]).toBeUndefined();
|
||||||
|
}
|
||||||
|
expect(process.env[allowedKey]).toBe('allowed');
|
||||||
|
|
||||||
|
reloadEnvironment(settings, workspace);
|
||||||
|
for (const key of Object.keys(projectKeys)) {
|
||||||
|
expect(process.env[key]).toBeUndefined();
|
||||||
|
}
|
||||||
|
expect(process.env[allowedKey]).toBe('allowed');
|
||||||
|
|
||||||
|
const snapshot = buildRuntimeEnvironment(settings, workspace, {});
|
||||||
|
for (const key of Object.keys(projectKeys)) {
|
||||||
|
expect(snapshot.effectiveEnv[key]).toBeUndefined();
|
||||||
|
}
|
||||||
|
expect(snapshot.effectiveEnv[allowedKey]).toBe('allowed');
|
||||||
|
},
|
||||||
|
);
|
||||||
|
|
||||||
|
it('preserves the operator-supplied catalog URL against project configuration', () => {
|
||||||
|
const workspace = makeWorkspace();
|
||||||
|
process.env['QWEN_CODE_MODELS_DEV_URL'] = 'https://mirror.corp/api.json';
|
||||||
|
fs.writeFileSync(
|
||||||
|
path.join(workspace, '.env'),
|
||||||
|
'QWEN_CODE_MODELS_DEV_URL=https://project.example.com/api.json\n',
|
||||||
|
);
|
||||||
|
|
||||||
|
loadEnvironment(testSettings({}), workspace);
|
||||||
|
expect(process.env['QWEN_CODE_MODELS_DEV_URL']).toBe(
|
||||||
|
'https://mirror.corp/api.json',
|
||||||
|
);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
describe('daemon registration capacity environment', () => {
|
describe('daemon registration capacity environment', () => {
|
||||||
it.each([undefined, '32'])(
|
it.each([undefined, '32'])(
|
||||||
'keeps project files from overriding operator capacity %s',
|
'keeps project files from overriding operator capacity %s',
|
||||||
|
|||||||
@@ -307,6 +307,22 @@ describe('isHardcodedProjectEnvExclusion', () => {
|
|||||||
});
|
});
|
||||||
|
|
||||||
it('matches the newly added hardcoded exclusions case-insensitively', () => {
|
it('matches the newly added hardcoded exclusions case-insensitively', () => {
|
||||||
|
expect(isHardcodedProjectEnvExclusion('QWEN_CODE_MODELS_DEV_URL')).toBe(
|
||||||
|
true,
|
||||||
|
);
|
||||||
|
expect(isHardcodedProjectEnvExclusion('qwen_code_models_dev_url')).toBe(
|
||||||
|
true,
|
||||||
|
);
|
||||||
|
// The two switches the URL key sits beside: a project must not flip the
|
||||||
|
// catalog daemon-wide (or over the operator's exported `off`) either.
|
||||||
|
expect(isHardcodedProjectEnvExclusion('QWEN_CODE_MODELS_DEV')).toBe(true);
|
||||||
|
expect(isHardcodedProjectEnvExclusion('qwen_code_models_dev')).toBe(true);
|
||||||
|
expect(isHardcodedProjectEnvExclusion('QWEN_CODE_MODELS_DEV_REFRESH')).toBe(
|
||||||
|
true,
|
||||||
|
);
|
||||||
|
expect(isHardcodedProjectEnvExclusion('qwen_code_models_dev_refresh')).toBe(
|
||||||
|
true,
|
||||||
|
);
|
||||||
expect(isHardcodedProjectEnvExclusion('SSL_CERT_FILE')).toBe(true);
|
expect(isHardcodedProjectEnvExclusion('SSL_CERT_FILE')).toBe(true);
|
||||||
expect(isHardcodedProjectEnvExclusion('ssl_cert_file')).toBe(true);
|
expect(isHardcodedProjectEnvExclusion('ssl_cert_file')).toBe(true);
|
||||||
expect(isHardcodedProjectEnvExclusion('GIT_SSH_COMMAND')).toBe(true);
|
expect(isHardcodedProjectEnvExclusion('GIT_SSH_COMMAND')).toBe(true);
|
||||||
|
|||||||
@@ -45,6 +45,19 @@ export const PROJECT_ENV_HARDCODED_EXCLUSIONS = [
|
|||||||
'QWEN_CODE_SYSTEM_DEFAULTS_PATH',
|
'QWEN_CODE_SYSTEM_DEFAULTS_PATH',
|
||||||
// Downloaded updates execute as the user; a project must not select them.
|
// Downloaded updates execute as the user; a project must not select them.
|
||||||
'QWEN_UPDATE_BASE_URL',
|
'QWEN_UPDATE_BASE_URL',
|
||||||
|
// The model catalog is written to the global cache that every project
|
||||||
|
// reads, so all three of its keys are operator decisions. A project must
|
||||||
|
// not choose where it is downloaded from; and because `qwen serve`
|
||||||
|
// snapshots and freezes this environment before any workspace file loads
|
||||||
|
// — every workspace child inherits that snapshot, and a workspace can only
|
||||||
|
// add absent keys (`setRuntimeEnvIfUnset`) — a repository's own config
|
||||||
|
// would otherwise decide catalog behaviour for every *other* workspace the
|
||||||
|
// daemon hosts, and override an operator's exported `off` on the reload
|
||||||
|
// path. The rejection is silent (it is not a loader key), so nothing else
|
||||||
|
// reports it either.
|
||||||
|
'QWEN_CODE_MODELS_DEV_URL',
|
||||||
|
'QWEN_CODE_MODELS_DEV',
|
||||||
|
'QWEN_CODE_MODELS_DEV_REFRESH',
|
||||||
// This points to a host temp file that carries build warnings. A project
|
// This points to a host temp file that carries build warnings. A project
|
||||||
// `.env` must not redirect it to an arbitrary file to read or delete.
|
// `.env` must not redirect it to an arbitrary file to read or delete.
|
||||||
'QWEN_CODE_WARNINGS_FILE',
|
'QWEN_CODE_WARNINGS_FILE',
|
||||||
|
|||||||
@@ -391,6 +391,9 @@ describe('getSandboxPassthroughEnvArgs', () => {
|
|||||||
it('passes update relaunch state into container sandboxes', () => {
|
it('passes update relaunch state into container sandboxes', () => {
|
||||||
expect(
|
expect(
|
||||||
getSandboxPassthroughEnvArgs({
|
getSandboxPassthroughEnvArgs({
|
||||||
|
QWEN_CODE_MODELS_DEV: 'off',
|
||||||
|
QWEN_CODE_MODELS_DEV_REFRESH: 'off',
|
||||||
|
QWEN_CODE_MODELS_DEV_URL: 'https://mirror.example/api.json',
|
||||||
QWEN_CODE_SKIP_UPDATE_CHECK_ONCE: 'true',
|
QWEN_CODE_SKIP_UPDATE_CHECK_ONCE: 'true',
|
||||||
QWEN_CODE_CUSTOM_SANDBOX_IMAGE: 'example.com/qwen:1.0.0',
|
QWEN_CODE_CUSTOM_SANDBOX_IMAGE: 'example.com/qwen:1.0.0',
|
||||||
QWEN_CODE_HOST_UPDATE_RELAUNCH: 'false',
|
QWEN_CODE_HOST_UPDATE_RELAUNCH: 'false',
|
||||||
@@ -398,6 +401,12 @@ describe('getSandboxPassthroughEnvArgs', () => {
|
|||||||
QWEN_CODE_DESKTOP: '1',
|
QWEN_CODE_DESKTOP: '1',
|
||||||
}),
|
}),
|
||||||
).toEqual([
|
).toEqual([
|
||||||
|
'--env',
|
||||||
|
'QWEN_CODE_MODELS_DEV=off',
|
||||||
|
'--env',
|
||||||
|
'QWEN_CODE_MODELS_DEV_REFRESH=off',
|
||||||
|
'--env',
|
||||||
|
'QWEN_CODE_MODELS_DEV_URL=https://mirror.example/api.json',
|
||||||
'--env',
|
'--env',
|
||||||
'QWEN_CODE_SKIP_UPDATE_CHECK_ONCE=true',
|
'QWEN_CODE_SKIP_UPDATE_CHECK_ONCE=true',
|
||||||
'--env',
|
'--env',
|
||||||
|
|||||||
@@ -22,6 +22,11 @@ import { Storage } from '@qwen-code/qwen-code-core/config/storage.js';
|
|||||||
import { resolveBundleDir } from '@qwen-code/qwen-code-core/utils/bundlePaths.js';
|
import { resolveBundleDir } from '@qwen-code/qwen-code-core/utils/bundlePaths.js';
|
||||||
import { FatalSandboxError } from '@qwen-code/qwen-code-core/utils/errors.js';
|
import { FatalSandboxError } from '@qwen-code/qwen-code-core/utils/errors.js';
|
||||||
import { isSubpath } from '@qwen-code/qwen-code-core/utils/paths.js';
|
import { isSubpath } from '@qwen-code/qwen-code-core/utils/paths.js';
|
||||||
|
import { MODEL_CATALOG_ENV } from '@qwen-code/qwen-code-core/models/model-catalog.js';
|
||||||
|
import {
|
||||||
|
MODEL_CATALOG_REFRESH_ENV,
|
||||||
|
MODEL_CATALOG_URL_ENV,
|
||||||
|
} from '@qwen-code/qwen-code-core/models/model-catalog-refresh.js';
|
||||||
import { BWRAP_MIGRATION_MESSAGE } from '../config/execution-sandbox-settings.js';
|
import { BWRAP_MIGRATION_MESSAGE } from '../config/execution-sandbox-settings.js';
|
||||||
import { randomBytes } from 'node:crypto';
|
import { randomBytes } from 'node:crypto';
|
||||||
import { writeStderrLine } from '../utils/stdioHelpers.js';
|
import { writeStderrLine } from '../utils/stdioHelpers.js';
|
||||||
@@ -83,6 +88,9 @@ export function getSandboxPassthroughEnvArgs(
|
|||||||
return [
|
return [
|
||||||
'QWEN_DEBUG_LOG_FILE',
|
'QWEN_DEBUG_LOG_FILE',
|
||||||
'QWEN_CODE_LEGACY_MCP_BLOCKING',
|
'QWEN_CODE_LEGACY_MCP_BLOCKING',
|
||||||
|
MODEL_CATALOG_ENV,
|
||||||
|
MODEL_CATALOG_REFRESH_ENV,
|
||||||
|
MODEL_CATALOG_URL_ENV,
|
||||||
SKIP_UPDATE_CHECK_ENV_VAR,
|
SKIP_UPDATE_CHECK_ENV_VAR,
|
||||||
CUSTOM_SANDBOX_IMAGE_ENV_VAR,
|
CUSTOM_SANDBOX_IMAGE_ENV_VAR,
|
||||||
HOST_UPDATE_RELAUNCH_ENV_VAR,
|
HOST_UPDATE_RELAUNCH_ENV_VAR,
|
||||||
|
|||||||
@@ -19,6 +19,10 @@ if (process.env['QWEN_SERVE_NO_PERSISTENT_REGISTRATION'] === undefined) {
|
|||||||
process.env['QWEN_SERVE_NO_PERSISTENT_REGISTRATION'] = '1';
|
process.env['QWEN_SERVE_NO_PERSISTENT_REGISTRATION'] = '1';
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Model limits and modalities come from the regex tables unless a test opts
|
||||||
|
// into the models.dev catalog.
|
||||||
|
process.env['QWEN_CODE_MODELS_DEV'] = 'off';
|
||||||
|
|
||||||
// The review sandbox policy is the OPERATOR's setting for their own reviews,
|
// The review sandbox policy is the OPERATOR's setting for their own reviews,
|
||||||
// and this suite must not inherit it. A maintainer who turns the feature on
|
// and this suite must not inherit it. A maintainer who turns the feature on
|
||||||
// and then runs `npm test` would otherwise watch the review tests refuse to
|
// and then runs `npm test` would otherwise watch the review tests refuse to
|
||||||
|
|||||||
@@ -12,6 +12,7 @@ import {
|
|||||||
import type { SessionSourceService } from '../services/session-sources.js';
|
import type { SessionSourceService } from '../services/session-sources.js';
|
||||||
|
|
||||||
import { resolveProviderProtocol } from '../models/modelRegistry.js';
|
import { resolveProviderProtocol } from '../models/modelRegistry.js';
|
||||||
|
import { refreshModelCatalog } from '../models/model-catalog-refresh.js';
|
||||||
import {
|
import {
|
||||||
captureReasoningSnapshot,
|
captureReasoningSnapshot,
|
||||||
validateReasoningCapabilities,
|
validateReasoningCapabilities,
|
||||||
@@ -4045,6 +4046,7 @@ export class Config {
|
|||||||
): Promise<void> {
|
): Promise<void> {
|
||||||
this.debugLogger.info('Config initialization started');
|
this.debugLogger.info('Config initialization started');
|
||||||
await this.proxyDispatcherReady;
|
await this.proxyDispatcherReady;
|
||||||
|
void refreshModelCatalog();
|
||||||
options?.signal?.throwIfAborted();
|
options?.signal?.throwIfAborted();
|
||||||
// Omni multimodal support declares ffmpeg/ffprobe as hard runtime
|
// Omni multimodal support declares ffmpeg/ffprobe as hard runtime
|
||||||
// prerequisites: fail fast at startup with an actionable message
|
// prerequisites: fail fast at startup with an actionable message
|
||||||
|
|||||||
@@ -0,0 +1,44 @@
|
|||||||
|
/**
|
||||||
|
* @license
|
||||||
|
* Copyright 2025 Qwen Team
|
||||||
|
* SPDX-License-Identifier: Apache-2.0
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { describe, it, expect, vi } from 'vitest';
|
||||||
|
import { defaultModalities } from './modalityDefaults.js';
|
||||||
|
|
||||||
|
vi.mock('../models/model-catalog.js', () => {
|
||||||
|
const entries: Record<string, { modalities?: Record<string, boolean> }> = {
|
||||||
|
'qwen-catalog-vision': { modalities: { image: true } },
|
||||||
|
'qwen3-vl-catalog': { modalities: { pdf: true } },
|
||||||
|
'catalog-only-model': { modalities: { audio: true } },
|
||||||
|
'catalog-pdf-model': { modalities: { pdf: true } },
|
||||||
|
'catalog-vision-model': { modalities: { image: true } },
|
||||||
|
};
|
||||||
|
return { lookupModelCatalog: (model: string) => entries[model] };
|
||||||
|
});
|
||||||
|
|
||||||
|
describe('models.dev catalog', () => {
|
||||||
|
it('adds catalog modalities to a text-only family', () => {
|
||||||
|
expect(defaultModalities('qwen-catalog-vision')).toEqual({ image: true });
|
||||||
|
});
|
||||||
|
|
||||||
|
it('keeps PDF controlled by an existing family rule', () => {
|
||||||
|
expect(defaultModalities('qwen3-vl-catalog')).toEqual({
|
||||||
|
image: true,
|
||||||
|
video: true,
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
it('uses the catalog alone for a model no family pattern matches', () => {
|
||||||
|
expect(defaultModalities('catalog-only-model')).toEqual({ audio: true });
|
||||||
|
expect(defaultModalities('catalog-pdf-model')).toEqual({});
|
||||||
|
expect(defaultModalities('unknown-model')).toEqual({});
|
||||||
|
});
|
||||||
|
|
||||||
|
it('looks the catalog up by normalized id', () => {
|
||||||
|
expect(defaultModalities('provider/Catalog-Vision-Model:free')).toEqual({
|
||||||
|
image: true,
|
||||||
|
});
|
||||||
|
});
|
||||||
|
});
|
||||||
@@ -4,7 +4,11 @@
|
|||||||
* SPDX-License-Identifier: Apache-2.0
|
* SPDX-License-Identifier: Apache-2.0
|
||||||
*/
|
*/
|
||||||
|
|
||||||
import { describe, it, expect } from 'vitest';
|
import * as fs from 'node:fs';
|
||||||
|
import * as os from 'node:os';
|
||||||
|
import * as path from 'node:path';
|
||||||
|
import { afterAll, beforeAll, describe, it, expect } from 'vitest';
|
||||||
|
import { invalidateModelCatalog } from '../models/model-catalog.js';
|
||||||
import {
|
import {
|
||||||
defaultModalities,
|
defaultModalities,
|
||||||
isQwenFamilyWireModel,
|
isQwenFamilyWireModel,
|
||||||
@@ -50,6 +54,38 @@ type Case = [label: string, check: Check, model?: string];
|
|||||||
const casesFor = (prefix: string, rows: Case[]) =>
|
const casesFor = (prefix: string, rows: Case[]) =>
|
||||||
it.each(rows)(`${prefix} %s`, (label, check, model) => check(model ?? label));
|
it.each(rows)(`${prefix} %s`, (label, check, model) => check(model ?? label));
|
||||||
|
|
||||||
|
// Run against the real bundled catalog (the production default). QWEN_HOME is
|
||||||
|
// pinned to an empty dir so a host's refreshed cache cannot replace the
|
||||||
|
// bundle, and deleting the kill-switch opts back out of the test-setup's
|
||||||
|
// regex-only default.
|
||||||
|
let tempDir: string;
|
||||||
|
let previousHome: string | undefined;
|
||||||
|
let previousSwitch: string | undefined;
|
||||||
|
|
||||||
|
beforeAll(() => {
|
||||||
|
tempDir = fs.mkdtempSync(path.join(os.tmpdir(), 'modality-defaults-'));
|
||||||
|
previousHome = process.env['QWEN_HOME'];
|
||||||
|
previousSwitch = process.env['QWEN_CODE_MODELS_DEV'];
|
||||||
|
process.env['QWEN_HOME'] = path.join(tempDir, '.qwen');
|
||||||
|
delete process.env['QWEN_CODE_MODELS_DEV'];
|
||||||
|
invalidateModelCatalog();
|
||||||
|
});
|
||||||
|
|
||||||
|
afterAll(() => {
|
||||||
|
if (previousHome === undefined) {
|
||||||
|
delete process.env['QWEN_HOME'];
|
||||||
|
} else {
|
||||||
|
process.env['QWEN_HOME'] = previousHome;
|
||||||
|
}
|
||||||
|
if (previousSwitch === undefined) {
|
||||||
|
delete process.env['QWEN_CODE_MODELS_DEV'];
|
||||||
|
} else {
|
||||||
|
process.env['QWEN_CODE_MODELS_DEV'] = previousSwitch;
|
||||||
|
}
|
||||||
|
invalidateModelCatalog();
|
||||||
|
fs.rmSync(tempDir, { recursive: true, force: true });
|
||||||
|
});
|
||||||
|
|
||||||
describe('defaultModalities', () => {
|
describe('defaultModalities', () => {
|
||||||
it('does not infer modalities for an unrecognized batch route', () => {
|
it('does not infer modalities for an unrecognized batch route', () => {
|
||||||
expect(defaultModalities('google/gemini-2.5-flash:batch')).toEqual({});
|
expect(defaultModalities('google/gemini-2.5-flash:batch')).toEqual({});
|
||||||
@@ -105,6 +141,9 @@ describe('defaultModalities', () => {
|
|||||||
['qwen3.8-flash', IMAGE_VIDEO],
|
['qwen3.8-flash', IMAGE_VIDEO],
|
||||||
['qwen3.8-plus', IMAGE_VIDEO],
|
['qwen3.8-plus', IMAGE_VIDEO],
|
||||||
['qwen3.6-35b variants', ONLY_IMAGE_VIDEO, 'qwen3.6-35b-a3b-nvfp4'],
|
['qwen3.6-35b variants', ONLY_IMAGE_VIDEO, 'qwen3.6-35b-a3b-nvfp4'],
|
||||||
|
// The bundled catalog adds video; its pdf stays endpoint-gated by the
|
||||||
|
// lookupModelCatalog correction.
|
||||||
|
['qwen3.8-max', IMAGE_VIDEO],
|
||||||
]);
|
]);
|
||||||
|
|
||||||
casesFor('returns text-only for', [
|
casesFor('returns text-only for', [
|
||||||
@@ -114,7 +153,6 @@ describe('defaultModalities', () => {
|
|||||||
]);
|
]);
|
||||||
|
|
||||||
casesFor('returns image for', [
|
casesFor('returns image for', [
|
||||||
['qwen3.8-max', has({ image: true, video: undefined })],
|
|
||||||
[
|
[
|
||||||
'qwen3.8-max-preview (provider-prefixed)',
|
'qwen3.8-max-preview (provider-prefixed)',
|
||||||
IMAGE,
|
IMAGE,
|
||||||
|
|||||||
@@ -6,6 +6,7 @@
|
|||||||
|
|
||||||
import type { InputModalities } from './contentGenerator.js';
|
import type { InputModalities } from './contentGenerator.js';
|
||||||
import { normalize } from './tokenLimits.js';
|
import { normalize } from './tokenLimits.js';
|
||||||
|
import { lookupModelCatalog } from '../models/model-catalog.js';
|
||||||
import { parseModelReasoningCapabilities } from './reasoning-effort.js';
|
import { parseModelReasoningCapabilities } from './reasoning-effort.js';
|
||||||
|
|
||||||
const FULL_MULTIMODAL: InputModalities = {
|
const FULL_MULTIMODAL: InputModalities = {
|
||||||
@@ -112,18 +113,22 @@ const MODALITY_PATTERNS: Array<[RegExp, InputModalities]> = [
|
|||||||
/**
|
/**
|
||||||
* Return the default input modalities for a model based on its name.
|
* Return the default input modalities for a model based on its name.
|
||||||
*
|
*
|
||||||
* Uses the same normalize-then-regex pattern as {@link tokenLimit}.
|
* Uses the same normalize-then-regex pattern as {@link tokenLimit}, merged
|
||||||
* Unknown models default to text-only (empty object) to avoid sending
|
* with the models.dev catalog entry. PDF stays explicit because it selects a
|
||||||
* unsupported media types that would cause unrecoverable API errors.
|
* different file-reading path; other catalog modalities can extend a known
|
||||||
|
* family. A model neither source knows stays text-only (empty object) to avoid
|
||||||
|
* sending unsupported media types that would cause unrecoverable API errors.
|
||||||
*/
|
*/
|
||||||
export function defaultModalities(model: string): InputModalities {
|
export function defaultModalities(model: string): InputModalities {
|
||||||
const norm = normalize(model);
|
const norm = normalize(model);
|
||||||
|
const fromCatalog = { ...lookupModelCatalog(norm)?.modalities };
|
||||||
|
delete fromCatalog.pdf;
|
||||||
for (const [regex, modalities] of MODALITY_PATTERNS) {
|
for (const [regex, modalities] of MODALITY_PATTERNS) {
|
||||||
if (regex.test(norm)) {
|
if (regex.test(norm)) {
|
||||||
return { ...modalities };
|
return { ...fromCatalog, ...modalities };
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
return {};
|
return fromCatalog;
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
|
|||||||
@@ -1,8 +1,9 @@
|
|||||||
import { describe, it, expect } from 'vitest';
|
import { describe, it, expect, vi } from 'vitest';
|
||||||
import {
|
import {
|
||||||
normalize,
|
normalize,
|
||||||
tokenLimit,
|
tokenLimit,
|
||||||
knownTokenLimit,
|
knownTokenLimit,
|
||||||
|
hasExplicitOutputLimit,
|
||||||
clampOutputTokensToWindow,
|
clampOutputTokensToWindow,
|
||||||
outputClampMargin,
|
outputClampMargin,
|
||||||
defaultOutputCeiling,
|
defaultOutputCeiling,
|
||||||
@@ -14,6 +15,15 @@ import {
|
|||||||
OUTPUT_TOKEN_CEILING,
|
OUTPUT_TOKEN_CEILING,
|
||||||
} from './tokenLimits.js';
|
} from './tokenLimits.js';
|
||||||
|
|
||||||
|
vi.mock('../models/model-catalog.js', () => {
|
||||||
|
const entries: Record<string, { context?: number; output?: number }> = {
|
||||||
|
'catalog-model': { context: 123_456, output: 7_890 },
|
||||||
|
'qwen-catalog-context-only': { context: 50_000 },
|
||||||
|
'qwen-catalog-tiny': { context: 5 },
|
||||||
|
};
|
||||||
|
return { lookupModelCatalog: (model: string) => entries[model] };
|
||||||
|
});
|
||||||
|
|
||||||
describe('normalize', () => {
|
describe('normalize', () => {
|
||||||
it('keeps unrecognized batch routing tags out of token lookup', () => {
|
it('keeps unrecognized batch routing tags out of token lookup', () => {
|
||||||
expect(normalize('google/gemini-2.5-flash:batch')).toBe('batch');
|
expect(normalize('google/gemini-2.5-flash:batch')).toBe('batch');
|
||||||
@@ -649,3 +659,32 @@ describe('reconcileMaxTokens', () => {
|
|||||||
expect(reconcileMaxTokens(null, null)).toBeUndefined();
|
expect(reconcileMaxTokens(null, null)).toBeUndefined();
|
||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
|
describe('models.dev catalog', () => {
|
||||||
|
it('takes limits from the catalog before the regex tables', () => {
|
||||||
|
expect(tokenLimit('catalog-model', 'input')).toBe(123_456);
|
||||||
|
expect(tokenLimit('catalog-model', 'output')).toBe(7_890);
|
||||||
|
expect(knownTokenLimit('catalog-model', 'output')).toBe(7_890);
|
||||||
|
expect(hasExplicitOutputLimit('catalog-model')).toBe(false);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('falls back per field when the catalog entry is partial', () => {
|
||||||
|
expect(tokenLimit('qwen-catalog-context-only', 'input')).toBe(50_000);
|
||||||
|
expect(tokenLimit('qwen-catalog-context-only', 'output')).toBe(32_768);
|
||||||
|
expect(hasExplicitOutputLimit('qwen-catalog-context-only')).toBe(true);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('ignores catalog windows too small for the agent loop', () => {
|
||||||
|
expect(tokenLimit('qwen-catalog-tiny', 'input')).toBe(262_144);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('looks the catalog up by normalized id', () => {
|
||||||
|
expect(tokenLimit('provider/Catalog-Model:free', 'input')).toBe(123_456);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('keeps the regex tables and defaults for models the catalog lacks', () => {
|
||||||
|
expect(tokenLimit('gpt-5', 'input')).toBe(272_000);
|
||||||
|
expect(tokenLimit('unknown-model', 'input')).toBe(DEFAULT_TOKEN_LIMIT);
|
||||||
|
expect(hasExplicitOutputLimit('unknown-model')).toBe(false);
|
||||||
|
});
|
||||||
|
});
|
||||||
@@ -1,3 +1,5 @@
|
|||||||
|
import { lookupModelCatalog } from '../models/model-catalog.js';
|
||||||
|
|
||||||
type Model = string;
|
type Model = string;
|
||||||
type TokenCount = number;
|
type TokenCount = number;
|
||||||
|
|
||||||
@@ -11,6 +13,11 @@ export type TokenLimitType = 'input' | 'output';
|
|||||||
export const DEFAULT_TOKEN_LIMIT: TokenCount = 200_000; // 200K tokens
|
export const DEFAULT_TOKEN_LIMIT: TokenCount = 200_000; // 200K tokens
|
||||||
export const DEFAULT_OUTPUT_TOKEN_LIMIT: TokenCount = 32_000; // 32K tokens
|
export const DEFAULT_OUTPUT_TOKEN_LIMIT: TokenCount = 32_000; // 32K tokens
|
||||||
|
|
||||||
|
// Below this window, the 85% auto-compaction threshold can leave less than
|
||||||
|
// the 4K minimum output budget. Keep those catalog values from becoming
|
||||||
|
// automatic defaults; existing family tables remain the fallback.
|
||||||
|
export const MIN_AUTO_DETECTED_CONTEXT_WINDOW: TokenCount = 32_000;
|
||||||
|
|
||||||
export const ESCALATED_MAX_TOKENS: TokenCount = 64_000;
|
export const ESCALATED_MAX_TOKENS: TokenCount = 64_000;
|
||||||
|
|
||||||
/**
|
/**
|
||||||
@@ -329,6 +336,15 @@ function findTokenLimit(
|
|||||||
type: TokenLimitType = 'input',
|
type: TokenLimitType = 'input',
|
||||||
): TokenCount | undefined {
|
): TokenCount | undefined {
|
||||||
const norm = normalize(model);
|
const norm = normalize(model);
|
||||||
|
const catalog = lookupModelCatalog(norm);
|
||||||
|
const fromCatalog = type === 'output' ? catalog?.output : catalog?.context;
|
||||||
|
const usableCatalogContext =
|
||||||
|
type === 'input' &&
|
||||||
|
fromCatalog !== undefined &&
|
||||||
|
fromCatalog >= MIN_AUTO_DETECTED_CONTEXT_WINDOW;
|
||||||
|
if (usableCatalogContext) {
|
||||||
|
return fromCatalog;
|
||||||
|
}
|
||||||
const patterns = type === 'output' ? OUTPUT_PATTERNS : PATTERNS;
|
const patterns = type === 'output' ? OUTPUT_PATTERNS : PATTERNS;
|
||||||
|
|
||||||
for (const [regex, limit] of patterns) {
|
for (const [regex, limit] of patterns) {
|
||||||
@@ -337,16 +353,12 @@ function findTokenLimit(
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
return undefined;
|
return type === 'input' ? undefined : fromCatalog;
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Check if a model has an explicitly defined output token limit.
|
* Whether a curated output limit caps explicit requests. Catalog-only limits
|
||||||
* This distinguishes between models with known limits in OUTPUT_PATTERNS
|
* supply defaults, but must not clamp a user's endpoint-specific override.
|
||||||
* and unknown models that would fallback to DEFAULT_OUTPUT_TOKEN_LIMIT.
|
|
||||||
*
|
|
||||||
* @param model - The model name to check
|
|
||||||
* @returns true if the model has an explicit output limit definition, false if it uses the default fallback
|
|
||||||
*/
|
*/
|
||||||
export function hasExplicitOutputLimit(model: Model): boolean {
|
export function hasExplicitOutputLimit(model: Model): boolean {
|
||||||
const norm = normalize(model);
|
const norm = normalize(model);
|
||||||
|
|||||||
File diff suppressed because it is too large.
Load diff
@@ -0,0 +1,783 @@
|
|||||||
|
/**
|
||||||
|
* @license
|
||||||
|
* Copyright 2025 Qwen Team
|
||||||
|
* SPDX-License-Identifier: Apache-2.0
|
||||||
|
*/
|
||||||
|
|
||||||
|
import * as fs from 'node:fs';
|
||||||
|
import * as os from 'node:os';
|
||||||
|
import * as path from 'node:path';
|
||||||
|
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
|
||||||
|
import { defaultOutputCeiling, tokenLimit } from '../core/tokenLimits.js';
|
||||||
|
import bundled from './generated/model-registry.json' with { type: 'json' };
|
||||||
|
import {
|
||||||
|
getModelCatalogCachePath,
|
||||||
|
invalidateModelCatalog,
|
||||||
|
loadModelCatalog,
|
||||||
|
lookupModelCatalog,
|
||||||
|
MODEL_CATALOG_PROJECTION_VERSION,
|
||||||
|
} from './model-catalog.js';
|
||||||
|
import {
|
||||||
|
MODELS_DEV_PROVIDERS,
|
||||||
|
MODELS_DEV_URL,
|
||||||
|
refreshModelCatalog,
|
||||||
|
trimModelsDevCatalog,
|
||||||
|
type ModelsDevApi,
|
||||||
|
} from './model-catalog-refresh.js';
|
||||||
|
|
||||||
|
const NOW = '2026-09-16T00:00:00.000Z';
|
||||||
|
const LONG_AGO = '2000-01-01T00:00:00.000Z';
|
||||||
|
|
||||||
|
const chat = (
|
||||||
|
id: string,
|
||||||
|
limit: { context?: number; input?: number; output?: number },
|
||||||
|
input: string[] = ['text'],
|
||||||
|
) => ({
|
||||||
|
id,
|
||||||
|
tool_call: true,
|
||||||
|
limit,
|
||||||
|
modalities: { input, output: ['text'] },
|
||||||
|
});
|
||||||
|
|
||||||
|
const providerStubs = Object.fromEntries(
|
||||||
|
MODELS_DEV_PROVIDERS.map((provider) => [
|
||||||
|
provider,
|
||||||
|
{
|
||||||
|
models: {
|
||||||
|
placeholder: {
|
||||||
|
id: `${provider}-placeholder`,
|
||||||
|
tool_call: false,
|
||||||
|
modalities: { input: ['text'], output: ['text'] },
|
||||||
|
},
|
||||||
|
},
|
||||||
|
},
|
||||||
|
]),
|
||||||
|
) as ModelsDevApi;
|
||||||
|
|
||||||
|
const api: ModelsDevApi = {
|
||||||
|
...providerStubs,
|
||||||
|
anthropic: {
|
||||||
|
models: {
|
||||||
|
'claude-x': chat('claude-x', { context: 200000, output: 64000 }, [
|
||||||
|
'text',
|
||||||
|
'image',
|
||||||
|
'pdf',
|
||||||
|
]),
|
||||||
|
},
|
||||||
|
},
|
||||||
|
openai: {
|
||||||
|
models: {
|
||||||
|
'gpt-x': chat(
|
||||||
|
'gpt-x',
|
||||||
|
{ context: 400000, input: 272000, output: 128000 },
|
||||||
|
['text', 'image'],
|
||||||
|
),
|
||||||
|
// Not an agent model: no tool calling.
|
||||||
|
'embed-x': {
|
||||||
|
id: 'embed-x',
|
||||||
|
tool_call: false,
|
||||||
|
limit: { context: 8192, output: 1 },
|
||||||
|
modalities: { input: ['text'], output: ['text'] },
|
||||||
|
},
|
||||||
|
// Not an agent model: does not answer in text.
|
||||||
|
'tts-x': {
|
||||||
|
id: 'tts-x',
|
||||||
|
tool_call: true,
|
||||||
|
limit: { context: 8192, output: 16384 },
|
||||||
|
modalities: { input: ['text'], output: ['audio'] },
|
||||||
|
},
|
||||||
|
},
|
||||||
|
},
|
||||||
|
zai: {
|
||||||
|
models: {
|
||||||
|
// Same id as the alibaba-cn entry below, different serving limits.
|
||||||
|
'glm-x': chat('glm-x', { context: 204800, output: 131072 }),
|
||||||
|
},
|
||||||
|
},
|
||||||
|
'alibaba-cn': {
|
||||||
|
models: {
|
||||||
|
'qwen-x': chat('qwen-x', { context: 1000000, output: 65536 }, [
|
||||||
|
'text',
|
||||||
|
'image',
|
||||||
|
'video',
|
||||||
|
]),
|
||||||
|
'glm-x': chat('glm-x', { context: 202752, output: 16384 }),
|
||||||
|
// Both normalize onto `foo`, neither verbatim, and they disagree.
|
||||||
|
'foo-v3': chat('foo-v3', { context: 3000 }),
|
||||||
|
'foo-v4': chat('foo-v4', { context: 4000 }),
|
||||||
|
// Invalid limits must not become catalog defaults.
|
||||||
|
'nothing-known': chat('nothing-known', { context: -1, output: 0 }),
|
||||||
|
'tiny-context': chat('tiny-context', {
|
||||||
|
context: 8_192,
|
||||||
|
output: 4_096,
|
||||||
|
}),
|
||||||
|
},
|
||||||
|
},
|
||||||
|
alibaba: {
|
||||||
|
models: {
|
||||||
|
'qwen-x': chat('qwen-x', { context: 1000000, output: 65536 }, [
|
||||||
|
'text',
|
||||||
|
'image',
|
||||||
|
'video',
|
||||||
|
]),
|
||||||
|
},
|
||||||
|
},
|
||||||
|
openrouter: {
|
||||||
|
models: {
|
||||||
|
'router-only': chat('router-only', { context: 9, output: 9 }),
|
||||||
|
},
|
||||||
|
},
|
||||||
|
thinkingmachines: {
|
||||||
|
models: {
|
||||||
|
// First-party vendor outside the allowlist (issue #8558): its
|
||||||
|
// modalities project, its endpoint limits do not.
|
||||||
|
inkling: chat('inkling', { context: 65536, output: 65536 }, [
|
||||||
|
'text',
|
||||||
|
'image',
|
||||||
|
]),
|
||||||
|
},
|
||||||
|
},
|
||||||
|
};
|
||||||
|
|
||||||
|
const trimmed = {
|
||||||
|
'claude-x': {
|
||||||
|
context: 200000,
|
||||||
|
output: 64000,
|
||||||
|
modalities: { image: true, pdf: true },
|
||||||
|
},
|
||||||
|
'gpt-x': { context: 272000, output: 128000, modalities: { image: true } },
|
||||||
|
inkling: { modalities: { image: true } },
|
||||||
|
// Text-only declaration survives as an empty modalities record on an entry
|
||||||
|
// that ships limits: fidelity a future narrowing can tell from "unknown".
|
||||||
|
'tiny-context': { output: 4_096, modalities: {} },
|
||||||
|
'qwen-x': {
|
||||||
|
context: 1000000,
|
||||||
|
output: 65536,
|
||||||
|
modalities: { image: true, video: true },
|
||||||
|
},
|
||||||
|
};
|
||||||
|
|
||||||
|
const ENV_NAMES = [
|
||||||
|
'QWEN_HOME',
|
||||||
|
'QWEN_CODE_MODELS_DEV',
|
||||||
|
'QWEN_CODE_MODELS_DEV_REFRESH',
|
||||||
|
'QWEN_CODE_MODELS_DEV_URL',
|
||||||
|
];
|
||||||
|
|
||||||
|
describe('trimModelsDevCatalog', () => {
|
||||||
|
it('keeps limits from allowlisted providers and unions modalities across all of them, keyed by normalized id', () => {
|
||||||
|
expect(trimModelsDevCatalog(api, NOW)).toEqual({
|
||||||
|
source: MODELS_DEV_URL,
|
||||||
|
fetchedAt: NOW,
|
||||||
|
projection: MODEL_CATALOG_PROJECTION_VERSION,
|
||||||
|
models: trimmed,
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
it('ignores pure-mirror providers so a hub cannot veto first-party numbers', () => {
|
||||||
|
// modelscope republishes GLM under a vendor-qualified id with different
|
||||||
|
// serving limits; it is not in MODELS_DEV_PROVIDERS, so the first-party
|
||||||
|
// z.ai entry wins outright instead of the key being dropped for
|
||||||
|
// disagreement. A dated alias inside one first-party provider still
|
||||||
|
// conflicts and drops.
|
||||||
|
const models = trimModelsDevCatalog(
|
||||||
|
{
|
||||||
|
zai: { models: { 'glm-4.6': chat('glm-4.6', { output: 131072 }) } },
|
||||||
|
modelscope: {
|
||||||
|
models: {
|
||||||
|
'ZhipuAI/GLM-4.6': chat('ZhipuAI/GLM-4.6', { output: 98304 }),
|
||||||
|
},
|
||||||
|
},
|
||||||
|
deepseek: {
|
||||||
|
models: {
|
||||||
|
'deepseek-r1': chat('deepseek-r1', { output: 16384 }),
|
||||||
|
'deepseek-r1-0528': chat('deepseek-r1-0528', { output: 32768 }),
|
||||||
|
},
|
||||||
|
},
|
||||||
|
},
|
||||||
|
NOW,
|
||||||
|
).models;
|
||||||
|
expect(models['glm-4.6']).toEqual({ output: 131072, modalities: {} });
|
||||||
|
expect(models).not.toHaveProperty('deepseek-r1');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('does not widen non-vision DeepSeek models', () => {
|
||||||
|
const models = trimModelsDevCatalog(
|
||||||
|
{
|
||||||
|
deepseek: {
|
||||||
|
models: {
|
||||||
|
bare: chat('deepseek-v4-flash', {}, ['text', 'image']),
|
||||||
|
next: chat('deepseek-v4.1-flash', {}, ['text', 'image']),
|
||||||
|
vision: chat('deepseek-v4-flash-vision-exp', {}, ['text', 'image']),
|
||||||
|
// normalize('deepseek-v3') === 'deepseek': the bare-model record
|
||||||
|
// the #10270 narrowing exists for collapses onto the family root,
|
||||||
|
// which passes isModelCatalogKey — the guard must key on the
|
||||||
|
// family, not the hyphenated prefix.
|
||||||
|
root: chat('deepseek-v3', {}, ['text', 'image']),
|
||||||
|
},
|
||||||
|
},
|
||||||
|
},
|
||||||
|
NOW,
|
||||||
|
).models;
|
||||||
|
|
||||||
|
expect(models['deepseek-v4-flash']).toBeUndefined();
|
||||||
|
expect(models['deepseek-v4.1-flash']).toBeUndefined();
|
||||||
|
expect(models['deepseek']?.modalities?.image).toBeUndefined();
|
||||||
|
expect(models['deepseek-v4-flash-vision-exp']).toEqual({
|
||||||
|
modalities: { image: true },
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
it('drops an id whose providers disagree instead of picking one', () => {
|
||||||
|
// glm-x is served by zai and alibaba-cn with different output limits, so
|
||||||
|
// the regex tables keep their current answer.
|
||||||
|
expect(trimModelsDevCatalog(api, NOW).models).not.toHaveProperty('glm-x');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('drops an id whose aliases disagree and none is verbatim', () => {
|
||||||
|
expect(trimModelsDevCatalog(api, NOW).models).not.toHaveProperty('foo');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('keeps an id two providers agree on', () => {
|
||||||
|
// qwen-x is listed identically by alibaba-cn and alibaba.
|
||||||
|
expect(trimModelsDevCatalog(api, NOW).models['qwen-x']).toBeDefined();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('drops models that cannot drive an agent turn', () => {
|
||||||
|
const models = trimModelsDevCatalog(api, NOW).models;
|
||||||
|
expect(models).not.toHaveProperty('embed-x');
|
||||||
|
expect(models).not.toHaveProperty('tts-x');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('drops a model with no usable field and providers outside the list', () => {
|
||||||
|
const models = trimModelsDevCatalog(api, NOW).models;
|
||||||
|
expect(models).not.toHaveProperty('nothing-known');
|
||||||
|
expect(models).not.toHaveProperty('router-only');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('projects modalities, but not limits, for a first-party vendor outside the allowlist', () => {
|
||||||
|
// issue #8558: thinkingmachines/inkling's image support was invisible
|
||||||
|
// because the allowlist gated the whole entry.
|
||||||
|
expect(trimModelsDevCatalog(api, NOW).models['inkling']).toEqual({
|
||||||
|
modalities: { image: true },
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
it('ignores modality claims from untrusted routers', () => {
|
||||||
|
const models = trimModelsDevCatalog(
|
||||||
|
{
|
||||||
|
zai: {
|
||||||
|
models: {
|
||||||
|
'glm-x': chat('glm-x', { context: 204800 }, ['text', 'image']),
|
||||||
|
},
|
||||||
|
},
|
||||||
|
openrouter: {
|
||||||
|
models: { 'glm-x': chat('glm-x', { context: 999 }, ['text', 'pdf']) },
|
||||||
|
},
|
||||||
|
},
|
||||||
|
NOW,
|
||||||
|
).models;
|
||||||
|
expect(models['glm-x']).toEqual({
|
||||||
|
context: 204800,
|
||||||
|
modalities: { image: true },
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
it('records the source it was trimmed from', () => {
|
||||||
|
expect(trimModelsDevCatalog({}, NOW, 'https://mirror/api.json')).toEqual({
|
||||||
|
source: 'https://mirror/api.json',
|
||||||
|
fetchedAt: NOW,
|
||||||
|
projection: MODEL_CATALOG_PROJECTION_VERSION,
|
||||||
|
models: {},
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
it('omits a key that is unreachable by its own normalized spelling', () => {
|
||||||
|
// normalize('acme-v3-0324') -> 'acme-v3', which itself normalizes to
|
||||||
|
// 'acme': the entry could never be looked up by the id it describes.
|
||||||
|
const models = trimModelsDevCatalog(
|
||||||
|
{
|
||||||
|
alibaba: {
|
||||||
|
models: {
|
||||||
|
'acme-v3-0324': chat('acme-v3-0324', {
|
||||||
|
context: 163840,
|
||||||
|
output: 163840,
|
||||||
|
}),
|
||||||
|
},
|
||||||
|
},
|
||||||
|
},
|
||||||
|
NOW,
|
||||||
|
).models;
|
||||||
|
expect(models).not.toHaveProperty('acme-v3');
|
||||||
|
expect(models).toEqual({});
|
||||||
|
});
|
||||||
|
|
||||||
|
it('omits lossy and generic normalized keys', () => {
|
||||||
|
const models = trimModelsDevCatalog(
|
||||||
|
{
|
||||||
|
anthropic: {
|
||||||
|
models: {
|
||||||
|
numeric: chat('claude-sonnet-4:thinking:32768', {
|
||||||
|
context: 200000,
|
||||||
|
}),
|
||||||
|
tagged: chat('claude-sonnet-4@default', { context: 200000 }),
|
||||||
|
generic: chat('anthropic/auto', { context: 200000 }),
|
||||||
|
},
|
||||||
|
},
|
||||||
|
},
|
||||||
|
NOW,
|
||||||
|
).models;
|
||||||
|
expect(models).not.toHaveProperty('32768');
|
||||||
|
expect(models).not.toHaveProperty('claude-sonnet-4@default');
|
||||||
|
expect(models).not.toHaveProperty('auto');
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe('refreshModelCatalog', () => {
|
||||||
|
let tempDir: string;
|
||||||
|
let previousEnv: Record<string, string | undefined>;
|
||||||
|
const fetchMock = vi.fn<typeof fetch>();
|
||||||
|
|
||||||
|
function jsonResponse(
|
||||||
|
body: unknown,
|
||||||
|
headers: Record<string, string> = {},
|
||||||
|
): Response {
|
||||||
|
return new Response(JSON.stringify(body), {
|
||||||
|
status: 200,
|
||||||
|
headers: { 'content-type': 'application/json', ...headers },
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
function readJson(filePath: string): {
|
||||||
|
source: string;
|
||||||
|
fetchedAt: string;
|
||||||
|
projection: number;
|
||||||
|
etag?: string;
|
||||||
|
models: unknown;
|
||||||
|
} {
|
||||||
|
return JSON.parse(fs.readFileSync(filePath, 'utf8'));
|
||||||
|
}
|
||||||
|
|
||||||
|
function writeJson(filePath: string, content: unknown): void {
|
||||||
|
fs.mkdirSync(path.dirname(filePath), { recursive: true });
|
||||||
|
fs.writeFileSync(filePath, JSON.stringify(content));
|
||||||
|
}
|
||||||
|
|
||||||
|
beforeEach(() => {
|
||||||
|
tempDir = fs.mkdtempSync(path.join(os.tmpdir(), 'model-catalog-'));
|
||||||
|
previousEnv = Object.fromEntries(
|
||||||
|
ENV_NAMES.map((name) => [name, process.env[name]]),
|
||||||
|
);
|
||||||
|
for (const name of ENV_NAMES) {
|
||||||
|
delete process.env[name];
|
||||||
|
}
|
||||||
|
process.env['QWEN_HOME'] = path.join(tempDir, '.qwen');
|
||||||
|
fetchMock.mockReset();
|
||||||
|
vi.stubGlobal('fetch', fetchMock);
|
||||||
|
invalidateModelCatalog();
|
||||||
|
});
|
||||||
|
|
||||||
|
afterEach(() => {
|
||||||
|
vi.useRealTimers();
|
||||||
|
vi.unstubAllGlobals();
|
||||||
|
for (const name of ENV_NAMES) {
|
||||||
|
const value = previousEnv[name];
|
||||||
|
if (value === undefined) {
|
||||||
|
delete process.env[name];
|
||||||
|
} else {
|
||||||
|
process.env[name] = value;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
invalidateModelCatalog();
|
||||||
|
fs.rmSync(tempDir, { recursive: true, force: true });
|
||||||
|
});
|
||||||
|
|
||||||
|
it('writes the trimmed catalog to the cache', async () => {
|
||||||
|
// Pin the clock just ahead of the bundled snapshot's committed stamp: the
|
||||||
|
// fresh cache must win loadModelCatalog's `fetchedAt` compare no matter
|
||||||
|
// what the runner's wall clock says about a file generated on another host.
|
||||||
|
vi.useFakeTimers();
|
||||||
|
vi.setSystemTime(Date.parse(bundled.fetchedAt) + 60_000);
|
||||||
|
fetchMock.mockResolvedValue(jsonResponse(api, { etag: '"abc"' }));
|
||||||
|
|
||||||
|
await refreshModelCatalog();
|
||||||
|
|
||||||
|
expect(fetchMock).toHaveBeenCalledTimes(1);
|
||||||
|
expect(fetchMock.mock.calls[0]?.[0]).toBe(MODELS_DEV_URL);
|
||||||
|
const cache = readJson(getModelCatalogCachePath());
|
||||||
|
expect(cache.source).toBe(MODELS_DEV_URL);
|
||||||
|
expect(cache.etag).toBe('"abc"');
|
||||||
|
expect(cache.models).toEqual(trimmed);
|
||||||
|
expect(lookupModelCatalog('qwen-x')).toEqual(trimmed['qwen-x']);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('keeps the current context and output defaults until the next process', async () => {
|
||||||
|
vi.setSystemTime(new Date(Date.parse(bundled.fetchedAt) + 60_000));
|
||||||
|
const model = 'seed-oss-catalog-refresh-check';
|
||||||
|
expect(lookupModelCatalog(model)).toBeUndefined();
|
||||||
|
const contextBefore = tokenLimit(model);
|
||||||
|
const outputBefore = defaultOutputCeiling(model);
|
||||||
|
const updated = chat(model, { context: 131_072, output: 65_536 });
|
||||||
|
fetchMock.mockResolvedValue(
|
||||||
|
jsonResponse({
|
||||||
|
...api,
|
||||||
|
volcengine: { models: { [model]: updated } },
|
||||||
|
}),
|
||||||
|
);
|
||||||
|
|
||||||
|
await refreshModelCatalog();
|
||||||
|
|
||||||
|
expect(readJson(getModelCatalogCachePath()).models).toHaveProperty(model, {
|
||||||
|
context: 131_072,
|
||||||
|
output: 65_536,
|
||||||
|
modalities: {},
|
||||||
|
});
|
||||||
|
expect(tokenLimit(model)).toBe(contextBefore);
|
||||||
|
expect(defaultOutputCeiling(model)).toBe(outputBefore);
|
||||||
|
|
||||||
|
// A new process has no memoized catalog.
|
||||||
|
invalidateModelCatalog();
|
||||||
|
expect(tokenLimit(model)).toBe(131_072);
|
||||||
|
expect(defaultOutputCeiling(model)).toBe(64_000);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('refetches a cache whose stamp is ahead of the local clock', async () => {
|
||||||
|
// A stamp written by a faster clock must not serve indefinitely: the age
|
||||||
|
// is negative, which the freshness check treats as "refetch", and the
|
||||||
|
// re-stamp uses the local clock so the skew heals on this fetch.
|
||||||
|
writeJson(getModelCatalogCachePath(), {
|
||||||
|
source: MODELS_DEV_URL,
|
||||||
|
fetchedAt: new Date(Date.now() + 60_000).toISOString(),
|
||||||
|
projection: MODEL_CATALOG_PROJECTION_VERSION,
|
||||||
|
models: { 'kept-model': { context: 7 } },
|
||||||
|
});
|
||||||
|
fetchMock.mockResolvedValue(jsonResponse(api));
|
||||||
|
|
||||||
|
await refreshModelCatalog();
|
||||||
|
|
||||||
|
expect(fetchMock).toHaveBeenCalledTimes(1);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('refreshes a complete multi-megabyte upstream response', async () => {
|
||||||
|
fetchMock.mockResolvedValue(
|
||||||
|
jsonResponse({
|
||||||
|
...api,
|
||||||
|
pad: {
|
||||||
|
models: {
|
||||||
|
big: {
|
||||||
|
id: 'big',
|
||||||
|
tool_call: false,
|
||||||
|
note: 'x'.repeat(5 * 1024 * 1024),
|
||||||
|
},
|
||||||
|
},
|
||||||
|
},
|
||||||
|
}),
|
||||||
|
);
|
||||||
|
|
||||||
|
await refreshModelCatalog();
|
||||||
|
|
||||||
|
expect(readJson(getModelCatalogCachePath()).models).toEqual(trimmed);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('keeps the cache when a catalog body exceeds the download budget', async () => {
|
||||||
|
const previous = {
|
||||||
|
source: MODELS_DEV_URL,
|
||||||
|
fetchedAt: LONG_AGO,
|
||||||
|
projection: MODEL_CATALOG_PROJECTION_VERSION,
|
||||||
|
models: { 'kept-model': { context: 128_000 } },
|
||||||
|
};
|
||||||
|
writeJson(getModelCatalogCachePath(), previous);
|
||||||
|
// Valid JSON that would project fine, but larger than the budget: the
|
||||||
|
// refresh must refuse it before buffering unboundedly.
|
||||||
|
const oversized = {
|
||||||
|
...api,
|
||||||
|
pad: {
|
||||||
|
models: {
|
||||||
|
big: {
|
||||||
|
id: 'big',
|
||||||
|
tool_call: false,
|
||||||
|
note: 'x'.repeat(17 * 1024 * 1024),
|
||||||
|
},
|
||||||
|
},
|
||||||
|
},
|
||||||
|
} as ModelsDevApi;
|
||||||
|
fetchMock.mockResolvedValue(jsonResponse(oversized));
|
||||||
|
|
||||||
|
await expect(refreshModelCatalog()).resolves.toBeUndefined();
|
||||||
|
|
||||||
|
expect(readJson(getModelCatalogCachePath())).toEqual(previous);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('rejects a payload missing an allowlisted provider and keeps the cache', async () => {
|
||||||
|
const previous = {
|
||||||
|
source: MODELS_DEV_URL,
|
||||||
|
fetchedAt: LONG_AGO,
|
||||||
|
projection: MODEL_CATALOG_PROJECTION_VERSION,
|
||||||
|
models: { 'kept-model': { context: 128_000 } },
|
||||||
|
};
|
||||||
|
writeJson(getModelCatalogCachePath(), previous);
|
||||||
|
// A truncated download or a partial mirror: valid JSON that projects
|
||||||
|
// fine, but every allowlisted provider except one is absent. The refresh
|
||||||
|
// must treat it as fatal rather than shadow the bundled snapshot with a
|
||||||
|
// partial catalog for a day.
|
||||||
|
fetchMock.mockResolvedValue(
|
||||||
|
jsonResponse({
|
||||||
|
anthropic: api['anthropic'],
|
||||||
|
}),
|
||||||
|
);
|
||||||
|
|
||||||
|
await expect(refreshModelCatalog()).resolves.toBeUndefined();
|
||||||
|
|
||||||
|
expect(readJson(getModelCatalogCachePath())).toEqual(previous);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('rejects a present-but-non-JSON content type', async () => {
|
||||||
|
fetchMock.mockResolvedValue(
|
||||||
|
new Response(JSON.stringify(api), {
|
||||||
|
status: 200,
|
||||||
|
headers: { 'content-type': 'text/html' },
|
||||||
|
}),
|
||||||
|
);
|
||||||
|
|
||||||
|
await expect(refreshModelCatalog()).resolves.toBeUndefined();
|
||||||
|
|
||||||
|
expect(fs.existsSync(getModelCatalogCachePath())).toBe(false);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('skips the network while the cache is fresh', async () => {
|
||||||
|
writeJson(getModelCatalogCachePath(), {
|
||||||
|
source: MODELS_DEV_URL,
|
||||||
|
fetchedAt: new Date().toISOString(),
|
||||||
|
projection: MODEL_CATALOG_PROJECTION_VERSION,
|
||||||
|
models: { 'kept-model': { context: 7 } },
|
||||||
|
});
|
||||||
|
|
||||||
|
await refreshModelCatalog();
|
||||||
|
|
||||||
|
expect(fetchMock).not.toHaveBeenCalled();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('re-fetches a fresh cache that came from a different URL', async () => {
|
||||||
|
writeJson(getModelCatalogCachePath(), {
|
||||||
|
source: 'https://old-mirror/api.json',
|
||||||
|
fetchedAt: new Date().toISOString(),
|
||||||
|
projection: MODEL_CATALOG_PROJECTION_VERSION,
|
||||||
|
etag: '"old"',
|
||||||
|
// Non-empty, so only the source mismatch can force the re-fetch.
|
||||||
|
models: { 'kept-model': { context: 7 } },
|
||||||
|
});
|
||||||
|
fetchMock.mockResolvedValue(jsonResponse(api));
|
||||||
|
|
||||||
|
await refreshModelCatalog();
|
||||||
|
|
||||||
|
expect(fetchMock).toHaveBeenCalledTimes(1);
|
||||||
|
expect(fetchMock.mock.calls[0]?.[1]?.headers).toEqual({});
|
||||||
|
expect(readJson(getModelCatalogCachePath()).source).toBe(MODELS_DEV_URL);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('revalidates a stale cache with its ETag and keeps the models on 304', async () => {
|
||||||
|
writeJson(getModelCatalogCachePath(), {
|
||||||
|
source: MODELS_DEV_URL,
|
||||||
|
fetchedAt: LONG_AGO,
|
||||||
|
projection: MODEL_CATALOG_PROJECTION_VERSION,
|
||||||
|
etag: '"abc"',
|
||||||
|
models: { 'kept-model': { context: 7 } },
|
||||||
|
});
|
||||||
|
fetchMock.mockResolvedValue(new Response(null, { status: 304 }));
|
||||||
|
|
||||||
|
await refreshModelCatalog();
|
||||||
|
|
||||||
|
expect(fetchMock.mock.calls[0]?.[1]?.headers).toEqual({
|
||||||
|
'If-None-Match': '"abc"',
|
||||||
|
});
|
||||||
|
const cache = readJson(getModelCatalogCachePath());
|
||||||
|
expect(cache.models).toEqual({ 'kept-model': { context: 7 } });
|
||||||
|
expect(cache.etag).toBe('"abc"');
|
||||||
|
expect(Date.parse(cache.fetchedAt)).toBeGreaterThan(Date.parse(LONG_AGO));
|
||||||
|
});
|
||||||
|
|
||||||
|
it.each([undefined, 0, 2])(
|
||||||
|
'does not revalidate an older projection on 304 (%s)',
|
||||||
|
async (projection) => {
|
||||||
|
writeJson(getModelCatalogCachePath(), {
|
||||||
|
source: MODELS_DEV_URL,
|
||||||
|
fetchedAt: LONG_AGO,
|
||||||
|
projection,
|
||||||
|
etag: '"old"',
|
||||||
|
models: { 'deepseek-v4-flash': { modalities: { image: true } } },
|
||||||
|
});
|
||||||
|
fetchMock.mockResolvedValue(new Response(null, { status: 304 }));
|
||||||
|
|
||||||
|
await refreshModelCatalog();
|
||||||
|
|
||||||
|
expect(fetchMock.mock.calls[0]?.[1]?.headers).toEqual({});
|
||||||
|
expect(readJson(getModelCatalogCachePath()).fetchedAt).toBe(LONG_AGO);
|
||||||
|
invalidateModelCatalog();
|
||||||
|
expect(loadModelCatalog().fetchedAt).toBe(bundled.fetchedAt);
|
||||||
|
expect(lookupModelCatalog('deepseek-v4-flash')).toBeUndefined();
|
||||||
|
},
|
||||||
|
);
|
||||||
|
|
||||||
|
it('reprojects an old cache without sending its ETag', async () => {
|
||||||
|
writeJson(getModelCatalogCachePath(), {
|
||||||
|
source: MODELS_DEV_URL,
|
||||||
|
fetchedAt: new Date().toISOString(),
|
||||||
|
projection: 0,
|
||||||
|
etag: '"old"',
|
||||||
|
models: { 'deepseek-v4-flash': { modalities: { image: true } } },
|
||||||
|
});
|
||||||
|
fetchMock.mockResolvedValue(jsonResponse(api));
|
||||||
|
|
||||||
|
await refreshModelCatalog();
|
||||||
|
|
||||||
|
expect(fetchMock.mock.calls[0]?.[1]?.headers).toEqual({});
|
||||||
|
expect(readJson(getModelCatalogCachePath())).toMatchObject({
|
||||||
|
projection: MODEL_CATALOG_PROJECTION_VERSION,
|
||||||
|
models: trimmed,
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
it('leaves the cache untouched when the fetch fails', async () => {
|
||||||
|
writeJson(getModelCatalogCachePath(), {
|
||||||
|
source: MODELS_DEV_URL,
|
||||||
|
fetchedAt: LONG_AGO,
|
||||||
|
projection: MODEL_CATALOG_PROJECTION_VERSION,
|
||||||
|
models: { 'kept-model': { context: 7 } },
|
||||||
|
});
|
||||||
|
fetchMock.mockRejectedValue(new Error('offline'));
|
||||||
|
|
||||||
|
await expect(refreshModelCatalog()).resolves.toBeUndefined();
|
||||||
|
|
||||||
|
expect(readJson(getModelCatalogCachePath()).fetchedAt).toBe(LONG_AGO);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('leaves the cache untouched on an HTTP error', async () => {
|
||||||
|
writeJson(getModelCatalogCachePath(), {
|
||||||
|
source: MODELS_DEV_URL,
|
||||||
|
fetchedAt: LONG_AGO,
|
||||||
|
projection: MODEL_CATALOG_PROJECTION_VERSION,
|
||||||
|
models: {},
|
||||||
|
});
|
||||||
|
fetchMock.mockResolvedValue(new Response('nope', { status: 500 }));
|
||||||
|
|
||||||
|
await refreshModelCatalog();
|
||||||
|
|
||||||
|
expect(readJson(getModelCatalogCachePath()).fetchedAt).toBe(LONG_AGO);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('leaves the cache in force when a 200 response projects to nothing', async () => {
|
||||||
|
writeJson(getModelCatalogCachePath(), {
|
||||||
|
source: MODELS_DEV_URL,
|
||||||
|
fetchedAt: LONG_AGO,
|
||||||
|
projection: MODEL_CATALOG_PROJECTION_VERSION,
|
||||||
|
models: { 'kept-model': { context: 7 } },
|
||||||
|
});
|
||||||
|
// Valid JSON with no allowlisted provider, e.g. a gateway error envelope.
|
||||||
|
fetchMock.mockResolvedValue(jsonResponse({ message: 'rate limited' }));
|
||||||
|
|
||||||
|
await expect(refreshModelCatalog()).resolves.toBeUndefined();
|
||||||
|
|
||||||
|
const cache = readJson(getModelCatalogCachePath());
|
||||||
|
expect(cache.fetchedAt).toBe(LONG_AGO);
|
||||||
|
expect(cache.models).toEqual({ 'kept-model': { context: 7 } });
|
||||||
|
// The stale cache stays on disk but is older than the bundle, so the
|
||||||
|
// bundled snapshot is what answers.
|
||||||
|
expect(lookupModelCatalog('claude-fable-5')).toEqual(
|
||||||
|
bundled.models['claude-fable-5'],
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('rejects an incomplete 200 response with one valid model', async () => {
|
||||||
|
writeJson(getModelCatalogCachePath(), {
|
||||||
|
source: MODELS_DEV_URL,
|
||||||
|
fetchedAt: LONG_AGO,
|
||||||
|
projection: MODEL_CATALOG_PROJECTION_VERSION,
|
||||||
|
models: { 'kept-model': { context: 7 } },
|
||||||
|
});
|
||||||
|
fetchMock.mockResolvedValue(
|
||||||
|
jsonResponse({
|
||||||
|
openai: {
|
||||||
|
models: {
|
||||||
|
'gpt-x': chat('gpt-x', { context: 128000 }, ['text', 'image']),
|
||||||
|
},
|
||||||
|
},
|
||||||
|
}),
|
||||||
|
);
|
||||||
|
|
||||||
|
await refreshModelCatalog();
|
||||||
|
|
||||||
|
expect(readJson(getModelCatalogCachePath())).toMatchObject({
|
||||||
|
fetchedAt: LONG_AGO,
|
||||||
|
projection: MODEL_CATALOG_PROJECTION_VERSION,
|
||||||
|
models: { 'kept-model': { context: 7 } },
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
it('rejects a complete provider shape that projects to nothing', async () => {
|
||||||
|
writeJson(getModelCatalogCachePath(), {
|
||||||
|
source: MODELS_DEV_URL,
|
||||||
|
fetchedAt: LONG_AGO,
|
||||||
|
projection: MODEL_CATALOG_PROJECTION_VERSION,
|
||||||
|
models: { 'kept-model': { context: 7 } },
|
||||||
|
});
|
||||||
|
fetchMock.mockResolvedValue(jsonResponse(providerStubs));
|
||||||
|
|
||||||
|
await refreshModelCatalog();
|
||||||
|
|
||||||
|
expect(readJson(getModelCatalogCachePath())).toMatchObject({
|
||||||
|
fetchedAt: LONG_AGO,
|
||||||
|
projection: MODEL_CATALOG_PROJECTION_VERSION,
|
||||||
|
models: { 'kept-model': { context: 7 } },
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
it('does not reuse an empty cache: fetches without the ETag and heals', async () => {
|
||||||
|
writeJson(getModelCatalogCachePath(), {
|
||||||
|
source: MODELS_DEV_URL,
|
||||||
|
fetchedAt: LONG_AGO,
|
||||||
|
projection: MODEL_CATALOG_PROJECTION_VERSION,
|
||||||
|
etag: '"poisoned"',
|
||||||
|
models: {},
|
||||||
|
});
|
||||||
|
fetchMock.mockResolvedValue(jsonResponse(api, { etag: '"new"' }));
|
||||||
|
|
||||||
|
await refreshModelCatalog();
|
||||||
|
|
||||||
|
expect(fetchMock.mock.calls[0]?.[1]?.headers).toEqual({});
|
||||||
|
const cache = readJson(getModelCatalogCachePath());
|
||||||
|
expect(cache.models).toEqual(trimmed);
|
||||||
|
expect(cache.etag).toBe('"new"');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('does nothing when the refresh or the whole catalog is switched off', async () => {
|
||||||
|
process.env['QWEN_CODE_MODELS_DEV_REFRESH'] = 'off';
|
||||||
|
await refreshModelCatalog();
|
||||||
|
delete process.env['QWEN_CODE_MODELS_DEV_REFRESH'];
|
||||||
|
process.env['QWEN_CODE_MODELS_DEV'] = 'off';
|
||||||
|
await refreshModelCatalog();
|
||||||
|
|
||||||
|
expect(fetchMock).not.toHaveBeenCalled();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('fetches from QWEN_CODE_MODELS_DEV_URL when set', async () => {
|
||||||
|
process.env['QWEN_CODE_MODELS_DEV_URL'] = 'https://mirror.example/api.json';
|
||||||
|
fetchMock.mockResolvedValue(jsonResponse(api));
|
||||||
|
|
||||||
|
await refreshModelCatalog();
|
||||||
|
|
||||||
|
expect(fetchMock.mock.calls[0]?.[0]).toBe(
|
||||||
|
'https://mirror.example/api.json',
|
||||||
|
);
|
||||||
|
expect(readJson(getModelCatalogCachePath()).source).toBe(
|
||||||
|
'https://mirror.example/api.json',
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('shares one in-flight fetch between concurrent callers', async () => {
|
||||||
|
fetchMock.mockResolvedValue(jsonResponse(api));
|
||||||
|
|
||||||
|
await Promise.all([refreshModelCatalog(), refreshModelCatalog()]);
|
||||||
|
|
||||||
|
expect(fetchMock).toHaveBeenCalledTimes(1);
|
||||||
|
});
|
||||||
|
});
|
||||||
@@ -0,0 +1,410 @@
|
|||||||
|
/**
|
||||||
|
* @license
|
||||||
|
* Copyright 2025 Qwen Team
|
||||||
|
* SPDX-License-Identifier: Apache-2.0
|
||||||
|
*/
|
||||||
|
|
||||||
|
import * as fs from 'node:fs';
|
||||||
|
import * as path from 'node:path';
|
||||||
|
import type { InputModalities } from '../core/contentGenerator.js';
|
||||||
|
import {
|
||||||
|
MIN_AUTO_DETECTED_CONTEXT_WINDOW,
|
||||||
|
normalize,
|
||||||
|
} from '../core/tokenLimits.js';
|
||||||
|
import { atomicWriteJSON } from '../utils/atomicFileWrite.js';
|
||||||
|
import { createDebugLogger } from '../utils/debugLogger.js';
|
||||||
|
import { getErrorMessage } from '../utils/errors.js';
|
||||||
|
import {
|
||||||
|
CATALOG_MODALITIES,
|
||||||
|
getModelCatalogCachePath,
|
||||||
|
isModelCatalogDisabled,
|
||||||
|
isModelCatalogKey,
|
||||||
|
MODEL_CATALOG_PROJECTION_VERSION,
|
||||||
|
MODEL_CATALOG_URL_ENV,
|
||||||
|
MODELS_DEV_URL,
|
||||||
|
parseModelCatalog,
|
||||||
|
type ModelCatalog,
|
||||||
|
type ModelCatalogEntry,
|
||||||
|
} from './model-catalog.js';
|
||||||
|
|
||||||
|
const debugLogger = createDebugLogger('MODEL_CATALOG');
|
||||||
|
|
||||||
|
export { MODEL_CATALOG_URL_ENV, MODELS_DEV_URL } from './model-catalog.js';
|
||||||
|
/** `QWEN_CODE_MODELS_DEV_REFRESH=off` stops the once-a-day download; a downloaded cache still serves while it is newer than the bundled snapshot. */
|
||||||
|
export const MODEL_CATALOG_REFRESH_ENV = 'QWEN_CODE_MODELS_DEV_REFRESH';
|
||||||
|
|
||||||
|
const REFRESH_INTERVAL_MS = 24 * 60 * 60 * 1000;
|
||||||
|
const FETCH_TIMEOUT_MS = 10_000;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* models.dev providers whose token limits feed the catalog. Conflicting
|
||||||
|
* normalized ids lose their limits so endpoint-specific values fall back to
|
||||||
|
* existing tables. Pure mirrors are left out — they republish vendor models
|
||||||
|
* under their own aliases and limits, and a mirror's hosted-inference numbers
|
||||||
|
* otherwise veto the first-party vendor's (measured 2026-09-30 against the
|
||||||
|
* live payload: modelscope's only contributions were four dated `-2507`
|
||||||
|
* qwen3 snapshots recorded under the bare ids plus the `glm-4.6` veto, both
|
||||||
|
* the harm this list exists to avoid). `volcengine` stays because it is the
|
||||||
|
* first-party endpoint for the doubao/seed family — dropping it costs the
|
||||||
|
* catalog all 16 of those keys. Modalities are not gated by this list: they
|
||||||
|
* describe the weights, not the endpoint, so every provider that serves a
|
||||||
|
* model contributes them (see trimModelsDevCatalog).
|
||||||
|
*/
|
||||||
|
export const MODELS_DEV_PROVIDERS: readonly string[] = [
|
||||||
|
'anthropic',
|
||||||
|
'openai',
|
||||||
|
'google',
|
||||||
|
'deepseek',
|
||||||
|
'moonshotai',
|
||||||
|
'zai',
|
||||||
|
'minimax',
|
||||||
|
'xai',
|
||||||
|
'alibaba-cn',
|
||||||
|
'alibaba',
|
||||||
|
'volcengine',
|
||||||
|
];
|
||||||
|
|
||||||
|
interface ModelsDevModel {
|
||||||
|
id: string;
|
||||||
|
tool_call?: boolean;
|
||||||
|
limit?: { context?: number; input?: number; output?: number };
|
||||||
|
modalities?: { input?: string[]; output?: string[] };
|
||||||
|
}
|
||||||
|
|
||||||
|
export type ModelsDevApi = Record<
|
||||||
|
string,
|
||||||
|
{ models?: Record<string, ModelsDevModel> } | undefined
|
||||||
|
>;
|
||||||
|
|
||||||
|
type Limits = Pick<ModelCatalogEntry, 'context' | 'output'>;
|
||||||
|
|
||||||
|
function toLimits(model: ModelsDevModel): Limits | undefined {
|
||||||
|
const limits: Limits = {};
|
||||||
|
const context = model.limit?.input || model.limit?.context;
|
||||||
|
if (
|
||||||
|
context !== undefined &&
|
||||||
|
Number.isSafeInteger(context) &&
|
||||||
|
context >= MIN_AUTO_DETECTED_CONTEXT_WINDOW
|
||||||
|
) {
|
||||||
|
limits.context = context;
|
||||||
|
}
|
||||||
|
if (
|
||||||
|
model.limit?.output !== undefined &&
|
||||||
|
Number.isSafeInteger(model.limit.output) &&
|
||||||
|
model.limit.output > 0
|
||||||
|
) {
|
||||||
|
limits.output = model.limit.output;
|
||||||
|
}
|
||||||
|
return Object.keys(limits).length > 0 ? limits : undefined;
|
||||||
|
}
|
||||||
|
|
||||||
|
function toModalities(model: ModelsDevModel): InputModalities | undefined {
|
||||||
|
// An absent input list means upstream said nothing; an input list with no
|
||||||
|
// known key is a positive text-only declaration, which must survive as {}
|
||||||
|
// so a future narrowing can tell the two apart (every allowlisted model on
|
||||||
|
// models.dev declares the array).
|
||||||
|
if (model.modalities?.input === undefined) {
|
||||||
|
return undefined;
|
||||||
|
}
|
||||||
|
const modalities: InputModalities = {};
|
||||||
|
for (const modality of CATALOG_MODALITIES) {
|
||||||
|
if (model.modalities.input.includes(modality)) {
|
||||||
|
modalities[modality] = true;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return modalities;
|
||||||
|
}
|
||||||
|
|
||||||
|
function sortedModels(
|
||||||
|
entries: Iterable<readonly [string, ModelCatalogEntry]>,
|
||||||
|
): Record<string, ModelCatalogEntry> {
|
||||||
|
return Object.fromEntries(
|
||||||
|
[...entries].sort(([a], [b]) => (a < b ? -1 : a > b ? 1 : 0)),
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Only models that can drive the agent loop are worth recording: they must
|
||||||
|
* accept tool calls and answer in text. models.dev also lists embedding,
|
||||||
|
* text-to-speech, image and video models whose limits mean something else
|
||||||
|
* entirely — `gemini-embedding-001` reports an output limit of 1 and
|
||||||
|
* `veo-3.1-generate` a context of 480 — and those numbers would then outrank
|
||||||
|
* the family fallbacks that keep such ids harmless today.
|
||||||
|
*/
|
||||||
|
function servesAgentTurns(model: ModelsDevModel): boolean {
|
||||||
|
return (
|
||||||
|
model.tool_call === true &&
|
||||||
|
(model.modalities?.output ?? []).includes('text')
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
/** `toLimits` builds its keys in a fixed order, so this compares by value. */
|
||||||
|
function sameEntry(a: ModelCatalogEntry, b: ModelCatalogEntry): boolean {
|
||||||
|
return JSON.stringify(a) === JSON.stringify(b);
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Projects a models.dev `api.json` payload onto the catalog shape: one entry
|
||||||
|
* per normalized model id with only the fields the limit and modality tables
|
||||||
|
* consume.
|
||||||
|
*
|
||||||
|
* Two ids can land on the same key, either because they normalize together
|
||||||
|
* (`qwen3-max` and `qwen3-max-20260123`) or because several providers serve
|
||||||
|
* the same model. Limits and modalities follow different trust rules:
|
||||||
|
*
|
||||||
|
* A context window or output limit is a property of the endpoint, not of the
|
||||||
|
* weights — DashScope caps GLM-5 output at 16,384 while Z.ai allows 131,072,
|
||||||
|
* and both numbers are correct for their own endpoint — so only allowlisted
|
||||||
|
* providers contribute limits, and if any of them disagree the limits are
|
||||||
|
* dropped rather than guessed: the catalog cannot tell which endpoint a
|
||||||
|
* request will reach, so such models keep the answer the regex tables give
|
||||||
|
* them today.
|
||||||
|
*
|
||||||
|
* Modalities describe the weights, so trusted first-party providers contribute
|
||||||
|
* them. `thinkingmachines` is included for issue #8558 without trusting its
|
||||||
|
* endpoint limits. Router claims are excluded because they can widen a
|
||||||
|
* model's capabilities beyond what its owner reports.
|
||||||
|
*/
|
||||||
|
export function trimModelsDevCatalog(
|
||||||
|
api: ModelsDevApi,
|
||||||
|
fetchedAt: string,
|
||||||
|
source: string = MODELS_DEV_URL,
|
||||||
|
): ModelCatalog {
|
||||||
|
const limitCandidates = new Map<string, Limits[]>();
|
||||||
|
const modalityCandidates = new Map<string, InputModalities[]>();
|
||||||
|
for (const [provider, bucket] of Object.entries(api)) {
|
||||||
|
const trustedForLimits = MODELS_DEV_PROVIDERS.includes(provider);
|
||||||
|
const trustedForModalities =
|
||||||
|
trustedForLimits || provider === 'thinkingmachines';
|
||||||
|
for (const model of Object.values(bucket?.models ?? {})) {
|
||||||
|
if (typeof model.id !== 'string' || !servesAgentTurns(model)) {
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
const key = normalize(model.id);
|
||||||
|
// Lookups key on normalize(user input), so a key that is not its own
|
||||||
|
// normalized form is unreachable by its own spelling while a dated
|
||||||
|
// alias still hits it (`deepseek-v3-0324` -> `deepseek-v3` ->
|
||||||
|
// `deepseek`). Omit it so both spellings share the regex answer.
|
||||||
|
if (!isModelCatalogKey(key)) {
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
// DeepSeek models are text-only unless the id names a `vision` variant
|
||||||
|
// (#10270); some upstream bare-model records overstate image support.
|
||||||
|
// Key on the family root, not the hyphenated prefix: normalize() folds
|
||||||
|
// bare-model records onto it (`deepseek-v3` -> `deepseek`), and the
|
||||||
|
// root passes isModelCatalogKey, so a `deepseek-`-only guard would let
|
||||||
|
// exactly the records it exists for through.
|
||||||
|
const modalities =
|
||||||
|
/^deepseek(?:-|$)/.test(key) && !key.includes('vision')
|
||||||
|
? undefined
|
||||||
|
: toModalities(model);
|
||||||
|
if (modalities && trustedForModalities) {
|
||||||
|
const existing = modalityCandidates.get(key);
|
||||||
|
if (existing) {
|
||||||
|
existing.push(modalities);
|
||||||
|
} else {
|
||||||
|
modalityCandidates.set(key, [modalities]);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if (trustedForLimits) {
|
||||||
|
const limits = toLimits(model);
|
||||||
|
if (limits) {
|
||||||
|
const existing = limitCandidates.get(key);
|
||||||
|
if (existing) {
|
||||||
|
existing.push(limits);
|
||||||
|
} else {
|
||||||
|
limitCandidates.set(key, [limits]);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
const agreed: Array<readonly [string, ModelCatalogEntry]> = [];
|
||||||
|
for (const key of new Set([
|
||||||
|
...limitCandidates.keys(),
|
||||||
|
...modalityCandidates.keys(),
|
||||||
|
])) {
|
||||||
|
const entry: ModelCatalogEntry = {};
|
||||||
|
const limits = limitCandidates.get(key);
|
||||||
|
if (
|
||||||
|
limits &&
|
||||||
|
limits.every((candidate) => sameEntry(candidate, limits[0]!))
|
||||||
|
) {
|
||||||
|
Object.assign(entry, limits[0]);
|
||||||
|
}
|
||||||
|
const allModalities = modalityCandidates.get(key);
|
||||||
|
if (allModalities) {
|
||||||
|
const merged = Object.assign({}, ...allModalities);
|
||||||
|
// A declared-text-only record ({}) is kept only on entries that ship
|
||||||
|
// limits: there it is fidelity the union can later narrow against. On
|
||||||
|
// its own it carries no usable fact, so it must not create an entry —
|
||||||
|
// otherwise every conflict-dropped or limit-less model would survive
|
||||||
|
// as a bare `{}` marker.
|
||||||
|
if (Object.keys(merged).length > 0 || Object.keys(entry).length > 0) {
|
||||||
|
entry.modalities = merged;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if (Object.keys(entry).length > 0) {
|
||||||
|
agreed.push([key, entry]);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return {
|
||||||
|
source,
|
||||||
|
fetchedAt,
|
||||||
|
projection: MODEL_CATALOG_PROJECTION_VERSION,
|
||||||
|
models: sortedModels(agreed),
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
async function readCacheFile(
|
||||||
|
cachePath: string,
|
||||||
|
): Promise<ModelCatalog | undefined> {
|
||||||
|
try {
|
||||||
|
return parseModelCatalog(
|
||||||
|
JSON.parse(await fs.promises.readFile(cachePath, 'utf8')),
|
||||||
|
);
|
||||||
|
} catch {
|
||||||
|
return undefined;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// The full upstream feed is several MiB; 200 KiB budgets only the projection.
|
||||||
|
const MAX_CATALOG_BYTES = 16 * 1024 * 1024;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Reads a 200 body with a hard size ceiling, so a hostile or broken endpoint
|
||||||
|
* cannot make the once-a-day refresh buffer an unbounded payload. A present
|
||||||
|
* but non-JSON Content-Type is a gateway error page and fails here, one step
|
||||||
|
* earlier than JSON.parse; a missing one is tolerated for bare mirrors.
|
||||||
|
*/
|
||||||
|
async function readBoundedCatalogJson(
|
||||||
|
response: Response,
|
||||||
|
): Promise<ModelsDevApi> {
|
||||||
|
const contentType = response.headers.get('content-type');
|
||||||
|
if (contentType && !contentType.toLowerCase().includes('json')) {
|
||||||
|
await response.body?.cancel().catch(() => {});
|
||||||
|
throw new Error(`unexpected catalog content type: ${contentType}`);
|
||||||
|
}
|
||||||
|
if (!response.body) {
|
||||||
|
throw new Error('catalog response has no body');
|
||||||
|
}
|
||||||
|
const reader = response.body.getReader();
|
||||||
|
const chunks: Uint8Array[] = [];
|
||||||
|
let total = 0;
|
||||||
|
for (;;) {
|
||||||
|
const { done, value } = await reader.read();
|
||||||
|
if (done) {
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
total += value.byteLength;
|
||||||
|
if (total > MAX_CATALOG_BYTES) {
|
||||||
|
await reader.cancel().catch(() => {});
|
||||||
|
throw new Error(
|
||||||
|
`catalog response exceeds the ${MAX_CATALOG_BYTES}-byte budget`,
|
||||||
|
);
|
||||||
|
}
|
||||||
|
chunks.push(value);
|
||||||
|
}
|
||||||
|
return JSON.parse(Buffer.concat(chunks).toString('utf8')) as ModelsDevApi;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* A payload missing any allowlisted provider is truncated or partial-mirror
|
||||||
|
* data, not a catalog: it must never replace a complete snapshot or cache.
|
||||||
|
* This is a coverage check, not an allowlist of who may appear — providers
|
||||||
|
* outside MODELS_DEV_PROVIDERS (e.g. thinkingmachines) still contribute
|
||||||
|
* modalities. Shared by the runtime refresh and the snapshot generator so
|
||||||
|
* both paths reject the same payloads.
|
||||||
|
*/
|
||||||
|
export function assertCatalogPayloadComplete(api: ModelsDevApi): void {
|
||||||
|
const missingProvider = MODELS_DEV_PROVIDERS.find(
|
||||||
|
(provider) => Object.keys(api[provider]?.models ?? {}).length === 0,
|
||||||
|
);
|
||||||
|
if (missingProvider) {
|
||||||
|
throw new Error(`catalog is missing provider ${missingProvider}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async function refreshRemote(url: string, cachePath: string): Promise<void> {
|
||||||
|
const cached = await readCacheFile(cachePath);
|
||||||
|
// Old projections and unusable entries must be fetched again, not re-stamped
|
||||||
|
// by a 304 that only validates the unchanged upstream payload.
|
||||||
|
const reusable =
|
||||||
|
cached?.projection === MODEL_CATALOG_PROJECTION_VERSION &&
|
||||||
|
cached.source === url &&
|
||||||
|
Object.keys(cached.models).some(isModelCatalogKey)
|
||||||
|
? cached
|
||||||
|
: undefined;
|
||||||
|
if (reusable) {
|
||||||
|
const ageMs = Date.now() - Date.parse(reusable.fetchedAt);
|
||||||
|
// A negative or unreadable age means the stamp came from a different
|
||||||
|
// clock; refetch — the re-stamp below uses the local clock, so a skewed
|
||||||
|
// cache self-heals here instead of serving indefinitely.
|
||||||
|
if (Number.isFinite(ageMs) && ageMs >= 0 && ageMs < REFRESH_INTERVAL_MS) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
const headers: Record<string, string> = {};
|
||||||
|
if (reusable?.etag) {
|
||||||
|
headers['If-None-Match'] = reusable.etag;
|
||||||
|
}
|
||||||
|
const response = await fetch(url, {
|
||||||
|
headers,
|
||||||
|
signal: AbortSignal.timeout(FETCH_TIMEOUT_MS),
|
||||||
|
});
|
||||||
|
const fetchedAt = new Date().toISOString();
|
||||||
|
let next: ModelCatalog;
|
||||||
|
if (response.status === 304 && reusable) {
|
||||||
|
next = { ...reusable, fetchedAt };
|
||||||
|
} else if (response.ok) {
|
||||||
|
const api = await readBoundedCatalogJson(response);
|
||||||
|
assertCatalogPayloadComplete(api);
|
||||||
|
next = trimModelsDevCatalog(api, fetchedAt, url);
|
||||||
|
// A 200 that projects to nothing is not a catalog (a renamed upstream
|
||||||
|
// field or a gateway error body); keep the previous data instead of
|
||||||
|
// shadowing the bundled snapshot with an empty one for a day.
|
||||||
|
if (Object.keys(next.models).length === 0) {
|
||||||
|
throw new Error(`no catalog entries projected from ${url}`);
|
||||||
|
}
|
||||||
|
const etag = response.headers.get('etag');
|
||||||
|
if (etag) {
|
||||||
|
next.etag = etag;
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
// Release the socket: an unconsumed error body pins the connection.
|
||||||
|
await response.body?.cancel().catch(() => {});
|
||||||
|
throw new Error(`HTTP ${response.status}`);
|
||||||
|
}
|
||||||
|
await fs.promises.mkdir(path.dirname(cachePath), { recursive: true });
|
||||||
|
await atomicWriteJSON(cachePath, next);
|
||||||
|
debugLogger.debug(
|
||||||
|
`Model catalog refreshed from ${url}: ${Object.keys(next.models).length} models`,
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
let inFlight: Promise<void> | undefined;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Best-effort background refresh. Call after installing the proxy dispatcher;
|
||||||
|
* the selected catalog stays fixed for this process; the downloaded cache
|
||||||
|
* is available to the next process.
|
||||||
|
*/
|
||||||
|
export function refreshModelCatalog(): Promise<void> {
|
||||||
|
if (
|
||||||
|
isModelCatalogDisabled() ||
|
||||||
|
process.env[MODEL_CATALOG_REFRESH_ENV] === 'off'
|
||||||
|
) {
|
||||||
|
return Promise.resolve();
|
||||||
|
}
|
||||||
|
const url = process.env[MODEL_CATALOG_URL_ENV] || MODELS_DEV_URL;
|
||||||
|
inFlight ??= refreshRemote(url, getModelCatalogCachePath())
|
||||||
|
.catch((error: unknown) => {
|
||||||
|
debugLogger.debug(
|
||||||
|
`Model catalog refresh skipped: ${getErrorMessage(error)}`,
|
||||||
|
);
|
||||||
|
})
|
||||||
|
.finally(() => {
|
||||||
|
inFlight = undefined;
|
||||||
|
});
|
||||||
|
return inFlight;
|
||||||
|
}
|
||||||
@@ -0,0 +1,523 @@
|
|||||||
|
/**
|
||||||
|
* @license
|
||||||
|
* Copyright 2025 Qwen Team
|
||||||
|
* SPDX-License-Identifier: Apache-2.0
|
||||||
|
*/
|
||||||
|
|
||||||
|
import * as fs from 'node:fs';
|
||||||
|
import * as os from 'node:os';
|
||||||
|
import * as path from 'node:path';
|
||||||
|
import { afterEach, beforeEach, describe, expect, it } from 'vitest';
|
||||||
|
import type { Config } from '../config/config.js';
|
||||||
|
import { DefaultOpenAICompatibleProvider } from '../core/openaiContentGenerator/provider/default.js';
|
||||||
|
import { AuthType } from '../core/contentGenerator.js';
|
||||||
|
import {
|
||||||
|
clampOutputTokensToWindow,
|
||||||
|
defaultOutputCeiling,
|
||||||
|
hasExplicitOutputLimit,
|
||||||
|
tokenLimit,
|
||||||
|
} from '../core/tokenLimits.js';
|
||||||
|
import { computeThresholds } from '../services/chatCompressionService.js';
|
||||||
|
import { resolveModelConfig } from './modelConfigResolver.js';
|
||||||
|
import bundled from './generated/model-registry.json' with { type: 'json' };
|
||||||
|
import {
|
||||||
|
getModelCatalogCachePath,
|
||||||
|
invalidateModelCatalog,
|
||||||
|
isModelCatalogKey,
|
||||||
|
loadModelCatalog,
|
||||||
|
lookupModelCatalog,
|
||||||
|
MODEL_CATALOG_PROJECTION_VERSION,
|
||||||
|
MODEL_CATALOG_URL_ENV,
|
||||||
|
MODELS_DEV_URL,
|
||||||
|
parseModelCatalog,
|
||||||
|
} from './model-catalog.js';
|
||||||
|
|
||||||
|
const FAR_FUTURE = '9999-01-01T00:00:00.000Z';
|
||||||
|
const LONG_AGO = '2000-01-01T00:00:00.000Z';
|
||||||
|
|
||||||
|
function restoreEnv(name: string, value: string | undefined): void {
|
||||||
|
if (value === undefined) {
|
||||||
|
delete process.env[name];
|
||||||
|
} else {
|
||||||
|
process.env[name] = value;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
function writeJson(filePath: string, content: unknown): void {
|
||||||
|
fs.mkdirSync(path.dirname(filePath), { recursive: true });
|
||||||
|
fs.writeFileSync(
|
||||||
|
filePath,
|
||||||
|
typeof content === 'string' ? content : JSON.stringify(content),
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
describe('model catalog', () => {
|
||||||
|
let tempDir: string;
|
||||||
|
let previousHome: string | undefined;
|
||||||
|
let previousSwitch: string | undefined;
|
||||||
|
let previousUrl: string | undefined;
|
||||||
|
const [bundledId, bundledEntry] = Object.entries(bundled.models)[0] as [
|
||||||
|
string,
|
||||||
|
Record<string, unknown>,
|
||||||
|
];
|
||||||
|
|
||||||
|
beforeEach(() => {
|
||||||
|
tempDir = fs.mkdtempSync(path.join(os.tmpdir(), 'model-catalog-'));
|
||||||
|
previousHome = process.env['QWEN_HOME'];
|
||||||
|
previousSwitch = process.env['QWEN_CODE_MODELS_DEV'];
|
||||||
|
previousUrl = process.env[MODEL_CATALOG_URL_ENV];
|
||||||
|
process.env['QWEN_HOME'] = path.join(tempDir, '.qwen');
|
||||||
|
delete process.env['QWEN_CODE_MODELS_DEV'];
|
||||||
|
delete process.env[MODEL_CATALOG_URL_ENV];
|
||||||
|
invalidateModelCatalog();
|
||||||
|
});
|
||||||
|
|
||||||
|
afterEach(() => {
|
||||||
|
restoreEnv('QWEN_HOME', previousHome);
|
||||||
|
restoreEnv('QWEN_CODE_MODELS_DEV', previousSwitch);
|
||||||
|
restoreEnv(MODEL_CATALOG_URL_ENV, previousUrl);
|
||||||
|
invalidateModelCatalog();
|
||||||
|
fs.rmSync(tempDir, { recursive: true, force: true });
|
||||||
|
});
|
||||||
|
|
||||||
|
it('serves the bundled snapshot when no cache exists', () => {
|
||||||
|
expect(loadModelCatalog().fetchedAt).toBe(bundled.fetchedAt);
|
||||||
|
expect(lookupModelCatalog(bundledId)).toEqual(bundledEntry);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('returns undefined for a model the catalog does not list', () => {
|
||||||
|
expect(lookupModelCatalog('no-such-model')).toBeUndefined();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('is switched off by QWEN_CODE_MODELS_DEV=off', () => {
|
||||||
|
process.env['QWEN_CODE_MODELS_DEV'] = 'off';
|
||||||
|
expect(lookupModelCatalog(bundledId)).toBeUndefined();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('prefers a cache newer than the bundled snapshot', () => {
|
||||||
|
writeJson(getModelCatalogCachePath(), {
|
||||||
|
source: MODELS_DEV_URL,
|
||||||
|
fetchedAt: FAR_FUTURE,
|
||||||
|
projection: MODEL_CATALOG_PROJECTION_VERSION,
|
||||||
|
models: { 'cached-model': { context: 123, output: 45 } },
|
||||||
|
});
|
||||||
|
expect(lookupModelCatalog('cached-model')).toEqual({
|
||||||
|
context: 123,
|
||||||
|
output: 45,
|
||||||
|
});
|
||||||
|
expect(lookupModelCatalog(bundledId)).toBeUndefined();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('ignores a newer cache from a different configured source', () => {
|
||||||
|
process.env[MODEL_CATALOG_URL_ENV] = 'https://mirror.example/api.json';
|
||||||
|
writeJson(getModelCatalogCachePath(), {
|
||||||
|
source: MODELS_DEV_URL,
|
||||||
|
fetchedAt: FAR_FUTURE,
|
||||||
|
projection: MODEL_CATALOG_PROJECTION_VERSION,
|
||||||
|
models: { 'cached-model': { context: 123 } },
|
||||||
|
});
|
||||||
|
invalidateModelCatalog();
|
||||||
|
|
||||||
|
expect(lookupModelCatalog('cached-model')).toBeUndefined();
|
||||||
|
expect(loadModelCatalog().fetchedAt).toBe(bundled.fetchedAt);
|
||||||
|
});
|
||||||
|
|
||||||
|
it.each([undefined, 0, 2])(
|
||||||
|
'ignores a newer cache from another projection (%s)',
|
||||||
|
(projection) => {
|
||||||
|
writeJson(getModelCatalogCachePath(), {
|
||||||
|
source: MODELS_DEV_URL,
|
||||||
|
fetchedAt: FAR_FUTURE,
|
||||||
|
projection,
|
||||||
|
models: { 'deepseek-v4-flash': { modalities: { image: true } } },
|
||||||
|
});
|
||||||
|
|
||||||
|
expect(loadModelCatalog().fetchedAt).toBe(bundled.fetchedAt);
|
||||||
|
expect(lookupModelCatalog('deepseek-v4-flash')).toBeUndefined();
|
||||||
|
expect(lookupModelCatalog(bundledId)).toEqual(bundledEntry);
|
||||||
|
},
|
||||||
|
);
|
||||||
|
|
||||||
|
it('ignores a cache older than the bundled snapshot', () => {
|
||||||
|
writeJson(getModelCatalogCachePath(), {
|
||||||
|
source: 'test',
|
||||||
|
fetchedAt: LONG_AGO,
|
||||||
|
projection: MODEL_CATALOG_PROJECTION_VERSION,
|
||||||
|
models: { 'cached-model': { context: 123 } },
|
||||||
|
});
|
||||||
|
expect(lookupModelCatalog('cached-model')).toBeUndefined();
|
||||||
|
expect(loadModelCatalog().fetchedAt).toBe(bundled.fetchedAt);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('falls back to the bundled snapshot when the cache is malformed', () => {
|
||||||
|
writeJson(getModelCatalogCachePath(), '{not json');
|
||||||
|
expect(loadModelCatalog().fetchedAt).toBe(bundled.fetchedAt);
|
||||||
|
writeJson(getModelCatalogCachePath(), { fetchedAt: FAR_FUTURE });
|
||||||
|
invalidateModelCatalog();
|
||||||
|
expect(loadModelCatalog().fetchedAt).toBe(bundled.fetchedAt);
|
||||||
|
// Well-formed, but empty once every entry fails validation: the refresh
|
||||||
|
// side already refuses to write this shape, so a stale one must not win.
|
||||||
|
writeJson(getModelCatalogCachePath(), {
|
||||||
|
fetchedAt: FAR_FUTURE,
|
||||||
|
projection: MODEL_CATALOG_PROJECTION_VERSION,
|
||||||
|
models: { bad: { context: 'x' } },
|
||||||
|
});
|
||||||
|
invalidateModelCatalog();
|
||||||
|
expect(loadModelCatalog().fetchedAt).toBe(bundled.fetchedAt);
|
||||||
|
expect(lookupModelCatalog('qwen-flash')).toBeDefined();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('uses real bundled defaults without overriding explicit model settings', () => {
|
||||||
|
const sources = {
|
||||||
|
authType: AuthType.USE_OPENAI,
|
||||||
|
cli: {},
|
||||||
|
settings: {},
|
||||||
|
env: { OPENAI_MODEL: 'qwen-flash' },
|
||||||
|
};
|
||||||
|
expect(resolveModelConfig(sources).config.contextWindowSize).toBe(
|
||||||
|
bundled.models['qwen-flash'].context,
|
||||||
|
);
|
||||||
|
expect(
|
||||||
|
resolveModelConfig({
|
||||||
|
...sources,
|
||||||
|
settings: {
|
||||||
|
generationConfig: {
|
||||||
|
contextWindowSize: 12345,
|
||||||
|
modalities: { image: false },
|
||||||
|
},
|
||||||
|
},
|
||||||
|
}).config,
|
||||||
|
).toMatchObject({
|
||||||
|
contextWindowSize: 12345,
|
||||||
|
modalities: { image: false },
|
||||||
|
});
|
||||||
|
process.env['QWEN_CODE_MODELS_DEV'] = 'off';
|
||||||
|
expect(resolveModelConfig(sources).config.contextWindowSize).toBe(262144);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('uses the bundled gpt-4o limit on the env-only configuration path', () => {
|
||||||
|
const result = resolveModelConfig({
|
||||||
|
authType: AuthType.USE_OPENAI,
|
||||||
|
cli: {},
|
||||||
|
settings: {},
|
||||||
|
env: {
|
||||||
|
OPENAI_API_KEY: 'test-key',
|
||||||
|
OPENAI_BASE_URL: 'http://localhost:8000/v1',
|
||||||
|
OPENAI_MODEL: 'gpt-4o',
|
||||||
|
},
|
||||||
|
});
|
||||||
|
|
||||||
|
expect(result.config.contextWindowSize).toBe(128_000);
|
||||||
|
expect(result.sources['contextWindowSize'].kind).toBe('computed');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('preserves existing output limits across the entire bundled snapshot', () => {
|
||||||
|
for (const id of Object.keys(bundled.models)) {
|
||||||
|
process.env['QWEN_CODE_MODELS_DEV'] = 'off';
|
||||||
|
const pinned = hasExplicitOutputLimit(id);
|
||||||
|
const previous = tokenLimit(id, 'output');
|
||||||
|
delete process.env['QWEN_CODE_MODELS_DEV'];
|
||||||
|
if (pinned) {
|
||||||
|
expect({ id, output: tokenLimit(id, 'output') }).toEqual({
|
||||||
|
id,
|
||||||
|
output: previous,
|
||||||
|
});
|
||||||
|
}
|
||||||
|
}
|
||||||
|
expect(defaultOutputCeiling('glm-4.7')).toBe(16_384);
|
||||||
|
expect(tokenLimit('qwen-vl-max', 'output')).toBe(32_768);
|
||||||
|
expect(tokenLimit('claude-sonnet-4-6')).toBe(1_000_000);
|
||||||
|
expect(tokenLimit('claude-sonnet-5')).toBe(1_000_000);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('keeps bundled automatic windows safe at the compaction threshold', () => {
|
||||||
|
const unsafe: Array<{ id: string; total: number; window: number }> = [];
|
||||||
|
for (const id of Object.keys(bundled.models)) {
|
||||||
|
const window = tokenLimit(id);
|
||||||
|
const prompt = computeThresholds(window).auto;
|
||||||
|
const output = clampOutputTokensToWindow(
|
||||||
|
defaultOutputCeiling(id),
|
||||||
|
window,
|
||||||
|
prompt,
|
||||||
|
);
|
||||||
|
if (prompt + output > window) {
|
||||||
|
unsafe.push({ id, total: prompt + output, window });
|
||||||
|
}
|
||||||
|
}
|
||||||
|
expect(unsafe).toEqual([]);
|
||||||
|
expect(tokenLimit('gpt-4-1106-preview')).toBe(131_072);
|
||||||
|
expect(tokenLimit('qwen-math-plus')).toBe(262_144);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('ships the current modality-only projection for offline startup', () => {
|
||||||
|
expect(lookupModelCatalog('inkling')).toEqual({
|
||||||
|
modalities: { image: true },
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
it('keys every bundled entry by a usable model id', () => {
|
||||||
|
for (const id of Object.keys(bundled.models)) {
|
||||||
|
expect(isModelCatalogKey(id)).toBe(true);
|
||||||
|
}
|
||||||
|
// Both spellings of a model must resolve alike; a dated alias must not
|
||||||
|
// land on a key the canonical id cannot reach (deepseek-v3 did).
|
||||||
|
expect(lookupModelCatalog('deepseek-v3-0324')).toBeUndefined();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('drops cache keys no normalized spelling can reach', () => {
|
||||||
|
// A cache written before the projection's fixed-point filter can carry
|
||||||
|
// `deepseek-v3`; its own normalized spelling (deepseek) never reaches it,
|
||||||
|
// while it poisons every dated alias that resolves through the lookup.
|
||||||
|
writeJson(getModelCatalogCachePath(), {
|
||||||
|
source: 'https://models.dev/api.json',
|
||||||
|
fetchedAt: FAR_FUTURE,
|
||||||
|
projection: MODEL_CATALOG_PROJECTION_VERSION,
|
||||||
|
models: { 'deepseek-v3': { context: 163_840, output: 163_840 } },
|
||||||
|
});
|
||||||
|
invalidateModelCatalog();
|
||||||
|
expect(lookupModelCatalog('deepseek-v3')).toBeUndefined();
|
||||||
|
expect(tokenLimit('deepseek-v3-0324', 'output')).toBe(32_000);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('drops lossy and generic keys from an older cache', () => {
|
||||||
|
writeJson(getModelCatalogCachePath(), {
|
||||||
|
source: MODELS_DEV_URL,
|
||||||
|
fetchedAt: FAR_FUTURE,
|
||||||
|
projection: MODEL_CATALOG_PROJECTION_VERSION,
|
||||||
|
models: {
|
||||||
|
'32768': { modalities: { pdf: true } },
|
||||||
|
'model@default': { context: 123 },
|
||||||
|
auto: { modalities: { image: true } },
|
||||||
|
'valid-model': { context: 456 },
|
||||||
|
},
|
||||||
|
});
|
||||||
|
invalidateModelCatalog();
|
||||||
|
expect(lookupModelCatalog('32768')).toBeUndefined();
|
||||||
|
expect(lookupModelCatalog('model@default')).toBeUndefined();
|
||||||
|
expect(lookupModelCatalog('auto')).toBeUndefined();
|
||||||
|
expect(lookupModelCatalog('valid-model')).toEqual({ context: 456 });
|
||||||
|
});
|
||||||
|
|
||||||
|
it('keeps output pins after refresh while filling unknown model limits', () => {
|
||||||
|
writeJson(getModelCatalogCachePath(), {
|
||||||
|
source: 'https://models.dev/api.json',
|
||||||
|
fetchedAt: FAR_FUTURE,
|
||||||
|
projection: MODEL_CATALOG_PROJECTION_VERSION,
|
||||||
|
models: {
|
||||||
|
'glm-4.7': { output: 131_072 },
|
||||||
|
'qwen-vl-max': { output: 8_192 },
|
||||||
|
'new-model': { context: 200_000, output: 8_000 },
|
||||||
|
},
|
||||||
|
});
|
||||||
|
invalidateModelCatalog();
|
||||||
|
expect(defaultOutputCeiling('glm-4.7')).toBe(16_384);
|
||||||
|
expect(tokenLimit('qwen-vl-max', 'output')).toBe(32_768);
|
||||||
|
expect(tokenLimit('new-model', 'output')).toBe(8_000);
|
||||||
|
expect(hasExplicitOutputLimit('new-model')).toBe(false);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('uses catalog output defaults without clipping explicit provider requests', () => {
|
||||||
|
const previous = process.env['QWEN_CODE_MAX_OUTPUT_TOKENS'];
|
||||||
|
try {
|
||||||
|
for (const [model, defaultTokens] of [
|
||||||
|
['qwq-32b', 8_192],
|
||||||
|
['kimi-k2-thinking', 16_384],
|
||||||
|
['qvq-max', 8_192],
|
||||||
|
] as const) {
|
||||||
|
const provider = new DefaultOpenAICompatibleProvider(
|
||||||
|
{ model },
|
||||||
|
{} as Config,
|
||||||
|
);
|
||||||
|
const request = { model, messages: [] };
|
||||||
|
delete process.env['QWEN_CODE_MAX_OUTPUT_TOKENS'];
|
||||||
|
expect(provider.buildRequest(request, 'test').max_tokens).toBe(
|
||||||
|
defaultTokens,
|
||||||
|
);
|
||||||
|
expect(
|
||||||
|
provider.buildRequest({ ...request, max_tokens: 32_768 }, 'test')
|
||||||
|
.max_tokens,
|
||||||
|
).toBe(32_768);
|
||||||
|
process.env['QWEN_CODE_MAX_OUTPUT_TOKENS'] = '32768';
|
||||||
|
expect(provider.buildRequest(request, 'test').max_tokens).toBe(32_768);
|
||||||
|
}
|
||||||
|
// The only case where the explicit env budget exceeds a curated pin,
|
||||||
|
// so the isKnownModel clamp must win over the env value.
|
||||||
|
process.env['QWEN_CODE_MAX_OUTPUT_TOKENS'] = '32768';
|
||||||
|
const provider = new DefaultOpenAICompatibleProvider(
|
||||||
|
{ model: 'glm-4.7' },
|
||||||
|
{} as Config,
|
||||||
|
);
|
||||||
|
expect(
|
||||||
|
provider.buildRequest({ model: 'glm-4.7', messages: [] }, 'test')
|
||||||
|
.max_tokens,
|
||||||
|
).toBe(16_384);
|
||||||
|
} finally {
|
||||||
|
restoreEnv('QWEN_CODE_MAX_OUTPUT_TOKENS', previous);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
it('requires explicit Qwen PDF support for both bundled and refreshed data', () => {
|
||||||
|
const sources = {
|
||||||
|
authType: AuthType.USE_OPENAI,
|
||||||
|
cli: {},
|
||||||
|
settings: {},
|
||||||
|
env: { OPENAI_MODEL: 'qwen3.8-max' },
|
||||||
|
};
|
||||||
|
expect(resolveModelConfig(sources).config.modalities).toMatchObject({
|
||||||
|
image: true,
|
||||||
|
video: true,
|
||||||
|
});
|
||||||
|
expect(resolveModelConfig(sources).config.modalities?.pdf).toBeUndefined();
|
||||||
|
writeJson(getModelCatalogCachePath(), {
|
||||||
|
source: 'https://models.dev/api.json',
|
||||||
|
fetchedAt: FAR_FUTURE,
|
||||||
|
projection: MODEL_CATALOG_PROJECTION_VERSION,
|
||||||
|
models: { 'qwen3.8-max': { modalities: { image: true, pdf: true } } },
|
||||||
|
});
|
||||||
|
invalidateModelCatalog();
|
||||||
|
expect(resolveModelConfig(sources).config.modalities?.pdf).toBeUndefined();
|
||||||
|
expect(
|
||||||
|
resolveModelConfig({
|
||||||
|
...sources,
|
||||||
|
settings: { generationConfig: { modalities: { pdf: true } } },
|
||||||
|
}).config.modalities?.pdf,
|
||||||
|
).toBe(true);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('keeps Sonnet 4.5 at its default API limit even after a refresh', () => {
|
||||||
|
expect(lookupModelCatalog('claude-sonnet-4-5')?.context).toBe(200_000);
|
||||||
|
writeJson(getModelCatalogCachePath(), {
|
||||||
|
source: 'https://models.dev/api.json',
|
||||||
|
fetchedAt: FAR_FUTURE,
|
||||||
|
projection: MODEL_CATALOG_PROJECTION_VERSION,
|
||||||
|
models: {
|
||||||
|
'claude-sonnet-4-5': { context: 1_000_000, output: 64_000 },
|
||||||
|
'claude-sonnet-4-6': { context: 1_000_000 },
|
||||||
|
},
|
||||||
|
});
|
||||||
|
invalidateModelCatalog();
|
||||||
|
expect(lookupModelCatalog('claude-sonnet-4-5')).toEqual({
|
||||||
|
context: 200_000,
|
||||||
|
output: 64_000,
|
||||||
|
});
|
||||||
|
expect(lookupModelCatalog('claude-sonnet-4-6')?.context).toBe(1_000_000);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('keeps qwen3-coder-plus at its vendor-declared 1M window even after a refresh', () => {
|
||||||
|
expect(tokenLimit('qwen3-coder-plus')).toBe(1_000_000);
|
||||||
|
expect(
|
||||||
|
computeThresholds(tokenLimit('qwen3-coder-plus')).hard,
|
||||||
|
).toBeLessThanOrEqual(1_000_000);
|
||||||
|
writeJson(getModelCatalogCachePath(), {
|
||||||
|
source: 'https://models.dev/api.json',
|
||||||
|
fetchedAt: FAR_FUTURE,
|
||||||
|
projection: MODEL_CATALOG_PROJECTION_VERSION,
|
||||||
|
models: { 'qwen3-coder-plus': { context: 1_048_576, output: 65_536 } },
|
||||||
|
});
|
||||||
|
invalidateModelCatalog();
|
||||||
|
expect(lookupModelCatalog('qwen3-coder-plus')).toEqual({
|
||||||
|
context: 1_000_000,
|
||||||
|
output: 65_536,
|
||||||
|
});
|
||||||
|
expect(tokenLimit('qwen3-coder-plus')).toBe(1_000_000);
|
||||||
|
});
|
||||||
|
|
||||||
|
// models.dev buckets each of these above the window the curated table in
|
||||||
|
// tokenLimits.ts and the provider presets declare, so the declared value
|
||||||
|
// must survive both the bundled snapshot and a newer downloaded one.
|
||||||
|
it.each([
|
||||||
|
['kimi-k3', 1_000_000],
|
||||||
|
['minimax-m2.5', 196_608],
|
||||||
|
['minimax-m2.5-highspeed', 196_608],
|
||||||
|
['glm-4.7', 202_752],
|
||||||
|
])(
|
||||||
|
'keeps %s at its declared %i window even after a refresh',
|
||||||
|
(id, declared) => {
|
||||||
|
expect(tokenLimit(id)).toBe(declared);
|
||||||
|
expect(computeThresholds(tokenLimit(id)).hard).toBeLessThanOrEqual(
|
||||||
|
declared,
|
||||||
|
);
|
||||||
|
writeJson(getModelCatalogCachePath(), {
|
||||||
|
source: 'https://models.dev/api.json',
|
||||||
|
fetchedAt: FAR_FUTURE,
|
||||||
|
projection: MODEL_CATALOG_PROJECTION_VERSION,
|
||||||
|
models: { [id]: { context: declared + 48_576, output: 65_536 } },
|
||||||
|
});
|
||||||
|
invalidateModelCatalog();
|
||||||
|
expect(lookupModelCatalog(id)?.context).toBe(declared);
|
||||||
|
expect(tokenLimit(id)).toBe(declared);
|
||||||
|
},
|
||||||
|
);
|
||||||
|
|
||||||
|
it('drops malformed entries while parsing', () => {
|
||||||
|
const parsed = parseModelCatalog({
|
||||||
|
fetchedAt: FAR_FUTURE,
|
||||||
|
projection: MODEL_CATALOG_PROJECTION_VERSION,
|
||||||
|
etag: '"abc"',
|
||||||
|
models: {
|
||||||
|
good: { context: 1, modalities: { image: true } },
|
||||||
|
bad: { context: 'x' },
|
||||||
|
worse: null,
|
||||||
|
negative: { context: -1 },
|
||||||
|
zero: { output: 0 },
|
||||||
|
infinite: { context: Infinity },
|
||||||
|
fractional: { output: 1.5 },
|
||||||
|
// A wrong-typed or false flag drops only the modality field — the
|
||||||
|
// entry's limits survive a malformed modalities member.
|
||||||
|
invalidModality: { context: 5, modalities: { image: 'false' } },
|
||||||
|
falseModality: { context: 6, modalities: { image: false } },
|
||||||
|
// A modality key this build does not know (e.g. written by a newer
|
||||||
|
// one) degrades that field, not the whole entry.
|
||||||
|
futureModality: {
|
||||||
|
context: 100,
|
||||||
|
output: 10,
|
||||||
|
modalities: { image: true, hologram: true },
|
||||||
|
},
|
||||||
|
// An own-property `__proto__` key (the refresh path can legitimately
|
||||||
|
// write one via Object.fromEntries; a literal would invoke the setter
|
||||||
|
// here, so it comes in through spread). The parse must skip it rather
|
||||||
|
// than mutate the map's prototype.
|
||||||
|
...Object.fromEntries([['__proto__', { context: 1 }]]),
|
||||||
|
},
|
||||||
|
});
|
||||||
|
expect(parsed).toEqual({
|
||||||
|
source: '',
|
||||||
|
fetchedAt: FAR_FUTURE,
|
||||||
|
projection: MODEL_CATALOG_PROJECTION_VERSION,
|
||||||
|
etag: '"abc"',
|
||||||
|
models: {
|
||||||
|
good: { context: 1, modalities: { image: true } },
|
||||||
|
invalidModality: { context: 5, modalities: {} },
|
||||||
|
falseModality: { context: 6, modalities: {} },
|
||||||
|
futureModality: {
|
||||||
|
context: 100,
|
||||||
|
output: 10,
|
||||||
|
modalities: { image: true },
|
||||||
|
},
|
||||||
|
},
|
||||||
|
});
|
||||||
|
expect(Object.getPrototypeOf(parsed!.models)).toBe(Object.prototype);
|
||||||
|
expect(parseModelCatalog({ models: {} })).toBeUndefined();
|
||||||
|
expect(parseModelCatalog('nope')).toBeUndefined();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('keeps every entry of the committed snapshot parseable', () => {
|
||||||
|
// The bundled file is the floor every offline user starts from, so it
|
||||||
|
// must satisfy the same parser the downloaded cache goes through.
|
||||||
|
const parsed = parseModelCatalog(bundled);
|
||||||
|
expect(parsed).toBeDefined();
|
||||||
|
expect(Object.keys(parsed!.models).sort()).toEqual(
|
||||||
|
Object.keys(bundled.models).sort(),
|
||||||
|
);
|
||||||
|
expect(Buffer.byteLength(JSON.stringify(bundled))).toBeLessThanOrEqual(
|
||||||
|
200 * 1024,
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('stamps the committed snapshot with the canonical catalog URL', () => {
|
||||||
|
// Provenance must name where the data comes from, not the local path a
|
||||||
|
// maintainer happened to generate from (scripts/generate-model-catalog.ts
|
||||||
|
// stamps MODELS_DEV_URL for local-path inputs).
|
||||||
|
expect(bundled.source).toBe(MODELS_DEV_URL);
|
||||||
|
});
|
||||||
|
});
|
||||||
@@ -0,0 +1,261 @@
|
|||||||
|
/**
|
||||||
|
* @license
|
||||||
|
* Copyright 2025 Qwen Team
|
||||||
|
* SPDX-License-Identifier: Apache-2.0
|
||||||
|
*/
|
||||||
|
|
||||||
|
import * as fs from 'node:fs';
|
||||||
|
import * as path from 'node:path';
|
||||||
|
import { Storage } from '../config/storage.js';
|
||||||
|
import type { InputModalities } from '../core/contentGenerator.js';
|
||||||
|
import { normalize } from '../core/tokenLimits.js';
|
||||||
|
import bundledCatalog from './generated/model-registry.json' with { type: 'json' };
|
||||||
|
|
||||||
|
export interface ModelCatalogEntry {
|
||||||
|
/** Context window: models.dev `limit.input`, else `limit.context`. */
|
||||||
|
context?: number;
|
||||||
|
/** Maximum output tokens: models.dev `limit.output`. */
|
||||||
|
output?: number;
|
||||||
|
/** Non-text input modalities the model accepts. */
|
||||||
|
modalities?: InputModalities;
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface ModelCatalog {
|
||||||
|
source: string;
|
||||||
|
fetchedAt: string;
|
||||||
|
projection: number;
|
||||||
|
etag?: string;
|
||||||
|
models: Record<string, ModelCatalogEntry>;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Bump when projection rules change so older caches are fetched and reprojected.
|
||||||
|
export const MODEL_CATALOG_PROJECTION_VERSION = 1;
|
||||||
|
|
||||||
|
/** `QWEN_CODE_MODELS_DEV=off` restores the regex-only model tables. */
|
||||||
|
export const MODEL_CATALOG_ENV = 'QWEN_CODE_MODELS_DEV';
|
||||||
|
export const MODELS_DEV_URL = 'https://models.dev/api.json';
|
||||||
|
/** Replaces the models.dev URL, e.g. with a corporate mirror. */
|
||||||
|
export const MODEL_CATALOG_URL_ENV = 'QWEN_CODE_MODELS_DEV_URL';
|
||||||
|
|
||||||
|
const GENERIC_MODEL_KEYS = new Set([
|
||||||
|
'auto',
|
||||||
|
'fast',
|
||||||
|
'free',
|
||||||
|
'latest',
|
||||||
|
'low',
|
||||||
|
'max',
|
||||||
|
]);
|
||||||
|
|
||||||
|
export function isModelCatalogDisabled(): boolean {
|
||||||
|
return process.env[MODEL_CATALOG_ENV] === 'off';
|
||||||
|
}
|
||||||
|
|
||||||
|
export function isModelCatalogKey(key: string): boolean {
|
||||||
|
return (
|
||||||
|
normalize(key) === key &&
|
||||||
|
!/^\d+$/.test(key) &&
|
||||||
|
!key.includes('@') &&
|
||||||
|
!GENERIC_MODEL_KEYS.has(key) &&
|
||||||
|
(key.length >= 5 || /^o\d+$/i.test(key))
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
export function getModelCatalogCachePath(): string {
|
||||||
|
return path.join(Storage.getGlobalQwenDir(), 'model-registry.json');
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Modality keys the catalog understands. This is the single list: the
|
||||||
|
* refresh projection writes exactly these (`toModalities`), so a new
|
||||||
|
* `InputModalities` key fails tsc here at declaration time rather than
|
||||||
|
* silently discarding entries at parse time.
|
||||||
|
*/
|
||||||
|
export const CATALOG_MODALITIES: ReadonlyArray<keyof InputModalities> = [
|
||||||
|
'image',
|
||||||
|
'pdf',
|
||||||
|
'audio',
|
||||||
|
'video',
|
||||||
|
];
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Narrows a raw object key to one the catalog understands. Widening to
|
||||||
|
* `readonly string[]` is what lets `includes` accept an arbitrary key, but it
|
||||||
|
* also stops the result from narrowing — hence the predicate.
|
||||||
|
*/
|
||||||
|
function isCatalogModality(key: string): key is keyof InputModalities {
|
||||||
|
return (CATALOG_MODALITIES as readonly string[]).includes(key);
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Keeps the fields that are valid instead of discarding the entry for one
|
||||||
|
* that is not: an unrecognised modality key (e.g. written by a newer build
|
||||||
|
* that knows more modalities) degrades that one field rather than taking the
|
||||||
|
* model's context window and output limit down with it. Context/output stay
|
||||||
|
* strict — a corrupt limit is worse than no limit.
|
||||||
|
*/
|
||||||
|
function sanitizeEntry(value: unknown): ModelCatalogEntry | undefined {
|
||||||
|
if (!value || typeof value !== 'object' || Array.isArray(value)) {
|
||||||
|
return undefined;
|
||||||
|
}
|
||||||
|
const { context, output, modalities } = value as ModelCatalogEntry;
|
||||||
|
if (
|
||||||
|
(context !== undefined &&
|
||||||
|
!(Number.isSafeInteger(context) && context > 0)) ||
|
||||||
|
(output !== undefined && !(Number.isSafeInteger(output) && output > 0))
|
||||||
|
) {
|
||||||
|
return undefined;
|
||||||
|
}
|
||||||
|
const entry: ModelCatalogEntry = {
|
||||||
|
...(context !== undefined ? { context } : {}),
|
||||||
|
...(output !== undefined ? { output } : {}),
|
||||||
|
};
|
||||||
|
if (modalities !== undefined) {
|
||||||
|
if (
|
||||||
|
typeof modalities !== 'object' ||
|
||||||
|
modalities === null ||
|
||||||
|
Array.isArray(modalities)
|
||||||
|
) {
|
||||||
|
return undefined;
|
||||||
|
}
|
||||||
|
const clean: InputModalities = {};
|
||||||
|
for (const [key, flag] of Object.entries(modalities)) {
|
||||||
|
if (isCatalogModality(key) && flag === true) {
|
||||||
|
clean[key] = true;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
entry.modalities = clean;
|
||||||
|
}
|
||||||
|
return entry;
|
||||||
|
}
|
||||||
|
|
||||||
|
export function parseModelCatalog(raw: unknown): ModelCatalog | undefined {
|
||||||
|
if (!raw || typeof raw !== 'object') {
|
||||||
|
return undefined;
|
||||||
|
}
|
||||||
|
const { source, fetchedAt, projection, etag, models } =
|
||||||
|
raw as Partial<ModelCatalog>;
|
||||||
|
if (typeof fetchedAt !== 'string' || !models || typeof models !== 'object') {
|
||||||
|
return undefined;
|
||||||
|
}
|
||||||
|
const valid: Record<string, ModelCatalogEntry> = {};
|
||||||
|
for (const [id, entry] of Object.entries(models)) {
|
||||||
|
// '__proto__' would invoke the prototype setter instead of creating an
|
||||||
|
// own property — the refresh path can legitimately write that key
|
||||||
|
// (Object.fromEntries uses [[DefineOwnProperty]]), so tolerate it here.
|
||||||
|
if (id === '__proto__') {
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
const sanitized = sanitizeEntry(entry);
|
||||||
|
if (sanitized) {
|
||||||
|
valid[id] = sanitized;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return {
|
||||||
|
source: typeof source === 'string' ? source : '',
|
||||||
|
fetchedAt,
|
||||||
|
projection:
|
||||||
|
typeof projection === 'number' && Number.isSafeInteger(projection)
|
||||||
|
? projection
|
||||||
|
: 0,
|
||||||
|
...(typeof etag === 'string' ? { etag } : {}),
|
||||||
|
models: valid,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
let loaded: ModelCatalog | undefined;
|
||||||
|
function readCache(cachePath: string): ModelCatalog | undefined {
|
||||||
|
try {
|
||||||
|
return parseModelCatalog(JSON.parse(fs.readFileSync(cachePath, 'utf8')));
|
||||||
|
} catch {
|
||||||
|
return undefined;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Client-owned context windows. models.dev's `limit.input` is the source of
|
||||||
|
* truth for input limits, but for these ids it is bucketed above the window
|
||||||
|
* the curated table in `tokenLimits.ts` *and* the provider presets declare,
|
||||||
|
* so the declared window wins. Only ids the resolved catalog actually
|
||||||
|
* carries are corrected.
|
||||||
|
*/
|
||||||
|
const CATALOG_CONTEXT_CORRECTIONS: Readonly<Record<string, number>> = {
|
||||||
|
// Sonnet 4.5's retired 1M beta must not override the default API limit.
|
||||||
|
// https://platform.claude.com/docs/en/build-with-claude/context-windows
|
||||||
|
'claude-sonnet-4-5': 200_000,
|
||||||
|
// models.dev rounds the vendor-declared window of these ids up to the next
|
||||||
|
// binary size: 1,048,576 for the 1M ids, 204,800 for the MiniMax-M2.5 and
|
||||||
|
// GLM-4.7 rounds. The over-stated window is larger than anything the
|
||||||
|
// curated row or the presets declare, so compaction thresholds computed
|
||||||
|
// from it fire too late and the request 400s at the vendor.
|
||||||
|
'qwen3-coder-plus': 1_000_000,
|
||||||
|
'kimi-k3': 1_000_000,
|
||||||
|
'minimax-m2.5': 196_608,
|
||||||
|
'minimax-m2.5-highspeed': 196_608,
|
||||||
|
'glm-4.7': 202_752,
|
||||||
|
};
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The refreshed cache wins only when it is newer than the snapshot bundled
|
||||||
|
* with this build, so upgrading the CLI never serves stale cached data.
|
||||||
|
*/
|
||||||
|
export function loadModelCatalog(): ModelCatalog {
|
||||||
|
if (!loaded) {
|
||||||
|
const bundled = bundledCatalog as ModelCatalog;
|
||||||
|
const cached = readCache(getModelCatalogCachePath());
|
||||||
|
const source = process.env[MODEL_CATALOG_URL_ENV] || MODELS_DEV_URL;
|
||||||
|
// A cache written before the projection's guards can hold keys no
|
||||||
|
// normalized spelling reaches (deepseek-v3 did) or no usable entries at
|
||||||
|
// all; neither may displace the bundled snapshot.
|
||||||
|
const usableEntries = Object.entries(cached?.models ?? {}).filter(([key]) =>
|
||||||
|
isModelCatalogKey(key),
|
||||||
|
);
|
||||||
|
const usable =
|
||||||
|
cached &&
|
||||||
|
cached.projection === MODEL_CATALOG_PROJECTION_VERSION &&
|
||||||
|
cached.source === source &&
|
||||||
|
usableEntries.length > 0
|
||||||
|
? { ...cached, models: Object.fromEntries(usableEntries) }
|
||||||
|
: undefined;
|
||||||
|
let base =
|
||||||
|
usable && usable.fetchedAt > bundled.fetchedAt ? usable : bundled;
|
||||||
|
const models = { ...base.models };
|
||||||
|
let corrected = false;
|
||||||
|
for (const [id, context] of Object.entries(CATALOG_CONTEXT_CORRECTIONS)) {
|
||||||
|
const entry = models[id];
|
||||||
|
if (entry) {
|
||||||
|
models[id] = { ...entry, context };
|
||||||
|
corrected = true;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if (corrected) {
|
||||||
|
base = { ...base, models };
|
||||||
|
}
|
||||||
|
loaded = base;
|
||||||
|
}
|
||||||
|
return loaded;
|
||||||
|
}
|
||||||
|
|
||||||
|
export function invalidateModelCatalog(): void {
|
||||||
|
loaded = undefined;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* `model` must already be normalized (`normalize()` in tokenLimits.ts) so
|
||||||
|
* the catalog keys and the regex tables see the same id.
|
||||||
|
*/
|
||||||
|
export function lookupModelCatalog(
|
||||||
|
model: string,
|
||||||
|
): ModelCatalogEntry | undefined {
|
||||||
|
if (isModelCatalogDisabled()) {
|
||||||
|
return undefined;
|
||||||
|
}
|
||||||
|
const entry = loadModelCatalog().models[model];
|
||||||
|
// DashScope PDF support depends on endpoint and protocol (not Responses).
|
||||||
|
// Keep it opt-in through explicit model configuration until scoped lookup.
|
||||||
|
if (model === 'qwen3.8-max' && entry?.modalities?.pdf) {
|
||||||
|
const modalities = { ...entry.modalities };
|
||||||
|
delete modalities.pdf;
|
||||||
|
return { ...entry, modalities };
|
||||||
|
}
|
||||||
|
return entry;
|
||||||
|
}
|
||||||
@@ -740,9 +740,13 @@ describe('modelConfigResolver', () => {
|
|||||||
});
|
});
|
||||||
|
|
||||||
it('env-var-only path: contextWindowSize auto-detected for a model whose limit differs from the global default', () => {
|
it('env-var-only path: contextWindowSize auto-detected for a model whose limit differs from the global default', () => {
|
||||||
// gpt-4o resolves to 131,072 ≠ DEFAULT_TOKEN_LIMIT (200,000), so this
|
// This suite runs with the catalog off (packages/core/test-setup.ts),
|
||||||
// assertion fails if the fallback applies the generic default instead
|
// so 131,072 is the regex-table answer, not the shipped one — the
|
||||||
// of the model-specific limit.
|
// catalog-on twin of this exact env-only path pins the production value
|
||||||
|
// (128,000) in model-catalog.test.ts ('uses the bundled gpt-4o limit on
|
||||||
|
// the env-only configuration path'). gpt-4o resolves to 131,072 ≠
|
||||||
|
// DEFAULT_TOKEN_LIMIT (200,000), so this assertion fails if the fallback
|
||||||
|
// applies the generic default instead of the model-specific limit.
|
||||||
const result = envOnly('gpt-4o');
|
const result = envOnly('gpt-4o');
|
||||||
|
|
||||||
expect(result.config.model).toBe('gpt-4o');
|
expect(result.config.model).toBe('gpt-4o');
|
||||||
|
|||||||
@@ -20,6 +20,10 @@ if (process.env['QWEN_DEBUG_LOG_FILE'] === undefined) {
|
|||||||
// Disable 429 simulation globally for all tests
|
// Disable 429 simulation globally for all tests
|
||||||
setSimulate429(false);
|
setSimulate429(false);
|
||||||
|
|
||||||
|
// Model limits and modalities come from the regex tables unless a test opts
|
||||||
|
// into the models.dev catalog.
|
||||||
|
process.env['QWEN_CODE_MODELS_DEV'] = 'off';
|
||||||
|
|
||||||
// Keep managed auto-memory test fixtures under per-test temp project roots.
|
// Keep managed auto-memory test fixtures under per-test temp project roots.
|
||||||
if (process.env['QWEN_CODE_MEMORY_LOCAL'] === undefined) {
|
if (process.env['QWEN_CODE_MEMORY_LOCAL'] === undefined) {
|
||||||
process.env['QWEN_CODE_MEMORY_LOCAL'] = '1';
|
process.env['QWEN_CODE_MEMORY_LOCAL'] = '1';
|
||||||
|
|||||||
@@ -4,12 +4,44 @@
|
|||||||
* SPDX-License-Identifier: Apache-2.0
|
* SPDX-License-Identifier: Apache-2.0
|
||||||
*/
|
*/
|
||||||
|
|
||||||
import { describe, expect, it } from 'vitest';
|
import * as fs from 'node:fs';
|
||||||
|
import * as os from 'node:os';
|
||||||
|
import * as path from 'node:path';
|
||||||
|
import { afterAll, beforeAll, describe, expect, it } from 'vitest';
|
||||||
|
import { invalidateModelCatalog } from '@qwen-code/qwen-code-core/models/model-catalog.js';
|
||||||
import {
|
import {
|
||||||
extractModelInfoFromNewSessionResult,
|
extractModelInfoFromNewSessionResult,
|
||||||
extractSessionModelState,
|
extractSessionModelState,
|
||||||
} from './acpModelInfo.js';
|
} from './acpModelInfo.js';
|
||||||
|
|
||||||
|
let tempDir: string;
|
||||||
|
let previousHome: string | undefined;
|
||||||
|
let previousSwitch: string | undefined;
|
||||||
|
|
||||||
|
beforeAll(() => {
|
||||||
|
tempDir = fs.mkdtempSync(path.join(os.tmpdir(), 'acp-model-info-'));
|
||||||
|
previousHome = process.env['QWEN_HOME'];
|
||||||
|
previousSwitch = process.env['QWEN_CODE_MODELS_DEV'];
|
||||||
|
process.env['QWEN_HOME'] = tempDir;
|
||||||
|
delete process.env['QWEN_CODE_MODELS_DEV'];
|
||||||
|
invalidateModelCatalog();
|
||||||
|
});
|
||||||
|
|
||||||
|
afterAll(() => {
|
||||||
|
if (previousHome === undefined) {
|
||||||
|
delete process.env['QWEN_HOME'];
|
||||||
|
} else {
|
||||||
|
process.env['QWEN_HOME'] = previousHome;
|
||||||
|
}
|
||||||
|
if (previousSwitch === undefined) {
|
||||||
|
delete process.env['QWEN_CODE_MODELS_DEV'];
|
||||||
|
} else {
|
||||||
|
process.env['QWEN_CODE_MODELS_DEV'] = previousSwitch;
|
||||||
|
}
|
||||||
|
invalidateModelCatalog();
|
||||||
|
fs.rmSync(tempDir, { recursive: true, force: true });
|
||||||
|
});
|
||||||
|
|
||||||
describe('extractSessionModelState', () => {
|
describe('extractSessionModelState', () => {
|
||||||
it('extracts full model state from NewSessionResponse.models', () => {
|
it('extracts full model state from NewSessionResponse.models', () => {
|
||||||
const result = extractSessionModelState({
|
const result = extractSessionModelState({
|
||||||
|
|||||||
@@ -0,0 +1,78 @@
|
|||||||
|
/**
|
||||||
|
* @license
|
||||||
|
* Copyright 2025 Qwen team
|
||||||
|
* SPDX-License-Identifier: Apache-2.0
|
||||||
|
*/
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Refreshes the bundled models.dev snapshot that ships with the CLI as the
|
||||||
|
* offline floor for model context windows, output limits, and modalities.
|
||||||
|
*
|
||||||
|
* Usage: npm run generate:model-catalog [-- <url-or-path-to-api.json>]
|
||||||
|
*
|
||||||
|
* The runtime refresh (`packages/core/src/models/model-catalog-refresh.ts`)
|
||||||
|
* applies the same projection to whatever it downloads, so the committed
|
||||||
|
* file and the per-user cache always share one shape.
|
||||||
|
*/
|
||||||
|
|
||||||
|
import * as fs from 'node:fs';
|
||||||
|
import * as path from 'node:path';
|
||||||
|
import { fileURLToPath } from 'node:url';
|
||||||
|
|
||||||
|
import {
|
||||||
|
assertCatalogPayloadComplete,
|
||||||
|
MODELS_DEV_URL,
|
||||||
|
trimModelsDevCatalog,
|
||||||
|
type ModelsDevApi,
|
||||||
|
} from '../packages/core/src/models/model-catalog-refresh.js';
|
||||||
|
|
||||||
|
const MAX_BYTES = 200 * 1024;
|
||||||
|
|
||||||
|
const input = process.argv[2] ?? MODELS_DEV_URL;
|
||||||
|
// The provenance stamped into the committed snapshot must stay meaningful to
|
||||||
|
// its readers: a local input path is machine-specific, so stamp the
|
||||||
|
// canonical catalog URL instead — the payload is models.dev data regardless
|
||||||
|
// of where this machine happened to store it.
|
||||||
|
const source = /^https?:\/\//.test(input) ? input : MODELS_DEV_URL;
|
||||||
|
|
||||||
|
async function fetchApi(url: string): Promise<ModelsDevApi> {
|
||||||
|
const response = await fetch(url);
|
||||||
|
if (!response.ok) {
|
||||||
|
// Release the socket before throwing so the process exits cleanly.
|
||||||
|
await response.body?.cancel().catch(() => {});
|
||||||
|
throw new Error(`fetching ${url}: HTTP ${response.status}`);
|
||||||
|
}
|
||||||
|
return (await response.json()) as ModelsDevApi;
|
||||||
|
}
|
||||||
|
|
||||||
|
const api: ModelsDevApi = /^https?:\/\//.test(input)
|
||||||
|
? await fetchApi(input)
|
||||||
|
: JSON.parse(fs.readFileSync(input, 'utf8'));
|
||||||
|
|
||||||
|
// Same coverage guard the runtime refresh applies: a truncated download or
|
||||||
|
// a mirror that omits one provider must not replace the committed snapshot.
|
||||||
|
assertCatalogPayloadComplete(api);
|
||||||
|
const catalog = trimModelsDevCatalog(api, new Date().toISOString(), source);
|
||||||
|
// A 200 that projects to nothing is not a catalog (a renamed upstream field
|
||||||
|
// or a gateway error body); never overwrite the committed snapshot with it.
|
||||||
|
if (Object.keys(catalog.models).length === 0) {
|
||||||
|
throw new Error(
|
||||||
|
`no catalog entries projected from ${input}; leaving the committed snapshot untouched`,
|
||||||
|
);
|
||||||
|
}
|
||||||
|
const json = JSON.stringify(catalog, null, 2) + '\n';
|
||||||
|
const bytes = Buffer.byteLength(json);
|
||||||
|
if (bytes > MAX_BYTES) {
|
||||||
|
throw new Error(
|
||||||
|
`Trimmed catalog is ${bytes} bytes, over the ${MAX_BYTES}-byte budget; tighten MODELS_DEV_PROVIDERS or the projection.`,
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
const outputPath = path.resolve(
|
||||||
|
path.dirname(fileURLToPath(import.meta.url)),
|
||||||
|
'../packages/core/src/models/generated/model-registry.json',
|
||||||
|
);
|
||||||
|
fs.writeFileSync(outputPath, json);
|
||||||
|
console.log(
|
||||||
|
`Generated model catalog at: ${outputPath} (${Object.keys(catalog.models).length} models, ${bytes} bytes)`,
|
||||||
|
);
|
||||||
Reference in new issue
Block a user