mirror of
https://github.com/QwenLM/qwen-code.git
synced 2026-11-07 18:36:00 -10:00
docs: finish scrubbing tokenizer references after estimator-cluster removal
Follow-up to 0ee17632c7/1871bb5b81 (review round 2): - supportedImageFormats.ts header and getSupportedImageFormatsString doc no longer describe a tokenizer decode/metadata-extraction stage; the list is now documented as the vision-input acceptance list, with token accounting noted as the flat DEFAULT_IMAGE_TOKEN_ESTIMATE. - web-shell-image-drag-and-drop.md's BMP rationale no longer claims ImageTokenizer parses BMP dimensions; dated sync note added stating BMP support rests on SUPPORTED_IMAGE_MIME_TYPES plus converter passthrough since PR #9676.
This commit is contained in:
1 parent
9bf6087809
commit
18f08c0924
2 files changed
+18
-9
No files matched your search
@@ -502,9 +502,14 @@ oversized placeholder。它们验证既有服务端契约;Web Shell helper 测
|
||||
候选顺序和 encoded-data 剩余预算。
|
||||
|
||||
BMP 以 `image/bmp` 进入缩略图 data URL 和 daemon image block。Core 的
|
||||
`SUPPORTED_IMAGE_MIME_TYPES` 明确包含 `image/bmp`,`ImageTokenizer` 解析 BMP 尺寸,
|
||||
OpenAI converter 把启用 image modality 的 `inlineData` 原样构造成
|
||||
`data:image/bmp;base64,...`;Gemini 路径保留相同 `inlineData`。因此 V1 不在浏览器转码。
|
||||
`SUPPORTED_IMAGE_MIME_TYPES` 明确包含 `image/bmp`,OpenAI converter 把启用 image modality 的
|
||||
`inlineData` 原样构造成 `data:image/bmp;base64,...`;Gemini 路径保留相同 `inlineData`。
|
||||
因此 V1 不在浏览器转码。
|
||||
|
||||
> **2026-08-24 同步注记(PR #9676)**:request-tokenizer 估计器簇(含 `ImageTokenizer`
|
||||
> 及其 BMP 尺寸解析)已作为孤儿代码删除。BMP 支持现在仅依赖 `SUPPORTED_IMAGE_MIME_TYPES`
|
||||
> 接受清单与 converter 透传;token 计数使用 `compactionInputSlimming.ts` 中的固定
|
||||
> `DEFAULT_IMAGE_TOKEN_ESTIMATE`。下文对 BMP 路径的 E2E/人工验收要求不变。
|
||||
浏览器若不能解码缩略图,不影响附件数据传输,但 E2E 必须覆盖 Chromium 解码,
|
||||
Firefox/Linux 必须完成人工验收。
|
||||
|
||||
|
||||
@@ -5,11 +5,15 @@
|
||||
*/
|
||||
|
||||
/**
|
||||
* Image MIME types the image tokenizer can decode for metadata extraction.
|
||||
* This is a capability list, not an acceptance contract: the file-read path
|
||||
* only forwards a narrower set to model endpoints (see
|
||||
* PROVIDER_SAFE_IMAGE_MIME_TYPES in fileUtils.ts and #9291), so some types
|
||||
* here are omitted from requests before they ever reach the tokenizer.
|
||||
* Image MIME types accepted for vision input (attachment/thumbnail paths).
|
||||
* This is an acceptance list for inputs, not a decode-capability list:
|
||||
* token accounting uses the flat DEFAULT_IMAGE_TOKEN_ESTIMATE in
|
||||
* compactionInputSlimming.ts — the former request-tokenizer estimator
|
||||
* cluster, including ImageTokenizer and its dimension parsing, was removed
|
||||
* as orphaned in PR #9676. The file-read path forwards only the narrower
|
||||
* PIPELINE_IMAGE_MIME_TYPES subset to model endpoints (see
|
||||
* PROVIDER_SAFE_IMAGE_MIME_TYPES in fileUtils.ts and #9291); anything else
|
||||
* is omitted from requests with an in-band notice.
|
||||
*/
|
||||
export const SUPPORTED_IMAGE_MIME_TYPES = [
|
||||
'image/bmp',
|
||||
@@ -58,7 +62,7 @@ export function isSupportedImageMimeType(
|
||||
|
||||
/**
|
||||
* Get a human-readable list of image formats the pipeline forwards to the
|
||||
* model (not the tokenizer's wider decode capability).
|
||||
* model (the narrower pipeline subset, not the full acceptance list above).
|
||||
* @returns Comma-separated string of forwarded formats
|
||||
*/
|
||||
export function getSupportedImageFormatsString(): string {
|
||||
|
||||
Reference in new issue
Block a user