feat(tui): cap request images after a provider refuses the image count

A session whose history held more images than the route accepts in one
request became permanently unsendable: the provider 400 became
LlmError::InvalidRequest, the transport did not retry it, and every
later turn, compaction and restart rebuilt the same over-limit history.
When the 400 also mentioned image_url/vision, is_image_input_rejection_
message misread it as "no image input" and stripped every image.

Learn from rejection only (no proactive default, no RouteLimits,
models.dev or custom_models field in this slice): a route that never
refuses sends every image exactly as before.

- llm_client::image_count_rejection sits next to is_context_length_
  message: phrase list ("too many images", "maximum number of images",
  "max images", "images per request/prompt/message", "image(s) may be
  provided", "number of images" + exceed/more than/at most/limit, "at
  most N image") plus a stated-limit parser over the matched clause
  (at most / max is / maximum of / limit of / up to / allowed / max ...).
  It never matches a context-length message.
- model_step classifies a count rejection before recover_child_request
  and before the image-input branch. cap = stated limit when it is below
  the images sent, else sent/2, 0 when sent <= 1. A cap of 0 reuses the
  existing ImageInputRejectedResent strip-and-resend branch (now the
  shared resend_without_images helper). A cap >= 1 resends at most 3
  times per turn, each strictly smaller; then the turn ends with a typed
  ErrorEnvelope::image_count_exceeded (InvalidInput, Error,
  recoverable, code image_count_exceeded) carrying the localized
  ImageCountRejected hint plus the provider text. The image-input
  branch now also requires the rejected request to carry an image, so it
  can never resend an image-free request unchanged.
- is_image_input_rejection_message returns false for a count rejection
  unless its stated limit is 0.
- is_transient_subagent_provider_error returns false for a count
  rejection, so a child never retries one as transient (including the
  untyped HTTP/1.1 "stream request" path).
- process_stream: a mid-stream count rejection is terminal for that
  request (never resumed or transparently resent) and still teaches the
  cap.
- Learned caps live on the Engine, keyed by (provider identity key,
  model), in memory only; child engines learn their own. A learned 0
  marks image input unsupported for that route.
- image_attach::cap_request_images applies the window to the outbound
  copy right after strip_images_when_unsupported: each ImageUrl counts 1,
  each tool result carrying an image counts 1, oldest first, with
  E = 0 if N <= cap else ceil((N-cap)/K)*K, K = max(1, cap/4). Stored
  history is untouched. Placeholders are fixed bytes with neutral
  claims; the tool note says to take a fresh observation instead of
  repeating an action.
- The same pure policy (route identity + learned map) feeds the turn
  (preparation.rs), the compaction summary request (PreparedCompaction-
  Envelope.image_cap and config.image_input, threaded into create_
  summary) and /preview-request, which now also applies the strip for
  its planned route.
- Model-visible means logged: when the window (route, cap, elided)
  changes, the engine emits Event::RequestImageWindow {provider, model,
  total, elided, cap, cap_source, message} with a protocol twin
  (request_image_window in EVENT_KINDS), runtime_threads persists it as
  a Status item with metadata code request_image_window, exec stream-json
  prints it, the TUI shows it as a toast, and the prefix tracker records
  reset:image_window. The compaction window goes through the compaction
  notice sink. New MessageIds RequestImageWindow and ImageCountRejected
  are translated in all 15 packs.

KV-cache effect: the pinned system+tools prefix is never touched. The
elision point moves only once every K new images, so K-1 of every K
requests stay a pure append of the previous one; a move re-prefills from
the first changed image (Anthropic: from the history breakpoint). With
K = cap/4 the amortized re-prefill is about 3 images' worth per step
whatever the cap, against cap images per step for a naive keep-newest.

Known limits (also in CHANGELOG [Unreleased]): nothing changes until a
provider refuses once; no proactive or documented per-model limits; the
learned cap is not persisted, so a restart or new child pays one refused
request; elision is by age, user attachments before tool screenshots; a
mid-stream refusal only teaches the next send; a gateway 5xx count
refusal is retried unchanged by the transport first; a request-body
size (HTTP 413) refusal on an ordinary turn still has no recovery; a
per-message limit lowers the whole request's cap; a parse that finds no
number falls back to halving; the bounded child-report request is capped
by the window but does not learn from a refusal.

Verification:
- scripts/dev-test.sh tui image_attach llm_client error_taxonomy
  compaction preview turn_loop model_step context runtime_threads
  protocol_parity subagent exec_agent: 2183 tests run: 2183 passed,
  13168 skipped (nextest).
- scripts/dev-test.sh protocol: 75 tests run: 75 passed.
- scripts/dev-cargo.sh nextest run -p codewhale-localization --lib
  --locked: 54 tests run: 54 passed.
- CI clippy (workspace, all targets, all features, -D warnings with the
  CI allows): exit 0, no warnings; run before a rustfmt-only reformat of
  three files. cargo fmt --all -- --check: clean after that reformat.
- npm test: exit 0 (node 101/101, runtime-sdk 19/19, extension-host 413
  passed + 7 skipped of 420, web dependency-security 7/7, vitest 772/772
  in 74 files). npm run check:web: exit 0.
- sync-changelog.sh and release/check-versions.sh --require-dated-
  release: OK. npm --prefix web run prebuild: no generated change
  (docs/CACHE.md does not feed agent-docs).
- Real binary (target/debug/codewhale exec --output-format stream-json,
  isolated HOME, DEEPSEEK_BASE_URL to a local stub that returns 400
  "At most N image(s) may be provided in one prompt." when a request
  carries more than N images):
  - 6 attachments, N=3: request 1 400 with 6 image_url parts; request 2
    200 with 3 image_url parts and 3 "Image not attached" notes; no
    consecutive identical bodies; stream-json request_image_window
    {total 6, elided 3, cap 3, cap_source provider_rejection}; saved
    session still holds all 6 image_url blocks and no notes.
  - 2 attachments, N=0: request 1 400; request 2 200 with 0 image_url
    parts and 2 strip notes.
  - 16 attachments, unstated "Too many images in request." always:
    16 -> 8 -> 4 -> 2 images (all distinct bodies), then the turn ends
    with "probe-model refused the 2 image(s) in this request; Codewhale
    did not resend it unchanged..." after 3 recoveries.

Not verified: the runtime_threads Status item and the TUI toast were not
exercised in a running app-server or TUI (compile and existing tests
only); no real provider was called; the Anthropic and Responses wires
were not probed (Chat Completions only); a mid-stream count rejection
and the child-engine path were not driven end to end; the compaction
summary window was not driven by a live compaction.

Refs CW-7235

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01NBKbhx19BRbZtvWK8rVKjz
This commit is contained in:
CodeWhale BotandClaude Opus 5.5 committed 2026-10-16 13:23:01 -07:00
1 parent b095416f55
commit f0913c3f62
39 files changed
+646 -21

No files matched your search

+30
View File
@@ -7,6 +7,36 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased]
- A long session whose history holds more images than its model accepts in
one request no longer becomes permanently unsendable. When a provider
refuses the number of images (for example vLLM's "At most 3 image(s) may
be provided in one prompt", "too many images" or "maximum number of
images"), Codewhale learns that limit for the provider and model (the
number the provider states, otherwise half the images sent) and resends
with only the newest images attached. Each older image is replaced by a
fixed note telling the model it remains in the session and not to guess
what it showed; a tool image's note says to take a fresh observation, such
as a new screenshot, rather than repeat the action. Each resend carries
strictly fewer images, there are at most three per turn and never an
unchanged one; after that the turn ends with an `image_count_exceeded`
error and the next message uses the learned limit. A stated limit of 0
sends images as text, as for a model without image input. Before, a count
rejection either ended every later turn the same way or was misread as
"no image input" and stripped every image. The window moves in steps of a
quarter of the limit, so the cached history prefix changes only once every
few new images. Each change is recorded as a `request_image_window` event
(runtime threads store it as a status item with that code, `exec
--output-format stream-json` prints it and the TUI shows it), and
`/preview-request` and the compaction summary request apply the same
window. A sub-agent no longer retries a count rejection as a transient
failure. Known limits: nothing changes until a provider has refused once,
and no documented per-model limits are applied; the learned limit is kept
in memory per engine and per sub-agent, so a restart learns it again with
one refused request; the oldest images go first, user attachments
included; a refusal that arrives mid-stream ends that request and only the
next send uses the limit; a gateway that reports the refusal as a 5xx is
retried unchanged by the transport first; and a request-body size (HTTP
413) refusal on an ordinary turn still has no recovery (Refs CW-7235).
- Switching provider is now refused while a turn is running, as model,
thinking and permission changes already were. The refusal covers
`/provider <id>`, the provider picker, a custom provider, setup and login
+2
View File
@@ -370,6 +370,8 @@
"ScreenModeUnchanged": "Pantalla sense canvis: {reason}.",
"ImageInputRejectedResent": "{model} no accepta imatges — s'han reenviat com a text; usa image_ocr per llegir-les",
"ImageInputOmitted": "{model} no pot veure imatges — {count} imatge(s) s'han enviat com a text en aquest torn; usa image_ocr per llegir-les o /model per canviar",
"RequestImageWindow": "{model} accepta com a màxim {cap} imatge(s) per sol·licitud: s'envien les {kept} més recents de {total}; {elided} imatge(s) més antigues van com a notes de text",
"ImageCountRejected": "{model} ha rebutjat les {count} imatge(s) d'aquesta sol·licitud; Codewhale no l'ha tornat a enviar sense canvis. Torna a enviar per reintentar-ho amb menys imatges, o canvia de model",
"ProviderToolCallMissing": "El proveïdor ha acabat amb `{reason}`, però no ha proporcionat cap crida a una eina. Torna a provar aquest torn per continuar.",
"ConfigLabelShowThinking": "Raonament del model al xat",
"ConfigLabelThinkingHighlight": "Ressaltat de fons del raonament",
+2
View File
@@ -370,6 +370,8 @@
"ScreenModeUnchanged": "Bildschirm unverändert: {reason}.",
"ImageInputRejectedResent": "{model} akzeptiert keine Bilder — sie wurden als Text erneut gesendet; nutze image_ocr zum Lesen",
"ImageInputOmitted": "{model} kann keine Bilder sehen — {count} Bild(er) wurden in diesem Zug als Text gesendet; nutze image_ocr zum Lesen oder /model zum Wechseln",
"RequestImageWindow": "{model} akzeptiert höchstens {cap} Bild(er) pro Anfrage: Die neuesten {kept} von {total} werden gesendet; {elided} ältere Bild(er) gehen als Textnotizen mit",
"ImageCountRejected": "{model} hat die {count} Bild(er) dieser Anfrage abgelehnt; Codewhale hat sie nicht unverändert erneut gesendet. Sende erneut, um es mit weniger Bildern zu versuchen, oder wechsle das Modell",
"ProviderToolCallMissing": "Der Anbieter hat mit `{reason}` beendet, aber keinen Werkzeugaufruf geliefert. Wiederhole diesen Durchgang, um fortzufahren.",
"ConfigLabelShowThinking": "Modell-Reasoning im Chat",
"ConfigLabelThinkingHighlight": "Reasoning-Hintergrund hervorheben",
+2
View File
@@ -371,6 +371,8 @@
"ScreenModeUnchanged": "Screen unchanged: {reason}.",
"ImageInputRejectedResent": "{model} does not accept images — resent as text; use image_ocr to read them",
"ImageInputOmitted": "{model} can't see images — {count} image(s) sent as text this turn; use image_ocr to read them or /model to switch",
"RequestImageWindow": "{model} accepts at most {cap} image(s) per request: sending the newest {kept} of {total}; {elided} older image(s) go as text notes",
"ImageCountRejected": "{model} refused the {count} image(s) in this request; Codewhale did not resend it unchanged. Send again to retry with fewer images, or switch models",
"ProviderToolCallMissing": "Provider ended with `{reason}` but supplied no tool call. Retry the turn to continue.",
"ConfigLabelShowThinking": "Model reasoning in chat",
"ConfigLabelThinkingHighlight": "Thinking background highlight",
+2
View File
@@ -370,6 +370,8 @@
"ScreenModeUnchanged": "Pantalla sin cambios: {reason}.",
"ImageInputRejectedResent": "{model} no acepta imágenes — se reenviaron como texto; usa image_ocr para leerlas",
"ImageInputOmitted": "{model} no puede ver imágenes — {count} imagen(es) se enviaron como texto en este turno; usa image_ocr para leerlas o /model para cambiar",
"RequestImageWindow": "{model} acepta como máximo {cap} imagen(es) por solicitud: se envían las {kept} más recientes de {total}; {elided} imagen(es) más antiguas van como notas de texto",
"ImageCountRejected": "{model} rechazó las {count} imagen(es) de esta solicitud; Codewhale no la reenvió sin cambios. Envía de nuevo para reintentar con menos imágenes, o cambia de modelo",
"ProviderToolCallMissing": "El proveedor terminó con `{reason}`, pero no incluyó ninguna llamada a herramientas. Vuelve a intentar este turno para continuar.",
"ConfigLabelShowThinking": "Razonamiento del modelo en el chat",
"ConfigLabelThinkingHighlight": "Resaltado de fondo del razonamiento",
+2
View File
@@ -370,6 +370,8 @@
"ScreenModeUnchanged": "Écran inchangé : {reason}.",
"ImageInputRejectedResent": "{model} n’accepte pas les images — elles ont été renvoyées sous forme de texte ; utilisez image_ocr pour les lire",
"ImageInputOmitted": "{model} ne voit pas les images — {count} image(s) envoyée(s) sous forme de texte à ce tour ; utilisez image_ocr pour les lire ou /model pour changer",
"RequestImageWindow": "{model} accepte au plus {cap} image(s) par requête : envoi des {kept} plus récentes sur {total} ; {elided} image(s) plus ancienne(s) partent sous forme de notes texte",
"ImageCountRejected": "{model} a refusé les {count} image(s) de cette requête ; Codewhale ne l'a pas renvoyée telle quelle. Renvoyez pour réessayer avec moins d'images, ou changez de modèle",
"ProviderToolCallMissing": "Le fournisseur a terminé avec `{reason}`, mais sans fournir d’appel d’outil. Réessayez ce tour pour continuer.",
"ConfigLabelShowThinking": "Raisonnement du modèle dans le chat",
"ConfigLabelThinkingHighlight": "Surlignage du fond du raisonnement",
+2
View File
@@ -370,6 +370,8 @@
"ScreenModeUnchanged": "स्क्रीन अपरिवर्तित: {reason}।",
"ImageInputRejectedResent": "{model} इमेज स्वीकार नहीं करता — उन्हें टेक्स्ट के रूप में फिर भेजा गया; पढ़ने के लिए image_ocr का उपयोग करें",
"ImageInputOmitted": "{model} इमेज नहीं देख सकता — इस टर्न में {count} इमेज टेक्स्ट के रूप में भेजी गईं; पढ़ने के लिए image_ocr या बदलने के लिए /model का उपयोग करें",
"RequestImageWindow": "{model} प्रति अनुरोध अधिकतम {cap} इमेज स्वीकार करता है: {total} में से नवीनतम {kept} भेजी जा रही हैं; {elided} पुरानी इमेज टेक्स्ट नोट के रूप में जाती हैं",
"ImageCountRejected": "{model} ने इस अनुरोध की {count} इमेज अस्वीकार कर दीं; Codewhale ने इसे बिना बदलाव के दोबारा नहीं भेजा। कम इमेज के साथ फिर से प्रयास करने के लिए दोबारा भेजें, या मॉडल बदलें",
"ProviderToolCallMissing": "प्रदाता ने `{reason}` के साथ उत्तर समाप्त किया, लेकिन कोई टूल कॉल नहीं दिया। जारी रखने के लिए इस टर्न को फिर से आज़माएँ।",
"ConfigLabelShowThinking": "चैट में मॉडल रीज़निंग",
"ConfigLabelThinkingHighlight": "रीज़निंग पृष्ठभूमि हाइलाइट",
+2
View File
@@ -370,6 +370,8 @@
"ScreenModeUnchanged": "Layar tidak berubah: {reason}.",
"ImageInputRejectedResent": "{model} tidak menerima gambar — gambar dikirim ulang sebagai teks; gunakan image_ocr untuk membacanya",
"ImageInputOmitted": "{model} tidak dapat melihat gambar — {count} gambar dikirim sebagai teks pada giliran ini; gunakan image_ocr untuk membacanya atau /model untuk beralih",
"RequestImageWindow": "{model} menerima paling banyak {cap} gambar per permintaan: mengirim {kept} terbaru dari {total}; {elided} gambar lama dikirim sebagai catatan teks",
"ImageCountRejected": "{model} menolak {count} gambar dalam permintaan ini; Codewhale tidak mengirimnya ulang tanpa perubahan. Kirim lagi untuk mencoba dengan lebih sedikit gambar, atau ganti model",
"ProviderToolCallMissing": "Penyedia mengakhiri respons dengan `{reason}`, tetapi tidak menyertakan panggilan alat. Coba ulang giliran ini untuk melanjutkan.",
"ConfigLabelShowThinking": "Penalaran model di chat",
"ConfigLabelThinkingHighlight": "Sorotan latar penalaran",
+2
View File
@@ -370,6 +370,8 @@
"ScreenModeUnchanged": "画面は変更されませんでした:{reason}。",
"ImageInputRejectedResent": "{model} は画像を受け付けないため、画像をテキストとして再送しました。画像を読むには image_ocr を使用してください",
"ImageInputOmitted": "{model} は画像を認識できないため、このターンの画像 {count} 件をテキストとして送信しました。読むには image_ocr、切り替えるには /model を使用してください",
"RequestImageWindow": "{model} は 1 リクエストあたり最大 {cap} 件の画像しか受け付けません。{total} 件中の新しい {kept} 件を送信し、古い {elided} 件はテキストの注記として送ります",
"ImageCountRejected": "{model} がこのリクエストの画像 {count} 件を拒否しました。Codewhale は同じ内容のまま再送していません。もう一度送信すると画像を減らして再試行します。またはモデルを切り替えてください",
"ProviderToolCallMissing": "プロバイダーは `{reason}` で応答を終了しましたが、ツール呼び出しがありません。続けるにはこのターンを再試行してください。",
"ConfigLabelShowThinking": "会話にモデル推論を表示",
"ConfigLabelThinkingHighlight": "推論の背景ハイライト",
+2
View File
@@ -370,6 +370,8 @@
"ScreenModeUnchanged": "화면 변경 없음: {reason}.",
"ImageInputRejectedResent": "{model}은(는) 이미지를 지원하지 않아 텍스트로 다시 보냈습니다. image_ocr를 사용해 읽으세요",
"ImageInputOmitted": "{model}은(는) 이미지를 볼 수 없어 이번 턴의 이미지 {count}개를 텍스트로 보냈습니다. image_ocr로 읽거나 /model로 전환하세요",
"RequestImageWindow": "{model}은(는) 요청당 최대 {cap}개의 이미지만 받습니다. {total}개 중 최신 {kept}개를 보내고, 오래된 {elided}개는 텍스트 메모로 보냅니다",
"ImageCountRejected": "{model}이(가) 이 요청의 이미지 {count}개를 거부했습니다. Codewhale은 변경 없이 다시 보내지 않았습니다. 다시 보내면 더 적은 이미지로 재시도하며, 모델을 바꿀 수도 있습니다",
"ProviderToolCallMissing": "공급자가 `{reason}`으로 응답을 종료했지만 도구 호출을 제공하지 않았습니다. 계속하려면 이 턴을 다시 시도하세요.",
"ConfigLabelShowThinking": "채팅에 모델 추론 표시",
"ConfigLabelThinkingHighlight": "추론 배경 강조",
+2
View File
@@ -370,6 +370,8 @@
"ScreenModeUnchanged": "Tela inalterada: {reason}.",
"ImageInputRejectedResent": "{model} não aceita imagens — reenviado como texto; use image_ocr para lê-las",
"ImageInputOmitted": "{model} não consegue ver imagens — {count} imagem(ns) enviada(s) como texto neste turno; use image_ocr para lê-las ou /model para trocar",
"RequestImageWindow": "{model} aceita no máximo {cap} imagem(ns) por solicitação: enviando as {kept} mais recentes de {total}; {elided} imagem(ns) mais antiga(s) vão como notas de texto",
"ImageCountRejected": "{model} recusou as {count} imagem(ns) desta solicitação; o Codewhale não a reenviou sem alterações. Envie novamente para tentar com menos imagens, ou troque de modelo",
"ProviderToolCallMissing": "O provedor encerrou com `{reason}`, mas não forneceu nenhuma chamada de ferramenta. Tente este turno novamente para continuar.",
"ConfigLabelShowThinking": "Raciocínio do modelo no chat",
"ConfigLabelThinkingHighlight": "Destaque do fundo do raciocínio",
+2
View File
@@ -370,6 +370,8 @@
"ScreenModeUnchanged": "Экран без изменений: {reason}.",
"ImageInputRejectedResent": "{model} не принимает изображения — они отправлены повторно как текст; используйте image_ocr для чтения",
"ImageInputOmitted": "{model} не видит изображения — {count} изображ. в этом ходе отправлено как текст; используйте image_ocr для чтения или /model для смены модели",
"RequestImageWindow": "{model} принимает не более {cap} изображ. за запрос: отправляются {kept} новейших из {total}; {elided} старых изображ. уходят как текстовые заметки",
"ImageCountRejected": "{model} отклонил {count} изображ. в этом запросе; Codewhale не отправил его повторно без изменений. Отправьте снова, чтобы повторить с меньшим числом изображений, или смените модель",
"ProviderToolCallMissing": "Провайдер завершил ответ с `{reason}`, но не передал вызов инструмента. Повторите этот ход, чтобы продолжить.",
"ConfigLabelShowThinking": "Рассуждения модели в чате",
"ConfigLabelThinkingHighlight": "Подсветка фона рассуждений",
+2
View File
@@ -370,6 +370,8 @@
"ScreenModeUnchanged": "Екран без змін: {reason}.",
"ImageInputRejectedResent": "{model} не приймає зображення — їх повторно надіслано як текст; скористайтеся image_ocr, щоб їх прочитати",
"ImageInputOmitted": "{model} не бачить зображень — {count} зображ. у цьому ході надіслано як текст; скористайтеся image_ocr, щоб прочитати, або /model, щоб змінити модель",
"RequestImageWindow": "{model} приймає не більше {cap} зображ. за запит: надсилаються {kept} найновіших із {total}; {elided} старіших зображ. йдуть як текстові нотатки",
"ImageCountRejected": "{model} відхилив {count} зображ. у цьому запиті; Codewhale не надіслав його повторно без змін. Надішліть ще раз, щоб повторити з меншою кількістю зображень, або змініть модель",
"ProviderToolCallMissing": "Провайдер завершив відповідь із `{reason}`, але не передав виклик інструмента. Повторіть цей хід, щоб продовжити.",
"ConfigLabelShowThinking": "Міркування моделі в чаті",
"ConfigLabelThinkingHighlight": "Підсвічування тла міркувань",
+2
View File
@@ -370,6 +370,8 @@
"ScreenModeUnchanged": "Màn hình không thay đổi: {reason}.",
"ImageInputRejectedResent": "{model} không chấp nhận hình ảnh — đã gửi lại dưới dạng văn bản; hãy dùng image_ocr để đọc chúng",
"ImageInputOmitted": "{model} không thể xem hình ảnh — {count} hình ảnh đã được gửi dưới dạng văn bản trong lượt này; hãy dùng image_ocr để đọc hoặc /model để chuyển",
"RequestImageWindow": "{model} chỉ nhận tối đa {cap} hình ảnh mỗi yêu cầu: gửi {kept} hình mới nhất trong {total}; {elided} hình cũ hơn được gửi dưới dạng ghi chú văn bản",
"ImageCountRejected": "{model} đã từ chối {count} hình ảnh trong yêu cầu này; Codewhale không gửi lại nguyên trạng. Gửi lại để thử với ít hình ảnh hơn, hoặc đổi mô hình",
"ProviderToolCallMissing": "Nhà cung cấp kết thúc với `{reason}` nhưng không gửi lệnh gọi công cụ nào. Hãy thử lại lượt này để tiếp tục.",
"ConfigLabelShowThinking": "Hiển thị suy luận trong trò chuyện",
"ConfigLabelThinkingHighlight": "Tô nền phần suy luận",
+2
View File
@@ -371,6 +371,8 @@
"ScreenModeUnchanged": "屏幕未改变:{reason}。",
"ImageInputRejectedResent": "{model} 不接受图像,已将图像替换为文本重新发送;请使用 image_ocr 读取图像",
"ImageInputOmitted": "{model} 无法查看图像,本回合 {count} 张图像已作为文本发送;请使用 image_ocr 读取,或用 /model 切换模型",
"RequestImageWindow": "{model} 每个请求最多接受 {cap} 张图像:发送 {total} 张中最新的 {kept} 张;较早的 {elided} 张以文本说明代替",
"ImageCountRejected": "{model} 拒绝了此请求中的 {count} 张图像;Codewhale 没有原样重发。再次发送将以更少的图像重试,或切换模型",
"ProviderToolCallMissing": "提供商以 `{reason}` 结束响应,但未提供工具调用。请重试本回合以继续。",
"ConfigLabelShowThinking": "对话中显示模型推理",
"ConfigLabelThinkingHighlight": "思考背景高亮",
+2
View File
@@ -369,6 +369,8 @@
"ScreenModeUnchanged": "畫面未變更:{reason}。",
"ImageInputRejectedResent": "{model} 不接受圖片,已將圖片替換為文字重新傳送;請使用 image_ocr 讀取圖片",
"ImageInputOmitted": "{model} 無法檢視圖片,本回合 {count} 張圖片已作為文字傳送;請使用 image_ocr 讀取,或用 /model 切換模型",
"RequestImageWindow": "{model} 每個請求最多接受 {cap} 張圖片:傳送 {total} 張中最新的 {kept} 張;較早的 {elided} 張以文字說明代替",
"ImageCountRejected": "{model} 拒絕了此請求中的 {count} 張圖片;Codewhale 沒有原樣重送。再次傳送將以更少的圖片重試,或切換模型",
"ProviderToolCallMissing": "供應商以 `{reason}` 結束回應,但未提供工具呼叫。請重試本回合以繼續。",
"ConfigLabelShowThinking": "對話中顯示模型推理",
"ConfigLabelThinkingHighlight": "思考背景醒目提示",
+4
View File
@@ -190,6 +190,8 @@ pub enum MessageId {
ScreenModeUnchanged,
ImageInputRejectedResent,
ImageInputOmitted,
RequestImageWindow,
ImageCountRejected,
ProviderToolCallMissing,
// First-run anonymous usage disclosure.
TelemetryNoticeDefaultOn,
@@ -2867,6 +2869,8 @@ pub const ALL_MESSAGE_IDS: &[MessageId] = &[
MessageId::ScreenModeUnchanged,
MessageId::ImageInputRejectedResent,
MessageId::ImageInputOmitted,
MessageId::RequestImageWindow,
MessageId::ImageCountRejected,
MessageId::ProviderToolCallMissing,
MessageId::TelemetryNoticeDefaultOn,
MessageId::TelemetryNoticeHeadline,
+27
View File
@@ -300,6 +300,18 @@ pub enum EventMsg {
omitted_tool_count: u64,
},
RequestImageWindow {
thread_id: ThreadId,
session_id: SessionId,
provider: String,
model: String,
total: u64,
elided: u64,
cap: u64,
cap_source: String,
message: String,
},
/// Workspace snapshots (undo) are off for this workspace. `reason` is one
/// rendered, localized line: the consequence, the gate that refused, and
/// the recovery that actually lifts *that* gate (the size cap's config key
@@ -800,6 +812,7 @@ pub struct EventEnvelope {
/// proves the list and `kind_str` agree.
pub const EVENT_KINDS: &[&str] = &[
"tool_projection_warning",
"request_image_window",
"message_started",
"response_delta",
"message_complete",
@@ -856,6 +869,7 @@ impl EventMsg {
pub fn kind_str(&self) -> &'static str {
match self {
Self::ToolProjectionWarning { .. } => "tool_projection_warning",
Self::RequestImageWindow { .. } => "request_image_window",
Self::SnapshotsDisabled { .. } => "snapshots_disabled",
Self::MessageStarted { .. } => "message_started",
Self::ResponseDelta { .. } => "response_delta",
@@ -915,6 +929,7 @@ impl EventMsg {
pub fn thread_id(&self) -> &ThreadId {
match self {
Self::ToolProjectionWarning { thread_id, .. }
| Self::RequestImageWindow { thread_id, .. }
| Self::SnapshotsDisabled { thread_id, .. }
| Self::MessageStarted { thread_id, .. }
| Self::ResponseDelta { thread_id, .. }
@@ -974,6 +989,7 @@ impl EventMsg {
pub fn session_id(&self) -> &SessionId {
match self {
Self::ToolProjectionWarning { session_id, .. }
| Self::RequestImageWindow { session_id, .. }
| Self::SnapshotsDisabled { session_id, .. }
| Self::MessageStarted { session_id, .. }
| Self::ResponseDelta { session_id, .. }
@@ -1081,6 +1097,17 @@ mod tests {
omitted_tool_names: vec!["a".into()],
omitted_tool_count: 3,
},
EventMsg::RequestImageWindow {
thread_id: t.clone(),
session_id: s.clone(),
provider: "vllm".into(),
model: "m".into(),
total: 6,
elided: 3,
cap: 3,
cap_source: "provider_rejection".into(),
message: "m".into(),
},
EventMsg::MessageStarted {
thread_id: t.clone(),
session_id: s.clone(),
+30
View File
@@ -7,6 +7,36 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased]
- A long session whose history holds more images than its model accepts in
one request no longer becomes permanently unsendable. When a provider
refuses the number of images (for example vLLM's "At most 3 image(s) may
be provided in one prompt", "too many images" or "maximum number of
images"), Codewhale learns that limit for the provider and model (the
number the provider states, otherwise half the images sent) and resends
with only the newest images attached. Each older image is replaced by a
fixed note telling the model it remains in the session and not to guess
what it showed; a tool image's note says to take a fresh observation, such
as a new screenshot, rather than repeat the action. Each resend carries
strictly fewer images, there are at most three per turn and never an
unchanged one; after that the turn ends with an `image_count_exceeded`
error and the next message uses the learned limit. A stated limit of 0
sends images as text, as for a model without image input. Before, a count
rejection either ended every later turn the same way or was misread as
"no image input" and stripped every image. The window moves in steps of a
quarter of the limit, so the cached history prefix changes only once every
few new images. Each change is recorded as a `request_image_window` event
(runtime threads store it as a status item with that code, `exec
--output-format stream-json` prints it and the TUI shows it), and
`/preview-request` and the compaction summary request apply the same
window. A sub-agent no longer retries a count rejection as a transient
failure. Known limits: nothing changes until a provider has refused once,
and no documented per-model limits are applied; the learned limit is kept
in memory per engine and per sub-agent, so a restart learns it again with
one refused request; the oldest images go first, user attachments
included; a refusal that arrives mid-stream ends that request and only the
next send uses the limit; a gateway that reports the refusal as a 5xx is
retried unchanged by the transport first; and a request-body size (HTTP
413) refusal on an ordinary turn still has no recovery (Refs CW-7235).
- Switching provider is now refused while a turn is running, as model,
thinking and permission changes already were. The refusal covers
`/provider <id>`, the provider picker, a custom provider, setup and login
+23
View File
@@ -105,6 +105,7 @@ pub struct PreparedCompactionEnvelope {
/// request that omits it shares no cacheable prefix with the turn it
/// summarizes and re-bills the whole history uncached.
pub reasoning_effort: Option<String>,
pub image_cap: Option<usize>,
/// User-visible progress sink supplied by the host that owns the run
/// (the interactive engine, typically). `None` keeps every notice in the
/// log only — the pass itself never depends on it.
@@ -118,6 +119,7 @@ impl std::fmt::Debug for PreparedCompactionEnvelope {
.field("session_id", &self.session_id)
.field("tools", &self.tools)
.field("reasoning_effort", &self.reasoning_effort)
.field("image_cap", &self.image_cap)
.field("notice_sink", &self.notice_sink.as_ref().map(|_| "<sink>"))
.finish()
}
@@ -131,6 +133,7 @@ impl PartialEq for PreparedCompactionEnvelope {
&& self.session_id == other.session_id
&& self.tools == other.tools
&& self.reasoning_effort == other.reasoning_effort
&& self.image_cap == other.image_cap
}
}
@@ -142,6 +145,7 @@ impl PreparedCompactionEnvelope {
session_id: None,
tools: None,
reasoning_effort: None,
image_cap: None,
notice_sink: None,
}
}
@@ -1447,6 +1451,7 @@ pub async fn compact_messages_safe(
system_prompt,
prepared.tools.as_deref(),
prepared.reasoning_effort.as_deref(),
prepared.image_cap,
prepared.notice_sink.as_deref(),
&mut quality_retries,
invocation_usage,
@@ -1613,6 +1618,7 @@ async fn compact_messages(
None,
None,
None,
None,
&mut quality_retries,
&mut invocation_usage,
)
@@ -1627,6 +1633,7 @@ async fn compact_messages_with_metadata(
system_prompt: Option<&SystemPrompt>,
tools: Option<&[Tool]>,
reasoning_effort: Option<&str>,
image_cap: Option<usize>,
notice_sink: Option<&dyn CompactionNoticeSink>,
quality_retries: &mut u32,
invocation_usage: &mut Usage,
@@ -1642,6 +1649,7 @@ async fn compact_messages_with_metadata(
system_prompt,
tools,
reasoning_effort,
image_cap,
notice_sink,
quality_retries,
invocation_usage,
@@ -1877,6 +1885,7 @@ async fn create_summary(
system_prompt: Option<&SystemPrompt>,
tools: Option<&[Tool]>,
reasoning_effort: Option<&str>,
image_cap: Option<usize>,
notice_sink: Option<&dyn CompactionNoticeSink>,
quality_retries: &mut u32,
invocation_usage: &mut Usage,
@@ -1895,6 +1904,20 @@ async fn create_summary(
"Compaction omitted {stripped_images} image block(s) unsupported by its route"
));
}
if let Some(cap) = image_cap {
let window = crate::image_attach::cap_request_images(&mut request_messages, cap);
if window.elided > 0 {
let message = format!(
"Making room: the summary request sends the newest {} of {} images; {} older go as text notes (limit {} per request).",
window.total - window.elided,
window.total,
window.elided,
window.cap,
);
logging::warn(&message);
deliver_compaction_notice(notice_sink, message);
}
}
request_messages.push(Message {
role: Role::User,
content: vec![ContentBlock::Text {
+95
View File
@@ -1053,6 +1053,8 @@ pub struct Engine {
/// key, and wire protocol. Part of the route identity the input-bill
/// carry-over is keyed on; `None` until a route is installed.
active_route_endpoint: Option<codewhale_config::route::ResolvedEndpoint>,
learned_image_caps: HashMap<(String, String), usize>,
last_image_window: Option<(String, String, usize, usize)>,
rx_op: mpsc::Receiver<Op>,
live_runtime_authority: Arc<StdMutex<LiveRuntimeAuthorityState>>,
compaction_cancellation: Arc<StdMutex<CompactionCancellationState>>,
@@ -1748,6 +1750,97 @@ impl Engine {
format!("{message}\n\n{hint}")
}
pub(super) fn image_route_provider_key(&self) -> String {
self.api_provider_identity.as_ref().map_or_else(
|| self.api_provider.as_str().to_string(),
|identity| identity.key.to_string(),
)
}
pub(super) fn request_image_policy(
&self,
provider: &str,
model: &str,
route_image_input: CapabilityState,
) -> (CapabilityState, Option<usize>) {
match self
.learned_image_caps
.get(&(provider.to_string(), model.to_string()))
{
Some(0) => (CapabilityState::Unsupported, None),
cap => (route_image_input, cap.copied()),
}
}
pub(super) fn learn_image_cap(
&mut self,
model: &str,
sent: usize,
rejection: crate::llm_client::ImageCountRejection,
) -> Option<usize> {
if sent == 0 {
return None;
}
let cap = match rejection.limit {
Some(limit) if limit < sent => limit,
_ if sent <= 1 => 0,
_ => sent / 2,
};
let key = (self.image_route_provider_key(), model.to_string());
let learned = self.learned_image_caps.entry(key).or_insert(cap);
*learned = (*learned).min(cap);
crate::logging::warn(format!(
"request_images: model {model} refused {sent} image(s) in one request (stated limit {:?}); learned cap {}",
rejection.limit, *learned
));
Some(*learned)
}
pub(super) async fn note_request_image_window(
&mut self,
provider: String,
window: crate::image_attach::ImageWindow,
) {
let model = self.session.model.clone();
let current = (window.elided > 0)
.then(|| (provider.clone(), model.clone(), window.cap, window.elided));
if current == self.last_image_window {
return;
}
self.last_image_window = current;
if window.elided == 0 {
return;
}
let cap_source = "provider_rejection";
crate::logging::warn(format!(
"request_images: provider={provider} model={model} total={} elided={} cap={} cap_source={cap_source}",
window.total, window.elided, window.cap
));
if let Some(pm) = self.session.prefix_stability.as_mut() {
pm.note_history_reset("image_window");
}
let message = codewhale_localization::tr(
codewhale_localization::resolve_locale(&self.config.locale_tag),
codewhale_localization::MessageId::RequestImageWindow,
)
.replace("{model}", &model)
.replace("{cap}", &window.cap.to_string())
.replace("{kept}", &(window.total - window.elided).to_string())
.replace("{total}", &window.total.to_string())
.replace("{elided}", &window.elided.to_string());
let _ = self
.send_event(Event::RequestImageWindow {
provider,
model,
total: window.total,
elided: window.elided,
cap: window.cap,
cap_source: cap_source.to_string(),
message,
})
.await;
}
/// A provider's input bill describes one route's tokenization of one
/// prompt. The auto-compaction gate and the preflight guard lift the
/// honest estimate to the last bill, so a bill carried across a route
@@ -2361,6 +2454,8 @@ impl Engine {
active_route_limits,
active_route_capabilities: codewhale_config::route::RouteCapabilities::default(),
active_route_endpoint: None,
learned_image_caps: HashMap::new(),
last_image_window: None,
rx_op,
live_runtime_authority: Arc::clone(&live_runtime_authority),
compaction_cancellation: Arc::clone(&compaction_cancellation),
+7
View File
@@ -228,7 +228,14 @@ impl Engine {
config
.workspace
.get_or_insert_with(|| self.config.workspace.clone());
let (image_input, image_cap) = self.request_image_policy(
&self.image_route_provider_key(),
&config.model,
config.image_input,
);
config.image_input = image_input;
let mut prepared = PreparedCompactionEnvelope::new(config);
prepared.image_cap = image_cap;
prepared.session_id = Some(self.session.id.clone());
prepared.notice_sink = Some(std::sync::Arc::new(EngineCompactionNoticeSink {
tx: self.tx_event.clone(),
+5
View File
@@ -872,6 +872,11 @@ pub(super) fn context_does_not_fit_message(
pub(super) fn is_image_input_rejection_message(message: &str) -> bool {
let lower = message.to_lowercase();
if crate::llm_client::image_count_rejection(&lower)
.is_some_and(|rejection| rejection.limit != Some(0))
{
return false;
}
let image_signal = lower.contains("image_url")
|| lower.contains("content.type")
|| lower.contains("content type")
+14 -1
View File
@@ -465,7 +465,7 @@ impl Engine {
system_prompt.as_ref(),
);
let request = prepare_primary_turn_request(PrimaryTurnRequest {
let mut request = prepare_primary_turn_request(PrimaryTurnRequest {
model: model.clone(),
messages: outbound_messages,
max_tokens: effective_max_output_tokens_for_route(provider, &model, limits),
@@ -474,6 +474,19 @@ impl Engine {
tool_choice: tool_choice.clone(),
reasoning_effort: effective_reasoning_effort,
});
let (image_input, image_cap) = self.request_image_policy(
route.identity.key.as_str(),
&model,
route_context.capabilities.image_input,
);
crate::image_attach::strip_images_when_unsupported(
&mut request.messages,
image_input,
&model,
);
if let Some(cap) = image_cap {
crate::image_attach::cap_request_images(&mut request.messages, cap);
}
let prepared = match route.client.prepare_outbound_request(request, true) {
Ok(prepared) => {
+24
View File
@@ -28,6 +28,7 @@ use codewhale_core::request::{PrimaryTurnRequest, prepare_primary_turn_request};
use codewhale_models::Role;
const MAX_APPROVAL_INTENT_SUMMARY_CHARS: usize = 2_000;
const MAX_IMAGE_COUNT_RECOVERIES: u8 = 3;
// Private bookkeeping for this one outer loop. The existing TurnContext,
// Engine session, clocks, event queue, prompt and approval owners remain authoritative.
@@ -38,6 +39,7 @@ struct TurnLoopProgress {
context_recovery_attempts: u8,
auto_compaction_suppressed: bool,
image_rejection_recovered: bool,
image_count_recoveries: u8,
mode: AppMode,
tool_catalog: Vec<codewhale_models::Tool>,
active_tool_names: std::collections::HashSet<String>,
@@ -1564,6 +1566,7 @@ impl Engine {
context_recovery_attempts,
auto_compaction_suppressed,
image_rejection_recovered,
image_count_recoveries: 0,
mode,
tool_catalog,
active_tool_names,
@@ -4550,6 +4553,27 @@ impl Engine {
let message = self.decorate_auth_error_message(e.to_string());
let user_message =
stream_read_error_user_message(&message, any_content_received);
if self.rlm_host.is_none()
&& let Some(rejection) = crate::llm_client::image_count_rejection(
&format!("{e:#}").to_lowercase(),
)
{
let sent =
crate::image_attach::count_request_images(&stream_request.messages);
let model = self.session.model.clone();
if self.learn_image_cap(&model, sent, rejection).is_some() {
terminal_stream_error = true;
stream_error.get_or_insert(user_message.clone());
let _ = self
.send_stream_event(Event::error(
crate::error_taxonomy::ErrorEnvelope::image_count_exceeded(
user_message,
),
))
.await;
break;
}
}
let envelope =
crate::error_taxonomy::envelope_for_llm_error(e, user_message.clone());
if self.child_host.is_some() {
@@ -175,6 +175,15 @@ impl Engine {
Ok(false) => {}
}
}
if self.rlm_host.is_none()
&& let Some(rejection) =
crate::llm_client::image_count_rejection(&format!("{e:#}").to_lowercase())
&& let Some(result) = self
.recover_image_count_rejection(progress, &stream_request, rejection, &e)
.await
{
return result;
}
if let Some(recovery) = self.recover_child_request(progress, &e).await {
return recovery;
}
@@ -201,23 +210,9 @@ impl Engine {
&& is_image_input_rejection_message(&message)
&& self.active_route_capabilities.image_input != CapabilityState::Unsupported
&& !progress.image_rejection_recovered
&& crate::image_attach::count_request_images(&stream_request.messages) > 0
{
progress.image_rejection_recovered = true;
// This path tells the user itself; the resend must
// not announce the same omission a second time.
progress.image_omission_notified = true;
self.active_route_capabilities.image_input = CapabilityState::Unsupported;
crate::logging::warn(format!(
"model {} rejected image content; resending with images replaced by text",
self.session.model
));
let status = codewhale_localization::tr(
codewhale_localization::resolve_locale(&self.config.locale_tag),
codewhale_localization::MessageId::ImageInputRejectedResent,
)
.replace("{model}", &self.session.model);
let _ = self.send_event(Event::status(status)).await;
return PhaseResult::Retry;
return self.resend_without_images(progress).await;
}
let display_message = self.decorate_auth_error_message(
initial_stream_error_user_message(&self.config.locale_tag, &e),
@@ -1040,6 +1035,71 @@ impl Engine {
prepared_output_tokens: stream_request.max_tokens,
})
}
async fn resend_without_images(
&mut self,
progress: &mut TurnLoopProgress,
) -> PhaseResult<AcceptedModelStep> {
progress.image_rejection_recovered = true;
// This path tells the user itself; the resend must
// not announce the same omission a second time.
progress.image_omission_notified = true;
self.active_route_capabilities.image_input = CapabilityState::Unsupported;
crate::logging::warn(format!(
"model {} rejected image content; resending with images replaced by text",
self.session.model
));
let status = codewhale_localization::tr(
codewhale_localization::resolve_locale(&self.config.locale_tag),
codewhale_localization::MessageId::ImageInputRejectedResent,
)
.replace("{model}", &self.session.model);
let _ = self.send_event(Event::status(status)).await;
PhaseResult::Retry
}
async fn recover_image_count_rejection(
&mut self,
progress: &mut TurnLoopProgress,
request: &codewhale_models::MessageRequest,
rejection: crate::llm_client::ImageCountRejection,
error: &anyhow::Error,
) -> Option<PhaseResult<AcceptedModelStep>> {
let sent = crate::image_attach::count_request_images(&request.messages);
let model = self.session.model.clone();
let cap = self.learn_image_cap(&model, sent, rejection)?;
if cap == 0 {
if self.active_route_capabilities.image_input != CapabilityState::Unsupported
&& !progress.image_rejection_recovered
{
return Some(self.resend_without_images(progress).await);
}
} else if progress.image_count_recoveries < MAX_IMAGE_COUNT_RECOVERIES {
progress.image_count_recoveries = progress.image_count_recoveries.saturating_add(1);
return Some(PhaseResult::Retry);
}
let hint = codewhale_localization::tr(
codewhale_localization::resolve_locale(&self.config.locale_tag),
codewhale_localization::MessageId::ImageCountRejected,
)
.replace("{model}", &model)
.replace("{count}", &sent.to_string());
let detail = self.decorate_auth_error_message(initial_stream_error_user_message(
&self.config.locale_tag,
error,
));
let display_message = format!("{hint}\n{detail}");
progress.turn_error = Some(display_message.clone());
let _ = self
.send_event(Event::error(
crate::error_taxonomy::ErrorEnvelope::image_count_exceeded(display_message),
))
.await;
Some(PhaseResult::Return((
TurnOutcomeStatus::Failed,
progress.turn_error.take(),
)))
}
async fn recover_child_request(
&mut self,
progress: &mut TurnLoopProgress,
@@ -608,9 +608,15 @@ impl Engine {
// outbound copy is rewritten, and it is rewritten to text that says
// why rather than being dropped.
let fresh_images = crate::image_attach::images_since_last_user_prompt(&request.messages);
let image_provider = self.image_route_provider_key();
let (image_input, image_cap) = self.request_image_policy(
&image_provider,
&self.session.model,
self.active_route_capabilities.image_input,
);
let stripped_images = crate::image_attach::strip_images_when_unsupported(
&mut request.messages,
self.active_route_capabilities.image_input,
image_input,
&self.session.model,
);
if stripped_images > 0 {
@@ -629,6 +635,10 @@ impl Engine {
let _ = self.send_event(Event::status(status)).await;
}
}
if let Some(cap) = image_cap {
let window = crate::image_attach::cap_request_images(&mut request.messages, cap);
self.note_request_image_window(image_provider, window).await;
}
let tool_request_snapshot =
crate::tool_inspection::ToolInspectionSnapshot::from_prepared_request_with_surface(
&turn.id,
+10
View File
@@ -208,6 +208,16 @@ pub enum Event {
omitted_tool_count: usize,
},
RequestImageWindow {
provider: String,
model: String,
total: usize,
elided: usize,
cap: usize,
cap_source: String,
message: String,
},
/// Workspace snapshots (undo) could not be enabled for this workspace.
/// Emitted once per session/workspace so another session cannot consume
/// its notice. The disabled state also remains visible in `/status` (#5930).
+19
View File
@@ -454,6 +454,25 @@ pub fn event_to_protocol(event: &Event, ids: &ProtocolIds) -> wire::EventMsg {
omitted_tool_names: omitted_tool_names.clone(),
omitted_tool_count: count(*omitted_tool_count),
},
Event::RequestImageWindow {
provider,
model,
total,
elided,
cap,
cap_source,
message,
} => wire::EventMsg::RequestImageWindow {
thread_id,
session_id,
provider: provider.clone(),
model: model.clone(),
total: count(*total),
elided: count(*elided),
cap: count(*cap),
cap_source: cap_source.clone(),
message: message.clone(),
},
Event::SnapshotsDisabled { workspace, reason } => wire::EventMsg::SnapshotsDisabled {
thread_id,
session_id,
+11
View File
@@ -156,6 +156,17 @@ impl ErrorEnvelope {
)
}
#[must_use]
pub fn image_count_exceeded(message: impl Into<String>) -> Self {
Self::new(
ErrorCategory::InvalidInput,
ErrorSeverity::Error,
true,
"image_count_exceeded",
message,
)
}
/// A stop Codewhale applied to itself (step ceiling, per-turn wall clock).
/// The turn failed, so the card is an error, but the session stays usable:
/// sending another message continues, so it is `recoverable` and does not
+23
View File
@@ -923,6 +923,29 @@ pub(crate) async fn run_exec_agent(
)
);
}
Event::RequestImageWindow {
provider,
model,
total,
elided,
cap,
cap_source,
message,
} => {
if output_format == ExecOutputFormat::StreamJson {
emit_exec_stream_event(&ExecStreamEvent::RequestImageWindow {
provider,
model,
total,
elided,
cap,
cap_source,
message,
})?;
} else if !json_output {
eprintln!("{message}");
}
}
Event::ToolCallStarted {
id, name, input, ..
} => {
+79
View File
@@ -1005,6 +1005,85 @@ pub fn strip_images_when_unsupported(
stripped
}
pub(crate) const ELIDED_IMAGE_NOTE: &str = "[Image not attached to this request: this model accepts only a limited number of images per request, so the oldest are left out. It remains in the session; do not guess what it showed.]";
pub(crate) const ELIDED_TOOL_IMAGE_NOTE: &str = "[Tool image not attached to this request: this model accepts only a limited number of images per request, so the oldest are left out; do not guess what it showed. Take a fresh observation (for example a new screenshot) if you need the current state; do not repeat an action to recover its image.]";
#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)]
pub(crate) struct ImageWindow {
pub total: usize,
pub elided: usize,
pub cap: usize,
}
fn block_carries_image(block: &ContentBlock) -> bool {
match block {
ContentBlock::ImageUrl { .. } => true,
ContentBlock::ToolResult { content_blocks, .. } => {
content_blocks.as_ref().is_some_and(|blocks| {
blocks.iter().any(|value| {
value.get("type").and_then(serde_json::Value::as_str) == Some("image")
})
})
}
_ => false,
}
}
#[must_use]
pub(crate) fn count_request_images(messages: &[codewhale_models::Message]) -> usize {
messages
.iter()
.flat_map(|message| &message.content)
.filter(|block| block_carries_image(block))
.count()
}
#[must_use]
pub(crate) fn images_to_elide(total: usize, cap: usize) -> usize {
if total <= cap {
return 0;
}
let chunk = (cap / 4).max(1);
((total - cap).div_ceil(chunk) * chunk).min(total)
}
pub(crate) fn cap_request_images(
messages: &mut [codewhale_models::Message],
cap: usize,
) -> ImageWindow {
let total = count_request_images(messages);
let elided = images_to_elide(total, cap);
let mut remaining = elided;
for block in messages.iter_mut().flat_map(|message| &mut message.content) {
if remaining == 0 {
break;
}
if !block_carries_image(block) {
continue;
}
match block {
ContentBlock::ImageUrl { .. } => {
*block = ContentBlock::Text {
text: ELIDED_IMAGE_NOTE.to_string(),
cache_control: None,
};
}
ContentBlock::ToolResult {
content,
content_blocks,
..
} => {
*content_blocks = None;
*content = format!("{content}\n{ELIDED_TOOL_IMAGE_NOTE}");
}
_ => continue,
}
remaining -= 1;
}
ImageWindow { total, elided, cap }
}
/// Total inline-image byte budget for one compaction retry that follows a
/// provider body-size rejection.
///
+10
View File
@@ -13887,6 +13887,16 @@ struct ExecStreamInputAnalysis {
enum ExecStreamEvent {
#[serde(rename = "status")]
Status { message: String },
#[serde(rename = "request_image_window")]
RequestImageWindow {
provider: String,
model: String,
total: usize,
elided: usize,
cap: usize,
cap_source: String,
message: String,
},
#[serde(rename = "content")]
Content { content: String },
#[serde(rename = "tool_use")]
+64
View File
@@ -787,6 +787,70 @@ pub(crate) fn is_context_length_message(lower: &str) -> bool {
.any(|phrase| lower.contains(phrase))
}
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub(crate) struct ImageCountRejection {
pub limit: Option<usize>,
}
pub(crate) fn image_count_rejection(lower: &str) -> Option<ImageCountRejection> {
if is_context_length_message(lower) {
return None;
}
let phrase = [
"too many images",
"maximum number of images",
"max number of images",
"max images",
"maximum images",
"images per request",
"images per prompt",
"images per message",
"image(s) may be provided",
"images may be provided",
]
.iter()
.find_map(|phrase| lower.find(*phrase));
let number_of_images = lower.find("number of images").filter(|_| {
["exceed", "more than", "at most", "limit"]
.iter()
.any(|word| lower.contains(*word))
});
let at_most_images = lower.match_indices("at most ").find_map(|(index, marker)| {
let rest = &lower[index + marker.len()..];
let digits = rest.len() - rest.trim_start_matches(|c: char| c.is_ascii_digit()).len();
(digits > 0 && rest[digits..].trim_start().starts_with("image")).then_some(index)
});
let start = phrase.or(number_of_images).or(at_most_images)?;
let clause_start = lower[..start]
.rfind(['.', '\n', '{', '"'])
.map_or(0, |index| index + 1);
let clause = &lower[clause_start..];
let limit = [
"at most ",
"maximum of ",
"max of ",
"max is ",
"maximum is ",
"limit is ",
"limit of ",
"up to ",
"allowed: ",
"allowed ",
"maximum: ",
"max: ",
"max ",
"maximum ",
]
.iter()
.find_map(|marker| {
let rest = &clause[clause.find(*marker)? + marker.len()..];
let rest = rest.trim_start();
let digits = rest.len() - rest.trim_start_matches(|c: char| c.is_ascii_digit()).len();
rest[..digits].parse::<usize>().ok()
});
Some(ImageCountRejection { limit })
}
fn explicit_quota_code(body: &str) -> Option<String> {
let value: Value = serde_json::from_str(body).ok()?;
[
+42
View File
@@ -18772,6 +18772,48 @@ impl RuntimeThreadManager {
)
.await?;
}
EngineEvent::RequestImageWindow {
provider,
model,
total,
elided,
cap,
cap_source,
message,
} => {
let item = TurnItemRecord {
schema_version: CURRENT_RUNTIME_SCHEMA_VERSION,
id: runtime_record_id("item"),
turn_id: turn_id.clone(),
kind: TurnItemKind::Status,
status: TurnItemLifecycleStatus::Completed,
summary: summarize_text(&message, SUMMARY_LIMIT),
detail: Some(message),
metadata: Some(json!({
"code": "request_image_window",
"provider": provider,
"model": model,
"total": total,
"elided": elided,
"cap": cap,
"cap_source": cap_source,
})),
artifact_refs: Vec::new(),
artifacts: Vec::new(),
started_at: Some(Utc::now()),
ended_at: Some(Utc::now()),
};
self.store.save_item(&item)?;
self.attach_item_to_turn(&turn_id, &item.id)?;
self.emit_event(
&thread_id,
Some(&turn_id),
Some(&item.id),
"item.completed",
json!({ "item": item }),
)
.await?;
}
EngineEvent::Error { envelope, .. } => {
turn_status = Some(RuntimeTurnStatus::Failed);
turn_error = Some(envelope.message.clone());
+4 -1
View File
@@ -15137,6 +15137,10 @@ fn is_transient_subagent_provider_error(error: &anyhow::Error) -> bool {
// quota error mentioning "429" is not a transient rate limit. Retain the
// transport fallback for generic/parse errors and relay HTTP 400s, which
// can carry an upstream timeout rather than an invalid request.
let message = format!("{error:#}").to_ascii_lowercase();
if crate::llm_client::image_count_rejection(&message).is_some() {
return false;
}
match error.downcast_ref::<LlmError>() {
Some(
LlmError::QuotaExhausted(_)
@@ -15150,7 +15154,6 @@ fn is_transient_subagent_provider_error(error: &anyhow::Error) -> bool {
_ => {}
}
let message = format!("{error:#}").to_ascii_lowercase();
[
"did not receive response headers",
"response headers",
+3
View File
@@ -3691,6 +3691,9 @@ pub(crate) async fn run_event_loop(
.replace("{tools}", &tools);
app.push_status_toast(message, StatusToastLevel::Warning, Some(12_000));
}
EngineEvent::RequestImageWindow { message, .. } => {
app.push_status_toast(message, StatusToastLevel::Warning, Some(12_000));
}
EngineEvent::SnapshotsDisabled { reason, .. } => {
// Undo is silently off otherwise: the engine's stderr
// notice never reaches the alternate screen (#5930).
+3 -1
View File
@@ -49,7 +49,9 @@ one-request miss):
| Session construction | `initial` |
History resets that legitimately invalidate the tail (not the header) are
logged as `reset:<what>` — `reset:compaction`, `reset:clear`.
logged as `reset:<what>` — `reset:compaction`, `reset:clear`, and
`reset:image_window` when a provider's per-request image limit moves the
request image window (the oldest images go as fixed text notes).
Anything else that changes the header bytes with **no** declared reason is
**drift**: it is logged as `drift:<component>`, the original pin is **kept**
+2 -1
View File
@@ -46,7 +46,8 @@
| 会话构建 | `initial` |
历史重置会合法地让尾部(而不是头部)失效,记录为 `reset:<what>`——
`reset:compaction`、`reset:clear`。
`reset:compaction`、`reset:clear`,以及在提供方的单请求图像上限移动请求图像窗口时记录的
`reset:image_window`(最早的图像以固定的文本说明代替)。
除此之外,任何在**没有**声明原因的情况下改动头部字节的行为都算**漂移**:
它会记为 `drift:<component>`,原来的固定点被**保留**下来。同一个未声明的前缀