serve: /v1/responses accepts max_output_tokens of 0 and -1 (#4036)

The Responses validator still refused anything below 1 after the chat
path made max_tokens of -1 and 0 mean no cap (documented in
docs/DETAILS.md). It now accepts -1 and 0 as no cap; values below -1,
floats, booleans and strings stay a 400. Nothing else changes: the
server already passes max_output_tokens or 0 to prepare(), which gives
the answer the room left in the context.

Tests: OverHttp.test_max_output_tokens_zero_and_minus_one_mean_no_cap
(stream and non-stream, status completed) and
test_max_output_tokens_bad_values_are_400. The first fails without the
change. serve/test_responses.py 43 tests OK.
This commit is contained in:
Niko1221 committed 2026-10-26 18:45:04 +01:00
1 parent 41188ec734
commit 9c2d35d978
3 files changed
+20 -3

No files matched your search

+1 -1
View File
@@ -703,7 +703,7 @@ endpoints are refused with an error that says so.
| `tools` | `function`, `namespace` and `custom` tools. Hosted tools (`web_search`, `file_search`, ...) are left out because the model cannot run them |
| `tool_choice` | `"none"` hides the tools; anything else lets the model choose (it cannot be forced) |
| `reasoning.effort` | `none`/`minimal`, `low`, `medium`, `high`/`xhigh`; without it the config's `reasoning_effort`, else the model's default |
| `max_output_tokens` | The output cap, thinking included. Running out ends the response `incomplete` |
| `max_output_tokens` | The output cap, thinking included. Running out ends the response `incomplete`. `0` and `-1` (or leaving it out) mean no cap, as for `max_tokens`; anything below `-1` is a 400 |
| `text.format` | `json_schema` and `json_object` use the JSON formats above (checked, not constrained) |
| `client_metadata["x-codex-turn-metadata"]` thread-title request | Opt-in `"codex_thread_titles": true`: answered without the engine as `{"title": "<the user's line, cut to 36 characters>"}` so the conversation keeps the prompt cache |
+4 -2
View File
@@ -443,8 +443,10 @@ def check_request(req: dict) -> None:
raise ResponsesError("stored prompt templates are not supported: send instructions and input",
"prompt", "unsupported_parameter")
n = req.get("max_output_tokens")
if n is not None and (isinstance(n, bool) or not isinstance(n, int) or n < 1):
raise ResponsesError("max_output_tokens must be a positive whole number", "max_output_tokens")
# -1 and 0 mean no cap, as max_tokens does on the chat path (docs/DETAILS.md); anything lower is a mistake
if n is not None and (isinstance(n, bool) or not isinstance(n, int) or n < -1):
raise ResponsesError("max_output_tokens must be a whole number, 1 or more, or -1 or 0 for no cap",
"max_output_tokens")
reasoning = req.get("reasoning")
if reasoning is not None and not isinstance(reasoning, dict):
raise ResponsesError("reasoning must be an object", "reasoning")
+15
View File
@@ -389,6 +389,21 @@ class OverHttp(Server):
code, events = self.post({"model": "m", "input": "hi", "max_output_tokens": 5, "stream": True})
self.assertEqual(events[-1]["type"], "response.incomplete")
def test_max_output_tokens_zero_and_minus_one_mean_no_cap(self):
# #4036: the documented "no cap" values; the answer is complete, not cut at the cap
for n in (0, -1):
code, r = self.post({"model": "m", "input": "hi", "max_output_tokens": n})
self.assertEqual((code, r["status"]), (200, "completed"), (n, r))
self.assertIsNone(r["incomplete_details"])
code, events = self.post({"model": "m", "input": "hi", "max_output_tokens": n, "stream": True})
self.assertEqual((code, events[-1]["type"]), (200, "response.completed"), n)
def test_max_output_tokens_bad_values_are_400(self):
for n in (-2, -100, 1.5, True, "5"):
code, r = self.post({"model": "m", "input": "hi", "max_output_tokens": n})
self.assertEqual(code, 400, (n, r))
self.assertEqual(r["error"]["param"], "max_output_tokens", n)
def test_effort_none_does_not_think(self):
self.engine.scripts = [self.tok.encode("Hello.<|im_end|>", parse_special=True)]
self.engine.script = self.engine.scripts[0]