mirror of
https://github.com/QwenLM/qwen-code.git
synced 2026-11-07 06:19:43 -10:00
docs: configure Qwen3.x served under a custom model name (#16342)
* docs: configure Qwen3.x served under a custom model name The local self-hosted section never pointed at the reasoning override, so a Qwen3.x model served as e.g. --served-model-name chat kept thinking on in side requests and used the 200K default window. Add a short subsection with a model entry that declares the qwen-chat-template profile and the server's context window, and the server flags tool calling needs. Refs #16340 Co-Authored-By: Claude Opus 5.5 <[email protected]> * docs: move the custom model name subsection to the end of the local section Keep the API key and note blocks under Local Self-Hosted Models, as suggested in review. Co-Authored-By: Claude Opus 5.5 <[email protected]> --------- Co-authored-by: Claude Opus 5.5 <[email protected]>
This commit is contained in:
1 parent
b9e20830f7
commit
a18ecb26c8
1 file changed
+30
@@ -588,6 +588,36 @@ export VLLM_API_KEY="not-needed"
|
||||
>
|
||||
> Each provider example includes an `env` field to illustrate how the API key should be configured.
|
||||
|
||||
#### Qwen3.x served under a custom model name
|
||||
|
||||
Thinking control and the context window are inferred from the model id. If a
|
||||
vLLM or SGLang server exposes a Qwen3.x model under an id that does not start
|
||||
with `qwen` (for example `--served-model-name chat`), neither applies: side
|
||||
requests such as the auto-mode permission classifier keep thinking on, and the
|
||||
context window falls back to the 200K default. Declare both on the model entry,
|
||||
as described in [Override reasoning capabilities](#override-reasoning-capabilities):
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "chat",
|
||||
"name": "Qwen3.6 35B A3B (vLLM)",
|
||||
"envKey": "VLLM_API_KEY",
|
||||
"baseUrl": "http://localhost:8000/v1",
|
||||
"capabilities": { "reasoning": { "profile": "qwen-chat-template" } },
|
||||
"generationConfig": { "contextWindowSize": 262144 }
|
||||
}
|
||||
```
|
||||
|
||||
`qwen-chat-template` sends `chat_template_kwargs.enable_thinking`, which these
|
||||
servers pass to the Qwen chat template; a top-level `enable_thinking` field is
|
||||
not read by them. Set `contextWindowSize` to the server's maximum model length
|
||||
(vLLM reports it as `max_model_len` in `/v1/models`). Qwen Code sends tool
|
||||
definitions on every request, so the server also needs tool calling and a
|
||||
reasoning parser enabled; on vLLM that is `--enable-auto-tool-choice`, a
|
||||
`--tool-call-parser` for the model (for example `qwen3_xml`) and
|
||||
`--reasoning-parser qwen3`. Parser names change between server releases, so
|
||||
check your server's documentation.
|
||||
|
||||
## Alibaba Cloud Coding Plan
|
||||
|
||||
Alibaba Cloud Coding Plan provides a pre-configured set of Qwen models optimized for coding tasks. This feature is available for users with Alibaba Cloud Coding Plan API access and offers a simplified setup experience with automatic model configuration updates.
|
||||
|
||||
Reference in new issue
Block a user