Compare commits

..
Author SHA1 Message Date
Kevin Codex b549f5928f refactor(config): drop the unreachable SetNotify wrapper
SetNotify was kept in #1001 as a thin replace-form wrapper over
UpdateNotify for existing tests, but every product surface (the CLI
`zero config notify` and the TUI /notify picker) calls UpdateNotify's
merge form directly, so the advisory deadcode job reports it as
unreachable. Move the replace helper into writer_test.go as the
unexported setNotify, keep every test it served unchanged in behaviour,
and point the two CLI doc comments at UpdateNotify.

`make deadcode` no longer lists internal/config; the other advisory
findings in that job are pre-existing and untouched.
2026-10-14 15:58:35 +08:00
Kevin Codex 1335ca6671 feat(providers): add GMI Cloud provider preset and partner listing (#1127)
Register GMI Cloud (https://api.gmi-serving.com/v1) as an
OpenAI-compatible catalog provider and add it to the README Partners
section.

- The catalog id `gmicloud` matches the models.dev provider id, so the
  remote model catalog resolves without a mapping. `gmi-cloud`,
  `GMI Cloud` and `gmi` resolve to it.
- Reads GMI_API_KEY (the name in GMI's docs) and GMICLOUD_API_KEY (the
  name models.dev uses).
- Defaults to Qwen/Qwen3.8-Max. The curated list is Qwen 3.8 Max and
  Flash (GMI's partner models), then DeepSeek V4 Pro and Flash. All four
  are served by GMI's /v1/models.
- README and README_ZH: key line, setup command, and the partner logo in
  light and dark variants under docs/assets/.

Verified live with a GMI key: `zero providers check gmicloud
--connectivity` passes, and `zero exec` completes a read-then-write task
on both Qwen 3.8 models.
2026-10-14 12:55:29 +08:00
Fabio Tavares LeitãoandClaude Sonnet 5.5 ddc45f2c81 feat(providers): opt-in ZERO_RESPONSE_HEADER_TIMEOUT override for slow first-byte deployments (#1106)
* feat(providers): opt-in ZERO_RESPONSE_HEADER_TIMEOUT override for slow first-byte deployments

The shared HTTP transport waits at most 120s for a response header. A local
model server that has to load a large model can take longer than that to
produce the first byte (measured 1-5 minutes on a throttled local Ollama), so
an otherwise-alive request is aborted.

Add ZERO_RESPONSE_HEADER_TIMEOUT, mirroring ZERO_STREAM_IDLE_TIMEOUT: a Go
duration or a bare number of seconds; "0", "off", "none" or "disabled" remove
the limit; an unparseable or non-positive value falls back to the default
instead of removing the limit. The default stays 120s, so nothing changes for
anyone who does not set the variable. The constant and the resolver sit next to
the idle-timeout equivalents in providerio.

Checked against a throttled local Ollama: ZERO_RESPONSE_HEADER_TIMEOUT=5s
fails at ~6.4s, =300s succeeds at ~223s (a request that would have hit the
120s ceiling).

Refs #1038

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_0142QEjA7eEFcdXTpcTmcAZk

* fix(providers): keep the 120s header timeout when bare seconds overflow

A value such as 36028797018963968 passes strconv.Atoi and wraps to zero
when multiplied by time.Second, which would disable the default. Values
that do not fit in time.Duration stay on the default.

* fix(providers): keep timeout defaults when bare seconds overflow

ResolveStreamIdleTimeout multiplied bare seconds by time.Second the same way the header timeout did, so a large count wrapped to zero and disabled the watchdog. Both resolvers now share a bounds-checked conversion.

* fix(providers): apply the header timeout when the shared client is taken

The stall-hardened transport baked ResponseHeaderTimeout in at package init, so a later ZERO_RESPONSE_HEADER_TIMEOUT never reached the real client and the 120s check depended on the env at init. Each resolved value now keeps its own transport, and a slow header proves that transport enforces it.

---------

Co-authored-by: Claude Sonnet 5.5 <[email protected]>
2026-10-13 12:49:55 +08:00
Lortu | AEGIS Core eebb384676 feat(runtime): add process-local per-model RPM admission limits (#1116)
* feat(runtime): add process-local per-model RPM admission limits

* fix(config): preserve model RPM through provider fallback

* test(config): pin provider-command RPM cap precedence

* style(config): gofmt provider-command RPM test
2026-10-13 10:15:11 +08:00
Kevin Codex 39d6560137 docs(readme): add Trendshift badges and Partners section (#1126)
Add three Trendshift badges under the tagline (#1 Go repository of the
day, #3 Go repository of the week, #3 on GitHub Trending) and a Partners
section listing Twigpine, Xiaomi MiMo, Novita AI, Concentrate, and Exa,
following the layout used in the OpenClaude README. README_ZH.md gets
the same badge row and a matching section.

The Novita, Concentrate, and Exa logos are vendored under docs/assets
with light and dark variants. The Twigpine and Xiaomi MiMo logos are
loaded from their own sites, as OpenClaude does; Twigpine switches to
its dark variant under a dark color scheme.
2026-10-13 09:40:10 +08:00
37 changed files with 1042 additions and 78 deletions

No files matched your search

+20
View File
@@ -78,6 +78,19 @@ zero exec --output-format stream-json < turns.jsonl
<td align="center"><a href="https://concentrate.ai/"><strong>Concentrate</strong></a></td>
<td align="center"><a href="https://exa.ai/"><strong>Exa</strong></a></td>
</tr>
<tr>
<td align="center" width="150" height="80">
<a href="https://www.gmicloud.ai/">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="docs/assets/gmi-cloud-logo-dark.svg">
<img src="docs/assets/gmi-cloud-logo.svg" alt="GMI Cloud logo" width="120">
</picture>
</a>
</td>
</tr>
<tr>
<td align="center"><a href="https://www.gmicloud.ai/"><strong>GMI Cloud</strong></a></td>
</tr>
</table>
## Why Zero
@@ -190,6 +203,7 @@ export GEMINI_API_KEY=...
export AIMLAPI_API_KEY=...
export LONGCAT_API_KEY=...
export FIREWORKS_API_KEY=...
export GMI_API_KEY=...
export MINIMAX_API_KEY=...
export MINIMAXI_API_KEY=...
```
@@ -212,6 +226,12 @@ To configure Fireworks AI directly, run:
zero providers setup fireworks --set-active
```
To configure GMI Cloud directly, run:
```bash
zero providers setup gmicloud --set-active
```
MiniMax presets use the Anthropic-compatible endpoints for the global and China
regions:
+20
View File
@@ -76,6 +76,19 @@ zero exec --output-format stream-json < turns.jsonl
<td align="center"><a href="https://concentrate.ai/"><strong>Concentrate</strong></a></td>
<td align="center"><a href="https://exa.ai/"><strong>Exa</strong></a></td>
</tr>
<tr>
<td align="center" width="150" height="80">
<a href="https://www.gmicloud.ai/">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="docs/assets/gmi-cloud-logo-dark.svg">
<img src="docs/assets/gmi-cloud-logo.svg" alt="GMI Cloud logo" width="120">
</picture>
</a>
</td>
</tr>
<tr>
<td align="center"><a href="https://www.gmicloud.ai/"><strong>GMI Cloud</strong></a></td>
</tr>
</table>
## 为什么选择 Zero
@@ -163,6 +176,7 @@ export OPENAI_API_KEY=sk-...
export ANTHROPIC_API_KEY=...
export GEMINI_API_KEY=...
export LONGCAT_API_KEY=...
export GMI_API_KEY=...
```
要直接配置美团 LongCat(LongCat-2.0),运行:
@@ -171,6 +185,12 @@ export LONGCAT_API_KEY=...
zero providers setup longcat --set-active
```
要直接配置 GMI Cloud,运行:
```bash
zero providers setup gmicloud --set-active
```
对于本地模型,运行 Ollama、LM Studio 或 [Atomic Chat](https://atomic.chat) 桌面应用,
然后使用 `zero setup` 或 `zero providers detect`。使用 Atomic Chat 时,请先加载模型并启用
本地 OpenAI 兼容 API(默认地址为 `http://127.0.0.1:1337/v1`),再选择 `atomic-chat-local`。
+26
View File
@@ -0,0 +1,26 @@
# Local per-model completion limits
Add `modelRPM` to the existing Zero config file:
```json
{"modelRPM": {"gpt-4.1": 30, "claude-sonnet-4.5": 15}}
```
Keys resolve through the model registry; custom IDs match exactly after trimming
whitespace. Missing keys and zero mean unlimited. Negative limits and conflicting
aliases are configuration errors. Project config may add or tighten a user cap,
but cannot relax it. Changes take effect on the next Zero process start.
For `zero exec` and interactive agent turns, admission happens immediately before
the wrapped turn session's `Stream`. N admissions fit in any sliding 60-second
window; N+1 returns a local rate-limit error before entering that method. The
error reports when a slot expires. Exec uses its existing provider-error exit
code plus a local-limit hint. There is no sleep or hidden retry. Failed admitted
requests still consume a slot.
Sessions and model switches share a limiter in one process. Separate processes
have independent windows; restarting clears them. This is not an account-wide
cap. It counts Stream admissions, not all HTTP requests: setup/prewarm,
compaction, provider-internal retries, discovery and direct calls outside that
boundary are not counted. No TPM, cost budgets, persistence, cross-process
coordination or new runtime/Python dependency is introduced.
+1
View File
@@ -6,6 +6,7 @@ Zero.
## User Docs
- [Install](INSTALL.md)
- [Local per-model completion limits](MODEL_RPM.md)
- [Update flow](UPDATE.md)
- [OAuth logins and subscription-backed providers](oauth-subscriptions.md)
+7
View File
@@ -0,0 +1,7 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 34 718 165" width="718" height="165" fill="#fff">
<path d="M123.292 132.486h-33.1c-2.644 0-4.808-2.395-4.808-5.363v-21.436c0-2.95 2.148-5.363 4.809-5.363h33.099c2.645 0 4.808-2.395 4.808-5.363V73.525c0-2.95 2.147-5.363 4.808-5.363h33.099c2.646 0 4.809-2.395 4.809-5.363V41.363c0-2.95-2.148-5.363-4.809-5.363H90.208c-2.645 0-4.808 2.395-4.808 5.363V62.8c0 2.95-2.147 5.363-4.808 5.363h-33.1c-2.645 0-4.808 2.395-4.808 5.363v21.436c0 2.95-2.147 5.363-4.808 5.363H4.808c-2.645 0-4.808 2.395-4.808 5.363v21.436c0 2.95 2.147 5.363 4.808 5.363h33.1c2.645 0 4.808 2.395 4.808 5.363v21.436c0 2.95 2.147 5.363 4.808 5.363h33.099c2.645 0 4.808 2.395 4.808 5.363v21.436c0 2.95 2.148 5.363 4.809 5.363h75.798c2.646 0 4.809-2.395 4.809-5.363v-21.436c0-2.95-2.148-5.363-4.809-5.363H132.94c-2.646 0-4.809-2.395-4.809-5.363v-21.436c0-2.95-2.147-5.363-4.808-5.363z"/>
<path d="M251.408 100.341c2.646 0 4.809 2.395 4.809 5.363v21.436c0 2.95-2.148 5.363-4.809 5.363H218.31c-2.646 0-4.809 2.395-4.809 5.363v21.436c0 2.95-2.147 5.363-4.808 5.363h-33.099c-2.645 0-4.808-2.395-4.808-5.363v-53.58c0-2.951 2.147-5.364 4.808-5.364h75.799zM510.68 40.105h-28.795v153.33h28.795zM717.43 40.105h-28.795v153.33h28.795z"/>
<path d="M532.41 40.105h-28.83l35.391 153.33H567.8zM653.242 40.105h-28.795v153.33h28.795z"/>
<path d="M596.594 193.435h-28.795l35.39-153.33h28.795zM452.028 114.062v27.042s-.139.174-.191.261h-83.069V114.08h83.295-.035z"/>
<path d="M452.028 141.347v35.651c-17.479 12.306-39.834 19.769-62.953 19.769-59.673 0-97.858-35.981-97.858-80.362s34.54-80.362 97.858-80.362c22.511 0 45.127 7.689 62.953 20.36v35.685c-13.105-16.42-37.856-28.743-62.953-28.743-44.052 0-70.695 20.846-70.695 53.077 0 32.232 28.691 53.077 70.695 53.077 25.323 0 49.727-11.906 62.744-28.1h.191v-.035z"/>
</svg>

After

Width:  |  Height:  |  Size: 1.8 KiB

+7
View File
@@ -0,0 +1,7 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 34 718 165" width="718" height="165" fill="#0D0E10">
<path d="M123.292 132.486h-33.1c-2.644 0-4.808-2.395-4.808-5.363v-21.436c0-2.95 2.148-5.363 4.809-5.363h33.099c2.645 0 4.808-2.395 4.808-5.363V73.525c0-2.95 2.147-5.363 4.808-5.363h33.099c2.646 0 4.809-2.395 4.809-5.363V41.363c0-2.95-2.148-5.363-4.809-5.363H90.208c-2.645 0-4.808 2.395-4.808 5.363V62.8c0 2.95-2.147 5.363-4.808 5.363h-33.1c-2.645 0-4.808 2.395-4.808 5.363v21.436c0 2.95-2.147 5.363-4.808 5.363H4.808c-2.645 0-4.808 2.395-4.808 5.363v21.436c0 2.95 2.147 5.363 4.808 5.363h33.1c2.645 0 4.808 2.395 4.808 5.363v21.436c0 2.95 2.147 5.363 4.808 5.363h33.099c2.645 0 4.808 2.395 4.808 5.363v21.436c0 2.95 2.148 5.363 4.809 5.363h75.798c2.646 0 4.809-2.395 4.809-5.363v-21.436c0-2.95-2.148-5.363-4.809-5.363H132.94c-2.646 0-4.809-2.395-4.809-5.363v-21.436c0-2.95-2.147-5.363-4.808-5.363z"/>
<path d="M251.408 100.341c2.646 0 4.809 2.395 4.809 5.363v21.436c0 2.95-2.148 5.363-4.809 5.363H218.31c-2.646 0-4.809 2.395-4.809 5.363v21.436c0 2.95-2.147 5.363-4.808 5.363h-33.099c-2.645 0-4.808-2.395-4.808-5.363v-53.58c0-2.951 2.147-5.364 4.808-5.364h75.799zM510.68 40.105h-28.795v153.33h28.795zM717.43 40.105h-28.795v153.33h28.795z"/>
<path d="M532.41 40.105h-28.83l35.391 153.33H567.8zM653.242 40.105h-28.795v153.33h28.795z"/>
<path d="M596.594 193.435h-28.795l35.39-153.33h28.795zM452.028 114.062v27.042s-.139.174-.191.261h-83.069V114.08h83.295-.035z"/>
<path d="M452.028 141.347v35.651c-17.479 12.306-39.834 19.769-62.953 19.769-59.673 0-97.858-35.981-97.858-80.362s34.54-80.362 97.858-80.362c22.511 0 45.127 7.689 62.953 20.36v35.685c-13.105-16.42-37.856-28.743-62.953-28.743-44.052 0-70.695 20.846-70.695 53.077 0 32.232 28.691 53.077 70.695 53.077 25.323 0 49.727-11.906 62.744-28.1h.191v-.035z"/>
</svg>

After

Width:  |  Height:  |  Size: 1.8 KiB

+3 -1
View File
@@ -968,7 +968,9 @@ func runInteractiveTUIWithSetup(stderr io.Writer, deps appDeps, permissionMode a
// From the engine the session's commands run through, so the notice
// describes the sandbox they actually get.
startupNotices := []string{sandboxEngine.DegradedNotice()}
modelRPM := zeroruntime.NewModelRPMLimiter(resolved.ModelRPM)
return deps.runTUI(context.Background(), tui.Options{
ModelRPM: modelRPM,
Cwd: workspaceRoot,
Version: version,
Theme: theme,
@@ -991,7 +993,7 @@ func runInteractiveTUIWithSetup(stderr io.Writer, deps appDeps, permissionMode a
if provider == nil {
return nil
}
if optimized, ok := providers.OptimizedTurnSessions(profile, provider, providers.Options{}); ok {
if optimized, ok := providers.ConfiguredTurnSessions(profile, provider, providers.Options{ModelRPM: modelRPM}); ok {
return optimized
}
return providers.DefaultTurnSessions(profile, provider, providers.Options{})
+8 -2
View File
@@ -343,8 +343,10 @@ func TestRunNoArgsFallsBackToUsableProviderWhenNoneMarkedActive(t *testing.T) {
// config.json has providers configured but none marked active (e.g. a
// blank/stale activeProvider field) — Resolve returns the successfully
// normalized providers list alongside ErrNoActiveProvider.
return config.ResolvedConfig{Providers: []config.ProviderProfile{usable}},
fmt.Errorf("%w: active provider %q not found", config.ErrNoActiveProvider, "")
return config.ResolvedConfig{
Providers: []config.ProviderProfile{usable},
ModelRPM: map[string]int{"gpt-test": 1},
}, fmt.Errorf("%w: active provider %q not found", config.ErrNoActiveProvider, "")
},
newProvider: func(profile config.ProviderProfile) (zeroruntime.Provider, error) {
providerProfile = profile
@@ -374,6 +376,10 @@ func TestRunNoArgsFallsBackToUsableProviderWhenNoneMarkedActive(t *testing.T) {
}
if providerProfile.Name != "work" {
t.Fatalf("provider used = %q, want fallback to the usable saved provider %q", providerProfile.Name, "work")
}
if launchedOptions.ModelRPM == nil {
t.Fatal("ModelRPM = nil, want limiter preserved through provider fallback")
}
}
+2 -2
View File
@@ -10,12 +10,12 @@ import (
// runConfigNotify implements `zero config notify`: with no flags it prints the
// stored mode/focusMode; --mode/--focus update them via the same
// config.SetNotify writer the TUI /notify command uses, so all surfaces stay
// config.UpdateNotify writer the TUI /notify command uses, so all surfaces stay
// in lockstep; --reset blanks both fields so the TUI's effective default
// applies again (an unconfigured headless run stays silent).
//
// The command manages a user preference, so it talks ONLY to the user's own
// config file (config.UserNotify / config.SetNotify) and never runs the full
// config file (config.UserNotify / config.UpdateNotify) and never runs the full
// config resolution: resolving providers would fail with ErrNoActiveProvider
// for a brand-new user, locking them out of setting notifications before they
// have even configured a provider (CodeRabbit review, PR #1001). The display
+6 -6
View File
@@ -463,20 +463,20 @@ func runExec(args []string, stdout io.Writer, stderr io.Writer, deps appDeps) in
// escalation so post-switch turns are attributed to the escalated model.
currentModel := resolved.Provider.Model
// Optimized OpenAI turn sessions (ZERO_OPENAI_TURN_SESSION, default off). nil
// when gated off or the profile is ineligible: agent.Run then wraps the
// provider in its default adapter. This is the run's STARTING session provider
// and is used whether or not escalation is enabled.
turnSessions, _ := providers.OptimizedTurnSessions(resolved.Provider, provider, providers.Options{})
// when gated off or ineligible, unless a configured RPM limiter needs to
// wrap the default adapter. The limiter is shared across model switches.
sessionOptions := providers.Options{ModelRPM: zeroruntime.NewModelRPMLimiter(resolved.ModelRPM)}
turnSessions, _ := providers.ConfiguredTurnSessions(resolved.Provider, provider, sessionOptions)
// Both switchers come from one shared builder so exec and the interactive TUI
// cannot drift on the nil contracts the agent loop depends on. The session
// switcher is nil unless this run STARTED optimized, which is what keeps a
// default-adapter run on the default adapter.
// switcher preserves the starting transport and shares the RPM limiter.
var modelSwitcher func(context.Context, string) (agent.Provider, error)
var modelSessionSwitcher func(context.Context, string) (zeroruntime.TurnSessionProvider, error)
if options.allowEscalation {
modelSwitcher, modelSessionSwitcher = providers.EscalationSwitchers(
resolved.Provider, provider, deps.newProvider,
func(modelID string) { currentModel = modelID },
sessionOptions,
)
}
+83
View File
@@ -0,0 +1,83 @@
package cli
import (
"bytes"
"context"
"errors"
"github.com/Gitlawb/zero/internal/config"
"github.com/Gitlawb/zero/internal/mcp"
"github.com/Gitlawb/zero/internal/tools"
"github.com/Gitlawb/zero/internal/tui"
"github.com/Gitlawb/zero/internal/zeroruntime"
"strings"
"testing"
)
type rpmExecProvider struct{ calls int }
func (p *rpmExecProvider) StreamCompletion(context.Context, zeroruntime.CompletionRequest) (<-chan zeroruntime.StreamEvent, error) {
p.calls++
ch := make(chan zeroruntime.StreamEvent, 4)
if p.calls == 1 {
ch <- zeroruntime.StreamEvent{Type: zeroruntime.StreamEventToolCallStart, ToolCallID: "fixture", ToolName: "read_file"}
ch <- zeroruntime.StreamEvent{Type: zeroruntime.StreamEventToolCallDelta, ToolCallID: "fixture", ArgumentsFragment: `{"path":"missing-fixture.txt"}`}
ch <- zeroruntime.StreamEvent{Type: zeroruntime.StreamEventToolCallEnd, ToolCallID: "fixture"}
} else {
ch <- zeroruntime.StreamEvent{Type: zeroruntime.StreamEventText, Content: "unexpected second completion"}
}
ch <- zeroruntime.StreamEvent{Type: zeroruntime.StreamEventDone}
close(ch)
return ch, nil
}
func rpmTestDeps(t *testing.T, p *rpmExecProvider) appDeps {
t.Helper()
root := t.TempDir()
for _, k := range []string{"HOME", "USERPROFILE", "APPDATA", "LOCALAPPDATA", "XDG_CONFIG_HOME", "XDG_CACHE_HOME", "XDG_DATA_HOME", "XDG_STATE_HOME"} {
t.Setenv(k, root)
}
return appDeps{getwd: func() (string, error) { return root, nil }, resolveConfig: func(string, config.Overrides) (config.ResolvedConfig, error) {
c := execResolvedConfig()
c.ModelRPM = map[string]int{c.Provider.Model: 1}
return c, nil
}, newProvider: func(config.ProviderProfile) (zeroruntime.Provider, error) { return p, nil }, registerMCPTools: func(context.Context, *tools.Registry, config.MCPConfig, mcp.RegisterOptions) (mcpToolRuntime, error) {
return noopMCPRuntime{}, nil
}}
}
func TestRunExecRPMRefusesSecondCompletion(t *testing.T) {
p := &rpmExecProvider{}
deps := rpmTestDeps(t, p)
var out, err bytes.Buffer
code := runWithDeps([]string{"exec", "fixture"}, &out, &err, deps)
if code != exitProvider || p.calls != 1 || !strings.Contains(err.String(), "Local modelRPM cap reached") {
t.Fatalf("exit=%d calls=%d stdout=%s stderr=%s", code, p.calls, out.String(), err.String())
}
}
func TestInteractiveRPMFactorySharesWindow(t *testing.T) {
p := &rpmExecProvider{}
deps := rpmTestDeps(t, p)
launched := false
deps.runTUI = func(ctx context.Context, o tui.Options) int {
launched = true
if o.ModelRPM == nil {
t.Fatal("TUI escalation lacks limiter")
}
for i := 0; i < 2; i++ {
ss := o.NewTurnSessionProvider(o.ProviderProfile, p)
s, e := ss.OpenTurnSession(ctx)
if e != nil {
t.Fatal(e)
}
_, e = s.Stream(ctx, zeroruntime.CompletionRequest{})
var hit *zeroruntime.RPMLimitError
if (i == 0 && e != nil) || (i == 1 && !errors.As(e, &hit)) {
t.Fatalf("run=%d err=%v", i, e)
}
}
return exitSuccess
}
var out, err bytes.Buffer
code := runWithDeps(nil, &out, &err, deps)
if !launched || code != exitSuccess || p.calls != 1 {
t.Fatalf("launched=%v exit=%d calls=%d stderr=%s", launched, code, p.calls, err.String())
}
}
@@ -20,6 +20,7 @@ func TestProviderCatalogRuntimeProvidersShowOnboardingSetupHints(t *testing.T) {
{id: "groq", name: "Groq", transport: "openai-compatible", defaultModel: "llama-3.3-70b-versatile"},
{id: "longcat", name: "LongCat", transport: "openai-compatible", defaultModel: "LongCat-2.0"},
{id: "fireworks", name: "Fireworks AI", transport: "openai-compatible", defaultModel: "accounts/fireworks/models/kimi-k2p7-code"},
{id: "gmicloud", name: "GMI Cloud", transport: "openai-compatible", defaultModel: "Qwen/Qwen3.8-Max"},
{id: "ollama-cloud", name: "Ollama Cloud", transport: "openai-compatible", defaultModel: "qwen3-coder:480b"},
{id: "ollama", name: "Ollama Local", transport: "openai-compatible", defaultModel: "llama3.1"},
}
+48
View File
@@ -0,0 +1,48 @@
package config
import (
"fmt"
"github.com/Gitlawb/zero/internal/modelregistry"
"strings"
)
func normalizeModelRPM(limits map[string]int) (map[string]int, error) {
if len(limits) == 0 {
return nil, nil
}
registry, err := modelregistry.DefaultRegistry()
if err != nil {
return nil, err
}
out := make(map[string]int, len(limits))
for key, n := range limits {
model := strings.TrimSpace(key)
if model == "" || n < 0 {
return nil, fmt.Errorf("invalid modelRPM entry %q: model must be non-empty and limit >= 0", key)
}
if id, ok := registry.ResolveID(model); ok {
model = id
}
if old, exists := out[model]; exists && old != n {
return nil, fmt.Errorf("conflicting modelRPM entries resolve to %q", model)
}
out[model] = n
}
return out, nil
}
// Project config can tighten a user cap but cannot relax it.
func mergeModelRPM(dst *map[string]int, src map[string]int, tighten bool) {
if len(src) == 0 {
return
}
if *dst == nil {
*dst = make(map[string]int)
}
for model, n := range src {
old := (*dst)[model]
if !tighten || old == 0 || (n > 0 && n < old) {
(*dst)[model] = n
}
}
}
+54
View File
@@ -0,0 +1,54 @@
package config
import (
"encoding/json"
"reflect"
"testing"
)
func TestModelRPMRoundTrip(t *testing.T) {
var c FileConfig
if e := json.Unmarshal([]byte(`{"modelRPM":{" OPENAI:GPT-4.1 ":2,"custom":0},"future":true}`), &c); e != nil {
t.Fatal(e)
}
want := map[string]int{"gpt-4.1": 2, "custom": 0}
if !reflect.DeepEqual(c.ModelRPM, want) {
t.Fatal(c.ModelRPM)
}
b, e := json.Marshal(c)
if e != nil {
t.Fatal(e)
}
var d FileConfig
if e = json.Unmarshal(b, &d); e != nil {
t.Fatal(e)
}
if !reflect.DeepEqual(d.ModelRPM, want) || string(d.Extra["future"]) != "true" {
t.Fatalf("round trip=%s", b)
}
}
func TestModelRPMInvalid(t *testing.T) {
for _, s := range []string{`{"modelRPM":{"":1}}`, `{"modelRPM":{"a":-1}}`, `{"modelRPM":{"a":1.5}}`, `{"modelRPM":{"a":"2"}}`, `{"modelRPM":{"gpt-4.1":1,"openai:gpt-4.1":2}}`} {
var c FileConfig
if e := json.Unmarshal([]byte(s), &c); e == nil {
t.Fatalf("accepted %s", s)
}
}
}
func TestModelRPMProjectCanOnlyTighten(t *testing.T) {
user := writeConfig(t, `{"modelRPM":{"gpt-4.1":2},"activeProvider":"test","providers":[{"name":"test","provider":"openai","model":"gpt-4.1"}]}`)
for _, n := range []string{"0", "1", "5"} {
project := writeConfig(t, `{"modelRPM":{"openai:gpt-4.1":`+n+`}}`)
c, e := Resolve(ResolveOptions{UserConfigPath: user, ProjectConfigPath: project, Env: map[string]string{}})
if e != nil {
t.Fatal(e)
}
want := 2
if n == "1" {
want = 1
}
if c.ModelRPM["gpt-4.1"] != want {
t.Fatal(c.ModelRPM)
}
}
}
+7
View File
@@ -104,6 +104,9 @@ func Resolve(options ResolveOptions) (ResolvedConfig, error) {
// trusted global-config merge, which must keep honouring the setting.
commandConfig.Sandbox.Enabled = nil
commandConfig.CrossSessionInbound = ""
// External provider commands cannot relax the user cap.
mergeModelRPM(&cfg.ModelRPM, commandConfig.ModelRPM, true)
commandConfig.ModelRPM = nil
mergeConfig(&cfg, commandConfig)
}
@@ -163,6 +166,7 @@ func Resolve(options ResolveOptions) (ResolvedConfig, error) {
// dropping them would let the TUI's unconfigured notify default overwrite
// a stored explicit opt-out on startup (maintainer review, PR #1001).
partial := ResolvedConfig{
ModelRPM: cfg.ModelRPM,
Providers: providers,
MaxTurns: cfg.MaxTurns,
MCP: cfg.MCP,
@@ -180,6 +184,7 @@ func Resolve(options ResolveOptions) (ResolvedConfig, error) {
}
return ResolvedConfig{
ModelRPM: cfg.ModelRPM,
ActiveProvider: active.Name,
Providers: providers,
Provider: active,
@@ -248,6 +253,7 @@ func loadConfigFile(path string) (FileConfig, error) {
}
func mergeConfig(dst *FileConfig, src FileConfig) {
mergeModelRPM(&dst.ModelRPM, src.ModelRPM, false)
if activeProvider := strings.TrimSpace(src.ActiveProvider); activeProvider != "" {
dst.ActiveProvider = activeProvider
}
@@ -309,6 +315,7 @@ func mergeConfig(dst *FileConfig, src FileConfig) {
}
func mergeProjectConfig(dst *FileConfig, src FileConfig) error {
mergeModelRPM(&dst.ModelRPM, src.ModelRPM, true)
if activeProvider := strings.TrimSpace(src.ActiveProvider); activeProvider != "" {
dst.ActiveProvider = activeProvider
}
+38 -4
View File
@@ -1109,6 +1109,34 @@ func TestResolveProviderCommandOverridesEnvProviderFields(t *testing.T) {
}
}
func TestResolveProviderCommandCannotRelaxUserModelRPM(t *testing.T) {
userPath := writeConfig(t, `{
"modelRPM": {
"gpt-command": 5
}
}`)
command := writeCommand(t, commandScript{
Stdout: `{"activeProvider":"cmd","providers":[{"name":"cmd","provider":"openai","apiKey":"sk-command","model":"gpt-command"}],"modelRPM":{"gpt-command":10}}`,
})
resolved, err := Resolve(ResolveOptions{
UserConfigPath: userPath,
ProviderCommand: command,
Env: map[string]string{},
})
if err != nil {
t.Fatalf("Resolve() error = %v", err)
}
if got := resolved.ModelRPM["gpt-command"]; got != 5 {
t.Fatalf(
"ModelRPM[gpt-command] = %d, want user cap 5 preserved against provider-command relaxation",
got,
)
}
}
func TestResolveAllowsNoConfiguredProviders(t *testing.T) {
resolved, err := Resolve(ResolveOptions{Env: map[string]string{}})
if err != nil {
@@ -1166,10 +1194,13 @@ func TestResolveKeepsNormalizedProvidersWhenNoneMarkedActive(t *testing.T) {
// activeProvider is blank/stale — a caller like the interactive TUI still
// needs the normalized list to fall back to an already-usable provider
// instead of forcing a full re-onboarding wizard.
path := writeConfig(t, `{"providers":[
{"name":"work","provider_kind":"openai","apiKey":"sk-test","model":"gpt-test"},
{"name":"other","provider_kind":"openai","apiKey":"sk-other","model":"gpt-test"}
]}`)
path := writeConfig(t, `{
"modelRPM":{"gpt-test":1},
"providers":[
{"name":"work","provider_kind":"openai","apiKey":"sk-test","model":"gpt-test"},
{"name":"other","provider_kind":"openai","apiKey":"sk-other","model":"gpt-test"}
]
}`)
resolved, err := Resolve(ResolveOptions{ProjectConfigPath: path, Env: map[string]string{}})
if !errors.Is(err, ErrNoActiveProvider) {
@@ -1178,6 +1209,9 @@ func TestResolveKeepsNormalizedProvidersWhenNoneMarkedActive(t *testing.T) {
if len(resolved.Providers) != 2 {
t.Fatalf("Providers = %#v, want the 2 normalized profiles preserved despite the error", resolved.Providers)
}
if got := resolved.ModelRPM["gpt-test"]; got != 1 {
t.Fatalf("ModelRPM[gpt-test] = %d, want 1 preserved despite ErrNoActiveProvider", got)
}
}
func TestResolveTrimsProviderProfileAliasesBeforeFallback(t *testing.T) {
+10
View File
@@ -364,6 +364,7 @@ func (cfg *ToolsConfig) UnmarshalJSON(data []byte) error {
}
type FileConfig struct {
ModelRPM map[string]int `json:"modelRPM,omitempty"`
ActiveProvider string `json:"activeProvider,omitempty"`
Providers []ProviderProfile `json:"providers,omitempty"`
MaxTurns int `json:"maxTurns,omitempty"`
@@ -389,6 +390,7 @@ type FileConfig struct {
func (cfg FileConfig) MarshalJSON() ([]byte, error) {
type rawConfig struct {
ModelRPM map[string]int `json:"modelRPM,omitempty"`
ActiveProvider string `json:"activeProvider,omitempty"`
Providers []ProviderProfile `json:"providers,omitempty"`
MaxTurns int `json:"maxTurns,omitempty"`
@@ -405,6 +407,7 @@ func (cfg FileConfig) MarshalJSON() ([]byte, error) {
Sessions *SessionsConfig `json:"sessions,omitempty"`
}
raw := rawConfig{
ModelRPM: cfg.ModelRPM,
ActiveProvider: cfg.ActiveProvider,
Providers: cfg.Providers,
MaxTurns: cfg.MaxTurns,
@@ -483,6 +486,7 @@ type Overrides struct {
}
type ResolvedConfig struct {
ModelRPM map[string]int
ActiveProvider string
Providers []ProviderProfile
Provider ProviderProfile
@@ -551,6 +555,7 @@ type MCPOAuthConfig struct {
func (cfg *FileConfig) UnmarshalJSON(data []byte) error {
type rawConfig struct {
ModelRPM map[string]int `json:"modelRPM"`
ActiveProvider string `json:"activeProvider"`
Providers []ProviderProfile `json:"providers"`
MaxTurns int `json:"maxTurns"`
@@ -586,6 +591,11 @@ func (cfg *FileConfig) UnmarshalJSON(data []byte) error {
extra = nil
}
cfg.ActiveProvider = raw.ActiveProvider
limits, err := normalizeModelRPM(raw.ModelRPM)
if err != nil {
return err
}
cfg.ModelRPM = limits
cfg.Providers = raw.Providers
// A negative maxTurns is unambiguously invalid; without this it would be
// silently dropped by the `MaxTurns > 0` merge gates and fall back to the
-18
View File
@@ -879,24 +879,6 @@ func UpdateNotify(path string, merge func(current NotifyConfig) NotifyConfig) (r
return next, nil
}
// SetNotify replaces the stored notification preference with value. It is a
// thin wrapper over the transactional UpdateNotify so existing callers keep
// their replace semantics under the same lock and byte-preserving write.
func SetNotify(path string, value NotifyConfig) (FileConfig, error) {
next, err := UpdateNotify(path, func(NotifyConfig) NotifyConfig { return value })
if err != nil {
return FileConfig{}, err
}
// Re-read the full config so callers that inspect unrelated fields (and
// existing tests) keep their FileConfig-shaped result.
cfg, err := loadConfigFile(path)
if err != nil {
return FileConfig{}, err
}
cfg.Notify = next
return cfg, nil
}
// SetPet persists only the terminal-pet preference while preserving every
// unrelated user setting through the config writer's atomic replace path.
func SetPet(path string, pet string) (result FileConfig, err error) {
+38 -21
View File
@@ -428,6 +428,23 @@ func TestSetThemePreservesUnknownTopLevelFields(t *testing.T) {
}
}
// setNotify replaces the stored notify block with value through UpdateNotify
// and returns the re-read config, so the tests below can assert on a
// FileConfig-shaped result. Every product surface uses UpdateNotify's merge
// form directly; this replace form exists only for these tests.
func setNotify(path string, value NotifyConfig) (FileConfig, error) {
next, err := UpdateNotify(path, func(NotifyConfig) NotifyConfig { return value })
if err != nil {
return FileConfig{}, err
}
cfg, err := loadConfigFile(path)
if err != nil {
return FileConfig{}, err
}
cfg.Notify = next
return cfg, nil
}
func TestSetNotifyPersistsValidValues(t *testing.T) {
path := filepath.Join(t.TempDir(), "zero.json")
writeConfigFixture(t, path, FileConfig{
@@ -437,9 +454,9 @@ func TestSetNotifyPersistsValidValues(t *testing.T) {
},
}, 0o600)
cfg, err := SetNotify(path, NotifyConfig{Mode: " both ", FocusMode: " unfocused "})
cfg, err := setNotify(path, NotifyConfig{Mode: " both ", FocusMode: " unfocused "})
if err != nil {
t.Fatalf("SetNotify() error = %v", err)
t.Fatalf("setNotify() error = %v", err)
}
if cfg.Notify.Mode != "both" || cfg.Notify.FocusMode != "unfocused" {
t.Fatalf("Notify = %+v, want mode=both focusMode=unfocused (trimmed)", cfg.Notify)
@@ -449,14 +466,14 @@ func TestSetNotifyPersistsValidValues(t *testing.T) {
t.Fatalf("persisted Notify = %+v, want mode=both focusMode=unfocused", persisted.Notify)
}
if persisted.ActiveProvider != "openai" || len(persisted.Providers) != 1 {
t.Fatalf("provider config was not preserved by SetNotify: %#v", persisted)
t.Fatalf("provider config was not preserved by setNotify: %#v", persisted)
}
}
func TestSetNotifyRejectsInvalidMode(t *testing.T) {
path := filepath.Join(t.TempDir(), "zero.json")
writeConfigFixture(t, path, FileConfig{ActiveProvider: "openai"}, 0o600)
if _, err := SetNotify(path, NotifyConfig{Mode: "loud", FocusMode: "unfocused"}); err == nil {
if _, err := setNotify(path, NotifyConfig{Mode: "loud", FocusMode: "unfocused"}); err == nil {
t.Fatal("expected error for invalid notify.mode")
}
}
@@ -464,25 +481,25 @@ func TestSetNotifyRejectsInvalidMode(t *testing.T) {
func TestSetNotifyRejectsInvalidFocusMode(t *testing.T) {
path := filepath.Join(t.TempDir(), "zero.json")
writeConfigFixture(t, path, FileConfig{ActiveProvider: "openai"}, 0o600)
if _, err := SetNotify(path, NotifyConfig{Mode: "off", FocusMode: "sideways"}); err == nil {
if _, err := setNotify(path, NotifyConfig{Mode: "off", FocusMode: "sideways"}); err == nil {
t.Fatal("expected error for invalid notify.focusMode")
}
}
func TestSetNotifyRejectsEmptyConfigPath(t *testing.T) {
if _, err := SetNotify("", NotifyConfig{Mode: "off"}); err == nil {
if _, err := setNotify("", NotifyConfig{Mode: "off"}); err == nil {
t.Fatal("expected error for empty config path")
}
}
func TestSetNotifyBlankValuesPreservedAsDefaults(t *testing.T) {
// An empty mode/focusMode stored on disk is a valid "use the built-in
// defaults" signal — SetNotify must not reject blanks, and they must round
// defaults" signal — setNotify must not reject blanks, and they must round
// trip unchanged.
path := filepath.Join(t.TempDir(), "zero.json")
writeConfigFixture(t, path, FileConfig{ActiveProvider: "openai"}, 0o600)
if _, err := SetNotify(path, NotifyConfig{}); err != nil {
t.Fatalf("SetNotify({}) should accept blank values, got error: %v", err)
if _, err := setNotify(path, NotifyConfig{}); err != nil {
t.Fatalf("setNotify({}) should accept blank values, got error: %v", err)
}
persisted := readConfigFixture(t, path)
if persisted.Notify.Mode != "" || persisted.Notify.FocusMode != "" {
@@ -494,7 +511,7 @@ func TestSetNotifyBlankValuesPreservedAsDefaults(t *testing.T) {
// values whose EXPLICIT presence matters. The typed serializer's omitempty
// cannot round-trip tools.deferThreshold: 0 (explicit "never defer" vs unset
// "use default 3") or an MCP server's disabled: false (explicitly enabled), so
// SetNotify/UpdateNotify edit only the notify member's bytes instead. The
// UpdateNotify edits only the notify member's bytes instead. The
// assertions run against the RAW file, because reading through FileConfig
// would hide exactly the presence loss being tested.
func TestSetNotifyPreservesExplicitUnrelatedValues(t *testing.T) {
@@ -517,8 +534,8 @@ func TestSetNotifyPreservesExplicitUnrelatedValues(t *testing.T) {
t.Fatal(err)
}
if _, err := SetNotify(path, NotifyConfig{Mode: "off", FocusMode: "always"}); err != nil {
t.Fatalf("SetNotify: %v", err)
if _, err := setNotify(path, NotifyConfig{Mode: "off", FocusMode: "always"}); err != nil {
t.Fatalf("setNotify: %v", err)
}
raw, err := os.ReadFile(path)
@@ -561,8 +578,8 @@ func TestSetNotifyResetRemovesEveryDuplicateNotifyMember(t *testing.T) {
t.Fatal(err)
}
if _, err := SetNotify(path, NotifyConfig{}); err != nil {
t.Fatalf("SetNotify({}) reset: %v", err)
if _, err := setNotify(path, NotifyConfig{}); err != nil {
t.Fatalf("setNotify({}) reset: %v", err)
}
raw, err := os.ReadFile(path)
@@ -586,8 +603,8 @@ func TestSetNotifyResetRemovesEveryDuplicateNotifyMember(t *testing.T) {
if err := os.WriteFile(path, []byte(duplicated), 0o600); err != nil {
t.Fatal(err)
}
if _, err := SetNotify(path, NotifyConfig{Mode: "off", FocusMode: "always"}); err != nil {
t.Fatalf("SetNotify replace: %v", err)
if _, err := setNotify(path, NotifyConfig{Mode: "off", FocusMode: "always"}); err != nil {
t.Fatalf("setNotify replace: %v", err)
}
raw, err = os.ReadFile(path)
if err != nil {
@@ -1394,7 +1411,7 @@ func TestUpdateNotifyRacesExistingConfigMutatorWithoutLoss(t *testing.T) {
// fields of duplicate JSON members into one struct, so a full replacement that
// omits a field could inherit it from an earlier duplicate —
// {"notify":{"mode":"off"},"notify":{"focusMode":"focused"}} +
// SetNotify({FocusMode:"always"}) must yield blank/always, not off/always.
// setNotify({FocusMode:"always"}) must yield blank/always, not off/always.
// The editor removes every duplicate and inserts the replacement fresh.
func TestSetNotifyReplacementInheritsNoFieldFromEarlierDuplicate(t *testing.T) {
path := filepath.Join(t.TempDir(), "zero.json")
@@ -1403,8 +1420,8 @@ func TestSetNotifyReplacementInheritsNoFieldFromEarlierDuplicate(t *testing.T) {
t.Fatal(err)
}
if _, err := SetNotify(path, NotifyConfig{FocusMode: "always"}); err != nil {
t.Fatalf("SetNotify: %v", err)
if _, err := setNotify(path, NotifyConfig{FocusMode: "always"}); err != nil {
t.Fatalf("setNotify: %v", err)
}
stored, err := UserNotify(path)
if err != nil {
@@ -1437,8 +1454,8 @@ func TestSetNotifySingleMemberReplacedInPlace(t *testing.T) {
if err := os.WriteFile(path, []byte(original), 0o600); err != nil {
t.Fatal(err)
}
if _, err := SetNotify(path, NotifyConfig{Mode: "off", FocusMode: "always"}); err != nil {
t.Fatalf("SetNotify: %v", err)
if _, err := setNotify(path, NotifyConfig{Mode: "off", FocusMode: "always"}); err != nil {
t.Fatalf("setNotify: %v", err)
}
raw, err := os.ReadFile(path)
if err != nil {
+7
View File
@@ -24,6 +24,7 @@ const (
Connectivity
ModelNotFound
ContextOverflow
LocalRPM
)
// providerMarkers are the prefixes the provider layer attaches to every
@@ -59,6 +60,8 @@ func Classify(err error) Category {
return Unknown
}
switch {
case strings.Contains(m, "rate limit error: local rpm limit for"):
return LocalRPM
case containsAny(m, "auth error:", "unauthorized", "api key", "api_key", "invalid_api_key",
"authentication", "permission denied", "forbidden") || containsStatusCode(m, "401", "403"):
return Auth
@@ -89,6 +92,8 @@ func TUIHint(err error) string {
return "API key rejected — run /provider to re-check your credentials"
case RateLimit:
return "Rate limited — wait a moment, or switch model with /model"
case LocalRPM:
return "Local modelRPM cap reached — wait for the window to expire, or switch model with /model"
case Connectivity:
return "Can't reach the provider — run /doctor --connectivity"
case ModelNotFound:
@@ -109,6 +114,8 @@ func CLIHint(err error) string {
return "API key rejected — run `zero setup`, `zero auth openrouter` for OpenRouter, or set the provider's API key"
case RateLimit:
return "Rate limited — wait a moment, or switch model with --model"
case LocalRPM:
return "Local modelRPM cap reached — wait for the window to expire, or switch model with --model"
case Connectivity:
return "Can't reach the provider — run `zero doctor`"
case ModelNotFound:
+1
View File
@@ -17,6 +17,7 @@ func TestClassify(t *testing.T) {
{"raw 401", "provider request error: 401 Unauthorized", Auth},
{"invalid api key", "provider request error: invalid_api_key: incorrect key provided", Auth},
{"rate limit prefix", "rate limit error: 429 too many requests", RateLimit},
{"local RPM", `rate limit error: local RPM limit for "gpt-4.1" (2 requests/60s); retry in 12s`, LocalRPM},
{"overloaded", "provider error: model is overloaded, please retry", RateLimit},
{"resource exhausted gemini", "provider stream error: rpc error: code = ResourceExhausted desc = quota exceeded", RateLimit},
{"context length openai", "provider request error: this model's maximum context length is 128000 tokens", ContextOverflow},
+4
View File
@@ -143,6 +143,10 @@ var descriptors = []Descriptor{
openAICompat("dashscope", "DashScope", "https://dashscope-intl.aliyuncs.com/compatible-mode/v1", "qwen-plus", []string{"DASHSCOPE_API_KEY", "QWEN_API_KEY"}, "qwen"),
openAICompat("moonshot", "Moonshot AI", "https://api.moonshot.ai/v1", "kimi-k2-0905-preview", []string{"MOONSHOT_API_KEY"}, "kimi"),
openAICompat("atlascloud", "Atlas Cloud", "https://api.atlascloud.ai/v1", "qwen/qwen3.5-flash", []string{"ATLASCLOUD_API_KEY"}, "atlas cloud", "atlas"),
// GMI Cloud — OpenAI-compatible inference. The id matches the models.dev
// provider id so the remote catalog resolves without a mapping; GMI's own
// docs name the key GMI_API_KEY, models.dev names it GMICLOUD_API_KEY.
openAICompat("gmicloud", "GMI Cloud", "https://api.gmi-serving.com/v1", "Qwen/Qwen3.8-Max", []string{"GMI_API_KEY", "GMICLOUD_API_KEY"}, "gmi cloud", "gmi"),
openAICompat("longcat", "LongCat", "https://api.longcat.chat/openai", "LongCat-2.0", []string{"LONGCAT_API_KEY"}, "meituan longcat", "meituan", "longcat-2.0"),
openAICompat("nvidia-nim", "NVIDIA NIM", "https://integrate.api.nvidia.com/v1", "nvidia/llama-3.1-nemotron-70b-instruct", []string{"NVIDIA_API_KEY"}, "nvidia nim"),
anthropicCompat("minimax", "MiniMax", "https://api.minimax.io/anthropic", "MiniMax-M3", []string{"MINIMAX_API_KEY"}, "mini-max", "mini_max"),
+30 -1
View File
@@ -27,6 +27,7 @@ var expectedCatalogIDs = []string{
"dashscope",
"moonshot",
"atlascloud",
"gmicloud",
"longcat",
"nvidia-nim",
"minimax",
@@ -201,6 +202,31 @@ func TestAtlasCloudDescriptor(t *testing.T) {
}
}
func TestGMICloudDescriptor(t *testing.T) {
descriptor, err := Require("gmicloud")
if err != nil {
t.Fatalf("Require(gmicloud) error = %v", err)
}
if descriptor.Name != "GMI Cloud" {
t.Fatalf("Name = %q, want GMI Cloud", descriptor.Name)
}
if descriptor.DefaultBaseURL != "https://api.gmi-serving.com/v1" {
t.Fatalf("DefaultBaseURL = %q, want GMI Cloud OpenAI-compatible endpoint", descriptor.DefaultBaseURL)
}
if descriptor.DefaultModel != "Qwen/Qwen3.8-Max" {
t.Fatalf("DefaultModel = %q, want Qwen/Qwen3.8-Max", descriptor.DefaultModel)
}
if descriptor.Transport != TransportOpenAICompatible {
t.Fatalf("Transport = %q, want %q", descriptor.Transport, TransportOpenAICompatible)
}
if !descriptor.RequiresAuth {
t.Fatal("RequiresAuth = false, want true")
}
if !reflect.DeepEqual(descriptor.AuthEnvVars, []string{"GMI_API_KEY", "GMICLOUD_API_KEY"}) {
t.Fatalf("AuthEnvVars = %#v, want GMI_API_KEY then GMICLOUD_API_KEY", descriptor.AuthEnvVars)
}
}
func TestCatalogDescriptorsExposeRequiredDefaults(t *testing.T) {
for _, descriptor := range All() {
if descriptor.ID == "" {
@@ -321,6 +347,9 @@ func TestLookupNormalizesIDsAndAliases(t *testing.T) {
"mini_max": "minimax",
"Moonshot": "moonshot",
"Atlas Cloud": "atlascloud",
"GMI Cloud": "gmicloud",
"gmi-cloud": "gmicloud",
"gmi": "gmicloud",
"nvidia nim": "nvidia-nim",
"xiaomi mimo": "xiaomi-mimo",
"custom_openai_compatible": "custom-openai-compatible",
@@ -371,7 +400,7 @@ func TestListByTransportPreservesCatalogOrder(t *testing.T) {
TransportBedrock: {"bedrock"},
TransportVertex: {"vertex"},
TransportAnthropicCompat: {"minimax", "minimaxi-cn", "opencode-go-anthropic-compatible", "custom-anthropic-compatible"},
TransportOpenAICompat: {"gitlawb-opengateway", "aimlapi", "ollama-cloud", "ollama", "lmstudio", "atomic-chat-local", "openrouter", "huggingface", "chatgpt", "groq", "deepseek", "together", "fireworks", "dashscope", "moonshot", "atlascloud", "longcat", "nvidia-nim", "mistral", "github", "xai", "venice", "xiaomi-mimo", "bankr", "zai", "zai-cn", "kilocode", "opencode", "opencode-go", "atomic-chat", "chatgpt-proxy", "custom-openai-compatible"},
TransportOpenAICompat: {"gitlawb-opengateway", "aimlapi", "ollama-cloud", "ollama", "lmstudio", "atomic-chat-local", "openrouter", "huggingface", "chatgpt", "groq", "deepseek", "together", "fireworks", "dashscope", "moonshot", "atlascloud", "gmicloud", "longcat", "nvidia-nim", "mistral", "github", "xai", "venice", "xiaomi-mimo", "bankr", "zai", "zai-cn", "kilocode", "opencode", "opencode-go", "atomic-chat", "chatgpt-proxy", "custom-openai-compatible"},
}
for transport, wantIDs := range cases {
+6
View File
@@ -149,6 +149,12 @@ var curatedModels = map[string][]Model{
{ID: "qwen/qwen3.5-flash", Description: "catalog default"},
{ID: "deepseek-ai/deepseek-v4-pro", Description: "reasoning model"},
},
"gmicloud": {
{ID: "Qwen/Qwen3.8-Max", Description: "catalog default"},
{ID: "Qwen/Qwen3.8-Flash", Description: "fast model"},
{ID: "deepseek-ai/DeepSeek-V4-Pro", Description: "reasoning model"},
{ID: "deepseek-ai/DeepSeek-V4-Flash", Description: "fast coding model"},
},
"nvidia-nim": {
{ID: "nvidia/llama-3.1-nemotron-70b-instruct", Description: "catalog default"},
{ID: "meta/llama-3.1-70b-instruct", Description: "general model"},
@@ -48,6 +48,11 @@ func TestModelsAreProviderScoped(t *testing.T) {
want: []string{"qwen/qwen3.5-flash", "deepseek-ai/deepseek-v4-pro"},
notWant: []string{"gpt-4.1", "claude-sonnet-4.5", "openai/gpt-4.1"},
},
{
provider: "gmicloud",
want: []string{"Qwen/Qwen3.8-Max", "Qwen/Qwen3.8-Flash", "deepseek-ai/DeepSeek-V4-Pro", "deepseek-ai/DeepSeek-V4-Flash"},
notWant: []string{"gpt-4.1", "claude-sonnet-4.5", "deepseek-ai/deepseek-v4-pro"},
},
{
provider: "mistral",
want: []string{"mistral-large-latest", "codestral-latest"},
+15 -8
View File
@@ -22,14 +22,15 @@ import (
// An error is reported to the loop, which records a note and continues on the
// current model.
//
// The session switcher comes back nil unless the run STARTED optimized. A run
// that began on the default adapter stays on it, so escalation cannot quietly
// change the transport underneath a session.
// The session switcher is installed for an optimized start or configured RPM
// limits. Default-adapter starts retain that transport after escalation.
// The same limiter wraps switched sessions so switching cannot reset a cap.
func EscalationSwitchers(
profile config.ProviderProfile,
provider zeroruntime.Provider,
newProvider func(config.ProviderProfile) (zeroruntime.Provider, error),
onSwitch func(modelID string),
sessionOptions ...Options,
) (
func(context.Context, string) (zeroruntime.Provider, error),
func(context.Context, string) (zeroruntime.TurnSessionProvider, error),
@@ -37,6 +38,10 @@ func EscalationSwitchers(
if newProvider == nil {
return nil, nil
}
options := Options{}
if len(sessionOptions) > 0 {
options = sessionOptions[0]
}
// The escalated profile is the run's profile with the model replaced, so the
// credential, base URL and headers travel with it. Callers pass a newProvider
// that already applies the stored key, which is why there is no per-site key
@@ -62,8 +67,8 @@ func EscalationSwitchers(
return switchedProvider, nil
}
turnSessions, _ := OptimizedTurnSessions(profile, provider, Options{})
if turnSessions == nil {
turnSessions, _ := OptimizedTurnSessions(profile, provider, options)
if turnSessions == nil && options.ModelRPM == nil {
return modelSwitcher, nil
}
@@ -78,12 +83,14 @@ func EscalationSwitchers(
if onSwitch != nil {
onSwitch(modelID)
}
if optimized, ok := OptimizedTurnSessions(switchedProfile, switchedProvider, Options{}); ok {
return optimized, nil
if turnSessions != nil {
if optimized, ok := OptimizedTurnSessions(switchedProfile, switchedProvider, options); ok {
return limitTurnSessions(switchedProfile, optimized, options), nil
}
}
// Ineligible target: the default adapter, but carrying the switched
// model's own capability projection rather than the original's.
return DefaultTurnSessions(switchedProfile, switchedProvider, Options{}), nil
return limitTurnSessions(switchedProfile, DefaultTurnSessions(switchedProfile, switchedProvider, options), options), nil
}
return modelSwitcher, sessionSwitcher
}
+1
View File
@@ -21,6 +21,7 @@ import (
// Options configures provider construction.
type Options struct {
ModelRPM *zeroruntime.ModelRPMLimiter
UserAgent string
HTTPClient *http.Client
ModelRegistry *modelregistry.Registry
+97
View File
@@ -0,0 +1,97 @@
package providers
import (
"context"
"errors"
"github.com/Gitlawb/zero/internal/config"
"github.com/Gitlawb/zero/internal/zeroruntime"
"io"
"net/http"
"strings"
"sync/atomic"
"testing"
)
type rpmRoundTripper struct{ calls atomic.Int64 }
func (r *rpmRoundTripper) RoundTrip(q *http.Request) (*http.Response, error) {
r.calls.Add(1)
return &http.Response{StatusCode: 200, Header: http.Header{"Content-Type": {"text/event-stream"}}, Body: io.NopCloser(strings.NewReader("data: [DONE]\n\n")), Request: q}, nil
}
func TestConfiguredRPMBlocksNPlusOneBeforeHTTP(t *testing.T) {
for _, opt := range []string{"0", "1"} {
t.Run("optimized="+opt, func(t *testing.T) {
t.Setenv(openaiTurnSessionEnv, opt)
r := &rpmRoundTripper{}
o := Options{HTTPClient: &http.Client{Transport: r}, ModelRPM: zeroruntime.NewModelRPMLimiter(map[string]int{"gpt-4.1": 2})}
profile := config.ProviderProfile{Name: "test", ProviderKind: config.ProviderKindOpenAI, Model: "openai:gpt-4.1", APIKey: "fixture-only"}
p, e := New(profile, o)
if e != nil {
t.Fatal(e)
}
for i := 0; i < 3; i++ {
if i == 2 {
profile.Model = "gpt-4.1"
}
ss, ok := ConfiguredTurnSessions(profile, p, o)
if !ok {
t.Fatal("missing wrapper")
}
s, e := ss.OpenTurnSession(context.Background())
if e != nil {
t.Fatal(e)
}
ch, e := s.Stream(context.Background(), zeroruntime.CompletionRequest{Messages: []zeroruntime.Message{{Role: zeroruntime.MessageRoleUser, Content: "fixture"}}})
if i == 2 {
var hit *zeroruntime.RPMLimitError
if !errors.As(e, &hit) || ch != nil {
t.Fatalf("N+1 was not denied: %v", e)
}
} else {
if e != nil {
t.Fatal(e)
}
for range ch {
}
}
if e := s.Close(); e != nil {
t.Fatal(e)
}
}
if r.calls.Load() != 2 {
t.Fatalf("transport=%d want 2", r.calls.Load())
}
})
}
}
func TestEscalationSharesRPMWhenStartingUnoptimized(t *testing.T) {
t.Setenv(openaiTurnSessionEnv, "0")
o := Options{ModelRPM: zeroruntime.NewModelRPMLimiter(map[string]int{"gpt-4.1": 1})}
profile := config.ProviderProfile{Name: "test", ProviderKind: config.ProviderKindOpenAI, Model: "gpt-4.1"}
p := escalationStubProvider{}
ss, _ := ConfiguredTurnSessions(profile, p, o)
s, e := ss.OpenTurnSession(context.Background())
if e != nil {
t.Fatal(e)
}
if _, e = s.Stream(context.Background(), zeroruntime.CompletionRequest{}); e != nil {
t.Fatal(e)
}
_, switcher := EscalationSwitchers(profile, p, func(config.ProviderProfile) (zeroruntime.Provider, error) { return p, nil }, nil, o)
if switcher == nil {
t.Fatal("missing shared session switcher")
}
ss, e = switcher(context.Background(), "openai:gpt-4.1")
if e != nil {
t.Fatal(e)
}
s, e = ss.OpenTurnSession(context.Background())
if e != nil {
t.Fatal(e)
}
_, e = s.Stream(context.Background(), zeroruntime.CompletionRequest{})
var hit *zeroruntime.RPMLimitError
if !errors.As(e, &hit) {
t.Fatalf("switch reset cap: %v", e)
}
}
+98 -15
View File
@@ -14,6 +14,7 @@ import (
"runtime"
"strconv"
"strings"
"sync"
"time"
"github.com/Gitlawb/zero/internal/trace"
@@ -89,6 +90,22 @@ func ContentStallTimeout(idleTimeout time.Duration) time.Duration {
// entirely (streams may then hang until the HTTP/transport layer gives up).
const streamIdleTimeoutEnv = "ZERO_STREAM_IDLE_TIMEOUT"
// bareSecondsDuration converts a positive second count into a time.Duration.
// time.Duration is int64 nanoseconds, so a count that passes strconv.Atoi can
// still overflow when multiplied by time.Second and wrap to 0, which would
// disable a timeout that should have stayed at its default. ok is false when
// the product does not fit.
func bareSecondsDuration(secs int) (time.Duration, bool) {
if secs <= 0 {
return 0, false
}
const maxSecs = int64(math.MaxInt64) / int64(time.Second)
if int64(secs) > maxSecs {
return 0, false
}
return time.Duration(secs) * time.Second, true
}
// ResolveStreamIdleTimeout selects the effective stream idle timeout. Precedence:
// an explicit positive option (e.g. set by a test) wins; otherwise the
// ZERO_STREAM_IDLE_TIMEOUT env override if set and valid; otherwise
@@ -105,15 +122,59 @@ func ResolveStreamIdleTimeout(option time.Duration) time.Duration {
if d, err := time.ParseDuration(raw); err == nil && d > 0 {
return d
}
if secs, err := strconv.Atoi(raw); err == nil && secs > 0 {
return time.Duration(secs) * time.Second
if secs, err := strconv.Atoi(raw); err == nil {
if d, ok := bareSecondsDuration(secs); ok {
return d
}
}
// Unparseable / non-positive: fall through to the default rather than
// silently disabling the watchdog on a typo.
// Unparseable / non-positive / overflow: fall through to the default
// rather than silently disabling the watchdog.
}
return DefaultStreamIdleTimeout
}
// DefaultResponseHeaderTimeout is how long the shared HTTP transport waits for
// a response header after the request is written. 120s, not 60s: a slow cloud
// proxy (e.g. ollama `*:cloud`) can withhold its 200 response header until the
// upstream model emits a first token, so a 60s cap risked aborting a
// legitimately-slow-but-alive request. 120s still bounds a truly dead reused
// connection (which never responds) while tolerating slow header delivery; slow
// first tokens after the header are covered by the idle + content-stall
// watchdogs.
const DefaultResponseHeaderTimeout = 120 * time.Second
// responseHeaderTimeoutEnv is the global override for the response header
// timeout. It accepts the same forms as ZERO_STREAM_IDLE_TIMEOUT: a Go duration
// ("5m", "300s", "90s") or a bare number of seconds ("300"). A value of "0",
// "off", "none", or "disabled" removes the limit entirely (a connection that
// never answers may then wait until the request context ends). Useful when a
// local model server needs longer than DefaultResponseHeaderTimeout to produce
// the first byte, for example a cold model load on a throttled Ollama.
const responseHeaderTimeoutEnv = "ZERO_RESPONSE_HEADER_TIMEOUT"
// ResolveResponseHeaderTimeout selects the effective response header timeout:
// the ZERO_RESPONSE_HEADER_TIMEOUT env override if set and valid, otherwise
// DefaultResponseHeaderTimeout. A returned value <= 0 means no limit.
func ResolveResponseHeaderTimeout() time.Duration {
if raw := strings.TrimSpace(os.Getenv(responseHeaderTimeoutEnv)); raw != "" {
switch strings.ToLower(raw) {
case "0", "off", "none", "disabled":
return 0
}
if d, err := time.ParseDuration(raw); err == nil && d > 0 {
return d
}
if secs, err := strconv.Atoi(raw); err == nil {
if d, ok := bareSecondsDuration(secs); ok {
return d
}
}
// Unparseable / non-positive / overflow: fall through to the default
// rather than silently removing the limit.
}
return DefaultResponseHeaderTimeout
}
// NormalizeBaseURL trims trailing slashes and validates an HTTP API base URL.
func NormalizeBaseURL(baseURL string, defaultBaseURL string, label string) (string, error) {
baseURL = strings.TrimSpace(baseURL)
@@ -127,7 +188,18 @@ func NormalizeBaseURL(baseURL string, defaultBaseURL string, label string) (stri
return baseURL, nil
}
// sharedHTTPClient is the process-wide client used when a provider supplies none.
// sharedHTTPClients holds one stall-hardened client per resolved response-header
// timeout. The timeout is read when HTTPClient is called, not at package init:
// a test (or a process that sets ZERO_RESPONSE_HEADER_TIMEOUT before building
// a provider) must see that value on the transport it actually dials. The
// transport field is never rewritten after creation, so concurrent requests
// keep the conn pool of the timeout they resolved.
var sharedHTTPClients = struct {
mu sync.Mutex
byTimeout map[time.Duration]*http.Client
}{}
// stallHardenedClient builds the client used when a provider supplies none.
// It tunes the default transport to defeat the stale-pooled-connection hang: Go
// keeps idle keep-alive connections in a pool, and a later request can reuse one
// the server/NAT has silently dropped. Because the model call is a POST (non-
@@ -159,15 +231,11 @@ func NormalizeBaseURL(baseURL string, defaultBaseURL string, label string) (stri
// to the minutes-long stalls this avoids — and this doesn't touch
// Linux/Windows, where the underlying OS doesn't keep dead/degraded
// pooled connections around as long.
var sharedHTTPClient = func() *http.Client {
func stallHardenedClient(headerTimeout time.Duration) *http.Client {
transport := http.DefaultTransport.(*http.Transport).Clone()
// 120s, not 60s: a slow cloud proxy (e.g. ollama `*:cloud`) can withhold its
// 200 response header until the upstream model emits a first token, so a 60s cap
// risked aborting a legitimately-slow-but-alive request. 120s still bounds a
// truly dead reused connection (which never responds) while tolerating slow
// header delivery; slow first tokens after the header are covered by the idle +
// content-stall watchdogs.
transport.ResponseHeaderTimeout = 120 * time.Second
// DefaultResponseHeaderTimeout (120s) unless ZERO_RESPONSE_HEADER_TIMEOUT
// overrides it; see the constant for why the default is 120s and not 60s.
transport.ResponseHeaderTimeout = headerTimeout
transport.IdleConnTimeout = 30 * time.Second
// Periodically close idle connections to prevent stale HTTP/2
// connections from causing PROTOCOL_ERROR on the next request.
@@ -183,7 +251,22 @@ var sharedHTTPClient = func() *http.Client {
}
transport.DisableKeepAlives = runtime.GOOS == "darwin"
return &http.Client{Transport: transport}
}()
}
func sharedStallClient() *http.Client {
timeout := ResolveResponseHeaderTimeout()
sharedHTTPClients.mu.Lock()
defer sharedHTTPClients.mu.Unlock()
if sharedHTTPClients.byTimeout == nil {
sharedHTTPClients.byTimeout = make(map[time.Duration]*http.Client)
}
if client := sharedHTTPClients.byTimeout[timeout]; client != nil {
return client
}
client := stallHardenedClient(timeout)
sharedHTTPClients.byTimeout[timeout] = client
return client
}
const defaultIdleConnCloseInterval = 30 * time.Second
@@ -221,7 +304,7 @@ func HTTPClient(client *http.Client) *http.Client {
if client != nil {
return client
}
return sharedHTTPClient
return sharedStallClient()
}
// SendEvent writes a provider event without blocking cancellation cleanup.
@@ -340,6 +340,10 @@ func TestStreamTimeoutMessage(t *testing.T) {
// response-header wait + shorter idle-conn reuse) that defeats the macOS stale-
// pooled-connection hang; an explicit client is returned untouched.
func TestHTTPClientReturnsStallHardenedSharedClient(t *testing.T) {
// Package init must not bake the timeout. An ambient
// ZERO_RESPONSE_HEADER_TIMEOUT would otherwise make this assertion
// depend on whatever the process inherited.
t.Setenv("ZERO_RESPONSE_HEADER_TIMEOUT", "")
got := HTTPClient(nil)
if got == nil {
t.Fatal("HTTPClient(nil) returned nil")
@@ -374,6 +378,66 @@ func TestHTTPClientReturnsStallHardenedSharedClient(t *testing.T) {
}
}
// The shared transport must enforce the timeout ResolveResponseHeaderTimeout
// returns. Checking the resolver alone stays green if the client still carries
// the value captured at init.
func TestHTTPClientTransportUsesResolvedHeaderTimeout(t *testing.T) {
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
delay, err := time.ParseDuration(r.URL.Query().Get("delay"))
if err != nil {
http.Error(w, "bad delay", http.StatusBadRequest)
return
}
time.Sleep(delay)
w.WriteHeader(http.StatusNoContent)
}))
t.Cleanup(srv.Close)
t.Setenv("ZERO_RESPONSE_HEADER_TIMEOUT", "200ms")
short := HTTPClient(nil)
tr, ok := short.Transport.(*http.Transport)
if !ok {
t.Fatalf("transport type = %T, want *http.Transport", short.Transport)
}
if tr.ResponseHeaderTimeout != ResolveResponseHeaderTimeout() || tr.ResponseHeaderTimeout != 200*time.Millisecond {
t.Fatalf("ResponseHeaderTimeout = %v, resolver = %v, want 200ms", tr.ResponseHeaderTimeout, ResolveResponseHeaderTimeout())
}
started := time.Now()
resp, err := short.Get(srv.URL + "?delay=1s")
if resp != nil {
_ = resp.Body.Close()
}
if err == nil {
t.Fatal("slow header succeeded; the transport did not apply the 200ms timeout")
}
if !strings.Contains(err.Error(), "timeout awaiting response headers") {
t.Fatalf("error = %v, want the transport's response-header timeout", err)
}
if elapsed := time.Since(started); elapsed >= 800*time.Millisecond {
t.Fatalf("header wait took %v; the transport waited on the handler instead of the resolved timeout", elapsed)
}
t.Setenv("ZERO_RESPONSE_HEADER_TIMEOUT", "2s")
long := HTTPClient(nil)
if long == short {
t.Fatal("a new resolved timeout reused the transport that still has the previous timeout")
}
if got := long.Transport.(*http.Transport).ResponseHeaderTimeout; got != 2*time.Second {
t.Fatalf("ResponseHeaderTimeout = %v, want 2s", got)
}
resp, err = long.Get(srv.URL + "?delay=50ms")
if err != nil {
t.Fatalf("header within the resolved timeout failed: %v", err)
}
_ = resp.Body.Close()
if resp.StatusCode != http.StatusNoContent {
t.Fatalf("status = %d, want 204", resp.StatusCode)
}
if again := HTTPClient(nil); again != long {
t.Fatal("the same resolved timeout must keep one shared client")
}
}
// startIdleConnCloser periodically closes idle pooled connections so stale HTTP/2
// connections are not reused across long idle periods.
func TestStartIdleConnCloserClosesIdleConnections(t *testing.T) {
@@ -0,0 +1,73 @@
package providerio
import (
"testing"
"time"
)
func TestResolveResponseHeaderTimeout(t *testing.T) {
const env = "ZERO_RESPONSE_HEADER_TIMEOUT"
t.Run("default when env is unset or empty", func(t *testing.T) {
t.Setenv(env, "")
if got := ResolveResponseHeaderTimeout(); got != DefaultResponseHeaderTimeout {
t.Fatalf("got %v, want default %v", got, DefaultResponseHeaderTimeout)
}
})
t.Run("default keeps the value that was previously hardcoded", func(t *testing.T) {
if DefaultResponseHeaderTimeout != 120*time.Second {
t.Fatalf("default is %v; this override must not change the 120s default", DefaultResponseHeaderTimeout)
}
})
t.Run("env Go duration", func(t *testing.T) {
t.Setenv(env, "240s")
if got := ResolveResponseHeaderTimeout(); got != 240*time.Second {
t.Fatalf("got %v, want 240s", got)
}
t.Setenv(env, "5m")
if got := ResolveResponseHeaderTimeout(); got != 5*time.Minute {
t.Fatalf("got %v, want 5m", got)
}
})
t.Run("env bare seconds", func(t *testing.T) {
t.Setenv(env, "300")
if got := ResolveResponseHeaderTimeout(); got != 300*time.Second {
t.Fatalf("got %v, want 300s", got)
}
})
t.Run("env value is trimmed", func(t *testing.T) {
t.Setenv(env, " 90s ")
if got := ResolveResponseHeaderTimeout(); got != 90*time.Second {
t.Fatalf("got %v, want 90s", got)
}
})
t.Run("env removes the limit", func(t *testing.T) {
for _, value := range []string{"0", "off", "none", "disabled", "OFF", "Disabled"} {
t.Setenv(env, value)
if got := ResolveResponseHeaderTimeout(); got != 0 {
t.Fatalf("%q: got %v, want 0 (no limit)", value, got)
}
}
})
t.Run("bare seconds that overflow time.Duration keep the default", func(t *testing.T) {
t.Setenv(env, "36028797018963968")
if got := ResolveResponseHeaderTimeout(); got != DefaultResponseHeaderTimeout {
t.Fatalf("got %v, want default %v (overflow must not become a zero timeout)", got, DefaultResponseHeaderTimeout)
}
})
t.Run("invalid env falls back to default, not unlimited", func(t *testing.T) {
for _, value := range []string{"banana", "-5s", "-1", "1.5x"} {
t.Setenv(env, value)
if got := ResolveResponseHeaderTimeout(); got != DefaultResponseHeaderTimeout {
t.Fatalf("%q: got %v, want default %v (a typo must not remove the limit)", value, got, DefaultResponseHeaderTimeout)
}
}
})
}
@@ -49,6 +49,13 @@ func TestResolveStreamIdleTimeout(t *testing.T) {
}
})
t.Run("bare seconds that overflow time.Duration keep the default", func(t *testing.T) {
t.Setenv(env, "36028797018963968")
if got := ResolveStreamIdleTimeout(0); got != DefaultStreamIdleTimeout {
t.Fatalf("got %v, want default %v (overflow must not disable the watchdog)", got, DefaultStreamIdleTimeout)
}
})
t.Run("invalid env falls back to default, not disabled", func(t *testing.T) {
t.Setenv(env, "banana")
if got := ResolveStreamIdleTimeout(0); got != DefaultStreamIdleTimeout {
+22
View File
@@ -101,3 +101,25 @@ func DefaultTurnSessions(profile config.ProviderProfile, provider zeroruntime.Pr
}
return zeroruntime.NewProviderTurnSessionProvider(provider, caps)
}
// ConfiguredTurnSessions preserves transport selection and wraps the starting
// session when RPM is configured. Without either feature, the default is nil.
func ConfiguredTurnSessions(profile config.ProviderProfile, p zeroruntime.Provider, o Options) (zeroruntime.TurnSessionProvider, bool) {
s, ok := OptimizedTurnSessions(profile, p, o)
if o.ModelRPM == nil {
return s, ok
}
if !ok {
s = DefaultTurnSessions(profile, p, o)
}
return limitTurnSessions(profile, s, o), true
}
func limitTurnSessions(profile config.ProviderProfile, s zeroruntime.TurnSessionProvider, o Options) zeroruntime.TurnSessionProvider {
model := strings.TrimSpace(profile.Model)
if registry, err := defaultRegistry(o.ModelRegistry); err == nil {
if id, ok := registry.ResolveID(model); ok {
model = id
}
}
return o.ModelRPM.Wrap(model, s)
}
+3
View File
@@ -89,6 +89,7 @@ type model struct {
// allowEscalation mirrors Options.AllowEscalation: it gates the per-run model
// switchers, and the caller gates the escalate_model tool on the same flag.
allowEscalation bool
modelRPM *zeroruntime.ModelRPMLimiter
newProvider func(config.ProviderProfile) (zeroruntime.Provider, error)
newTurnSessionProvider func(config.ProviderProfile, zeroruntime.Provider) zeroruntime.TurnSessionProvider
probeProviderHealth func(context.Context, providerhealth.Options) providerhealth.Result
@@ -1055,6 +1056,7 @@ func newModel(ctx context.Context, options Options) model {
sandboxSetupCommand: options.SandboxSetupCommand,
agentOptions: options.AgentOptions,
allowEscalation: options.AllowEscalation,
modelRPM: options.ModelRPM,
sessionCompactor: options.SessionCompactor,
runtimeMessageSink: options.RuntimeMessageSink,
permissionMode: permissionMode,
@@ -5681,6 +5683,7 @@ func (m model) runAgentWithOptions(runID int, runCtx context.Context, prompt str
// currentModel: every usage event after a real escalation is billed
// to the escalated model, not the one the run started on.
func(modelID string) { usageModelID = modelID },
providers.Options{ModelRPM: m.modelRPM},
)
}
options.SupportsVision = func(modelID string) bool {
+1
View File
@@ -22,6 +22,7 @@ import (
// Options configures the reusable Zero terminal UI shell.
type Options struct {
ModelRPM *zeroruntime.ModelRPMLimiter
Cwd string
Version string // CLI build version, shown on the home screen; empty hides it
UserConfigPath string
+101
View File
@@ -0,0 +1,101 @@
package zeroruntime
import (
"context"
"fmt"
"sync"
"time"
)
// ModelRPMLimiter shares a sliding 60-second window across turn sessions in
// one runtime process. Separate processes do not share state.
type ModelRPMLimiter struct {
mu sync.Mutex
limits map[string]int
windows map[string][]time.Time
now func() time.Time
}
// NewModelRPMLimiter copies validated canonical-model limits. Zero/unset is off.
func NewModelRPMLimiter(limits map[string]int) *ModelRPMLimiter {
active := make(map[string]int)
for m, n := range limits {
if n > 0 {
active[m] = n
}
}
if len(active) == 0 {
return nil
}
return &ModelRPMLimiter{limits: active, windows: make(map[string][]time.Time), now: time.Now}
}
// RPMLimitError denotes rejection before the wrapped Stream method is called.
type RPMLimitError struct {
Model string
Limit int
RetryAfter time.Duration
}
func (e *RPMLimitError) Error() string {
retry := e.RetryAfter.Round(time.Second)
if retry < e.RetryAfter {
retry += time.Second
}
return fmt.Sprintf("rate limit error: local RPM limit for %q (%d requests/60s); retry in %s", e.Model, e.Limit, retry)
}
// Wrap gates Stream admissions, not setup/prewarm, compaction or transport retries.
func (l *ModelRPMLimiter) Wrap(model string, p TurnSessionProvider) TurnSessionProvider {
if l == nil || l.limits[model] == 0 || p == nil {
return p
}
return rpmProvider{TurnSessionProvider: p, limiter: l, model: model}
}
func (l *ModelRPMLimiter) admit(ctx context.Context, model string) error {
l.mu.Lock()
defer l.mu.Unlock()
if err := ctx.Err(); err != nil {
return err
}
now := l.now()
w := l.windows[model]
first := 0
for first < len(w) && !w[first].After(now.Add(-time.Minute)) {
first++
}
w = w[:copy(w, w[first:])]
l.windows[model] = w
if len(w) >= l.limits[model] {
return &RPMLimitError{Model: model, Limit: l.limits[model], RetryAfter: w[0].Add(time.Minute).Sub(now)}
}
l.windows[model] = append(w, now)
return nil
}
type rpmProvider struct {
TurnSessionProvider
limiter *ModelRPMLimiter
model string
}
func (p rpmProvider) OpenTurnSession(ctx context.Context) (TurnSession, error) {
s, err := p.TurnSessionProvider.OpenTurnSession(ctx)
if err != nil {
return nil, err
}
return rpmSession{TurnSession: s, limiter: p.limiter, model: p.model}, nil
}
type rpmSession struct {
TurnSession
limiter *ModelRPMLimiter
model string
}
func (s rpmSession) Stream(ctx context.Context, r CompletionRequest) (<-chan StreamEvent, error) {
if err := s.limiter.admit(ctx, s.model); err != nil {
return nil, err
}
return s.TurnSession.Stream(ctx, r)
}
+128
View File
@@ -0,0 +1,128 @@
package zeroruntime
import (
"context"
"errors"
"sync"
"sync/atomic"
"testing"
"time"
)
type rpmTransport struct{ calls atomic.Int64 }
func (p *rpmTransport) StreamCompletion(context.Context, CompletionRequest) (<-chan StreamEvent, error) {
p.calls.Add(1)
ch := make(chan StreamEvent)
close(ch)
return ch, nil
}
func rpmSessionFor(t *testing.T, l *ModelRPMLimiter, m string, p Provider) TurnSession {
t.Helper()
s, e := l.Wrap(m, NewProviderTurnSessionProvider(p, ProviderCapabilities{Model: m})).OpenTurnSession(context.Background())
if e != nil {
t.Fatal(e)
}
return s
}
func TestRPMRejectsNPlusOneBeforeTransportAcrossSessions(t *testing.T) {
l := NewModelRPMLimiter(map[string]int{"m": 3})
clock := time.Unix(0, 0)
l.now = func() time.Time { return clock }
p := &rpmTransport{}
for i := 0; i < 3; i++ {
if _, e := rpmSessionFor(t, l, "m", p).Stream(context.Background(), CompletionRequest{}); e != nil {
t.Fatal(e)
}
}
s := rpmSessionFor(t, l, "m", p)
ch, e := s.Stream(context.Background(), CompletionRequest{})
var hit *RPMLimitError
if !errors.As(e, &hit) || ch != nil || hit.RetryAfter != time.Minute {
t.Fatalf("N+1: stream=%v error=%v", ch, e)
}
if p.calls.Load() != 3 {
t.Fatalf("transport calls=%d, want 3", p.calls.Load())
}
clock = clock.Add(time.Minute)
if _, e = s.Stream(context.Background(), CompletionRequest{}); e != nil {
t.Fatal(e)
}
}
func TestRPMSlidingWindow(t *testing.T) {
l := NewModelRPMLimiter(map[string]int{"m": 2})
clock := time.Unix(59, 0)
l.now = func() time.Time { return clock }
s := rpmSessionFor(t, l, "m", &rpmTransport{})
for i := 0; i < 2; i++ {
if _, e := s.Stream(context.Background(), CompletionRequest{}); e != nil {
t.Fatal(e)
}
}
clock = time.Unix(60, 0)
_, e := s.Stream(context.Background(), CompletionRequest{})
var hit *RPMLimitError
if !errors.As(e, &hit) || hit.RetryAfter != 59*time.Second {
t.Fatalf("minute boundary reset window: %v", e)
}
}
func TestRPMConcurrentAdmissions(t *testing.T) {
l := NewModelRPMLimiter(map[string]int{"m": 7})
l.now = func() time.Time { return time.Unix(0, 0) }
p := &rpmTransport{}
var grants, unexpected atomic.Int64
var wg sync.WaitGroup
start := make(chan struct{})
for i := 0; i < 100; i++ {
s := rpmSessionFor(t, l, "m", p)
wg.Add(1)
go func() {
defer wg.Done()
<-start
_, e := s.Stream(context.Background(), CompletionRequest{})
var hit *RPMLimitError
if e == nil {
grants.Add(1)
} else if !errors.As(e, &hit) {
unexpected.Add(1)
}
}()
}
close(start)
wg.Wait()
if grants.Load() != 7 || p.calls.Load() != 7 || unexpected.Load() != 0 {
t.Fatalf("grants=%d transport=%d unexpected=%d", grants.Load(), p.calls.Load(), unexpected.Load())
}
}
func TestRPMCancellationFailedAttemptsAndIndependentModels(t *testing.T) {
l := NewModelRPMLimiter(map[string]int{"a": 1, "b": 1})
p := &recordingProvider{err: errors.New("fixture transport failure")}
s := rpmSessionFor(t, l, "a", p)
ctx, cancel := context.WithCancel(context.Background())
cancel()
if _, e := s.Stream(ctx, CompletionRequest{}); !errors.Is(e, context.Canceled) {
t.Fatal(e)
}
if _, e := s.Stream(context.Background(), CompletionRequest{}); !errors.Is(e, p.err) {
t.Fatal(e)
}
_, e := s.Stream(context.Background(), CompletionRequest{})
var hit *RPMLimitError
if !errors.As(e, &hit) || len(p.requests) != 1 {
t.Fatalf("failed attempt refunded: %v", e)
}
_, _ = rpmSessionFor(t, l, "b", p).Stream(context.Background(), CompletionRequest{})
for i := 0; i < 3; i++ {
_, _ = rpmSessionFor(t, l, "unlimited", p).Stream(context.Background(), CompletionRequest{})
}
if len(p.requests) != 5 {
t.Fatal("other model incorrectly blocked")
}
}
func TestRPMDisabled(t *testing.T) {
l := NewModelRPMLimiter(map[string]int{"m": 0})
p := &providerTurnSessionProvider{provider: &rpmTransport{}}
if l != nil || l.Wrap("m", p) != p {
t.Fatal("disabled limit altered provider")
}
}