Compare commits

...
Author SHA1 Message Date
Gnanam 3498ee4882 fix(providerio): disable HTTP keep-alive reuse on macOS to defeat degraded-connection stalls
This codebase already has a documented, previously-diagnosed macOS-only
bug: "sharedHTTPClient... defeats the stale-pooled-connection hang...
reproduced on BOTH chatgpt and ollama... surfaces on macOS far more than
Linux because macOS keeps dead pooled connections around longer." The
existing mitigation (ResponseHeaderTimeout=120s, IdleConnTimeout=30s)
catches a reused connection that never responds at all, or one that sat
idle too long.

Traced a live report that doesn't fit either case: a session accumulated
301 real completion tokens over ~12.5 minutes (~0.4 tok/s) on
chatgpt/gpt-5.5 before being manually cancelled - not silent, just
catastrophically slow, on a retry shortly after a prior attempt on the
same host. That's a third failure mode: a reused connection that's
alive-but-severely-degraded, still delivering real bytes (enough to keep
resetting Zero's stream idle/content-stall watchdogs, which only fire on
true silence) but never at a usable rate. Basic network diagnostics on
the affected machine (DNS, TCP connect, TLS handshake) were all healthy
and routed directly with no VPN/proxy involved, pointing at the same
connection-reuse class of bug rather than a local network misconfig.

DisableKeepAlives removes pooling from the equation entirely rather than
just tightening the timeout window, since a degraded-not-dead connection
isn't reliably distinguishable from genuine backend slowness once inside
the stream. Scoped to darwin only via runtime.GOOS: a fresh TCP+TLS
handshake per request costs low tens of milliseconds (measured ~92ms
against a real host on the affected machine) against potential
minutes-long stalls, and Linux/Windows don't exhibit this OS-level
pooled-connection behavior, so they keep full connection reuse.

New test: TestHTTPClientReturnsStallHardenedSharedClient extended to
assert DisableKeepAlives matches runtime.GOOS == "darwin".
2026-07-01 21:38:30 +05:30
2 changed files with 29 additions and 0 deletions

No files matched your search

@@ -10,6 +10,7 @@ import (
"net/url"
"os"
"regexp"
"runtime"
"strconv"
"strings"
"time"
@@ -122,6 +123,21 @@ func NormalizeBaseURL(baseURL string, defaultBaseURL string, label string) (stri
// unaffected (the stream-idle + content watchdogs cover that).
// - IdleConnTimeout is shortened so a connection that went idle across a pause
// is closed (and re-dialed fresh) rather than reused after it has gone stale.
// - DisableKeepAlives on darwin only: ResponseHeaderTimeout/IdleConnTimeout
// above catch a reused connection that's fully dead (never responds) or
// idle past the timeout, but not one that's alive-but-severely-degraded —
// e.g. reused shortly after a prior request on the same host (well within
// 30s, common across quick retries of the same turn), where it still
// delivers real bytes, just at a crippled rate, resetting Zero's stream
// idle/content-stall watchdogs (which only fire on true silence) without
// ever recovering. That degraded-not-dead case is indistinguishable from
// genuine backend slowness from inside the stream, so the only reliable
// fix is removing pooling from the equation entirely on the one platform
// where this class of bug has actually reproduced. A fresh TCP+TLS
// handshake per request costs low tens of milliseconds — negligible next
// to the minutes-long stalls this avoids — and this doesn't touch
// Linux/Windows, where the underlying OS doesn't keep dead/degraded
// pooled connections around as long.
var sharedHTTPClient = func() *http.Client {
transport := http.DefaultTransport.(*http.Transport).Clone()
// 120s, not 60s: a slow cloud proxy (e.g. ollama `*:cloud`) can withhold its
@@ -132,6 +148,7 @@ var sharedHTTPClient = func() *http.Client {
// content-stall watchdogs.
transport.ResponseHeaderTimeout = 120 * time.Second
transport.IdleConnTimeout = 30 * time.Second
transport.DisableKeepAlives = runtime.GOOS == "darwin"
return &http.Client{Transport: transport}
}()
@@ -5,6 +5,7 @@ import (
"errors"
"io"
"net/http"
"runtime"
"strings"
"testing"
"time"
@@ -324,6 +325,17 @@ func TestHTTPClientReturnsStallHardenedSharedClient(t *testing.T) {
if tr.IdleConnTimeout != 30*time.Second {
t.Fatalf("IdleConnTimeout = %v, want 30s", tr.IdleConnTimeout)
}
// DisableKeepAlives is the stronger mitigation scoped to darwin only: the
// two timeouts above catch a reused connection that's fully dead or idle
// too long, but not one that's alive-and-degraded (still delivers real
// bytes, just at a crippled rate) — which resets Zero's stream watchdogs
// without ever recovering. Removing pooling from the equation entirely is
// only worth its reconnect cost on the platform this class of bug has
// actually reproduced on.
wantDisableKeepAlives := runtime.GOOS == "darwin"
if tr.DisableKeepAlives != wantDisableKeepAlives {
t.Fatalf("DisableKeepAlives = %v, want %v (GOOS=%s)", tr.DisableKeepAlives, wantDisableKeepAlives, runtime.GOOS)
}
if HTTPClient(nil) != got {
t.Fatal("HTTPClient(nil) must return a shared instance so the conn pool is reused")
}