Files
98736457bd feat(web-search): add AI/ML API Toolbox as an opt-in backend (#1189)
* feat(aimlapi): shared service key and attribution for aimlapi.com services

Adds the pieces the Toolbox and MCP integrations share:

- internal/aimlapi: X-AIMLAPI-Source (agent/zero), the fixed Toolbox and
  MCP URLs, and an exact-hostname check (https, default port, no
  userinfo) for api/tools/mcp.aimlapi.com before the key or attribution
  headers are attached. The partner id still comes from ResolvePartnerID.
- internal/config: AIMLAPIServiceKey picks the key from the aimlapi
  profile (or AIMLAPI_API_KEY without one), reading the credential
  store, and withholds it when any aimlapi-keyed profile points at a
  proxy base URL or AIMLAPI_INFERENCE_URL is overridden. Keys that are
  not a single token are refused.

* feat(web-search): AI/ML API Toolbox as an opt-in web_search backend

ZERO_WEBSEARCH_PROVIDER=aimlapi runs the Toolbox tool web/search at the
fixed https://tools.aimlapi.com/v1/tools URL. It is selected only by that
setting; an AI/ML API key alone never enables it or registers web_search.

- The key comes from config.LoadAIMLAPIServiceKey on every search; a
  missing, proxy-profile or invalid key fails before any request.
  ZERO_WEBSEARCH_BASE_URL and ZERO_WEBSEARCH_API_KEY are ignored.
- The key and attribution headers are attached only after
  aimlapi.IsAttributedURL accepts the URL, and redirects are refused.
- Every run carries max_cost_usd 0.025 and a fresh idempotency key;
  limit maps to max_results and domains to include_domains, with the
  local domain filter kept.
- A queued run (202) is polled until it ends, honouring
  next_poll_hint_ms clamped to 250 ms..5 s. This backend gets a 30 s
  budget with wait_ms 20 s; the generic backends keep 10 s.
- Non-2xx responses, is_error runs, failed/stopped/timed_out runs and
  malformed bodies are errors carrying the error code and request id,
  with the key scrubbed.

* fix(aimlapi): preserve store errors and isolate Windows config tests

* fix(web-search): redact results and preserve domain errors

* docs(web-search): clarify minimum search charge

* fix(aimlapi): reject Unicode whitespace in service keys

* fix(aimlapi): report store reads after proxy eligibility checks

* fix(web-search): keep plain no-results for unscoped backends

Only a backend that applied the domain allowlist itself (the AI/ML API
Toolbox) should turn an empty result into the "no results matched
domains" error. A generic backend never saw the allowlist, so an empty
result from it is an ordinary "No results for query", as before this
branch.

Also clear ZERO_WEBSEARCH_PROVIDER in the unconfigured registry test:
ZERO_WEBSEARCH_PROVIDER=aimlapi alone now registers web_search, so an
exported value flipped the test's oracle.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(aimlapi): resolve the Toolbox key without an active provider

A user config that lists several providers but marks none active (or
names a removed one) makes Resolve return ErrNoActiveProvider together
with the normalized profiles. The app keeps running on that partial
config, but LoadAIMLAPIServiceKey stopped on the error, so web_search
failed even with an aimlapi profile or AIMLAPI_API_KEY set. Look the
key up in the partial config instead; the proxy check still scans every
resolved profile, and all other resolve errors are still returned.

Also state the key rules in the README as implemented: user config
only (project .zero providers are not used), which aimlapi profile is
used, and that any aimlapi profile or any profile reading
AIMLAPI_API_KEY on a non-canonical base URL withholds the key.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(aimlapi): withhold the Toolbox key for dropped proxy profiles

Resolve drops a non-active profile it cannot normalize (an unknown or
unsupported catalog entry, a missing base URL, ...), so the proxy check
never saw it. A dropped profile that reads AIMLAPI_API_KEY from a proxy
still shows the key belongs to that proxy, yet the key was sent to
tools.aimlapi.com. LoadAIMLAPIServiceKey now also checks the configured
user profiles, merged the same way Resolve merges them, and withholds
the key when a dropped profile carries it and is not explicitly on the
canonical base URL.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* test(web-search): pin the generic backend route in BASE_URL registration tests

defaultSearchBackend now checks ZERO_WEBSEARCH_PROVIDER=aimlapi before
ZERO_WEBSEARCH_BASE_URL. TestWebSearchRegisteredInCoreNetworkTools and
TestCoreNetworkToolsExposeSafetyMetadata set only the base URL and inherit
the provider from the process environment, so with an exported
ZERO_WEBSEARCH_PROVIDER=aimlapi they took the Toolbox route and still passed
even if generic base-URL registration broke. Clear the provider so they
exercise the generic route they describe, matching
TestCoreNetworkToolsOmitWebSearchWhenUnconfigured.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(redaction): redact caller-supplied secrets split by invisible characters

RedactString replaced Options.ExtraSecretValues only as contiguous text,
and the gap-tolerant pass from #1067 covers only the built-in credential
shapes. An opaque key such as the AI/ML API service key, echoed by a
service with a NUL or zero-width space inside it, matched neither pass.
Both fragments then reached Toolbox web_search errors (before clipping)
and successful results sent to the model and transcript.

Match each caller-supplied secret with runs of the same invisible
separators allowed between its characters, right after the contiguous
replacement. Separators inside the match are covered; those beside it
stay. Text without a separator takes the old path unchanged.

Regressions: split explicit values in the redaction package, and split
keys in the Toolbox HTTP error, run error and successful-result paths.
Raised by Copilot on #1189.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(redaction): claim each caller secret whole or split before shorter ones

RedactString replaced every ExtraSecretValues entry contiguously, longest
first, and only then ran the split match. A shorter secret inside a
longer one that was split by an invisible character was replaced first,
so "abc\x00def" with secrets abcdef and abc became "[REDACTED]\x00def"
and the rest of the longer secret stayed visible.

Run the contiguous and split match together per value, longest first,
so a longer value is claimed in either form before any shorter one.
Regressions cover a shorter secret at the start, end and middle of a
split longer one, and alongside it. Raised by CodeRabbit on #1189.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(redaction): merge overlapping caller-secret occurrences before replacing

ExtraSecretValues were replaced one occurrence at a time, so each
replacement could consume bytes another occurrence needed. With secret
abcabc, "abc\x00abcabc" became "abc\x00[REDACTED]". A key whose first and
last characters match lost all but one character the same way
("xY7q​[REDACTED]"), and two partly overlapping values left the
rest of one visible. Reordering the passes cannot fix this: the
contiguous and regexp replacements are both non-overlapping.

Find every occurrence of every value, whole or split by invisible
separators, in one separator-free copy of the text (linear KMP, all
overlapping starts), map the spans back, merge overlapping ones, and
replace each merged span once. Occurrences that only touch stay
separate replacements; separators beside a match stay; tab, newline
and carriage return are still not gaps. This honours the existing
TestRedactString_OverlappingExtraSecrets contract for both forms.
Raised by CodeRabbit on #1189.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(web-search): keep the request URL out of Toolbox transport errors

When http.Client.Do failed, the backend scrubbed err.Error() from the
*url.Error, which quotes the request URL. A polling URL carries the run
id the Toolbox returned, percent-escaped, so a key echoed in that id
appeared as "Lp9%00Xr4" or "Lp9%E2%80%8BXr4". The key scrubber matches
the key whole or split by raw invisible separators, not percent-escapes,
so the key stayed recoverable in the web_search error.

Report only the cause inside the *url.Error. The message prefix, the
redirect refusal text and key scrubbing are unchanged. The regression
fails a polling GET after a queued run whose id carries a NUL- or
ZWSP-split key and checks the output has neither the polling URL nor
the key. Raised by CodeRabbit on #1189.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(redaction): always redact a caller secret as written

Once the text held any invisible separator, redactExtraSecretValues
matched each caller secret only with separators removed from both. That
replaced the contiguous match instead of adding to it: a secret made
only of separators compacted to nothing and was skipped, and separators
at either end of a secret fell outside its match. RedactString("a\x00b")
with secret "\x00" returned the text unchanged, where main replaced it.

In the separator path, also record every occurrence of the secret as
written, then merge spans as before. Text without a separator takes the
unchanged path. Raised by CodeRabbit on #1189.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

---------

Co-authored-by: Stan <[email protected]>
Co-authored-by: Hugo <[email protected]>
Co-authored-by: Claude Opus 5.5 <[email protected]>
2026-11-04 18:24:09 -08:00
..