`logComparativeEvaluationFeedback` looked each run up by the keys of the
score map
returned by the user's comparative evaluator, but posted the feedback
whether or not
the lookup hit. On a miss, `POST /api/v1/feedback` went out with a
`run_id` but no
`session_id` — while still carrying this comparison's
`comparative_experiment_id` and
`feedback_group_id`. `ComparisonEvaluationResult.scores` is authored
entirely by the
evaluator and was never checked against the runs it was handed, so a
nested run id, or
one an LLM judge invented, was enough to trigger it.
Such an entry is now skipped with a `WARNING`, and `.sessionId(...)` is
unconditional in
the builder. No fallback session is used: with two experiments in play
there is no way to
tell which one an unrecognised run belongs to. Evaluator-supplied text
is sanitized before
logging so a judge's output can't forge log records.
The other feedback paths were checked and are unaffected — `evaluate()`
supplies
`session_id` via `TraceConfig`, and `evaluateExisting()` via the
required
`RunSchema.session_id`.
New `EvaluateSupportTest` calls the internal helper directly with a
mocked
`FeedbackService`. This path had no coverage before.
## Test Plan
- [x] `./gradlew :langsmith-java-core:test` green (422 test classes);
`lintKotlin` clean
## Summary
Port of langchain-ai/langsmith-sdk#3284 to the Java SDK. There was no
stripping here at all, so this implements it.
The generated request paths spell their own prefix out —
`RunServiceImpl` emits `.addPathSegments("api", "v1", "runs", "query")`,
`.addPathSegments("v2", "runs", "query")` — so for those the base URL
has to point at the deployment root. A user who sets
`LANGSMITH_ENDPOINT` to the suffixed URL the Python and JS SDKs accept
(`https://host/api/v1`) currently gets requests to
`https://host/api/v1/api/v1/runs/query`.
Handwritten requests are the opposite case: they are root-relative, so
the prefix they sit behind comes from the configured endpoint — `/api`
on self-hosted, nothing on SaaS. That is deployment specific and cannot
be reconstructed from a normalized root, so those requests keep the URL
exactly as configured.
## Which requests are handwritten
Determined with `make show-custom-code TARGET=java` in
`langchainplus/smith-sdks`, which diffs pure generated output against
the integrated SDK. Only two endpoint families build a URL from the
client base URL outside the generated paths:
| Handwritten request | Files |
|---|---|
| `GET {base}/info` | `InfoServiceImpl`, `InfoServiceAsyncImpl` |
| `POST {base}/runs/multipart` | `RunServiceImpl`, `RunServiceAsyncImpl`
|
`AutoBatchQueue`, `RunMultipartBatch`, `RunCompression` and
`evaluation/*` are custom but route through those or through generated
methods. `OtelTraceExporter` uses its own OTLP endpoint. The one other
handwritten request, the OAuth token refresh in `ProfileConfig`,
deliberately targets the root and already stripped `/api/v1`.
## Changes
- **`BaseUrls.kt`** (new, custom) — `normalizeBaseUrl` trims a trailing
`/api/v1` or `/api`. Only whole trailing path segments of the *parsed*
URL are considered, so a host such as `api.example.com` is untouched and
the scheme/host are never rewritten. The result keeps a trailing slash,
matching `ClientOptions.PRODUCTION_URL`. A URL that fails to parse, or
that carries a query or fragment, is returned unchanged. A bare `/v1` is
not stripped, matching the Python and JS SDKs.
- **`ClientOptions.kt`** — `baseUrl()` now returns the normalized root,
and a new `configuredBaseUrl()` returns the value as configured.
Normalizing in the **getter** rather than in `Builder.baseUrl` matters:
it keeps the configured value recoverable, including across a
`toBuilder()` round trip.
- **The four handwritten call sites** — switched to
`clientOptions.configuredBaseUrl()`.
- **`ProfileConfig.kt`** — `normalizeConfigUrl` delegates to the shared
helper, so the OAuth token endpoint strips `/api` too rather than only
`/api/v1`.
`PrepareRequest.kt` carries no custom code, so it is left untouched.
## Tests
New custom `BaseUrlsTest.kt` (the generated `ClientOptionsTest.kt` is
left alone), 21 tests: the stripped and preserved cases for
`normalizeBaseUrl`; handwritten `/info` and `/runs/multipart` keeping
the configured URL for endpoints of `""`, `/api` and `/api/v1`;
generated paths resolving against the root in all three; both accessors
on `ClientOptions`; the `toBuilder()` round trip; and the null fallback
to `PRODUCTION_URL`.
- `./gradlew :langsmith-java-core:test` → 1986 tests, 0 failures
- `./gradlew format` / `lint` clean
Note: `AutoBatchQueueTest.batch size limit starts a new batch for
subsequent runs` is timing sensitive and fails intermittently when run
in isolation, on a clean `main` checkout as well as here. Unrelated to
this change.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Kiewan Villatel