feat(telemetry): carry inbound trace id into daemon access log with telemetry off

Telemetry off (the default) left daemon logs without any trace id: with no
request span, the log trace prefix never fires, so a caller forwarding W3C
traceparent could not be joined to its daemon log lines.

The middleware now parses the header with a plain regex
(extractInboundTraceId — same shape/all-zero/ff rejections as the W3C
propagator, no OTel machinery) and stores the trace id on the per-response
telemetry context. The access log emits it as the camelCase traceId field
of "request completed", keeping the log-based join alive with no telemetry
config and no trace backend. With telemetry on nothing changes: the request
span already carries the caller's trace id into the log prefix.

docs/design/2026-08-18-daemon-http-inbound-trace-context.md

@@ -32,10 +32,12 @@ the plumbing already exists: `withDaemonSpan` accepts an explicit
    `req.headers` per request (fail-closed to `undefined`, telemetry never
    affects handling) and passes the context only when extraction succeeded —
    requests without a valid header keep the exact current span shape.
-   Extraction is gated on `isTelemetrySdkInitialized()` so telemetry-off
-   deployments pay no hot-path parse cost, and a present-but-invalid header
-   emits a debug daemon log (`qwen-code.daemon.traceparent.invalid`) so a
-   rejected header is diagnosable from daemon logs alone.
+   Span-context extraction is gated on `isTelemetrySdkInitialized()`, so
+   telemetry-off deployments pay no OTel machinery on the hot path — only the
+   single-regex trace-id capture described under Log correlation — and a
+   present-but-invalid header emits a debug daemon log
+   (`qwen-code.daemon.traceparent.invalid`) so a rejected header is
+   diagnosable from daemon logs alone.
 
    Note the whole subtree relocates with the request span, not just
    `daemon.request` itself: session-subprocess spans reached via `_meta`
@@ -43,6 +45,31 @@ the plumbing already exists: `withDaemonSpan` accepts an explicit
    aggregating or alerting by `traceId` sees session-side spans change
    ownership too.
 
+## Log correlation
+
+Daemon log lines already carry a `[trace_id=… span_id=…]` prefix when written
+inside an active recording span (`getActiveTraceContext`, #9084). Telemetry
+on, this PR closes the loop end to end: the request span parents to the
+caller's trace, so every daemon log line for that request — including the
+access log's `request completed` — is prefixed with the caller's trace id.
+(The access log's `finish` listener reads the active span through the same
+AsyncLocalStorage propagation as the logger, so ordering of the two `finish`
+listeners is irrelevant for the prefix.)
+
+Telemetry off — the default — there is no span, so the prefix never fires.
+A separate lightweight path keeps the log-based join alive with no telemetry
+config and no trace backend: the middleware parses the `traceparent` header
+with a plain regex (`extractInboundTraceId`, same shape/all-zero/`ff`
+rejections as the W3C propagator; future-version leniency and `tracestate`
+stay in the propagator path since a log line only needs a plausible trace
+id) and stores the trace id on the per-response telemetry context (the same
+symbol the workspace hash uses). The access log's `finish` callback reads it
+back and emits it as the camelCase `traceId` field of `request completed` —
+distinct from the logger's reserved snake_case `trace_id` prefix keys, so a
+caller cannot spoof the span-derived prefix. The field is omitted, not
+empty, when no valid header is present, and every step stays fail-closed
+(telemetry must not affect request handling).
+
 ## Sampling policy
 
 The caller's `sampled` bit is not adopted verbatim on the HTTP path. Under
@@ -93,6 +120,14 @@ policy at the HTTP edge.
   verbatim), middleware pass-through (present vs omitted key, telemetry-off
   skip, rejected-header debug log), and a type-level guard keeping
   `parentContext` on `DaemonRequestSpanOptions`.
+- Unit (telemetry-off log join): `extractInboundTraceId` (valid / absent /
+  malformed / all-zero ids / version `ff` / array value / future version,
+  plus no-propagator-needed in the fresh-module state), middleware capture
+  (telemetry off with and without a header, telemetry on skipping capture,
+  handler-resolved context initialization not clobbering the stored id), and
+  the access log emitting / omitting the `traceId` field.
 - Dry run: `serve` with `QWEN_TELEMETRY_OUTFILE`, one curl with a fixed
   `traceparent` — exported span must share the header's traceId and parent to
   its spanId; a control request without the header must stay on its own trace.
+  With telemetry disabled, the same curl must still log
+  `request completed` with `traceId` matching the header.

packages/cli/src/serve/server/access-log.test.ts

@@ -9,10 +9,23 @@ import { context, ROOT_CONTEXT } from '@opentelemetry/api';
 import type { Application, RequestHandler } from 'express';
 import { afterEach, describe, expect, it, vi } from 'vitest';
 import type { DaemonLogContext, DaemonLogger } from '../daemon-logger.js';
+
+const telemetryMocks = vi.hoisted(() => ({
+  getDaemonTelemetryInboundTraceId: vi.fn((): string | undefined => undefined),
+}));
+
+// The access log reads the caller trace id through this seam; mocking it
+// keeps the suite off the real telemetry module (and its core import graph).
+vi.mock('./telemetry.js', () => ({
+  getDaemonTelemetryInboundTraceId:
+    telemetryMocks.getDaemonTelemetryInboundTraceId,
+}));
+
 import { installAccessLogMiddleware } from './access-log.js';
 
 afterEach(() => {
   vi.restoreAllMocks();
+  telemetryMocks.getDaemonTelemetryInboundTraceId.mockReset();
 });
 
 function fakeLogger(): DaemonLogger {
@@ -115,6 +128,30 @@ describe('installAccessLogMiddleware', () => {
     );
   });
 
+  it('joins the request log line to the caller trace when telemetry is off', () => {
+    telemetryMocks.getDaemonTelemetryInboundTraceId.mockReturnValueOnce(
+      '3'.repeat(32),
+    );
+    const h = harness();
+    h.begin({ path: '/traced' }).response.emit('finish');
+
+    expect(h.logger.info).toHaveBeenCalledWith(
+      'request completed',
+      expect.objectContaining({ traceId: '3'.repeat(32) }),
+    );
+  });
+
+  it('omits the traceId field when no inbound trace id was captured', () => {
+    const h = harness();
+    h.begin({ path: '/untraced' }).response.emit('finish');
+
+    const context = vi.mocked(h.logger.info).mock.calls[0]?.[1] as
+      | DaemonLogContext
+      | undefined;
+    expect(context).toBeDefined();
+    expect('traceId' in (context ?? {})).toBe(false);
+  });
+
   it('caps UTF-8 fields, uses the first raw client header, and tolerates clock retreat', () => {
     const h = harness();
     const sessionId = '你'.repeat(100);

packages/cli/src/serve/server/access-log.ts

@@ -8,6 +8,7 @@ import { performance } from 'node:perf_hooks';
 import { context, ROOT_CONTEXT } from '@opentelemetry/api';
 import type { Application } from 'express';
 import type { DaemonLogContext, DaemonLogger } from '../daemon-logger.js';
+import { getDaemonTelemetryInboundTraceId } from './telemetry.js';
 
 const SESSION_ID_RE = /\/session\/([^/]+)/;
 const ACCESS_LOG_BURST = 60;
@@ -158,6 +159,11 @@ export function installAccessLogMiddleware(
         const clientId = rawClientId
           ? truncateUtf8(rawClientId, CLIENT_ID_MAX_BYTES)
           : undefined;
+        // With telemetry on, the daemon request span stamps the trace prefix
+        // on this line already; this field covers telemetry-off deployments,
+        // where it is the only traceId link between a daemon log line and the
+        // caller that sent the traceparent header.
+        const inboundTraceId = getDaemonTelemetryInboundTraceId(res);
         const ctx = {
           route: route.value,
           ...(route.originalBytes
@@ -179,6 +185,7 @@ export function installAccessLogMiddleware(
                   : {}),
               }
             : {}),
+          ...(inboundTraceId ? { traceId: inboundTraceId } : {}),
           status,
           durationMs: Math.max(0, Math.round(monotonicNow() - startMs)),
         };

packages/cli/src/serve/server/telemetry.test.ts

@@ -11,6 +11,7 @@ import type { NextFunction, Request, Response } from 'express';
 const coreMocks = vi.hoisted(() => ({
   emitDaemonLog: vi.fn(),
   extractDaemonHttpTraceContext: vi.fn((): unknown => undefined),
+  extractInboundTraceId: vi.fn((): string | undefined => undefined),
   hashDaemonWorkspace: vi.fn((workspace: string) => `hash:${workspace}`),
   isTelemetrySdkInitialized: vi.fn(() => true),
   recordDaemonError: vi.fn(),
@@ -34,6 +35,7 @@ vi.mock('@qwen-code/qwen-code-core', () => ({
 
 import {
   daemonTelemetryMiddleware,
+  getDaemonTelemetryInboundTraceId,
   legacySessionTelemetryRoutes,
   resolveDaemonTelemetryRoute,
   setDaemonTelemetryWorkspace,
@@ -181,8 +183,9 @@ describe('daemonTelemetryMiddleware — recordRequest seam', () => {
     expect(coreMocks.emitDaemonLog).not.toHaveBeenCalled();
   });
 
-  it('skips extraction entirely when the telemetry SDK is not initialized', () => {
+  it('skips span-context extraction when the telemetry SDK is not initialized', () => {
     coreMocks.isTelemetrySdkInitialized.mockReturnValueOnce(false);
+    coreMocks.extractInboundTraceId.mockReturnValueOnce('3'.repeat(32));
     const res = mockRes(200);
 
     daemonTelemetryMiddleware(() => '/ws')(
@@ -192,15 +195,75 @@ describe('daemonTelemetryMiddleware — recordRequest seam', () => {
       res,
       vi.fn() as unknown as NextFunction,
     );
+
+    // Telemetry off: no span parent, but the trace id is still parsed for
+    // the access log's log-based join.
+    expect(coreMocks.extractDaemonHttpTraceContext).not.toHaveBeenCalled();
+    expect(coreMocks.extractInboundTraceId).toHaveBeenCalledWith({
+      traceparent: `00-${'3'.repeat(32)}-${'4'.repeat(16)}-01`,
+    });
+    expect(getDaemonTelemetryInboundTraceId(res)).toBe('3'.repeat(32));
+
     res.emit('finish');
 
-    expect(coreMocks.extractDaemonHttpTraceContext).not.toHaveBeenCalled();
     const options = coreMocks.withDaemonRequestSpan.mock
       .calls[0]?.[0] as Record<string, unknown>;
     expect('parentContext' in options).toBe(false);
     expect(coreMocks.emitDaemonLog).not.toHaveBeenCalled();
   });
 
+  it('leaves the response without an inbound trace id when telemetry is off and the header is absent', () => {
+    coreMocks.isTelemetrySdkInitialized.mockReturnValueOnce(false);
+    const res = mockRes(200);
+
+    daemonTelemetryMiddleware(() => '/ws')(
+      mockReq('GET', '/daemon/status', {}),
+      res,
+      vi.fn() as unknown as NextFunction,
+    );
+
+    expect(getDaemonTelemetryInboundTraceId(res)).toBeUndefined();
+
+    res.emit('finish');
+  });
+
+  it('does not capture the inbound trace id when telemetry is on (the request span already carries it)', () => {
+    const res = mockRes(200);
+
+    daemonTelemetryMiddleware(() => '/ws')(
+      mockReq('GET', '/daemon/status', {
+        traceparent: `00-${'3'.repeat(32)}-${'4'.repeat(16)}-01`,
+      }),
+      res,
+      vi.fn() as unknown as NextFunction,
+    );
+
+    expect(coreMocks.extractInboundTraceId).not.toHaveBeenCalled();
+    expect(getDaemonTelemetryInboundTraceId(res)).toBeUndefined();
+
+    res.emit('finish');
+  });
+
+  it('keeps the captured trace id when the handler-resolved branch initializes the response context', () => {
+    coreMocks.isTelemetrySdkInitialized.mockReturnValueOnce(false);
+    coreMocks.extractInboundTraceId.mockReturnValueOnce('3'.repeat(32));
+    const res = mockRes(200);
+
+    daemonTelemetryMiddleware(() => '/ws')(
+      mockReq('GET', '/session/abc/artifacts', {
+        traceparent: `00-${'3'.repeat(32)}-${'4'.repeat(16)}-01`,
+      }),
+      res,
+      vi.fn() as unknown as NextFunction,
+    );
+
+    // The handler_resolved branch must not clobber the context the trace id
+    // write already created.
+    expect(getDaemonTelemetryInboundTraceId(res)).toBe('3'.repeat(32));
+
+    res.emit('finish');
+  });
+
   it('logs at debug severity when a present traceparent header is rejected', () => {
     const res = mockRes(200);
 

packages/cli/src/serve/server/telemetry.ts

@@ -7,6 +7,7 @@
 import {
   emitDaemonLog,
   extractDaemonHttpTraceContext,
+  extractInboundTraceId,
   hashDaemonWorkspace,
   isTelemetrySdkInitialized,
   recordDaemonError,
@@ -385,6 +386,13 @@ interface ResolvedDaemonTelemetryRoute {
 
 interface DaemonTelemetryResponseContext {
   workspaceCwd?: string;
+  /**
+   * Caller trace id from a valid inbound `traceparent` header, captured with
+   * telemetry disabled (with telemetry on the request span already carries
+   * it into the log line's trace prefix). Consumed by the access log for the
+   * no-backend log-based join.
+   */
+  inboundTraceId?: string;
 }
 
 const daemonTelemetryResponseContext = Symbol('daemonTelemetryResponseContext');
@@ -475,6 +483,23 @@ export function setDaemonTelemetryWorkspace(
   }
 }
 
+/**
+ * The caller trace id captured from a valid inbound `traceparent` header when
+ * telemetry is disabled. The access log reads it so a request's log line
+ * still joins with the caller's logs (or trace backend) with no daemon-side
+ * telemetry at all.
+ */
+export function getDaemonTelemetryInboundTraceId(
+  res: Response,
+): string | undefined {
+  try {
+    return (res as TelemetryResponse)[daemonTelemetryResponseContext]
+      ?.inboundTraceId;
+  } catch {
+    return undefined;
+  }
+}
+
 // Route handlers are split across `routes/*.ts`; any added or renamed route
 // that needs daemon telemetry must keep these patterns in sync.
 export function resolveDaemonTelemetryRoute(
@@ -745,9 +770,12 @@ export function daemonTelemetryMiddleware(
         : undefined;
     const deferredRuntime = getDeferredRuntimeRequestTiming(req);
     const startMs = deferredRuntime?.startedAt.getTime() ?? Date.now();
-    // Skip extraction entirely when telemetry is off — withDaemonSpan would
-    // short-circuit anyway, so the header parse would be wasted hot-path work.
+    // With telemetry on, extract the full W3C context (span parent + forced
+    // sampling). With telemetry off there is no span to parent, so only the
+    // trace id is parsed out for the access log — a single regex, no OTel
+    // machinery on the hot path.
     let parentContext: ReturnType<typeof extractDaemonHttpTraceContext>;
+    let inboundTraceId: string | undefined;
     if (isTelemetrySdkInitialized()) {
       try {
         parentContext = extractDaemonHttpTraceContext(req.headers);
@@ -779,11 +807,33 @@ export function daemonTelemetryMiddleware(
           },
         );
       }
+    } else {
+      // With telemetry off there is no request span, so the log line's trace
+      // prefix never fires. Still surface the caller's trace id to the access
+      // log — a plain header parse keeps the log-based join alive with no
+      // telemetry config and no trace backend.
+      try {
+        inboundTraceId = extractInboundTraceId(req.headers);
+      } catch {
+        // Telemetry must not affect request handling.
+      }
     }
     const telemetryRes = res as TelemetryResponse;
+    if (inboundTraceId !== undefined) {
+      try {
+        const context = telemetryRes[daemonTelemetryResponseContext];
+        if (context) {
+          context.inboundTraceId = inboundTraceId;
+        } else {
+          telemetryRes[daemonTelemetryResponseContext] = { inboundTraceId };
+        }
+      } catch {
+        // Telemetry must not affect request handling.
+      }
+    }
     if (route.attribution === 'handler_resolved') {
       try {
-        telemetryRes[daemonTelemetryResponseContext] = {};
+        telemetryRes[daemonTelemetryResponseContext] ??= {};
       } catch {
         // Telemetry must not affect request handling.
       }

packages/core/src/telemetry/daemon-tracing.test.ts

@@ -30,6 +30,7 @@ import {
   createDaemonBridgeTelemetry,
   extractDaemonHttpTraceContext,
   extractDaemonTraceContext,
+  extractInboundTraceId,
   hashDaemonWorkspace,
   injectDaemonTraceContext,
   runWithDaemonTelemetryContext,
@@ -227,6 +228,49 @@ describe('daemon-tracing', () => {
     ).toBeUndefined();
   });
 
+  it('extracts only the trace id for the telemetry-off log join', () => {
+    expect(
+      extractInboundTraceId({
+        traceparent: `00-${'3'.repeat(32)}-${'4'.repeat(16)}-01`,
+      }),
+    ).toBe('3'.repeat(32));
+    // future versions pass the plain shape check; version-specific leniency
+    // (extension fields, tracestate) stays in the propagator path
+    expect(
+      extractInboundTraceId({
+        traceparent: `01-${'3'.repeat(32)}-${'4'.repeat(16)}-01`,
+      }),
+    ).toBe('3'.repeat(32));
+  });
+
+  it('rejects invalid headers on the telemetry-off trace id path', () => {
+    expect(extractInboundTraceId(undefined)).toBeUndefined();
+    expect(extractInboundTraceId({})).toBeUndefined();
+    expect(
+      extractInboundTraceId({ traceparent: 'not-a-traceparent' }),
+    ).toBeUndefined();
+    expect(
+      extractInboundTraceId({
+        traceparent: `00-${'0'.repeat(32)}-${'4'.repeat(16)}-01`,
+      }),
+    ).toBeUndefined();
+    expect(
+      extractInboundTraceId({
+        traceparent: `00-${'3'.repeat(32)}-${'0'.repeat(16)}-01`,
+      }),
+    ).toBeUndefined();
+    expect(
+      extractInboundTraceId({
+        traceparent: `ff-${'3'.repeat(32)}-${'4'.repeat(16)}-01`,
+      }),
+    ).toBeUndefined();
+    expect(
+      extractInboundTraceId({
+        traceparent: [`00-${'3'.repeat(32)}-${'4'.repeat(16)}-01`],
+      }),
+    ).toBeUndefined();
+  });
+
   it('yields no parent context until the SDK chunk injects the fallback propagator', async () => {
     // Fresh module registry: daemon-tracing without the sdk-impl injection.
     // The global propagator stays a no-op in tests (nothing registers one),
@@ -240,6 +284,13 @@ describe('daemon-tracing', () => {
         traceparent: `00-${'3'.repeat(32)}-${'4'.repeat(16)}-01`,
       }),
     ).toBeUndefined();
+    // The telemetry-off trace id path needs no propagator — that is the
+    // whole point of the plain regex parse.
+    expect(
+      fresh.extractInboundTraceId({
+        traceparent: `00-${'3'.repeat(32)}-${'4'.repeat(16)}-01`,
+      }),
+    ).toBe('3'.repeat(32));
     expect(
       fresh.extractDaemonTraceContext({
         _meta: {

packages/core/src/telemetry/daemon-tracing.ts

@@ -395,6 +395,38 @@ export function extractDaemonHttpTraceContext(
   );
 }
 
+const TRACEPARENT_RE =
+  /^([0-9a-f]{2})-([0-9a-f]{32})-([0-9a-f]{16})-([0-9a-f]{2})$/;
+const ALL_ZERO_TRACE_ID = '0'.repeat(32);
+const ALL_ZERO_SPAN_ID = '0'.repeat(16);
+
+/**
+ * Extract the caller's trace id from an inbound `traceparent` header without
+ * any OpenTelemetry machinery. Unlike {@link extractDaemonHttpTraceContext}
+ * (which builds a span parent and needs the W3C propagator — only installed
+ * once the telemetry SDK starts), this is a plain format check so the daemon
+ * log can carry the caller's trace id even with telemetry disabled: the
+ * log-based join then works with no trace backend at all. Version-specific
+ * leniency beyond the shape check (future versions, tracestate) stays in the
+ * propagator path; a log line only needs a plausible, non-zero trace id.
+ */
+export function extractInboundTraceId(
+  headers: Record<string, unknown> | undefined,
+): string | undefined {
+  const traceparent = headers?.['traceparent'];
+  if (typeof traceparent !== 'string' || traceparent.length === 0) {
+    return undefined;
+  }
+  const match = TRACEPARENT_RE.exec(traceparent);
+  if (!match) return undefined;
+  const [, version, traceId, spanId] = match;
+  if (version === 'ff') return undefined;
+  if (traceId === ALL_ZERO_TRACE_ID || spanId === ALL_ZERO_SPAN_ID) {
+    return undefined;
+  }
+  return traceId;
+}
+
 export interface DaemonBridgeTelemetryMetrics {
   sessionLifecycle(action: 'spawn' | 'close' | 'die'): void;
   channelLifecycle(action: 'spawn' | 'exit', expected?: boolean): void;