feat(mcp): compact input schema profile for size-capped providers (kimi-cli/Moonshot) (#1514)

* feat(mcp): add compact input schema profile for size-capped providers

Moonshot's function-calling API rejects a request while any single tool
parameters schema exceeds its literal serialized size cap, so clients
such as kimi-cli fail every request while the default catalog is
attached (#1484).

Add an opt-in REA_MCP_INPUT_SCHEMA_PROFILE=compact server presentation.
Each advertised input schema is rendered within a 13,312-byte budget,
measured to sit below the observed provider cap: annotation prose
(nested descriptions, examples, defaults, titles) is dropped, repeated
annotation-free subschemas are shared through schema-local $defs/$ref
references the provider validates without expansion, and the three
structurally oversized schemas are advertised in a documented reduced
form. The canonical catalog, tool set, output schemas, and server-side
canonical validation are unchanged; the default full profile is
byte-identical to today's advertisement.

Boundary tests pin every compact schema to the budget, keep them valid
JSON Schema that still accepts every canonical example, hold reduced
presentations at three or fewer, and verify canonical rejection still
guards a permissive advertisement. Verified against the real Moonshot
API through kimi-cli: prompts succeed with the compact catalog attached
where the default catalog fails every request with 400.

* fix(mcp): preserve compact schemas through Kimi conversion

---------

Co-authored-by: morluto <[email protected]>
This commit is contained in:
Haythem Sellami
2026-10-10 16:58:43 +08:00
committed by GitHub
co-authored by morluto
parent 2b6f14f528
commit b6ad1f2c3d
15 changed files with 1190 additions and 18 deletions
+43
View File
@@ -36,6 +36,49 @@ after SDK conversion and check advertised validation and actual calls.
Individual model APIs can impose additional nesting limits; complete producer
captures can exceed ten structural levels.
### Compact input schema profile
Some function-calling providers reject a request while any single tool's
`parameters` schema exceeds a literal serialized size cap; Moonshot clients
such as kimi-cli fail every request while the default catalog is attached
([#1484](https://github.com/morluto/rea/issues/1484)). Setting the server's
`REA_MCP_INPUT_SCHEMA_PROFILE` environment variable to `compact` renders every
advertised input schema within a 13,312-byte budget, measured to sit below
Moonshot's observed cap (community measurements: a 13.7 KB schema is accepted
while a ~15 KB one is rejected). The setting is read at startup; restart the
server to apply changes, as with `REA_MCP_MAX_RESPONSE_BYTES`. The default
value `full` keeps the complete advertisement unchanged.
The compact profile renders the same tool set, output schemas, annotations,
and canonical server-side validation. It changes only the advertised input
JSON Schema:
- Annotation prose leaves the advertised form: nested property descriptions,
literal examples, defaults, and titles are dropped. The root keeps its
description because it states the accepted input groups.
- Schema-local references are inlined within the budget. Kimi Code expands
references before sending provider requests, so sharing definitions cannot
establish the provider-facing size. Expansion stops at the budget before
allocating an oversized presentation. Unconstrained JSON fields advertise
all JSON value types explicitly so Kimi cannot infer string for Evidence
objects or arrays.
- The few schemas whose validation structure alone exceeds the budget
(`compare_web_captures`, `build_reconstruction_obligation_ledger`,
`evaluate_reconstruction_coverage`, `project_managed_application_graph`,
`compare_application_versions`, `capture_browser_scenario`) are advertised
in a reduced form: property names, reference-resolved types (including union
alternatives), first-level guidance, and the root's required-field constraints,
with nested validation detail elided. The
canonical schema still rejects malformed calls server-side, so callers
following the reduced advertisement receive the canonical typed error
instead of silent acceptance.
Boundary tests pin every advertised compact schema to the budget, keep them
valid JSON Schema that still accepts every canonical example, and hold the
set of six reduced presentations as contracts evolve. Client verification
also checks provider-facing sizes and canonical examples after conversion by
the actual Kimi Code CLI; local captures do not establish Moonshot acceptance.
Self-contained output schemas advertise a content-bound `$id`, including their
declared dialect. SDK validators can reuse compiled schemas across complete
catalog refreshes and equivalent tool outputs; changing the schema changes its
+2
View File
@@ -5,11 +5,13 @@ import { ConfigurationError } from "../domain/configurationErrors.js";
import { err, ok, type Result } from "../domain/result.js";
import { analysisProviderSelectorSchema } from "../contracts/providerSelection.js";
import { ghidraStartupTimeoutSchema } from "./ghidraStartupTimeout.js";
import { mcpInputSchemaProfileSchema } from "./mcpInputSchemaProfile.js";
import { mcpResponseBudgetSchema } from "./mcpResponseBudget.js";
const environmentSchema = z.object({
REA_ANALYSIS_PROVIDER: analysisProviderSelectorSchema.default("auto"),
REA_MCP_MAX_RESPONSE_BYTES: mcpResponseBudgetSchema.optional(),
REA_MCP_INPUT_SCHEMA_PROFILE: mcpInputSchemaProfileSchema.optional(),
REA_IDA_MCP_CONFIG: z
.string()
.min(1)
+38
View File
@@ -0,0 +1,38 @@
import { z } from "zod";
import { ConfigurationError } from "../domain/configurationErrors.js";
import { err, ok, type Result } from "../domain/result.js";
// Moonshot's function-calling API rejects requests while any single tool
// parameters schema exceeds its literal serialized size cap. Community
// measurements bound the cap between ~13.7 KB (accepted) and ~15 KB
// (rejected); 13 KiB leaves margin under the lower bound. See
// https://github.com/morluto/rea/issues/1484 for the measured evidence.
export const COMPACT_INPUT_SCHEMA_BUDGET_BYTES = 13 * 1024;
export const mcpInputSchemaProfileSchema = z.enum(["full", "compact"]);
const profileConstraint =
'REA_MCP_INPUT_SCHEMA_PROFILE must be "full" (default) or "compact". The compact profile advertises input schemas without nested annotation prose for providers that cap each tool schema\'s serialized size.';
export type McpInputSchemaProfile = z.infer<typeof mcpInputSchemaProfileSchema>;
/** Parse the advertised input schema presentation selection. */
export const parseMcpInputSchemaProfile = (
value: string | undefined,
): Result<McpInputSchemaProfile | undefined, ConfigurationError> => {
if (value === undefined) return ok(undefined);
const parsed = mcpInputSchemaProfileSchema.safeParse(value);
return parsed.success
? ok(parsed.data)
: err(
new ConfigurationError(profileConstraint, {
settings: [
{
setting: "REA_MCP_INPUT_SCHEMA_PROFILE",
constraint: profileConstraint,
},
],
}),
);
};
+3
View File
@@ -50,6 +50,9 @@ export const parseConfig = (
...(env.REA_MCP_MAX_RESPONSE_BYTES === undefined
? {}
: { mcpMaxResponseBytes: env.REA_MCP_MAX_RESPONSE_BYTES }),
...(env.REA_MCP_INPUT_SCHEMA_PROFILE === undefined
? {}
: { mcpInputSchemaProfile: env.REA_MCP_INPUT_SCHEMA_PROFILE }),
analysisProvider: env.REA_ANALYSIS_PROVIDER,
...(env.REA_IDA_MCP_CONFIG === undefined
? {}
+3
View File
@@ -2,8 +2,11 @@ import type { LogLevel } from "../logger.js";
import type { ReferenceSourcePolicy } from "../domain/referenceSourcePolicy.js";
import type { AnalysisProviderSelector } from "../contracts/providerSelection.js";
import type { McpInputSchemaProfile } from "./mcpInputSchemaProfile.js";
export interface AppConfig {
readonly mcpMaxResponseBytes?: number;
readonly mcpInputSchemaProfile?: McpInputSchemaProfile;
readonly analysisProvider: AnalysisProviderSelector;
readonly idaMcpConfigPath?: string;
readonly ghidraInstallDir: string | undefined;
+314
View File
@@ -0,0 +1,314 @@
import { Ajv2020 } from "ajv/dist/2020.js";
import { describe, expect, it } from "vitest";
import { compactAdvertisedInputSchema } from "./compactInputSchema.js";
const serializedBytes = (value: unknown): number =>
Buffer.byteLength(JSON.stringify(value), "utf8");
const repeatedObservation = {
type: "object",
properties: {
captured_url: { type: "string", minLength: 1 },
captured_at: { type: "string", minLength: 1 },
status_code: { type: "integer", minimum: 100, maximum: 599 },
headers: {
type: "object",
additionalProperties: { type: "string" },
minProperties: 1,
},
body_digest: { type: "string", minLength: 64, maxLength: 64 },
},
required: ["captured_url", "captured_at", "status_code"],
additionalProperties: false,
} as const;
const annotatedSchema = {
type: "object",
description: "Root group guidance stays.",
properties: {
query: {
type: "string",
description: "Field guidance leaves.",
examples: ["seed"],
default: "seed",
title: "Query",
minLength: 1,
},
limits: {
type: "object",
description: "Nested guidance leaves.",
properties: {
max_results: {
type: "integer",
description: "Leaves.",
minimum: 1,
maximum: 500,
default: 50,
},
},
required: ["max_results"],
additionalProperties: false,
},
},
required: ["query"],
dependentRequired: { limits: ["query"] },
additionalProperties: false,
} as const;
const oversizedPairSchema = (): Record<string, unknown> => {
const deepStructure = {
type: "object",
description: "Nested guidance that leaves.",
properties: Object.fromEntries(
Array.from({ length: 40 }, (_, index) => [
`field_${index}`,
{ type: "string", minLength: 1, description: `Field ${index}` },
]),
),
required: Array.from({ length: 40 }, (_, index) => `field_${index}`),
additionalProperties: false,
};
return {
type: "object",
description: "Provide a complete capture pair.",
properties: {
before: structuredClone(deepStructure),
after: structuredClone(deepStructure),
},
required: ["before", "after"],
additionalProperties: false,
};
};
describe("compact advertised input schema", () => {
it("drops nested annotation prose while keeping validation keywords", () => {
const { schema, presentation } = compactAdvertisedInputSchema(
structuredClone(annotatedSchema),
13 * 1024,
);
expect(presentation).toBe("inline");
expect(schema.description).toBe("Root group guidance stays.");
const query = (schema.properties as Record<string, unknown>)
.query as Record<string, unknown>;
expect(query).toEqual({ type: "string", minLength: 1 });
const limits = (schema.properties as Record<string, unknown>)
.limits as Record<string, unknown>;
expect(limits.description).toBeUndefined();
expect(limits.required).toEqual(["max_results"]);
expect(schema.dependentRequired).toEqual({ limits: ["query"] });
expect(schema.required).toEqual(["query"]);
});
it("keeps annotation-shaped keys inside literal const and enum data", () => {
const literalMode = {
description: "saved data",
default: { keep: "me" },
};
const { schema } = compactAdvertisedInputSchema(
{
type: "object",
properties: {
mode: { enum: [literalMode] },
pinned: { const: { description: "literal" } },
},
},
13 * 1024,
);
const properties = schema.properties as Record<string, unknown>;
expect((properties.mode as Record<string, unknown>).enum).toEqual([
literalMode,
]);
expect((properties.pinned as Record<string, unknown>).const).toEqual({
description: "literal",
});
});
it("inlines local references without changing validation or literal data", () => {
const original = {
type: "object",
$defs: { "observation/~": structuredClone(repeatedObservation) },
properties: {
before: { $ref: "#/$defs/observation~1~0" },
after: { $ref: "#/$defs/observation~1~0" },
literal: { const: { $ref: "saved data", $defs: { keep: true } } },
},
required: ["before", "after"],
additionalProperties: false,
};
const { schema, presentation } = compactAdvertisedInputSchema(
original,
13312,
);
expect(presentation).toBe("inline");
expect(schema.$defs).toBeUndefined();
const ajv = new Ajv2020({ strict: false });
const canonical = ajv.compile(original);
const compact = ajv.compile(schema);
const observation = {
captured_url: "https://example.com",
captured_at: "now",
status_code: 200,
};
for (const input of [
{ before: observation, after: observation },
{ before: observation, after: { ...observation, status_code: 99 } },
{
before: observation,
after: observation,
literal: original.properties.literal.const,
},
])
expect(compact(input)).toBe(canonical(input));
});
it("keeps unconstrained JSON values explicit for client type inference", () => {
const { schema } = compactAdvertisedInputSchema(
{
type: "object",
properties: { normalized_result: {}, literal: { const: {} } },
},
13312,
);
const properties = schema.properties as Record<string, unknown>;
expect(properties.normalized_result).toEqual({
type: ["null", "boolean", "object", "array", "number", "string"],
});
expect(properties.literal).toEqual({ const: {} });
const validate = new Ajv2020({ strict: false }).compile(schema);
for (const normalized_result of [null, true, {}, [], 42, "result"])
expect(validate({ normalized_result })).toBe(true);
});
it("preserves independent validation constraints on reference siblings", () => {
const original = {
type: "object",
$defs: { text: { type: "string", minLength: 3 } },
properties: {
query: { $ref: "#/$defs/text", minLength: 1, maxLength: 5 },
},
};
const { schema } = compactAdvertisedInputSchema(original, 13312);
const validate = new Ajv2020({ strict: false }).compile(schema);
expect(validate({ query: "a" })).toBe(false);
expect(validate({ query: "abc" })).toBe(true);
expect(validate({ query: "abcdef" })).toBe(false);
});
});
describe("compact input schema budget and reduction", () => {
it("budgets repeated-reference expansion before materializing it", () => {
const definitions: Record<string, unknown> = {
leaf: structuredClone(repeatedObservation),
};
for (let index = 0; index < 30; index++) {
const $ref = `#/$defs/${index === 0 ? "leaf" : `level${index - 1}`}`;
definitions[`level${index}`] = {
type: "object",
properties: { left: { $ref }, right: { $ref } },
};
}
const { schema, presentation } = compactAdvertisedInputSchema(
{
type: "object",
$defs: definitions,
properties: { tree: { $ref: "#/$defs/level29" } },
},
1024,
);
expect(presentation).toBe("reduced");
expect(serializedBytes(schema)).toBeLessThanOrEqual(1024);
expect(schema.properties).toEqual({ tree: { type: "object" } });
});
it("retains referenced object, array and nullable types in reduced inputs", () => {
const original = {
type: "object",
$defs: {
pair: oversizedPairSchema(),
observations: {
type: "array",
items: structuredClone(repeatedObservation),
},
nullable: { anyOf: [{ type: "string" }, { type: "null" }] },
},
properties: {
capture: { $ref: "#/$defs/pair", description: "Complete capture." },
observations: { $ref: "#/$defs/observations" },
label: { $ref: "#/$defs/nullable" },
},
};
const { schema, presentation } = compactAdvertisedInputSchema(
original,
1024,
);
expect(presentation).toBe("reduced");
expect(schema.properties).toEqual({
capture: { type: "object", description: "Complete capture." },
observations: { type: "array" },
label: { type: ["string", "null"] },
});
const validate = new Ajv2020({ strict: false }).compile(schema);
expect(validate({ capture: {}, observations: [], label: null })).toBe(true);
expect(validate({ capture: {}, observations: [], label: "capture" })).toBe(
true,
);
expect(validate({ capture: "{}", observations: [], label: null })).toBe(
false,
);
});
it("reduces structurally oversized schemas to the shallow property presentation", () => {
const compacted = compactAdvertisedInputSchema(oversizedPairSchema(), 1024);
expect(compacted.presentation).toBe("reduced");
expect(serializedBytes(compacted.schema)).toBeLessThanOrEqual(1024);
expect(compacted.schema.type).toBe("object");
expect(compacted.schema.description).toContain(
"Provide a complete capture pair.",
);
expect(compacted.schema.description).toContain(
"compact input schema profile",
);
const properties = compacted.schema.properties as Record<string, unknown>;
expect(properties.before).toEqual({
type: "object",
description: "Nested guidance that leaves.",
});
expect(properties.after).toEqual(properties.before);
expect(compacted.schema.required).toEqual(["before", "after"]);
});
it("produces deterministic output for equal input", () => {
const schema = {
type: "object",
properties: {
before: structuredClone(repeatedObservation),
after: structuredClone(repeatedObservation),
},
};
expect(compactAdvertisedInputSchema(schema, 13 * 1024)).toEqual(
compactAdvertisedInputSchema(structuredClone(schema), 13 * 1024),
);
});
it("keeps inline presentations under the measured provider budget", () => {
const { schema, presentation } = compactAdvertisedInputSchema(
{
type: "object",
description: "Group guidance.",
properties: {
before: structuredClone(repeatedObservation),
after: structuredClone(repeatedObservation),
normalization: {
type: "object",
description: "Rules.",
properties: { rules: { type: "array", items: { type: "string" } } },
},
},
},
13312,
);
expect(presentation).toBe("inline");
expect(serializedBytes(schema)).toBeLessThanOrEqual(13312);
});
});
+311
View File
@@ -0,0 +1,311 @@
import {
schemaArrays,
schemaChildren,
schemaMaps,
} from "./inputSchemaPresentation.js";
// Kimi expands schema-local references before sending tools to Moonshot.
// Compact advertisements must fit after expansion, independently of client
// reference handling. Canonical server-side validation remains unchanged.
/** Annotation keys that carry guidance but no validation semantics. */
const annotationKeys = new Set(["description", "examples", "default", "title"]);
/** Keys whose values are literal data, never nested schemas to rewrite. */
const dataKeys = new Set(["const", "enum", "examples", "default"]);
const definitionKeys = new Set(["$defs", "definitions"]);
// An empty schema accepts every JSON value. Kimi otherwise infers string.
const jsonValueTypes = [
"null",
"boolean",
"object",
"array",
"number",
"string",
];
export type AdvertisedInputSchemaPresentation = "inline" | "reduced";
export interface CompactedInputSchema {
readonly schema: Record<string, unknown>;
readonly presentation: AdvertisedInputSchemaPresentation;
}
const isSchemaObject = (value: unknown): value is Record<string, unknown> =>
typeof value === "object" && value !== null && !Array.isArray(value);
const serializedBytes = (value: unknown): number =>
Buffer.byteLength(JSON.stringify(value), "utf8");
/** Rewrite every schema-valued position; literal data stays opaque. */
const rebuildSchemaPositions = (
node: unknown,
rebuild: (child: unknown) => unknown,
): unknown => {
if (Array.isArray(node)) return node.map((child) => rebuild(child));
if (!isSchemaObject(node)) return node;
const out: Record<string, unknown> = {};
for (const [key, value] of Object.entries(node)) {
if (dataKeys.has(key)) {
out[key] = value;
continue;
}
if (schemaMaps.has(key) && isSchemaObject(value)) {
out[key] = Object.fromEntries(
Object.entries(value).map(([name, child]) => [name, rebuild(child)]),
);
continue;
}
if (schemaArrays.has(key) && Array.isArray(value)) {
out[key] = value.map((child) => rebuild(child));
continue;
}
if (schemaChildren.has(key)) {
out[key] = Array.isArray(value)
? value.map((child) => rebuild(child))
: rebuild(value);
continue;
}
out[key] = value;
}
return out;
};
/**
* Drop annotation prose from every subschema while preserving all validation
* keywords. The root keeps its own description because the root-union
* projection encodes accepted input groups there; the root's echoed literal
* examples leave with the rest of the annotation prose.
*/
const stripAnnotations = (
schema: Record<string, unknown>,
): Record<string, unknown> => {
const rebuilt = rebuildSchemaPositions(
schema,
stripChildAnnotations,
) as Record<string, unknown>;
const out: Record<string, unknown> = {};
for (const [key, value] of Object.entries(rebuilt)) {
if (annotationKeys.has(key) && key !== "description") continue;
out[key] = value;
}
return out;
};
const stripChildAnnotations = (child: unknown): unknown => {
const rebuilt = rebuildSchemaPositions(child, stripChildAnnotations);
if (!isSchemaObject(rebuilt)) return rebuilt;
const kept: Record<string, unknown> = {};
for (const [key, value] of Object.entries(rebuilt))
if (!annotationKeys.has(key)) kept[key] = value;
return kept;
};
const resolveLocalReference = (
root: Record<string, unknown>,
reference: string,
): unknown => {
if (reference === "#") return root;
if (!reference.startsWith("#/")) return undefined;
let value: unknown = root;
for (const token of reference.slice(2).split("/")) {
const key = token.replaceAll("~1", "/").replaceAll("~0", "~");
if (!isSchemaObject(value) || !Object.hasOwn(value, key)) return undefined;
value = value[key];
}
return value;
};
/** Stop serialization at the budget before allocating an expanded schema. */
const inlineWithinBudget = (
root: Record<string, unknown>,
budgetBytes: number,
): Record<string, unknown> | undefined => {
let resolved = true;
function* serialize(
value: unknown,
position: "schema" | "map" | "data",
active: ReadonlySet<string>,
): Generator<string> {
if (Array.isArray(value)) {
yield "[";
for (const [index, child] of value.entries()) {
if (index > 0) yield ",";
yield* serialize(child, position, active);
}
yield "]";
} else if (isSchemaObject(value)) {
if (position === "schema" && Object.keys(value).length === 0) {
yield JSON.stringify({ type: jsonValueTypes });
return;
}
if (position === "schema" && typeof value.$ref === "string") {
const target = resolveLocalReference(root, value.$ref);
if (target === undefined || active.has(value.$ref)) {
resolved = false;
return;
}
const { $ref, ...siblings } = value;
// Reference siblings are independent constraints in draft 2020-12.
// Merging their keys into the target would overwrite validation rules.
yield* serialize(
Object.keys(siblings).length === 0
? target
: { allOf: [target, siblings] },
"schema",
new Set([...active, $ref]),
);
return;
}
yield "{";
let first = true;
for (const [key, child] of Object.entries(value)) {
if (position === "schema" && definitionKeys.has(key)) continue;
if (!first) yield ",";
first = false;
yield JSON.stringify(key) + ":";
const childPosition =
position === "map"
? "schema"
: position === "schema" && schemaMaps.has(key)
? "map"
: position === "schema" &&
(schemaArrays.has(key) || schemaChildren.has(key))
? "schema"
: "data";
yield* serialize(child, childPosition, active);
}
yield "}";
} else {
yield JSON.stringify(value);
}
}
const chunks: string[] = [];
let bytes = 0;
for (const chunk of serialize(root, "schema", new Set())) {
bytes += Buffer.byteLength(chunk, "utf8");
if (bytes > budgetBytes || !resolved) return undefined;
chunks.push(chunk);
}
return resolved
? (JSON.parse(chunks.join("")) as Record<string, unknown>)
: undefined;
};
/** Preserve every possible type, including referenced and nullable unions. */
const detectedTypes = (
schema: unknown,
root: Record<string, unknown>,
active: ReadonlySet<string> = new Set(),
): readonly string[] | undefined => {
if (!isSchemaObject(schema)) return undefined;
if (typeof schema.$ref === "string") {
if (active.has(schema.$ref)) return undefined;
return detectedTypes(
resolveLocalReference(root, schema.$ref),
root,
new Set([...active, schema.$ref]),
);
}
if (typeof schema.type === "string") return [schema.type];
if (
Array.isArray(schema.type) &&
schema.type.every((type) => typeof type === "string")
)
return schema.type;
const alternatives = Array.isArray(schema.anyOf)
? schema.anyOf
: Array.isArray(schema.oneOf)
? schema.oneOf
: [];
if (alternatives.length === 0) return undefined;
const types = alternatives.map((branch) =>
detectedTypes(branch, root, active),
);
if (types.some((type) => type === undefined)) return undefined;
return [...new Set(types.flatMap((type) => type ?? []))];
};
const REDUCED_ANNOTATION =
"Advertised in reduced form by the compact input schema profile; the server still validates complete canonical inputs.";
/**
* Present only the root property shape with per-property guidance. Used for
* schemas whose validation structure alone exceeds the compact budget; the
* canonical schema still rejects malformed calls server-side.
*/
const reduceToShallowProperties = (
schema: Record<string, unknown>,
): Record<string, unknown> => {
const properties = isSchemaObject(schema.properties) ? schema.properties : {};
const description = [
typeof schema.description === "string" ? schema.description : undefined,
REDUCED_ANNOTATION,
]
.filter((text) => text !== undefined)
.join(" ");
const $schema =
typeof schema.$schema === "string" ? { $schema: schema.$schema } : {};
const shallow = Object.fromEntries(
Object.entries(properties).map(([name, value]) => {
const types = detectedTypes(value, schema);
const type = types?.length === 1 ? types[0] : types;
const description =
isSchemaObject(value) && typeof value.description === "string"
? { description: value.description }
: {};
return [
name,
{ ...(type === undefined ? {} : { type }), ...description },
];
}),
);
return {
...$schema,
type: "object",
description,
properties: shallow,
...(Array.isArray(schema.required)
? { required: [...schema.required] }
: {}),
...(typeof schema.minProperties === "number"
? { minProperties: schema.minProperties }
: {}),
...(isSchemaObject(schema.dependentRequired)
? { dependentRequired: { ...schema.dependentRequired } }
: {}),
...(schema.additionalProperties === false
? { additionalProperties: false }
: {}),
};
};
/**
* Render an advertised input schema within a per-schema byte budget for
* providers that validate the literal serialized schema size. Validation
* keywords and the canonical contract are preserved; only
* annotation prose and, for structurally oversized schemas, nested
* validation detail leave the advertised form.
*/
export const compactAdvertisedInputSchema = (
schema: Record<string, unknown>,
budgetBytes: number,
): CompactedInputSchema => {
const inline = inlineWithinBudget(stripAnnotations(schema), budgetBytes);
if (inline !== undefined) return { schema: inline, presentation: "inline" };
const reduced = reduceToShallowProperties(schema);
if (serializedBytes(reduced) <= budgetBytes)
return { schema: reduced, presentation: "reduced" };
// A future contract with an extreme root-property count still advertises
// an honest, budget-fitting shape instead of an oversized one.
return {
schema: reduceToShallowProperties({
type: "object",
description:
(typeof schema.description === "string"
? `${schema.description} `
: "") + REDUCED_ANNOTATION,
}),
presentation: "reduced",
};
};
+3 -3
View File
@@ -1,6 +1,6 @@
import { isDeepStrictEqual } from "node:util";
const schemaMaps = new Set([
export const schemaMaps = new Set([
"$defs",
"definitions",
"properties",
@@ -8,8 +8,8 @@ const schemaMaps = new Set([
"dependentSchemas",
"dependencies",
]);
const schemaArrays = new Set(["allOf", "anyOf", "oneOf", "prefixItems"]);
const schemaChildren = new Set([
export const schemaArrays = new Set(["allOf", "anyOf", "oneOf", "prefixItems"]);
export const schemaChildren = new Set([
"items",
"additionalItems",
"contains",
+43
View File
@@ -95,6 +95,46 @@ export const toolOutputSchemaWithMetadata = <Contract extends ToolContract>(
},
);
/**
* Apply a wire-presentation transform to one schema's advertised input
* projection. The wrapper preserves the schema's own parser and standard
* metadata, so both Zod registrations and plain standard schemas keep their
* canonical validation while only the advertised JSON changes.
*/
export const transformAdvertisedInputJsonSchema = <Schema>(
schema: Schema,
transform: (projected: Record<string, unknown>) => Record<string, unknown>,
): Schema => {
const standard = Reflect.get(schema as object, "~standard");
if (!isSchemaRecord(standard)) return schema;
const jsonSchema = Reflect.get(standard, "jsonSchema");
if (!isSchemaRecord(jsonSchema)) return schema;
const project = Reflect.get(jsonSchema, "input");
if (typeof project !== "function") return schema;
const byTarget = new Map<string, Record<string, unknown>>();
const transformedInput = (options: unknown): unknown => {
const target = Reflect.get(options as object, "target");
if (typeof target === "string") {
const cached = byTarget.get(target);
if (cached !== undefined) return cached;
}
const projected = (project as (options: unknown) => unknown)(options);
const transformed = isSchemaRecord(projected)
? transform(projected)
: projected;
if (typeof target === "string" && isSchemaRecord(transformed))
byTarget.set(target, transformed);
return transformed;
};
return {
...(schema as object),
"~standard": {
...standard,
jsonSchema: { ...jsonSchema, input: transformedInput },
},
} as Schema;
};
const hasRelativeSchemaReference = (
schema: Record<string, unknown>,
): boolean => {
@@ -164,6 +204,9 @@ const memoizeByTarget = (
};
};
const isSchemaRecord = (value: unknown): value is Record<string, unknown> =>
typeof value === "object" && value !== null && !Array.isArray(value);
const fallbackPropertyDescription = (property: string): string | undefined => {
const explicit = Object.hasOwn(PROPERTY_DESCRIPTIONS, property)
? PROPERTY_DESCRIPTIONS[property]
+76
View File
@@ -5,14 +5,22 @@ import {
isJSONRPCNotification,
isJSONRPCResultResponse,
type Implementation,
type Icon,
type JSONRPCMessage,
type McpServerOptions,
type RegisteredTool,
type RequestId,
type ScopeChallengeHandler,
type StandardSchemaWithJSON,
type ToolAnnotations,
type ToolCallback,
type Transport,
type TransportSendOptions,
} from "@modelcontextprotocol/server";
import { analysisErrorProjectionSchema } from "../contracts/errorSchemas.js";
import { compactAdvertisedInputSchema } from "../contracts/compactInputSchema.js";
import { transformAdvertisedInputJsonSchema } from "../contracts/toolSchemaMetadata.js";
import type { EvidenceWriter } from "../application/investigation/InvestigationRecordPort.js";
import { AnalysisResourceConstraintError } from "../domain/analysisErrorCore.js";
import { projectAnalysisError } from "../domain/analysisErrorProjection.js";
@@ -21,6 +29,24 @@ import { jsonObjectSchema } from "../domain/jsonValue.js";
import type { ToolResultDelivery } from "./toolResult.js";
import { encodeToolResult } from "./toolResultEncoding.js";
/** Compact advertised input schemas within this per-schema byte budget. */
export interface CompactInputSchemaPresentation {
readonly budgetBytes: number;
}
/** A Zod schema whose standard interface exposes a JSON Schema projection. */
const hasAdvertisedInputJsonSchema = (value: unknown): boolean => {
if (typeof value !== "object" || value === null) return false;
const standard = Reflect.get(value, "~standard");
if (typeof standard !== "object" || standard === null) return false;
const jsonSchema = Reflect.get(standard, "jsonSchema");
return (
typeof jsonSchema === "object" &&
jsonSchema !== null &&
Reflect.get(jsonSchema, "input") !== undefined
);
};
/** Bind oversized MCP error recovery to this server's Evidence ledger. */
export class EvidenceMcpServer extends McpServer {
constructor(
@@ -30,10 +56,60 @@ export class EvidenceMcpServer extends McpServer {
| EvidenceWriter["recordEvidence"]
| undefined,
readonly delivery: ToolResultDelivery,
private readonly compactPresentation:
| CompactInputSchemaPresentation
| undefined = undefined,
) {
super(info, options);
}
/**
* Compact the advertised input schema of every registered tool when the
* compact profile is active. The registered canonical schema, its parser,
* and output schemas are untouched; only the advertised JSON Schema
* projection is rendered within the provider's per-schema size budget.
* The signatures mirror the SDK's registrations so call sites keep their
* inferred handler argument types.
*/
override registerTool<
OutputArgs extends StandardSchemaWithJSON,
InputArgs extends StandardSchemaWithJSON | undefined = undefined,
>(
name: string,
config: {
title?: string;
description?: string;
inputSchema?: InputArgs;
outputSchema?: OutputArgs;
annotations?: ToolAnnotations;
icons?: Icon[];
scopeChallenge?: ScopeChallengeHandler;
_meta?: Record<string, unknown>;
},
callback: ToolCallback<InputArgs>,
): RegisteredTool {
const budget = this.compactPresentation?.budgetBytes;
const inputSchema =
budget !== undefined &&
config !== undefined &&
hasAdvertisedInputJsonSchema(config.inputSchema)
? transformAdvertisedInputJsonSchema(
config.inputSchema as Parameters<
typeof transformAdvertisedInputJsonSchema
>[0],
(projected) =>
compactAdvertisedInputSchema(projected, budget).schema,
)
: undefined;
return super.registerTool(
name,
(inputSchema === undefined
? config
: { ...config, inputSchema }) as never,
callback as never,
);
}
/** Preserve the SDK transport lifecycle while retaining oversized errors. */
override connect(transport: Transport): Promise<void> {
return super.connect(
+24 -1
View File
@@ -15,6 +15,11 @@ import {
type ToolResultDelivery,
} from "./toolResult.js";
import { parseMcpResponseBudget } from "../config/mcpResponseBudget.js";
import {
COMPACT_INPUT_SCHEMA_BUDGET_BYTES,
parseMcpInputSchemaProfile,
} from "../config/mcpInputSchemaProfile.js";
import type { CompactInputSchemaPresentation } from "./EvidenceMcpServer.js";
import { isAbsolute } from "node:path";
import type { BinarySessionPort } from "../application/binary/BinarySessionPort.js";
@@ -148,6 +153,7 @@ const installSessionToolAvailability = (
const createMcpServer = (
session: BinarySessionPort | undefined,
delivery: ToolResultDelivery,
compactPresentation: CompactInputSchemaPresentation | undefined,
): EvidenceMcpServer =>
new EvidenceMcpServer(
{
@@ -165,6 +171,7 @@ const createMcpServer = (
? undefined
: (evidence) => session.recordEvidence(evidence),
delivery,
compactPresentation,
);
/**
@@ -183,7 +190,11 @@ export const createServer = (
const selectedOptions = { ...options, environment, delivery };
const startedAt = new Date().toISOString();
const logger = options.logger ?? silentLogger;
const server = createMcpServer(session, delivery);
const server = createMcpServer(
session,
delivery,
selectCompactInputSchemaPresentation(environment),
);
const android =
options.androidAnalysis ?? createAndroidAnalysisProvider(environment);
const availability = installSessionToolAvailability(
@@ -256,6 +267,18 @@ const selectToolResultDelivery = (
return createToolResultDelivery(configured.value);
};
const selectCompactInputSchemaPresentation = (
environment: Readonly<NodeJS.ProcessEnv>,
): CompactInputSchemaPresentation | undefined => {
const configured = parseMcpInputSchemaProfile(
environment.REA_MCP_INPUT_SCHEMA_PROFILE,
);
if (!configured.ok) throw configured.error;
return configured.value === "compact"
? { budgetBytes: COMPACT_INPUT_SCHEMA_BUDGET_BYTES }
: undefined;
};
const registerConfiguredAnalysisTools = (
{
server,
@@ -0,0 +1,281 @@
import { Client, InMemoryTransport } from "@modelcontextprotocol/client";
import { STDIO_DEFAULT_MAX_BUFFER_SIZE } from "@modelcontextprotocol/server";
import { Ajv2020 } from "ajv/dist/2020.js";
import { describe, expect, it } from "vitest";
import { TOOL_CONTRACTS } from "../../../src/contracts/toolContracts.js";
import { compactAdvertisedInputSchema } from "../../../src/contracts/compactInputSchema.js";
import {
COMPACT_INPUT_SCHEMA_BUDGET_BYTES,
parseMcpInputSchemaProfile,
} from "../../../src/config/mcpInputSchemaProfile.js";
import { processScenarioSchema } from "../../../src/domain/process/processScenario.js";
import { EvidenceMcpServer } from "../../../src/server/EvidenceMcpServer.js";
import { ToolResultDelivery } from "../../../src/server/toolResult.js";
import { toolRegistrationOptions } from "../../../src/server/toolRegistrationOptions.js";
interface AdvertisedTool {
readonly name: string;
readonly inputSchema: Record<string, unknown>;
readonly outputSchema?: Record<string, unknown> | undefined;
readonly description?: string | undefined;
}
const serializedBytes = (value: unknown): number =>
Buffer.byteLength(JSON.stringify(value), "utf8");
const listToolsThroughTransport = async (
server: EvidenceMcpServer,
): Promise<readonly AdvertisedTool[]> => {
const client = new Client({ name: "compact-profile", version: "0" });
const [clientTransport, serverTransport] =
InMemoryTransport.createLinkedPair();
try {
await server.connect(serverTransport);
await client.connect(clientTransport);
return (await client.listTools()).tools;
} finally {
await Promise.allSettled([client.close(), server.close()]);
}
};
const registerCatalog = (server: EvidenceMcpServer): void => {
for (const contract of TOOL_CONTRACTS)
server.registerTool(
contract.name,
toolRegistrationOptions(contract),
async () => ({ content: [] }),
);
};
const compactServer = (): EvidenceMcpServer =>
new EvidenceMcpServer(
{ name: "compact-profile", version: "0" },
{ capabilities: {} },
undefined,
new ToolResultDelivery(STDIO_DEFAULT_MAX_BUFFER_SIZE),
{ budgetBytes: COMPACT_INPUT_SCHEMA_BUDGET_BYTES },
);
const fullServer = (): EvidenceMcpServer =>
new EvidenceMcpServer(
{ name: "full-profile", version: "0" },
{ capabilities: {} },
undefined,
new ToolResultDelivery(STDIO_DEFAULT_MAX_BUFFER_SIZE),
);
describe("MCP compact input schema profile", () => {
it("advertises every tool input schema within the provider budget", async () => {
const server = compactServer();
registerCatalog(server);
const tools = await listToolsThroughTransport(server);
expect(tools.map(({ name }) => name).sort()).toEqual(
TOOL_CONTRACTS.map(({ name }) => name).sort(),
);
const oversized = tools.filter(
({ inputSchema }) =>
serializedBytes(inputSchema) > COMPACT_INPUT_SCHEMA_BUDGET_BYTES,
);
expect(oversized.map(({ name }) => name)).toEqual([]);
});
it("keeps output schemas and tool descriptions identical to the default profile", async () => {
const compact = compactServer();
const full = fullServer();
registerCatalog(compact);
registerCatalog(full);
const [compactTools, fullTools] = await Promise.all([
listToolsThroughTransport(compact),
listToolsThroughTransport(full),
]);
const fullByName = new Map(fullTools.map((tool) => [tool.name, tool]));
expect(compactTools.length).toBe(fullTools.length);
for (const tool of compactTools) {
const reference = fullByName.get(tool.name);
if (reference === undefined) throw new Error(`Missing ${tool.name}`);
expect(tool.outputSchema, tool.name).toEqual(reference.outputSchema);
expect(tool.description, tool.name).toBe(reference.description);
}
});
it("keeps compact schemas valid and accepting canonical examples", async () => {
const server = compactServer();
registerCatalog(server);
const tools = await listToolsThroughTransport(server);
const ajv = new Ajv2020({ strict: false, validateFormats: false });
const contractsByName = new Map<string, (typeof TOOL_CONTRACTS)[number]>(
TOOL_CONTRACTS.map((contract) => [contract.name, contract]),
);
for (const tool of tools) {
expect(
ajv.validateSchema(tool.inputSchema),
`${tool.name}: ${ajv.errorsText(ajv.errors)}`,
).toBe(true);
const contract = contractsByName.get(tool.name);
const validate = ajv.compile(tool.inputSchema);
for (const example of contract?.examples ?? [])
expect(validate(example.input), `${tool.name}: ${example.title}`).toBe(
true,
);
}
});
it("reduces at most the structurally oversized tools", async () => {
const full = fullServer();
registerCatalog(full);
const tools = await listToolsThroughTransport(full);
const reduced: string[] = [];
for (const tool of tools) {
const { presentation } = compactAdvertisedInputSchema(
tool.inputSchema,
COMPACT_INPUT_SCHEMA_BUDGET_BYTES,
);
if (presentation === "reduced") reduced.push(tool.name);
}
expect(reduced.sort()).toEqual([
"build_reconstruction_obligation_ledger",
"capture_browser_scenario",
"compare_application_versions",
"compare_web_captures",
"evaluate_reconstruction_coverage",
"project_managed_application_graph",
]);
expect(tools.length).toBe(TOOL_CONTRACTS.length);
});
it("retains parameter types and avoids client-side reference expansion", async () => {
const server = compactServer();
registerCatalog(server);
const tools = await listToolsThroughTransport(server);
for (const tool of tools) {
// Kimi Code 2.1.1 expands references and defaults description-only
// properties to string. Inline schemas with explicit shallow types
// avoid both conversions for the structurally oversized tools.
expect(JSON.stringify(tool.inputSchema), tool.name).not.toContain(
'"$ref":',
);
}
const comparison = tools.find(
({ name }) => name === "compare_managed_members",
);
expect(comparison?.inputSchema.properties).toMatchObject({
left: {
properties: {
normalized_result: {
type: ["null", "boolean", "object", "array", "number", "string"],
},
},
},
});
const ledger = tools.find(
({ name }) => name === "build_reconstruction_obligation_ledger",
);
expect(ledger?.inputSchema.properties).toMatchObject({
evidence_bundle: { type: "object" },
reviewed_obligations: { type: "array" },
manifest: { type: "object" },
});
const coverage = tools.find(
({ name }) => name === "evaluate_reconstruction_coverage",
);
expect(coverage?.inputSchema.properties).toMatchObject({
coverage: { type: "object" },
boundary_id: { type: "string" },
});
});
it("parses the profile selection the server startup reads", () => {
expect(parseMcpInputSchemaProfile("compact")).toEqual({
ok: true,
value: "compact",
});
expect(parseMcpInputSchemaProfile("surprise")).not.toHaveProperty("value");
});
});
describe("MCP compact input schema canonical enforcement", () => {
it("still rejects canonical-invalid arguments the compact advertisement accepts", async () => {
const server = compactServer();
const contract = TOOL_CONTRACTS.find(
({ name }) => name === "capture_process_scenario",
);
if (contract === undefined)
throw new Error("Process capture contract was not registered");
let handlerCalled = false;
server.registerTool(
contract.name,
toolRegistrationOptions(contract),
async () => {
handlerCalled = true;
return {
content: [{ type: "text" as const, text: "handler ran" }],
isError: true,
};
},
);
const client = new Client({ name: "compact-validation", version: "0" });
const [clientTransport, serverTransport] =
InMemoryTransport.createLinkedPair();
try {
await server.connect(serverTransport);
await client.connect(clientTransport);
const advertised = (await client.listTools()).tools.find(
({ name }) => name === contract.name,
);
if (advertised === undefined)
throw new Error("Process capture tool was not advertised");
const ajv = new Ajv2020({ strict: false, validateFormats: false });
const validate = ajv.compile(advertised.inputSchema);
const valid = { executable: "node", environment: { APP_MODE: "test" } };
const reserved = {
executable: "node",
environment: { REA_PROCESS_RUN_ID: "caller-value" },
};
expect(validate(valid)).toBe(true);
expect(processScenarioSchema.safeParse(valid).success).toBe(true);
expect(processScenarioSchema.safeParse(reserved).success).toBe(false);
const result = await client.callTool({
name: contract.name,
arguments: reserved,
});
expect(result.isError).toBe(true);
expect(handlerCalled).toBe(false);
} finally {
await Promise.allSettled([client.close(), server.close()]);
}
});
it("compacts plain standard schemas that are not Zod instances", async () => {
// Mirrors registerWebNetworkCaptureTool: a plain standard schema reusing
// the advertised projection while owning its own permissive validator.
const server = compactServer();
const contract = TOOL_CONTRACTS.find(
({ name }) => name === "inspect_web_network_capture",
);
if (contract === undefined)
throw new Error("Network capture contract was not registered");
const registration = toolRegistrationOptions(contract);
const inputSchema = {
"~standard": {
...registration.inputSchema["~standard"],
vendor: "rea-application",
validate: (value: unknown) => ({ value }),
},
};
server.registerTool(
contract.name,
{ ...registration, inputSchema },
async () => ({ content: [] }),
);
const tools = await listToolsThroughTransport(server);
const advertised = tools.find(({ name }) => name === contract.name);
if (advertised === undefined)
throw new Error("Network capture tool was not advertised");
expect(serializedBytes(advertised.inputSchema)).toBeLessThanOrEqual(
COMPACT_INPUT_SCHEMA_BUDGET_BYTES,
);
const ajv = new Ajv2020({ strict: false, validateFormats: false });
expect(ajv.validateSchema(advertised.inputSchema)).toBe(true);
});
});
@@ -43,12 +43,14 @@ const exerciseDeliveryFailure = async (
recordEvidence,
delivery,
);
server.registerTool("large_failure", { inputSchema: {} }, async () =>
oversizedFailure(),
server.registerTool(
"large_failure",
{ inputSchema: z.object({}) },
async () => oversizedFailure(),
);
server.registerTool(
"healthy",
{ inputSchema: {} },
{ inputSchema: z.object({}) },
async (): Promise<CallToolResult> => ({
content: [{ type: "text", text: "still connected" }],
}),
@@ -161,7 +163,7 @@ it("delivers an error whose text fits although structured and text copies would
},
delivery,
);
server.registerTool("mid_failure", { inputSchema: {} }, async () =>
server.registerTool("mid_failure", { inputSchema: z.object({}) }, async () =>
failureWithName(nameBytes),
);
const client = new Client({ name: "mid-sized-error-client", version: "1" });
@@ -0,0 +1,29 @@
import { describe, expect, it } from "vitest";
import { parseMcpInputSchemaProfile } from "../../../src/config/mcpInputSchemaProfile.js";
describe("MCP input schema profile configuration", () => {
it("accepts the full and compact presentations", () => {
expect(parseMcpInputSchemaProfile(undefined)).toEqual({
ok: true,
value: undefined,
});
expect(parseMcpInputSchemaProfile("full")).toEqual({
ok: true,
value: "full",
});
expect(parseMcpInputSchemaProfile("compact")).toEqual({
ok: true,
value: "compact",
});
});
it.each(["", "FULL", "compact ", "minimal", "0"])(
"rejects unknown profiles: %j",
(value) => {
const parsed = parseMcpInputSchemaProfile(value);
if (parsed.ok) throw new Error("Expected invalid profile");
expect(parsed.error.message).toContain("REA_MCP_INPUT_SCHEMA_PROFILE");
},
);
});
@@ -1,6 +1,7 @@
import { Client, InMemoryTransport } from "@modelcontextprotocol/client";
import { STDIO_DEFAULT_MAX_BUFFER_SIZE } from "@modelcontextprotocol/server";
import { afterEach, expect, it, vi } from "vitest";
import { z } from "zod";
import { createAnalysisExecution } from "../../src/application/AnalysisProvider.js";
import { AnalysisInputError } from "../../src/domain/analysisErrorCore.js";
@@ -73,16 +74,19 @@ it("keeps independent server budgets for successful results and oversized errors
Math.ceil((smallBudget + 65536) / 17),
);
for (const server of [small, large])
server.registerTool("selected_failure", { inputSchema: {} }, async () =>
server.delivery.toErrorToolResult(
new AnalysisInputError("fixture", undefined, [
{
path: ["procedure"],
reason: "invalid_value",
message: diagnostic,
},
]),
),
server.registerTool(
"selected_failure",
{ inputSchema: z.object({}) },
async () =>
server.delivery.toErrorToolResult(
new AnalysisInputError("fixture", undefined, [
{
path: ["procedure"],
reason: "invalid_value",
message: diagnostic,
},
]),
),
);
const [smallClient, largeClient] = await Promise.all([