diff --git a/docs/android-analysis.md b/docs/android-analysis.md index efcb05c64..849aeb2b1 100644 --- a/docs/android-analysis.md +++ b/docs/android-analysis.md @@ -126,8 +126,13 @@ also stops target hashing before provider admission. These operations do not execute the APK or provide split APK/AAB handling, signature validation, full resource-table semantics, native-library analysis or -Android runtime capture. Existing artifact inventory/extraction tools can supply -archive evidence. Project that Evidence with the execution-free inventory tool: +Android runtime capture. An `.aab` suffix is a ZIP archive, not an APK, so +Android tools and the application graph do not treat it as a standalone APK. +Existing artifact inventory/extraction tools can supply archive evidence. +Inventory keeps ZIP names that differ only in case and records that a +case-insensitive destination cannot store both spellings. Extraction fails +only when that destination directory cannot keep both names. Project that +Evidence with the execution-free inventory tool: ```sh rea project-android-application-graph '{"inventory_evidence":[]}' diff --git a/docs/installation.md b/docs/installation.md index 38ea09f5a..acc285e69 100644 --- a/docs/installation.md +++ b/docs/installation.md @@ -135,10 +135,17 @@ listed after the table because its connector is not one of these files: | Antigravity | `antigravity` | | GitHub Copilot CLI | `copilot_cli` | | Command Code | `commandcode` | +| Qwen Code | `qwen_code` | | VS Code | `vscode` | | Grok Build | `grok_build` | | OMP | `omp` | +For Qwen Code, setup registers `rea` in the `mcpServers` table of the +user-level `~/.qwen/settings.json`, or `$QWEN_HOME/settings.json` when configured. +Setup and doctor follow Qwen Code's tilde and working-directory-relative home +overrides. Qwen Code also discovers personal skills in `~/.agents/skills`, +where setup installs the bundled workflow. + For OMP, setup writes a `type: "stdio"` entry to the user-level `~/.omp/agent/mcp.json`. It follows `PI_CONFIG_DIR`, an absolute `PI_CODING_AGENT_DIR`, and the profile selected by `OMP_PROFILE` or @@ -194,10 +201,10 @@ save verified paths for an existing Ghidra installation. The bundled skill is installed where each selected client discovers personal skills: Claude Code uses `~/.claude/skills` (or -`$CLAUDE_CONFIG_DIR/skills` when configured), while other supported clients -use the shared `~/.agents/skills` directory. A mixed selection plans both -paths. Selecting the skill without a client uses the shared directory. Setup -leaves existing skill copies in other locations untouched. +`$CLAUDE_CONFIG_DIR/skills` when configured), while other supported clients, +including Qwen Code, use the shared `~/.agents/skills` directory. A mixed selection +plans the relevant paths. Selecting the skill without a client uses the shared +directory. Setup leaves existing skill copies in other locations untouched. `doctor --skill --json` verifies the selected copies against the bundled instructions and references. Consumers should use `identity.skill.state`, diff --git a/docs/process-capture.md b/docs/process-capture.md index c315e3e9e..5cc246ec2 100644 --- a/docs/process-capture.md +++ b/docs/process-capture.md @@ -110,11 +110,14 @@ while comparisons and trace assertions use the normalized text. Retaining the original adds at most `limits.output_bytes` of text, because it comes from the same admitted PTY chunks rather than a second unbounded stream. -Rendered frame `lines` hold the visible rows after normalization, without -trailing spaces; pad a line to the frame's `columns` to restore the fixed-width -row. `serialized_state` keeps the exact terminal state. Captures commit this -line format in their comparison contract, so a capture whose lines were padded -to full width does not compare as the same contract. +Rendered frame `lines` hold the visible rows after normalization, with trailing +U+0020 spaces removed. Blank rows and internal spaces remain. `columns` records +terminal cell width; normalization and Unicode can make string lengths differ +from that width. `serialized_state` holds terminal serialization after the +selected normalization. Original admitted PTY chunks are available through +`raw_data ?? data`, as described above. Captures commit this line format in their +comparison contract, so a capture whose lines were padded to full width does not +compare as the same contract. Port normalization recognizes explicit `port`, `tcp_port`, `udp_port`, and `listen` fields, URL authorities, IP endpoints, and `localhost` endpoints, with diff --git a/scripts/verify-package-environment.mjs b/scripts/verify-package-environment.mjs index f9ee010af..1a97cfbab 100644 --- a/scripts/verify-package-environment.mjs +++ b/scripts/verify-package-environment.mjs @@ -85,6 +85,7 @@ export async function verifyPackageEnvironment({ PI_CODING_AGENT_DIR: undefined, PI_CONFIG_DIR: undefined, PI_PROFILE: undefined, + QWEN_HOME: undefined, OPENCODE_CONFIG: undefined, XDG_CONFIG_HOME: join(home, ".config"), PATH: `${fakeBin}:${process.env.PATH ?? ""}`, diff --git a/src/application/ClientRegistrationStatus.ts b/src/application/ClientRegistrationStatus.ts index a657f1df4..1f5a12473 100644 --- a/src/application/ClientRegistrationStatus.ts +++ b/src/application/ClientRegistrationStatus.ts @@ -91,6 +91,7 @@ export const readClientRegistrationStatuses = async ( PI_CODING_AGENT_DIR: options.environment.PI_CODING_AGENT_DIR, PI_CONFIG_DIR: options.environment.PI_CONFIG_DIR, PI_PROFILE: options.environment.PI_PROFILE, + QWEN_HOME: options.environment.QWEN_HOME, SAND_DATA_ROOT: options.environment.SAND_DATA_ROOT, XDG_CONFIG_HOME: options.environment.XDG_CONFIG_HOME, }, diff --git a/src/application/SupportedClients.ts b/src/application/SupportedClients.ts index 97467a227..52f5577ca 100644 --- a/src/application/SupportedClients.ts +++ b/src/application/SupportedClients.ts @@ -1,4 +1,4 @@ -import { isAbsolute, join } from "node:path"; +import { isAbsolute, join, resolve } from "node:path"; import { lstatSync } from "node:fs"; /** One supported client configuration location. */ @@ -35,6 +35,7 @@ interface ClientPathContext { readonly PI_CODING_AGENT_DIR?: string | undefined; readonly PI_CONFIG_DIR?: string | undefined; readonly PI_PROFILE?: string | undefined; + readonly QWEN_HOME?: string | undefined; readonly SAND_DATA_ROOT?: string | undefined; readonly XDG_CONFIG_HOME?: string | undefined; }; @@ -85,6 +86,25 @@ const claudeCodeMarkerDirectory = ({ home, env }: ClientPathContext): string => const codexDirectory = ({ home, env }: ClientPathContext): string => env.CODEX_HOME ?? join(home, ".codex"); +/** Match Qwen Code's home override, including tilde and cwd-relative paths. */ +const qwenCodeDirectory = ({ home, env }: ClientPathContext): string => { + const override = env.QWEN_HOME; + if (!override) return join(home, ".qwen"); + const expanded = + override === "~" + ? home + : override.startsWith("~/") || override.startsWith("~\\") + ? join( + home, + ...override + .slice(2) + .split(/[/\\]+/) + .filter(Boolean), + ) + : override; + return isAbsolute(expanded) ? expanded : resolve(expanded); +}; + const grokDirectory = ({ home, env }: ClientPathContext): string => env.GROK_HOME ?? join(home, ".grok"); @@ -272,6 +292,14 @@ export const SUPPORTED_CLIENT_DEFINITIONS = [ markerPath: commandCodeDirectory, format: "commandcode", }, + { + name: "qwen_code", + displayName: "Qwen Code", + configPath: (context: ClientPathContext) => + join(qwenCodeDirectory(context), "settings.json"), + markerPath: qwenCodeDirectory, + format: "json", + }, { name: "vscode", displayName: "VS Code", @@ -323,6 +351,7 @@ export const supportedClients = ( PI_CODING_AGENT_DIR: process.env.PI_CODING_AGENT_DIR, PI_CONFIG_DIR: process.env.PI_CONFIG_DIR, PI_PROFILE: process.env.PI_PROFILE, + QWEN_HOME: process.env.QWEN_HOME, SAND_DATA_ROOT: process.env.SAND_DATA_ROOT, XDG_CONFIG_HOME: process.env.XDG_CONFIG_HOME, }, diff --git a/src/artifacts/ArtifactPaths.ts b/src/artifacts/ArtifactPaths.ts index d5ec35cb4..ce8d6c515 100644 --- a/src/artifacts/ArtifactPaths.ts +++ b/src/artifacts/ArtifactPaths.ts @@ -1,9 +1,24 @@ import { posix } from "node:path"; +import { compareUnicodeCodePoints } from "../domain/unicodeCodePointOrder.js"; import { ArtifactReaderFailure } from "./ArtifactReader.js"; const DRIVE_OR_UNC = /^(?:[A-Za-z]:|\\|\/\/)/u; +/** + * English Unicode case folding used only to describe logical names a + * case-insensitive destination might collapse. It is not a filesystem case table. + */ +const ENGLISH_UNICODE_CASE_NOTE = + "This comparison is not a filesystem observation."; + +/** Inventory limitation when logical names collide under English Unicode case folding. */ +export const ENGLISH_UNICODE_CASE_INVENTORY_LIMITATION = `Logical archive names collide under English (en-US) Unicode case folding; a case-insensitive destination cannot store both spellings. ${ENGLISH_UNICODE_CASE_NOTE}`; + +/** User-facing prefix for a destination that cannot keep two logical spellings. */ +export const DESTINATION_CASE_COLLISION_PREFIX = + "Destination filesystem cannot store both "; + /** Normalize an untrusted logical archive path without touching filesystem. */ export const normalizeArtifactPath = (input: string): string => { if ( @@ -30,7 +45,7 @@ export const normalizeArtifactPath = (input: string): string => { return normalized; }; -/** Reject exact, Unicode-normalized, case-folded, and prefix collisions. */ +/** Reject exact and same-spelling prefix collisions. Case-distinct names stay distinct. */ export class ArtifactPathRegistry { readonly #root: PathTrieNode = { kind: undefined, children: new Map() }; @@ -43,16 +58,10 @@ export class ArtifactPathRegistry { "path", `Artifact prefix conflict: ${path}`, ); - const foldedPart = part.toLocaleLowerCase("en-US"); - let child = node.children.get(foldedPart); - if (child !== undefined && child.spelling !== part) - throw new ArtifactReaderFailure( - "path", - `Artifact path collision: ${path} differs only in case from ${[...parts.slice(0, index), child.spelling].join("/")}`, - ); + let child = node.children.get(part); if (child === undefined) { - child = { spelling: part, kind: undefined, children: new Map() }; - node.children.set(foldedPart, child); + child = { kind: undefined, children: new Map() }; + node.children.set(part, child); } node = child; if (index === parts.length - 1) { @@ -73,7 +82,144 @@ export class ArtifactPathRegistry { } interface PathTrieNode { - readonly spelling?: string; kind: "file" | "directory" | "symlink" | "slice" | undefined; readonly children: Map; } + +interface CaseNode { + readonly bySpelling: Map; + readonly spellingsByFold: Map; + readonly terminals: string[]; +} + +const emptyCaseNode = (): CaseNode => ({ + bySpelling: new Map(), + spellingsByFold: new Map(), + terminals: [], +}); + +const englishCaseFold = (part: string): string => + part.toLocaleLowerCase("en-US"); + +/** + * Record English (en-US) Unicode case collisions on each affected occurrence. + * Returns the inventory limitation when any logical names collide. + */ +export const annotateEnglishUnicodeCaseCollisions = ( + occurrences: readonly { + readonly logical_path: string; + limitations: string[]; + }[], +): string | undefined => { + const collisions = englishUnicodeCaseCollisions( + occurrences.map(({ logical_path }) => logical_path), + ); + if (collisions.size === 0) return undefined; + for (const occurrence of occurrences) { + const others = collisions.get(occurrence.logical_path); + if (others === undefined || others.length === 0) continue; + occurrence.limitations.push( + `Logical path ${occurrence.logical_path} collides under English (en-US) Unicode case folding with ${others.join(", ")}; a case-insensitive destination cannot store both. ${ENGLISH_UNICODE_CASE_NOTE}`, + ); + } + return ENGLISH_UNICODE_CASE_INVENTORY_LIMITATION; +}; + +/** + * Name the directory-listing spelling that differs from the requested segment + * only by English (en-US) Unicode case. Exact matches are the same logical name. + */ +export const destinationCaseCollisionMessage = ( + requestedPath: string, + existingNames: readonly string[], +): string | undefined => { + const slash = requestedPath.lastIndexOf("/"); + const segment = slash < 0 ? requestedPath : requestedPath.slice(slash + 1); + if (existingNames.includes(segment)) return undefined; + const folded = englishCaseFold(segment); + const existing = existingNames + .filter((name) => englishCaseFold(name) === folded) + .sort(compareUnicodeCodePoints)[0]; + if (existing === undefined) return undefined; + const parent = slash < 0 ? "" : requestedPath.slice(0, slash + 1); + return `${DESTINATION_CASE_COLLISION_PREFIX}${requestedPath} and ${parent}${existing}; inventory retains both logical names.`; +}; + +const englishUnicodeCaseCollisions = ( + paths: readonly string[], +): ReadonlyMap => { + const root = emptyCaseNode(); + for (const path of paths) { + if (path.length === 0 || path === ".") continue; + let node = root; + for (const part of path.split("/")) { + const folded = englishCaseFold(part); + const spellings = node.spellingsByFold.get(folded); + if (spellings === undefined) node.spellingsByFold.set(folded, [part]); + else if (!spellings.includes(part)) spellings.push(part); + let child = node.bySpelling.get(part); + if (child === undefined) { + child = emptyCaseNode(); + node.bySpelling.set(part, child); + } + node = child; + } + node.terminals.push(path); + } + const collisions = new Map>(); + recordCaseCollisions(root, collisions); + return new Map( + [...collisions.entries()].map(([path, others]) => [ + path, + [...others].sort(compareUnicodeCodePoints), + ]), + ); +}; + +const recordCaseCollisions = ( + node: CaseNode, + collisions: Map>, +): void => { + for (const spellings of node.spellingsByFold.values()) + if (spellings.length > 1) noteFoldedSpellings(node, spellings, collisions); + for (const child of node.bySpelling.values()) + recordCaseCollisions(child, collisions); +}; + +const noteFoldedSpellings = ( + node: CaseNode, + spellings: readonly string[], + collisions: Map>, +): void => { + const groups = spellings.map((spelling) => { + const child = node.bySpelling.get(spelling); + return child === undefined ? [] : terminalsUnder(child); + }); + for (const [index, group] of groups.entries()) + noteCaseGroup( + group, + groups.flatMap((candidate, candidateIndex) => + candidateIndex === index ? [] : candidate, + ), + collisions, + ); +}; + +const noteCaseGroup = ( + group: readonly string[], + others: readonly string[], + collisions: Map>, +): void => { + for (const path of group) { + const set = collisions.get(path) ?? new Set(); + for (const other of others) if (other !== path) set.add(other); + collisions.set(path, set); + } +}; + +const terminalsUnder = (node: CaseNode): readonly string[] => { + const paths = [...node.terminals]; + for (const child of node.bySpelling.values()) + paths.push(...terminalsUnder(child)); + return paths; +}; diff --git a/src/artifacts/SafeOutputTree.ts b/src/artifacts/SafeOutputTree.ts index fafb6fee4..05ce0ac68 100644 --- a/src/artifacts/SafeOutputTree.ts +++ b/src/artifacts/SafeOutputTree.ts @@ -4,6 +4,7 @@ import { lstat, mkdir, open, + readdir, realpath, rm, type FileHandle, @@ -14,6 +15,7 @@ import { streamChunkToBuffer } from "./StreamBytes.js"; import { ArtifactPathRegistry, + destinationCaseCollisionMessage, normalizeArtifactPath, } from "./ArtifactPaths.js"; import { ArtifactReaderFailure } from "./ArtifactReader.js"; @@ -134,7 +136,8 @@ export class SafeOutputTree { constants.O_WRONLY | constants.O_NOFOLLOW, 0o600, - ).catch((cause: unknown) => { + ).catch(async (cause: unknown) => { + await throwIfDestinationCaseCollision(dirname(destination), path, cause); throw new ArtifactReaderFailure( "path", `Could not exclusively create extraction path: ${path}`, @@ -245,17 +248,28 @@ export class SafeOutputTree { if (fileName === undefined) throw new ArtifactReaderFailure("path", "Invalid extraction path"); let current = this.#outputRoot; + let logicalParent = ""; for (const part of parts) { + const parentDirectory = current; + const logicalPath = + logicalParent.length === 0 ? part : `${logicalParent}/${part}`; current = join(current, part); + let created = true; await mkdir(current, { mode: 0o700 }).catch((cause: unknown) => { if (!isAlreadyExists(cause)) throw cause; + created = false; }); + if (!created) + await throwIfDestinationCaseCollision(parentDirectory, logicalPath); const metadata = await lstat(current); - if (!metadata.isDirectory() || metadata.isSymbolicLink()) + if (!metadata.isDirectory() || metadata.isSymbolicLink()) { + await throwIfDestinationCaseCollision(parentDirectory, logicalPath); throw new ArtifactReaderFailure( "path", `Unsafe extraction parent: ${relativePath}`, ); + } + logicalParent = logicalPath; } return join(current, fileName); } @@ -330,6 +344,23 @@ const isNotFound = (cause: unknown): boolean => const isAlreadyExists = (cause: unknown): boolean => cause instanceof Error && "code" in cause && cause.code === "EEXIST"; +/** Fail when this directory already holds another spelling of the requested segment. */ +const throwIfDestinationCaseCollision = async ( + parentDirectory: string, + logicalPath: string, + cause?: unknown, +): Promise => { + const names = await readdir(parentDirectory).catch(() => undefined); + if (names === undefined) return; + const message = destinationCaseCollisionMessage(logicalPath, names); + if (message === undefined) return; + throw new ArtifactReaderFailure( + "path", + message, + cause === undefined ? undefined : { cause }, + ); +}; + const abortIfNeeded = (signal?: AbortSignal): void => { if (signal?.aborted === true) throw new ArtifactReaderFailure( diff --git a/src/artifacts/inventory/scanCanonical.ts b/src/artifacts/inventory/scanCanonical.ts index abd524693..0719e58f4 100644 --- a/src/artifacts/inventory/scanCanonical.ts +++ b/src/artifacts/inventory/scanCanonical.ts @@ -73,12 +73,12 @@ export const scanCanonicalArtifactInventory = async ( try { if (reader instanceof AsarArtifactReader) await reader.prepareContainer(rootDigest?.sha256, options.signal); - const { nodes, occurrences, pendingContradictions } = await scanReader( - reader, - ownedReaders, - options.signal, - integrity, - ); + const { + nodes, + occurrences, + pendingContradictions, + caseCollisionLimitation, + } = await scanReader(reader, ownedReaders, options.signal, integrity); outcome = { kind: "completed", snapshot: await buildInventorySnapshot({ @@ -91,6 +91,7 @@ export const scanCanonicalArtifactInventory = async ( nodes, occurrences, pendingContradictions, + caseCollisionLimitation, }), }; } catch (cause: unknown) { @@ -130,6 +131,7 @@ interface SnapshotBuildInput { readonly nodes: Map; readonly occurrences: MutableOccurrence[]; readonly pendingContradictions: PendingIntegrityContradiction[]; + readonly caseCollisionLimitation: string | undefined; } const buildInventorySnapshot = async ( @@ -202,7 +204,12 @@ const buildInventorySnapshot = async ( edges: orderedEdges, provenance: reader?.provenance() ?? [], integrity_contradictions: integrityContradictions, - limitations: buildLimitations(rootFormat, reader, integrityContradictions), + limitations: buildLimitations( + rootFormat, + reader, + integrityContradictions, + input.caseCollisionLimitation, + ), }; }; @@ -310,8 +317,10 @@ const buildLimitations = ( rootFormat: ArtifactOccurrence["artifact_format"], reader: ArtifactReader | undefined, integrityContradictions: readonly IntegrityContradiction[], + caseCollisionLimitation: string | undefined, ): string[] => [ ...inventoryLimitations(rootFormat, reader), + ...(caseCollisionLimitation === undefined ? [] : [caseCollisionLimitation]), ...(integrityContradictions.length === 0 ? [] : [ diff --git a/src/artifacts/inventory/scanReader.ts b/src/artifacts/inventory/scanReader.ts index 1ee691a74..89b1b628e 100644 --- a/src/artifacts/inventory/scanReader.ts +++ b/src/artifacts/inventory/scanReader.ts @@ -1,6 +1,7 @@ import { isAbsolute } from "node:path"; import { + annotateEnglishUnicodeCaseCollisions, ArtifactPathRegistry, normalizeArtifactPath, } from "../ArtifactPaths.js"; @@ -64,10 +65,16 @@ export const scanReader = async ( readonly nodes: Map; readonly occurrences: MutableOccurrence[]; readonly pendingContradictions: PendingIntegrityContradiction[]; + readonly caseCollisionLimitation: string | undefined; }> => { const { nodes, occurrences, pendingContradictions } = emptyScan(); if (reader === undefined) - return { nodes, occurrences, pendingContradictions }; + return { + nodes, + occurrences, + pendingContradictions, + caseCollisionLimitation: undefined, + }; const context: ScanContext = { reader, signal, @@ -90,7 +97,12 @@ export const scanReader = async ( context.occurrenceByPath, context.expandedContainerIds, )?.occurrence_id ?? null; - return { nodes, occurrences, pendingContradictions }; + return { + nodes, + occurrences, + pendingContradictions, + caseCollisionLimitation: annotateEnglishUnicodeCaseCollisions(occurrences), + }; }; const visitArtifactEntries = async ( diff --git a/src/contracts/artifactToolContracts.ts b/src/contracts/artifactToolContracts.ts index a6926a19f..744ea34c3 100644 --- a/src/contracts/artifactToolContracts.ts +++ b/src/contracts/artifactToolContracts.ts @@ -63,7 +63,7 @@ const artifact = < export const ARTIFACT_TOOL_CONTRACTS = [ artifact( "inspect_artifact", - "Inspect an active archive or application package in one call. Returns the complete content-addressed artifact graph, source Evidence, observations, relationships, integrity contradictions, hypotheses, unexplored branches, limitations, and next probes inline. Read-only DMG mounting is used automatically on supported macOS hosts. Integrity mismatches fail by default; record-and-continue keeps mismatches explicitly untrusted. It does not extract files; use extract_artifact to materialize its regular contents.", + "Inspect an active archive or application package in one call. Returns the complete content-addressed artifact graph once, as the nested inventory Evidence, with root and integrity observations, integrity contradictions, hypotheses, unexplored branches, limitations, and next probes inline. Read-only DMG mounting is used automatically on supported macOS hosts. Integrity mismatches fail by default; record-and-continue keeps mismatches explicitly untrusted. It does not extract files; use extract_artifact to materialize its regular contents.", artifactInventoryInputSchema, ), artifact( diff --git a/src/contracts/inputSchemaPresentation.ts b/src/contracts/inputSchemaPresentation.ts index 8b1bd5b9e..3b83dc4a7 100644 --- a/src/contracts/inputSchemaPresentation.ts +++ b/src/contracts/inputSchemaPresentation.ts @@ -59,7 +59,7 @@ const referencedDescription = ( const describeSchema = ( schema: unknown, root: Readonly>, - describeProperty: (property: string) => string, + describeProperty: (property: string) => string | undefined, ): unknown => { if (!isObject(schema)) return schema; return Object.fromEntries( @@ -70,16 +70,16 @@ const describeSchema = ( Object.fromEntries( Object.entries(value).map(([name, child]) => { const described = describeSchema(child, root, describeProperty); + if (key !== "properties" || !isObject(described)) + return [name, described]; + const description = + referencedDescription(root, described) ?? + describeProperty(name); return [ name, - key === "properties" && isObject(described) - ? { - ...described, - description: - referencedDescription(root, described) ?? - describeProperty(name), - } - : described, + description === undefined + ? described + : { ...described, description }, ]; }), ), @@ -223,7 +223,7 @@ const flattenRootObjectUnion = ( /** Describe canonical input fields and present the root as one object. */ export const presentInputJsonSchema = ( root: Record, - describeProperty: (property: string) => string, + describeProperty: (property: string) => string | undefined, ): Record => { const flattened = flattenRootObjectUnion(root); const described = describeSchema(flattened, flattened, describeProperty); diff --git a/src/contracts/toolInputReferences.test.ts b/src/contracts/toolInputReferences.test.ts index 4441ba9f7..c661fe040 100644 --- a/src/contracts/toolInputReferences.test.ts +++ b/src/contracts/toolInputReferences.test.ts @@ -97,9 +97,14 @@ describe("advertised input references", () => { before: { description: "Exact byte range in the selected artifact." }, after: { description: "Exact byte range in the selected artifact." }, metadata: { default: literal, examples: [literal] }, - constructor: { description: "Value for constructor." }, }, }); + // Inherited object keys never supply a description. + expect(inline).not.toHaveProperty([ + "properties", + "constructor", + "description", + ]); const ajv = target === "draft-2020-12" ? new Ajv2020({ strict: false }) diff --git a/src/contracts/toolSchemaMetadata.ts b/src/contracts/toolSchemaMetadata.ts index 8f43f1708..e2a6e50af 100644 --- a/src/contracts/toolSchemaMetadata.ts +++ b/src/contracts/toolSchemaMetadata.ts @@ -164,7 +164,7 @@ const memoizeByTarget = ( }; }; -const fallbackPropertyDescription = (property: string): string => { +const fallbackPropertyDescription = (property: string): string | undefined => { const explicit = Object.hasOwn(PROPERTY_DESCRIPTIONS, property) ? PROPERTY_DESCRIPTIONS[property] : undefined; @@ -192,5 +192,6 @@ const fallbackPropertyDescription = (property: string): string => { return `Local filesystem ${words} selected for this operation.`; if (property.startsWith("is_") || property.startsWith("has_")) return `Whether ${words}.`; - return `Value for ${words}.`; + // A generic sentence would only restate the property name in every schema. + return undefined; }; diff --git a/src/domain/analysisErrorPresentation.ts b/src/domain/analysisErrorPresentation.ts index 7c71a9b5d..b4fe4ad64 100644 --- a/src/domain/analysisErrorPresentation.ts +++ b/src/domain/analysisErrorPresentation.ts @@ -191,7 +191,7 @@ export const analysisErrorUserMessage = (error: AnalysisError): string => { const standardMessage = standardErrorMessage(error._tag); if (standardMessage !== undefined) return standardMessage; if (error instanceof ArtifactOperationError) - return artifactMessage(error.operation, error.reason); + return artifactMessage(error.operation, error.reason, error.detail); if (error instanceof EvidenceReferenceError) return error.reason === "missing" ? `Evidence ${error.evidenceId} is not retained in this session. Supply complete inline Evidence, re-run its producer, or import its bundle before using this reference.` @@ -329,13 +329,13 @@ const START_FAILURE_TAGS: ReadonlySet = new Set([ const artifactMessage = ( operation: ArtifactOperationError["operation"], reason: ArtifactOperationError["reason"], + detail?: string, ): string => { if (reason === "cancelled") return "Artifact operation was cancelled. Start it again when ready."; if (reason === "limit") return "Artifact is too large to process safely. Narrow the requested path or use a smaller artifact."; - if (reason === "path") - return "Artifact contains an unsafe or conflicting internal path. Inspect the reported path and correct the artifact before retrying."; + if (reason === "path") return artifactPathMessage(detail); if (reason === "unavailable") return "Artifact processing is unavailable for the current target or host. Check artifact support and required tools."; if (reason === "integrity" && operation === "analyze_javascript_application") @@ -345,6 +345,23 @@ const artifactMessage = ( return "Artifact could not be read or written. Check file access and try again."; }; +const artifactPathMessage = (detail: string | undefined): string => { + // Kept in step with DESTINATION_CASE_COLLISION_PREFIX. Domain does not import + // the artifact adapter that produces the detail. + if ( + detail !== undefined && + detail.startsWith("Destination filesystem cannot store both ") + ) + return detail; + if ( + detail !== undefined && + (detail.startsWith("Artifact path is absolute or unsafe:") || + detail.startsWith("Artifact path is not normalized:")) + ) + return "Artifact contains an unsafe internal path. Inspect the reported path and correct the artifact before retrying."; + return "Artifact contains a conflicting internal path. Inspect the reported path before retrying."; +}; + const evidenceFileMessage = ({ operation, reason, diff --git a/src/domain/android/androidApplication.ts b/src/domain/android/androidApplication.ts index 745b8e0c4..674425749 100644 --- a/src/domain/android/androidApplication.ts +++ b/src/domain/android/androidApplication.ts @@ -125,6 +125,8 @@ export const projectAndroidApplication = ( "Manifest, resource, signing, and bytecode semantics require a dedicated Android provider; this projection reports exact inventory paths and hashes only.", "Runtime families are inferred from inventory formats and paths; filename suffixes do not establish valid DEX or JVM class bytes.", "Bridge candidates are path-based hypotheses, not decoded JNI declarations or observed runtime calls.", + "A bridge basis is inferred from the native path and is repeated for every managed component.", + "ZIP inventory cannot see the APK Signing Block, so an empty signing array does not mean the APK is unsigned.", ...(bridgeProjection.coverage.status === "partial" ? [ `Bridge candidate pairs exceeded the projection safety budget; ${bridgeProjection.coverage.omitted_candidates} hypotheses are omitted. Component arrays still include every component from the supplied inventory pages.`, @@ -157,23 +159,12 @@ export const projectAndroidApplication = ( }; const classify = (all: readonly Component[]) => ({ - manifests: all.filter( - ({ path, format }) => - format === "android-manifest" || - /(?:^|\/)AndroidManifest\.xml$/u.test(path), - ), - resources: all.filter( - ({ path, format }) => - format === "android-resources" || /(?:^|\/)resources\.arsc$/u.test(path), - ), - dex: all.filter( - ({ path, format }) => - format === "dex" || /(?:^|\/)[^/]+\.dex$/iu.test(path), - ), - jvm_classes: all.filter( - ({ path, format }) => - format === "jvm-class" || /(?:^|\/)[^/]+\.class$/iu.test(path), + manifests: all.filter(({ path }) => + /(?:^|\/)AndroidManifest\.xml$/u.test(path), ), + resources: all.filter(({ path }) => /(?:^|\/)resources\.arsc$/u.test(path)), + dex: all.filter(({ path }) => /(?:^|\/)[^/]+\.dex$/iu.test(path)), + jvm_classes: all.filter(({ path }) => /(?:^|\/)[^/]+\.class$/iu.test(path)), native_libraries: all.filter( ({ path, format }) => format === "elf" || /(?:^|\/)lib\/[^/]+\/[^/]+\.so$/iu.test(path), @@ -189,19 +180,9 @@ const runtimeFamilies = (all: readonly Component[]) => { const families = new Set< AndroidApplicationProjectionResult["runtime_families"][number] >(); - if ( - all.some( - ({ path, format }) => - format === "dex" || path.toLowerCase().endsWith(".dex"), - ) - ) + if (all.some(({ path }) => path.toLowerCase().endsWith(".dex"))) families.add("dalvik-art"); - if ( - all.some( - ({ path, format }) => - format === "jvm-class" || path.toLowerCase().endsWith(".class"), - ) - ) + if (all.some(({ path }) => path.toLowerCase().endsWith(".class"))) families.add("java-kotlin"); if (all.some(({ format }) => format === "elf")) families.add("native"); if (all.some(({ format }) => format === "javascript-bundle")) diff --git a/src/domain/apple/interfaceBuilderGraph.test.ts b/src/domain/apple/interfaceBuilderGraph.test.ts index fb1c0b9ad..bfa2c71de 100644 --- a/src/domain/apple/interfaceBuilderGraph.test.ts +++ b/src/domain/apple/interfaceBuilderGraph.test.ts @@ -552,3 +552,32 @@ it("names the archive once per evidence record", () => { evidence.filter(({ description }) => description.includes(relativePath)), ).toEqual([]); }); + +it("states each object's class, label, and ID once", () => { + const { objects } = parseInterfaceBuilderRecords({ + "com.apple.ibtool.document.objects": { + "7": { + customClass: "NSButton", + label: "Start", + objectID: "7", + nibValues: { NSContents: "Start" }, + }, + button: { class: "UIButton", title: "Build", objectID: "authored" }, + }, + }); + expect(objects).toEqual([ + expect.objectContaining({ + id: "7", + class_name: "NSButton", + name: "Start", + attributes: { nibValues: { NSContents: "Start" } }, + }), + expect.objectContaining({ + id: "button", + class_name: "UIButton", + name: "Build", + // Values that differ from the canonical fields stay as authored. + attributes: { class: "UIButton", title: "Build", objectID: "authored" }, + }), + ]); +}); diff --git a/src/domain/apple/interfaceBuilderKeyedArchive.ts b/src/domain/apple/interfaceBuilderKeyedArchive.ts index 51b446b52..c0b7f3507 100644 --- a/src/domain/apple/interfaceBuilderKeyedArchive.ts +++ b/src/domain/apple/interfaceBuilderKeyedArchive.ts @@ -71,13 +71,23 @@ export const parseInterfaceBuilderRecords = ( attributes["name"], className, ); + // class_name, name, and id already carry these values; repeating them in + // attributes doubled them on every graph node. + const carried = new Set(); + if (attributes["customClass"] === className) carried.add("customClass"); + if (attributes["label"] === label) carried.add("label"); + if (attributes["objectID"] === id) carried.add("objectID"); objects.push( objectNode.parse({ id, kind: classifyObject(className), class_name: className, name: label ?? id, - attributes: jsonSafeRecord(attributes), + attributes: jsonSafeRecord( + Object.fromEntries( + Object.entries(attributes).filter(([key]) => !carried.has(key)), + ), + ), }), ); } diff --git a/src/domain/artifactInspection.ts b/src/domain/artifactInspection.ts index f4c197b9c..19ccb9b8f 100644 --- a/src/domain/artifactInspection.ts +++ b/src/domain/artifactInspection.ts @@ -16,30 +16,12 @@ const artifactIdSchema = prefixedDigestSchema("art"); const inspectionObservationSchema = z.strictObject({ observation_id: prefixedDigestSchema("aio"), - kind: z.enum(["root-manifest", "artifact", "occurrence", "integrity"]), + kind: z.enum(["root-manifest", "integrity"]), subject: textSchema, value: jsonValueSchema, evidence_id: evidenceIdSchema, }); -const inspectionRelationshipSchema = z.strictObject({ - relationship_id: prefixedDigestSchema("air"), - relation: z.enum([ - "contains", - "extracts", - "slice-of", - "embeds", - "loads", - "maps-source", - "derived-from", - ]), - source_artifact_id: artifactIdSchema, - target_artifact_id: artifactIdSchema, - occurrence_id: prefixedDigestSchema("occ"), - logical_path: textSchema.nullable(), - evidence_id: evidenceIdSchema, -}); - const inspectionHypothesisSchema = z.strictObject({ hypothesis_id: prefixedDigestSchema("aih"), statement: textSchema, @@ -97,7 +79,6 @@ export const artifactInspectionResultSchema = z.strictObject({ ) .min(1), observations: z.array(inspectionObservationSchema), - derived_relationships: z.array(inspectionRelationshipSchema), hypotheses: z.array(inspectionHypothesisSchema), contradictions: z.array(inspectionContradictionSchema), unexplored_branches: z.array(unexploredBranchSchema), @@ -107,8 +88,6 @@ export const artifactInspectionResultSchema = z.strictObject({ substeps_completed: z.literal(1), observations_retained: z.number().int().min(0), observations_omitted: z.number().int().min(0), - relationships_retained: z.number().int().min(0), - relationships_omitted: z.number().int().min(0), hypotheses_retained: z.number().int().min(0), hypotheses_omitted: z.number().int().min(0), unexplored_branches_retained: z.number().int().min(0), @@ -136,10 +115,6 @@ export const createArtifactInspection = ( inventory, inventoryEvidence.evidence_id, ); - const relationships = allRelationships( - inventory, - inventoryEvidence.evidence_id, - ); const hypotheses = allHypotheses(inventory, inventoryEvidence.evidence_id); const nextProbes = probesFor(inventory); const branches = allUnexploredBranches( @@ -149,14 +124,12 @@ export const createArtifactInspection = ( ); const retained = { observations, - relationships, hypotheses, branches, probes: nextProbes, }; const omissions = { observations: 0, - relationships: 0, hypotheses: 0, branches: 0, probes: 0, @@ -184,7 +157,6 @@ export const createArtifactInspection = ( }, ], observations: retained.observations, - derived_relationships: retained.relationships, hypotheses: retained.hypotheses, contradictions: contradictions(inventory, inventoryEvidence.evidence_id), unexplored_branches: retained.branches, @@ -194,8 +166,6 @@ export const createArtifactInspection = ( substeps_completed: 1 as const, observations_retained: retained.observations.length, observations_omitted: omissions.observations, - relationships_retained: retained.relationships.length, - relationships_omitted: omissions.relationships, hypotheses_retained: retained.hypotheses.length, hypotheses_omitted: omissions.hypotheses, unexplored_branches_retained: retained.branches.length, @@ -224,6 +194,8 @@ const parseInventoryEvidence = (input: unknown): Evidence => { return evidence; }; +// Artifact, occurrence, and edge records stay in the nested inventory Evidence; +// restating each one here would repeat the complete graph in every result. const allObservations = ( inventory: ArtifactInventoryResult, evidenceId: string, @@ -234,12 +206,6 @@ const allObservations = ( inventory.manifest, evidenceId, ), - ...inventory.nodes.map((node) => - observation("artifact", node.artifact_id, node, evidenceId), - ), - ...inventory.occurrences.map((occurrence) => - observation("occurrence", occurrence.occurrence_id, occurrence, evidenceId), - ), ...inventory.integrity_contradictions.map((value) => observation("integrity", value.contradiction_id, value, evidenceId), ), @@ -263,25 +229,6 @@ const observation = ( }; }; -const allRelationships = ( - inventory: ArtifactInventoryResult, - evidenceId: string, -): ArtifactInspectionResult["derived_relationships"] => - inventory.edges.map((edge) => { - const semantic = { - relation: edge.relation, - source_artifact_id: edge.parent_artifact_id, - target_artifact_id: edge.child_artifact_id, - occurrence_id: edge.occurrence_id, - logical_path: edge.logical_path, - evidence_id: evidenceId, - }; - return { - relationship_id: `air_${digestCanonicalValue(semantic, "Artifact inspection")}`, - ...semantic, - }; - }); - const allHypotheses = ( inventory: ArtifactInventoryResult, evidenceId: string, diff --git a/src/domain/javascript/javascriptApplicationGraph.test.ts b/src/domain/javascript/javascriptApplicationGraph.test.ts index efedf78c0..58f127928 100644 --- a/src/domain/javascript/javascriptApplicationGraph.test.ts +++ b/src/domain/javascript/javascriptApplicationGraph.test.ts @@ -547,6 +547,21 @@ describe("JavaScript Application Graph: source maps and observations", () => { /Edge identifier/u, ); }); + + it("creates exactly the graph the public schema accepts", () => { + const graph = buildSyntheticJavaScriptApplicationGraph(); + expect(JSON.stringify(graph)).toBe( + JSON.stringify( + javascriptApplicationGraphSchema.parse(structuredClone(graph)), + ), + ); + + const { graph_id: _graphId, ...semantic } = structuredClone(graph); + firstOf(semantic.edges, "edge").edge_id = `jag_edge_${"f".repeat(64)}`; + expect(() => createJavaScriptApplicationGraph(semantic)).toThrow( + /Edge identifier/u, + ); + }); }); describe("JavaScript Application Graph: duplicate and orphan rejection", () => { diff --git a/src/domain/javascript/javascriptApplicationGraph.ts b/src/domain/javascript/javascriptApplicationGraph.ts index ba122e1f0..e16aded75 100644 --- a/src/domain/javascript/javascriptApplicationGraph.ts +++ b/src/domain/javascript/javascriptApplicationGraph.ts @@ -304,8 +304,8 @@ const checkNode = ( type GraphRecord = z.infer; -const checkGraphInvariants = ( - graph: GraphRecord, +const checkGraphContent = ( + graph: JavaScriptApplicationGraphInput, context: z.RefinementCtx, ): void => { sortedUniqueIssue( @@ -394,7 +394,13 @@ const checkGraphInvariants = ( context, ); } +}; +const checkGraphInvariants = ( + graph: GraphRecord, + context: z.RefinementCtx, +): void => { + checkGraphContent(graph, context); const { graph_id: identifier, ...semantic } = graph; if ( identifier !== @@ -411,6 +417,9 @@ const checkGraphInvariants = ( export const javascriptApplicationGraphSchema = javascriptApplicationGraphRecordSchema.superRefine(checkGraphInvariants); +const javascriptApplicationGraphContentSchema = + javascriptApplicationGraphInputSchema.superRefine(checkGraphContent); + /** Fully validated JavaScript Application Graph. */ export type JavaScriptApplicationGraph = z.infer< typeof javascriptApplicationGraphSchema @@ -482,10 +491,11 @@ export const createJavaScriptApplicationGraph = ( }, limitations: uniqueSorted(parsed.limitations), }; - return javascriptApplicationGraphSchema.parse({ - ...semantic, - graph_id: `jag_${digestCanonicalValue(semantic, "JavaScript Application Graph")}`, - }); + const content = javascriptApplicationGraphContentSchema.parse(semantic); + return { + ...content, + graph_id: `jag_${digestCanonicalValue(content, "JavaScript Application Graph")}`, + }; }; const validatedImmutableApplicationGraphs = new WeakSet(); diff --git a/src/domain/javascript/javascriptSemanticGraph.ts b/src/domain/javascript/javascriptSemanticGraph.ts index 69b3bb967..9ab2dd485 100644 --- a/src/domain/javascript/javascriptSemanticGraph.ts +++ b/src/domain/javascript/javascriptSemanticGraph.ts @@ -127,10 +127,6 @@ export const createJavaScriptSemanticGraphNode = ( ...parsed, application_node_ids: uniqueSorted(parsed.application_node_ids), evidence: evidenceContexts.intern(parsed.evidence), - identifier_strategy: { - strategy: "semantic-content-sha256" as const, - stability: "artifact-version" as const, - }, }; return javaScriptSemanticNodeSchema.parse({ ...semantic, @@ -147,10 +143,6 @@ export const createJavaScriptSemanticGraphRelation = ( const semantic = { ...parsed, evidence: evidenceContexts.intern(parsed.evidence), - identifier_strategy: { - strategy: "semantic-content-sha256" as const, - stability: "relationship-exact" as const, - }, }; return javaScriptSemanticRelationSchema.parse({ ...semantic, diff --git a/src/domain/javascript/javascriptSemanticGraphSchemas.ts b/src/domain/javascript/javascriptSemanticGraphSchemas.ts index fb03f1a4d..4048ae698 100644 --- a/src/domain/javascript/javascriptSemanticGraphSchemas.ts +++ b/src/domain/javascript/javascriptSemanticGraphSchemas.ts @@ -233,11 +233,9 @@ export const javaScriptSemanticNodeInputSchema = z.strictObject({ export const javaScriptSemanticNodeSchema = javaScriptSemanticNodeInputSchema.extend({ evidence: javaScriptSemanticEvidenceReferenceSchema, - node_id: semanticNodeIdSchema, - identifier_strategy: z.strictObject({ - strategy: z.literal("semantic-content-sha256"), - stability: z.literal("artifact-version"), - }), + node_id: semanticNodeIdSchema.describe( + "SHA-256 of the node kind and identity; stable for one artifact version.", + ), }); /** One directed semantic relationship before its ID is derived. */ @@ -254,11 +252,9 @@ export const javaScriptSemanticRelationInputSchema = z.strictObject({ export const javaScriptSemanticRelationSchema = javaScriptSemanticRelationInputSchema.extend({ evidence: javaScriptSemanticEvidenceReferenceSchema, - relation_id: semanticRelationIdSchema, - identifier_strategy: z.strictObject({ - strategy: z.literal("semantic-content-sha256"), - stability: z.literal("relationship-exact"), - }), + relation_id: semanticRelationIdSchema.describe( + "SHA-256 of the exact relation content, including its evidence reference.", + ), }); /** Reasons semantic analysis can leave an explicit frontier. */ diff --git a/src/domain/zipPackageFormat.ts b/src/domain/zipPackageFormat.ts index c45109135..39a8f86a5 100644 --- a/src/domain/zipPackageFormat.ts +++ b/src/domain/zipPackageFormat.ts @@ -22,7 +22,8 @@ export const zipPackageFormatForPath = ( const lower = path.toLowerCase(); if (lower.endsWith(".zip")) return "zip"; if (lower.endsWith(".ipa")) return "ipa"; - if (lower.endsWith(".apk") || lower.endsWith(".aab")) return "apk"; + // An Android App Bundle is a ZIP archive, not a standalone APK. + if (lower.endsWith(".apk")) return "apk"; if (lower.endsWith(".msix") || lower.endsWith(".msixbundle")) return "msix"; if (lower.endsWith(".appx") || lower.endsWith(".appxbundle")) return "appx"; return undefined; diff --git a/tests/boundary/filesystem/artifactInventoryEntryOrder.test.ts b/tests/boundary/filesystem/artifactInventoryEntryOrder.test.ts index 6d106e335..9a6fbd2f6 100644 --- a/tests/boundary/filesystem/artifactInventoryEntryOrder.test.ts +++ b/tests/boundary/filesystem/artifactInventoryEntryOrder.test.ts @@ -100,7 +100,6 @@ describe("entry-order validation and reader controls", () => { it.each([ ["pkg/data.txt", "pkg/data.txt/"], ["pkg/data.txt/", "pkg/data.txt"], - ["pkg/data.txt", "PKG/other.txt"], ["pkg/data.txt", "pkg"], ["../escape.txt"], ])("keeps invalid/colliding paths rejected for %j", async (...entries) => { @@ -112,6 +111,25 @@ describe("entry-order validation and reader controls", () => { }); }); + it("keeps case-distinct directory names separate in either entry order", async () => { + const root = await createTestTempDirectory("rea-entry-order-case-"); + const archive = join(root, "case-distinct.zip"); + const entries = ["pkg/", "pkg/data.txt", "PKG/", "PKG/other.txt"]; + const parents = { + ".": null, + pkg: ".", + "pkg/data.txt": "pkg", + PKG: ".", + "PKG/other.txt": "PKG", + }; + for (const ordered of [entries, entries.toReversed()]) { + await writeOrderedZip(archive, ordered); + expect(artifactParentPaths(await inventoryArtifact(archive))).toEqual( + parents, + ); + } + }); + it("preserves ordinary directory and ASAR structure", async () => { const root = await createTestTempDirectory("rea-entry-order-readers-"); const directory = join(root, "source"); diff --git a/tests/boundary/filesystem/clientLifecycle.test.ts b/tests/boundary/filesystem/clientLifecycle.test.ts index 88eab9953..297787a10 100644 --- a/tests/boundary/filesystem/clientLifecycle.test.ts +++ b/tests/boundary/filesystem/clientLifecycle.test.ts @@ -59,6 +59,7 @@ describe("client configuration filesystem lifecycle", () => { "antigravity", "copilot_cli", "commandcode", + "qwen_code", "vscode", "grok_build", "omp", diff --git a/tests/boundary/filesystem/qwenClientConfiguration.test.ts b/tests/boundary/filesystem/qwenClientConfiguration.test.ts new file mode 100644 index 000000000..0fae29640 --- /dev/null +++ b/tests/boundary/filesystem/qwenClientConfiguration.test.ts @@ -0,0 +1,183 @@ +import { access, mkdir, readFile, writeFile } from "node:fs/promises"; +import { join, resolve } from "node:path"; + +import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"; +import { parse as parseJsonc } from "jsonc-parser"; + +import { readClientRegistrationStatuses } from "../../../src/application/ClientRegistrationStatus.js"; +import { configureClientConfiguration } from "../../../src/application/SetupClientConfiguration.js"; +import { systemSetupHost } from "../../../src/application/SetupHost.js"; +import { supportedClients } from "../../../src/application/SupportedClients.js"; +import { systemUninstallHost } from "../../../src/application/Uninstall.js"; +import { PRODUCT_IDENTITY } from "../../../src/identity.js"; +import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js"; +import { + clearClientLocationEnvironment, + NPX_REGISTRATION_COMMAND, +} from "../../fixtures/clientEnvironment.js"; + +const command = NPX_REGISTRATION_COMMAND; + +beforeEach(clearClientLocationEnvironment); +afterEach(() => vi.unstubAllEnvs()); + +const qwenPath = ( + home: string, + platform: NodeJS.Platform, + env: Parameters[2] = {}, +) => + supportedClients(home, platform, env).find( + ({ name }) => name === "qwen_code", + ); + +describe("Qwen Code configuration paths", () => { + it("resolves the user settings file and marker directory", () => { + expect(qwenPath("/home/a", "linux")).toEqual({ + name: "qwen_code", + displayName: "Qwen Code", + configPath: join("/home/a", ".qwen", "settings.json"), + markerPath: join("/home/a", ".qwen"), + format: "json", + }); + expect(qwenPath("/Users/a", "darwin")?.configPath).toBe( + join("/Users/a", ".qwen", "settings.json"), + ); + }); + + it.each([ + ["", join("/home/a", ".qwen")], + ["~", "/home/a"], + ["~/profiles/work", join("/home/a", "profiles", "work")], + ["~\\profiles\\work", join("/home/a", "profiles", "work")], + ["profiles/work", resolve("profiles/work")], + [" profile ", resolve(" profile ")], + ["$PROFILE", resolve("$PROFILE")], + ])("matches Qwen's QWEN_HOME resolution for %j", (override, directory) => { + vi.stubEnv("QWEN_HOME", override); + const client = qwenPath("/home/a", process.platform, { + QWEN_HOME: override, + }); + expect( + supportedClients("/home/a").find(({ name }) => name === "qwen_code"), + ).toEqual(client); + expect(client).toMatchObject({ + configPath: join(directory, "settings.json"), + markerPath: directory, + }); + }); +}); + +describe("Qwen Code client lifecycle", () => { + const qwenClient = async () => { + const home = await createTestTempDirectory("rea-qwen-"); + const client = qwenPath(home, process.platform, {}); + if (client?.markerPath === undefined) + throw new Error("missing Qwen Code client"); + await mkdir(client.markerPath, { recursive: true }); + return { home, client }; + }; + + it("registers rea in mcpServers while preserving unrelated settings and comments", async () => { + const { home, client } = await qwenClient(); + await writeFile( + client.configPath, + `{ + // Keep this note. + "model": { "name": "qwen3-coder-plus" }, + "mcpServers": { "other": { "command": "other-server" } } +} +`, + ); + expect( + (await configureClientConfiguration(client, {}, command)).status, + ).toBe("configured"); + const text = await readFile(client.configPath, "utf8"); + expect(text).toContain("// Keep this note."); + expect(parseJsonc(text)).toEqual({ + model: { name: "qwen3-coder-plus" }, + mcpServers: { + other: { command: "other-server" }, + rea: { command: "npx", args: command.slice(1) }, + }, + }); + expect(await configureClientConfiguration(client, {}, command)).toEqual({ + status: "unchanged", + }); + expect((await systemUninstallHost(home).removeClient(client)).status).toBe( + "removed", + ); + const removedText = await readFile(client.configPath, "utf8"); + const removed = parseJsonc(removedText); + expect(removed).toEqual({ + model: { name: "qwen3-coder-plus" }, + mcpServers: { other: { command: "other-server" } }, + }); + expect(removedText).toContain("// Keep this note."); + }); + + it("reports a package-runner registration as aligned", async () => { + const { home, client } = await qwenClient(); + expect( + (await configureClientConfiguration(client, {}, command)).status, + ).toBe("configured"); + const entry = resolve("scripts/rea.mjs"); + const status = ( + await readClientRegistrationStatuses(home, entry, { environment: {} }) + ).find(({ client: name }) => name === "qwen_code"); + expect(status).toMatchObject({ state: "aligned" }); + }); +}); + +describe("Qwen Code profile setup", () => { + it.each([false, true])( + "installs and removes the shared skill with a custom home: %s", + async (customHome) => { + const home = await createTestTempDirectory("rea-qwen-profile-"); + const profile = join(home, customHome ? "qwen-profile" : ".qwen"); + const environment = { + HOME: home, + USERPROFILE: home, + ...(customHome ? { QWEN_HOME: profile } : {}), + }; + await mkdir(profile); + const setup = systemSetupHost(undefined, environment); + const client = (await setup.detectedClients()).find( + ({ name }) => name === "qwen_code", + ); + if (client === undefined) throw new Error("missing Qwen Code client"); + expect(client.configPath).toBe(join(profile, "settings.json")); + expect(await setup.configureClient(client, {}, command)).toEqual({ + status: "configured", + }); + expect(await setup.installSkill([client.name])).toBe("installed"); + const skill = join(home, ".agents", "skills", PRODUCT_IDENTITY.skillName); + expect(await readFile(join(skill, "SKILL.md"), "utf8")).toContain( + "reverse-engineer-anything", + ); + expect(await setup.clientNeedsConfigure(client, {}, command)).toBe(false); + expect(await setup.skillNeedsInstall([client.name])).toBe(false); + expect(await setup.installSkill([client.name])).toBe("unchanged"); + expect( + ( + await readClientRegistrationStatuses( + home, + resolve("scripts/rea.mjs"), + { environment }, + ) + ).find(({ client: name }) => name === "qwen_code"), + ).toMatchObject({ state: "aligned" }); + const uninstall = systemUninstallHost(home, undefined, environment); + const registered = (await uninstall.clients()).find( + ({ name }) => name === "qwen_code", + ); + expect(registered).toEqual(client); + expect((await uninstall.removeClient(client)).status).toBe("removed"); + expect((await uninstall.removeSkill()).status).toBe("removed"); + await expect(access(skill)).rejects.toMatchObject({ code: "ENOENT" }); + if (customHome) + await expect(access(join(home, ".qwen"))).rejects.toMatchObject({ + code: "ENOENT", + }); + }, + ); +}); diff --git a/tests/boundary/mcp/toolSchemaValidity.test.ts b/tests/boundary/mcp/toolSchemaValidity.test.ts index 1540964fd..993a5e00f 100644 --- a/tests/boundary/mcp/toolSchemaValidity.test.ts +++ b/tests/boundary/mcp/toolSchemaValidity.test.ts @@ -117,7 +117,7 @@ function expectKnownAuthorityHints(tools: readonly ToolSchemas[]): void { expect(advertised.get(name), name).toMatchObject({ annotations }); } -function expectRecursivePropertyDescriptions( +function expectMeaningfulPropertyDescriptions( schema: unknown, path: string, root: unknown = schema, @@ -125,7 +125,7 @@ function expectRecursivePropertyDescriptions( ): void { if (!isRecord(schema)) return; if (typeof schema.$ref === "string" && !active.has(schema.$ref)) - expectRecursivePropertyDescriptions( + expectMeaningfulPropertyDescriptions( resolveReference(root, schema.$ref), `${path}.${schema.$ref}`, root, @@ -133,10 +133,13 @@ function expectRecursivePropertyDescriptions( ); if (isRecord(schema.properties)) { for (const [property, child] of Object.entries(schema.properties)) { - expect(child, `${path}.${property}`).toMatchObject({ - description: expect.any(String), - }); - expectRecursivePropertyDescriptions( + // Omit a description rather than restate the property name. + const description = isRecord(child) ? child.description : undefined; + if (description !== undefined) + expect(description, `${path}.${property}`).toEqual( + expect.not.stringMatching(/^(?:Value for .*)?$/u), + ); + expectMeaningfulPropertyDescriptions( child, `${path}.${property}`, root, @@ -147,12 +150,12 @@ function expectRecursivePropertyDescriptions( for (const key of ["items", "additionalProperties"]) if (schema[key] !== undefined) - expectRecursivePropertyDescriptions(schema[key], path, root, active); + expectMeaningfulPropertyDescriptions(schema[key], path, root, active); for (const key of ["allOf", "anyOf", "oneOf", "prefixItems"]) { const children = schema[key]; if (Array.isArray(children)) children.forEach((child: unknown, index: number) => - expectRecursivePropertyDescriptions( + expectMeaningfulPropertyDescriptions( child, `${path}.${key}[${index}]`, root, @@ -358,7 +361,7 @@ describe("MCP JSON Schema validity", () => { contract.inputSchema.safeParse(example.input).success, `${contract.name}: ${example.title}`, ).toBe(true); - expectRecursivePropertyDescriptions(tool?.inputSchema, contract.name); + expectMeaningfulPropertyDescriptions(tool?.inputSchema, contract.name); } } finally { await Promise.allSettled([client.close(), server.close()]); diff --git a/tests/boundary/providers/native/artifactProvider.archiveInventory.test.ts b/tests/boundary/providers/native/artifactProvider.archiveInventory.test.ts index 06fe70527..2da4a98f4 100644 --- a/tests/boundary/providers/native/artifactProvider.archiveInventory.test.ts +++ b/tests/boundary/providers/native/artifactProvider.archiveInventory.test.ts @@ -60,6 +60,15 @@ describe("artifact archive inventory", () => { expect(inspectionEvidence.evidence_links).toEqual( inspection.evidence_links, ); + // The graph is delivered once, in the nested inventory Evidence. + expect( + artifactInventoryResultSchema.parse( + inspection.substeps[0]?.evidence.normalized_result, + ).occurrences, + ).toHaveLength(3); + expect(inspection.observations.map(({ kind }) => kind)).toEqual([ + "root-manifest", + ]); const cancellation = new AbortController(); cancellation.abort(); await expect( diff --git a/tests/boundary/providers/native/artifactProvider.archiveSafety.test.ts b/tests/boundary/providers/native/artifactProvider.archiveSafety.test.ts index 368d029dc..8e8ff883e 100644 --- a/tests/boundary/providers/native/artifactProvider.archiveSafety.test.ts +++ b/tests/boundary/providers/native/artifactProvider.archiveSafety.test.ts @@ -1,4 +1,4 @@ -import { readFile, writeFile } from "node:fs/promises"; +import { lstat, readFile, writeFile } from "node:fs/promises"; import { join } from "node:path"; import { TextReader, Uint8ArrayWriter, ZipWriter } from "@zip.js/zip.js"; @@ -9,6 +9,8 @@ import { createTestTempDirectory } from "../../../fixtures/temporaryDirectory.js import { createDirectAnalysis } from "../../../../src/composition/directAnalysis.js"; import { ArtifactPathRegistry, + destinationCaseCollisionMessage, + ENGLISH_UNICODE_CASE_INVENTORY_LIMITATION, normalizeArtifactPath, } from "../../../../src/artifacts/ArtifactPaths.js"; import { ArtifactProvider } from "../../../../src/artifacts/ArtifactProvider.js"; @@ -18,6 +20,7 @@ import { artifactInventoryResultSchema } from "../../../../src/domain/artifactGr import type { BinaryTarget } from "../../../../src/domain/binaryTargetTypes.js"; import { parseBinaryTarget } from "../../../../src/application/BinaryTargetResolver.js"; import { parseEvidence } from "../../../../src/domain/evidence.js"; +import { projectAnalysisError } from "../../../../src/domain/analysisErrorProjection.js"; const { runProviderAnalysis } = createDirectAnalysis({}); @@ -95,17 +98,8 @@ describe("artifact archive safety", () => { }); const registry = new ArtifactPathRegistry(); registry.add("A.js", "file"); - let collisionError: unknown; - try { - registry.add("a.js", "file"); - } catch (error: unknown) { - collisionError = error; - } - expect(collisionError).toBeInstanceOf(ArtifactReaderFailure); - expect(collisionError).toMatchObject({ - reason: "path", - message: "Artifact path collision: a.js differs only in case from A.js", - }); + expect(() => registry.add("a.js", "file")).not.toThrow(); + expect(() => registry.add("A.js", "file")).toThrow(ArtifactReaderFailure); for (const [filePath, childPath] of [ ["Foo", "foo/bar"], @@ -113,11 +107,15 @@ describe("artifact archive safety", () => { ] as const) { const casePrefixRegistry = new ArtifactPathRegistry(); casePrefixRegistry.add(filePath, "file"); - expect(() => casePrefixRegistry.add(childPath, "file")).toThrow( - ArtifactReaderFailure, - ); + expect(() => casePrefixRegistry.add(childPath, "file")).not.toThrow(); } + const exactPrefixRegistry = new ArtifactPathRegistry(); + exactPrefixRegistry.add("Foo", "file"); + expect(() => exactPrefixRegistry.add("Foo/bar", "file")).toThrow( + ArtifactReaderFailure, + ); + const sameDirectoryRegistry = new ArtifactPathRegistry(); sameDirectoryRegistry.add("Foo/one.js", "file"); expect(() => sameDirectoryRegistry.add("Foo/two.js", "file")).not.toThrow(); @@ -126,8 +124,64 @@ describe("artifact archive safety", () => { caseVariantDirectoryRegistry.add("Foo/one.js", "file"); expect(() => caseVariantDirectoryRegistry.add("foo/two.js", "file"), - ).toThrow(ArtifactReaderFailure); + ).not.toThrow(); }); + + it("names a destination case collision from the directory listing", () => { + expect( + destinationCaseCollisionMessage("res/2f.xml", ["2F.xml", "other.xml"]), + ).toBe( + "Destination filesystem cannot store both res/2f.xml and res/2F.xml; inventory retains both logical names.", + ); + expect( + destinationCaseCollisionMessage("res/2f.xml", ["2f.xml", "2F.xml"]), + ).toBeUndefined(); + }); +}); + +it("extracts case-distinct names only when the destination can preserve them", async () => { + const root = await createTestTempDirectory("rea-case-distinct-zip-"); + await writeFile(join(root, "case-probe"), "lower"); + await writeFile(join(root, "CASE-PROBE"), "upper"); + const caseSensitive = + (await readFile(join(root, "case-probe"), "utf8")) === "lower"; + const path = join(root, "fixture.zip"); + const writer = new ZipWriter(new Uint8ArrayWriter()); + await writer.add("Main.js", new TextReader("upper")); + await writer.add("main.js", new TextReader("lower")); + await writeFile(path, await writer.close()); + const parsed = await parseBinaryTarget(path); + if (!parsed.ok) throw parsed.error; + const result = await inventory(parsed.value); + expect(result.limitations).toContain( + ENGLISH_UNICODE_CASE_INVENTORY_LIMITATION, + ); + expect( + result.occurrences + .filter(({ logical_path }) => logical_path !== ".") + .map(({ logical_path }) => logical_path) + .sort(), + ).toEqual(["Main.js", "main.js"]); + const output = join(root, "output"); + const extracted = await new ArtifactProvider(process.env) + .createClient(parsed.value) + .execute( + "extract_artifact", + artifactExtractionExecutionSchema.parse({ output_root: output }), + ); + if (!caseSensitive) { + if (extracted.ok) throw new Error("expected a destination case collision"); + expect(projectAnalysisError(extracted.error)).toMatchObject({ + message: + "Destination filesystem cannot store both main.js and Main.js; inventory retains both logical names.", + details: { operation: "extract_artifact", reason: "path" }, + }); + await expect(lstat(output)).rejects.toMatchObject({ code: "ENOENT" }); + return; + } + if (!extracted.ok) throw extracted.error; + expect(await readFile(join(output, "Main.js"), "utf8")).toBe("upper"); + expect(await readFile(join(output, "main.js"), "utf8")).toBe("lower"); }); const inventory = async (targetValue: BinaryTarget) => { const result = await new ArtifactProvider(process.env) diff --git a/tests/boundary/providers/native/artifactProvider.failureDetail.test.ts b/tests/boundary/providers/native/artifactProvider.failureDetail.test.ts index dcec03c93..e2b2fd385 100644 --- a/tests/boundary/providers/native/artifactProvider.failureDetail.test.ts +++ b/tests/boundary/providers/native/artifactProvider.failureDetail.test.ts @@ -6,30 +6,73 @@ import { writeOrderedZip } from "../../../fixtures/artifactEntryOrder.js"; import { createTestTempDirectory } from "../../../fixtures/temporaryDirectory.js"; import { parseBinaryTarget } from "../../../../src/application/BinaryTargetResolver.js"; +import { + destinationCaseCollisionMessage, + ENGLISH_UNICODE_CASE_INVENTORY_LIMITATION, +} from "../../../../src/artifacts/ArtifactPaths.js"; import { ArtifactProvider } from "../../../../src/artifacts/ArtifactProvider.js"; +import { artifactInventoryResultSchema } from "../../../../src/domain/artifactGraph.js"; +import { ArtifactOperationError } from "../../../../src/domain/artifactOperationError.js"; import { projectAnalysisError } from "../../../../src/domain/analysisErrorProjection.js"; -it.each([ - [ - ["Main.js", "main.js"], - "Artifact path collision: main.js differs only in case from Main.js", - ], - [["a\\b.js"], 'Artifact path is absolute or unsafe: "a\\\\b.js"'], -] as const)( - "reports the failed path constraint for %j", - async (entries, detail) => { - const root = await createTestTempDirectory("rea-artifact-detail-"); - const path = join(root, "fixture.zip"); - await writeOrderedZip(path, entries); - const target = await parseBinaryTarget(path); - if (!target.ok) throw new Error("expected a ZIP target"); - const result = await new ArtifactProvider(process.env) - .createClient(target.value) - .execute("inventory_artifact", {}); - if (result.ok) throw new Error("expected the path constraint to fail"); - expect(projectAnalysisError(result.error)).toMatchObject({ - code: "artifact_operation_failed", - details: { operation: "inventory_artifact", reason: "path", detail }, - }); - }, -); +it("keeps logical names that differ only in case and records the limitation", async () => { + const root = await createTestTempDirectory("rea-artifact-detail-"); + const path = join(root, "fixture.zip"); + await writeOrderedZip(path, ["Main.js", "main.js"]); + const target = await parseBinaryTarget(path); + if (!target.ok) throw new Error("expected a ZIP target"); + const result = await new ArtifactProvider(process.env) + .createClient(target.value) + .execute("inventory_artifact", {}); + if (!result.ok) throw result.error; + const inventory = artifactInventoryResultSchema.parse(result.value.result); + expect(inventory.occurrences.map(({ logical_path }) => logical_path)).toEqual( + expect.arrayContaining(["Main.js", "main.js"]), + ); + expect(inventory.limitations).toContain( + ENGLISH_UNICODE_CASE_INVENTORY_LIMITATION, + ); + for (const logicalPath of ["Main.js", "main.js"]) { + expect( + inventory.occurrences + .find((occurrence) => occurrence.logical_path === logicalPath) + ?.limitations.join("\n"), + ).toContain("English (en-US) Unicode case folding"); + } +}); + +it("reports an unsafe path as a hostile artifact path", async () => { + const root = await createTestTempDirectory("rea-artifact-detail-"); + const path = join(root, "fixture.zip"); + await writeOrderedZip(path, ["a\\b.js"]); + const target = await parseBinaryTarget(path); + if (!target.ok) throw new Error("expected a ZIP target"); + const result = await new ArtifactProvider(process.env) + .createClient(target.value) + .execute("inventory_artifact", {}); + if (result.ok) throw new Error("expected the path constraint to fail"); + expect(projectAnalysisError(result.error)).toMatchObject({ + code: "artifact_operation_failed", + message: + "Artifact contains an unsafe internal path. Inspect the reported path and correct the artifact before retrying.", + details: { + operation: "inventory_artifact", + reason: "path", + detail: 'Artifact path is absolute or unsafe: "a\\\\b.js"', + }, + }); +}); + +it("returns a destination case collision as the user-facing path message", () => { + const detail = destinationCaseCollisionMessage("res/2f.xml", ["2F.xml"]); + expect(detail).toEqual(expect.any(String)); + if (detail === undefined) return; + expect( + projectAnalysisError( + new ArtifactOperationError("extract_artifact", "path", undefined, detail), + ), + ).toMatchObject({ + message: detail, + details: { reason: "path", detail }, + }); +}); diff --git a/tests/composition/android/application.test.ts b/tests/composition/android/application.test.ts index 082a706b0..3529dc4e8 100644 --- a/tests/composition/android/application.test.ts +++ b/tests/composition/android/application.test.ts @@ -86,6 +86,12 @@ describe("Android application projection", () => { expect(left.bridge_candidates).toEqual([ expect.objectContaining({ basis: "react-native-convention" }), ]); + expect(left.limitations).toEqual( + expect.arrayContaining([ + "A bridge basis is inferred from the native path and is repeated for every managed component.", + "ZIP inventory cannot see the APK Signing Block, so an empty signing array does not mean the APK is unsigned.", + ]), + ); expect(JSON.stringify(left)).not.toContain("opaque signing"); }); @@ -146,6 +152,45 @@ describe("Android application projection", () => { }); }); +describe("case-distinct APK resource names", () => { + it("inventories and projects an APK whose resource names differ only in case", async () => { + const root = await createTestTempDirectory("rea-android-case-"); + const path = join(root, "Shrunk.apk"); + const writer = new ZipWriter(new Uint8ArrayWriter()); + await writer.add( + "AndroidManifest.xml", + new Uint8ArrayReader(Uint8Array.from([3, 0, 8, 0])), + ); + await writer.add( + "classes.dex", + new Uint8ArrayReader( + Uint8Array.from([0x64, 0x65, 0x78, 0x0a, 0x30, 0x33, 0x35, 0]), + ), + ); + await writer.add("res/2F.xml", new TextReader("upper")); + await writer.add("res/2f.xml", new TextReader("lower")); + await writeFile(path, await writer.close()); + + const inventory = parseEvidence( + await runProviderAnalysis(path, "inventory_artifact", {}), + ); + const result = projectAndroidApplicationEvidence({ + inventory_evidence: [inventory], + }); + const projection = androidApplicationProjectionResultSchema.parse( + requireSuccessfulProjection(result).normalized_result, + ); + expect(projection.root_format).toBe("apk"); + expect(projection.components.manifests).toHaveLength(1); + expect(projection.components.dex).toHaveLength(1); + expect(JSON.stringify(inventory)).toContain("res/2F.xml"); + expect(JSON.stringify(inventory)).toContain("res/2f.xml"); + expect(JSON.stringify(inventory)).toContain( + "English (en-US) Unicode case folding", + ); + }); +}); + describe("Android bridge candidate coverage", () => { it("returns every component and bridge candidate from the inventory", async () => { const root = await createTestTempDirectory("rea-android-complete-"); diff --git a/tests/fixtures/android/jadx.ts b/tests/fixtures/android/jadx.ts index c82469039..8e8ac4884 100644 --- a/tests/fixtures/android/jadx.ts +++ b/tests/fixtures/android/jadx.ts @@ -74,10 +74,11 @@ export const createJadxProtocolFixture = async ( ownsProcessLifetime: true, cleanup: () => cleanupOwnedProcessGroup(spawned.ownership), }); - const stopped = await supervisor.stop(); - expect(stopped, JSON.stringify(stopped)).not.toMatchObject({ - status: "incomplete", - }); + // The SDK can close the child transport before process exit settles. + // Retry retained ownership without bypassing the supervisor's proof. + await expect + .poll(() => supervisor.stop(), { timeout: 5_000 }) + .not.toMatchObject({ status: "incomplete" }); if (options.cwd !== undefined) await rm(options.cwd, { recursive: true, force: true }); }); diff --git a/tests/fixtures/clientEnvironment.ts b/tests/fixtures/clientEnvironment.ts index 79961e49d..0db5a69ec 100644 --- a/tests/fixtures/clientEnvironment.ts +++ b/tests/fixtures/clientEnvironment.ts @@ -22,6 +22,7 @@ export const CLIENT_LOCATION_ENVIRONMENT = [ "PI_CODING_AGENT_DIR", "PI_CONFIG_DIR", "PI_PROFILE", + "QWEN_HOME", "SAND_DATA_ROOT", "XDG_CONFIG_HOME", ] as const; diff --git a/website/AGENTS.md b/website/AGENTS.md index d96ab2b62..e97181ec6 100644 --- a/website/AGENTS.md +++ b/website/AGENTS.md @@ -14,6 +14,11 @@ for asset notes and the source of case-study claims. case study focused on one question, with further examples in native details. - Show what REA returns and how the agent uses it. Preserve source attribution, target identity and verification scope. +- Blog articles teach methods, decisions and workflows through concrete cases. + Lead with the takeaway. Essays can develop a personal argument through + experience; figures are optional and should add an explanation the prose + needs. Use engine details only when they explain a decision. Label working + outlines and mark them `noindex`. - Keep personal paths, account data, credentials and raw captures outside the public assets. Use generic inputs and label shortened display paths. - Follow the same clear writing standard in search titles and descriptions. diff --git a/website/README.md b/website/README.md index 2005c633d..93faf5e59 100644 --- a/website/README.md +++ b/website/README.md @@ -42,6 +42,8 @@ HTML files. No URL list needs to be maintained. - `public/examples/dino-lab/index.html`: adjustable-speed mini-game, recovered rule, original-game check and browser analysis steps. - `public/first-investigation/index.html`: a guided Notes export investigation, from setup to a checked CSV prediction. - `public/showcase/index.html`: the case-study index. +- `public/blog/index.html`: articles about reconstruction methods, ports, mods and reverse engineering. +- `public/blog/touhou-reconstruction/index.html`: a personal essay about agent-driven reconstruction, knowledge that compounds across TH08 and TH095, and the workflow's applicability to PC-98 TH04. - `public/showcase/aegis/index.html`: Aegis's Android login-code calculation, with an adjustable clock and reference checks. - `public/showcase/dx-ball/index.html`: sound-pan investigation and project status. - `public/showcase/notion/index.html`: Notion's Electron clipboard bridge and rich clipboard format. @@ -121,6 +123,19 @@ and Cloudflare's [managed robots behavior](https://developers.cloudflare.com/bot ## Content +Blog articles explain methods, decisions and ideas through project experience. +Lead with a takeaway, then develop the argument through concrete examples. +An essay can explore a change in perspective without becoming a step-by-step +guide. Add a figure when it contributes a useful explanation; an opening flow +is optional. Case studies remain focused on one inspected behavior. Keep engine +details in supporting references unless they explain a relevant decision. + +The Touhou article is an English essay. Its provenance, +quoted README passage and milestone counting rules are in +[evidence/touhou-reconstruction-method.md](evidence/touhou-reconstruction-method.md). +The completed text has normal search metadata and is included in the generated +sitemap. + Keep the copy direct and specific. Explain the task and the result before listing tool names. Setup commands and runtime requirements should match the released package. Keep the core learning path inside the site. Source, issue and evidence @@ -177,11 +192,14 @@ The homepage starts with a short installation prompt and the ordinary setup command, then explains what reverse engineering is and why someone would use it. The manual/agent comparison introduces REA’s role before the examples. Its opening links directly to Showcases and lets experienced readers skip to -the analysis guides. Case-study previews live on the Showcases page. +the analysis guides. A Blog link below that shortcut leads to reconstruction +methods and project notes. Case-study previews live on the Showcases page. The closing section offers copyable project prompts, from cloning `rea.tools` to reconstructing a game from its executable. The experienced-reader shortcut lands directly on the guide links below these prompts. -The page ends with FAQ, Discord and issue-report links under “Any questions?”. +“Any questions?” offers FAQ, Discord and issue-report links. The page ends +with “Join the community”, which links Discord and REA's X account. +Every content page's header and footer link GitHub, Discord and X. A right-side table of contents stays visible at widths of 1440px and above. On narrower screens it becomes a sticky, native disclosure; selecting a link closes the menu and focuses the destination. The homepage script follows diff --git a/website/evidence/touhou-reconstruction-method.md b/website/evidence/touhou-reconstruction-method.md new file mode 100644 index 000000000..99f3ceb33 --- /dev/null +++ b/website/evidence/touhou-reconstruction-method.md @@ -0,0 +1,175 @@ +# Touhou reconstruction method: evidence notes + +Checked on 10 October 2026. This document supports the full English essay +at `public/blog/touhou-reconstruction/index.html`. + +## Method sources + +The Factory checkpoint is +[`038b6bed`](https://github.com/N0zoM1z0/touhou-reconstruction-factory/tree/038b6bed51d273c8b3680553b6f43e1b97d19bd1). +Read its [agent autonomy](https://github.com/N0zoM1z0/touhou-reconstruction-factory/blob/038b6bed51d273c8b3680553b6f43e1b97d19bd1/docs/agent-autonomy.md), +[semantic workflow](https://github.com/N0zoM1z0/touhou-reconstruction-factory/blob/038b6bed51d273c8b3680553b6f43e1b97d19bd1/docs/semantic-reconstruction.md), +and [cross-game knowledge](https://github.com/N0zoM1z0/touhou-reconstruction-factory/blob/038b6bed51d273c8b3680553b6f43e1b97d19bd1/docs/knowledge-base.md) +documents for the underlying method: autonomous investigation, bounded changes, +empirical feedback, durable checkpoints and reusable lessons. + +The industrial-age analogy explains that method. The Factory's +software architecture also uses the Abstract Factory pattern; the article's +analogy should focus on the working process rather than that implementation. + +REA supplies analysis evidence to an agent. The historical projects also used +project-specific analysis bridges and tooling. The article should attribute +historical work to those projects and explain where REA fits in the general +workflow, without relabeling every historical tool result as REA output. + +## Collection and counting + +Public history was retrieved with `gh api`, including all pages of each +repository's default-branch commits, repository metadata and release records. +Pinned progress documents were read from the corresponding Git objects in the +local project checkouts. Every milestone SHA below appears in the public +GitHub history. No local-only revision or uncommitted change supplies a claim. + +Elapsed days are the difference between public commit **committer timestamps**, +in UTC, divided by 86,400 and rounded to two decimal places. The article uses +rounded calendar intervals. They measure recorded milestone spacing, not +compute time, person-hours or a controlled speedup benchmark. + +TH08's start is the independent continuation at `001bf3e`, not its inherited +2025 initial commit or its hosting creation date. Its imported GensokyoClub +history ends at `7ad3792`; preserve that attribution and license. TH095 starts +with zero confirmed authored functions, while using methods and infrastructure +from earlier projects. TH04 uses ReC98 as a credited source of existing work. + +The denominators change as candidates are classified and boundaries reviewed. +Read each pinned progress document's definitions. Avoid a cross-game function +rate or an executable-wide completion percentage. + +## Public history snapshot + +This is the initial 10 October research snapshot. The later TH04 phase update +below uses its own pinned checkpoint; it does not change these earlier counts. + +| Project | Default-branch commits | Commits whose subject begins `gpt-web:` | Public head | +| ------- | ---------------------: | --------------------------------------: | ------------------------------------------ | +| TH08 | 1,117 | 400 | `d2a00f4acb4a13916e07a00bd13f714961714628` | +| TH095 | 713 | 450 | `6084b36cfa90ac1720f0cba36caec3ee56fc5e1f` | +| TH04 | 957 | 623 | `33b80522f03565022dbfe4d5c8a88bdc57f566ae` | + +TH08 has 696 reachable continuation commits after the imported boundary; +its total includes the upstream history. Subject prefixes identify a recorded +workflow convention, not the number of sessions, experiments or verified +functions. Commit counts remain supporting research rather than article +headlines. + +## Milestone records + +### TH08 + +Start: [001bf3e9](https://github.com/N0zoM1z0/th08/commit/001bf3e9c91cc35b79c7a0e36b3565b86f494362) at 2026-08-13T02:23:48Z. + +| Checkpoint | UTC timestamp | Elapsed days | Recorded milestone | +| -------------------------------------------------------------------------------------------- | -------------------- | -----------: | ---------------------------------------------------------------- | +| [15caba73](https://github.com/N0zoM1z0/th08/commit/15caba73df6735fc44eac4783bc61b8d098ea830) | 2026-08-19T12:19:33Z | 6.41 | Restore clean authored build baseline | +| [da07b21c](https://github.com/N0zoM1z0/th08/commit/da07b21c8cb63230142b0a66228d6aac66555f3f) | 2026-08-24T10:43:33Z | 11.35 | Add playable Linux reconstruction port | +| [529eb1a3](https://github.com/N0zoM1z0/th08/commit/529eb1a3afa05ae36467731eaccb12e3e50e132f) | 2026-08-26T05:12:23Z | 13.12 | Link the playable TH08 Web edition | +| [5d314e21](https://github.com/N0zoM1z0/th08/commit/5d314e21ec725a0de7a38b7c735f683ad358e79f) | 2026-08-30T10:30:16Z | 17.34 | docs: publish native Linux 64-bit release | +| [bd1bae44](https://github.com/N0zoM1z0/th08/commit/bd1bae444803adfa1fc865bfe64415e9951945e6) | 2026-08-26T15:59:04Z | 13.57 | semantic: recover GameManager state and exact title registration | + +### TH095 + +Start: [8c31e6f8](https://github.com/N0zoM1z0/th095/commit/8c31e6f892d945423b7f7b439747a2bd915f82d8) at 2026-08-29T07:49:20Z. + +| Checkpoint | UTC timestamp | Elapsed days | Recorded milestone | +| --------------------------------------------------------------------------------------------- | -------------------- | -----------: | ---------------------------------------- | +| [0fa857fb](https://github.com/N0zoM1z0/th095/commit/0fa857fb2b2ec6237732e4d6be8d0a28069c9330) | 2026-09-07T05:23:52Z | 8.90 | gpt-web: match enemy movement exactly | +| [74baddfc](https://github.com/N0zoM1z0/th095/commit/74baddfccb712d1311a4d6e7487a63cd1100452e) | 2026-09-08T09:19:45Z | 10.06 | Match final ANM x87 draw functions | +| [33d46a0e](https://github.com/N0zoM1z0/th095/commit/33d46a0ee48f37060d973125a1a7632a40f8d998) | 2026-09-09T11:15:00Z | 11.14 | build: complete whole-program linkage | +| [3442dcf2](https://github.com/N0zoM1z0/th095/commit/3442dcf29d9e0b5bef384a49bd0c9ad32a711826) | 2026-09-10T07:07:28Z | 11.97 | Mark the Windows reconstruction playable | + +### TH04 + +Start: [f402cb30](https://github.com/N0zoM1z0/th04/commit/f402cb30b7e795ea4328b6de794e9eefc52c97b3) at 2026-09-06T09:54:05Z. + +| Checkpoint | UTC timestamp | Elapsed days | Recorded milestone | +| -------------------------------------------------------------------------------------------- | -------------------- | -----------: | --------------------------------------------------------------- | +| [1cd1a257](https://github.com/N0zoM1z0/th04/commit/1cd1a2577a3dbf4c17ac7a939faed5eeb81a6ebf) | 2026-09-26T21:16:40Z | 20.47 | gpt-web: accept MAIN SND_LOAD | +| [4b6a57f8](https://github.com/N0zoM1z0/th04/commit/4b6a57f8c0625da07c38258b78f5f205aae92805) | 2026-10-02T07:44:53Z | 25.91 | gpt-6.1-sol: build four native products and restore DOS handoff | + +## What the milestones establish + +- **TH08:** the 19 August progress ledger records 1,107 / 1,107 authored + functions with source. The 24 August change introduces a playable Linux + reconstruction port; the 30 August GitHub release publishes the 64-bit Linux + product. The 26 August exact ledger records 1,106 / 1,107 accepted functions. +- **TH095:** the initial ledger records zero confirmed/source-present authored + functions. The 7 September ledger records 697 source-present mappings; + 8 September records 696 exact functions out of 697 confirmed authored + functions. The 10 September checkpoint marks the Windows i386 reconstruction + playable and documents the exercised startup, gameplay and transition paths. + The remaining `Controller::GetInput` comparison is separate from playability. +- **TH04:** the 26 September ledger records 661 exact functions out of 663 + reviewed authored candidates across OP, MAIN, MAINE and ZUN. MAIN's byte + denominator covers its reviewed authored extents; other artifacts use decoded + comparisons. The 2 October checkpoint builds four standalone DOS products. + Runtime work and a modern x64 product have their own milestones; a four-product + build is not a whole-game runtime result. + +TH08's playable ports preceded later historical-platform runtime audits. The +Factory's current recommended order reflects lessons learned across those +projects; do not present that later recommendation as a process applied +perfectly from the beginning. + +## Editorial scope + +The article follows the author's change in perspective: previous manual RE, +starting agent-driven work in August 2026, TH08's continuation, TH095's reuse of +experience, and TH04's application of the method to another architecture. The +manual-RE background and August starting point are author-provided experience. +The industrial-era conclusion and the interpretation of the two contribution +models are the author's argument, supported by the described project record. +They are not a measured agent-versus-human performance ratio. + +Engine APIs, compiler flags, addresses and ledger schemas remain in references. +The full essay replaces the outline and removes its `noindex` directive; the +generated sitemap includes the article. The author approved publication on +10 October 2026, and the review labels were removed from the article and Blog hub. + +## GensokyoClub quotation and policy + +The public README was read with `gh api` on 10 October 2026. Its latest +README-changing commit is +[`e874b98e`](https://github.com/GensokyoClub/th08/blob/e874b98e210b3be1ed21c5e03d54b883eac5069c/README.md#important-notice), +dated 6 September. The notice announces a decision effective 5 September and +describes concerns about AI decompilations, ports, attribution and their effect +on the maintainers. Its stated plan is private development until completion. + +The article quotes 23 words from the passage's opening clause, followed by an +ellipsis; it paraphrases the remaining point about the psychological toll. +The excerpt was checked against the retrieved original, with whitespace +normalized for display. The linked notice supplies its context. + +The current public head is +[`f345c7de`](https://github.com/GensokyoClub/th08/tree/f345c7dee07b849f2ff93a0029ab5ede126bf12c). +Its [contribution policy](https://github.com/GensokyoClub/th08/blob/f345c7dee07b849f2ff93a0029ab5ede126bf12c/CODE_OF_CONDUCT.md) +excludes PRs created primarily using AI. The article paraphrases this policy +and discusses the different contribution models. It preserves GensokyoClub's +credit, describes the author's own manual and agent workflows, and does not +assign motives beyond what the public notice states. + +## TH04's current DOS and 64-bit phases + +The latest public main checkpoint checked for this revision is +[`bb9faca5`](https://github.com/N0zoM1z0/th04/blob/bb9faca54e2eb3c0d6c757c75d59ac29d2cd5e93/docs/RE_HANDOFF.md), +dated 10 October 2026. Its handoff records the standalone DOS products, +repaired normal and invincible variants, and the user's manual Windows +observations of complete Normal routes, endings and saves. It identifies +resumed x64 work as the current phase on the separate `port/modern-64` branch. +That branch was publicly present at `1ffc53a61e1e82409adb40c7658d4ee2f3a21916` +when inspected with `gh`. + +The essay's first-person DOS testing statement describes those maintainer +observations, also confirmed by the author in this revision request. It is +not attributed to a new automated website test or a fresh runtime replay. +The 64-bit port is described as in development. The earlier four-product build +and function-count milestones retain their historical scopes. diff --git a/website/figures.md b/website/figures.md index 72a32ac61..534d8ac24 100644 --- a/website/figures.md +++ b/website/figures.md @@ -94,6 +94,14 @@ APK's selected Java and an explanatory summary. APK, methods, source attribution and reconstruction checks. Keep the block formula, default settings, slider outputs and reference cases consistent. +## Touhou reconstruction essay + +The essay opens with a takeaway and develops its argument in prose. It has no +opening flow figure. Project dates and the attributed README excerpt support +the narrative; detailed milestone references are available in a native +disclosure. Their provenance is documented in +[evidence/touhou-reconstruction-method.md](evidence/touhou-reconstruction-method.md). + ## Original image generation prompts ## Prompt 1 diff --git a/website/public/assets/styles.css b/website/public/assets/styles.css index fb7093b9e..542baf892 100644 --- a/website/public/assets/styles.css +++ b/website/public/assets/styles.css @@ -163,10 +163,12 @@ section[id] { white-space: nowrap; } .navigation a:hover, -.navigation a[aria-current="page"] { +.navigation a[aria-current="page"], +.navigation a[aria-current="true"] { color: var(--ink); } -.navigation a[aria-current="page"] { +.navigation a[aria-current="page"], +.navigation a[aria-current="true"] { text-decoration: underline; } .navigation .nav-start { @@ -178,7 +180,8 @@ section[id] { gap: 8px; } .nav-github, -.nav-discord { +.nav-discord, +.nav-x { display: inline-flex; align-items: center; justify-content: center; @@ -192,12 +195,14 @@ section[id] { text-decoration: none; } .nav-github:hover, -.nav-discord:hover { +.nav-discord:hover, +.nav-x:hover { background: var(--soft); color: var(--ink); } .nav-github svg, -.nav-discord svg { +.nav-discord svg, +.nav-x svg { display: block; flex-shrink: 0; } @@ -675,6 +680,27 @@ pre code { .article .stats dd { color: var(--muted); } +.blog-takeaway { + max-width: 730px; + margin-top: 1.8rem; + padding-left: 1.25rem; + border-left: 2px solid var(--accent); +} +.blog-takeaway .eyebrow { + color: var(--accent); +} +.blog-takeaway p:last-child { + margin-bottom: 0; +} +.blog-article blockquote { + margin: 1.6rem 0; + padding-left: 1.25rem; + border-left: 2px solid var(--line); + font-size: 1.05rem; +} +.blog-article blockquote p { + color: var(--ink); +} .caption { font-size: 0.875rem; color: var(--muted); @@ -1372,6 +1398,9 @@ pre code { .home-expert-shortcut a { white-space: nowrap; } +.home-expert-shortcut + .home-expert-shortcut { + margin-top: 0.5rem; +} @media (min-width: 1440px) { .home-page-nav { position: fixed; @@ -2117,6 +2146,9 @@ pre code { right: calc(1rem + env(safe-area-inset-right, 0px)); bottom: calc(1rem + env(safe-area-inset-bottom, 0px)); } + .site-footer { + padding-bottom: calc(2rem + 44px + env(safe-area-inset-bottom, 0px)); + } .container { width: calc(100% - 2.5rem); } diff --git a/website/public/blog/index.html b/website/public/blog/index.html new file mode 100644 index 000000000..90bf245ad --- /dev/null +++ b/website/public/blog/index.html @@ -0,0 +1,174 @@ + + + + + + REA Blog — Thoughts on Reverse Engineering + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+ +

Blog.

+

Thoughts on reverse engineering.

+
+
+
+

Game reconstruction
Essay

+
+

The industrial revolution in reverse engineering

+

+ What manual RE and reconstructing TH08, TH095 and TH04 taught me + about agents, verification and knowledge that compounds across + projects. +

+ Read the article +

By N0zoM1z0 · 10 October 2026

+
+
+
+
+ +

+ + Top + + + diff --git a/website/public/blog/touhou-reconstruction/index.html b/website/public/blog/touhou-reconstruction/index.html new file mode 100644 index 000000000..2afa3f1d9 --- /dev/null +++ b/website/public/blog/touhou-reconstruction/index.html @@ -0,0 +1,669 @@ + + + + + + Reverse Engineering’s Industrial Revolution — REA + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+ +

Essay · Touhou reconstruction

+

The industrial revolution in reverse engineering.

+

By N0zoM1z0 · 10 October 2026

+
+

Takeaway

+

+ Agent-driven reverse engineering changes how the work can + accumulate. + An investigation can become a checked result, a failed experiment + can become a reusable test, and one project's experience can become + the next project's starting point. +

+

+ Reconstructing TH08, TH095 and TH04 convinced me that RE is entering + its industrial age. +

+
+
+
+ +
+
+

What changed in August 2026

+

+ Before this reconstruction work, I had done a lot of manual + reverse engineering. The familiar loop was to inspect + instructions, follow references, form an explanation, test it, and + revise it. Progress depended on how much of that loop I could + carry out myself, and how well I could keep the growing picture in + my head. +

+

+ In August 2026, I began exploring agent-driven RE seriously. I + wanted to see how far an agent could go when it had access to the + program, ordinary development tools, and feedback from real + checks. Touhou reconstruction became the place where I could + pursue that question through a substantial project. +

+

+ The early progress was startling. Tasks that I would have budgeted + around my own time and attention could keep moving while the agent + investigated, tried source formulations, compiled them, and + examined the result. My work increasingly involved choosing the + direction, improving the tools and checks, and deciding what the + accumulated evidence meant. +

+

+ That changed which projects seemed reasonable to attempt. A large + executable began to look like a collection of investigations we + could organize and keep advancing. Recovering source, making it + readable, and eventually using it for a port became parts of an + achievable project. +

+

+ The numbers first got my attention. The more interesting question + was what made that progress possible—and whether the process could + improve as it worked. +

+
+
+

TH08: building on human work

+

+ TH08, Imperishable + Night, began for me as a continuation of + GensokyoClub's reconstruction. Their public work already contained source, names, build + knowledge and a contribution history. It gave the continuation a + foundation. That foundation matters when discussing everything + that followed. +

+

+ I preserved the imported history through the public 10 August + checkpoint. The independent continuation began on 13 August. By 19 + August, the project's ledger recorded source for all 1,107 + identified game functions. A playable Linux reconstruction port + was committed on 24 August, roughly eleven days after the + continuation began. A Web edition followed, then a native Linux + 64-bit release on 30 August. +

+

+ Those dates made the change in pace tangible. What interested me + most was that the work extended beyond producing plausible C++. It + included comparisons, build repairs, recovery of meaningful names + and types, and the work needed to turn reconstructed pieces into a + program people could run. +

+

+ This also exposed the importance of feedback. A function can look + correct and compile correctly while its place in the whole program + is wrong. Two functions can accidentally use separate copies of + state that should be shared. A port can hide a problem that + appears when the historical build runs. Each failure adds + something useful if we can reproduce it and keep the check. +

+

+ Over time, more of the project lived outside any one conversation: + source, evidence, comparison scripts, regression tests, and + focused notes about what to do next. A fresh agent could inspect + that state and continue. + The repository was becoming the project's working + memory. +

+

+ A successful batch gave us both recovered code and a better + environment for the next batch. +

+
+
+

Two different ideas of reconstruction

+

+ There is a real disagreement around this work, and GensokyoClub's + public README expresses it directly. Its notice announces that + further development will happen privately until completion. One + passage reads: +

+
+

+ The rise of grifters (AI decompilations and ports) in this space + taking from our work paints a bad image for future decompilation + efforts… +

+
+

+ The notice also describes the psychological toll on the + maintainers. Their contribution policy excludes pull requests + produced primarily with AI. These are people putting their free + time into difficult work, and their published work helped make my + continuation possible. That effort deserves respect. The + disagreement I want to discuss concerns how reconstruction is done + and how contributions are evaluated. +

+

+ In the manual RE workflow I knew, expertise and execution were + closely connected. The person developing an understanding of a + function also carried out much of the investigation and + reconstruction. Personal knowledge, careful authorship and trust + in contributors were central to the project. +

+

+ Manual projects already preserve knowledge in code, build tools + and tests. What changed for me was having an agent use that + accumulated knowledge to investigate, implement and test the next + change. +

+

+ My continuation takes an agent-led approach. Agents carry out the + new engineering; the human sets direction and release decisions. + Understanding has to be retained in source and evidence, while + comparisons, builds and runtime experiments test the results. A + contribution can be useful even when an agent performed most of + the work, provided its reasoning and result can be examined. +

+

+ This creates a difficult transition for communities built around + the earlier model. Open source makes existing work available for + continuation, and agents can change the pace of that continuation + dramatically. Years of careful work may become the starting point + for something that moves much faster. Recognition, provenance and + the standards for accepting new work become more important in that + situation. +

+

+ The industrial analogy helps me make sense of the disagreement. A + craft contains skill, judgment and accumulated knowledge. When + machinery changes how execution happens, those skills find new + roles in designing the process, detecting defects and deciding + what should be produced. Different communities can reasonably + choose different ways to organize that work. +

+

+ My choice is to develop this continuation openly, preserve its + sources and history, and make the new work reviewable. The + technical question I want to pursue is how far agent-driven + reconstruction can go with strong evidence and a process that + keeps improving. +

+

+ Source: GensokyoClub's + README notice, checked on 10 October 2026, and its + contribution policy. The quotation is a shortened excerpt. TH08's + credits and provenance + record the continuation boundary. +

+
+
+

TH095: experience starts compounding

+

+ TH095, Shoot the + Bullet, made the accumulated experience much easier to see. Its + new repository began on 29 August with zero confirmed game + functions in the reconstruction ledger. It already had a way to + work: identify the target, bring up analysis and comparison tools, + keep tasks manageable, preserve evidence, and leave checkpoints + another session could use. +

+

+ By 7 September, all 697 identified game functions had source. By 8 + September, 696 of them were accepted as exact comparisons. The + whole program linked on 9 September, and the Windows i386 + reconstruction was marked playable on 10 September, about twelve + days after initialization. +

+

+ I found this more exciting than the first project's speed. TH095 + began with the process and experience developed during TH08 and + the other reconstruction work. We could approach a new target with + better questions, better checks and fewer assumptions left + implicit. +

+

+ For example, matching individual functions had taught us to pay + attention to the assembled program early. Does the source build + together? Do different parts of the game use the same state? Does + the reconstructed executable survive the transitions we need to + exercise? Those questions became part of how the next project was + organized. +

+

+ This is what I mean by knowledge compounding. A lesson that stays + in my head helps while I am present. A lesson retained as a test, + a script or a precise evidence note can help every later session. + A failure becomes valuable when the next agent can recognize it + without repeating the whole investigation. +

+

+ The method becomes part of the starting material for the next + game. + As that material improves, we can spend more attention on what is + new about the target. +

+

+ This compounds across people as well as projects. Someone joining + later can rerun a check, inspect the reason for a source choice, + or continue from a recorded unknown. The project can make use of + experience without requiring everyone to reconstruct its entire + history from scratch. +

+
+
+

TH04: the workflow survives a different architecture

+

+ TH04, Lotus Land + Story, took the work into the PC-98 DOS era. We moved from 32-bit + Windows games to a 16-bit environment with four cooperating + programs and hardware behavior that had to be understood on its + own terms. Existing + ReC98 work provided + valuable knowledge and source material here too. +

+

+ The DOS reconstruction is now working, including complete Normal + routes, endings and saves from my manual testing. The current work + is a native 64-bit port. These are distinct phases: first + establish a working reconstruction on the original platform, then + carry its behavior into a new environment. +

+

+ TH04's architecture changed the evidence we needed and the tools + used to collect it. It did not require inventing a different way + to organize the work. We could still isolate a question, inspect + the original behavior, reconstruct it, compare the result and + retain the lesson. +

+

+ The difficult parts became concrete experiments. Which part of + startup establishes a piece of state? What happens when gameplay + hands control to an ending? Which hardware effect must the + reconstructed code preserve? With observations and a useful check, + an agent could work through these questions in the same iterative + way as it did on the Windows titles. +

+

+ That is why TH04 matters to the larger argument. The process + generalized across a substantial change in platform. Architecture + determined the subject of the investigation; it did not become a + fundamental barrier to the workflow. +

+

+ The 64-bit port now gives that work another use. We can revisit + the recovered behavior, separate it from old platform services and + test new implementations against what we learned. The + reconstruction supplies both source and a growing body of + knowledge about how the game should behave. +

+

+ Project state as of 10 October 2026: + DOS reconstruction and manual testing + · + 64-bit port. The port remains in development. +

+
+
+

What makes this an industrial shift

+

+ These projects changed my view of the bottleneck. As more + investigation and implementation can be carried out by agents, the + working environment becomes increasingly important. The agent + needs access to the target, tools it can compose, feedback it can + act on, and memory that survives its current session. +

+

+ REA fits into that picture by making reverse-engineering tools + available through interfaces an agent can use. It can inspect + code, follow relationships and return evidence for the next + question. Reconstruction projects add their own compilers, + comparisons and runtime checks. Together, these give an agent a + way to move from an explanation to an experiment with an + observable result. +

+

+ Fast execution makes good feedback more valuable. A bad assumption + can produce a large amount of plausible source. A useful check can + reject that assumption early and point to the next investigation. + Improving a check improves many later decisions, which is why + tooling and verification deserve as much attention as the + recovered code. +

+

+ The work also needs continuity. A conversation can end while the + campaign continues. Its result should leave the repository in a + state another agent can understand: what changed, what was tested, + what remains uncertain, and where to continue. This lets a large + reconstruction proceed through many small experiments without + relying on one uninterrupted session. +

+

+ The + Touhou Reconstruction Factory + grew out of these experiences. It gives the projects shared ways + to retain evidence, run checks and carry useful lessons between + games. The point is to make the next investigation easier to + execute, easier to judge and easier to resume. +

+

+ This is the industrial shift I see: expertise increasingly lives + in a working process that can be reused and improved. Human + judgment shapes the process. Agents supply much of the execution. + Verified results and accumulated lessons make the process more + capable over time. +

+
+
+

The projects we can now consider

+

+ The pace matters because it changes the cost of beginning. When I + imagined doing every investigation manually, I also imagined + committing my own attention to each one. A sufficiently large + project could be interesting and still remain something I would + never realistically attempt. +

+

+ An agent with access, feedback and project memory changes that + calculation. Recovering a game, making its source readable, + building a port, or exploring a mod can become a sustained + engineering effort with a practical way to keep going. The work + still involves difficult questions, but there is now a process for + giving those questions sustained attention. +

+

+ TH08 showed me what an agent-led continuation could achieve on top + of human work. TH095 showed how the experience could carry + forward. TH04 showed that the method could cross into a very + different platform. Together, they made the industrial analogy + feel concrete. +

+

+ I now look at an unfamiliar program and ask: + what access, feedback and accumulated knowledge would let an + agent work on this reliably? +

+

+ That question opens up projects I would previously have left + alone. Each one can add code, checks and experience that make the + next more approachable. That is the future of reverse engineering + I want to keep exploring. +

+
+
+
+ Project milestones and sources +

+ The dates describe recorded project checkpoints, checked against + public GitHub history on 10 October 2026. Elapsed time is + calendar time between commits. Source presence, exact + comparisons, a build and a runtime result each name a different + milestone. +

+ +
+

+ More articles · + Explore a TH04 investigation +

+
+
+
+
+ +

+ Top + + diff --git a/website/public/examples/dino-lab/index.html b/website/public/examples/dino-lab/index.html index bfae79460..02452bda9 100644 --- a/website/public/examples/dino-lab/index.html +++ b/website/public/examples/dino-lab/index.html @@ -63,7 +63,8 @@ >Reverse Engineer Anything @@ -107,6 +108,25 @@ /> + + + @@ -383,7 +403,8 @@ SourceMIT licenseCommunity + >Join the communityFollow us on X diff --git a/website/public/faq/index.html b/website/public/faq/index.html index 60fe1183d..3ab51b9d3 100644 --- a/website/public/faq/index.html +++ b/website/public/faq/index.html @@ -63,6 +63,7 @@ >