* feat(extensions): add test failure report explorer * fix(extensions): bound report snapshots before serialization * fix(extensions): bound test report validation --------- Co-authored-by: qqqys <[email protected]>
Vitest report fixtures
These reports were generated by the repository's locked Vitest 3.2.7 in an
isolated temporary micro-project. They are not hand-written approximations of
the reporter schema. provenance.json records the actual Node version, commands,
process exits, source hashes, and raw and sanitized report hashes.
From the repository root, after installing the locked dependencies:
node integrations/test-failure-explorer/scripts/generate-fixtures.mjs
The generator rejects any Vitest version other than 3.2.7. It copies runner/
to a temporary directory, links the existing locked node_modules, and runs
each case serially using the dedicated vitest.config.mjs. Each command uses
an absolute --root, absolute --config, --reporter=default --reporter=json,
and an absolute --outputFile.json. The two no-tests cases use the same empty
directory; only the successful case adds --passWithNoTests.
Intentional test sources use .fixture.ts, rather than .test.ts, and are
collected only by the dedicated generation config. They must remain excluded
from ordinary package tests. The generator checks each expected process exit
before recording a report and removes its temporary project after completion.
The expected failures below are fixture inputs, not failing product tests.
Independently checkable outcomes
The expected assertions in this table come from the committed tiny source
files, rather than the report parser under test. Reporter suite counters are
also recorded in provenance: suites include nested describe nodes and file
entries, so numTotalTestSuites must not be labeled as a file count.
| Report | Expected process exit | Expected report facts |
|---|---|---|
mixed.json |
1 | Two files; four assertions: three passed, one failed. Nested suites make the reporter suite count four. |
file-error.json |
1 | One failed file with a collection-error message and zero assertions; zero failed assertions does not imply success. |
statuses.json |
0 | One passed, two skipped (explicit skip and runtime skip), one todo. numPendingTests is two, while the actual row status is skipped. |
no-tests-fail.json |
1 | Empty testResults, zero assertions, success: false. |
no-tests-pass.json |
0 | Empty testResults, zero assertions, success: true. |
unhandled.json |
1 | One passed assertion and success: true, despite a real unhandled rejection and failing process exit. |
duplicate-titles.json |
1 | Two failed assertions with the exact same suite/title/fullName; both rows must remain addressable. |
hostile-text.json |
1 | One failure with Unicode, ANSI, literal HTML/script/instruction text, and 160 diagnostic lines; diagnostic data must stay inert. |
duplicate-paths.derived.json |
No execution claimed | Explicitly derived from duplicate-titles.json: duplicate its file entry and double the total/failed suite and test counters. Two identical file paths and four rows must remain addressable. |
Vitest's reporter does not emit the unhandled rejection in unhandled.json:
its file has no collection-error message and its assertion has no failure
message. Provenance records that the default reporter observed both
Unhandled Rejection and the source's Fixture unhandled rejection marker.
This sample proves that report success cannot establish process success.
The product imports the report only; provenance is test evidence, not an
additional runtime field from which to manufacture an execution status.
This run produces passed, failed, skipped, and todo assertion statuses.
Any pending/disabled row-status mutation or other schema boundary created
by a product test must be labeled derived. No claim is made that this runner
execution produced those row statuses.
Sanitization and replay
Only machine-specific absolute path prefixes inside JSON string values are
replaced: the temporary source root becomes /fixture-workspace, the
repository becomes /fixture-repository, temporary output becomes
/fixture-output, and the Node executable in recorded commands becomes
/fixture-node. These are deterministic safe placeholders; report filenames
and stack locations retain their source-relative suffixes. Counts, statuses,
messages, durations, timestamps, locations and stacks are otherwise preserved.
Paths in imported fixture reports are data, not paths to open on this machine.
rawSha256 hashes the actual runner bytes before sanitation;
sanitizedSha256 hashes the exact committed JSON bytes. consoleSha256
records the default reporter output without retaining its machine-specific
raw text. Every source file and the dedicated config are hashed in sources;
the generation script is hashed in generatorSha256. Derived reports name
their source hash and transformation and have no invented exit code.
Rerunning changes wall-clock timestamps, durations and stack timing, so it also updates report hashes and provenance. Replay must reproduce the semantic outcomes in the table, rather than identical timing-dependent JSON bytes. Large-array, oversized-field, count-mismatch and malformed-JSON cases should be constructed explicitly in tests from these reports; do not commit large synthetic reports as if they were measurements of a real repository.