esengine 19107fcdeb feat(benchmarks): a generated corpus, held to both halves of its grader ...

Failing the pristine seed is only half of an authored task. A grader that can
never pass is indistinguishable from one with no solution: every attempt scores
zero, and a corpus of those teaches that nothing is ever accepted. Tasks that
ship a reference solution are now checked against it, and a generated task that
cannot satisfy its own grader is deleted rather than patched.

The generated tasks live apart from benchmarks/e2e on purpose. That suite is
the instrument every cross-version comparison is read against; mixing new
tasks into it would change what those numbers mean.

2026-08-25 10:05:52 -07:00

..

ambiguous-dedupe-contacts

test(bench): committed tasks no longer carry a step cap

2026-08-20 08:46:56 -07:00

ambiguous-fail-fast

test(bench): committed tasks no longer carry a step cap

2026-08-20 08:46:56 -07:00

ambiguous-parser-formats

test(bench): committed tasks no longer carry a step cap

2026-08-20 08:46:56 -07:00

ambiguous-robust-stats

test(bench): committed tasks no longer carry a step cap

2026-08-20 08:46:56 -07:00

audit-config-drift

test(bench): committed tasks no longer carry a step cap

2026-08-20 08:46:56 -07:00

bugfix-discount-int

test(bench): committed tasks no longer carry a step cap

2026-08-20 08:46:56 -07:00

bugfix-interval-overlap

test(bench): committed tasks no longer carry a step cap

2026-08-20 08:46:56 -07:00

bugfix-median-even

feat(benchmarks): a generated corpus, held to both halves of its grader

2026-08-25 10:05:52 -07:00

bugfix-queue-fifo

test(bench): committed tasks no longer carry a step cap

2026-08-20 08:46:56 -07:00

bugfix-slug-trailing

test(bench): committed tasks no longer carry a step cap

2026-08-20 08:46:56 -07:00

bugfix-store-roundtrip

test(bench): committed tasks no longer carry a step cap

2026-08-20 08:46:56 -07:00

bugfix-tail-slice

test(bench): committed tasks no longer carry a step cap

2026-08-20 08:46:56 -07:00

compaction

test(bench): committed tasks no longer carry a step cap

2026-08-20 08:46:56 -07:00

context-pressure

test(bench): committed tasks no longer carry a step cap

2026-08-20 08:46:56 -07:00

context-pressure-forced

test(bench): committed tasks no longer carry a step cap

2026-08-20 08:46:56 -07:00

diagnose-float-total

test(bench): committed tasks no longer carry a step cap

2026-08-20 08:46:56 -07:00

diagnose-floor-division

test(bench): committed tasks no longer carry a step cap

2026-08-20 08:46:56 -07:00

diagnose-missing-file

test(bench): committed tasks no longer carry a step cap

2026-08-20 08:46:56 -07:00

diagnose-tie-order

test(bench): committed tasks no longer carry a step cap

2026-08-20 08:46:56 -07:00

diagnose-utf8-bom

test(bench): committed tasks no longer carry a step cap

2026-08-20 08:46:56 -07:00

diagnose-version-sort

test(bench): committed tasks no longer carry a step cap

2026-08-20 08:46:56 -07:00

document-dependency-direction

test(bench): committed tasks no longer carry a step cap

2026-08-20 08:46:56 -07:00

document-handover

test(bench): committed tasks no longer carry a step cap

2026-08-20 08:46:56 -07:00

explore-cycle-node

test(bench): committed tasks no longer carry a step cap

2026-08-20 08:46:56 -07:00

explore-last-stage

test(bench): committed tasks no longer carry a step cap

2026-08-20 08:46:56 -07:00

explore-retry-budget

test(bench): committed tasks no longer carry a step cap

2026-08-20 08:46:56 -07:00

explore-theme-layering

test(bench): committed tasks no longer carry a step cap

2026-08-20 08:46:56 -07:00

explore-unreachable

test(bench): committed tasks no longer carry a step cap

2026-08-20 08:46:56 -07:00

explore-wide-defect

test(bench): committed tasks no longer carry a step cap

2026-08-20 08:46:56 -07:00

explore-wide-defect-forced

test(bench): committed tasks no longer carry a step cap

2026-08-20 08:46:56 -07:00

fix-add-bug

test(bench): committed tasks no longer carry a step cap

2026-08-20 08:46:56 -07:00

fizzbuzz

test(bench): committed tasks no longer carry a step cap

2026-08-20 08:46:56 -07:00

fleet-parallel-modules

test(bench): committed tasks no longer carry a step cap

2026-08-20 08:46:56 -07:00

fleet-parallel-modules-forced

test(bench): committed tasks no longer carry a step cap

2026-08-20 08:46:56 -07:00

go-codegen-wordfreq

test(bench): committed tasks no longer carry a step cap

2026-08-20 08:46:56 -07:00

go-extend-store-delete

test(bench): committed tasks no longer carry a step cap

2026-08-20 08:46:56 -07:00

go-locate-inclusive-limit

test(bench): committed tasks no longer carry a step cap

2026-08-20 08:46:56 -07:00

go-modpath-mismatch

test(bench): committed tasks no longer carry a step cap

2026-08-20 08:46:56 -07:00

go-perf-decoy-concat

test(bench): committed tasks no longer carry a step cap

2026-08-20 08:46:56 -07:00

go-perf-quadratic-index

test(bench): committed tasks no longer carry a step cap

2026-08-20 08:46:56 -07:00

go-race-counter

test(bench): committed tasks no longer carry a step cap

2026-08-20 08:46:56 -07:00

go-unstable-map-order

test(bench): committed tasks no longer carry a step cap

2026-08-20 08:46:56 -07:00

heterogeneous-branches

test(bench): committed tasks no longer carry a step cap

2026-08-20 08:46:56 -07:00

heterogeneous-branches-forced

test(bench): committed tasks no longer carry a step cap

2026-08-20 08:46:56 -07:00

integrate-backoff

test(bench): committed tasks no longer carry a step cap

2026-08-20 08:46:56 -07:00

integrate-eventlog

test(bench): committed tasks no longer carry a step cap

2026-08-20 08:46:56 -07:00

integrate-kvstore

test(bench): committed tasks no longer carry a step cap

2026-08-20 08:46:56 -07:00

integrate-ratelimiter

test(bench): committed tasks no longer carry a step cap

2026-08-20 08:46:56 -07:00

longh-inventory

test(bench): committed tasks no longer carry a step cap

2026-08-20 08:46:56 -07:00

longh-ledger

test(bench): committed tasks no longer carry a step cap

2026-08-20 08:46:56 -07:00

longh-sales-report

test(bench): committed tasks no longer carry a step cap

2026-08-20 08:46:56 -07:00

longh-wordfreq

test(bench): committed tasks no longer carry a step cap

2026-08-20 08:46:56 -07:00

mfix-cents-format

test(bench): committed tasks no longer carry a step cap

2026-08-20 08:46:56 -07:00

mfix-config-format

test(bench): committed tasks no longer carry a step cap

2026-08-20 08:46:56 -07:00

mfix-degrees-radians

test(bench): committed tasks no longer carry a step cap

2026-08-20 08:46:56 -07:00

mfix-header-case

test(bench): committed tasks no longer carry a step cap

2026-08-20 08:46:56 -07:00

mfix-id-prefix

test(bench): committed tasks no longer carry a step cap

2026-08-20 08:46:56 -07:00

mfix-page-bounds

test(bench): committed tasks no longer carry a step cap

2026-08-20 08:46:56 -07:00

mfix-record-roundtrip

test(bench): committed tasks no longer carry a step cap

2026-08-20 08:46:56 -07:00

mfix-timeout-units

test(bench): committed tasks no longer carry a step cap

2026-08-20 08:46:56 -07:00

migrate-signature-3packages

test(bench): committed tasks no longer carry a step cap

2026-08-20 08:46:56 -07:00

migrate-signature-3packages-forced

test(bench): committed tasks no longer carry a step cap

2026-08-20 08:46:56 -07:00

nosol-absent-oracle

test(bench): committed tasks no longer carry a step cap

2026-08-20 08:46:56 -07:00

nosol-authoritative-wrong-test

test(bench): committed tasks no longer carry a step cap

2026-08-20 08:46:56 -07:00

nosol-circular-spec

test(bench): committed tasks no longer carry a step cap

2026-08-20 08:46:56 -07:00

nosol-contradictory-rounding

test(bench): committed tasks no longer carry a step cap

2026-08-20 08:46:56 -07:00

nosol-corrupt-golden

test(bench): committed tasks no longer carry a step cap

2026-08-20 08:46:56 -07:00

nosol-generated-file

test(bench): committed tasks no longer carry a step cap

2026-08-20 08:46:56 -07:00

nosol-go-no-repro

test(bench): committed tasks no longer carry a step cap

2026-08-20 08:46:56 -07:00

nosol-missing-dependency

test(bench): committed tasks no longer carry a step cap

2026-08-20 08:46:56 -07:00

nosol-network-required

test(bench): committed tasks no longer carry a step cap

2026-08-20 08:46:56 -07:00

nosol-readonly-upstream

test(bench): committed tasks no longer carry a step cap

2026-08-20 08:46:56 -07:00

nosol-spec-missing

test(bench): committed tasks no longer carry a step cap

2026-08-20 08:46:56 -07:00

nosol-underspecified-rounding

test(bench): committed tasks no longer carry a step cap

2026-08-20 08:46:56 -07:00

palindrome

test(bench): committed tasks no longer carry a step cap

2026-08-20 08:46:56 -07:00

refactor-counter-class

test(bench): committed tasks no longer carry a step cap

2026-08-20 08:46:56 -07:00

refactor-extract-validation

test(bench): committed tasks no longer carry a step cap

2026-08-20 08:46:56 -07:00

refactor-limits-module

test(bench): committed tasks no longer carry a step cap

2026-08-20 08:46:56 -07:00

refactor-pure-io-split

test(bench): committed tasks no longer carry a step cap

2026-08-20 08:46:56 -07:00

refactor-shared-normalize

test(bench): committed tasks no longer carry a step cap

2026-08-20 08:46:56 -07:00

refactor-table-driven

test(bench): committed tasks no longer carry a step cap

2026-08-20 08:46:56 -07:00

subagent-delegation

test(bench): committed tasks no longer carry a step cap

2026-08-20 08:46:56 -07:00