fix(benchmarks): two generated graders reach the same verdict off POSIX

Both have failed every run of CI's windows-latest job since the corpus
landed, and neither is a flake.

gen-topo-cycle timed each call with signal.alarm. Windows Python has no
SIGALRM, so the grader raised while installing the handler, before it
graded anything. A daemon worker thread keeps the semantics exactly —
_Timeout on a hang, the original exception re-raised, so the cyclic case
still reads ValueError — and cannot hold the interpreter open at exit.

gen-csv-quotes compared cli.py's stdout against an expected string.
Windows Python writes CRLF in text mode, so that comparison could never
hold; the carriage returns come off first, because the grader checks the
fields and not the host's line endings. Its mktemp template also read
TMPDIR, which under Git Bash is a Windows path — the rest of the corpus
asks for a bare mktemp -d, and now so does this one.
This commit is contained in:
esengine committed 2026-08-26 07:35:14 -07:00
1 parent 42bba371f9
commit e3031b5fd7
2 files changed
+24 -15

No files matched your search

@@ -4,7 +4,7 @@
# so it references csv_reader.py and cli.py directly.
set -e
tmpdir="$(mktemp -d "${TMPDIR:-/tmp}/csv-verify.XXXXXX")"
tmpdir="$(mktemp -d)"
trap 'rm -rf "$tmpdir"' EXIT
# 1. Behavioural checks against the public API (python3 stdlib only).
@@ -46,7 +46,9 @@ name,note
EOF
expected=$'name|note\nAda, Countess of Lovelace|wrote "notes"'
got="$(python3 cli.py "$tmpdir/sample.csv")"
# Windows' python writes CRLF in text mode, so strip the carriage returns
# before comparing: the grader checks the fields, not the host's line endings.
got="$(python3 cli.py "$tmpdir/sample.csv" | tr -d '\r')"
if [ "$got" != "$expected" ]; then
printf 'FAIL: cli.py printed:\n%s\n\nexpected:\n%s\n' "$got" "$expected" >&2
exit 1
+20 -13
View File
@@ -5,8 +5,8 @@ set -e
cd "$(dirname "$0")"
python3 - <<'PYEOF'
import signal
import sys
import threading
import topo
@@ -15,19 +15,26 @@ class _Timeout(Exception):
pass
def _alarm_handler(signum, frame):
raise _Timeout("topological_sort did not return within the time budget")
signal.signal(signal.SIGALRM, _alarm_handler)
# A worker thread rather than signal.alarm: Windows has no SIGALRM, and this
# grader has to reach the same verdict on every host. The thread is a daemon so
# a solution that never returns cannot hold the interpreter open at exit.
def run_with_timeout(fn, *args, **kwargs):
signal.alarm(5)
try:
return fn(*args, **kwargs)
finally:
signal.alarm(0)
outcome = {}
def call():
try:
outcome["value"] = fn(*args, **kwargs)
except BaseException as exc:
outcome["error"] = exc
worker = threading.Thread(target=call, daemon=True)
worker.start()
worker.join(5)
if worker.is_alive():
raise _Timeout("topological_sort did not return within the time budget")
if "error" in outcome:
raise outcome["error"]
return outcome["value"]
def is_topo_order(n, edges, order):