mirror of
https://github.com/esengine/DeepSeek-Reasonix.git
synced 2026-11-07 10:45:59 -10:00
fix(benchmarks): two generated graders reach the same verdict off POSIX
Both have failed every run of CI's windows-latest job since the corpus landed, and neither is a flake. gen-topo-cycle timed each call with signal.alarm. Windows Python has no SIGALRM, so the grader raised while installing the handler, before it graded anything. A daemon worker thread keeps the semantics exactly — _Timeout on a hang, the original exception re-raised, so the cyclic case still reads ValueError — and cannot hold the interpreter open at exit. gen-csv-quotes compared cli.py's stdout against an expected string. Windows Python writes CRLF in text mode, so that comparison could never hold; the carriage returns come off first, because the grader checks the fields and not the host's line endings. Its mktemp template also read TMPDIR, which under Git Bash is a Windows path — the rest of the corpus asks for a bare mktemp -d, and now so does this one.
This commit is contained in:
1 parent
42bba371f9
commit
e3031b5fd7
2 files changed
+24
-15
No files matched your search
@@ -4,7 +4,7 @@
|
||||
# so it references csv_reader.py and cli.py directly.
|
||||
set -e
|
||||
|
||||
tmpdir="$(mktemp -d "${TMPDIR:-/tmp}/csv-verify.XXXXXX")"
|
||||
tmpdir="$(mktemp -d)"
|
||||
trap 'rm -rf "$tmpdir"' EXIT
|
||||
|
||||
# 1. Behavioural checks against the public API (python3 stdlib only).
|
||||
@@ -46,7 +46,9 @@ name,note
|
||||
EOF
|
||||
|
||||
expected=$'name|note\nAda, Countess of Lovelace|wrote "notes"'
|
||||
got="$(python3 cli.py "$tmpdir/sample.csv")"
|
||||
# Windows' python writes CRLF in text mode, so strip the carriage returns
|
||||
# before comparing: the grader checks the fields, not the host's line endings.
|
||||
got="$(python3 cli.py "$tmpdir/sample.csv" | tr -d '\r')"
|
||||
if [ "$got" != "$expected" ]; then
|
||||
printf 'FAIL: cli.py printed:\n%s\n\nexpected:\n%s\n' "$got" "$expected" >&2
|
||||
exit 1
|
||||
|
||||
@@ -5,8 +5,8 @@ set -e
|
||||
cd "$(dirname "$0")"
|
||||
|
||||
python3 - <<'PYEOF'
|
||||
import signal
|
||||
import sys
|
||||
import threading
|
||||
|
||||
import topo
|
||||
|
||||
@@ -15,19 +15,26 @@ class _Timeout(Exception):
|
||||
pass
|
||||
|
||||
|
||||
def _alarm_handler(signum, frame):
|
||||
raise _Timeout("topological_sort did not return within the time budget")
|
||||
|
||||
|
||||
signal.signal(signal.SIGALRM, _alarm_handler)
|
||||
|
||||
|
||||
# A worker thread rather than signal.alarm: Windows has no SIGALRM, and this
|
||||
# grader has to reach the same verdict on every host. The thread is a daemon so
|
||||
# a solution that never returns cannot hold the interpreter open at exit.
|
||||
def run_with_timeout(fn, *args, **kwargs):
|
||||
signal.alarm(5)
|
||||
try:
|
||||
return fn(*args, **kwargs)
|
||||
finally:
|
||||
signal.alarm(0)
|
||||
outcome = {}
|
||||
|
||||
def call():
|
||||
try:
|
||||
outcome["value"] = fn(*args, **kwargs)
|
||||
except BaseException as exc:
|
||||
outcome["error"] = exc
|
||||
|
||||
worker = threading.Thread(target=call, daemon=True)
|
||||
worker.start()
|
||||
worker.join(5)
|
||||
if worker.is_alive():
|
||||
raise _Timeout("topological_sort did not return within the time budget")
|
||||
if "error" in outcome:
|
||||
raise outcome["error"]
|
||||
return outcome["value"]
|
||||
|
||||
|
||||
def is_topo_order(n, edges, order):
|
||||
|
||||
Reference in new issue
Block a user