Two ad-hoc A/B loops this session produced a 4.7% regression and a 6%
improvement that both evaporated on a second look. Running the same module
against itself says why:
fannkuch A 1.309s B 1.084s 0.828x individual runs 1.119-1.633
nbody A 1.681s B 1.599s 0.951x individual runs 1.604-2.479
dicts A 0.550s B 0.532s 0.967x individual runs 0.544-0.646
Seventeen per cent apart on identical bytes. I had been reporting fractions
of that as findings.
Three causes, all now addressed, and each was visible only once the one
before it was fixed:
* **Order.** The loops ran A then B every round, so whatever warms up
warmed up for B -- which came out ahead in all three, the signature of
exactly that. Rounds alternate now.
* **Length.** A one-second benchmark under V8 mostly measures where its
tiering landed. `BENCH_SCALE` makes each kernel run for several seconds
and the per-run spread falls from 46-83% to 18-45%.
* **A warm page cache.** The first calibration handed the same *path* to
both sides, so the second read a cache the first had filled, and won
everything. It copies the module now.
Calibrated after all three, six rounds, idle machine:
fannkuch 0.974x nbody 1.059x dicts 0.986x
About six per cent, and no side systematically ahead. That is the floor;
anything smaller is not a result.
Two claims are corrected by it. `bench/python.sh` said V8 was "a little
behind on the other two" -- 2% on fannkuch and 8% on dicts, at or under the
floor. Only nbody's 34% survives, and that is the claim the header now
makes. And #74 recorded bulk object recompilation as 4.5-8% slower on
CPython; that is not established either, and the task says so.
Also reverted here: polling only on non-return indirect branches. The
argument is sound -- among direct branches every cycle contains a backward
edge, so only a computed branch can close a cycle that never polls, and a
cycle closed by `ret` alone must grow the stack. It measured 1.028x, 0.938x
and 1.000x, which is to say nothing, and it buys a subtler correctness
argument and coarser preemption for it.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01KcidnrZ5aLqKoBJBqdjx9d
A backward direct branch is the cheap approximation of a loop, and it misses
the loop that matters most: one whose back edge is computed. Two hundred
million iterations of a loop closed only by `br x16` produced zero
safepoints -- so a thread inside a bytecode interpreter's dispatch, or
anything jump-threaded, could not be preempted, could not take a signal,
could not be interrupted by a timer, and appeared in no trace at all.
The first probe of this said the opposite, and was wrong: its loop still had
a direct backward branch at the bottom, and a constant-indexed computed goto
that the compiler folded into a direct branch. With a volatile selector the
branch survives as a real `br x16` and the count goes from 195 beats to 0.
With the poll, 97.
Priced on the worst case there is, CPython under V8, five interleaved
rounds:
fannkuch before 1.263s after 1.322s 1.047x
nbody before 1.633s after 1.640s 1.004x
dicts before 0.559s after 0.557s 0.996x
Five per cent on the most dispatch-heavy, nothing measurable on the other
two, and one per cent of code size. That is the price of a runtime that can
be interrupted at all.
It does not fix the glxgears stall, which was the reason for looking.
-- and the build --
`gexc` had not compiled since the `/proc` generation counters went in:
`ProcState` gained two fields and one site in the *native* shim listed every
field by name. The browser target compiled fine, so every check came back
green while the static recompiler sat four hours stale. That site now takes
`..Default::default()`.
It has a consequence worth stating: the CPython comparison recorded here
last time had the `v8` column running with two syscall fixes that the `gex`
column did not, because one was built from the current tree and the other
was not. Rerun from one tree, V8 is a long way ahead on floating point and a
little behind on integer and allocation -- mixed, not the win first
recorded. The header now says so.
Also: `--enable-logging=stderr` for the headless browser, without which the
page's console goes nowhere. The renderer is a separate wasm module with its
own libc and cannot reach the shim's stderr, so its diagnostics were being
written and dropped.
139 of 139 integration tests pass, built from one tree.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01KcidnrZ5aLqKoBJBqdjx9d
This file called itself "a measurement of the static recompiler and nothing
else". It was measuring it through Cranelift and wasmtime, which is the
debug adapter. Now it also runs the same module under V8.
Five rounds, alternating, idle machine, ratios against qemu-user:
fannkuch (integer, list) qemu 0.732s gex 1.581x v8 1.466x
nbody (float) qemu 2.724s gex 0.800x v8 0.766x
dicts (allocation) qemu 0.329s gex 1.681x v8 1.696x
V8 is ahead on two of the three and level on the third, running a CPython
whose every opcode handler was translated ahead of time -- a bytecode
interpreter being the shape a block-per-function translation is supposed to
be worst at. It agrees with what nbench already said, where V8 takes six
kernels of ten.
Every run's answers are compared against every other run's, whichever engine
produced them, and they agree.
Two things the browser leg needed:
* A subset of the rootfs. `--dir` bundles eagerly into the guest's
filesystem and 665 MB of Debian exhausts the shim's heap before the
program starts. It fails as `rust_oom` from inside wasm, which is not an
obvious way to be told the harness asked for too much. Twenty megabytes
-- the interpreter, its four libraries and the standard library -- is
enough.
* The tree mounted at `/`. Mounted anywhere else the ELF's interpreter
path does not resolve and the loader refuses the program, saying only
that it could not load it.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01KcidnrZ5aLqKoBJBqdjx9d
CPython would not run with the runtime translator off: the module failed to
validate 182 MB in, "expected i64, found i32". It was `smov x0, v0.h[2]`.
Every wasm lane extract narrower than 64 bits yields an i32 whatever the
lifter's node says its type is, so the 64-bit forms of `smov` need a
widening the narrower ones do not. Nothing gex had ever run used one.
That is the shape of bug exhaustive translation is good at finding, so this
makes a test of it. `gexc build` now validates what it writes, and
tests/exhaustive.sh translates every aarch64 ELF in a root filesystem
starting a block at every decodable address. 589 binaries, 82 MiB of guest
text, about thirteen minutes.
It found a second one immediately. `rev16`, `rev32` and `rev64` reverse
elements inside a group of fixed width, so an element at least as wide as
its group is a reserved encoding -- `rev16 v0.4s` is not an instruction. The
decoder accepted them and the lifter computed `group / lane_bytes - 1`,
which underflowed into a byte permutation with lane indices of 248. wasm
rejects a whole module for one of those, so a reserved encoding sitting in
libhogweed, in code nothing executes, made the library unloadable. The
decoder now rejects them and the lifter checks again, because a wrapped
permutation index is not a wrong answer, it is a module that will not load.
Both are covered by neon-move.c, 27 differential cases against qemu: every
smov and umov width, insert from a general register and from another lane,
and every element/group combination the reverse family actually allows.
With these, CPython runs with the JIT off entirely -- exhaustively
recompiled, nothing translated at run time -- at the same speed as with it:
fannkuch 0.950, nbody 2.037, dicts 0.567 against 0.88, 1.91, 0.54. What the
runtime translator buys is startup, not throughput.
589 validated, 0 failed. The 716,363 blocks that end in an instruction gex
cannot execute are mostly padding and literal pools decoded as code; the
count is there so its growth is visible, not because it is a failure.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01KcidnrZ5aLqKoBJBqdjx9d
Debian's python3-minimal links the interpreter into the executable, so every
instruction that matters is translated ahead of time and none of these
numbers contain any runtime translation. It is also the hardest shape a
block-per-function translation meets: a bytecode interpreter is one enormous
function of indirect dispatch.
Against qemu-user, medians over three invocations:
fannkuch (integer, list) 1.04 1.08 0.99
nbody (float) 0.68 0.66 0.69
dicts (allocation) 1.03 1.08 1.08
At parity on integer and list work, five to eight per cent behind on
allocation, a third faster on floating point. Every run is checked against
every other, whichever engine produced it.
Regions were the obvious thing to try, a dispatch loop being a loop. Paired
medians of five, alternating which ran first: 1.010, 0.968, 0.961. Neutral
to four per cent, the same answer the other workloads gave. The interpreter
loop is not one loop to discovery -- it is a jump table into a few hundred
separate handlers, and a region cannot hold that.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01KcidnrZ5aLqKoBJBqdjx9d
