From a793d0b87807fd356ae9c59689a7fb067e2e8fcc Mon Sep 17 00:00:00 2001 From: Ludovic Marie <136255825+ludovic111@users.noreply.github.com> Date: Wed, 4 Nov 2026 17:30:18 +0100 Subject: [PATCH] benchmarks: label the 2026-11-04 Mac mini runs offscreen, not comparable Horizon ran offscreen (no presenting) while vanilla drew into a window, so the Mac rows are not a fair comparison; the website keeps the Mac mini's 2026-10-22 runs. The reports stay as a record, labelled. Co-Authored-By: Claude Opus 5.5 --- benchmarks/perf/site-2026-11-04.md | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/benchmarks/perf/site-2026-11-04.md b/benchmarks/perf/site-2026-11-04.md index 0b447b89bc..437089bf67 100644 --- a/benchmarks/perf/site-2026-11-04.md +++ b/benchmarks/perf/site-2026-11-04.md @@ -1,6 +1,10 @@ # Horizon Nightly 2026-11-04 vs vanilla 26.3 on the Mac mini and the desktop (2026-11-04) -The runs behind playhorizon.dev/benchmarks as of 2026-11-04 (horizon-rust/horizon-site#58): `tools/perf_bench.py` +> **The Mac mini reports (`*-macmini-rd*.json`) are offscreen, not comparable.** Horizon rendered offscreen (no presenting) +> while vanilla drew into a window, so the Mac rows below are not a Horizon-vs-vanilla comparison, and the website does not +> publish them: it keeps the Mac mini's 2026-10-22 runs. They are kept here only as a record of that day's offscreen run. + +The desktop runs are the ones behind playhorizon.dev/benchmarks as of 2026-11-04 (horizon-rust/horizon-site#58): `tools/perf_bench.py` reports, render distance 12 and 32, three games each, every game from a fresh world (seed 1234), 1280x720, VSync off. Files: `{horizon,vanilla}-macmini-rd{12,32}.json` and `{horizon,vanilla}-desktop-rd{12,32}.json` next to this page. @@ -13,7 +17,7 @@ reports, render distance 12 and 32, three games each, every game from a fresh wo ## Machines and conditions -- **Mac mini, Apple M4 Pro (14 cores, 20-core GPU), macOS.** The session was locked, so Horizon ran `--offscreen`: a locked +- **Mac mini, Apple M4 Pro (14 cores, 20-core GPU), macOS: offscreen, not comparable.** The session was locked, so Horizon ran `--offscreen`: a locked session gives a window no frames, and windowed Horizon measured 0 FPS. Vanilla drew into its window. Offscreen skips presenting, which favours Horizon, so its averages aren't held at the 144 Hz display as they were on 2026-10-22. Each game started at a 1-minute load of 17-18 (the run waited for < 20); during the runs the load rose to 30-50. @@ -29,8 +33,8 @@ reports, render distance 12 and 32, three games each, every game from a fresh wo | Machine | RD | Horizon | Vanilla | Horizon ÷ vanilla (avg, 1% low) | |---|---|---|---|---| -| Mac mini | 12 | 774 / 294 (offscreen) | 132 / 27 | 5.9x, 10.9x | -| Mac mini | 32 | 297 / 159 (offscreen) | 68 / 16 | 4.4x, 9.9x | +| Mac mini (**offscreen, not comparable**) | 12 | 774 / 294 | 132 / 27 | — | +| Mac mini (**offscreen, not comparable**) | 32 | 297 / 159 | 68 / 16 | — | | Desktop | 12 | 2,224 / 308 | 1,824 / 201 | 1.2x, 1.5x | | Desktop | 32 | 800 / 140 | 696 / 64 | 1.15x, 2.2x |