Compare commits

...
Author SHA1 Message Date
Horizon agentandClaude Opus 5.5 8406d2c586 horizon-threads: the stack dump test samples only once the busy loop runs
The worker sent its handle and went straight into the loop, so the signal could land
while it was still waking the receiver (futex_wake in the mpsc send), and the busy
frame was not yet on its stack. It failed that way on CI (run 43194576407).

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-12 14:38:30 +01:00
Ludovic Marie f24ee1accf Merge pull request #15361 from UMCEKO/perf-platform
🤖 Agent: queued for merge at Ludovic's request.
2026-11-12 12:12:30 +01:00
umcekoandClaude Opus 5.5 327daa3899 Merge origin/main into perf-platform
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-12 13:12:23 +03:00
Kacper ba91f75cb2 Merge pull request #13731 from TheApple05/fix/13666-chest-boat-loot
Fix chest boat loot for hopper transfers and destruction
2026-11-12 11:08:09 +01:00
Kacper 5164ada2a6 Merge branch 'main' into fix/13666-chest-boat-loot 2026-11-12 10:36:41 +01:00
Kacper d0c091ae1d Merge branch 'main' into fix/13666-chest-boat-loot 2026-11-12 08:52:13 +01:00
umcekoandClaude Opus 5.5 7798130a18 Merge origin/main into perf-platform
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-12 05:24:24 +03:00
umcekoandClaude Opus 5.5 474e9eb4c8 Perf: the entity context lines live in the per-mob and per-projectile helpers
creatures.rs may not grow after main moved the mob loop out; end_mob clears the mob's context, Projectile::tick sets its own, and the fangs start with none.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-12 03:59:29 +03:00
umcekoandClaude Opus 5.5 9fd3efbf81 Merge origin/main into perf-platform-clean
The mob loop moved into creatures/mob_group.rs; its per-mob sample takes start_mob/end_mob there.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-12 03:53:38 +03:00
Kacper c26677ed47 Merge branch 'main' into fix/13666-chest-boat-loot 2026-11-12 01:04:14 +01:00
umcekoandClaude Opus 5.5 47b40ecd35 Merge remote-tracking branch 'origin/main' into perf-platform-clean
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-12 02:03:21 +03:00
Kacper cccf79f63a Merge branch 'main' into fix/13666-chest-boat-loot 2026-11-11 23:56:45 +01:00
umcekoandClaude Opus 5.5 77b12a46b1 server_profile_e2e: the disconnect step starts once the timed capture's file has arrived
The client refuses a server start while the last file is still coming in.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-12 01:40:12 +03:00
umcekoandClaude Opus 5.5 3da49d7a02 Perf: the inflation-cap test checks the bomb against the announced cap
Its gzip size depends on the flate2 backend that feature unification picks
(3.7 MB alone, 5.2 MB in a workspace test run); what matters is that the
announced-size cap lets it through, so the inflated cap is what stops it.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-12 00:58:35 +03:00
umcekoandClaude Opus 5.5 eae9be3b4f Perf: rustfmt the sampler's ring attribute
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-12 00:51:16 +03:00
umcekoandClaude Opus 5.5 f21daf7263 Perf: horizon-server report prints a short summary again
On main `horizon-server report` printed the report's text summary; with the
capture it printed only the file's path, and the text form moved to
horizon-perf, which no release ships, so a host on a VPS could not read the
result there. horizon-perf-format, already shipped through the platform,
gets server_brief: the kind and length, TPS and MSPT, CPU, RSS, players,
chunks and entities, and the five heaviest tick phases. The report prints
it before the path.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-12 00:47:34 +03:00
umcekoandClaude Opus 5.5 21a8b3bff1 Perf: the one-shot Open server retries a failed accept until its deadline
Any accept() error other than WouldBlock ended the server that offers a
capture to the viewer, before the viewer had fetched it: an aborted
connection (ECONNABORTED, common on BSDs and macOS, WSAECONNRESET on
Windows) or a full file table (EMFILE) left Open broken for that click.
Errors now wait 10 ms and try again until the offer's deadline or a cancel.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-12 00:45:29 +03:00
umcekoandClaude Opus 5.5 1979dc6dc2 Perf: a thread's sampler ring grows when a session needs more
A thread's ring was sized at its first session and kept, but the size a
session needs depends on its Hz, --wall (twice the entries) and the unwind
mode (about 16 KiB per stack copy), so a later capture with --wall or a
higher --hz lost most of its samples to `dropped` (95 of 99 in the new
test). A session now gets a new, larger ring when the thread's is smaller
than it needs. A replaced ring stays allocated for the thread's life, since
a late signal's handler may still hold it; rings only ever grow, so this is
at most a few per thread.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-12 00:44:51 +03:00
umcekoandClaude Opus 5.5 b2df8cee34 Perf: a server profile stops when the guest who asked for it leaves
The requester was keyed by connection id and never cleared on disconnect,
so after the admin's client dropped during an open-ended `profile server
start` everyone got AlreadyRunning until the 600 s window ended, and the
same admin, reconnected under a new id, got NotRunning when stopping. Each
tick's requests now include a stop (reason DISCONNECTED) once the
requester's connection is gone; the file stays in the server's reports/ and
its path is logged, and the slot frees as soon as it is saved, so anyone
(the same admin included) can start again. server_profile_e2e covers the
disconnect and the start after reconnecting.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-12 00:43:06 +03:00
umcekoandClaude Opus 5.5 ec1fb9a182 Perf: Share uploads again when the receipt's upload is gone
share_saved always reused an existing .hzperf.share.json and only re-pinned
it, so once the upload expired (a temporary one after an hour, a shared one
after 30 days) or was deleted, Share failed for that capture until the
receipt was removed by hand. A 404 or 410 on the pin of a reused receipt now
uploads the capture again, replaces the receipt (still 0600, written beside
and renamed over) and pins the new upload.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-12 00:40:09 +03:00
umcekoandClaude Opus 5.5 cec503f33e Perf: a typed /profiler goes where its completion came from
with_profiler keeps a server's or mod's own `profiler` node, so the chat
completed the server's command while handle_chat_input still answered
/profiler locally, and the server's command could not be reached. Horizon's
nodes now carry their own command id; a typed /profiler runs locally only
when the chat tree's `profiler` is Horizon's (or there is no server), and
otherwise goes to the server or mod that owns the name. The client's own
buttons keep running locally.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-12 00:36:22 +03:00
umcekoandClaude Opus 5.5 5ccb6cd15b Perf: a server's click can no longer run /profiler on the client
click_command answered any clicked /profiler line itself, so a server's
chat component, book or dialog with run_command "/profiler share" uploaded
the player's last capture after one click. The client's own [Open],
[Share], [Start], [Stop] and retry buttons now carry a client-only click
action that no server text decodes to (as [Folder]'s open_file already
was); a clicked run_command is a plain command again, sent to the server.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-12 00:27:09 +03:00
umcekoandClaude Opus 5.5 91491459d5 Perf: a server part whose last chunk fails to finish still gets its Stop
The merger was taken out before finish(), so when finish() failed the
abandoned part was closed with a fresh merger that had yielded nothing,
and no Stop{DISCONNECTED} was written for a part that began. finish() now
borrows the merger, which keeps what it yielded for cut_short.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-12 00:14:36 +03:00
umcekoandClaude Opus 5.5 4cce8f8484 Perf: cap the server's part and take only its own side's fields
A client took whatever size a server announced and inflated every chunk
without limit, and appended fields of any side into the player's capture.
The announced size is now capped at 64 MiB (the share endpoint's own cap),
the inflated part at 512 MiB (the timeline's 256 MiB cap plus the rest of
a 600 s capture; real captures inflate about 3x from 130-230 KB a minute),
and one field at 64 MiB, which also bounds what waits half-arrived. A field
whose message is not side = SERVER, or has no side (an attachment), refuses
the part; what arrived stays with the incomplete warning, as for any part
that fails.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-12 00:12:29 +03:00
umceko c7227eb3f7 Merge remote-tracking branch 'origin/main' into perf-platform-clean
# Conflicts:
#	crates/horizon-client-render/src/gfx/pack_timing.rs
#	crates/horizon-platform/src/lib.rs
2026-11-12 00:03:27 +03:00
umceko fb85eb77e0 Merge remote-tracking branch 'origin/main' into perf-platform-clean
# Conflicts:
#	crates/horizon-client/src/game/creatures.rs
2026-11-11 21:18:57 +03:00
umcekoandClaude Opus 5.5 a3efbe017d profile_client_e2e: the stop line reads "Profile saved" since the chat redesign
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-11 21:18:38 +03:00
umcekoandClaude Opus 5.5 239b96621a Performance platform: files over 3000 lines don't grow
The pass timing switches and readings and the world pass helpers move
from gfx.rs to gfx/pass_timing_api.rs, the streamer's queue census and
job_stats to streaming/, Horizon's mark sprite to mark_sprite.rs, and
`horizon-ctl lan op` to game/lan_op.rs. A guest's plugin channel
routing moves beside horizon_channel, the client takes the profile
channel's replies before handle_packet, and tick_profile starts and
ends a mob's tick for both the sampler and the mob costs. The server
bench test reads the saved line the server prints now.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-11 19:53:39 +03:00
umcekoandClaude Opus 5.5 08b55819b1 Performance platform: leave out what is not the profiler
Removes the fork's review and claim tooling and its docs, and the scene
corpus, which the design keeps separate (an input fixture, not a
measurement). The design doc describes the viewer and upload API in
horizon-rust/horizon-site and drops the planning sections; tests and
doc comments point at playhorizon.dev and horizon-site. The debug keys'
comment is upstream's again.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-11 18:21:33 +03:00
umcekoandClaude Opus 5.5 d27e19390d Merge upstream main into the performance platform
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-11 18:18:28 +03:00
Umut Cevdet Koçak ad491915a5 Merge pull request #280 from sunrise-org/perf-idle-cost
Profiler docs: idle hooks' cost within the margin of error
2026-11-11 18:16:50 +03:00
umcekoandClaude Opus 5.5 f356ceda41 Profiler docs: the idle hooks' cost is within the margin of error
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-11 18:14:56 +03:00
Umut Cevdet Koçak add317d016 Merge pull request #277 from sunrise-org/perf-m1-heap
Sampled heap profiling: allocations and live memory by call stack
2026-11-11 03:39:28 +03:00
umcekoandClaude Opus 5.5 b944c970c3 Heap samples leave out the profiler's own threads
The e2e capture's live heap was all on horizon-profile: the CPU
sampler's drain thread copies the binary's unwind sections (~9 MB,
framehop's Module::new from StackUnwinder::refresh) at the start of the
capture and holds them to the end, so the heap sampler profiled the
profiler. Samples were already tagged with the allocating thread.

heap_sample::ignore_this_thread() stops sampling a thread's allocations
(its frees still leave the live set). The sampler's drain thread, the
capture writer (WriterThread::spawn_with), the finishing thread and
the gauges thread call it.

server_perf_e2e checks live bytes over every thread, none on a profiler
thread, and that the top live function is not the profiler's; it
samples every 64 KiB so its 4 s capture holds some. It now passes: 1.0
MiB live on tick, workers and chunk threads, 19,453 samples.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-11 03:39:05 +03:00
umceko 3030aa3652 Merge remote-tracking branch 'fork/perf-platform-upstream' into perf-m1-heap
# Conflicts:
#	tools/server_perf_e2e.py
2026-11-11 03:20:39 +03:00
Umut Cevdet Koçak f65b2574e1 Merge pull request #279 from sunrise-org/perf-chat-calm
Profiler chat: calm lines, action bar progress, and the mark sprite fixed
2026-11-11 03:03:02 +03:00
umcekoandClaude Sonnet 5.5 f80e0cfbf0 Profiler chat: one calm line per outcome, progress on the action bar, and the mark sprite drawn
Chat keeps the lines with a next step (started, saved, shared, failures); saving,
uploading, opened and marked go to the action bar; the saved line carries the
capture's length from when stop was asked; [Retry] names its profile.

Text::styled dropped the style's inline object (Style::inherit never passes one
on), so the profiler tag drew the font's [OBJ] box instead of horizon:gui/mark.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-11 03:02:11 +03:00
Umut Cevdet Koçak b97a7d0dec Merge pull request #278 from sunrise-org/perf-m2-reader-memory
Perf platform M2: finer sample blocks, memory_labels/counters ops, samples in tick, complete server memory report
2026-11-11 01:48:42 +03:00
Umut Cevdet Koçak b6832dc940 Merge pull request #276 from sunrise-org/perf-l1-layers
Profiler on upstream's layers: Open and Share through a hook, no built-in address
2026-11-11 01:48:38 +03:00
umcekoandClaude Sonnet 5.5 7942b6d4c3 Memory report: a typed, complete server report with entities, players, chunk send caches, queues and unattributed_mb
`horizon-ctl memory` and a capture's memory snapshots are built from `#[derive(Serialize)]` structs instead
of `json!`; existing keys keep their shape. A dedicated server (no window) names its columns `columns`;
single player keeps `client_columns` and `server_columns`.

New, all estimated from capacities:
- `entities`: count and bytes by entity type (top 30), across every level
- `players`: count and, per player by name, the send queue (packets and bytes), chunks waiting, chunks sent
  and tracked entities
- `chunk_send`: the encoded section and heightmap caches and the shared compressed frames
- `light_queue`, `generation_queue`: lengths and the bytes of their tables
- `allocator`: `live_heap_mb` (mimalloc's statistics), `commit_mb`, `peak_commit_mb`, `overhead_mb`
- `named_mb` and `unattributed_mb` (resident minus the named parts) on every side

The server e2e asserts a stop snapshot has `players`, `entities` and `unattributed_mb`.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-11 01:47:31 +03:00
umcekoandClaude Sonnet 5.5 c7a26889ff horizon-perf: tick carries the top functions of the samples around the tick
`hot` is the top 30 functions (the `hot` op's row) of the main thread's samples in the time blocks
overlapping the tick, only the slow samples when the tick is slow, and `samples_from`/`samples_to` say the
window those blocks cover. With 1 s blocks that is the second around the tick, not the tick's own samples.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-11 01:47:31 +03:00
umcekoandClaude Sonnet 5.5 46b588e72a horizon-perf: memory_labels lists a capture's memory snapshots and counters its counters
`memory_labels` (CLI `query memory-labels`, MCP and wasm through the shared op list) gives one row per
snapshot, `{label, t, side, rows}`, in time order. `counters` gives one row per counter definition,
`{id, name, unit, kind, labels, side, samples}`. Each has a golden reply. `memory` also ranks a negative
megabyte row by its size, so `unattributed_mb` can show.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-11 01:47:31 +03:00
umcekoandClaude Sonnet 5.5 2843049551 Profiler: CPU sample time blocks are 1 s, up to 1800, and a time range selects the blocks it overlaps
`block_s` defaults to 1.0 and `MAX_TIME_BLOCKS` is 1800 (30 minutes, the block index a `u16`); a longer
session keeps the rest in the last block, as before. The reader selects every block a `--from`/`--to`
range overlaps, a range inside one block picks that block, and the reply's `selection.from`/`to` name the
blocks the weights cover, so a 2 s range no longer leaves Functions and the flame graph empty.

Capture size before and after on the 200-bot benchmark and a client capture is not measured yet: pending a
quiet machine. If growth is over 2x, blocks older than 5 minutes get merged at write time.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-11 01:47:31 +03:00
umcekoandClaude Opus 5.5 9e7925e2e9 Sampled heap profiling: allocations and live memory by call stack
horizon-threads' heap_sample wraps mimalloc and is the game's allocator
on Linux (alloc-count and alloc-census still replace it). While a
capture runs it samples about one allocation per 512 KiB
(HORIZON_HEAP_SAMPLE_KIB; 0 turns it off) with an exponential countdown
per thread, records the stack, thread and span context, and weighs
each sample interval / (1 - exp(-size / interval)) bytes. Frees look a
pointer up in a 1 MiB table of per-bucket counts and only take the lock
on a hit. One session per process: the first capture to start owns it,
so the integrated server's joined side does not start another.

Captures write two more kinds, alloc and live (bytes, period 1), into
the same tables as the CPU samples, without the allocator's own frames
at the leaf end.

Measured (fast profile, Linux x86_64, 24 threads, machine at load ~50
from other builds):
- Off: alloc_count_bench 64 B alloc+free pairs, +0.4-0.6 ns a pair
  (+9-11% of mimalloc's 5 ns on 1 thread; +1-14% on 8, noisy): one
  relaxed load and branch on each side, as designed.
- On, 200 walking bots with a capture: the sampler is 1.2% of the
  tick thread's CPU samples (1.24 and 0.77 ms a tick), about 70% of it
  .eh_frame unwinding, at ~40-70 us a sample; 63k and 100k samples in
  ~2 min. MSPT avg on 89/148/224/97 vs off 108/69/217/65 ms: the
  overloaded server (TPS 5-14) and the machine load swamp the A/B.
- Capture size: 209 and 231 KB on vs 171 and 171 KB off (+22-35%).

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-11 01:40:00 +03:00
umcekoandClaude Opus 5.5 81b07fb2d6 Profiler format and reader: live heap samples
SampleSource gains LIVE (sampled heap bytes still allocated when the
capture stopped, at their allocation's time block), and Overhead gains
heap_samples and heap_sample_ns.

horizon-perf reads them: --select live works in query hot/tree/callers
and in the folded and pprof exports, in bytes (a total, not a rate per
tick); the summary prints a heap line (live and allocated MiB, the
functions holding the most) and the heap sampler's cost.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-11 01:40:00 +03:00
umcekoandClaude Sonnet 5.5 bd94713fdd /profiler: stop only saves; [Open] and [Share] are the player's to click
`/profiler stop` prints the saved file and never uploads. Where a captures part is
registered (the official build) the line carries [Open] and [Share] buttons that run
`/profiler open <id>` and `/profiler share <id>` (no id: the last capture saved);
a build without the part answers both "not available in this build". horizon-ctl
gains `profile open FILE` and `profile share FILE`. The profiler harnesses read the
new saved line, and `profile_client_e2e.py --official` checks the buttons and open_at.
Docs: saving, Open and Share are independent of telemetry; Share is the only upload.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-11 00:28:07 +03:00
umcekoandClaude Sonnet 5.5 a86ab9d4bc Layers: the profiler crates have layers, and benchmarks sit on horizon-system
The benchmark report format reads its hardware through `horizon-system`, not
`horizon-telemetry`, so `horizon-bench` is an engine crate and the two `-> horizon-bench`
exceptions go. `horizon-perf-format` is engine, `horizon-perf` a tool, `horizon-perf-share`
Horizon. The launcher and `horizon-server` reach the report format and the capture files
through `horizon_platform::{bench, perf_format}`. `horizon-threads` joins the portable
crates, which `horizon-world` already used.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-11 00:28:07 +03:00
umcekoandClaude Sonnet 5.5 ccf6b2c913 Perf captures: Open and Share go through a hook, not a built-in address
Saving a capture is local. A registered `Hooks::captures` offers the viewer page,
Open (serve the file once on 127.0.0.1 and open the viewer on it) and Share (the
upload). `horizon-perf-share`, a Horizon crate in `horizon-official`'s parts, holds
both, with the addresses in `horizon-addresses` and their env overrides; the upload
code moves there from `horizon-perf`, which gains `horizon-perf open`.

The saved line and the `profile stop` reply carry no address: `open_at` is present
only when a build has the hook. `horizon-client` keeps `horizon-perf` for tests only.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-11 00:28:07 +03:00
Umut Cevdet Koçak e301d92133 Merge branch 'horizon-rust:main' into perf-platform-upstream 2026-11-10 22:13:29 +03:00
umcekoandClaude Opus 5.5 fc9eb64042 Captures name players: the privacy section and its harness say so
A server owner has to know which connection floods or lags, so net rows carry each player's name, client
brand and view distance. server_perf_e2e still checks that the world, seed, MOTD, mob and sign names and the
IP stay out, and now checks that net names both bots.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-10 16:22:22 +03:00
umcekoandClaude Opus 5.5 fc9acc4a74 Profiler: a client capture records its connection, and a server-thread start waits for the last file
The window now hands its server connection's byte counters to the capture each frame, so a client capture's
network tab has data. A server-thread capture asked for while the previous one is still being written waits
for that file instead of refusing, which the window had already reported as started.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-10 16:22:22 +03:00
umcekoandClaude Sonnet 5.5 79b1df4051 Golden net replies carry each connection's name, brand and view distance
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-10 15:58:46 +03:00
umcekoandClaude Sonnet 5.5 31ae839a81 Format the ported profiler work
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-10 15:18:40 +03:00
umcekoandClaude Sonnet 5.5 226fb118f8 Captures name each connection: player, client brand and view distance; a client records its own server traffic
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-10 15:17:58 +03:00
umcekoandClaude Sonnet 5.5 4390baae27 Profiler chat: Horizon's mark as the line's tag
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-10 15:15:50 +03:00
umcekoandClaude Sonnet 5.5 aa3df3b4f7 Read a single-player capture's GPU frames without naming the side
The profile e2e checks the GPU summary and frame series too.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-10 15:01:15 +03:00
umcekoandClaude Sonnet 5.5 51fa349fe4 Record every GPU-timed frame in captures and split the world pass into draw groups
A capture made with --gpu now writes each timed frame (gpu_frames = 24: its
start on the capture clock, first-to-last timestamp total, every pass and part)
in blocks about once a second, with the CPU ms of every drawn frame. GpuPasses
ms becomes the median over the whole capture instead of the last 15 frames.

Where the device stamps inside passes, every timed frame splits the world
passes into their draw groups (sky, solid terrain, occlusion boxes, entities,
translucent terrain, clouds, weather, hand...) and compute passes into their
dispatches, so GPU culling's parts show without detail mode; Metal keeps pass
boundaries, recorded as GpuDevice.pass_split.

query gpu gains per-pass and per-part samples, p50/p95/p99/max and share of the
GPU frame p50, plus a summary row with GPU and CPU frame percentiles and a
gpu/cpu/mixed bound verdict; query gpu_frames gives the series with slow frames
flagged. Older captures read with empty statistics.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-10 14:44:54 +03:00
umcekoandClaude Sonnet 5.5 034ffdbfef Parse /profiler through its command tree, with gpu literals and spark-style chat lines
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-10 14:35:00 +03:00
umcekoandClaude Sonnet 5.5 28fa20f13d Share in-game profiler captures through direct viewer links
Upload completed chat-started captures on a background worker and show a clickable capture URL. Preserve the local file and an owner-only receipt before pinning, so /profiler share can retry without uploading twice. Use an application user agent accepted by the deployed API. Keep control-socket and environment captures local.

Validated timed and manual in-game completion against the live API with byte-identical download, plus retry/ownership regression, command completion and Clippy. Test uploads removed after validation.

Fixes sunrise-org/horizon#267

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-10 14:11:21 +03:00
umcekoandClaude Sonnet 5.5 ac784c6832 Record complete 1% and 0.1% timing lows for performance captures
Keep bounded raw unit timings during recording and expose exact slow-tail means through meta. Tick rates express work capacity capped at 20 TPS; old or incomplete captures omit lows. Related: sunrise-org/horizon#264.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-10 14:08:24 +03:00
umcekoandClaude Sonnet 5.5 9cddcfb071 Carry the remaining profiler and benchmark work onto upstream's layout
Benchmark A/B runs and --benchmark-set/--profile, the memory section, span
laps through streaming and the world update, the server report and the
/profiler command tree, ported from the original patches onto the files
upstream split out of game.rs.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-10 13:48:22 +03:00
umcekoandClaude Sonnet 5.5 4039f0eeb6 Merge origin/main into perf-platform-upstream
Conflicts kept both sides: the platform's pass timing feature moves into the
device search's new open_adapter, its Options fields and work-attribution
calls follow the code upstream moved out of game.rs, and the perf channel
joins upstream's list of Horizon channels.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-10 13:37:36 +03:00
Umut Cevdet Koçak 18c2782391 Merge pull request #259 from sunrise-org/fix/perf-upload-public-check
Fix the capture upload E2E driver for the public CDN
2026-11-10 00:47:14 +03:00
umcekoandCodex gpt-6.1-sol eb150b0297 Make the capture upload check work through the public CDN
Co-Authored-By: Codex gpt-6.1-sol <[email protected]>
2026-11-10 00:29:46 +03:00
Umut Cevdet Koçak 88cc456f65 Merge pull request #258 from sunrise-org/feat/perf-sharing
Move the perf viewer to the site and implement capture sharing in the CLI
2026-11-10 00:25:02 +03:00
umcekoandCodex gpt-6.1-sol b2d4f65e77 Move the perf viewer to the site and add capture uploads and share links
Keep Rust reader artifacts in the game repository, move the viewer source into sunrise-org/horizon-site, and add CLI uploads and remote capture reads. Update fork automation for the organization transfer.

Co-Authored-By: Codex gpt-6.1-sol <[email protected]>
2026-11-10 00:03:48 +03:00
AppaandCodex aac293db9b Keep the boat identity across loot advancement rewards
Co-Authored-By: Codex (GPT-6) <[email protected]>
2026-11-09 00:25:43 -08:00
AppaandCodex a328e9c5db Keep chest boat loot handling in the boat module
Respect the repository size limit on commands and LAN action modules.

Co-Authored-By: Codex (GPT-6) <[email protected]>
2026-11-09 00:24:50 -08:00
AppaandCodex 4272268cbc Guard chest boat loot fixtures with a fresh hopper cooldown
Co-Authored-By: Codex (GPT-6) <[email protected]>
2026-11-09 00:21:18 -08:00
AppaandCodex 3ef0ebf52f Fix chest boat loot unpacking for hopper access and destruction
Roll pending loot before automation accesses chest-boat slots or destroying
removals spill them. Keep occupied slots, synchronize open-menu copies, and
award container loot only for the player opening a pending table.

Co-Authored-By: Codex (GPT-6) <[email protected]>
2026-11-09 00:17:30 -08:00
umcekoandClaude Opus 5.5 3d11f0b6c6 Merge upstream main into perf-platform
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-08 21:15:53 +03:00
umcekoandClaude Opus 5.5 db571535c2 Merge perf-platform web work into the upstream sync
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-08 21:07:08 +03:00
Umut Cevdet Koçak 7d9d35c729 Merge pull request #257 from UMCEKO/plat/web-finish
Perf web: call tree, flame graph, info points, copy id, bench and compare in every view
2026-11-08 21:03:37 +03:00
umcekoandClaude Opus 5.5 6354608451 Merge perf-platform into the finished web viewer
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-08 20:47:45 +03:00
umcekoandClaude Sonnet 5.5 1fc4ccd439 Perf web: UI review fixes (numbers from the wasm, no file name, no duplicated views, units and labels)
- Summary tiles read the wasm's own series stats (new optional `stats` on series rows, asked for with filters.stats); the old browser-side statistics disagreed with the CLI summary (MSPT 8.9 against 10.7, CPU 8.47 against 6.6).
- Queries no longer carry the real file name, so the Query tab never shows it.
- Slow ticks: every kept tick opens into its full span tree; the meaningless min/median/p95 of the slowest 16, the histogram and the sparkline are gone. The summary no longer repeats the worst ticks and lists only the top steps of the tick.
- Metadata: tabs that only repeated Memory, Network and World are gone, rows without data are left out, empty tabs are hidden, enum values read as words, and the mod crate column reads the `crate` field the wasm sends.
- Units on every number, frame figures say frames, warnings are worded from their codes, compare rows have readable names and drop an all-"not enough data" verdict column, network packet columns hide when nothing was counted, TPS windows longer than the capture are not shown.
- Keyboard focus is no longer hidden under the sticky tab bar, and rows that open show a chevron.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-08 20:42:38 +03:00
umcekoandClaude Sonnet 5.5 ce9fdee6d4 Perf web: offer only the exports the capture can fill, group by thread ignores the picked thread, one verdict colour scheme, strings through t()
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-08 20:33:14 +03:00
umcekoandClaude Sonnet 5.5 8d1b4c8cca Perf web: every row view shows the change and verdict against a second capture
The diff gains opt-in function and call-path rows (--calls, calls in hp_diff), matched by stable id. One comparison state feeds Summary, Spans, Functions and the Call tree; rows only the second capture has are listed apart.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-08 20:22:47 +03:00
umcekoandClaude Sonnet 5.5 fee41659bd Perf web: hide the sample picker when there is one choice, drop an unsafe cast in export
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-08 20:04:27 +03:00
umcekoandClaude Sonnet 5.5 b0154fc77d Merge plat/web-bench
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-08 19:57:30 +03:00
umceko 2dbff9aefe Merge plat/web-points 2026-11-08 19:57:16 +03:00
umceko dee8aecd7a Merge plat/web-flame 2026-11-08 19:57:15 +03:00
umceko 3a0a7390bf Merge plat/web-tree 2026-11-08 19:57:13 +03:00
umcekoandClaude Sonnet 5.5 e434d37311 Perf web: info points on well-known spans and functions, and a copy-id control on every row
Spans and functions the profiler meets most (tick_blocks, tick_lan, streaming,
tick_mobs, lan_chunks, random_ticks, allocator and memory calls...) get a short
explanation keyed by stable id in the translation file, shown as an accessible
hover and focus tooltip. Every row of the Summary, Spans, Functions, Call tree
and Slow ticks views copies its stable id from a control that appears on row
hover or keyboard focus, so rows stay clean at rest.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-08 19:56:27 +03:00
umcekoandClaude Sonnet 5.5 544600c10a Perf web: flame graph view for the call tree
A List / Flame graph toggle in the Call tree tab draws the current side, thread,
selection, grouping, merge mode and focus as a canvas flame graph: widths are time,
colours are the tree's crate groups, hover shows name, time and share, a click zooms
in with a trail back, and arrow keys, Enter and Escape do the same from the keyboard.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-08 19:49:44 +03:00
umcekoandClaude Sonnet 5.5 75e0f7387b Perf web: call tree with colour by crate, group by, selections, detail panel and search
The tree rows carry their crate, and a request can ask for each row's own time as a percent.
The tree tab is split into focused components; chains of single-child functions share a line.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-08 19:41:38 +03:00
Umut Cevdet Koçak 285846c5ee Merge pull request #256 from UMCEKO/perf/w1-web-ui
Web UI for horizon-perf
2026-11-08 19:33:32 +03:00
umcekoandClaude Sonnet 5.5 9df9c5bde2 Perf web: Bench tab with frame times per phase, repeat statistics and A/B verdicts
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-08 19:29:59 +03:00
umcekoandClaude Opus 5.5 28561fd634 Launcher: a native mod's generated main uses the game's current allocator
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-08 19:19:52 +03:00
umcekoandClaude Opus 5.5 f3613a539d Perf web: landing page, tabs only where there is data, and no duplicated views
The no-capture screen is now a landing page: what the viewer reads and how to get a capture. A tab appears only when its query answers with rows for the side on show, so GPU is gone from a server and empty tabs are gone everywhere; Metadata's sub-tabs follow the same rule.

Health, and Summary's network and dimension cards, repeated other views and are gone. So are the file name and size, the report box, New profile, the sort direction and row limit, the placeholder flame graph, and id chips.

Fixes: slow ticks say they cover only the slowest kept, in time order; query errors show their real message; Compare is a table of real rows for the side on show; table headers line up with their columns.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-08 19:18:46 +03:00
umcekoandClaude Opus 5.5 04a9f56511 Perf web: review fixes, design-system polish and no upload path
Removes sharing entirely: the Share button, share links, the upload API, its S3 storage and the CLI's `share` and `upload`. A capture stays on the machine.

Fixes:
- ureq and the snapshot files are gone, so the wasm builds and the tree is 7,600 lines lighter
- merge mode hides wrappers and merges a function across its callers
- `group --by mod` backs the mods view, charging a sample to its innermost mod frame
- a failed compare shows its error, and the compared capture is closed when replaced
- Hot shows milliseconds rather than calling them percent; Tree bars share one scale
- tick details line up; wasm replies are typed structs; launcher lock is current

Polish follows docs/design-system.md: Geist, list rows with aligned number columns, one accent, one sticky tab row, a real opening screen, focus states and reduced motion. Panels share one query hook, and dead icons, strings and exports are removed.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-08 19:00:20 +03:00
Claude 5ea2d3c0fb Perf web UI: host-machine tab, new profile button, drop filename/top drop title
Co-Authored-By: Claude Opus 4.7 <[email protected]>
2026-11-08 16:20:42 +01:00
Claude be3db9da27 Perf platform web UI: complete UI1-UI7 client and U1/U2 upload API
Fan out implementation of the remaining perf-platform web UI pieces and upload API:
health/metadata, call tree/group-by/select/detail/search, flat/mods/merge/info,
flame graph and ticks tab, compare/GPU/export/copy/share/open links, and
upload route handlers, S3 storage, validation, share pin, sweep, rate limit.

Co-Authored-By: Claude Opus 4.7 <[email protected]>
2026-11-08 16:12:05 +01:00
Claude 0d6d1f1f66 Perf platform wasm + web UI updates
Co-Authored-By: Claude Code <[email protected]>
2026-11-08 15:43:10 +01:00
Claude bc4559b024 Improve web/perf UI: tabs, paste, panels, theme, typography
Co-Authored-By: Claude Code <[email protected]>
2026-11-08 15:42:48 +01:00
umcekoandClaude Opus 5.5 993525d333 Merge upstream main into perf-platform
Upstream moved Game::shut_down and idle to game/window_handler.rs and loading_frame and start_game to game/loading_window.rs; finish_profile and the benchmark set and profile fields move with them.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-08 10:14:49 +03:00
Umut Cevdet Koçak 3bd70dbc06 Merge pull request #255 from UMCEKO/plat/L1
Perf platform L1: retire the slow-tick span log lines; docs point to captures
2026-11-08 07:59:40 +03:00
umcekoandClaude Sonnet 5.5 0b4c261d0c Perf platform L1: keep per-event attribution as a log line, pacing_bench drops pre-capture frames
The slow ring no longer shows up as span lists in the log, but whose packets or events took a tick
stay in the server owner's log as a separate line. pacing_bench keeps only frames over the threshold
recorded during the capture.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-08 07:53:56 +03:00
umcekoandClaude Sonnet 5.5 caac772100 Perf platform L1: slow-tick and slow-frame span lists leave the log; pacing_bench reads captures; docs point to captures
The slow ring and the captures keep the steps of slow ticks and frames; the notes channel that only fed the log line (event timings, packet notes, spawn counters) goes, and the spawn counters move to the counter registry.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-08 07:36:33 +03:00
Umut Cevdet Koçak 1d7e15c969 Merge pull request #254 from UMCEKO/plat/R1b
Perf platform R1b: rate-limited transfer of the server's capture into the client's file
2026-11-08 07:12:22 +03:00
umcekoandClaude Sonnet 5.5 68685775d9 Perf platform R1b: rate-limited transfer of the server's capture into the client's file
The server sends its finished file in 32 KiB chunks over horizon:perf, at most 256 KiB/s and not while the guest has a backlog. The client merges it field by field into its own stream (PartMerger, no decode), or saves it unchanged for profile server alone; a cut transfer ends the file with SERVER_PART_INCOMPLETE.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-08 07:07:33 +03:00
Umut Cevdet Koçak e87894a81d Merge pull request #253 from UMCEKO/plat/R1a
Perf platform R1a: profile a remote Horizon server from a client (channel, level 4, server copy)
2026-11-08 06:42:23 +03:00
umcekoandClaude Sonnet 5.5 35d8ee793f Perf platform R1a: profile a remote Horizon server from a client over horizon:perf
A client asks a Horizon server (LAN host, dedicated server) to capture its side: operator level 4, one capture at a time, the server logs who profiled and keeps its copy in reports/. profile server start|stop work on a joined server; a bare profile also asks and a refusal is a warning in the client file.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-08 06:35:26 +03:00
Umut Cevdet Koçak 78caa5b352 Merge pull request #252 from UMCEKO/plat/C2
Perf platform C2: single player captures both sides; horizon-ctl profile server
2026-11-08 06:05:14 +03:00
umcekoandClaude Sonnet 5.5 3597945405 Perf platform C2: the gpu e2e checks read the client side's options
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-08 06:05:00 +03:00
umcekoandClaude Sonnet 5.5 87623d9714 Perf platform C2: fit the server capture options and tests to the gpu and timeline pieces after the rebase
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-08 06:00:17 +03:00
umcekoandClaude Sonnet 5.5 71a09a89c0 Perf platform C2: every mark reply is on the game clock; start lists the sides asked for
A server-only mark replied capture-relative and a joined one on the game clock; both now use the game clock.
The e2e checks the server mark against the client mark's game-clock t.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-08 05:54:54 +03:00
umcekoandClaude Sonnet 5.5 18c344ff3c Fix profile mark and status routing for joined and finished server captures
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-08 05:54:54 +03:00
umcekoandClaude Sonnet 5.5 1b28cc6feb Perf platform C2: e2e reads the client side by name, docs say what stop and mark do per side
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-08 05:54:41 +03:00
umcekoandClaude Sonnet 5.5 af837708aa Perf platform C2: single player captures both sides; profile server
The integrated server thread writes its side (kind INTEGRATED) into the window's capture file, on the same clock, and a bare profile means both. profile server writes the server thread's own file.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-08 05:54:41 +03:00
Umut Cevdet Koçak ba04e6991d Merge pull request #251 from UMCEKO/plat/E1
Perf platform E1: horizon-perf export to pprof, folded and Perfetto
2026-11-08 05:47:20 +03:00
umcekoandClaude Sonnet 5.5 56cf9c3301 Perf platform E1: typed pprof labels and sample states, finer pprof period, two-side export tests
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-08 05:46:48 +03:00
umcekoandClaude Sonnet 5.5 05925da3cd Perf platform E1: horizon-perf export to pprof, folded stacks and a Perfetto trace
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-08 05:46:48 +03:00
Umut Cevdet Koçak a072c2275f Merge pull request #250 from UMCEKO/plat/B1
Perf platform B1: benchmark reports in the capture, horizon-perf attach, server_bench --profile
2026-11-08 05:42:43 +03:00
umcekoandClaude Sonnet 5.5 4e27118c3b Perf platform B1: fit the bench op and stop_with to the gpu piece after the rebase
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-08 05:41:55 +03:00
umcekoandClaude Sonnet 5.5 415797f30a perf: keep local paths, label and bot errors out of captures
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-08 05:37:05 +03:00
umcekoandClaude Sonnet 5.5 c283f0ce85 Perf platform B1: benchmark reports in the capture, horizon-perf attach, --benchmark --profile, server_bench --profile
Adds BenchResult (field 21) and Attachment (field 22) to the capture contract. horizon-perf reads bare benchmarks/*.json as one-sided captures, answers query bench and prints bench lines in the summary, and attach appends an attachment on the wire bytes so unknown fields survive. The game's --benchmark --profile records a capture and embeds the report through profile stop --bench; server_bench.py --profile records Horizon's measure window with horizon-report and attaches the run record.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-08 05:37:03 +03:00
Umut Cevdet Koçak 5bc42b54cf Merge pull request #249 from UMCEKO/plat/T1
Perf platform T1: timeline in the capture; HORIZON_TRACE and trace write .hzperf
2026-11-08 05:35:15 +03:00
umcekoandClaude Sonnet 5.5 980844073d Merge fix: drop the needless options update after rebasing T1 onto G1
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-08 05:35:00 +03:00
umcekoandClaude Sonnet 5.5 780a965ae2 Fix T1 review findings: trace seconds range, TLS-safe span, docs
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-08 05:29:27 +03:00
umcekoandClaude Sonnet 5.5 7b7afe9a8f Perf platform T1: timeline chunks in the capture; HORIZON_TRACE, trace and horizon-trace write .hzperf
The Chrome-trace writer in spans.rs is replaced by per-thread buffers drained once a second into
TimelineChunk (field 15), capped at 60 s or 256 MiB with timeline_stopped_t and a warning.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-08 05:29:18 +03:00
Umut Cevdet Koçak dc191e58df Merge pull request #248 from UMCEKO/plat/G1
Perf platform G1: GPU pass timings and counters in the capture
2026-11-08 05:09:48 +03:00
umcekoandClaude Sonnet 5.5 9f08382773 Perf platform G1: GPU pass timings and counters in the capture
Adds the gpu field (GpuPasses per timer) to the contract, written from the
renderer's PassTiming readings at stop; profile start --gpu[=detail], the
HORIZON_SHADER_PACK_TIMING and HORIZON_GPU_PASS_DETAIL aliases, horizon-perf
query gpu with fragments_per_pixel, a summary line and a meta count.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-08 05:01:54 +03:00
Umut Cevdet Koçak b787473097 Merge pull request #247 from UMCEKO/plat/C1
Perf platform C1: client captures: horizon-ctl profile client, F3+O, HORIZON_PROFILE
2026-11-08 04:31:55 +03:00
umcekoandClaude Sonnet 5.5 89feab9f8f Perf platform C1: adapt the client capture to the span, world and net recorders
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-08 04:31:42 +03:00
umcekoandClaude Sonnet 5.5 dd5fd7e80d Perf platform C1: fix review findings
One-line docstring and debug-key comment, F3+O feedback decided by who pressed stop, hardware described off the render thread, and the gauge probe spawned on the first frame with this second's counts.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-08 04:25:38 +03:00
umcekoandClaude Sonnet 5.5 e715b5b743 Perf platform C1: client captures with horizon-ctl profile, F3+O and HORIZON_PROFILE
The window's frames feed an always-on client health history and slow ring.
profile start|stop|status|mark (and F3+O, HORIZON_PROFILE=DIR) write a .hzperf
into reports/ in the game home, with the render thread's samples, retention and
the translated saved message. The capture session is shared with the dedicated
server's through a Role; the gauge probes run off the frame thread.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-08 04:25:38 +03:00
Umut Cevdet Koçak 4dbc406616 Merge pull request #246 from UMCEKO/plat/S1b
Perf platform S1b: spans, workers, net and world move from report.rs into the capture
2026-11-08 03:56:40 +03:00
umcekoandClaude Sonnet 5.5 c5a342f764 Fix S1b review: rustfmt the rule asserts, check only the report's span bit in the drop test
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-08 03:52:19 +03:00
umcekoandClaude Sonnet 5.5 6efe76a432 Fix S1b review: spans and job stats off when a recording drops, rule and option tests, far ring 0 when off
A Recording now turns spans and job stats off when it is dropped, so a capture that failed
to start or a shutdown no longer leaves them on. A leaver's unread bytes are lost (noted on
TrafficRecorder). The e2e script's docstring is one line and its build command is in
docs/perf-platform.md, which also says the privacy checks' split and the dropped between-tick spans.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-08 03:52:18 +03:00
umcekoandClaude Sonnet 5.5 958af813dd Perf platform S1b: spans, workers, net and world move from report.rs into the capture
The dedicated server's capture now carries span_blocks (5 s blocks with parents), the spans
summary (totals, histogram, 16 worst ticks with their trees), workers, per-player net bytes by
join order and the world at start and stop with sampled mob costs. perf-*.json and .txt are no
longer written. capture.proto gains the game_rules and horizon_options allowlists and the mob
timing overhead fields. Spans recorded between ticks no longer count in a tick.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-08 03:52:18 +03:00
Umut Cevdet Koçak da474503d8 Merge pull request #245 from UMCEKO/plat/M1
Perf platform M1: horizon-perf mcp server
2026-11-08 03:38:50 +03:00
umcekoandClaude Sonnet 5.5 50386e55df horizon-perf: mcp server with perf_summary, perf_query, perf_diff, perf_list and perf_follow (M1)
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-08 03:36:48 +03:00
Umut Cevdet Koçak fc1972ffe7 Merge pull request #244 from UMCEKO/plat/Q2
Perf platform Q2: sample queries hot, tree, callers, callees, split, group, frames
2026-11-08 03:28:49 +03:00
umcekoandClaude Sonnet 5.5 05b2a7afee perf Q2: split t:<thread> uses that thread's units; frames warns on mixed threads; real-writer smoke test
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-08 03:24:19 +03:00
umcekoandClaude Sonnet 5.5 4e71e99f98 Perf platform Q2: sample queries hot, tree, callers, callees, split, group, frames; hot functions in the summary
Reads the samples section: functions merged by normalized name, ids over the
normalized name and call path, filters (thread, select, slow, where, window),
several captures for frames, meta counts samples, summary gains hot functions
and the entity/block/chunk splits.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-08 03:15:17 +03:00
Umut Cevdet Koçak 8fc4c27855 Merge pull request #243 from UMCEKO/plat/S2
Perf platform S2: sampler call trees in the capture; horizon-report samples by default
2026-11-08 02:58:17 +03:00
umcekoandClaude Sonnet 5.5 35ddec5e9e Fix S2 review findings: one sample node per (parent, function), one-line doc, sampler share test, design doc notes
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-08 02:30:00 +03:00
umcekoandClaude Sonnet 5.5 9102edfb56 Perf platform S2: sampler call trees in the capture; horizon-report samples by default
The samples field (SampleProfile) is built from horizon_threads::Profile in
horizon-client-render: a deduplicated function table, a call-node chain per
stack, a context table and weights per thread, state, slow flag and time block.
The sampler's aggregate gains time blocks. horizon-report samples the tick
thread on Linux unless --no-samples, and finishes the file off the tick thread.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-08 02:04:12 +03:00
Umut Cevdet Koçak be4be0ce6b Merge pull request #242 from UMCEKO/plat/S1a
Perf platform S1a: dedicated servers: horizon-report writes .hzperf
2026-11-08 01:00:06 +03:00
umcekoandClaude Sonnet 5.5 d7b9e267e0 Fix S1a review: PERF_UI_URL points at /perf, note report output in hosting docs
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-08 00:53:36 +03:00
umcekoandClaude Sonnet 5.5 769cdcd232 Fix S1a review findings: one-line comments, unknown slow-tick cpu is 0, empty capture is incomplete, report checks completion only after the window, record files exception in the design doc
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-08 00:53:35 +03:00
umcekoandClaude Sonnet 5.5 3d9ef21f68 Perf platform S1a: horizon-report writes a .hzperf on the dedicated server
The always-on server health history and slow ring feed a capture file
(Start, history, slow_history, health, counters, slow, Stop) in reports/,
with retention of 30 files or 1 GiB that never touches renamed files.
horizon-report accepts --no-samples and --profile; horizon-server report
prints the saved path and the web UI address.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-08 00:53:35 +03:00
Umut Cevdet Koçak 6634e4ba98 Merge pull request #241 from UMCEKO/plat/typed
horizon-perf: every request, reply and vocabulary is typed
2026-11-08 00:33:32 +03:00
umceko ad01e692c9 horizon-perf: every request, reply and vocabulary is typed
Replies are serde structs (one Envelope<R>, a row struct per op, typed meta, watch events, diff and bench diff) and
closed vocabularies are enums (Op, Select, ContextKey, GroupBy, Unit, Verdict, HealthField, Flag, the proto name enums).
The CLI parses strings into them at the edge; no serde_json::Value, json! or string-keyed match is left in the library.

The JSON is the same as before as parsed JSON (tests/golden holds 49 cases captured from the old code). Intentional
differences, each visible in the golden diff:
- --where dimension=X is now the key dim, so dim and dimension merge in selection.where
- follow --json spells a stop reason window_cap (it said "window cap"), like query meta
- an unknown --by is a usage error at the edge instead of a world-only query error
- JSON object keys now come out in declaration order, not alphabetical
2026-11-08 00:32:47 +03:00
umceko 1547313299 horizon-perf: the golden follow case includes a window-cap stop 2026-11-08 00:28:45 +03:00
umceko 9874954fd0 horizon-perf: golden outputs of every query, diff and follow --json case, from before the typing refactor 2026-11-08 00:18:35 +03:00
Umut Cevdet Koçak 2e68167379 Merge pull request #240 from UMCEKO/plat/D1
Perf platform D1: horizon-perf diff with block statistics, and summary --baseline
2026-11-07 23:57:34 +03:00
umcekoandClaude Sonnet 5.5 14f833917a horizon-perf: diff with two-sample block statistics, drift warning, summary --baseline, benchmark report ab (D1)
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-07 23:54:36 +03:00
Umut Cevdet Koçak 9ca66bdb02 Merge pull request #239 from UMCEKO/plat/Q1
Perf platform Q1: horizon-perf queries without samples, stable ids and filters
2026-11-07 23:44:15 +03:00
umcekoandClaude Sonnet 5.5 0da566c2ed horizon-perf: summary sections, filters and tests for the span queries (Q1)
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-07 23:43:16 +03:00
umcekoandClaude Sonnet 5.5 8883d77504 horizon-perf: span, tick, series, world, net, meta, memory, mark and warning queries with stable ids (Q1)
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-07 23:43:16 +03:00
Umut Cevdet Koçak 838d140cbc Merge pull request #238 from UMCEKO/plat/F2
Perf platform F2: health recorder, slow ring and counter registry
2026-11-07 23:37:30 +03:00
umcekoandClaude Sonnet 5.5 d2825d34f8 Perf platform F2: health recorder tiers, slow ring and counter registry
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-07 23:35:06 +03:00
Umut Cevdet Koçak e6f80a4ee4 Merge pull request #237 from UMCEKO/plat/D
Perf platform D: alloc-count, per-thread allocation counts in the frame stats
2026-11-07 23:30:32 +03:00
umcekoandClaude Sonnet 5.5 1d4d4feaf5 alloc-count: ON is false with system-allocator; rustfmt
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-07 23:25:49 +03:00
umcekoandClaude Sonnet 5.5 ff458a358d alloc-count: one-line comments, rustfmt
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-07 23:25:49 +03:00
umcekoandClaude Sonnet 5.5 c61ca75ca4 Add alloc-count: per-thread allocation counts, and alloc in the frame stats
horizon-threads gets a feature alloc-count (alloc-census implies it) with
CountingAllocator: each thread owns a cache-line slot of four counters, so a
block costs two plain load/store bumps. GameAllocator picks the census, the
counting mimalloc or plain mimalloc, and main.rs and the scene_replay example
declare it; without the feature the allocator is unchanged. The census counts
the same way.

In a counting build stats, bench and fly reply with alloc (window-thread
allocations and KiB per frame, all-thread rates, net growth), and stats raw and
fly raw add alloc_count and alloc_bytes beside frame_ms. Other builds reply
null. docs/memory.md gets the section with the measured cost.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-07 23:25:49 +03:00
umcekoandClaude Sonnet 5.5 cc52c95887 Docs: memory.md says what holds the game's memory and what the heap census costs
Describes only what exists: horizon-ctl memory and what each number means, the
alloc-census build (how tags are chosen, what it costs: +1.8% resident, frame
CPU time within the run-to-run spread, measured with --benchmark standard),
spans/trace/stats, resident memory in benchmark reports, scene_replay --memory
and the system-allocator feature for heaptrack on small probes.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-07 23:25:40 +03:00
Umut Cevdet Koçak e0f6c4f5e8 Merge pull request #236 from UMCEKO/plat/H
Perf platform H: in-process sampling profiler with symbolized function trees
2026-11-07 23:19:38 +03:00
umcekoandClaude Sonnet 5.5 91813c5ae0 horizon-threads: in-process symbolization and merged function trees for the sampler's aggregates
symbols.rs resolves addresses against this process's own binaries (DWARF through addr2line, the ELF symbol table for full paths), keeping base file names only. tree.rs names the aggregate's trie, merges it into a function tree and ranks functions by own and total samples. The pprof, folded and HTML writers stay out; the capture's samples field replaces them. Also fixes the spans test's work:: path and a clippy as_chunks lint.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-07 23:16:34 +03:00
umcekoandClaude Sonnet 5.5 a8a0d2cc25 horizon-threads: a profile session drains the sampler into bounded aggregates, with wall-clock samples
`profile::start` runs a writer thread that drains the sampler's rings every 100 ms, unwinds the stack copies there and folds each record
into a call-stack trie (131k nodes at most) plus counters per stack and context (span, entity type, chunk, block, dimension, slow tick).
A tick marker decides afterwards whether the samples of that tick count as slow, so the report can keep only the ticks over a threshold.
Tick durations go to a fixed log-bucket histogram per thread with the ten worst ticks. `Session::status` reads progress while it runs.

`Config::wall` arms a second timer per thread on the monotonic clock, signalling SIGRTMIN so it queues instead of merging with the
CPU-time SIGPROF; each handler blocks the other, so they never nest on one ring. A wall sample is marked running when the thread's CPU
clock advanced by more than half the period since its previous wall sample (the handlers' own microseconds don't count).

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-07 23:16:33 +03:00
umcekoandClaude Sonnet 5.5 8e88e778a6 Profiler: the game loops store what they are doing in the sampler's context
The innermost span (spans::guard), the entity type before each mob, item,
cart, projectile and block entity tick, the chunk and block of random and
scheduled block ticks, the chunk in natural spawning, the level's dimension,
and a tick marker with its duration at the end of each server tick and each
client frame. Plain stores into the sampler's thread-local; the new
horizon_threads::profile module makes them no-ops where the sampler does not
run and interns the names behind the ids.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-07 23:16:33 +03:00
umcekoandClaude Sonnet 5.5 7b5cf13614 horizon-threads: the sampler unwinds copies of the stack with .eh_frame when frame pointers are off
Frame pointers cost 29-34% world load in release builds, so the release build keeps them off and the SIGPROF handler now copies
the interrupted registers and the top 16 KiB of the thread's stack (clamped to its bounds) into a byte ring, memcpy only.
`Session::drain`, on the caller's thread, unwinds each copy with the loaded objects' `.eh_frame` tables. The frame-pointer walk
stays in the handler as the fast path when this build keeps frame pointers (`Unwind::Auto`, decided at start from a probe).
Ring entries are variable length; a ring holds two seconds of samples by default, and a full one drops and counts as before.

New dependencies (Linux only): `framehop` 0.16 (MIT/Apache-2.0, samply's unwinder; default features off, so no Mach-O/PE
parsers) unwinds from `.eh_frame`, and `object` 0.39 locates the sections in each loaded ELF. There is no in-tree unwinder, and
writing a CFI interpreter is the alternative. Both share one `gimli` 0.33, which was not in the lock before.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-07 23:16:33 +03:00
umcekoandClaude Sonnet 5.5 1514d3d16a horizon-threads: a frame-pointer unwinder and a Linux CPU-time stack sampler
`unwind` walks frame pointers within a thread's stack bounds (shared with the allocation profiler). `sampler` arms a per-thread
CPU-time POSIX timer that signals SIGPROF to its own thread; the async-signal-safe handler copies the thread's context and
stack into a preallocated ring, and `tick_end` writes tick-duration markers so a report can keep only slow ticks.
Nothing is installed or armed until `sampler::start`. Frame pointers stay off in `.cargo/config.toml`: the benchmark
shows world load +29% (7 of 8 pairs) with `-C force-frame-pointers=yes`.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-07 23:16:32 +03:00
Umut Cevdet Koçak 3e093c2c6f Merge pull request #235 from UMCEKO/plat/C
Perf platform C: GPU pass timing
2026-11-07 23:16:19 +03:00
umcekoandClaude Sonnet 5.5 e9db26a19a Shorten the detail test doc to one line
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-07 23:10:35 +03:00
umcekoandClaude Sonnet 5.5 267d39c511 GPU pass timing: fix review findings (docs on tables, uploads entry, phase 2 test)
step moves above tables' doc comment. The docs now say GPU culling uploads is a pass entry of its
own, only in detail frames on devices with encoder timestamps, not a part of GPU culling. A new
GPU test times both phases and the uploads and checks detail leaves the counts and draws as an
untimed frame's; the test device now asks for SHADER_EARLY_DEPTH_TEST so phase 2 can run.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-07 23:10:35 +03:00
umcekoandClaude Sonnet 5.5 3b3de54fa3 GPU pass timing: where detail shows (gpu-passes detail, F3, scene_replay)
`horizon-ctl gpu-passes detail` turns on dispatch times and, on Vulkan, per-pass counters;
stats and bench add `gpu_pass_detail`, `gpu_pass_parts` and `gpu_counters`, F3 lists the parts
and the six passes with the most fragments, and `scene_replay --gpu-detail on` reports both per
view after the timed frames. RADV counts fragments per 2x2 quad, so a calibration draw at setup
finds each device's `fragment_unit` and `fragments_per_pixel` is computed from it.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-07 23:10:35 +03:00
umcekoandClaude Sonnet 5.5 a1cf28d100 GPU pass timing: pipeline statistics on the world passes
`gpu-passes detail` frames now alternate between a parts frame (times) and a counters frame
(vertex, clipper and fragment shader invocations per world pass, split into segments such as
before translucent, occlusion boxes and translucent terrain). A counters frame throws its times
away, and CountedPass ends its query on drop, so none stays open at the end of a pass. Vulkan
only; a device without pipeline statistics keeps parts frames.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-07 23:10:35 +03:00
umcekoandClaude Sonnet 5.5 517546d1af GPU pass timing: dispatch parts inside a pass, and GPU culling's dispatches and uploads
`gpu-passes detail` (the renderer's `set_pass_detail`, or HORIZON_GPU_PASS_DETAIL) times each
dispatch of GPU culling (apply, cull, scan, scatter; count2, scan2, scatter2) and its uploads
plus clear_buffer, so a slow pass can be split. A timestamp pair brackets each part inside one
compute pass where the device has TIMESTAMP_QUERY_INSIDE_PASSES, else each part gets a pass of
its own. Off by default; `on` records the same queries and passes as before.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-07 23:10:35 +03:00
umcekoandClaude Opus 5.5 cb7a5c204e gpu-passes: time the frames the submit thread submits too
land_frame told the GPU timer and a shader pack's pass timer that their frame
was submitted, but not Horizon's own pass timer: with the submit thread on, its
first timed frame was read and every later one waited forever, so gpu-passes,
F3 and scene_replay showed the first frame's pass times only.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-07 23:10:35 +03:00
Umut Cevdet Koçak 7833ed6c05 Merge pull request #234 from UMCEKO/plat/P1
Perf platform P1: horizon-perf readers, latest/list/follow and a health-only summary
2026-11-07 23:07:48 +03:00
umcekoandClaude Sonnet 5.5 8b7cc25b66 perf: stream captures in 64 KiB slices, one-line doc, pin gzip trailer and header paths
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-07 23:05:29 +03:00
umcekoandClaude Sonnet 5.5 0689bc6505 horizon-perf: whole-file and follow readers, latest, list, follow and a health-only summary (P1)
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-07 23:01:49 +03:00
Umut Cevdet Koçak e9e7050f32 Merge pull request #233 from UMCEKO/plat/A
Perf platform A: paired A/B statistics and the travel benchmark scenario
2026-11-07 22:51:54 +03:00
umcekoandClaude Sonnet 5.5 afdac528fc Benchmark A/B: drop the references to the removed bench_stats.py
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-07 22:42:49 +03:00
umcekoandClaude Sonnet 5.5 875c7d7a2b Benchmark: the travel scenario, --benchmark-set and bench raw
horizon-client --benchmark travel flies 512 blocks at 16 blocks/s with the time
and weather frozen, then holds at the end for four blocks: the renderer after
travel, where GPU culling's slowdown shows. Scenario::parse replaces scenes();
--benchmark-set SWITCH=VALUE sets one of the SWITCHES (defaults as gfx has them)
before the world opens, and a refused switch ends the run with the game's
reason. The bench control word takes raw like stats and fly, and the report gets
gpu.<phase>.p50_ms and cpu.<phase>.p50_ms for the trip and hold phases.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-07 22:30:08 +03:00
umcekoandClaude Sonnet 5.5 9a2931de68 Benchmark report: paired A/B statistics and the optional ab field
horizon_bench::compare pairs the ab.a/ab.b blocks (B A A B order) and gives the
Hodges-Lehmann effect, the exact Wilcoxon signed-rank interval, a verdict, the
blocks' noise and the first-block flag. tools/bench_stats.py computes the same
numbers in Python and checks a report's stored result against its frames.
Report gains an optional ab field; the format stays version 1.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-07 22:30:08 +03:00
umcekoandClaude Opus 5.5 52128e9f79 docs: perf platform's gzip writer goes through flate2 (pure Rust), as F1 built it
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-07 22:17:42 +03:00
Umut Cevdet Koçak e635a9b0b4 horizon-perf-format: the .hzperf capture contract (F1) (#232)
horizon-perf-format: the .hzperf capture contract (F1)
2026-11-07 22:17:18 +03:00
umcekoandClaude Opus 5.5 0d21d07827 horizon-perf-format: the .hzperf capture contract (F1)
The new crate holds piece F1 of docs/perf-platform.md: capture.proto (Capture's
top level with Start, Stop, strings, marks, warnings, health, history, the slow
ring and counters), compiled by protox + prost-build in build.rs, and the gzip
writer (flate2, rust_backend) that appends whole top-level fields with a sync
flush after each, on a horizon-perf-writer thread fed by a 4,096-field bounded
channel that drops and counts instead of waiting.

Tests: the frozen tests/fixtures/v1.hzperf decodes to its listed values; after
every flush the file on disk inflates to exactly the fields written; a file cut
at random points yields only whole fields, in order; unknown field numbers are
skipped; the descriptor lint passes capture.proto and rejects a missing
_UNSPECIFIED, a JSON-name collision and a group; a full channel drops, counts
and never waits, and Stop records the drops.

Fixes #2

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-07 22:16:46 +03:00
umcekoandClaude Opus 5.5 b91b5516f3 fork: /claim bot and review batches for the perf platform fork
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-07 20:31:53 +03:00
umceko be1de92536 Merge branch 'docs/perf-platform-spec' into perf-platform 2026-11-07 20:28:31 +03:00
umcekoandClaude Opus 5.5 b3be633edf docs: perf platform tracker lives on UMCEKO/horizon#1
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-07 20:22:26 +03:00
umcekoandClaude Opus 5.5 ae1555ab94 docs: perf platform pieces have no PR size limit; tracker #11902
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-07 20:17:03 +03:00
umcekoandClaude Opus 5.5 ee3a790501 docs: design spec for one performance platform (.hzperf, horizon-perf, web UI)
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-07 20:10:46 +03:00
347 changed files with 54114 additions and 1779 deletions

No files matched your search

+4
View File
@@ -0,0 +1,4 @@
# `cargo xtask …` runs the repository's task crate, so the commands in docs/perf-platform.md and
# web/perf/README.md work as written. `cargo run -p xtask -- …` stays equivalent.
[alias]
xtask = "run --package xtask --"
+56
View File
@@ -0,0 +1,56 @@
name: perf-wasm
# The site consumes this reader bundle from horizon-rust/horizon-site.
on:
push:
branches: [main]
paths:
- 'crates/horizon-perf/**'
- 'crates/horizon-perf-format/**'
- 'crates/horizon-bench/**'
- 'xtask/**'
- 'Cargo.lock'
- 'Cargo.toml'
- '.github/workflows/perf-wasm.yml'
pull_request:
types: [opened, synchronize, reopened]
paths:
- 'crates/horizon-perf/**'
- 'crates/horizon-perf-format/**'
- 'crates/horizon-bench/**'
- 'xtask/**'
- 'Cargo.lock'
- 'Cargo.toml'
- '.github/workflows/perf-wasm.yml'
workflow_dispatch:
permissions:
contents: read
concurrency:
group: perf-wasm-${{ github.event.pull_request.number || github.sha }}
cancel-in-progress: ${{ github.event_name == 'pull_request' }}
jobs:
build:
runs-on: ubuntu-latest
timeout-minutes: 30
env:
HORIZON_TELEMETRY_URL: "off"
steps:
- uses: actions/checkout@v7
- uses: dtolnay/rust-toolchain@stable
with:
targets: wasm32-unknown-unknown
- uses: Swatinem/rust-cache@v2
with:
key: perf-wasm
# The agreement test: the C ABI, called natively, must print the bytes the CLI prints.
- name: The wasm ABI agrees with the CLI
run: cargo test -p horizon-perf --features wasm --test wasm
- name: Build the wasm the page loads
run: cargo run -p xtask -- perf-wasm
- uses: actions/upload-artifact@v4
with:
name: perf-wasm
path: target/perf-wasm
if-no-files-found: error
+1
View File
@@ -102,6 +102,7 @@ Game workspace (root `Cargo.toml`); `launcher/` and `crates/horizon-telemetry-se
| `horizon-shader-api` | Horizon's own WGSL shader packs ([docs/api/shaders.md](docs/api/shaders.md)): `horizon-shaders.toml`, the `.hshader` package; shared by the game, the launcher and the registry |
| `horizon-shaders` | Shader packs (the `shaders/` folder format), rebuilt clean-room from a written spec of the format; no wgpu |
| `horizon-theme` | Design tokens and fonts for the launcher, the game's own menus and the website |
| `horizon-perf-format` | The `.hzperf` capture contract ([docs/perf-platform.md](docs/perf-platform.md) §3): `proto/horizon/perf/v1/capture.proto`, its generated types, the gzip writer and its thread. No Horizon dependencies; nothing in the game reads a capture |
| `horizon-telemetry`, `horizon-telemetry-server`, `horizon-bench` | Opt-in data collection, its ingest server, the benchmark report format |
| `horizon-system` | The machine the game runs on (OS, CPU, memory, drive, power, GPU maker and memory), for the crash report, the renderer's GPU info and telemetry |
| `horizon-platform` | The platform's public API: what the Horizon layer may use of the engine ([docs/layers.md](docs/layers.md)); re-exports `horizon-system` today |
Generated
+341 -3
View File
@@ -18,6 +18,15 @@ version = "0.1.10"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "366ffbaa4442f4684d91e2cd7c5ea7c4ed8add41959a31447066e279e432b618"
[[package]]
name = "addr2line"
version = "0.26.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "59317f77929f0e679d39364702289274de2f0f0b22cbf50b2b8cff2169a0b27a"
dependencies = [
"gimli",
]
[[package]]
name = "adler2"
version = "2.0.1"
@@ -129,6 +138,12 @@ dependencies = [
"libc",
]
[[package]]
name = "anyhow"
version = "1.0.104"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "330a5ed07fa54e4702c9d6c4174f74427fc0ef6e214bbd677ae50a5099946470"
[[package]]
name = "arboard"
version = "3.7.0"
@@ -264,6 +279,12 @@ version = "1.8.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2af50177e190e07a26ab74f8b1efbfe2ef87da2116221318cb1c2e82baf7de06"
[[package]]
name = "beef"
version = "0.5.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3a8241f3ebb85c056b509d4327ad0358fbbba6ffb340bf388f26350aeda225b1"
[[package]]
name = "bit-set"
version = "0.10.0"
@@ -559,7 +580,7 @@ checksum = "af491d569909a7e4dee0ad7db7f5341fef5c614d5b8ec8cf765732aba3cff681"
dependencies = [
"serde",
"termcolor",
"unicode-width",
"unicode-width 0.2.2",
]
[[package]]
@@ -1011,6 +1032,12 @@ version = "0.1.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7360491ce676a36bf9bb3c56c1aa791658183a54d2744120f27285738d90465a"
[[package]]
name = "fastrand"
version = "2.5.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "da7c62ceae207dd37ea5b845da6a0696c799f85e97da1ab5b7910be3c1c80223"
[[package]]
name = "fdeflate"
version = "0.3.7"
@@ -1109,6 +1136,18 @@ dependencies = [
"percent-encoding",
]
[[package]]
name = "framehop"
version = "0.16.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6f54fe4785e899d4d6f43793b151c63c5647240fc630b005509d2614a939f693"
dependencies = [
"arrayvec",
"cfg-if",
"fallible-iterator",
"gimli",
]
[[package]]
name = "fs4"
version = "1.1.0"
@@ -1246,6 +1285,15 @@ dependencies = [
"windows 0.62.2",
]
[[package]]
name = "gimli"
version = "0.33.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0bf7f043f89559805f8c7cacc432749b2fa0d0a0a9ee46ce47164ed5ba7f126c"
dependencies = [
"stable_deref_trait",
]
[[package]]
name = "gl_generator"
version = "0.14.0"
@@ -1426,7 +1474,7 @@ dependencies = [
name = "horizon-bench"
version = "0.1.0"
dependencies = [
"horizon-telemetry",
"horizon-system",
"serde",
"serde_json",
]
@@ -1482,6 +1530,8 @@ dependencies = [
"horizon-mods",
"horizon-narrator",
"horizon-optifine",
"horizon-perf",
"horizon-perf-format",
"horizon-protocol",
"horizon-server-files",
"horizon-shader-api",
@@ -1561,6 +1611,8 @@ dependencies = [
"horizon-mods",
"horizon-narrator",
"horizon-optifine",
"horizon-perf",
"horizon-perf-format",
"horizon-protocol",
"horizon-server-files",
"horizon-shader-api",
@@ -1615,6 +1667,8 @@ dependencies = [
"horizon-items",
"horizon-narrator",
"horizon-optifine",
"horizon-perf",
"horizon-perf-format",
"horizon-polytone",
"horizon-protocol",
"horizon-shader-api",
@@ -1832,6 +1886,7 @@ dependencies = [
"horizon-game",
"horizon-heartbeat",
"horizon-official-servers",
"horizon-perf-share",
"horizon-platform",
"log",
"winresource",
@@ -1856,12 +1911,54 @@ dependencies = [
"regex",
]
[[package]]
name = "horizon-perf"
version = "0.1.0"
dependencies = [
"flate2",
"horizon-bench",
"horizon-perf-format",
"horizon-perf-share",
"prost",
"regex",
"serde",
"serde_json",
]
[[package]]
name = "horizon-perf-format"
version = "0.1.0"
dependencies = [
"flate2",
"prost",
"prost-build",
"prost-types",
"protox",
]
[[package]]
name = "horizon-perf-share"
version = "0.1.0"
dependencies = [
"getrandom 0.3.4",
"horizon-addresses",
"horizon-platform",
"log",
"open",
"serde",
"serde_json",
"ureq",
"url",
]
[[package]]
name = "horizon-platform"
version = "0.1.0"
dependencies = [
"horizon-bench",
"horizon-core",
"horizon-hooks",
"horizon-perf-format",
"horizon-prepare",
"horizon-protocol",
"horizon-server-files",
@@ -1946,7 +2043,6 @@ dependencies = [
"chrono",
"getrandom 0.3.4",
"horizon-anticheat",
"horizon-bench",
"horizon-chat-filter",
"horizon-mcp",
"horizon-platform",
@@ -2030,9 +2126,14 @@ version = "0.1.0"
name = "horizon-threads"
version = "0.1.0"
dependencies = [
"addr2line",
"framehop",
"libc",
"libmimalloc-sys",
"memmap2",
"mimalloc",
"object",
"rustc-demangle",
"windows-sys 0.61.2",
]
@@ -2042,6 +2143,7 @@ version = "0.1.0"
dependencies = [
"flate2",
"horizon-core",
"horizon-threads",
"libm",
"log",
"lz4_flex",
@@ -2283,6 +2385,15 @@ dependencies = [
"once_cell",
]
[[package]]
name = "itertools"
version = "0.14.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2b192c782037fadd9cfa75548310488aabdbf3d2da73885b31bd0abd03351285"
dependencies = [
"either",
]
[[package]]
name = "itoa"
version = "1.0.18"
@@ -2509,6 +2620,72 @@ version = "0.4.34"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f9f8bd3e56ce4dfc153cf470fffbfa98c7620958b312ca5c3a4b8d5181fd13c6"
[[package]]
name = "logos"
version = "0.15.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ff472f899b4ec2d99161c51f60ff7075eeb3097069a36050d8037a6325eb8154"
dependencies = [
"logos-derive 0.15.1",
]
[[package]]
name = "logos"
version = "0.16.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "eb2c55a318a87600ea870ff8c2012148b44bf18b74fad48d0f835c38c7d07c5f"
dependencies = [
"logos-derive 0.16.1",
]
[[package]]
name = "logos-codegen"
version = "0.15.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "192a3a2b90b0c05b27a0b2c43eecdb7c415e29243acc3f89cc8247a5b693045c"
dependencies = [
"beef",
"fnv",
"lazy_static",
"proc-macro2",
"quote",
"regex-syntax",
"rustc_version",
"syn 2.0.119",
]
[[package]]
name = "logos-codegen"
version = "0.16.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "58b3ffaa284e1350d017a57d04ada118c4583cf260c8fb01e0fe28a2e9cf8970"
dependencies = [
"fnv",
"proc-macro2",
"quote",
"regex-automata",
"regex-syntax",
"syn 2.0.119",
]
[[package]]
name = "logos-derive"
version = "0.15.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "605d9697bcd5ef3a42d38efc51541aa3d6a4a25f7ab6d1ed0da5ac632a26b470"
dependencies = [
"logos-codegen 0.15.1",
]
[[package]]
name = "logos-derive"
version = "0.16.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "52d3a9855747c17eaf4383823f135220716ab49bea5fbea7dd42cc9a92f8aa31"
dependencies = [
"logos-codegen 0.16.1",
]
[[package]]
name = "lru"
version = "0.18.5"
@@ -2551,6 +2728,28 @@ dependencies = [
"libc",
]
[[package]]
name = "miette"
version = "7.6.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5f98efec8807c63c752b5bd61f862c165c115b0a35685bdcfd9238c7aeb592b7"
dependencies = [
"cfg-if",
"miette-derive",
"unicode-width 0.1.14",
]
[[package]]
name = "miette-derive"
version = "7.6.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "db5b29714e950dbb20d5e6f74f9dcec4edbcc1067bb7f8ed198c097b8c1a818b"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.119",
]
[[package]]
name = "mimalloc"
version = "0.1.52"
@@ -2604,6 +2803,12 @@ dependencies = [
"windows-sys 0.61.2",
]
[[package]]
name = "multimap"
version = "0.10.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1d87ecb2933e8aeadb3e3a02b828fed80a7528047e68b4f424523a0981a3a084"
[[package]]
name = "multiversion_no_op"
version = "1.0.0"
@@ -3184,6 +3389,15 @@ dependencies = [
"objc2-foundation 0.2.2",
]
[[package]]
name = "object"
version = "0.39.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2e5a6c098c7a3b6547378093f5cc30bc54fd361ce711e05293a5cc589562739b"
dependencies = [
"memchr",
]
[[package]]
name = "oid-registry"
version = "0.8.1"
@@ -3515,6 +3729,16 @@ version = "0.3.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e8cf8e6a8aa66ce33f63993ffc4ea4271eb5b0530a9002db8455ea6050c77bfa"
[[package]]
name = "prettyplease"
version = "0.2.37"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "479ca8adacdd7ce8f1fb39ce9ecccbfe93a3f1344b3d0d97f20bc0196208f62b"
dependencies = [
"proc-macro2",
"syn 2.0.119",
]
[[package]]
name = "proc-macro-crate"
version = "3.5.0"
@@ -3539,6 +3763,96 @@ version = "1.0.18"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3d595e54a326bc53c1c197b32d295e14b169e3cfeaa8dc82b529f947fba6bcf5"
[[package]]
name = "prost"
version = "0.14.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "528ac67416ff8646872a3c02cad9cc4ee5dc9f9540c9b10771855c95cb2e5ae1"
dependencies = [
"bytes",
"prost-derive",
]
[[package]]
name = "prost-build"
version = "0.14.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "03da047801ff44bb6a4d407d4860c05fd70bb81714e6b2f3812603d5b145b042"
dependencies = [
"heck",
"itertools",
"log",
"multimap",
"petgraph",
"prettyplease",
"prost",
"prost-types",
"regex",
"syn 2.0.119",
"tempfile",
]
[[package]]
name = "prost-derive"
version = "0.14.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b570b25f7617e43d59005d0990ccb79e950a423952cea19671b7a876da390adf"
dependencies = [
"anyhow",
"itertools",
"proc-macro2",
"quote",
"syn 2.0.119",
]
[[package]]
name = "prost-reflect"
version = "0.16.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "01b80ea363c31af2de2b92e3c07ed1156628f7838c4afb4df75ee78a37fedbd1"
dependencies = [
"logos 0.16.1",
"miette",
"prost",
"prost-types",
]
[[package]]
name = "prost-types"
version = "0.14.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f94967dc7688f3054c7fac87473ffae4cc4c3904800e2d9f5b857246d8963b0a"
dependencies = [
"prost",
]
[[package]]
name = "protox"
version = "0.9.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4f25a07a73c6717f0b9bbbd685918f5df9815f7efba450b83d9c9dea41f0e3a1"
dependencies = [
"bytes",
"miette",
"prost",
"prost-reflect",
"prost-types",
"protox-parse",
"thiserror 2.0.21",
]
[[package]]
name = "protox-parse"
version = "0.9.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "072eee358134396a4643dff81cfff1c255c9fbd3fb296be14bdb6a26f9156366"
dependencies = [
"logos 0.15.1",
"miette",
"prost-types",
"thiserror 2.0.21",
]
[[package]]
name = "quick-error"
version = "2.0.1"
@@ -3838,6 +4152,12 @@ dependencies = [
"smallvec",
]
[[package]]
name = "rustc-demangle"
version = "0.1.28"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b74b56ffa8bb2830709a538c2cbcae9aa062db0d2a42563bfb09bdaae44020eb"
[[package]]
name = "rustc-hash"
version = "1.1.0"
@@ -4396,6 +4716,18 @@ dependencies = [
"syn 3.0.6",
]
[[package]]
name = "tempfile"
version = "3.27.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "32497e9a4c7b38532efcdebeef879707aa9f794296a4f0244f6f69e9bc8574bd"
dependencies = [
"fastrand",
"once_cell",
"rustix 1.1.5",
"windows-sys 0.61.2",
]
[[package]]
name = "termcolor"
version = "1.4.1"
@@ -4665,6 +4997,12 @@ version = "1.13.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c6f5d3c3b1bf09027a88a6bc961fc00497d651009560b5463668dc81b0fa87a8"
[[package]]
name = "unicode-width"
version = "0.1.14"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7dd6e30e90baa6f72411720665d41d89b9a3d039dc45b8faea1ddd07f617f6af"
[[package]]
name = "unicode-width"
version = "0.2.2"
+3
View File
@@ -37,6 +37,9 @@ members = [
"crates/horizon-discord",
"crates/horizon-official-servers",
"crates/horizon-bench",
"crates/horizon-perf-format",
"crates/horizon-perf",
"crates/horizon-perf-share",
"crates/horizon-friends",
"crates/horizon-relay",
"crates/horizon-mods",
+6
View File
@@ -35,6 +35,12 @@ pub const COSMETICS_API: &str = "https://cosmetics.playhorizon.dev";
/// "I'm playing" ping, benchmark uploads (`HORIZON_TELEMETRY_URL` overrides it).
pub const TELEMETRY: &str = "https://telemetry.playhorizon.dev";
/// The performance viewer page a saved capture opens in (`HORIZON_PERF_VIEWER` overrides it).
pub const PERF_VIEWER: &str = "https://playhorizon.dev/perf";
/// The capture upload API behind Share (`HORIZON_PERF_CAPTURES` overrides it).
pub const PERF_CAPTURES: &str = "https://playhorizon.dev/api/perf/captures";
/// Public STUN servers for Play with Friends' NAT discovery: two independent operators, so one
/// outage doesn't break discovery (`HORIZON_STUN_SERVERS` overrides them).
pub const STUN_SERVERS: &[&str] = &["stun.cloudflare.com:3478", "stun.l.google.com:19302"];
+4 -2
View File
@@ -9,5 +9,7 @@ description = "The benchmark report format (docs/benchmark-report.md), shared by
[dependencies]
serde = { version = "1", features = ["derive"] }
serde_json = "1"
# The hardware probes shared with telemetry (cores, clocks, drive, VRAM, power, form).
horizon-telemetry = { path = "../horizon-telemetry", default-features = false }
# The hardware probes (cores, clocks, drive, VRAM, power, form); a reader of reports (the wasm
# build of horizon-perf) needs the types below, never the probes.
[target.'cfg(not(target_arch = "wasm32"))'.dependencies]
horizon-system = { path = "../horizon-system" }
+434
View File
@@ -0,0 +1,434 @@
//! Paired A/B comparison of benchmark blocks (docs/benchmark-report.md, "A/B runs").
use std::collections::BTreeMap;
use serde::{Deserialize, Serialize};
use crate::{round, Frames};
#[derive(Clone, Copy, Debug, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum Verdict {
TooFewPairs,
WithinNoise,
Negligible,
Slower,
Faster,
}
/// B against A over paired blocks; lower milliseconds are better.
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)]
pub struct Paired {
pub pairs: usize,
pub a: f64,
pub b: f64,
/// Hodges-Lehmann estimate of b - a.
pub diff: f64,
/// Absent when `a` is zero.
pub diff_pct: Option<f64>,
/// Exact Wilcoxon signed-rank interval; absent below 6 pairs.
pub ci95: Option<[f64; 2]>,
pub confidence: Option<f64>,
/// Median distance between neighbouring blocks of the same arm.
pub noise: Option<f64>,
pub verdict: Verdict,
}
type Getter = fn(&Frames) -> Option<f64>;
fn median(values: &[f64]) -> f64 {
let mut v = values.to_vec();
v.sort_by(f64::total_cmp);
match v.len() {
0 => 0.0,
n if n % 2 == 1 => v[n / 2],
n => (v[n / 2 - 1] + v[n / 2]) / 2.0,
}
}
fn round4(v: f64) -> f64 {
(v * 10_000.0).round() / 10_000.0
}
/// `(a, b)` per pair, `noise` left for the caller.
pub fn paired(pairs: &[(f64, f64)]) -> Paired {
let n = pairs.len();
let a = median(&pairs.iter().map(|p| p.0).collect::<Vec<_>>());
let b = median(&pairs.iter().map(|p| p.1).collect::<Vec<_>>());
let d: Vec<f64> = pairs.iter().map(|p| p.1 - p.0).collect();
let mut walsh = Vec::with_capacity(n * (n + 1) / 2);
for i in 0..n {
for j in i..n {
walsh.push((d[i] + d[j]) / 2.0);
}
}
walsh.sort_by(f64::total_cmp);
let diff = median(&walsh);
let interval = interval(n, &walsh);
let verdict = match interval {
None => Verdict::TooFewPairs,
Some(([lo, hi], _)) if lo <= 0.0 && 0.0 <= hi => Verdict::WithinNoise,
Some(_) if diff.abs() < 0.05_f64.max(0.02 * a) => Verdict::Negligible,
Some(([lo, _], _)) if lo > 0.0 => Verdict::Slower,
Some(_) => Verdict::Faster,
};
Paired {
pairs: n,
a: round(a) + 0.0,
b: round(b) + 0.0,
diff: round(diff) + 0.0,
diff_pct: (a != 0.0).then(|| round(100.0 * diff / a) + 0.0),
ci95: interval.map(|([lo, hi], _)| [round(lo) + 0.0, round(hi) + 0.0]),
confidence: interval.map(|(_, c)| round4(c)),
noise: None,
verdict,
}
}
/// The widest symmetric interval of the sorted Walsh averages whose tails hold at most 2.5% each, and its confidence.
fn interval(n: usize, walsh: &[f64]) -> Option<([f64; 2], f64)> {
let m = walsh.len();
// Null probabilities of T+ (halving is exact in binary, so these are the counts over 2^n).
let mut p = vec![0.0f64; m + 1];
p[0] = 1.0;
for r in 1..=n {
for t in (0..=m).rev() {
p[t] = (p[t] + if t >= r { p[t - r] } else { 0.0 }) / 2.0;
}
}
let (mut k, mut tail, mut best) = (0, 0.0, 0.0);
while k < m {
tail += p[k];
if tail > 0.025 {
break;
}
k += 1;
best = tail;
}
(k >= 1).then(|| ([walsh[k - 1], walsh[m - k]], 1.0 - 2.0 * best))
}
/// The result of an A/B run: pairs of blocks in B A A B order, recomputable from the report's `frames`.
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)]
pub struct Ab {
pub switch: String,
pub a: String,
pub b: String,
pub pairs: u32,
pub block_s: f64,
pub warmup_s: f64,
/// Block 0 (measured before any switch) lies outside the later B blocks.
pub first_block_outside_range: bool,
/// By `gpu_p50_ms`, `cpu_p50_ms`, `frame_p50_ms`; a metric missing from any block is left out.
pub result: BTreeMap<String, Paired>,
}
impl Ab {
/// Pairs the `ab.a`/`ab.b` entries of `frames` by `repeat` (0, 1, 2...): pair k is blocks 2k and 2k+1.
pub fn from_frames(
switch: &str,
a: &str,
b: &str,
block_s: f64,
warmup_s: f64,
frames: &[Frames],
) -> Result<Ab, String> {
let mut blocks: Vec<&Frames> = frames
.iter()
.filter(|f| f.phase == "ab.a" || f.phase == "ab.b")
.collect();
blocks.sort_by_key(|f| f.repeat);
if blocks.is_empty()
|| !blocks.len().is_multiple_of(2)
|| blocks
.iter()
.enumerate()
.any(|(i, f)| f.repeat as usize != i)
{
return Err("A/B blocks must be numbered 0..N with N even".to_owned());
}
if blocks.chunks(2).any(|p| p[0].phase == p[1].phase) {
return Err("each pair needs one ab.a and one ab.b block".to_owned());
}
let pairs = blocks.len() / 2;
let metrics: [(&str, Getter); 3] = [
("gpu_p50_ms", |f| f.gpu_p50_ms),
("cpu_p50_ms", |f| f.cpu_p50_ms),
("frame_p50_ms", |f| Some(f.p50_ms)),
];
let mut result = BTreeMap::new();
let mut series: BTreeMap<&str, Vec<f64>> = BTreeMap::new();
for (name, get) in metrics {
let Some(values) = blocks.iter().map(|f| get(f)).collect::<Option<Vec<f64>>>() else {
continue;
};
let by_arm: Vec<(f64, f64)> = blocks
.chunks(2)
.zip(values.chunks(2))
.map(|(p, v)| {
if p[0].phase == "ab.a" {
(v[0], v[1])
} else {
(v[1], v[0])
}
})
.collect();
let mut stat = paired(&by_arm);
let gaps: Vec<f64> = (1..values.len().saturating_sub(1))
.step_by(2)
.filter(|&i| blocks[i].phase == blocks[i + 1].phase)
.map(|i| (values[i] - values[i + 1]).abs())
.collect();
stat.noise = (!gaps.is_empty()).then(|| round(median(&gaps)) + 0.0);
result.insert(name.to_owned(), stat);
series.insert(name, values);
}
let first_block_outside_range = series
.get("gpu_p50_ms")
.or_else(|| series.get("frame_p50_ms"))
.is_some_and(|values| {
let later: Vec<f64> = (1..values.len())
.filter(|&i| blocks[i].phase == "ab.b")
.map(|i| values[i])
.collect();
blocks[0].phase == "ab.b"
&& !later.is_empty()
&& (values[0] < later.iter().copied().fold(f64::INFINITY, f64::min)
|| values[0] > later.iter().copied().fold(f64::NEG_INFINITY, f64::max))
});
Ok(Ab {
switch: switch.to_owned(),
a: a.to_owned(),
b: b.to_owned(),
pairs: pairs as u32,
block_s,
warmup_s,
first_block_outside_range,
result,
})
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::{App, Kind, Report};
// Expected values: exact-rational Walsh averages and a brute-force sign enumeration, run once outside this code.
fn check(
a: &[f64],
b: &[f64],
diff: f64,
ci: Option<[f64; 2]>,
conf: Option<f64>,
verdict: Verdict,
) {
let pairs: Vec<(f64, f64)> = a.iter().copied().zip(b.iter().copied()).collect();
let p = paired(&pairs);
assert_eq!(
(p.pairs, p.diff, p.ci95, p.confidence, p.verdict),
(a.len(), diff, ci, conf, verdict),
"{b:?}"
);
}
#[test]
fn the_decided_vectors() {
use Verdict::*;
let z = |n: usize| vec![0.0; n];
let up = |n: usize| (1..=n).map(|v| v as f64).collect::<Vec<_>>();
check(&z(8), &up(8), 4.5, Some([2.0, 7.0]), Some(0.9609), Slower);
let wobble = [-0.3, 0.2, -0.1, 0.4, -0.2, 0.1, 0.3, -0.4];
check(
&z(8),
&wobble,
0.0,
Some([-0.3, 0.3]),
Some(0.9609),
WithinNoise,
);
check(&z(6), &up(6), 3.5, Some([1.0, 6.0]), Some(0.9688), Slower);
check(&z(5), &up(5), 3.0, None, None, TooFewPairs);
let a = [3.65, 3.70, 3.62, 3.68, 3.66, 3.71, 3.63, 3.69];
let b = [5.63, 5.60, 5.70, 5.58, 5.65, 5.62, 5.66, 5.61];
check(&a, &b, 1.96, Some([1.905, 2.03]), Some(0.9609), Slower);
let tiny: Vec<f64> = (1..=8).map(|v| 10.0 + v as f64 / 100.0).collect();
check(
&[10.0; 8],
&tiny,
0.045,
Some([0.02, 0.07]),
Some(0.9609),
Negligible,
);
let down: Vec<f64> = up(8).iter().map(|_| 0.0).collect();
check(
&up(8),
&down,
-4.5,
Some([-7.0, -2.0]),
Some(0.9609),
Faster,
);
let mixed = [0.5, -0.2, 0.0, 0.3, 0.9, -0.1, 0.4];
check(
&z(7),
&mixed,
0.225,
Some([-0.1, 0.65]),
Some(0.9531),
WithinNoise,
);
let ten = [0.3, -0.1, 0.8, 0.2, -0.5, 0.6, 0.1, 0.4, 0.7, 0.05];
check(
&z(10),
&ten,
0.25,
Some([-0.025, 0.55]),
Some(0.9512),
WithinNoise,
);
}
#[test]
fn medians_percent_and_empty_input() {
let a = [3.65, 3.70, 3.62, 3.68, 3.66, 3.71, 3.63, 3.69];
let b = [5.63, 5.60, 5.70, 5.58, 5.65, 5.62, 5.66, 5.61];
let p = paired(&a.iter().copied().zip(b).collect::<Vec<_>>());
assert_eq!(
(p.a, p.b, p.diff_pct, p.noise),
(3.67, 5.625, Some(53.406), None)
);
assert_eq!(
paired(&[(0.0, 1.0); 8]).diff_pct,
None,
"no percent of zero"
);
assert_eq!(paired(&[]).verdict, Verdict::TooFewPairs);
}
fn block(phase: &str, repeat: u32, frame: f64, cpu: Option<f64>, gpu: Option<f64>) -> Frames {
Frames {
phase: phase.into(),
repeat,
frames: 100,
seconds: 4.0,
avg_fps: 250.0,
low_1pct_fps: 100.0,
low_01pct_fps: 90.0,
p50_ms: frame,
p95_ms: frame + 1.0,
p99_ms: frame + 2.0,
p999_ms: frame + 3.0,
max_ms: frame + 4.0,
cpu_p50_ms: cpu,
gpu_p50_ms: gpu,
histogram: vec![0; crate::HISTOGRAM_EDGES_MS.len()],
}
}
// B A A B B A A B B A A B, as gpu milliseconds; frame time is gpu + 1.
fn twelve(first_b: f64) -> Vec<Frames> {
let gpu = [
first_b, 3.0, 3.2, 5.0, 5.2, 3.1, 3.1, 5.1, 5.4, 3.4, 3.3, 5.3,
];
gpu.iter()
.enumerate()
.map(|(i, &g)| {
block(
if matches!(i % 4, 0 | 3) {
"ab.b"
} else {
"ab.a"
},
i as u32,
g + 1.0,
Some(g / 2.0),
Some(g),
)
})
.collect()
}
fn from(frames: &[Frames]) -> Result<Ab, String> {
Ab::from_frames("gpu-culling", "off", "on", 4.0, 2.0, frames)
}
#[test]
fn noise_and_the_first_block_flag() {
let ab = from(&twelve(6.4)).unwrap();
assert!(
ab.first_block_outside_range,
"6.4 is 1 ms above the other B blocks"
);
let gpu = &ab.result["gpu_p50_ms"];
// Neighbours (1,2) (3,4) (5,6) (7,8) (9,10): 0.2 0.2 0 0.3 0.1.
assert_eq!(
(gpu.pairs, gpu.a, gpu.b, gpu.noise),
(6, 3.15, 5.25, Some(0.2))
);
assert_eq!(
(gpu.ci95, gpu.verdict),
(Some([1.8, 3.4]), Verdict::Slower),
"every d is positive: min and max Walsh averages"
);
assert_eq!(ab.result["cpu_p50_ms"].noise, Some(0.1));
assert_eq!(
(ab.pairs, ab.switch.as_str(), ab.a.as_str(), ab.b.as_str()),
(6, "gpu-culling", "off", "on")
);
assert!(!from(&twelve(5.2)).unwrap().first_block_outside_range);
}
#[test]
fn a_metric_missing_from_one_block_is_left_out() {
let mut frames = twelve(5.2);
frames[7].gpu_p50_ms = None;
let ab = from(&frames).unwrap();
assert_eq!(
ab.result.keys().collect::<Vec<_>>(),
["cpu_p50_ms", "frame_p50_ms"]
);
let mut frames = twelve(6.4);
frames[0].gpu_p50_ms = None;
frames[0].p50_ms = 6.0;
assert!(
!from(&frames).unwrap().first_block_outside_range,
"frame time 6.0 is inside 6.0..6.4 of the later B blocks"
);
}
#[test]
fn blocks_that_do_not_pair_are_refused() {
assert!(from(&[]).is_err());
assert!(from(&twelve(5.2)[..11]).is_err(), "odd number of blocks");
let mut frames = twelve(5.2);
frames[1].phase = "ab.b".into();
assert!(from(&frames).is_err(), "a pair of two B blocks");
let mut frames = twelve(5.2);
frames[3].repeat = 9;
assert!(from(&frames).is_err(), "a gap in the numbering");
}
#[test]
fn a_report_with_ab_round_trips_and_old_ones_still_parse() {
let mut r = Report::new(Kind::Game, App::default());
let old = r.to_json();
assert!(!old.contains("\"ab\""));
assert_eq!(Report::parse(&old).unwrap().ab, None);
r.frames = twelve(5.2);
r.ab = Some(from(&r.frames).unwrap());
let json = r.to_json();
assert_eq!(Report::parse(&json).unwrap(), r);
assert!(
json.contains("\"first_block_outside_range\": false")
&& json.contains("\"verdict\": \"slower\"")
);
assert_eq!(r.for_upload().ab, r.ab);
let mut few = paired(&[(1.0, 2.0); 3]);
few.noise = None;
assert!(serde_json::to_string(&few)
.unwrap()
.contains("\"ci95\":null,\"confidence\":null"));
}
}
+19 -10
View File
@@ -2,7 +2,7 @@
//! come from the program that rendered (the game knows its adapter, the launcher its window).
#[cfg(any(target_os = "macos", target_os = "linux"))]
use horizon_telemetry::system::command_output as output;
use horizon_system::command_output as output;
use serde::{Deserialize, Serialize};
#[derive(Clone, Debug, Default, PartialEq, Serialize, Deserialize)]
@@ -62,6 +62,7 @@ pub struct Display {
pub refresh_hz: Option<u32>,
}
#[cfg(not(target_arch = "wasm32"))]
impl Hardware {
pub fn detect() -> Hardware {
let probes = probes();
@@ -69,12 +70,12 @@ impl Hardware {
os: std::env::consts::OS.to_owned(),
os_version: os_version(),
arch: std::env::consts::ARCH.to_owned(),
cpu: horizon_telemetry::system::cpu(),
cpu: horizon_system::cpu(),
cpu_threads: std::thread::available_parallelism().ok().map(|n| n.get()),
cpu_cores: probes.cores,
cpu_base_mhz: probes.base,
cpu_boost_mhz: probes.boost,
ram_mb: horizon_telemetry::system::ram_bytes().map(|b| b >> 20),
ram_mb: horizon_system::ram_bytes().map(|b| b >> 20),
storage: None,
form: probes.form,
gpu: None,
@@ -85,13 +86,14 @@ impl Hardware {
/// [`Hardware::detect`] with the kind of drive `game_dir` is on.
pub fn detect_in(game_dir: &std::path::Path) -> Hardware {
Hardware {
storage: horizon_telemetry::system::storage(game_dir).map(str::to_owned),
storage: horizon_system::storage(game_dir).map(str::to_owned),
..Hardware::detect()
}
}
}
/// What the telemetry probes read (`horizon_telemetry::system`).
#[cfg(not(target_arch = "wasm32"))]
/// What `horizon_system::Machine` reads.
struct Probes {
cores: Option<u16>,
base: Option<u32>,
@@ -99,27 +101,31 @@ struct Probes {
form: Option<String>,
}
#[cfg(not(target_arch = "wasm32"))]
fn probes() -> Probes {
let s = horizon_telemetry::payload::System::detect();
let s = horizon_system::Machine::detect();
let some = |n: u32| (n > 0).then_some(n);
Probes {
cores: (s.cores > 0).then_some(s.cores),
base: some(s.cpu_base_mhz),
boost: some(s.cpu_boost_mhz),
form: (!s.form.is_empty()).then_some(s.form),
form: (!s.form.is_empty()).then(|| s.form.to_owned()),
}
}
#[cfg(not(target_arch = "wasm32"))]
/// `ac` or `battery`; `None` where it can't be read (and on machines without a battery on Linux).
pub fn power_source() -> Option<&'static str> {
horizon_telemetry::system::power()
horizon_system::power()
}
#[cfg(not(target_arch = "wasm32"))]
#[cfg(target_os = "macos")]
fn os_version() -> Option<String> {
output("sw_vers", &["-productVersion"])
}
#[cfg(not(target_arch = "wasm32"))]
#[cfg(target_os = "linux")]
fn os_version() -> Option<String> {
let release = std::fs::read_to_string("/etc/os-release").ok()?;
@@ -132,6 +138,7 @@ fn os_version() -> Option<String> {
})
}
#[cfg(not(target_arch = "wasm32"))]
#[cfg(target_os = "linux")]
fn key_value<'a>(text: &'a str, key: &str, sep: char) -> Option<&'a str> {
text.lines().find_map(|line| {
@@ -140,17 +147,19 @@ fn key_value<'a>(text: &'a str, key: &str, sep: char) -> Option<&'a str> {
})
}
#[cfg(not(target_arch = "wasm32"))]
#[cfg(target_os = "windows")]
fn os_version() -> Option<String> {
horizon_telemetry::system::windows_version()
horizon_system::windows_version()
}
#[cfg(not(target_arch = "wasm32"))]
#[cfg(not(any(target_os = "macos", target_os = "linux", target_os = "windows")))]
fn os_version() -> Option<String> {
None
}
#[cfg(test)]
#[cfg(all(test, not(target_arch = "wasm32")))]
mod tests {
use super::*;
+11 -2
View File
@@ -11,11 +11,15 @@ use std::time::{SystemTime, UNIX_EPOCH};
use serde::{Deserialize, Serialize};
pub mod compare;
mod hardware;
pub use hardware::{power_source, Display, Gpu, Hardware};
#[cfg(not(target_arch = "wasm32"))]
pub use hardware::power_source;
pub use hardware::{Display, Gpu, Hardware};
/// The process probes, from the platform (`horizon-system`).
pub use horizon_telemetry::system::{cpu_seconds, load_average, rss_bytes};
#[cfg(not(target_arch = "wasm32"))]
pub use horizon_system::{cpu_seconds, load_average, rss_bytes};
pub const FORMAT: &str = "horizon-benchmark-report";
pub const VERSION: u32 = 1;
@@ -236,6 +240,9 @@ pub struct Report {
pub metrics: BTreeMap<String, Summary>,
#[serde(default)]
pub frames: Vec<Frames>,
/// Present in A/B runs only.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub ab: Option<compare::Ab>,
/// Local only: a name the person running it chose.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub label: Option<String>,
@@ -246,6 +253,7 @@ pub struct Report {
impl Report {
/// An empty report for this machine, created now.
#[cfg(not(target_arch = "wasm32"))]
pub fn new(kind: Kind, app: App) -> Report {
Self::with_hardware(kind, app, Hardware::detect())
}
@@ -263,6 +271,7 @@ impl Report {
conditions: Conditions::default(),
metrics: BTreeMap::new(),
frames: Vec::new(),
ab: None,
label: None,
local: Vec::new(),
}
+32 -1
View File
@@ -61,6 +61,8 @@ pub enum ClickEvent {
/// A local file to open with the system handler.
/// Only allow this in locally constructed messages, never server/LAN text.
OpenFile(String),
/// A `/profiler` line the client's own buttons run; never decoded from a server's text, so only they reach the local profiler.
Profiler(String),
}
/// A non-character `minecraft:object` embedded in a text component.
@@ -202,7 +204,12 @@ impl Text {
}
}
pub fn styled(s: impl Into<String>, style: Style) -> Self {
Self::literal(s).with_style(&style)
let mut text = Self::literal(s).with_style(&style);
// `inherit` never passes an object on, but a styled span's own style carries it.
for span in &mut text.spans {
span.style.inline_object.clone_from(&style.inline_object);
}
text
}
pub fn colored(s: impl Into<String>, color: u32) -> Self {
Self::styled(s, Style::color(color))
@@ -619,6 +626,9 @@ fn style_json(style: &Style) -> Vec<(String, Value)> {
event
}
ClickEvent::OpenFile(path) => json!({ "action": "open_file", "path": path }),
ClickEvent::Profiler(command) => {
json!({ "action": "run_command", "command": command })
}
};
out.push(("click_event".to_owned(), event));
}
@@ -772,6 +782,27 @@ pub fn format_translation(template: &str, args: &[Text]) -> Text {
mod tests {
use super::*;
#[test]
fn a_styled_object_span_keeps_its_object_through_append_and_slice() {
let object = InlineObject::Atlas {
atlas: "minecraft:gui".into(),
sprite: "horizon:gui/mark".into(),
fallback: "H".into(),
};
let sprite = Text::styled(
"\u{fffc}",
Style {
inline_object: Some(object.clone()),
..Style::color(WHITE)
},
);
let line = Text::colored("[", DARK_GRAY)
.append(sprite)
.append(Text::colored("] ", DARK_GRAY));
let line = line.slice(0, 3);
assert_eq!(line.spans[1].style.inline_object, Some(object));
assert_eq!(line.plain(), "[H]");
}
#[test]
fn appending_builds_the_same_tree_in_linear_time() {
// A styled text is wrapped once, so its style doesn't reach what follows; a plain one
// takes the siblings itself; a copy taken before an append keeps its own tree.
+4
View File
@@ -27,6 +27,8 @@ horizon-mcp = { path = "../horizon-mcp" }
# The server thread ahead of the workers (MinecraftServer.spin's Thread.setPriority).
horizon-optifine = { path = "../horizon-optifine" }
horizon-threads = { path = "../horizon-threads" }
# The .hzperf capture format and writer (docs/perf-platform.md).
horizon-perf-format = { path = "../horizon-perf-format" }
# One pool of helper threads for the server tick's parallel phases (`parallel.rs`): threads made
# once, not for every phase of every tick.
rayon = "1.10"
@@ -130,6 +132,8 @@ workspace = true
horizon-theme = { path = "../horizon-theme" }
horizon-client-render = { path = "../horizon-client-render", features = ["test-support"] }
horizon-items = { path = "../horizon-items", features = ["test-support"] }
# Reads back the captures the server_perf tests write.
horizon-perf = { path = "../horizon-perf" }
# Every WGSL shader translated as wgpu's DirectX 12 backend does (gfx.rs tests).
naga = { version = "30.0.1", features = ["wgsl-in", "hlsl-out", "msl-out", "spv-out"] }
+5
View File
@@ -23,6 +23,7 @@ horizon-narrator = { path = "../horizon-narrator" }
# OptiFine-format pack files: random entity textures (hud.rs), connected textures
# (connected.rs) and custom item texture rules (cit.rs).
horizon-optifine = { path = "../horizon-optifine" }
horizon-perf-format = { path = "../horizon-perf-format" }
horizon-polytone = { path = "../horizon-polytone" }
horizon-protocol = { path = "../horizon-protocol" }
horizon-shader-api = { path = "../horizon-shader-api" }
@@ -74,6 +75,9 @@ wgpu = { version = "30.0.1", default-features = false, features = ["gles"] }
audio = ["dep:cpal"]
# Helpers horizon-client's tests build their fixtures with.
test-support = []
# The `scene_replay` example's allocator counts allocations (horizon-threads' `alloc-count`).
alloc-count = ["horizon-threads/alloc-count"]
# horizon-client's `horizon-extras`: asking the device for hardware ray tracing on request
# (`gfx/ray_query.rs`, `HORIZON_RAY_TRACING=1`). Without it the device is requested as without
# the variable.
@@ -83,6 +87,7 @@ horizon-extras = []
workspace = true
[dev-dependencies]
horizon-perf = { path = "../horizon-perf" }
# Tests: a TrueType font to load, and the design system the menus' motion curves match.
horizon-theme = { path = "../horizon-theme" }
horizon-items = { path = "../horizon-items", features = ["test-support"] }
@@ -6,13 +6,14 @@
//! [--mipmaps N] [--lod-distance CHUNKS] [--render-distance CHUNKS] [--size WxH]
//! [--section-draws flat|indirect] [--section-cull on|off] [--section-sides on|off]
//! [--gpu-culling on|off] [--root RESOURCES] [--screenshot PNG]
//! [--world SAVE --leaf-culling MODE] [--memory on]`
//! [--world SAVE --leaf-culling MODE] [--memory on] [--gpu-detail on|off]`
//! `--screenshot` saves the first view's last frame, after the timed frames.
//! `--world` meshes the captured sections again from that save's region files
//! (blocks, biomes and saved light; `ao`, `biomeBlendRadius` and `cutoutLeaves`
//! from `./options.txt`) with `--leaf-culling` (default `off`), so a mesher
//! change can be measured on the captured view. A section whose 3x3 columns are
//! not all saved keeps its captured mesh.
//! `--gpu-detail on` adds GPU culling's dispatch times and, on Vulkan, the world passes' counters.
//! `--memory on` adds the renderer's largest GPU resources and, on macOS, the process's
//! physical footprint by category (`footprint`), taken after the timed frames.
use horizon_client_render::{
@@ -24,6 +25,7 @@ use horizon_client_render::{
gfx::{
PreparedTextures, Renderer, ScreenshotRequest,
capture::{self, AtlasCheck, Scene},
pack_timing,
panorama::Faces,
},
graphics::LeafCullingMode,
@@ -61,7 +63,10 @@ use winit::{
// The game's allocator, so that `--memory`'s footprint is the game's: the system heap kept
// gigabytes of the meshes `--world` frees dirty.
#[global_allocator]
static ALLOCATOR: horizon_threads::Allocator = horizon_threads::ALLOCATOR;
static ALLOCATOR: horizon_threads::GameAllocator = horizon_threads::GAME_ALLOCATOR;
/// Frames drawn with detail on, enough for a few timed frames of each kind.
const DETAIL_FRAMES: usize = 60;
#[derive(Default)]
struct Options {
@@ -84,6 +89,7 @@ struct Options {
world: Option<PathBuf>,
leaf_culling: LeafCullingMode,
memory: bool,
gpu_detail: bool,
}
fn on_off(value: &str) -> Result<bool, String> {
@@ -136,6 +142,7 @@ fn options() -> Result<Options, String> {
"--screenshot" => o.screenshot = Some(value.into()),
"--world" => o.world = Some(value.into()),
"--memory" => o.memory = on_off(&value)?,
"--gpu-detail" => o.gpu_detail = on_off(&value)?,
"--leaf-culling" => {
o.leaf_culling = LeafCullingMode::ALL
.into_iter()
@@ -400,6 +407,23 @@ fn replay(window: Arc<Window>) -> Result<(), Box<dyn std::error::Error>> {
None => notes.push("screenshot: not saved".into()),
}
}
let mut detail = serde_json::Map::new();
if o.gpu_detail {
renderer.set_pass_detail(true);
for _ in 0..DETAIL_FRAMES {
draw(&mut renderer)?;
}
detail.insert(
"gpu_pass_parts_median_ms".into(),
pack_timing::parts_json(&renderer.own_pass_parts()),
);
let unit = renderer.pass_detail_support().and_then(|s| s.fragment_unit);
detail.insert(
"gpu_counters_median".into(),
pack_timing::counters_by_pass_json(&renderer.own_pass_counters(), unit),
);
renderer.set_pass_detail(false);
}
views.push(json!({
"yaw_offset": offset,
"gpu_ms": summary(&gpu),
@@ -417,6 +441,9 @@ fn replay(window: Arc<Window>) -> Result<(), Box<dyn std::error::Error>> {
// indices hidden.
"culled": renderer.culled_sections(),
}));
if let Some(Value::Object(view)) = views.last_mut() {
view.extend(detail);
}
}
let section_draws = if renderer.flat_draws() {
"flat"
@@ -424,6 +451,11 @@ fn replay(window: Arc<Window>) -> Result<(), Box<dyn std::error::Error>> {
"indirect"
};
let memory = o.memory.then(|| memory_report(&renderer));
let gpu_detail = o
.gpu_detail
.then(|| renderer.pass_detail_support())
.flatten()
.map(pack_timing::support_json);
let report = json!({
"capture": o.dir.display().to_string(),
"adapter": renderer.info.name,
@@ -454,6 +486,7 @@ fn replay(window: Arc<Window>) -> Result<(), Box<dyn std::error::Error>> {
"notes": notes,
"views": views,
"memory": memory,
"gpu_detail": gpu_detail,
});
println!("{}", serde_json::to_string_pretty(&report)?);
Ok(())
+82 -90
View File
@@ -247,9 +247,11 @@ mod pack_post;
mod pack_shadow;
mod pack_sky;
mod pack_textures;
pub(crate) mod pack_timing;
pub mod pack_timing;
pub mod pack_uniforms;
mod pack_weather;
mod pass_timing_api;
use pass_timing_api::{after_culling_pass, pixel_count, world_pass_after_culling};
mod page_runs;
pub mod panorama;
mod readback;
@@ -338,42 +340,6 @@ fn surface_pass<'e>(
..Default::default()
})
}
fn world_pass_after_culling<'e>(
encoder: &'e mut wgpu::CommandEncoder,
view: &wgpu::TextureView,
depth: &wgpu::TextureView,
keep_depth: bool,
timing: Option<&mut pack_timing::PassTiming>,
name: &str,
) -> wgpu::RenderPass<'e> {
encoder.begin_render_pass(&wgpu::RenderPassDescriptor {
label: Some("Horizon world, after culling"),
timestamp_writes: timing.and_then(|t| t.writes(name)),
color_attachments: &[Some(wgpu::RenderPassColorAttachment {
view,
resolve_target: None,
depth_slice: None,
ops: wgpu::Operations {
load: wgpu::LoadOp::Load,
store: wgpu::StoreOp::Store,
},
})],
depth_stencil_attachment: Some(wgpu::RenderPassDepthStencilAttachment {
view: depth,
depth_ops: Some(wgpu::Operations {
load: wgpu::LoadOp::Load,
// A shader pack reads it as depthtex0.
store: if keep_depth {
wgpu::StoreOp::Store
} else {
wgpu::StoreOp::Discard
},
}),
stencil_ops: None,
}),
..Default::default()
})
}
/// The point a `shaders/` pack's terrain is drawn relative to: the camera with the fraction of
/// each coordinate rounded to the nearest 1/65536 block. Read back with a probe pack of ours,
/// Iris 1.11.7's terrain `gl_Vertex` is the block's position minus this, for cameras from 13
@@ -1474,10 +1440,14 @@ pub struct Renderer {
pack_loading: Option<PackLoading>,
/// The post passes of one of Horizon's own shader packs; never beside `pack_post`'s.
native: Option<native_pack::NativePack>,
/// Time the shader pack's passes on the GPU (F3, or `HORIZON_SHADER_PACK_TIMING`).
/// Time the shader pack's passes on the GPU (F3, or a capture made with `--gpu`).
pass_timing: bool,
/// Times the parts of passes too (`set_pass_detail`).
pass_detail: bool,
/// Least time between timed frames (`set_pass_timing_gap`).
pass_timing_gap: std::time::Duration,
/// Whether the timers keep every timed frame for `take_timed_frames`.
keep_timed_frames: bool,
/// Times Horizon's own passes on the GPU, as `pass_timing` does (`own_pass_times`).
own_timing: Option<pack_timing::PassTiming>,
/// The entity vertices Horizon draws itself while a shader pack draws the others.
@@ -5086,8 +5056,10 @@ impl Renderer {
pack_post: None,
pack_loading: None,
native: None,
pass_timing: pack_timing::PassTiming::logging(),
pass_timing: false,
pass_detail: false,
pass_timing_gap: std::time::Duration::ZERO,
keep_timed_frames: false,
own_timing: None,
pack_entity_rest: Vec::new(),
cloud_bindings,
@@ -7449,51 +7421,6 @@ impl Renderer {
.terrain_inputs(self.pack_terrain_flat),
)
}
/// Times the shader pack's passes on the GPU while `on` (the F3 overlay is open), or always
/// with `HORIZON_SHADER_PACK_TIMING`.
pub fn set_pass_timing(&mut self, on: bool) {
self.pass_timing = on || pack_timing::PassTiming::logging();
}
/// Times a frame at most once per `gap` while passes are timed (the F3 overlay needs far fewer
/// samples than a measurement). `HORIZON_SHADER_PACK_TIMING` keeps every sample.
pub fn set_pass_timing_gap(&mut self, gap: std::time::Duration) {
self.pass_timing_gap = if pack_timing::PassTiming::logging() {
std::time::Duration::ZERO
} else {
gap
};
}
/// The running shader pack's timer, of either kind.
fn pass_timer<'a>(
pack: &'a mut Option<(i32, Option<pack_post::PackPost>)>,
native: &'a mut Option<native_pack::NativePack>,
) -> Option<&'a mut Option<pack_timing::PassTiming>> {
match (pack, native) {
(Some((_, Some(post))), _) => Some(&mut post.timing),
(_, Some(native)) => Some(&mut native.timing),
_ => None,
}
}
/// Each pass of the running shader pack with its median GPU time in milliseconds, while
/// they are timed (`set_pass_timing`).
pub fn shader_pass_times(&self) -> Vec<(String, f32)> {
let timer = match (&self.pack_post, &self.native) {
(Some((_, Some(post))), _) => post.timing.as_ref(),
(_, Some(native)) => native.timing.as_ref(),
_ => None,
};
timer
.map(pack_timing::PassTiming::medians)
.unwrap_or_default()
}
/// Each of Horizon's own passes with its median GPU time in milliseconds, while they are
/// timed (`set_pass_timing`).
pub fn own_pass_times(&self) -> Vec<(String, f32)> {
self.own_timing
.as_ref()
.map(pack_timing::PassTiming::medians)
.unwrap_or_default()
}
/// The macros this renderer gives shader packs: the GL extensions this device has
/// (subgroups, when it has them), and the vendor, renderer and mipmap levels
/// (`Macros::define_renderer`).
@@ -8102,6 +8029,8 @@ impl Renderer {
}
if let Some(t) = &mut self.own_timing {
t.gap = self.pass_timing_gap;
t.keep_frames(self.keep_timed_frames);
t.set_detail(self.pass_detail);
t.begin(&self.device, &self.queue);
}
if self.animated_textures {
@@ -8521,6 +8450,7 @@ impl Renderer {
&mut self.uploads,
&mut encoder,
&input,
self.own_timing.as_mut(),
)
});
// A frame whose planes leave the test's integers (the camera millions of blocks
@@ -9869,6 +9799,7 @@ impl Renderer {
}
if let Some(Some(t)) = Self::pass_timer(&mut self.pack_post, &mut self.native) {
t.gap = self.pass_timing_gap;
t.keep_frames(self.keep_timed_frames);
}
let pack_reads_world = pack_post || native_post;
laps.lap("r_uploads");
@@ -10393,7 +10324,8 @@ impl Renderer {
store: wgpu::StoreOp::Store,
},
})];
let mut pass = encoder.begin_render_pass(&wgpu::RenderPassDescriptor {
let world_pixels = pixel_count(world_depth);
let first_pass = encoder.begin_render_pass(&wgpu::RenderPassDescriptor {
label: Some("Horizon world"),
timestamp_writes: self.own_timing.as_mut().and_then(|t| t.writes("World")),
color_attachments: &attachments,
@@ -10413,9 +10345,16 @@ impl Renderer {
}),
..Default::default()
});
let mut pass = pack_timing::counted(
self.own_timing.as_mut(),
first_pass,
world_pixels,
"before translucent",
);
pass.set_bind_group(0, &self.group, &[]);
pack_frame(&mut pass);
if frame.title || (sky_visible && (frame.skybox || frame.end_sky)) {
pass.stage(self.own_timing.as_mut(), "sky");
if !frame.title && frame.skybox && !pack_draws_sky && !native_sky_pass {
match self
.native
@@ -10502,16 +10441,15 @@ impl Renderer {
.forget_lifetime();
pass.set_bind_group(0, &self.group, &[]);
pack_frame(&mut pass);
pass
pack_timing::CountedPass::plain(pass)
};
// The occlusion test's pipeline draws into the world's colour alone.
let plain_pass = |encoder: &mut wgpu::CommandEncoder| {
let mut pass =
world_pass_after_culling(encoder, world_view, world_depth, true, None, "")
.forget_lifetime();
let mut pass = after_culling_pass(encoder, world_view, world_depth, true, None)
.forget_lifetime();
pass.set_bind_group(0, &self.group, &[]);
pack_frame(&mut pass);
pass
pack_timing::CountedPass::plain(pass)
};
if let Some(surface) = &surface {
drop(pass);
@@ -10565,6 +10503,7 @@ impl Renderer {
keep_depth,
self.own_timing.as_mut(),
"World after visibility buffer",
world_pixels,
);
pass.set_bind_group(0, &self.group, &[]);
pack_frame(&mut pass);
@@ -10597,6 +10536,7 @@ impl Renderer {
flat.draw(pass, &flat.calls[calls.clone()], vertices);
}
};
pass.stage(self.own_timing.as_mut(), "solid terrain");
match cull {
// As below: phase 1, the test, then phase 2's draws
// that the test saw, copied after it.
@@ -10606,6 +10546,8 @@ impl Renderer {
drop(pass);
pass = plain_pass(&mut encoder);
}
pass.split(self.own_timing.as_mut(), "occlusion boxes");
pass.stage(self.own_timing.as_mut(), "occlusion boxes");
cull.test(&mut pass);
drop(pass);
cull.test_pass(&mut encoder, self.own_timing.as_mut());
@@ -10619,10 +10561,12 @@ impl Renderer {
keep_depth,
self.own_timing.as_mut(),
"World after culling",
world_pixels,
),
};
pass.set_bind_group(0, &self.group, &[]);
pack_frame(&mut pass);
pass.stage(self.own_timing.as_mut(), "solid terrain phase 2");
for (pipeline, _, calls) in passes {
pass.set_pipeline(pipeline);
flat.draw_phase2(&mut pass, calls.clone(), vertices);
@@ -10660,6 +10604,7 @@ impl Renderer {
let pulled = material.is_none() && self.flat.is_some();
pass.set_index_buffer(self.indices.slice(..), wgpu::IndexFormat::Uint32);
pass.set_vertex_buffer(1, self.instances.slice(..));
pass.stage(self.own_timing.as_mut(), "solid terrain");
if let Some(group) = warp_group {
pass.set_bind_group(2, group, &[]);
}
@@ -10684,6 +10629,8 @@ impl Renderer {
drop(pass);
pass = plain_pass(&mut encoder);
}
pass.split(self.own_timing.as_mut(), "occlusion boxes");
pass.stage(self.own_timing.as_mut(), "occlusion boxes");
cull.test(&mut pass);
drop(pass);
cull.test_pass(&mut encoder, self.own_timing.as_mut());
@@ -10696,6 +10643,7 @@ impl Renderer {
keep_depth,
self.own_timing.as_mut(),
"World after culling",
world_pixels,
),
};
pass.set_bind_group(0, &self.group, &[]);
@@ -10705,6 +10653,7 @@ impl Renderer {
wgpu::IndexFormat::Uint32,
);
pass.set_vertex_buffer(1, self.instances.slice(..));
pass.stage(self.own_timing.as_mut(), "solid terrain phase 2");
for (pipeline, _, draws) in passes {
pass.set_pipeline(pipeline);
self.draw_sections_from(
@@ -10752,6 +10701,8 @@ impl Renderer {
drop(pass);
pass = plain_pass(&mut encoder);
}
pass.split(self.own_timing.as_mut(), "occlusion boxes");
pass.stage(self.own_timing.as_mut(), "occlusion boxes");
gpu.test(&mut pass);
drop(pass);
gpu.compact(&mut encoder, self.own_timing.as_mut());
@@ -10764,6 +10715,7 @@ impl Renderer {
keep_depth,
self.own_timing.as_mut(),
"World after culling",
world_pixels,
),
};
pass.set_bind_group(0, &self.group, &[]);
@@ -10772,6 +10724,7 @@ impl Renderer {
self.indices.slice(..),
wgpu::IndexFormat::Uint32,
);
pass.stage(self.own_timing.as_mut(), "solid terrain phase 2");
for (pipeline, draws, _) in passes {
pass.set_pipeline(pipeline);
gpu.draw(&mut pass, &self.arena, page_group, 1);
@@ -10798,6 +10751,7 @@ impl Renderer {
&& let Some(d) = &n.displacement
&& let Some(pipeline) = n.material.as_ref().and_then(|m| m.displaced.as_ref())
{
pass.stage(self.own_timing.as_mut(), "displacement");
pass.set_bind_group(0, &self.group, &[]);
pack_frame(&mut pass);
d.draw(&mut pass, &d.view, pipeline);
@@ -10814,6 +10768,7 @@ impl Renderer {
let hdr_entity_mark =
warped.map_or(&self.hdr_entity_mark_pipeline, |w| &w.hdr_entity_mark);
if static_entity_quads > 0 {
pass.stage(self.own_timing.as_mut(), "static block entities");
pass.set_pipeline(
warped.map_or(&self.static_entity_pipeline, |w| &w.static_entities),
);
@@ -10839,6 +10794,7 @@ impl Renderer {
}
pass.set_index_buffer(self.indices.slice(..), wgpu::IndexFormat::Uint32);
if entity_quads > 0 {
pass.stage(self.own_timing.as_mut(), "entities");
pass.set_pipeline(warped.map_or(&self.entity_pipeline, |w| &w.entities));
pass.set_bind_group(0, &self.entity_group, &[]);
pass.set_vertex_buffer(0, self.entity_buffer.slice(..));
@@ -10850,6 +10806,7 @@ impl Renderer {
pass.set_bind_group(0, &self.group, &[]);
}
if text_quads > 0 && !oit_on {
pass.stage(self.own_timing.as_mut(), "entity text");
pass.set_pipeline(warped.map_or(&self.text_polygon_offset_pipeline, |w| &w.text));
self.bind_world_text(&mut pass, warped.is_none(), false);
pass.set_vertex_buffer(0, self.entity_buffer.slice(..));
@@ -10859,6 +10816,7 @@ impl Renderer {
// FeatureRenderDispatcher.PreparedFrame.executeTranslucent: the shadows
// come before the translucent items, which then cover them (#7026).
if shadow_quads > 0 && !oit_on {
pass.stage(self.own_timing.as_mut(), "shadows");
pass.set_pipeline(warped.map_or(&self.shadow_blob_pipeline, |w| &w.entity_shadows));
pass.set_bind_group(0, &self.entity_group, &[]);
pass.set_vertex_buffer(0, self.entity_buffer.slice(..));
@@ -10874,6 +10832,7 @@ impl Renderer {
pass.set_bind_group(0, &self.group, &[]);
}
if translucent_item_quads > 0 && !oit_on {
pass.stage(self.own_timing.as_mut(), "translucent items");
pass.set_pipeline(
warped.map_or(&self.translucent_item_pipeline, |w| &w.translucent_items),
);
@@ -10909,6 +10868,7 @@ impl Renderer {
.enumerate()
{
if count > 0 {
pass.stage(self.own_timing.as_mut(), "terrain models");
pass.set_pipeline(&self.terrain_model_pipelines[i]);
pass.set_bind_group(
0,
@@ -10929,6 +10889,7 @@ impl Renderer {
// translucent material, before terrain transparency. This material
// always writes depth and never joins the improved-transparency queues.
if fluid_quads > 0 {
pass.stage(self.own_timing.as_mut(), "fluid block entities");
pass.set_pipeline(&self.fluid_pipeline);
pass.set_bind_group(0, &self.fluid_group, &[]);
pass.set_vertex_buffer(0, self.entity_buffer.slice(..));
@@ -10941,6 +10902,7 @@ impl Renderer {
}
// Cracks multiply the existing light; scalar OIT coverage cannot carry that factor.
if crumbling_quads > 0 && (!oit_on || hdr_gain) {
pass.stage(self.own_timing.as_mut(), "block breaking");
pass.set_pipeline(&self.crumbling_pipeline);
pass.set_bind_group(0, &self.entity_group, &[]);
pass.set_vertex_buffer(0, self.entity_buffer.slice(..));
@@ -10954,6 +10916,7 @@ impl Renderer {
// BoatRenderer's water patch (RenderTypes.waterMask): depth only,
// before translucent terrain, so no water shows inside the boat.
if water_mask_quads > 0 && !oit_on {
pass.stage(self.own_timing.as_mut(), "boat water mask");
pass.set_pipeline(&self.water_mask_pipeline);
pass.set_bind_group(0, &self.entity_group, &[]);
pass.set_vertex_buffer(0, self.entity_buffer.slice(..));
@@ -10966,6 +10929,7 @@ impl Renderer {
pass.set_bind_group(0, &self.group, &[]);
}
if !frame.lightning.bolts.is_empty() && !oit_on {
pass.stage(self.own_timing.as_mut(), "lightning");
pass.set_pipeline(&self.lightning_pipeline);
pass.set_bind_group(0, &self.entity_group, &[]);
self.lightning.draw(&mut pass, &frame.lightning);
@@ -10976,6 +10940,7 @@ impl Renderer {
pass.set_bind_group(0, &self.group, &[]);
}
if swirl_quads > 0 && !oit_on {
pass.stage(self.own_timing.as_mut(), "energy swirl");
pass.set_pipeline(&self.energy_swirl_pipeline);
pass.set_bind_group(0, &self.entity_group, &[]);
pass.set_vertex_buffer(0, self.entity_buffer.slice(..));
@@ -10992,6 +10957,7 @@ impl Renderer {
pass.set_bind_group(0, &self.group, &[]);
}
if additive_quads > 0 && !oit_on {
pass.stage(self.own_timing.as_mut(), "additive entities");
pass.set_bind_group(0, &self.entity_group, &[]);
pass.set_vertex_buffer(0, self.entity_buffer.slice(..));
let mut first = 0;
@@ -11017,20 +10983,24 @@ impl Renderer {
pass.set_bind_group(0, &self.group, &[]);
}
if outline_edges > 0 && !frame.outline_after_terrain && !pack_draws_outline {
pass.stage(self.own_timing.as_mut(), "block outline");
self.draw_outline(&mut pass, outline_edges as u32, frame.outline_high_contrast);
}
// DrawableGizmoPrimitives' opaque group writes depth, the translucent one does not.
if debug_opaque > 0 {
pass.stage(self.own_timing.as_mut(), "debug lines");
pass.set_pipeline(&self.debug_lines_opaque);
pass.set_vertex_buffer(0, self.debug_line_buffer.slice(..));
pass.draw(0..4, 0..debug_opaque);
}
if debug_depth > debug_opaque {
pass.stage(self.own_timing.as_mut(), "debug lines");
pass.set_pipeline(&self.debug_lines);
pass.set_vertex_buffer(0, self.debug_line_buffer.slice(..));
pass.draw(0..4, debug_opaque..debug_depth);
}
if debug_top > 0 {
pass.stage(self.own_timing.as_mut(), "debug lines");
pass.set_pipeline(&self.debug_lines_on_top);
pass.set_vertex_buffer(0, self.debug_line_buffer.slice(..));
pass.draw(0..4, debug_depth..debug_depth + debug_top);
@@ -11058,6 +11028,7 @@ impl Renderer {
keep_depth,
self.own_timing.as_mut(),
"World after copy",
world_pixels,
);
pass.set_bind_group(0, &self.group, &[]);
pack_frame(&mut pass);
@@ -11231,9 +11202,11 @@ impl Renderer {
true,
self.own_timing.as_mut(),
"World after OIT",
world_pixels,
);
pass.set_bind_group(0, &self.group, &[]);
pass.set_index_buffer(self.indices.slice(..), wgpu::IndexFormat::Uint32);
pass.stage(self.own_timing.as_mut(), "OIT marks");
if cloud_quads > 0 && !pack_draws_clouds {
self.mark_clouds(&mut pass, cloud_quads);
}
@@ -11266,6 +11239,8 @@ impl Renderer {
self.mark_translucent_terrain(&mut pass, flat, solid_draws, translucent_draws);
}
Some(flat) => {
pass.split(self.own_timing.as_mut(), "translucent terrain");
pass.stage(self.own_timing.as_mut(), "translucent terrain");
pass.set_pipeline(pack_flat.map_or(&flat.translucent, |p| &p.1));
flat.draw(
&mut pass,
@@ -11280,8 +11255,11 @@ impl Renderer {
solid_draws,
translucent_draws,
);
pass.split(self.own_timing.as_mut(), "after translucent");
}
None => {
pass.split(self.own_timing.as_mut(), "translucent terrain");
pass.stage(self.own_timing.as_mut(), "translucent terrain");
pass.set_pipeline(translucent_pipeline);
if let Some(group) = warp_group {
pass.set_bind_group(2, group, &[]);
@@ -11304,6 +11282,7 @@ impl Renderer {
translucent_draws,
);
}
pass.split(self.own_timing.as_mut(), "after translucent");
}
}
// LevelRenderer.executeTranslucentAfterTerrain: vanilla's
@@ -11311,6 +11290,7 @@ impl Renderer {
// the quads write depth, so before it they would hide the glass
// or water behind them.
if translucent_particle_quads > 0 && !oit_on {
pass.stage(self.own_timing.as_mut(), "translucent particles");
pass.set_pipeline(warped.map_or(&self.entity_pipeline, |w| &w.entities));
if let Some(group) = warp_group {
pass.set_bind_group(2, group, &[]);
@@ -11339,6 +11319,7 @@ impl Renderer {
// particles (also after the OIT composite) and before the hand.
// Its own geometry is independent of block-entity lifetime.
if world_text_quads + world_text_offset_quads > 0 {
pass.stage(self.own_timing.as_mut(), "world text");
pass.set_bind_group(0, &self.entity_group, &[]);
pass.set_index_buffer(self.indices.slice(..), wgpu::IndexFormat::Uint32);
pass.set_vertex_buffer(0, self.entity_buffer.slice(..));
@@ -11361,9 +11342,11 @@ impl Renderer {
// LevelRenderer.executeClassicTransparency: translucent targets'
// outlines follow terrain, so its blending cannot wash them out.
if outline_edges > 0 && frame.outline_after_terrain && !pack_draws_outline {
pass.stage(self.own_timing.as_mut(), "block outline");
self.draw_outline(&mut pass, outline_edges as u32, frame.outline_high_contrast);
}
if translucent_crumbling_quads > 0 && (!oit_on || hdr_gain) {
pass.stage(self.own_timing.as_mut(), "block breaking");
pass.set_pipeline(&self.crumbling_pipeline);
pass.set_bind_group(0, &self.entity_group, &[]);
pass.set_vertex_buffer(0, self.entity_buffer.slice(..));
@@ -11374,6 +11357,7 @@ impl Renderer {
);
}
if cloud_quads > 0 && !pack_draws_clouds && !oit_on {
pass.stage(self.own_timing.as_mut(), "clouds");
pass.set_pipeline(&self.cloud_pipeline);
pass.set_bind_group(0, &self.entity_group, &[]);
pass.set_bind_group(1, &self.cloud_group, &[]);
@@ -11385,6 +11369,7 @@ impl Renderer {
// terrain and clouds, so the water behind the rain is drawn first.
pass.set_index_buffer(self.indices.slice(..), wgpu::IndexFormat::Uint32);
if !pack_draws_weather && !oit_on {
pass.stage(self.own_timing.as_mut(), "weather");
let native_weather = self
.native
.as_ref()
@@ -11396,6 +11381,7 @@ impl Renderer {
}
// LevelRenderer: the world border after translucent terrain and weather.
if let Some(b) = frame.border.as_ref().filter(|_| !frame.title && !oit_on) {
pass.stage(self.own_timing.as_mut(), "world border");
pass.set_pipeline(&self.border_pipeline);
pass.set_bind_group(0, &self.border_group, &[]);
pass.set_index_buffer(self.indices.slice(..), wgpu::IndexFormat::Uint32);
@@ -11408,6 +11394,7 @@ impl Renderer {
}
// LevelRenderer.executeSeeThrough, after the rest of the world.
if see_through_quads > 0 {
pass.stage(self.own_timing.as_mut(), "see-through");
pass.set_pipeline(&self.see_through_pipeline);
self.bind_world_text(&mut pass, true, true);
pass.set_vertex_buffer(0, self.entity_buffer.slice(..));
@@ -11420,9 +11407,11 @@ impl Renderer {
// Native mods' primitives: after the entities and the rest of the world, into its
// targets (its resolution, reversed depth and TAA), before the hand.
if world_primitives {
pass.stage(self.own_timing.as_mut(), "mod primitives");
self.world_primitives.draw(&mut pass);
}
if hand_in_world {
pass.stage(self.own_timing.as_mut(), "hand");
pass.set_pipeline(&self.depth_reset);
pass.set_bind_group(0, &self.group, &[]);
pass.draw(0..3, 0..1);
@@ -12410,6 +12399,9 @@ impl Renderer {
if let Some(Some(timer)) = Self::pass_timer(&mut self.pack_post, &mut self.native) {
timer.submitted(&self.queue);
}
if let Some(timer) = &mut self.own_timing {
timer.submitted(&self.queue);
}
if offscreen && let Some(submission) = submission {
self.in_flight.push_back(submission);
if !self.paced_elsewhere {
@@ -71,7 +71,7 @@ use super::{
Frustum, SectionKey,
arena::{Arena, try_create},
occlusion::Reach,
pack_timing::PassTiming,
pack_timing::{PassTiming, compute_parts},
resident,
submit::Queue,
upload::Uploads,
@@ -374,6 +374,21 @@ pub(super) enum Prepared {
OutOfMemory,
}
/// A part of a compute pass: `pipeline` over `[x, y]` workgroups, nothing when `x` is 0.
fn step<'a>(
group: &'a wgpu::BindGroup,
pipeline: &'a wgpu::ComputePipeline,
[x, y]: [u32; 2],
) -> impl FnMut(&mut wgpu::ComputePass<'_>) + 'a {
move |pass| {
if x > 0 {
pass.set_bind_group(0, group, &[]);
pass.set_pipeline(pipeline);
pass.dispatch_workgroups(x, y, 1);
}
}
}
/// The frame's `tables` before the regions, into `words`, and the parameters
/// the frustum test and facing read with them; `None` when the CPU draws the
/// frame (`Prepared::Cpu`): a sum of a plane's terms could leave i32
@@ -1190,12 +1205,13 @@ impl GpuCulling {
uploads: &mut Uploads,
encoder: &mut wgpu::CommandEncoder,
input: &FrameInputs<'_>,
timing: Option<&mut PassTiming>,
) -> Prepared {
self.pages.clear();
self.ids = 0;
self.n_updates = 0;
self.occluding = false;
self.prepare_buffers(device, queue, uploads, encoder, input)
self.prepare_buffers(device, queue, uploads, encoder, input, timing)
.unwrap_or(Prepared::OutOfMemory)
}
@@ -1206,6 +1222,7 @@ impl GpuCulling {
uploads: &mut Uploads,
encoder: &mut wgpu::CommandEncoder,
input: &FrameInputs<'_>,
mut timing: Option<&mut PassTiming>,
) -> Option<Prepared> {
// Every buffer grows before anything is written. One made anew drops
// the bind group at once, and a new section table needs every record:
@@ -1287,6 +1304,9 @@ impl GpuCulling {
occl = OCCLUDE | if probe { PROBE } else { 0 };
}
}
let uploading = timing
.as_mut()
.and_then(|t| t.encoder_begin(encoder, "GPU culling uploads"));
if self.records.all_dirty {
// The whole table, after it grew or more records changed than
// updates take (GPU culling switched on in a loaded world, a
@@ -1352,6 +1372,9 @@ impl GpuCulling {
// are written before they are read.
let cleared = if self.occluding { tests_at } else { phase1_end };
encoder.clear_buffer(&self.counts.buffer, 0, Some(cleared * 4));
if let Some(t) = timing {
t.encoder_end(encoder, uploading);
}
if self.group.is_none() {
self.group = Some(self.bind(device));
if let Some(test) = &mut self.test {
@@ -1403,23 +1426,24 @@ impl GpuCulling {
let Some(group) = self.group.as_ref().filter(|_| self.ids > 0) else {
return;
};
let mut pass = encoder.begin_compute_pass(&wgpu::ComputePassDescriptor {
label: Some("GPU culling"),
timestamp_writes: timing.and_then(|t| t.compute_writes("GPU culling")),
});
pass.set_bind_group(0, group, &[]);
let [apply, cull, scan, scatter] = &self.pipelines;
if self.n_updates > 0 {
pass.set_pipeline(apply);
pass.dispatch_workgroups(self.n_updates.div_ceil(WORKGROUP), 1, 1);
}
let blocks = self.ids.div_ceil(WORKGROUP);
pass.set_pipeline(cull);
pass.dispatch_workgroups(blocks.min(ROW), blocks.div_ceil(ROW), 1);
pass.set_pipeline(scan);
pass.dispatch_workgroups(self.pass_pages, 1, 1);
pass.set_pipeline(scatter);
pass.dispatch_workgroups(blocks.min(ROW), blocks.div_ceil(ROW), 1);
let rows = [blocks.min(ROW), blocks.div_ceil(ROW)];
let updates = [self.n_updates.div_ceil(WORKGROUP), 1];
let (mut apply, mut cull) = (step(group, apply, updates), step(group, cull, rows));
let pages = [self.pass_pages, 1];
let (mut scan, mut scatter) = (step(group, scan, pages), step(group, scatter, rows));
compute_parts(
encoder,
timing,
"GPU culling",
&mut [
("apply", &mut apply),
("cull", &mut cull),
("scan", &mut scan),
("scatter", &mut scatter),
],
);
}
/// The kept sections' solid draws, a count draw per page, with the quad
@@ -1494,19 +1518,22 @@ impl GpuCulling {
if !self.occluding {
return;
}
let mut pass = encoder.begin_compute_pass(&wgpu::ComputePassDescriptor {
label: Some("GPU culling phase 2"),
timestamp_writes: timing.and_then(|t| t.compute_writes("GPU culling phase 2")),
});
pass.set_bind_group(0, group, &[]);
let [count2, scan2, scatter2] = &test.phase2;
let blocks = self.ids.div_ceil(WORKGROUP);
pass.set_pipeline(count2);
pass.dispatch_workgroups(blocks.min(ROW), blocks.div_ceil(ROW), 1);
pass.set_pipeline(scan2);
pass.dispatch_workgroups(self.pass_pages, 1, 1);
pass.set_pipeline(scatter2);
pass.dispatch_workgroups(blocks.min(ROW), blocks.div_ceil(ROW), 1);
let rows = [blocks.min(ROW), blocks.div_ceil(ROW)];
let pages = [self.pass_pages, 1];
let (mut count, mut scatter) = (step(group, count2, rows), step(group, scatter2, rows));
let mut scan = step(group, scan2, pages);
compute_parts(
encoder,
timing,
"GPU culling phase 2",
&mut [
("count2", &mut count),
("scan2", &mut scan),
("scatter2", &mut scatter),
],
);
}
/// After the frame's world passes (phase 2's counts too), in the same
@@ -2720,6 +2747,11 @@ mod tests {
let adapter = pollster::block_on(instance.request_adapter(&Default::default())).ok()?;
let needed =
wgpu::Features::MULTI_DRAW_INDIRECT_COUNT | wgpu::Features::INDIRECT_FIRST_INSTANCE;
let timing = adapter.features()
& (wgpu::Features::TIMESTAMP_QUERY
| wgpu::Features::TIMESTAMP_QUERY_INSIDE_PASSES
| wgpu::Features::TIMESTAMP_QUERY_INSIDE_ENCODERS
| wgpu::Features::SHADER_EARLY_DEPTH_TEST);
if !adapter.features().contains(needed)
|| super::super::flat::is_broken_intel_vulkan(&adapter.get_info())
{
@@ -2727,7 +2759,7 @@ mod tests {
}
eprintln!("GPU culling on {} ({name})", adapter.get_info().name);
pollster::block_on(adapter.request_device(&wgpu::DeviceDescriptor {
required_features: needed,
required_features: needed | timing,
..Default::default()
}))
.ok()
@@ -2834,6 +2866,11 @@ mod tests {
// was drawn.
let (mut eyed, mut hidden, mut several) = (0, 0, 0);
let (mut edge, mut edge_drawn) = (None, false);
// Timed in detail as the renderer does: the commands, and so the draws, stay the same.
let mut timing = PassTiming::new(device, queue);
if let Some(t) = &mut timing {
t.set_detail(true);
}
for case in 0..25 {
let radius = [2, 5, 9][case % 3];
let at_edge = case == 24;
@@ -2981,6 +3018,9 @@ mod tests {
edge = Some(id);
}
let whole = gpu.records.all_dirty;
if let Some(t) = &mut timing {
t.begin(device, queue);
}
let mut encoder = device.create_command_encoder(&Default::default());
let input = FrameInputs {
camera,
@@ -3000,11 +3040,18 @@ mod tests {
camera: camera + DVec3::Y * 2.0e7,
..input
};
let prepared = gpu.prepare(device, queue, &mut uploads, &mut encoder, &up);
let prepared = gpu.prepare(device, queue, &mut uploads, &mut encoder, &up, None);
assert_eq!(prepared, Prepared::Cpu, "case 7");
assert!(gpu.records.all_dirty || !gpu.records.dirty.is_empty());
}
let prepared = gpu.prepare(device, queue, &mut uploads, &mut encoder, &input);
let prepared = gpu.prepare(
device,
queue,
&mut uploads,
&mut encoder,
&input,
timing.as_mut(),
);
assert_eq!(prepared, Prepared::Gpu, "case {case}");
match case {
4 => assert!(
@@ -3060,12 +3107,15 @@ mod tests {
culled += 1;
}
}
gpu.dispatch(&mut encoder, None);
gpu.dispatch(&mut encoder, timing.as_mut());
// Its statistics come back with a CPU part of the case's own.
let cpu = [case * 7, case * 3, case];
gpu.copy_stats(&mut encoder, case as u64, cpu);
uploads.finish(&encoder);
queue.submit([encoder.finish()]);
if let Some(t) = &mut timing {
t.submitted(queue);
}
gpu.submitted().expect("a readback free")();
let pages = u64::from(gpu.pass_pages);
let counts = read(device, queue, &gpu.counts.buffer, (STATS + pages) * 4);
@@ -3124,6 +3174,18 @@ mod tests {
"of {eyed} from an eye, {hidden} with a bucket left out, {several} in three runs"
);
assert!(edge_drawn, "the bucket at the section's edge");
if let Some(t) = &timing {
let parts = t.part_medians();
let names = |pass: &str| {
let parts = parts.iter().find(|p| p.0 == pass)?;
Some(parts.1.iter().map(|p| p.0.as_str()).collect::<Vec<_>>())
};
assert_eq!(
names("GPU culling"),
Some(vec!["apply", "cull", "scan", "scatter"]),
"{parts:?}"
);
}
assert!(culled > 2_000, "{culled} within the distance left out");
assert!(faded * 10 > compared, "{faded} fading of {compared} drawn");
assert!(
@@ -3136,6 +3198,113 @@ mod tests {
);
}
/// Detail times both phases and the uploads and leaves counts and draws as an untimed frame's.
#[test]
fn detail_times_both_phases_and_leaves_the_draws_alone() {
on_each_device(|device, queue| {
let mut rng = Rng(0x1F83_D9AB_FB41_BD6B);
let (frustum, _) = view(&mut rng);
let records: Vec<Record> = (0..400)
.map(|i| {
let key = (rng.range(-6, 6), rng.range(0, 23), rng.range(-6, 6));
Record::new(
key,
i % 3,
u64::from(i) * 1024,
1 + rng.below(500),
UNBOUNDED,
)
})
.collect();
let mut timing = PassTiming::new(device, queue);
let early = device
.features()
.contains(wgpu::Features::SHADER_EARLY_DEPTH_TEST);
let Some(t) = timing.as_mut().filter(|_| early) else {
eprintln!("no timestamps or early depth test: skipped");
return;
};
t.set_detail(true);
let run = |timing: &mut Option<PassTiming>| {
let mut gpu = GpuCulling::new(device, Some(wgpu::TextureFormat::Rgba8Unorm))
.expect("the pass");
let mut uploads = Uploads::new(device);
for r in &records {
gpu.records.insert(*r);
}
let mut out = Vec::new();
for frame in 0..12 {
if let Some(t) = timing {
t.begin(device, queue);
}
let mut encoder = device.create_command_encoder(&Default::default());
let input = FrameInputs {
camera: DVec3::new(0.5, 70.0, 0.5),
min_y: -64,
sections: 24,
radius: 5,
frustum: &frustum,
reach: None,
eye: None,
view_proj: Mat4::IDENTITY,
occlude: true,
};
let prepared = gpu.prepare(
device,
queue,
&mut uploads,
&mut encoder,
&input,
timing.as_mut(),
);
assert_eq!(prepared, Prepared::Gpu, "frame {frame}");
assert!(gpu.occluding(), "frame {frame}: phase 2 runs");
gpu.dispatch(&mut encoder, timing.as_mut());
gpu.compact(&mut encoder, timing.as_mut());
uploads.finish(&encoder);
queue.submit([encoder.finish()]);
if let Some(t) = timing {
t.submitted(queue);
}
device.poll(wgpu::PollType::wait_indefinitely()).unwrap();
let pages = u64::from(gpu.pass_pages);
out.push((
read(device, queue, &gpu.counts.buffer, (STATS + 2 * pages) * 4),
read(device, queue, &gpu.draws.buffer, gpu.draws.bytes),
));
}
out
};
let timed = run(&mut timing);
let plain = run(&mut None);
assert_eq!(timed, plain, "detail changes what is drawn");
let t = timing.as_ref().expect("the timer");
let parts = t.part_medians();
let names = |pass: &str| {
let parts = parts.iter().find(|p| p.0 == pass)?;
Some(parts.1.iter().map(|p| p.0.as_str()).collect::<Vec<_>>())
};
assert_eq!(
names("GPU culling"),
Some(vec!["apply", "cull", "scan", "scatter"]),
"{parts:?}"
);
assert_eq!(
names("GPU culling phase 2"),
Some(vec!["count2", "scan2", "scatter2"]),
"{parts:?}"
);
// The uploads are a pass of their own, never a part of one.
assert_eq!(names("GPU culling uploads"), None, "{parts:?}");
let passes = t.medians();
assert_eq!(
passes.iter().any(|p| p.0 == "GPU culling uploads"),
t.detail_support().encoder_stamps,
"{passes:?}"
);
});
}
/// The statistics come back with their frame, in place and from the submit
/// thread, submitted as gfx.rs submits frames (`submit_later`, or in place
/// with its `after` handed back): three frames on their way fill the ring, a
@@ -3377,7 +3546,7 @@ struct Quad {
view_proj: Mat4::IDENTITY,
occlude: false,
};
let prepared = gpu.prepare(device, queue, &mut uploads, &mut encoder, &input);
let prepared = gpu.prepare(device, queue, &mut uploads, &mut encoder, &input, None);
assert_eq!(prepared, Prepared::Gpu, "frame {frame}");
let mut params = tables(&input, &mut words).expect("the planes fit");
params.pages = gpu.pass_pages;
+4 -1
View File
@@ -696,13 +696,16 @@ impl Oit {
),
] {
{
let mut pass = encoder.begin_render_pass(&wgpu::RenderPassDescriptor {
let pass = encoder.begin_render_pass(&wgpu::RenderPassDescriptor {
label: Some(label),
timestamp_writes: timing.as_deref_mut().and_then(|t| t.writes(label)),
color_attachments: &first,
depth_stencil_attachment: read_only(terrain_depth),
..Default::default()
});
let pixels = super::pixel_count(terrain_depth);
let mut pass =
super::pack_timing::counted(timing.as_deref_mut(), pass, pixels, label);
pass.set_bind_group(GROUP, group, &[]);
draw(&mut pass, stage);
}
File diff suppressed because it is too large. Load diff
@@ -0,0 +1,169 @@
//! The world pass after culling, and the renderer's GPU pass timing switches and readings.
use super::{Renderer, native_pack, pack_post, pack_timing};
pub(super) fn world_pass_after_culling<'e>(
encoder: &'e mut wgpu::CommandEncoder,
view: &wgpu::TextureView,
depth: &wgpu::TextureView,
keep_depth: bool,
mut timing: Option<&mut pack_timing::PassTiming>,
name: &str,
pixels: u64,
) -> pack_timing::CountedPass<'e> {
let pass = after_culling_pass(
encoder,
view,
depth,
keep_depth,
timing.as_deref_mut().and_then(|t| t.writes(name)),
);
pack_timing::counted(timing, pass, pixels, "before translucent")
}
/// The pixels of a depth attachment, for `fragments_per_pixel`.
pub(super) fn pixel_count(depth: &wgpu::TextureView) -> u64 {
let texture = depth.texture();
u64::from(texture.width()) * u64::from(texture.height())
}
pub(super) fn after_culling_pass<'e>(
encoder: &'e mut wgpu::CommandEncoder,
view: &wgpu::TextureView,
depth: &wgpu::TextureView,
keep_depth: bool,
timestamp_writes: Option<wgpu::RenderPassTimestampWrites<'_>>,
) -> wgpu::RenderPass<'e> {
encoder.begin_render_pass(&wgpu::RenderPassDescriptor {
label: Some("Horizon world, after culling"),
timestamp_writes,
color_attachments: &[Some(wgpu::RenderPassColorAttachment {
view,
resolve_target: None,
depth_slice: None,
ops: wgpu::Operations {
load: wgpu::LoadOp::Load,
store: wgpu::StoreOp::Store,
},
})],
depth_stencil_attachment: Some(wgpu::RenderPassDepthStencilAttachment {
view: depth,
depth_ops: Some(wgpu::Operations {
load: wgpu::LoadOp::Load,
// A shader pack reads it as depthtex0.
store: if keep_depth {
wgpu::StoreOp::Store
} else {
wgpu::StoreOp::Discard
},
}),
stencil_ops: None,
}),
..Default::default()
})
}
impl Renderer {
/// Times the shader pack's passes on the GPU while `on` (the F3 overlay is open or a capture records them).
pub fn set_pass_timing(&mut self, on: bool) {
self.pass_timing = on;
}
/// Times the parts of Horizon's own passes while `on`.
pub fn set_pass_detail(&mut self, on: bool) {
self.pass_detail = on;
}
/// Times a frame at most once per `gap` while passes are timed (the F3 overlay needs far fewer
/// samples than a measurement).
pub fn set_pass_timing_gap(&mut self, gap: std::time::Duration) {
self.pass_timing_gap = gap;
}
/// Keeps each timed frame's pass times for `take_timed_frames` while `on` (a capture records them).
pub fn set_keep_timed_frames(&mut self, on: bool) {
self.keep_timed_frames = on;
}
/// The frames both timers timed since the last call, without waiting on the GPU.
pub fn take_timed_frames(&mut self) -> pack_timing::TimedFrames {
let own = self
.own_timing
.as_mut()
.map(pack_timing::PassTiming::take_frames)
.unwrap_or_default();
let pack = match Self::pass_timer(&mut self.pack_post, &mut self.native) {
Some(Some(t)) => t.take_frames(),
_ => Vec::new(),
};
pack_timing::TimedFrames { own, pack }
}
/// The running shader pack's timer, of either kind.
pub(super) fn pass_timer<'a>(
pack: &'a mut Option<(i32, Option<pack_post::PackPost>)>,
native: &'a mut Option<native_pack::NativePack>,
) -> Option<&'a mut Option<pack_timing::PassTiming>> {
match (pack, native) {
(Some((_, Some(post))), _) => Some(&mut post.timing),
(_, Some(native)) => Some(&mut native.timing),
_ => None,
}
}
/// Each pass of the running shader pack with its median GPU time in milliseconds, while
/// they are timed (`set_pass_timing`).
pub fn shader_pass_times(&self) -> Vec<(String, f32)> {
let timer = match (&self.pack_post, &self.native) {
(Some((_, Some(post))), _) => post.timing.as_ref(),
(_, Some(native)) => native.timing.as_ref(),
_ => None,
};
timer
.map(pack_timing::PassTiming::medians)
.unwrap_or_default()
}
/// Each of Horizon's own passes with its median GPU time in milliseconds, while they are
/// timed (`set_pass_timing`).
pub fn own_pass_times(&self) -> Vec<(String, f32)> {
self.own_timing
.as_ref()
.map(pack_timing::PassTiming::medians)
.unwrap_or_default()
}
/// Each of Horizon's own passes with its parts' median GPU times in milliseconds.
pub fn own_pass_parts(&self) -> Vec<(String, Vec<(String, f32)>)> {
self.own_timing
.as_ref()
.map(pack_timing::PassTiming::part_medians)
.unwrap_or_default()
}
/// Each counted pass's counters (medians), while detail is on and the device has them.
pub fn own_pass_counters(&self) -> Vec<(String, pack_timing::PassCounters)> {
self.own_timing
.as_ref()
.map(pack_timing::PassTiming::counter_medians)
.unwrap_or_default()
}
/// Both timers' readings and this adapter, for a capture; `None` when neither has timed a pass.
pub fn gpu_readout(&self) -> Option<pack_timing::GpuReadout> {
let own = self
.own_timing
.as_ref()
.map(pack_timing::PassTiming::readings)
.filter(|r| !r.is_empty());
let pack = match (&self.pack_post, &self.native) {
(Some((_, Some(post))), _) => post.timing.as_ref(),
(_, Some(native)) => native.timing.as_ref(),
_ => None,
}
.map(pack_timing::PassTiming::readings)
.filter(|r| !r.is_empty());
(own.is_some() || pack.is_some()).then(|| pack_timing::GpuReadout {
adapter: self.info.clone(),
own,
pack,
})
}
/// Whether dispatch parts and counters are being asked for (`set_pass_detail`).
pub fn pass_detail(&self) -> bool {
self.pass_detail
}
/// What detail timing can show on this device; `None` without pass timing.
pub fn pass_detail_support(&self) -> Option<pack_timing::DetailSupport> {
self.own_timing
.as_ref()
.map(pack_timing::PassTiming::detail_support)
}
}
@@ -2,8 +2,8 @@
//! an adapter that can't present to the window gives way to another of its backend
//! (#13493).
use super::{
Renderer, check_layers, cull, describe_adapter, is_software, oit, startup_step, surface_format,
timing,
Renderer, check_layers, cull, describe_adapter, is_software, oit, pack_timing, startup_step,
surface_format, timing,
};
// Ray tracing on request, which the device needs from its creation (gfx.rs may not grow).
#[path = "ray_query.rs"]
@@ -135,6 +135,7 @@ pub(super) async fn open_adapter(
// atomics on images: their buffers and images. Filtering 32-bit float textures: shader
// packs' RGBA32F buffers, which GL filters and mipmaps. Count draws: GPU culling's (`gpu_culling`).
let features = timing::GpuTimer::features(adapter)
| pack_timing::PassTiming::features(adapter)
| cull::Cull::features(adapter)
| oit::features(adapter)
| ray_query::features(adapter)
@@ -239,17 +239,17 @@ impl Visbuffer {
encoder: &'e mut wgpu::CommandEncoder,
depth: &wgpu::TextureView,
clear: bool,
timing: Option<&mut PassTiming>,
) -> wgpu::RenderPass<'e> {
mut timing: Option<&mut PassTiming>,
) -> super::pack_timing::CountedPass<'e> {
let (ids, _) = self.ids.as_ref().expect("fitted before the world pass");
let name = if clear {
"Visibility IDs"
} else {
"Visibility IDs after culling"
};
encoder.begin_render_pass(&wgpu::RenderPassDescriptor {
let pass = encoder.begin_render_pass(&wgpu::RenderPassDescriptor {
label: Some("Horizon visibility buffer"),
timestamp_writes: timing.and_then(|t| t.writes(name)),
timestamp_writes: timing.as_deref_mut().and_then(|t| t.writes(name)),
color_attachments: &[Some(wgpu::RenderPassColorAttachment {
view: ids,
resolve_target: None,
@@ -272,7 +272,8 @@ impl Visbuffer {
stencil_ops: None,
}),
..Default::default()
})
});
super::pack_timing::counted(timing, pass, super::pixel_count(depth), name)
}
/// Phase 1's solid calls into the IDs, as the normal passes draw them: the opaque
+3 -3
View File
@@ -1401,9 +1401,9 @@ impl HudAtlas {
edition.3 = pixels;
}
images.push(("horizon/white".into(), 4, 4, vec![255; 64]));
// The controller's art, unless a pack brought its own (walked above, as a mod's GUI
// textures are).
for image in crate::pad_glyph::images() {
// The controller's art and Horizon's mark, unless a pack brought its own (walked above, as a
// mod's GUI textures are).
for image in crate::pad_glyph::images().chain(crate::mark_sprite::image()) {
if !images.iter().any(|i| i.0 == image.0) {
images.push(image);
}
+4
View File
@@ -74,6 +74,7 @@ pub mod level_loading;
pub mod light;
pub mod lod;
pub mod logging;
pub mod mark_sprite;
pub mod menu_blur;
pub mod menu_style;
pub mod mesher;
@@ -102,6 +103,9 @@ pub mod pad_glyph;
pub mod paged_maps;
pub mod par;
pub mod particles;
#[cfg(target_os = "linux")]
pub mod perf_capture;
pub mod perf_gpu;
pub mod poi_sections;
pub mod polytone_post;
pub mod post_chain;
@@ -0,0 +1,11 @@
//! Horizon's mark (`docs/media/brand/mark-dark.svg`) on the text grid, for an inline text sprite.
/// The mark's name in the HUD atlas.
pub const SPRITE: &str = "horizon:gui/mark";
const PNG: &[u8] = include_bytes!("../../horizon-client/assets/horizon/textures/gui/mark.png");
/// The mark as the HUD atlas takes an image.
pub fn image() -> std::iter::Once<(String, u32, u32, Vec<u8>)> {
let (width, height, rgba) = crate::hud::decode_bytes(PNG).expect("Horizon's mark is PNG");
std::iter::once((SPRITE.to_owned(), width, height, rgba))
}
@@ -0,0 +1,406 @@
//! A stopped sampler session as the capture's `samples` message (docs/perf-platform.md §3.5).
use std::collections::HashMap;
use horizon_perf_format::Interner;
use horizon_perf_format::v1::{
BlockUnits, Contexts, Functions, Nodes, Overhead, SampleKind, SampleProfile, SampleSource,
SampleUnit, SampledThread, Side, Unit, Weights,
};
use horizon_threads::heap_sample::HeapProfile;
use horizon_threads::profile::{Aggregate, Frame, Profile, State, Symbols, symbolize};
use horizon_threads::work::{self, Space};
#[cfg(test)]
mod tests;
const UNKNOWN: &str = "[unknown]";
/// The name `work::set_dim` ids stand for: 1 overworld, 2 the nether, 3 the end.
fn dimension_name(id: u8) -> Option<&'static str> {
match id {
1 => Some("overworld"),
2 => Some("the_nether"),
3 => Some("the_end"),
_ => None,
}
}
/// The crate a demangled function name belongs to, or "" when it names none.
fn crate_of(name: &str) -> &str {
if let Some(inner) = name.strip_prefix('[') {
let module = inner.split([' ', ']']).next().unwrap_or("");
return if module == "unmapped" { "" } else { module };
}
let name = name.trim_start_matches(['<', '&', '*']);
let name = name.strip_prefix("dyn ").unwrap_or(name);
match name.split_once("::") {
Some((krate, _)) if krate.chars().all(|c| c.is_alphanumeric() || c == '_') => krate,
_ => "",
}
}
#[derive(Default)]
struct FunctionTable {
rows: Functions,
index: HashMap<(u32, u32), u32>,
allocator: Vec<bool>,
}
impl FunctionTable {
fn id(
&mut self,
strings: &mut Interner,
name: &str,
file: Option<&str>,
line: u32,
addr: u64,
) -> u32 {
let (name_id, file_id) = (
strings.intern(name),
file.map_or(0, |file| strings.intern(file)),
);
if let Some(&row) = self.index.get(&(name_id, file_id)) {
return row;
}
let row = self.rows.name.len() as u32;
self.rows.name.push(name_id);
self.rows.file.push(file_id);
self.rows.line.push(line);
self.rows.crate_.push(match crate_of(name) {
"" => 0,
krate => strings.intern(krate),
});
self.rows.addr.push(addr);
self.allocator.push(allocator_frame(name));
self.index.insert((name_id, file_id), row);
row
}
}
/// The thread each trie node belongs to; node 0 is the root.
fn thread_of(agg: &Aggregate) -> Vec<u16> {
let mut of = vec![0u16; agg.nodes()];
for n in 1..agg.nodes() {
of[n] = match agg.frame(n as u32) {
Frame::Thread(i) => i,
_ => of[agg.parent(n as u32) as usize],
};
}
of
}
/// The trie nodes some counted sample passes through.
fn live_nodes(agg: &Aggregate) -> Vec<bool> {
let mut live = vec![false; agg.nodes()];
for key in agg.counts.keys() {
let mut n = key.node as usize;
while n != 0 && !live[n] {
live[n] = true;
n = agg.parent(n as u32) as usize;
}
}
live
}
#[derive(Default)]
struct NodeTable {
rows: Nodes,
index: HashMap<(Option<u32>, u32, u16), u32>,
}
impl NodeTable {
fn id(&mut self, parent: Option<u32>, function: u32, thread: u16) -> u32 {
let key = (parent, function, if parent.is_none() { thread } else { 0 });
if let Some(&row) = self.index.get(&key) {
return row;
}
let nodes = &mut self.rows;
nodes.parent.push(parent.unwrap_or(0));
nodes.function.push(function);
nodes.thread_root.push(match parent {
None => u32::from(thread) + 1,
Some(_) => 0,
});
let row = nodes.parent.len() as u32 - 1;
self.index.insert(key, row);
row
}
}
fn named(strings: &mut Interner, space: Space, id: u16) -> u32 {
match id {
0 => 0,
id => match work::name(space, id) {
Some(name) => strings.intern(name),
None => strings.intern(&format!("#{id}")),
},
}
}
/// Whether `name` is the allocator's own frame, which a heap sample's stack drops from its leaf end.
fn allocator_frame(name: &str) -> bool {
let name = name.trim_start_matches('<');
name.starts_with("horizon_threads::heap_sample::")
|| name.starts_with("__rust")
|| name.starts_with("__rg_")
|| name.starts_with("__rdl_")
|| name.starts_with("alloc::alloc::")
|| name.starts_with("alloc::raw_vec::")
|| name.starts_with("mimalloc::")
}
/// One aggregate's trie nodes as rows of the shared tables: `last[n]` is the output node of trie node `n`.
fn add_tree(
agg: &Aggregate,
threads: &[u16],
symbols: &Symbols,
functions: &mut FunctionTable,
nodes: &mut NodeTable,
strings: &mut Interner,
) -> (Vec<u16>, Vec<Option<u32>>) {
let (local, live) = (thread_of(agg), live_nodes(agg));
let thread_of: Vec<u16> = local
.iter()
.map(|&t| threads.get(usize::from(t)).copied().unwrap_or(0))
.collect();
let mut last: Vec<Option<u32>> = vec![None; agg.nodes()];
for n in (1..agg.nodes()).filter(|&n| live[n]) {
let inherited = last[agg.parent(n as u32) as usize];
last[n] = match agg.frame(n as u32) {
Frame::Thread(_) => None,
Frame::Wall(_) => inherited,
Frame::Addr(addr) => {
let offset = symbols.module_offset(addr);
let mut at = inherited;
for sym in symbols.frames(addr).iter().rev() {
let f = functions.id(strings, &sym.name, sym.file.as_deref(), sym.line, offset);
at = Some(nodes.id(at, f, thread_of[n]));
}
if at == inherited {
let f = functions.id(strings, UNKNOWN, None, 0, offset);
at = Some(nodes.id(at, f, thread_of[n]));
}
at
}
};
}
(thread_of, last)
}
fn trie_addrs(agg: &Aggregate) -> impl Iterator<Item = u64> + '_ {
let live = live_nodes(agg);
(1..agg.nodes())
.filter(move |&n| live[n])
.filter_map(|n| match agg.frame(n as u32) {
Frame::Addr(a) => Some(a),
_ => None,
})
}
/// `profile` (and the heap session's `heap`) as a `SampleProfile` of `side`; strings go through `strings`, whose new entries the caller appends first.
pub fn sample_profile(
profile: &Profile,
heap: Option<&HeapProfile>,
side: Side,
unit: Unit,
block_name: &dyn Fn(u16) -> Option<String>,
strings: &mut Interner,
) -> SampleProfile {
let agg = &profile.agg;
let addrs: Vec<u64> = trie_addrs(agg)
.chain(heap.into_iter().flat_map(|h| trie_addrs(&h.agg)))
.collect();
let symbols = symbolize(addrs);
let mut names: Vec<String> = agg.threads.clone();
let cpu_threads: Vec<u16> = (0..agg.threads.len() as u16).collect();
let heap_threads: Vec<u16> = heap.map_or_else(Vec::new, |h| {
h.agg
.threads
.iter()
.map(|name| match names.iter().position(|n| n == name) {
Some(at) => at as u16,
None => {
names.push(name.clone());
names.len() as u16 - 1
}
})
.collect()
});
let mut functions = FunctionTable::default();
let mut nodes = NodeTable::default();
let mut trees = vec![(
agg,
add_tree(
agg,
&cpu_threads,
&symbols,
&mut functions,
&mut nodes,
strings,
),
)];
if let Some(h) = heap {
let tree = add_tree(
&h.agg,
&heap_threads,
&symbols,
&mut functions,
&mut nodes,
strings,
);
trees.push((&h.agg, tree));
}
let wall = agg.counts.keys().any(|k| k.state != State::Cpu);
let period_ms = 1000.0 / f64::from(profile.config.sampler.hz.max(1));
let kind = |source: SampleSource, unit: SampleUnit, period: f64| SampleKind {
source: source as i32,
unit: unit as i32,
period,
};
let mut kinds = vec![kind(SampleSource::Cpu, SampleUnit::Ms, period_ms)];
if wall {
kinds.push(kind(SampleSource::Wall, SampleUnit::Ms, period_ms));
}
if heap.is_some() {
kinds.push(kind(SampleSource::Alloc, SampleUnit::Bytes, 1.0));
kinds.push(kind(SampleSource::Live, SampleUnit::Bytes, 1.0));
}
let kind_of = |state: State| -> u32 {
let source = match state {
State::Cpu => SampleSource::Cpu,
State::Running | State::Waiting => SampleSource::Wall,
State::Alloc => SampleSource::Alloc,
State::Live => SampleSource::Live,
};
kinds
.iter()
.position(|k| k.source == source as i32)
.unwrap_or(0) as u32
};
let mut contexts = Contexts::default();
let mut context_ids: HashMap<[u32; 6], u32> = HashMap::new();
let mut weights = Weights::default();
let mut unknown_roots: HashMap<u16, u32> = HashMap::new();
for (agg, (thread_of, last)) in &trees {
let mut counted: Vec<_> = agg.counts.iter().collect();
counted.sort_by_key(|(k, _)| {
(
thread_of[k.node as usize],
k.node,
k.state as u8,
k.slow,
k.time_block,
(k.span, k.entity, k.block, k.dim, k.chunk),
)
});
for (key, &count) in counted {
let thread = thread_of[key.node as usize];
let mut node = match last[key.node as usize] {
Some(node) => node,
None => *unknown_roots.entry(thread).or_insert_with(|| {
let f = functions.id(strings, UNKNOWN, None, 0, 0);
nodes.id(None, f, thread)
}),
};
if matches!(key.state, State::Alloc | State::Live) {
while nodes.rows.thread_root[node as usize] == 0
&& functions.allocator[nodes.rows.function[node as usize] as usize]
{
node = nodes.rows.parent[node as usize];
}
}
let row = [
named(strings, Space::Span, key.span),
named(strings, Space::Entity, key.entity),
match key.block {
0 => 0,
id => match block_name(id) {
Some(name) => strings.intern(&name),
None => strings.intern(&format!("#{id}")),
},
},
dimension_name(key.dim).map_or(0, |name| strings.intern(name)),
key.chunk.0 as u32,
key.chunk.1 as u32,
];
let context = *context_ids.entry(row).or_insert_with(|| {
contexts.span.push(row[0]);
contexts.entity.push(row[1]);
contexts.block.push(row[2]);
contexts.dim.push(row[3]);
contexts.chunk_x.push(key.chunk.0);
contexts.chunk_z.push(key.chunk.1);
contexts.span.len() as u32 - 1
});
weights.node.push(node);
weights.thread.push(u32::from(thread));
weights.kind.push(kind_of(key.state));
weights.state.push(match key.state {
State::Running => 1,
State::Waiting => 2,
State::Cpu | State::Alloc | State::Live => 0,
});
weights.slow.push(key.slow);
weights.context.push(context);
weights.block.push(u32::from(key.time_block));
weights.count.push(count);
}
}
let blocks = agg.time_blocks();
let mut threads = Vec::new();
let mut block_units = Vec::new();
for (i, name) in names.iter().enumerate() {
let ticks = agg.ticks.get(&(i as u16)).filter(|_| i < agg.threads.len());
threads.push(SampledThread {
name: strings.intern(name),
units: ticks.map_or(0, |t| t.count),
slow_units: ticks.map_or(0, |t| t.slow),
unit: unit as i32,
});
if i >= agg.threads.len() {
continue;
}
let mut units = agg
.block_ticks
.get(&(i as u16))
.cloned()
.unwrap_or_default();
units.resize(blocks, 0);
block_units.push(BlockUnits {
thread: i as u32,
units,
});
}
SampleProfile {
side: side as i32,
kinds,
threads,
block_s: profile.config.block_s,
block_units,
functions: Some(functions.rows),
nodes: Some(nodes.rows),
contexts: Some(contexts),
weights: Some(weights),
slow_over_ms: profile.config.ticks_over_ms as f32,
}
}
/// What the sampler and its writer thread cost during the session.
pub fn overhead(profile: &Profile, heap: Option<&HeapProfile>) -> Overhead {
let wall_ns = profile.elapsed.as_nanos().max(1) as f64;
let handler_ns = profile.stats.handler_ns.unwrap_or(0.0) * profile.stats.samples as f64;
Overhead {
sampler_share_of_thread: (handler_ns / wall_ns) as f32,
writer_core_share: (profile.writer_cpu_ns as f64 / wall_ns) as f32,
dropped_samples: profile.stats.dropped + profile.agg.lost + heap.map_or(0, |h| h.agg.lost),
heap_samples: heap.map_or(0, |h| h.samples),
heap_sample_ns: heap.map_or(0, |h| h.sample_ns),
..Overhead::default()
}
}
@@ -0,0 +1,649 @@
use std::collections::HashMap;
use std::hint::black_box;
use std::sync::mpsc;
use std::thread;
use std::time::{Duration, SystemTime};
use horizon_perf_format::v1::{
Kind as CaptureKind, SampleProfile, Side, Start, Stop, StringEntry, Unit,
};
use horizon_perf_format::{CaptureWriter, Field};
use horizon_threads::profile::{self, Aggregate, Config, Profile};
use horizon_threads::sampler::{self, Context, Kind, Record, Stats};
use horizon_threads::unwind::MAX_FRAMES;
use horizon_threads::work;
use super::*;
#[inline(never)]
fn alpha() {
black_box(1);
}
#[inline(never)]
fn beta() {
black_box(2);
}
#[inline(never)]
fn gamma() {
black_box(3);
}
fn at(f: fn(), offset: usize) -> usize {
f as *const () as usize + offset
}
fn record(kind: Kind, ctx: Context, value: u64, stack: &[usize]) -> Record {
let mut frames = [0; MAX_FRAMES];
frames[..stack.len()].copy_from_slice(stack);
Record {
kind,
depth: stack.len() as u8,
ctx,
value,
frames,
}
}
fn profile_of(agg: Aggregate, config: Config) -> Profile {
Profile {
agg,
config,
stats: Stats::default(),
elapsed: Duration::from_secs(10),
started: SystemTime::now(),
writer_cpu_ns: 0,
}
}
struct Converted {
samples: SampleProfile,
strings: Vec<StringEntry>,
}
impl Converted {
fn text(&self, id: u32) -> &str {
match id {
0 => "",
id => &self.strings.iter().find(|s| s.id == id).unwrap().text,
}
}
fn function(&self, row: u32) -> &str {
self.text(self.samples.functions.as_ref().unwrap().name[row as usize])
}
/// Function names from the root down to `node`.
fn path(&self, node: u32) -> Vec<&str> {
let nodes = self.samples.nodes.as_ref().unwrap();
let mut path = vec![self.function(nodes.function[node as usize])];
let mut at = node as usize;
while nodes.thread_root[at] == 0 {
at = nodes.parent[at] as usize;
path.push(self.function(nodes.function[at]));
}
path.reverse();
path
}
/// Samples per entity name, optionally only the slow ones.
fn by_entity(&self, slow_only: bool) -> HashMap<String, u64> {
let (w, c) = (
self.samples.weights.as_ref().unwrap(),
self.samples.contexts.as_ref().unwrap(),
);
let mut by = HashMap::new();
for i in 0..w.node.len() {
if slow_only && !w.slow[i] {
continue;
}
let entity = self.text(c.entity[w.context[i] as usize]).to_owned();
*by.entry(entity).or_default() += w.count[i];
}
by
}
}
fn convert(profile: &Profile) -> Converted {
let mut interner = Interner::default();
let samples = sample_profile(
profile,
None,
Side::Server,
Unit::Tick,
&|id| (id == 7).then(|| "minecraft:hopper".to_owned()),
&mut interner,
);
Converted {
samples,
strings: interner.take_new(Side::Server),
}
}
fn short(name: &str) -> &str {
name.rsplit("::").next().unwrap()
}
#[test]
fn a_stack_becomes_a_root_to_leaf_chain_with_its_context() {
let entity = work::entity_id("test:zombie");
let ctx = Context {
tick: 1,
entity_type: entity,
block: 7,
dim: 2,
chunk: (-3, 9),
..Context::default()
};
let mut agg = Aggregate::new(50_000_000);
for _ in 0..5 {
agg.add(
"tick",
&record(Kind::Cpu, ctx, 0, &[at(alpha, 4), at(gamma, 3)]),
);
}
agg.add("tick", &record(Kind::Tick, ctx, 9_000_000, &[]));
agg.finish();
let c = convert(&profile_of(agg, Config::default()));
let w = c.samples.weights.as_ref().unwrap();
assert_eq!(
(w.node.len(), w.count[0], w.state[0], w.slow[0]),
(1, 5, 0, false)
);
let path: Vec<_> = c.path(w.node[0]).into_iter().map(short).collect();
assert_eq!(path, ["gamma", "alpha"]);
let nodes = c.samples.nodes.as_ref().unwrap();
assert_eq!(nodes.thread_root, [1, 0]);
let ctxs = c.samples.contexts.as_ref().unwrap();
let row = w.context[0] as usize;
assert_eq!(c.text(ctxs.entity[row]), "test:zombie");
assert_eq!(c.text(ctxs.block[row]), "minecraft:hopper");
assert_eq!(c.text(ctxs.dim[row]), "the_nether");
assert_eq!((ctxs.chunk_x[row], ctxs.chunk_z[row]), (-3, 9));
assert_eq!(c.text(c.samples.threads[0].name), "tick");
assert_eq!(
(c.samples.threads[0].units, c.samples.threads[0].slow_units),
(1, 0)
);
}
#[test]
fn addresses_of_one_function_share_one_row() {
let ctx = Context::default();
let mut agg = Aggregate::new(50_000_000);
for (inner, count) in [(4, 2), (8, 3)] {
for _ in 0..count {
agg.add(
"tick",
&record(Kind::Cpu, ctx, 0, &[at(alpha, inner), at(gamma, 3)]),
);
}
}
agg.add(
"tick",
&record(Kind::Cpu, ctx, 0, &[at(beta, 4), at(gamma, 3)]),
);
agg.finish();
let c = convert(&profile_of(agg, Config::default()));
let f = c.samples.functions.as_ref().unwrap();
let names: Vec<_> = (0..f.name.len() as u32)
.map(|row| short(c.function(row)))
.collect();
let mut sorted = names.clone();
sorted.sort_unstable();
assert_eq!(sorted, ["alpha", "beta", "gamma"]);
let w = c.samples.weights.as_ref().unwrap();
let on_alpha: u64 = (0..w.node.len())
.filter(|&i| short(c.path(w.node[i]).last().unwrap()) == "alpha")
.map(|i| w.count[i])
.sum();
assert_eq!(on_alpha, 5);
let n = c.samples.nodes.as_ref().unwrap();
let alphas: Vec<usize> = (0..n.function.len())
.filter(|&i| short(c.function(n.function[i])) == "alpha")
.collect();
assert_eq!(alphas.len(), 1, "one node per (parent, function)");
assert_eq!(
short(c.function(n.function[n.parent[alphas[0]] as usize])),
"gamma"
);
let weight_on_alpha: u64 = (0..w.node.len())
.filter(|&i| w.node[i] as usize == alphas[0])
.map(|i| w.count[i])
.sum();
assert_eq!(weight_on_alpha, 5);
let alpha_row = names.iter().position(|n| *n == "alpha").unwrap();
assert!(f.addr[alpha_row] > 0, "a module-relative address is kept");
}
#[test]
fn time_blocks_and_ticks_per_block_follow_when_they_were_added() {
let ctx = |tick| Context {
tick,
..Context::default()
};
let mut agg = Aggregate::new(50_000_000);
for (block, tick, samples) in [(0, 1, 2u64), (1, 2, 3), (1, 3, 1), (3, 4, 4)] {
agg.set_time_block(block);
for _ in 0..samples {
agg.add("tick", &record(Kind::Cpu, ctx(tick), 0, &[at(alpha, 1)]));
}
agg.add("tick", &record(Kind::Tick, ctx(tick), 1_000_000, &[]));
}
agg.finish();
let config = Config {
block_s: 2.5,
..Config::default()
};
let c = convert(&profile_of(agg, config));
let w = c.samples.weights.as_ref().unwrap();
let mut per_block = [0u64; 4];
for i in 0..w.node.len() {
per_block[w.block[i] as usize] += w.count[i];
}
assert_eq!(per_block, [2, 4, 0, 4]);
assert_eq!(c.samples.block_s, 2.5);
assert_eq!(c.samples.block_units[0].units, [1, 2, 0, 1]);
assert_eq!(c.samples.threads[0].units, 4);
}
#[test]
fn a_sample_with_no_stack_hangs_off_an_unknown_root() {
let mut agg = Aggregate::new(50_000_000);
agg.add("tick", &record(Kind::Cpu, Context::default(), 0, &[]));
agg.finish();
let c = convert(&profile_of(agg, Config::default()));
let w = c.samples.weights.as_ref().unwrap();
assert_eq!(c.path(w.node[0]), ["[unknown]"]);
}
#[test]
fn wall_samples_carry_their_state_and_a_second_kind() {
let mut agg = Aggregate::new(50_000_000);
let ctx = Context::default();
agg.add("tick", &record(Kind::Cpu, ctx, 0, &[at(alpha, 1)]));
agg.add("tick", &record(Kind::Wall, ctx, 1, &[at(alpha, 1)]));
agg.add("tick", &record(Kind::Wall, ctx, 0, &[at(beta, 1)]));
agg.finish();
let config = Config {
sampler: sampler::Config {
hz: 200,
..sampler::Config::default()
},
..Config::default()
};
let c = convert(&profile_of(agg, config));
assert_eq!(c.samples.kinds.len(), 2);
assert_eq!(c.samples.kinds[0].period, 5.0);
let w = c.samples.weights.as_ref().unwrap();
let mut seen: Vec<_> = (0..w.node.len())
.map(|i| {
(
short(c.path(w.node[i]).last().unwrap()).to_owned(),
w.kind[i],
w.state[i],
)
})
.collect();
seen.sort();
assert_eq!(
seen,
[
("alpha".into(), 0, 0),
("alpha".into(), 1, 1),
("beta".into(), 1, 2)
]
);
}
#[inline(never)]
fn spin_fast() {
spin(8);
black_box(0);
}
#[inline(never)]
fn spin_slow() {
spin(120);
black_box(1);
}
fn spin(cpu_ms: u64) {
let until = sampler::thread_cpu_ns() + cpu_ms * 1_000_000;
let mut x = 1u64;
while sampler::thread_cpu_ns() < until {
for _ in 0..1 << 14 {
x = black_box(x.wrapping_mul(6364136223846793005).wrapping_add(1));
}
}
black_box(x);
}
fn live_config() -> Config {
Config {
sampler: sampler::Config {
hz: 1000,
ring_samples: 512,
unwind: sampler::Unwind::StackCopy,
stack_copy_kib: 16,
wall: false,
},
ticks_over_ms: 50.0,
..Config::default()
}
}
/// Runs `body` on a thread opted in as `name`, profiled for exactly its duration.
fn profiled(name: &'static str, body: fn()) -> Profile {
let (ready, wait) = mpsc::channel();
let (go, start) = mpsc::channel::<()>();
let worker = thread::spawn(move || {
assert!(sampler::sampled_thread(name));
ready.send(()).unwrap();
start.recv().unwrap();
body();
});
wait.recv().unwrap();
let session = profile::start(Config {
threads: vec![name.into()],
..live_config()
})
.expect("start profiling");
go.send(()).unwrap();
worker.join().unwrap();
session.stop().expect("stop profiling")
}
fn entity_units() {
let (a, b) = (work::entity_id("test:a"), work::entity_id("test:b"));
for unit in 0..4 {
work::tick_start();
work::set_entity(if unit < 3 { a } else { b });
let started = sampler::thread_cpu_ns();
spin(40);
work::clear_work();
work::tick_end(sampler::thread_cpu_ns() - started);
}
}
#[test]
fn three_units_of_a_and_one_of_b_split_the_samples_three_to_one() {
let c = convert(&profiled("perf-capture-entities", entity_units));
let by = c.by_entity(false);
let total = (by["test:a"] + by["test:b"]) as f64;
let (a, b) = (by["test:a"] as f64 / total, by["test:b"] as f64 / total);
assert!((a - 0.75).abs() < 0.1 && (b - 0.25).abs() < 0.1, "{by:?}");
assert!(total > 100.0, "{total} samples");
assert_eq!(c.samples.threads[0].units, 4);
}
fn ticks_with_one_slow() {
for tick in 0..6 {
work::tick_start();
let started = sampler::thread_cpu_ns();
if tick == 3 {
spin_slow();
} else {
spin_fast();
}
work::tick_end(sampler::thread_cpu_ns() - started);
}
}
#[test]
fn only_the_slow_ticks_function_is_in_the_slow_weights() {
let c = convert(&profiled("perf-capture-slow", ticks_with_one_slow));
let w = c.samples.weights.as_ref().unwrap();
let (mut slow, mut fast) = (0, 0);
for i in 0..w.node.len() {
let names: Vec<_> = c.path(w.node[i]).into_iter().map(short).collect();
if w.slow[i] {
slow += w.count[i];
assert!(
names.contains(&"spin_slow") && !names.contains(&"spin_fast"),
"{names:?}"
);
} else {
fast += w.count[i];
assert!(!names.contains(&"spin_slow"), "{names:?}");
}
}
assert!(slow > 60 && fast > 20, "{slow} slow, {fast} other");
assert_eq!(
(c.samples.threads[0].units, c.samples.threads[0].slow_units),
(6, 1)
);
}
#[test]
fn overhead_counts_lost_samples_and_the_writers_share_of_a_core() {
let mut agg = Aggregate::new(1);
agg.lost = 4;
let mut profile = profile_of(agg, Config::default());
profile.stats.dropped = 3;
profile.stats.samples = 1000;
profile.stats.handler_ns = Some(2_000.0);
profile.writer_cpu_ns = 50_000_000;
let o = overhead(&profile, None);
assert_eq!(o.dropped_samples, 7);
assert!((o.sampler_share_of_thread - 0.0002).abs() < 1e-7);
assert!((o.writer_core_share - 0.005).abs() < 1e-6);
}
#[test]
fn crates_come_from_the_first_path_segment() {
for (name, krate) in [
(
"horizon_world::level::tick::random_tick_at",
"horizon_world",
),
(
"<horizon_world::X as core::ops::Drop>::drop",
"horizon_world",
),
("core::ptr::drop_in_place<alloc::vec::Vec<u8>>", "core"),
("clone3", ""),
("[horizon-server 0x4f2a]", "horizon-server"),
("[unmapped 0x7ffd]", ""),
] {
assert_eq!(crate_of(name), krate, "{name}");
}
}
fn ask(
file: &std::path::Path,
parts: &[&str],
) -> horizon_perf::wire::Envelope<horizon_perf::query::Row> {
let env = horizon_perf::cli::Env {
horizon_home: None,
poll: Duration::from_millis(2),
};
let mut args = vec!["query".to_owned()];
args.extend(parts.iter().map(|p| (*p).to_owned()));
args.push(file.to_string_lossy().into_owned());
let mut out = Vec::new();
if let Err(e) = horizon_perf::cli::run(&args, &env, &mut out) {
panic!("{e:?}");
}
serde_json::from_slice(&out)
.unwrap_or_else(|e| panic!("{e}: {}", String::from_utf8_lossy(&out)))
}
#[test]
fn a_converted_profile_written_to_a_file_reads_back_through_the_queries() {
let ctx = Context::default();
let mut agg = Aggregate::new(50_000_000);
for _ in 0..5 {
agg.add(
"tick",
&record(Kind::Cpu, ctx, 0, &[at(alpha, 4), at(gamma, 3)]),
);
}
for _ in 0..2 {
agg.add(
"tick",
&record(Kind::Cpu, ctx, 0, &[at(beta, 4), at(gamma, 3)]),
);
}
for _ in 0..2 {
agg.add("tick", &record(Kind::Tick, ctx, 9_000_000, &[]));
}
for _ in 0..3 {
agg.add("worker", &record(Kind::Cpu, ctx, 0, &[at(beta, 4)]));
}
agg.add("worker", &record(Kind::Tick, ctx, 9_000_000, &[]));
agg.finish();
let config = Config {
sampler: sampler::Config {
hz: 100,
..sampler::Config::default()
},
..Config::default()
};
let c = convert(&profile_of(agg, config));
let mut fields = vec![Field::Start(Start {
side: Side::Server as i32,
kind: CaptureKind::Dedicated as i32,
..Default::default()
})];
fields.extend(c.strings.into_iter().map(Field::String));
fields.push(Field::Samples(c.samples));
fields.push(Field::Stop(Stop {
side: Side::Server as i32,
t: 10.0,
..Default::default()
}));
let dir = std::env::temp_dir().join(format!("horizon-render-perf-{}", std::process::id()));
std::fs::create_dir_all(&dir).unwrap();
let file = dir.join("real.hzperf");
let mut writer = CaptureWriter::new(std::fs::File::create(&file).unwrap()).unwrap();
for field in fields {
writer.write(field).unwrap();
}
writer.finish().unwrap();
// 10 ms samples: alpha has 5 over the tick thread's 2 ticks.
let hot = ask(&file, &["hot"]);
let alpha = hot.rows().iter().find_map(|r| match r {
horizon_perf::query::Row::Hot(h) if h.full.ends_with("alpha") => Some(h),
_ => None,
});
assert_eq!(alpha.map(|h| (h.self_ms, h.samples)), Some((25.0, 5)));
let worker = ask(&file, &["split", "t:worker", "--by", "thread"]);
let sel = worker.selection.as_ref().unwrap();
assert_eq!(
(sel.thread.as_deref(), sel.ticks, sel.samples),
(Some("worker"), Some(1), Some(3))
);
let values: Vec<_> = worker
.rows()
.iter()
.map(|r| match r {
horizon_perf::query::Row::Split(s) => (s.key.clone(), s.value),
other => panic!("{other:?}"),
})
.collect();
assert_eq!(values, [(Some("t:worker".to_owned()), 30.0)]);
std::fs::remove_dir_all(&dir).unwrap();
}
#[test]
fn heap_samples_share_the_tables_without_the_allocators_own_frames() {
let mut cpu = Aggregate::new(50_000_000);
cpu.add(
"tick",
&record(
Kind::Cpu,
Context::default(),
0,
&[at(alpha, 4), at(gamma, 3)],
),
);
cpu.finish();
let mut heap = Aggregate::new(0);
let inside = HeapProfile::bytes as *const () as usize + 4;
let on_tick = heap.thread_index("tick");
let key = heap.stack_key(
on_tick,
profile::State::Alloc,
&Context::default(),
&[inside, at(beta, 4), at(gamma, 3)],
);
heap.add_weight(key, 3000);
heap.add_weight(
profile::Key {
state: profile::State::Live,
..key
},
1000,
);
let on_worker = heap.thread_index("worker");
let key = heap.stack_key(
on_worker,
profile::State::Live,
&Context::default(),
&[inside, at(alpha, 4)],
);
heap.add_weight(key, 500);
let heap = HeapProfile {
agg: heap,
interval: 4096,
samples: 3,
live_samples: 2,
sample_ns: 0,
freed: 2000,
elapsed: Duration::from_secs(10),
};
let mut interner = Interner::default();
let samples = sample_profile(
&profile_of(cpu, Config::default()),
Some(&heap),
Side::Server,
Unit::Tick,
&|_| None,
&mut interner,
);
let c = Converted {
samples,
strings: interner.take_new(Side::Server),
};
let sources: Vec<_> = c
.samples
.kinds
.iter()
.map(|k| (k.source(), k.period))
.collect();
assert_eq!(
sources,
[
(SampleSource::Cpu, 4.0),
(SampleSource::Alloc, 1.0),
(SampleSource::Live, 1.0)
]
);
let names: Vec<_> = c.samples.threads.iter().map(|t| c.text(t.name)).collect();
assert_eq!(names, ["tick", "worker"]);
let w = c.samples.weights.as_ref().unwrap();
let mut rows: Vec<_> = (0..w.node.len())
.map(|i| {
let path: Vec<_> = c.path(w.node[i]).into_iter().map(short).collect();
(w.kind[i], w.thread[i], path.join(">"), w.count[i])
})
.collect();
rows.sort();
assert_eq!(
rows,
[
(0, 0, "gamma>alpha".to_owned(), 1),
(1, 0, "gamma>beta".to_owned(), 3000),
(2, 0, "gamma>beta".to_owned(), 1000),
(2, 1, "alpha".to_owned(), 500),
]
);
let o = overhead(
&profile_of(Aggregate::new(0), Config::default()),
Some(&heap),
);
assert_eq!(o.heap_samples, 3);
}
@@ -0,0 +1,484 @@
//! The GPU pass timers' readings as the capture's `gpu` messages (docs/perf-platform.md §3.5); apart from `perf_capture`, which only Linux has.
use horizon_perf_format::Interner;
use horizon_perf_format::v1::{
GpuBackend, GpuCounter, GpuCounts, GpuDevice, GpuFragmentUnit, GpuFrame, GpuFrames, GpuPart,
GpuPartCounts, GpuPartsMode, GpuPass, GpuPassSplit, GpuPasses, GpuSource, Side,
};
use std::time::Instant;
use crate::gfx::pack_timing::{
Counters, FragmentUnit, GpuReadout, PartsMode, PassCounters, Readings, TimedFrame, TimedFrames,
};
/// Every time one timer's frames gave a pass or a part during a capture.
#[derive(Default)]
struct Times {
passes: Vec<(String, Vec<f32>)>,
parts: Vec<(String, Vec<(String, Vec<f32>)>)>,
}
fn times_of<'a>(list: &'a mut Vec<(String, Vec<f32>)>, name: &str) -> &'a mut Vec<f32> {
let i = match list.iter().position(|e| e.0 == name) {
Some(i) => i,
None => {
list.push((name.to_owned(), Vec::new()));
list.len() - 1
}
};
&mut list[i].1
}
impl Times {
fn add(&mut self, frame: &TimedFrame) {
for (pass, ms) in &frame.passes {
times_of(&mut self.passes, pass).push(*ms);
}
for (pass, parts) in &frame.parts {
let i = match self.parts.iter().position(|e| e.0 == *pass) {
Some(i) => i,
None => {
self.parts.push((pass.clone(), Vec::new()));
self.parts.len() - 1
}
};
for (part, ms) in parts {
times_of(&mut self.parts[i].1, part).push(*ms);
}
}
}
fn is_empty(&self) -> bool {
self.passes.is_empty() && self.parts.is_empty()
}
/// `readings` with its times the medians over the whole capture instead of its last frames.
fn medians_into(&self, readings: &Readings) -> Readings {
let median = |v: &[f32]| {
let mut v = v.to_vec();
v.sort_by(f32::total_cmp);
v[v.len() / 2]
};
Readings {
passes: self
.passes
.iter()
.map(|(name, v)| (name.clone(), median(v)))
.collect(),
parts: self
.parts
.iter()
.map(|(pass, parts)| {
let parts = parts.iter().map(|(n, v)| (n.clone(), median(v))).collect();
(pass.clone(), parts)
})
.collect(),
..readings.clone()
}
}
}
/// A capture's timed frames on their way into `GpuFrames` blocks, every time kept for the whole capture's medians.
#[derive(Default)]
pub struct FrameLog {
pending: Vec<GpuFrame>,
cpu_frame_ms: Vec<f32>,
own: Times,
pack: Times,
}
impl FrameLog {
/// The CPU's own ms of one frame the window drew.
pub fn cpu_frame(&mut self, ms: f32) {
self.cpu_frame_ms.push(ms);
}
/// The frames timed since the last call; `t_of` puts a frame's start on the capture clock.
pub fn add(
&mut self,
frames: &TimedFrames,
t_of: impl Fn(Instant) -> f64,
strings: &mut Interner,
) {
for (source, list, times) in [
(GpuSource::Horizon, &frames.own, &mut self.own),
(GpuSource::ShaderPack, &frames.pack, &mut self.pack),
] {
for frame in list {
times.add(frame);
self.pending
.push(frame_message(frame, source, t_of(frame.at), strings));
}
}
}
/// The frames and CPU frame times since the last block; none when there are neither.
pub fn take_block(&mut self, side: Side) -> Option<GpuFrames> {
if self.pending.is_empty() && self.cpu_frame_ms.is_empty() {
return None;
}
Some(GpuFrames {
side: side as i32,
frames: std::mem::take(&mut self.pending),
cpu_frame_ms: std::mem::take(&mut self.cpu_frame_ms),
})
}
}
fn frame_message(
frame: &TimedFrame,
source: GpuSource,
t: f64,
strings: &mut Interner,
) -> GpuFrame {
let mut passes: Vec<GpuPass> = frame
.passes
.iter()
.map(|(name, ms)| GpuPass {
name: strings.intern(name),
ms: Some(*ms),
parts: Vec::new(),
})
.collect();
for (pass, parts) in &frame.parts {
let parts = parts
.iter()
.map(|(part, ms)| GpuPart {
name: strings.intern(part),
ms: *ms,
})
.collect();
let name = strings.intern(pass);
match passes.iter_mut().find(|p| p.name == name) {
Some(p) => p.parts = parts,
None => passes.push(GpuPass {
name,
ms: None,
parts,
}),
}
}
GpuFrame {
source: source as i32,
t,
total_ms: frame.total_ms,
passes,
}
}
fn backend(backend: wgpu::Backend) -> GpuBackend {
match backend {
wgpu::Backend::Vulkan => GpuBackend::Vulkan,
wgpu::Backend::Metal => GpuBackend::Metal,
wgpu::Backend::Dx12 => GpuBackend::Dx12,
wgpu::Backend::Gl => GpuBackend::Gl,
_ => GpuBackend::Other,
}
}
fn counts(c: Counters) -> GpuCounts {
GpuCounts {
vertex_shader_invocations: c.vertex_shader_invocations,
clipper_invocations: c.clipper_invocations,
clipper_primitives_out: c.clipper_primitives_out,
fragment_shader_invocations: c.fragment_shader_invocations,
}
}
fn counter(strings: &mut Interner, pass: &str, c: &PassCounters) -> GpuCounter {
GpuCounter {
pass: strings.intern(pass),
total: Some(counts(c.total)),
pixels: c.pixels,
parts: c
.parts
.iter()
.map(|(name, part)| GpuPartCounts {
name: strings.intern(name),
counts: Some(counts(*part)),
})
.collect(),
}
}
fn device(adapter: &wgpu::AdapterInfo, readings: &Readings) -> GpuDevice {
let driver = format!("{} {}", adapter.driver, adapter.driver_info);
GpuDevice {
adapter: adapter.name.clone(),
backend: backend(adapter.backend) as i32,
driver: driver.trim().to_owned(),
parts_mode: match readings.support.parts {
PartsMode::InsidePasses => GpuPartsMode::InsidePasses,
PartsMode::SeparatePasses => GpuPartsMode::SeparatePasses,
} as i32,
pass_split: match readings.support.parts {
PartsMode::InsidePasses => GpuPassSplit::DrawGroups,
PartsMode::SeparatePasses => GpuPassSplit::PassesOnly,
} as i32,
counters: readings.support.counters,
fragment_unit: match readings.support.fragment_unit {
None => GpuFragmentUnit::Unspecified,
Some(FragmentUnit::Pixel) => GpuFragmentUnit::Pixel,
Some(FragmentUnit::Quad) => GpuFragmentUnit::Quad,
} as i32,
}
}
fn section(
adapter: &wgpu::AdapterInfo,
side: Side,
source: GpuSource,
readings: &Readings,
strings: &mut Interner,
) -> GpuPasses {
let mut passes: Vec<GpuPass> = readings
.passes
.iter()
.map(|(name, ms)| GpuPass {
name: strings.intern(name),
ms: Some(*ms),
parts: Vec::new(),
})
.collect();
for (pass, parts) in &readings.parts {
let name = strings.intern(pass);
let at = passes
.iter()
.position(|p| p.name == name)
.unwrap_or_else(|| {
passes.push(GpuPass {
name,
ms: None,
parts: Vec::new(),
});
passes.len() - 1
});
passes[at].parts = parts
.iter()
.map(|(part, ms)| GpuPart {
name: strings.intern(part),
ms: *ms,
})
.collect();
}
GpuPasses {
side: side as i32,
device: Some(device(adapter, readings)),
passes,
counters: readings
.counters
.iter()
.map(|(pass, c)| counter(strings, pass, c))
.collect(),
source: source as i32,
}
}
/// One `GpuPasses` per timer that has timed something, its times the medians of `log` where it has them; the caller appends `strings`' new entries first.
pub fn gpu_passes(
readout: &GpuReadout,
log: &FrameLog,
side: Side,
strings: &mut Interner,
) -> Vec<GpuPasses> {
[
(GpuSource::Horizon, &readout.own, &log.own),
(GpuSource::ShaderPack, &readout.pack, &log.pack),
]
.into_iter()
.filter_map(|(source, readings, times)| {
let readings = readings.as_ref().filter(|r| !r.is_empty())?;
let whole = (!times.is_empty()).then(|| times.medians_into(readings));
let readings = whole.as_ref().unwrap_or(readings);
Some(section(&readout.adapter, side, source, readings, strings))
})
.collect()
}
#[cfg(test)]
mod tests {
use super::*;
use crate::gfx::pack_timing::DetailSupport;
fn adapter() -> wgpu::AdapterInfo {
wgpu::AdapterInfo {
name: "Test GPU".into(),
vendor: 0x10de,
device: 1,
device_type: wgpu::DeviceType::DiscreteGpu,
device_pci_bus_id: String::new(),
driver: "NVIDIA".into(),
driver_info: "570.1".into(),
backend: wgpu::Backend::Vulkan,
subgroup_min_size: 0,
subgroup_max_size: 0,
transient_saves_memory: None,
limit_bucket: None,
}
}
fn support(unit: Option<FragmentUnit>) -> DetailSupport {
DetailSupport {
parts: PartsMode::InsidePasses,
encoder_stamps: true,
counters: true,
fragment_unit: unit,
}
}
#[test]
fn known_medians_parts_and_counters_come_through_with_their_names() {
let c = |v, f| Counters {
vertex_shader_invocations: v,
clipper_invocations: 2_050_000,
clipper_primitives_out: 1_200_000,
fragment_shader_invocations: f,
};
let own = Readings {
passes: vec![("World".into(), 1.5), ("HUD".into(), 0.25)],
parts: vec![("GPU culling".into(), vec![("cull".into(), 0.125)])],
counters: vec![(
"World".into(),
PassCounters {
total: c(4_100_000, 6_300_000),
pixels: 2_073_600,
parts: vec![
("before translucent".into(), c(3_000_000, 5_100_000)),
("translucent terrain".into(), c(1_100_000, 1_200_000)),
],
},
)],
support: support(Some(FragmentUnit::Quad)),
};
let readout = GpuReadout {
adapter: adapter(),
own: Some(own),
pack: None,
};
let mut strings = Interner::default();
let got = gpu_passes(&readout, &FrameLog::default(), Side::Client, &mut strings);
assert_eq!(got.len(), 1);
let g = &got[0];
assert_eq!((g.side(), g.source()), (Side::Client, GpuSource::Horizon));
let d = g.device.as_ref().unwrap();
assert_eq!(d.adapter, "Test GPU");
assert_eq!(d.backend(), GpuBackend::Vulkan);
assert_eq!(d.driver, "NVIDIA 570.1");
assert_eq!(d.parts_mode(), GpuPartsMode::InsidePasses);
assert!(d.counters);
assert_eq!(d.fragment_unit(), GpuFragmentUnit::Quad);
// World, HUD, then the pass known only through its parts.
let entries = strings.take_new(Side::Client);
let name = |id: u32| entries.iter().find(|e| e.id == id).unwrap().text.as_str();
let named: Vec<_> = g.passes.iter().map(|p| name(p.name)).collect();
assert_eq!(named, ["World", "HUD", "GPU culling"]);
let shown: Vec<_> = g.passes.iter().map(|p| p.ms).collect();
assert_eq!(shown, [Some(1.5), Some(0.25), None]);
assert_eq!(name(g.passes[2].parts[0].name), "cull");
assert_eq!(g.passes[2].parts[0].ms, 0.125);
let counted = &g.counters[0];
assert_eq!(counted.pixels, 2_073_600);
assert_eq!(
counted.total.as_ref().unwrap().fragment_shader_invocations,
6_300_000
);
assert_eq!(counted.parts.len(), 2);
assert_eq!(
counted.parts[1]
.counts
.as_ref()
.unwrap()
.vertex_shader_invocations,
1_100_000
);
// The pass's counters and its time name the same string.
assert_eq!(counted.pass, g.passes[0].name);
}
#[test]
fn a_timer_that_timed_nothing_adds_no_section() {
let idle = Readings {
passes: Vec::new(),
parts: Vec::new(),
counters: Vec::new(),
support: support(None),
};
let pack = Readings {
passes: vec![("composite".into(), 0.75)],
..idle.clone()
};
let readout = GpuReadout {
adapter: adapter(),
own: Some(idle),
pack: Some(pack),
};
let got = gpu_passes(
&readout,
&FrameLog::default(),
Side::Client,
&mut Interner::default(),
);
assert_eq!(got.len(), 1);
assert_eq!(got[0].source(), GpuSource::ShaderPack);
assert_eq!(
got[0].device.as_ref().unwrap().fragment_unit(),
GpuFragmentUnit::Unspecified
);
}
#[test]
fn the_end_section_takes_the_whole_capture_median_and_each_block_its_frames() {
let start = Instant::now();
let frame = |i: u32, world: f32| TimedFrame {
at: start + std::time::Duration::from_millis(u64::from(i) * 10),
total_ms: Some(world + 1.0),
passes: vec![("World".into(), world), ("HUD".into(), 0.5)],
parts: vec![("World".into(), vec![("sky".into(), world / 4.0)])],
};
let mut log = FrameLog::default();
let mut strings = Interner::default();
let worlds = [1.0, 2.0, 3.0, 4.0, 100.0];
let frames = TimedFrames {
own: worlds
.iter()
.enumerate()
.map(|(i, &w)| frame(i as u32, w))
.collect(),
pack: Vec::new(),
};
log.add(
&frames,
|at| at.duration_since(start).as_secs_f64(),
&mut strings,
);
log.cpu_frame(6.0);
let block = log.take_block(Side::Client).expect("a block");
assert_eq!(block.frames.len(), 5);
assert_eq!(block.cpu_frame_ms, [6.0]);
assert!((block.frames[4].t - 0.04).abs() < 1e-9);
assert_eq!(block.frames[4].total_ms, Some(101.0));
assert_eq!(block.frames[4].passes[0].parts[0].ms, 25.0);
assert!(log.take_block(Side::Client).is_none());
// The timer's own median (9.0) is of its last frames; the capture's is the third of five sorted.
let own = Readings {
passes: vec![("World".into(), 9.0)],
parts: Vec::new(),
counters: Vec::new(),
support: support(None),
};
let readout = GpuReadout {
adapter: adapter(),
own: Some(own),
pack: None,
};
let got = gpu_passes(&readout, &log, Side::Client, &mut strings);
let world = &got[0].passes[0];
assert_eq!(world.ms, Some(3.0));
assert_eq!(world.parts[0].ms, 0.75);
assert_eq!(
got[0].device.as_ref().unwrap().pass_split(),
GpuPassSplit::DrawGroups
);
}
}
+304 -171
View File
@@ -1,10 +1,4 @@
//! Named timings within a frame, for finding what makes a frame slow:
//! `HORIZON_SPIKES=MS` prints the spans of every frame whose CPU time
//! exceeds MS milliseconds, and of every server tick (dedicated or
//! integrated) longer than MS. Each span keeps its wall time and the
//! thread's CPU time inside it, so a span that waited (for a lock, the GPU
//! or a core on a busy machine) shows apart from one that worked. Costs two
//! clock reads per span when enabled and a relaxed load otherwise.
//! Named timings (wall and thread CPU) of a frame or tick, on with `HORIZON_SPIKES` or while a capture records; the slow ones go to the slow ring, not as span lists to the log.
use std::{
cell::{Cell, RefCell},
sync::atomic::{AtomicBool, Ordering},
@@ -58,10 +52,10 @@ pub fn record_here(on: bool) {
/// Whether this thread records spans: all of them on, or [`record_here`].
#[inline]
pub fn recording_here() -> bool {
ENABLED.load(Ordering::Relaxed) || HERE.with(Cell::get) || trace::on()
ENABLED.load(Ordering::Relaxed) || HERE.with(Cell::get) || timeline::on()
}
/// The thread's CPU time when spans are on for every thread (`HORIZON_SPIKES`): only they
/// print it.
/// The thread's CPU time when spans are on for every thread (`HORIZON_SPIKES`): spans recorded
/// by `record_here` alone keep wall time only.
#[inline]
fn cpu_if_enabled() -> Option<Duration> {
if ENABLED.load(Ordering::Relaxed) {
@@ -82,31 +76,41 @@ pub fn time<T>(name: &'static str, f: impl FnOnce() -> T) -> T {
pub struct Guard {
open: Option<(&'static str, Instant, Option<Duration>, u8)>,
_heap: Option<horizon_threads::heap_census::Tag>,
/// The sampler's span before this one.
outer: u16,
}
/// Start a span named `name` that ends when the returned [`Guard`] is dropped.
#[inline]
pub fn guard(name: &'static str) -> Guard {
let _heap = horizon_threads::heap_tag(name);
let outer = horizon_threads::work::enter_span(name);
if !recording_here() {
return Guard { open: None, _heap };
return Guard {
open: None,
_heap,
outer,
};
}
let depth = DEPTH.with(|d| d.replace(d.get().saturating_add(1)));
Guard {
open: Some((name, Instant::now(), cpu_if_enabled(), depth)),
_heap,
outer,
}
}
impl Drop for Guard {
fn drop(&mut self) {
horizon_threads::work::leave_span(self.outer);
let Some((name, started, cpu, depth)) = self.open.take() else {
return;
};
let ms = started.elapsed().as_secs_f32() * 1000.0;
let elapsed = started.elapsed();
let ms = elapsed.as_secs_f32() * 1000.0;
let cpu_ms = cpu
.and_then(|start| Some((thread_cpu()?, start)))
.map_or(ms, |(end, start)| (end - start).as_secs_f32() * 1000.0);
DEPTH.with(|d| d.set(depth));
trace::span(name, started, ms);
timeline::span(name, started, elapsed, depth);
push(Span {
name,
ms,
@@ -115,6 +119,22 @@ impl Drop for Guard {
});
}
}
/// How many spans this thread holds.
fn recorded() -> usize {
SPANS.with(|s| s.borrow().len())
}
/// Moves the spans recorded since `mark` one level in: they ran inside the lap now ending.
fn nest_since(mark: usize, depth: u8) {
SPANS.with(|s| {
let mut s = s.borrow_mut();
let start = mark.min(s.len());
for span in &mut s[start..] {
if span.depth >= depth {
span.depth = span.depth.saturating_add(1);
}
}
});
}
/// Keeps a finished span for the thread's next [`take_raw`].
fn push(span: Span) {
SPANS.with(|s| {
@@ -125,22 +145,6 @@ fn push(span: Span) {
s.push(span);
});
}
/// Attach per-tick counts without interpreting them as durations.
pub fn counters(counts: &[(&str, u64)]) {
if !enabled() {
return;
}
NOTES.with(|n| {
n.borrow_mut().push((
0.0,
counts
.iter()
.map(|(name, count)| format!("{name}={count}"))
.collect::<Vec<_>>()
.join(", "),
))
});
}
/// Notes that one step inside a span took `ms` milliseconds doing `what` (one player's
/// packets, say), when this thread records spans and the step took at least [`NOTE_MS`]: a
/// span says where a slow tick went, its notes whose doing it was. Only the slowest few are
@@ -173,34 +177,36 @@ pub fn take_notes() -> Vec<(f32, String)> {
}
/// Consecutive steps of one function timed as spans: each [`Laps::lap`] records the time since
/// the previous one (or since [`laps`]) under its name. Nothing is read while spans are off.
pub struct Laps(Option<(Instant, Option<Duration>)>);
pub struct Laps(Option<(Instant, Option<Duration>, usize)>);
/// Start timing the steps of a function (see [`Laps`]).
#[inline]
pub fn laps() -> Laps {
Laps(recording_here().then(|| (Instant::now(), cpu_if_enabled())))
Laps(recording_here().then(|| (Instant::now(), cpu_if_enabled(), recorded())))
}
impl Laps {
/// Record the step that just ended under `name`, and start the next.
#[inline]
pub fn lap(&mut self, name: &'static str) {
let Some((started, cpu)) = &mut self.0 else {
let Some((started, cpu, mark)) = &mut self.0 else {
return;
};
let now = Instant::now();
let now_cpu = cpu.and_then(|_| thread_cpu());
let ms = (now - *started).as_secs_f32() * 1000.0;
let elapsed = now - *started;
let ms = elapsed.as_secs_f32() * 1000.0;
let cpu_ms = now_cpu
.zip(*cpu)
.map_or(ms, |(end, start)| (end - start).as_secs_f32() * 1000.0);
let depth = DEPTH.with(Cell::get);
trace::span(name, *started, ms);
timeline::span(name, *started, elapsed, depth);
nest_since(*mark, depth);
push(Span {
name,
ms,
cpu_ms,
depth,
});
(*started, *cpu) = (now, now_cpu);
(*started, *cpu, *mark) = (now, now_cpu, recorded());
}
}
/// The CPU time this thread has used, where the platform tells it: a tick
@@ -283,173 +289,176 @@ pub fn merge(spans: &[Span]) -> Vec<Span> {
sums.sort_by(|a, b| b.ms.total_cmp(&a.ms));
sums
}
/// `name wall` for spans of at least 0.5 ms, with `/cpu` when the thread
/// worked for less than nine tenths of it: the slow-frame and slow-tick lines.
pub fn describe(spans: &[Span]) -> String {
let list: Vec<String> = spans
.iter()
.filter(|s| s.ms >= 0.5)
.map(|s| {
if s.cpu_ms < s.ms * 0.9 {
format!("{} {:.1}/{:.1}", s.name, s.ms, s.cpu_ms)
} else {
format!("{} {:.1}", s.name, s.ms)
}
})
.collect();
list.join(", ")
}
/// The run's timeline: every span ([`time`], [`guard`], [`Laps::lap`]) on every thread, as
/// Chrome trace events that Perfetto (ui.perfetto.dev) opens as they are. `HORIZON_TRACE=FILE`
/// writes it from the start; [`start`](trace::start) and [`stop`](trace::stop) write one for a
/// while (`horizon-trace` on a dedicated server's console, `trace` on the control socket). Spans
/// show on their threads where they ran, nested by time, so what ran beside what, and what a
/// thread waited for, can be read off. Off, a span costs one relaxed load more.
pub mod trace {
/// The run's timeline: every span on every thread, kept in per-thread buffers until a capture drains them as `timeline` chunks.
pub mod timeline {
use std::{
cell::Cell,
fs::File,
io::{BufWriter, Write as _},
path::Path,
cell::RefCell,
sync::{
Mutex, PoisonError,
Arc, Mutex, PoisonError,
atomic::{AtomicBool, AtomicU32, Ordering},
},
time::Instant,
time::{Duration, Instant},
};
/// The most spans a thread holds between two drains; a thread that records more drops them.
const THREAD_EVENTS_MAX: usize = 1 << 18;
static ON: AtomicBool = AtomicBool::new(false);
static SINK: Mutex<Option<Sink>> = Mutex::new(None);
/// Counts the traces started: a thread names itself again in each one.
static TRACES: AtomicU32 = AtomicU32::new(0);
static NEXT_THREAD: AtomicU32 = AtomicU32::new(1);
thread_local! {
/// This thread's id in traces (0: none yet).
static THREAD: Cell<u32> = const { Cell::new(0) };
/// The trace this thread last named itself in.
static NAMED_IN: Cell<u32> = const { Cell::new(0) };
}
/// How often a trace's buffer goes to the file, at most: a run that ends without
/// [`stop`] loses no more than this.
const FLUSH_EVERY: std::time::Duration = std::time::Duration::from_secs(1);
static GENERATION: AtomicU32 = AtomicU32::new(0);
static RUN: Mutex<Option<Run>> = Mutex::new(None);
struct Sink {
file: BufWriter<File>,
struct Run {
origin: Instant,
/// Until when it records ([`start`]'s `seconds`).
until: Option<Instant>,
flushed: Instant,
/// Events written: the next one is preceded by a comma.
events: u64,
}
impl Sink {
fn event(&mut self, event: std::fmt::Arguments) {
let comma = if self.events == 0 { "" } else { ",\n" };
let _ = write!(self.file, "{comma}{{\"pid\":1,{event}}}");
self.events += 1;
}
threads: Vec<Arc<Mutex<Thread>>>,
}
/// Whether a trace is being written.
struct Thread {
name: String,
label: String,
events: Vec<Event>,
}
/// This thread's buffer in the timeline it last joined.
struct Mine {
generation: u32,
origin: Instant,
thread: Arc<Mutex<Thread>>,
}
thread_local! {
static MINE: RefCell<Option<Mine>> = const { RefCell::new(None) };
}
/// One finished span.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub struct Event {
pub name: &'static str,
/// Microseconds from the timeline's start.
pub start_us: u64,
pub dur_us: u32,
/// How many spans it ran inside on its thread.
pub depth: u8,
}
/// The spans one thread finished since the last drain, in the order they finished.
#[derive(Debug, PartialEq, Eq)]
pub struct ThreadEvents {
/// The thread's name; threads alive together under one name get `#2`, `#3`... after it.
pub thread: String,
pub events: Vec<Event>,
}
/// Whether a timeline is being recorded.
#[inline]
pub fn on() -> bool {
ON.load(Ordering::Relaxed)
}
/// Starts a trace written to `path` (replacing a trace being written), for `seconds` or
/// until [`stop`].
pub fn start(path: &Path, seconds: Option<u32>) -> std::io::Result<()> {
let mut file = BufWriter::with_capacity(1 << 18, File::create(path)?);
file.write_all(b"[\n")?;
let now = Instant::now();
let sink = Sink {
file,
origin: now,
until: seconds.map(|s| now + std::time::Duration::from_secs(u64::from(s))),
flushed: now,
events: 0,
};
let mut slot = SINK.lock().unwrap_or_else(PoisonError::into_inner);
if let Some(old) = slot.take() {
close(old);
/// Starts recording, timed from `origin`; false when a timeline already runs (the process has one).
pub fn start(origin: Instant) -> bool {
let mut run = RUN.lock().unwrap_or_else(PoisonError::into_inner);
if run.is_some() {
return false;
}
*slot = Some(sink);
TRACES.fetch_add(1, Ordering::Relaxed);
*run = Some(Run {
origin,
threads: Vec::new(),
});
GENERATION.fetch_add(1, Ordering::Relaxed);
ON.store(true, Ordering::Relaxed);
Ok(())
true
}
/// Starts the trace `HORIZON_TRACE` names, if it names one.
pub fn start_from_env() {
if let Some(path) = std::env::var_os("HORIZON_TRACE")
&& let Err(e) = start(Path::new(&path), None)
{
log::warn!("Horizon: HORIZON_TRACE: {e}");
}
/// Takes what the threads finished since the last call.
pub fn drain() -> Vec<ThreadEvents> {
let run = RUN.lock().unwrap_or_else(PoisonError::into_inner);
run.as_ref().map_or_else(Vec::new, take_all)
}
/// Ends the trace being written: the file is complete JSON. The events written so far
/// and whether one was being written.
pub fn stop() -> Option<u64> {
/// Stops recording and returns what the threads finished since the last drain.
pub fn finish() -> Vec<ThreadEvents> {
ON.store(false, Ordering::Relaxed);
let sink = SINK.lock().unwrap_or_else(PoisonError::into_inner).take()?;
let events = sink.events;
close(sink);
Some(events)
let run = RUN.lock().unwrap_or_else(PoisonError::into_inner).take();
run.as_ref().map_or_else(Vec::new, take_all)
}
/// Whether the trace being written has run its time ([`start`]'s `seconds`): the caller
/// then [`stop`]s it.
pub fn due() -> bool {
on() && SINK
.lock()
.unwrap_or_else(PoisonError::into_inner)
.as_ref()
.and_then(|s| s.until)
.is_some_and(|until| Instant::now() >= until)
fn take_all(run: &Run) -> Vec<ThreadEvents> {
run.threads
.iter()
.filter_map(|thread| {
let mut thread = thread.lock().unwrap_or_else(PoisonError::into_inner);
(!thread.events.is_empty()).then(|| ThreadEvents {
thread: thread.label.clone(),
events: std::mem::take(&mut thread.events),
})
})
.collect()
}
fn close(mut sink: Sink) {
let _ = sink.file.write_all(b"\n]\n");
let _ = sink.file.flush();
/// Takes a buffer for this thread: the one of an exited thread of the same name, else a new one.
fn join() -> Option<Mine> {
let mut slot = RUN.lock().unwrap_or_else(PoisonError::into_inner);
let run = slot.as_mut()?;
let name = std::thread::current()
.name()
.unwrap_or("unnamed")
.to_owned();
let of_name =
|t: &Arc<Mutex<Thread>>| t.lock().unwrap_or_else(PoisonError::into_inner).name == name;
let thread = match run
.threads
.iter()
.find(|t| Arc::strong_count(t) == 1 && of_name(t))
{
Some(exited) => Arc::clone(exited),
None => {
let same = run.threads.iter().filter(|t| of_name(t)).count();
let label = if same == 0 {
name.clone()
} else {
format!("{name} #{}", same + 1)
};
let thread = Arc::new(Mutex::new(Thread {
name,
label,
events: Vec::new(),
}));
run.threads.push(Arc::clone(&thread));
thread
}
};
Some(Mine {
generation: GENERATION.load(Ordering::Relaxed),
origin: run.origin,
thread,
})
}
/// A span of this thread from `started`, `ms` long.
pub(super) fn span(name: &'static str, started: Instant, ms: f32) {
/// A span of this thread from `started`, `elapsed` long, inside `depth` others.
pub(super) fn span(name: &'static str, started: Instant, elapsed: Duration, depth: u8) {
if !on() {
return;
}
let tid = THREAD.with(|t| {
if t.get() == 0 {
t.set(NEXT_THREAD.fetch_add(1, Ordering::Relaxed));
let generation = GENERATION.load(Ordering::Relaxed);
let _ = MINE.try_with(|mine| {
let mut mine = mine.borrow_mut();
if mine.as_ref().is_none_or(|m| m.generation != generation) {
*mine = join();
}
let Some(mine) = mine.as_ref() else {
return;
};
let event = Event {
name,
start_us: started.saturating_duration_since(mine.origin).as_micros() as u64,
dur_us: elapsed.as_micros().min(u128::from(u32::MAX)) as u32,
depth,
};
let mut thread = mine.thread.lock().unwrap_or_else(PoisonError::into_inner);
if thread.events.len() < THREAD_EVENTS_MAX {
thread.events.push(event);
}
t.get()
});
let trace = TRACES.load(Ordering::Relaxed);
let mut slot = SINK.lock().unwrap_or_else(PoisonError::into_inner);
let Some(sink) = slot.as_mut() else {
return;
};
if NAMED_IN.with(|n| n.replace(trace)) != trace {
let name = std::thread::current()
.name()
.unwrap_or("unnamed")
.replace(['"', '\\'], "_");
sink.event(format_args!(
r#""ph":"M","name":"thread_name","tid":{tid},"args":{{"name":"{name}"}}"#
));
}
let ts = started.saturating_duration_since(sink.origin).as_secs_f64() * 1e6;
sink.event(format_args!(
r#""ph":"X","name":"{name}","tid":{tid},"ts":{ts:.3},"dur":{:.3}"#,
f64::from(ms) * 1e3
));
let now = Instant::now();
if now.duration_since(sink.flushed) >= FLUSH_EVERY {
let _ = sink.file.flush();
sink.flushed = now;
}
}
}
@@ -654,6 +663,46 @@ mod tests {
SWITCH.lock().unwrap_or_else(|e| e.into_inner())
}
/// Samples taken inside a named span carry its id, with spans off, and the span is left after it.
#[cfg(target_os = "linux")]
#[test]
fn samples_inside_a_span_carry_its_id() {
use horizon_threads::{sampler, work};
use std::time::{Duration, Instant};
let worker = std::thread::spawn(|| {
assert!(sampler::sampled_thread("spans-id-test"));
let config = sampler::Config {
hz: 1000,
..Default::default()
};
let session = sampler::start(config, |n| n == "spans-id-test").unwrap();
let busy = |ms| {
let until = Instant::now() + Duration::from_millis(ms);
while Instant::now() < until {
std::hint::black_box(0);
}
};
super::time("spans_id_test_named", || busy(60));
assert_eq!(sampler::span(), 0);
busy(60);
let mut spans = Vec::new();
session.stop(|_, r| spans.push(r.ctx.span));
spans
});
let spans = worker.join().unwrap();
let inside = |s: &u16| work::name(work::Space::Span, *s) == Some("spans_id_test_named");
let (named, plain) = (
spans.iter().filter(|s| inside(s)).count(),
spans.iter().filter(|s| **s == 0).count(),
);
assert!(
named > 30 && plain > 30,
"{named} named, {plain} outside of {} samples",
spans.len()
);
assert_eq!(named + plain, spans.len(), "no other span id");
}
/// A slow tick's notes are its slowest steps, slowest first; short steps and steps on a
/// thread that records no spans leave none.
#[test]
@@ -680,8 +729,7 @@ mod tests {
assert!(super::take_notes().is_empty(), "taking them clears them");
}
/// Spans of one name add up; a span the thread spent mostly waiting
/// shows its CPU time after the wall time, short ones are left out.
/// Spans of one name add up, the slowest first; a span that waited keeps its small CPU time.
#[test]
fn spans_sum_by_name_and_show_waits() {
super::SPANS.with(|s| {
@@ -723,7 +771,6 @@ mod tests {
}
);
assert_eq!((spans[1].name, spans[1].ms), ("tick", 5.0));
assert_eq!(super::describe(&spans), "submit 30.0/0.2, tick 5.0");
assert!(super::take().is_empty());
}
@@ -843,6 +890,22 @@ mod tests {
assert!(spans[0].ms >= 2.0);
}
/// A span that ran during a lap belongs inside that lap, and not inside the next one.
#[test]
fn a_span_during_a_lap_nests_under_that_lap() {
let _switch = spans_switch();
let _ = super::take_raw();
super::enable(true);
let mut laps = super::laps();
super::time("inner", || {});
laps.lap("first");
laps.lap("second");
super::enable(false);
let spans = super::take_raw();
let depths: Vec<_> = spans.iter().map(|s| (s.name, s.depth)).collect();
assert_eq!(depths, [("inner", 1), ("first", 0), ("second", 0)]);
}
/// The thread's CPU clock moves with work done on it, not with time
/// spent asleep.
#[cfg(any(unix, all(windows, target_arch = "x86_64")))]
@@ -865,4 +928,74 @@ mod tests {
assert!(slept.as_millis() < 25, "{slept:?} while asleep");
assert!(worked.as_millis() >= 10, "{worked:?} while busy");
}
/// Each thread's finished spans reach the timeline with their depth and nesting in time; threads alive together under one name get labels of their own, ones after another share a lane.
#[test]
fn the_timeline_keeps_each_threads_spans_with_their_depth_and_place_in_time() {
use super::timeline;
use std::sync::{Arc, Barrier};
use std::time::{Duration, Instant};
let _switch = spans_switch();
assert!(timeline::start(Instant::now()));
assert!(
!timeline::start(Instant::now()),
"the process has one timeline"
);
let spawn = |name: &str, alive_with: Option<Arc<Barrier>>| {
std::thread::Builder::new()
.name(name.into())
.spawn(move || {
super::time("outer", || {
std::thread::sleep(Duration::from_millis(4));
super::time("inner", || std::thread::sleep(Duration::from_millis(3)));
});
if let Some(barrier) = alive_with {
barrier.wait();
}
})
.unwrap()
};
spawn("tl-solo", None).join().unwrap();
let together = Arc::new(Barrier::new(2));
let twins = [
spawn("tl-twin", Some(together.clone())),
spawn("tl-twin", Some(together)),
];
for twin in twins {
twin.join().unwrap();
}
spawn("tl-seq", None).join().unwrap();
spawn("tl-seq", None).join().unwrap();
let mut threads = timeline::finish();
threads.retain(|t| t.thread.starts_with("tl-"));
threads.sort_by(|a, b| a.thread.cmp(&b.thread));
let labels: Vec<_> = threads.iter().map(|t| t.thread.as_str()).collect();
assert_eq!(labels, ["tl-seq", "tl-solo", "tl-twin", "tl-twin #2"]);
for thread in &threads {
let runs = if thread.thread == "tl-seq" { 2 } else { 1 };
assert_eq!(thread.events.len(), 2 * runs, "{thread:?}");
for pair in thread.events.chunks(2) {
let [inner, outer] = pair else { unreachable!() };
assert_eq!((inner.name, inner.depth), ("inner", 1));
assert_eq!((outer.name, outer.depth), ("outer", 0));
assert!(
inner.dur_us >= 3_000 && outer.dur_us >= 7_000,
"{inner:?} {outer:?}"
);
assert!(inner.start_us >= outer.start_us + 4_000);
assert!(
inner.start_us + u64::from(inner.dur_us)
<= outer.start_us + u64::from(outer.dur_us)
);
}
}
let seq = threads.iter().find(|t| t.thread == "tl-seq").unwrap();
assert!(seq.events[2].start_us >= seq.events[1].start_us + u64::from(seq.events[1].dur_us));
spawn("tl-after", None).join().unwrap();
assert!(
timeline::drain().is_empty(),
"a stopped timeline records nothing"
);
}
}
+13 -31
View File
@@ -28,37 +28,7 @@ pub const IDLE_RELEASE: std::time::Duration = std::time::Duration::from_secs(5);
/// Worker CPU time and job counts by kind, for benchmarks
/// (`examples/gen_light_bench.rs`). Off unless enabled: a relaxed load per
/// job then.
pub mod job_stats {
use std::sync::atomic::{AtomicBool, AtomicU64, Ordering};
pub const KINDS: [&str; 7] = [
"load", "generate", "decorate", "finish", "light", "mesh", "other",
];
static ENABLED: AtomicBool = AtomicBool::new(false);
static NANOS: [AtomicU64; 7] = [const { AtomicU64::new(0) }; 7];
static COUNT: [AtomicU64; 7] = [const { AtomicU64::new(0) }; 7];
pub fn enable(on: bool) {
ENABLED.store(on, Ordering::Relaxed);
}
pub(super) fn enabled() -> bool {
ENABLED.load(Ordering::Relaxed)
}
pub(super) fn add(kind: usize, nanos: u64) {
NANOS[kind].fetch_add(nanos, Ordering::Relaxed);
COUNT[kind].fetch_add(1, Ordering::Relaxed);
}
/// (kind, nanoseconds, count) since the last call; resets them.
pub fn take() -> Vec<(&'static str, u64, u64)> {
(0..KINDS.len())
.map(|i| {
(
KINDS[i],
NANOS[i].swap(0, Ordering::Relaxed),
COUNT[i].swap(0, Ordering::Relaxed),
)
})
.collect()
}
}
pub mod job_stats;
/// A chunk under generation, opaque to the streamer.
pub type Proto = Box<dyn std::any::Any + Send>;
/// Chunk generation in vanilla's stages: a neighbour-independent proto chunk
@@ -1070,6 +1040,8 @@ pub struct WorkerContext {
pub light: Arc<LightTables>,
}
mod bounded;
mod queue_census;
pub use queue_census::QueueCensus;
mod forced_cover;
mod forced_ring;
mod relight_now;
@@ -1683,6 +1655,7 @@ impl Streaming {
&mut self,
world: &mut World,
) -> (Vec<((i32, i32, i32), SectionMesh)>, Vec<Arc<Chunk>>) {
let mut hz_laps = crate::spans::laps();
let mut uploads = Vec::new();
if self.paused {
return (uploads, Vec::new());
@@ -1708,6 +1681,7 @@ impl Streaming {
self.scan_age = if sweep { 0 } else { self.scan_age + 1 };
self.requeue();
self.relight_edits(world);
hz_laps.lap("st_relight_edits");
let deferred = std::mem::take(&mut self.deferred);
let received = self.shared.done.take();
// Meshes made stale meanwhile, freed with the uploaded ones (`free`).
@@ -1757,6 +1731,7 @@ impl Streaming {
}
}
}
hz_laps.lap("st_results");
self.free(stale);
let mut removed = Vec::new();
if sweep {
@@ -1827,6 +1802,7 @@ impl Streaming {
}
self.save_checked_out(written);
}
hz_laps.lap("st_sweep");
let (protos, finishing) = (&self.protos, &self.finishing);
self.decorated.retain(|k| {
protos.contains_key(k) || finishing.contains(k) || world.chunks.contains_key(k)
@@ -1858,6 +1834,7 @@ impl Streaming {
self.submit(Some((x, z)), 0, Job::Finish(proto));
}
}
hz_laps.lap("st_finalize");
// Decorate chunks whose 3x3 proto neighbourhood is available, in
// vanilla's order: decoration is order dependent where write regions
// overlap (see `DecorationOrder`).
@@ -1983,7 +1960,9 @@ impl Streaming {
self.submit(Some((x, z)), 1, Job::Decorate(x, z, region, finals));
}
}
hz_laps.lap("st_decorate");
self.schedule_generation(world, workers);
hz_laps.lap("st_generate");
// Lighting once all neighbours exist, nearest first and only as
// much of it as the workers can chew. Vanilla's light engine also
// hands a bounded queue to its workers and reschedules what did
@@ -2019,6 +1998,7 @@ impl Streaming {
Job::Light(x, z, region, self.sequence, self.sky_light),
);
}
hz_laps.lap("st_light");
// A server draws nothing: its sections are never meshed.
if !self.mesh {
world.dirty_sections.clear();
@@ -2059,6 +2039,7 @@ impl Streaming {
!back
});
}
hz_laps.lap("st_waiting");
// Column by column: the checks below hold for all of a column's
// sections, and columns waiting for light cost one look per frame.
columns.extend(world.dirty_sections.columns());
@@ -2091,6 +2072,7 @@ impl Streaming {
}
candidates.push((self.distance(x, z), x, z, sections));
}
hz_laps.lap("st_candidates");
// Nearest first, and only as many sections as the workers can chew.
// Every ready column used to be submitted at once, so at a wide
// render distance the queue — and `meshing`, and each job's 3x3
@@ -0,0 +1,29 @@
use std::sync::atomic::{AtomicBool, AtomicU64, Ordering};
pub const KINDS: [&str; 7] = [
"load", "generate", "decorate", "finish", "light", "mesh", "other",
];
static ENABLED: AtomicBool = AtomicBool::new(false);
static NANOS: [AtomicU64; 7] = [const { AtomicU64::new(0) }; 7];
static COUNT: [AtomicU64; 7] = [const { AtomicU64::new(0) }; 7];
pub fn enable(on: bool) {
ENABLED.store(on, Ordering::Relaxed);
}
pub(super) fn enabled() -> bool {
ENABLED.load(Ordering::Relaxed)
}
pub(super) fn add(kind: usize, nanos: u64) {
NANOS[kind].fetch_add(nanos, Ordering::Relaxed);
COUNT[kind].fetch_add(1, Ordering::Relaxed);
}
/// (kind, nanoseconds, count) since the last call; resets them.
pub fn take() -> Vec<(&'static str, u64, u64)> {
(0..KINDS.len())
.map(|i| {
(
KINDS[i],
NANOS[i].swap(0, Ordering::Relaxed),
COUNT[i].swap(0, Ordering::Relaxed),
)
})
.collect()
}
@@ -0,0 +1,39 @@
//! What the streamer's queues hold, for `horizon-ctl memory` and captures.
use super::{Queued, Streaming};
/// What the streamer's work queues hold, each as (entries, estimated bytes).
#[derive(Debug, Default)]
pub struct QueueCensus {
/// Jobs queued and chunks generating.
pub generation: (usize, usize),
/// Chunks being lit and the edit log their jobs replay.
pub light: (usize, usize),
}
impl Streaming {
/// The queues' lengths and the bytes of their tables, from capacities. For `horizon-ctl memory`.
pub fn queue_census(&self) -> QueueCensus {
use std::mem::size_of;
let table = |capacity: usize, entry: usize| capacity * (entry + 1);
let queued = {
let queue = self.shared.queue.lock().unwrap();
(queue.len(), queue.capacity() * size_of::<Queued>())
};
QueueCensus {
generation: (
queued.0 + self.generating.len(),
queued.1 + table(self.generating.capacity(), size_of::<(i32, i32)>()),
),
light: (
self.lighting.len(),
table(self.lighting.capacity(), size_of::<((i32, i32), u64)>())
+ table(
self.light_snapshots.capacity(),
size_of::<((i32, i32), u64)>(),
)
+ self.light_log.capacity() * size_of::<(u64, (i32, i32, i32))>()
+ self.light_scratch.capacity() * size_of::<(i64, i32, i32)>(),
),
}
}
}
+5 -1
View File
@@ -26,6 +26,7 @@ horizon-hooks = { path = "../horizon-hooks" }
horizon-optifine = { path = "../horizon-optifine" }
# The server thread ahead of the workers (MinecraftServer.spin's Thread.setPriority).
horizon-threads = { path = "../horizon-threads" }
horizon-perf-format = { path = "../horizon-perf-format" }
horizon-hdr = { path = "../horizon-hdr" }
horizon-world = { path = "../horizon-world" }
horizon-protocol = { path = "../horizon-protocol" }
@@ -134,7 +135,9 @@ friends = ["dep:horizon-friends"]
rng-trace = ["horizon-core/rng-trace", "horizon-client-lib/rng-trace"]
# Rust's allocations go to the system heap instead of mimalloc, where heap profilers that replace
# malloc (heaptrack, valgrind's massif) see them. For profiling only: mimalloc is faster.
system-allocator = []
system-allocator = ["horizon-threads/system-allocator"]
# mimalloc that counts each thread's allocations and frees (`stats` replies with `alloc`).
alloc-count = ["horizon-threads/alloc-count"]
# mimalloc with a tag on each block: `horizon-ctl memory` lists the live heap by startup step and
# thread (`heap_by_tag_mb`). For measuring only: each block costs 16 bytes more.
alloc-census = ["horizon-threads/alloc-census"]
@@ -178,5 +181,6 @@ workspace = true
[dev-dependencies]
horizon-client-render = { path = "../horizon-client-render", features = ["test-support"] }
horizon-items = { path = "../horizon-items", features = ["test-support"] }
horizon-perf = { path = "../horizon-perf" }
# Every WGSL shader translated as wgpu's DirectX 12 backend does (gfx.rs tests).
naga = { version = "30.0.1", features = ["wgsl-in", "hlsl-out", "msl-out", "spv-out"] }
@@ -1129,6 +1129,34 @@
"horizon.recipe_transfer.no_room": "Inventory full",
"horizon.recipe_transfer.unavailable": "Recipe does not fit this menu",
"horizon.recipe_transfer.unsupported": "Server requires a known recipe",
"horizon.debug.pass_detail.title": "Horizon pass detail (%s)",
"horizon.debug.pass_detail.inside_passes": "timestamps inside passes",
"horizon.debug.pass_detail.separate_passes": "separate passes, adds GPU time",
"horizon.debug.pass_detail.part": "%s › %s: %s ms",
"horizon.debug.pass_detail.counters": "%s: %s fragments, %s per pixel, %s vertices, %s of %s triangles kept",
"horizon.debug.pass_detail.no_counters": "GPU counters: Vulkan only",
"horizon.perf.started": "Profiling the client and server. %s",
"horizon.perf.uploading": "Uploading the profile…",
"horizon.perf.shared": "Shared: %s %s",
"horizon.perf.upload_failed": "Could not share the profile: %s. %s",
"horizon.perf.upload_interrupted": "the upload worker stopped",
"horizon.perf.unavailable": "Opening and sharing profiles is not available in this build.",
"horizon.perf.share_missing": "No profile saved in this session yet. %s",
"horizon.perf.saved": "Profile saved (%s s). %s",
"horizon.perf.open_button": "[Open]",
"horizon.perf.open_hover": "Open this profile in your browser. It is served once to your own browser and goes nowhere else.",
"horizon.perf.share_button": "[Share]",
"horizon.perf.share_hover": "Upload this profile and get a link to share. Nothing is uploaded until you click this.",
"horizon.perf.opened": "Opened in your browser",
"horizon.perf.open_failed": "Could not open the profile: %s. %s",
"horizon.perf.not_found": "No saved profile named %s.",
"horizon.perf.error": "Profiler: %s",
"horizon.perf.not_horizon": "this server isn't running Horizon",
"horizon.perf.denied": "a server profile needs operator level 4",
"horizon.perf.already_running": "the server is already running a profile",
"horizon.perf.not_running": "no server profile of yours is running",
"horizon.perf.failed": "the server could not write the profile; see its log",
"horizon.perf.part_incomplete": "the server's part of the profile did not arrive whole",
"horizon.studio.timeline.freecam_on": "Freecam: ON",
"horizon.studio.timeline.freecam_off": "Freecam: OFF",
"horizon.studio.timeline.freecam_short": "Cam",
@@ -1164,5 +1192,25 @@
"horizon.studio.topbar.split_scene": "Split Scene",
"horizon.recipes.fuel.options": "Hold SHIFT to pin this tooltip, then click an ingredient to view recipes.",
"horizon.recipes.fuel.any": "Any Fuel",
"horizon.recipes.fuel.minecraft": "Minecraft"
"horizon.recipes.fuel.minecraft": "Minecraft",
"horizon.perf.idle": "Not profiling. %s",
"horizon.perf.marked": "Marked %s",
"horizon.perf.started.client": "Profiling the client. %s",
"horizon.perf.started.server": "Profiling the server. %s",
"horizon.perf.saving": "Saving the profile…",
"horizon.perf.running.client": "Profiling the client for %s s. %s",
"horizon.perf.running.server": "Profiling the server for %s s. %s",
"horizon.perf.running.both": "Profiling the client and server for %s s. %s",
"horizon.perf.client_only": "Profiling the client only: %s",
"horizon.perf.stop_button": "[Stop]",
"horizon.perf.stop_hover": "Stop profiling and save the profile.",
"horizon.perf.start_button": "[Start]",
"horizon.perf.start_hover": "Start profiling.",
"horizon.perf.folder_button": "[Folder]",
"horizon.perf.folder_hover": "Show %s in its folder.",
"horizon.perf.copy_button": "[Copy]",
"horizon.perf.copy_hover": "Copy the link.",
"horizon.perf.retry_button": "[Retry]",
"horizon.perf.retry_hover": "Try the upload again. The profile is still saved on your computer.",
"horizon.perf.link_hover": "Open the shared profile in your browser."
}
Binary file not shown.

After

Width:  |  Height:  |  Size: 335 B

+824 -92
View File
@@ -11,6 +11,7 @@ use std::sync::mpsc::{self, Sender};
use std::thread;
use std::time::{Duration, Instant};
use horizon_bench::compare::Ab;
use horizon_bench::{App, Frames, Kind, Report};
use serde_json::Value;
@@ -54,14 +55,105 @@ const FLIGHT_SECONDS: f64 = 12.0;
const SETTLE: Duration = Duration::from_secs(2);
const LOAD_LIMIT: Duration = Duration::from_secs(600);
/// `standard` flies every scene; a scene's name flies only that one.
pub fn scenes(scenario: &str) -> Option<Vec<Scene>> {
match scenario {
"standard" => Some(SCENES.to_vec()),
name => SCENES.iter().find(|s| s.0 == name).map(|s| vec![*s]),
/// Where the trip starts and ends: 512 blocks at 16 blocks/s, the pace that scrambles section ids.
const TRAVEL_FROM: [f64; 3] = [-960.0, 150.0, -176.0];
const TRAVEL_TO: [f64; 3] = [-960.0, 150.0, 336.0];
const TRAVEL_PITCH: f64 = 30.0;
const TRAVEL_SECONDS: f64 = 32.0;
const HOLD_BLOCKS: u32 = 4;
const BLOCK_SECONDS: f64 = 4.0;
const WARMUP_SECONDS: f64 = 2.0;
const PAIRS: u32 = 8;
/// Frame-rate settings the benchmark can set for a run; `a` is the default.
pub struct Switch {
pub name: &'static str,
pub word: &'static str,
pub key: &'static str,
pub a: &'static str,
pub b: &'static str,
}
pub const SWITCHES: [Switch; 4] = [
Switch {
name: "gpu-culling",
word: "gpu-culling",
key: "gpu_culling",
a: "off",
b: "on",
},
Switch {
name: "section-cull",
word: "section-cull",
key: "cull",
a: "on",
b: "off",
},
Switch {
name: "section-sides",
word: "section-sides",
key: "sides",
a: "on",
b: "off",
},
Switch {
name: "section-draws",
word: "section-draws",
key: "flat",
a: "flat",
b: "indirect",
},
];
/// What `--benchmark` runs.
pub enum Scenario {
/// `standard` flies every scene; a scene's name flies only that one.
Scenes(Vec<Scene>),
/// A 512-block flight, then held at its end.
Travel,
/// The flight with the switch's B applied, then its two values interleaved B A A B.
Ab(&'static Switch),
}
impl Scenario {
pub fn parse(name: &str) -> Result<Scenario, String> {
match name {
"standard" => Ok(Scenario::Scenes(SCENES.to_vec())),
"travel" => Ok(Scenario::Travel),
name if name.starts_with("ab:") => SWITCHES
.iter()
.find(|s| s.name == &name[3..])
.map(Scenario::Ab)
.ok_or_else(|| {
format!(
"unknown switch {}: {}",
&name[3..],
SWITCHES.map(|s| s.name).join(", ")
)
}),
name => SCENES
.iter()
.find(|s| s.0 == name)
.map(|s| Scenario::Scenes(vec![*s]))
.ok_or_else(|| {
format!("unknown benchmark {name}: standard, village, forest, ocean, mountain, travel or ab:SWITCH ({})", SWITCHES.map(|s| s.name).join(", "))
}),
}
}
}
/// `--benchmark-set SWITCH=VALUE`: the value must be one of the switch's two.
pub fn parse_set(value: &str) -> Result<(String, String), String> {
let (name, setting) = value.split_once('=').ok_or("expected SWITCH=VALUE")?;
let names = || SWITCHES.map(|s| s.name).join(", ");
let switch = SWITCHES
.iter()
.find(|s| s.name == name)
.ok_or_else(|| format!("unknown switch {name}: {}", names()))?;
if setting != switch.a && setting != switch.b {
return Err(format!("{name} is {} or {}", switch.a, switch.b));
}
Ok((name.to_owned(), setting.to_owned()))
}
/// What the game knows about the run before it starts.
pub struct Setup {
pub scenario: String,
@@ -73,6 +165,10 @@ pub struct Setup {
pub fps_cap: Option<u32>,
pub offscreen: bool,
pub game_dir: PathBuf,
/// `--benchmark-set`: a switch and its value, applied before the world opens.
pub set: Option<(String, String)>,
/// `--profile`: record a .hzperf capture of the run and embed the report in it.
pub profile: bool,
}
/// Starts the driver; the game quits when it is done, and the report's path is printed.
@@ -150,8 +246,7 @@ fn time_passes(control: &Control, scene: &str, report: &mut Report) -> Result<bo
}
fn run(control: &Control, setup: Setup) -> Result<PathBuf, String> {
let scenes =
scenes(&setup.scenario).ok_or_else(|| format!("unknown scenario {}", setup.scenario))?;
let scenario = Scenario::parse(&setup.scenario)?;
let mut report = Report::new(Kind::Game, setup.app.clone());
let pid = std::process::id();
report.conditions.load_avg_1m = horizon_bench::load_average();
@@ -191,85 +286,12 @@ fn run(control: &Control, setup: Setup) -> Result<PathBuf, String> {
horizon_telemetry::system::storage(&setup.game_dir).map(str::to_owned);
report.hardware.display = serde_json::from_value(title["display"].clone()).ok();
replace_world(&setup.game_dir)?;
control.call(&format!("open {WORLD} {SEED} creative"))?;
// The world opens on a worker: its folder may not exist yet when `open` replies.
std::fs::create_dir_all(setup.game_dir.join("saves").join(WORLD)).map_err(|e| e.to_string())?;
std::fs::write(
setup.game_dir.join("saves").join(WORLD).join(MARKER),
"Created by horizon-client --benchmark; replaced by the next run.\n",
)
.map_err(|e| e.to_string())?;
let loaded = wait_loaded(control, |_| true)?;
report.metric("startup.world_ms", &[ms(loaded - setup.started)]);
let mut steady = Vec::new();
for (name, start, end, pitch) in scenes {
control.call("time 6000")?;
let meshed = control.status()["meshed_sections"].as_f64().unwrap_or(0.0);
let generated = control.status()["generated_chunks"].as_f64().unwrap_or(0.0);
let [x, y, z] = start;
control.call(&format!("tp {x} {y} {z}"))?;
let teleported = Instant::now();
// A zero-length flight holds the player at the start, flying.
control.call(&format!("fly {x} {y} {z} {x} {y} {z} 0.1 {pitch}"))?;
control.call(&format!("look {:.3} {pitch}", yaw(start, end)))?;
let near = |p: &Value| {
let p: Vec<f64> = p
.as_array()
.into_iter()
.flatten()
.filter_map(Value::as_f64)
.collect();
p.len() == 3 && (0..3).map(|i| (p[i] - start[i]).powi(2)).sum::<f64>() < 4.0
};
let loaded = wait_loaded(control, near)?;
let load = loaded - teleported;
report.metric(&format!("chunks.{name}.load_ms"), &[ms(load)]);
let status = control.status();
let meshes = status["meshed_sections"].as_f64().unwrap_or(0.0) - meshed;
let chunks = status["generated_chunks"].as_f64().unwrap_or(0.0) - generated;
report.metric(&format!("chunks.{name}.meshed_count"), &[meshes]);
report.metric(&format!("chunks.{name}.generated_count"), &[chunks]);
if load.as_secs_f64() > 0.0 {
report.metric(
&format!("chunks.{name}.generated_per_s"),
&[chunks / load.as_secs_f64()],
);
}
if !time_passes(control, name, &mut report)? {
thread::sleep(SETTLE);
}
let [ex, ey, ez] = end;
let begin = Instant::now();
let flight = control.call(&format!(
"fly {x} {y} {z} {ex} {ey} {ez} {FLIGHT_SECONDS} {pitch} raw"
))?;
steady.push((begin, Instant::now()));
let series = |key: &str| -> Vec<f32> {
flight[key]
.as_array()
.into_iter()
.flatten()
.filter_map(Value::as_f64)
.map(|v| v as f32)
.collect()
};
let frames = Frames::of(
name,
0,
&series("frame_ms"),
&series("cpu_ms"),
&series("gpu_ms"),
)
.ok_or("the flight drew no frames")?;
report.metric(&format!("fps.{name}.avg_fps"), &[frames.avg_fps]);
report.metric(&format!("fps.{name}.low_1pct_fps"), &[frames.low_1pct_fps]);
report.metric(
&format!("fps.{name}.low_01pct_fps"),
&[frames.low_01pct_fps],
);
report.frames.push(frames);
match &scenario {
Scenario::Ab(switch) => keeping_switch(control, switch, || {
measure(control, &setup, &scenario, &mut report, &mut steady)
})?,
_ => measure(control, &setup, &scenario, &mut report, &mut steady)?,
}
let status = control.status();
@@ -287,8 +309,12 @@ fn run(control: &Control, setup: Setup) -> Result<PathBuf, String> {
c.vsync = Some(setup.vsync);
c.fps_cap = setup.fps_cap;
c.offscreen = Some(setup.offscreen);
c.repeats = Some(1);
c.warmup_s = Some(SETTLE.as_secs_f64());
let (repeats, warmup) = match scenario {
Scenario::Travel => (HOLD_BLOCKS, None),
Scenario::Ab(_) => (2 * PAIRS, Some(WARMUP_SECONDS)),
Scenario::Scenes(_) => (1, Some(SETTLE.as_secs_f64())),
};
(c.repeats, c.warmup_s) = (Some(repeats), warmup);
if let Some(bytes) = renderer["gpu_allocated_bytes"].as_f64() {
report.metric("memory.gpu_mb", &[bytes / f64::from(1 << 20)]);
}
@@ -312,9 +338,342 @@ fn run(control: &Control, setup: Setup) -> Result<PathBuf, String> {
report.metric("game.binary_bytes", &[meta.len() as f64]);
}
crate::telemetry::keep_benchmark(&crate::logging::game_home(), &report);
report
let written = report
.write_to(&setup.game_dir.join("benchmarks"))
.map_err(|e| e.to_string())
.map_err(|e| e.to_string())?;
if setup.profile {
match control.call(&stop_capture(&written)) {
Ok(done) => println!("Horizon: benchmark capture {}", text(&done["path"])),
Err(e) => eprintln!("Horizon: the benchmark capture failed: {e}"),
}
}
Ok(written)
}
/// Starts the capture the run's frames feed; its trigger is the benchmark's.
const START_CAPTURE: &str = "profile client start";
/// Ends the capture with the written report embedded in it.
fn stop_capture(report: &Path) -> String {
format!("profile client stop --bench {}", report.display())
}
/// Opens the benchmark world and flies the scenario in it.
fn measure(
control: &Control,
setup: &Setup,
scenario: &Scenario,
report: &mut Report,
steady: &mut Vec<(Instant, Instant)>,
) -> Result<(), String> {
match (scenario, &setup.set) {
(Scenario::Ab(switch), _) => apply_set(control, switch.name, switch.b)?,
(_, Some((name, value))) => apply_set(control, name, value)?,
_ => {}
}
replace_world(&setup.game_dir)?;
control.call(&format!("open {WORLD} {SEED} creative"))?;
// The world opens on a worker: its folder may not exist yet when `open` replies.
std::fs::create_dir_all(setup.game_dir.join("saves").join(WORLD)).map_err(|e| e.to_string())?;
std::fs::write(
setup.game_dir.join("saves").join(WORLD).join(MARKER),
"Created by horizon-client --benchmark; replaced by the next run.\n",
)
.map_err(|e| e.to_string())?;
let loaded = wait_loaded(control, |_| true)?;
report.metric("startup.world_ms", &[ms(loaded - setup.started)]);
if setup.profile {
control.call(START_CAPTURE)?;
}
match scenario {
Scenario::Scenes(scenes) => {
for &(name, start, end, pitch) in scenes {
control.call("time 6000")?;
load_scene(control, name, start, end, pitch, report)?;
if !time_passes(control, name, report)? {
thread::sleep(SETTLE);
}
let frames = fly(control, name, start, end, FLIGHT_SECONDS, pitch, steady)?;
fps_metrics(report, name, std::slice::from_ref(&frames));
report.frames.push(frames);
}
}
Scenario::Travel => travel(control, report, steady)?,
Scenario::Ab(switch) => {
trip(control, report, steady)?;
let plan = Plan {
pairs: PAIRS,
warmup: Duration::from_secs_f64(WARMUP_SECONDS),
};
ab_blocks(control, switch, &plan, report, steady)?;
}
}
Ok(())
}
/// Reads a switch before the run and puts it back after, whatever the run did and however it ended.
fn keeping_switch<T>(
control: &Control,
switch: &Switch,
run: impl FnOnce() -> Result<T, String>,
) -> Result<T, String> {
let initial = control.call(switch.word)?[switch.key].clone();
let result = run();
let restored = restore(control, switch, &initial);
match result {
Ok(value) => restored.map(|()| value),
Err(e) => Err(e),
}
}
/// Sends the arm whose reply has the value the switch had.
fn restore(control: &Control, switch: &Switch, initial: &Value) -> Result<(), String> {
if control.call(switch.word)?[switch.key] == *initial {
return Ok(());
}
for value in [switch.a, switch.b] {
if control.call(&format!("{} {value}", switch.word))?[switch.key] == *initial {
return Ok(());
}
}
Err(format!("{} could not be put back", switch.name))
}
/// Sets a switch with its control word; a refusal (GPU culling not offered) is the game's reason.
fn apply_set(control: &Control, name: &str, value: &str) -> Result<(), String> {
let switch = SWITCHES
.iter()
.find(|s| s.name == name)
.ok_or("unknown switch")?;
control.call(&format!("{} {value}", switch.word)).map(drop)
}
/// Teleports to a scene's start, holds there and waits for the view to load, reporting the load.
fn load_scene(
control: &Control,
name: &str,
start: [f64; 3],
end: [f64; 3],
pitch: f64,
report: &mut Report,
) -> Result<(), String> {
let meshed = control.status()["meshed_sections"].as_f64().unwrap_or(0.0);
let generated = control.status()["generated_chunks"].as_f64().unwrap_or(0.0);
let [x, y, z] = start;
control.call(&format!("tp {x} {y} {z}"))?;
let teleported = Instant::now();
// A zero-length flight holds the player at the start, flying.
control.call(&format!("fly {x} {y} {z} {x} {y} {z} 0.1 {pitch}"))?;
control.call(&format!("look {:.3} {pitch}", yaw(start, end)))?;
let loaded = wait_loaded(control, near(start))?;
let load = loaded - teleported;
report.metric(&format!("chunks.{name}.load_ms"), &[ms(load)]);
let status = control.status();
let meshes = status["meshed_sections"].as_f64().unwrap_or(0.0) - meshed;
let chunks = status["generated_chunks"].as_f64().unwrap_or(0.0) - generated;
report.metric(&format!("chunks.{name}.meshed_count"), &[meshes]);
report.metric(&format!("chunks.{name}.generated_count"), &[chunks]);
if load.as_secs_f64() > 0.0 {
report.metric(
&format!("chunks.{name}.generated_per_s"),
&[chunks / load.as_secs_f64()],
);
}
Ok(())
}
/// Whether a status position is within 2 blocks of `target`.
fn near(target: [f64; 3]) -> impl Fn(&Value) -> bool {
move |p| {
let p: Vec<f64> = p
.as_array()
.into_iter()
.flatten()
.filter_map(Value::as_f64)
.collect();
p.len() == 3 && (0..3).map(|i| (p[i] - target[i]).powi(2)).sum::<f64>() < 4.0
}
}
/// Flies the camera from `start` to `end` and returns the flight's frames.
fn fly(
control: &Control,
phase: &str,
start: [f64; 3],
end: [f64; 3],
seconds: f64,
pitch: f64,
steady: &mut Vec<(Instant, Instant)>,
) -> Result<Frames, String> {
let ([x, y, z], [ex, ey, ez]) = (start, end);
let begin = Instant::now();
let flight = control.call(&format!(
"fly {x} {y} {z} {ex} {ey} {ez} {seconds} {pitch} raw"
))?;
steady.push((begin, Instant::now()));
frames_of(&flight, phase, 0)
}
/// The frames of a `fly ... raw` or `bench ... raw` reply.
fn frames_of(reply: &Value, phase: &str, repeat: u32) -> Result<Frames, String> {
let series = |key: &str| -> Vec<f32> {
reply[key]
.as_array()
.into_iter()
.flatten()
.filter_map(Value::as_f64)
.map(|v| v as f32)
.collect()
};
Frames::of(
phase,
repeat,
&series("frame_ms"),
&series("cpu_ms"),
&series("gpu_ms"),
)
.ok_or_else(|| format!("{phase} drew no frames"))
}
/// The phase's frame figures over its blocks (one for a flight).
fn fps_metrics(report: &mut Report, phase: &str, blocks: &[Frames]) {
let each = |f: fn(&Frames) -> f64| blocks.iter().map(f).collect::<Vec<_>>();
report.metric(&format!("fps.{phase}.avg_fps"), &each(|b| b.avg_fps));
report.metric(
&format!("fps.{phase}.low_1pct_fps"),
&each(|b| b.low_1pct_fps),
);
report.metric(
&format!("fps.{phase}.low_01pct_fps"),
&each(|b| b.low_01pct_fps),
);
}
/// The median CPU and GPU time of a frame over the phase's blocks, where measured.
fn timing_metrics(report: &mut Report, phase: &str, blocks: &[Frames]) {
let cpu: Vec<f64> = blocks.iter().filter_map(|b| b.cpu_p50_ms).collect();
let gpu: Vec<f64> = blocks.iter().filter_map(|b| b.gpu_p50_ms).collect();
report.metric(&format!("cpu.{phase}.p50_ms"), &cpu);
report.metric(&format!("gpu.{phase}.p50_ms"), &gpu);
}
/// The flight, then the view at its end loaded.
fn trip(
control: &Control,
report: &mut Report,
steady: &mut Vec<(Instant, Instant)>,
) -> Result<(), String> {
for line in [
"command gamerule advance_time false",
"command gamerule advance_weather false",
"command weather clear",
"time 6000",
] {
control.call(line)?;
}
let (from, to) = (TRAVEL_FROM, TRAVEL_TO);
load_scene(control, "travel", from, to, TRAVEL_PITCH, report)?;
let flight = fly(
control,
"travel",
from,
to,
TRAVEL_SECONDS,
TRAVEL_PITCH,
steady,
)?;
let blocks = std::slice::from_ref(&flight);
fps_metrics(report, "travel", blocks);
timing_metrics(report, "travel", blocks);
report.frames.push(flight);
let [x, y, z] = to;
control.call(&format!("fly {x} {y} {z} {x} {y} {z} 0.1 {TRAVEL_PITCH}"))?;
control.call(&format!("look {:.3} {TRAVEL_PITCH}", yaw(from, to)))?;
wait_loaded(control, near(to))?;
Ok(())
}
/// The state that matters: fly 512 blocks over terrain that loads and unloads, then hold at the end.
fn travel(
control: &Control,
report: &mut Report,
steady: &mut Vec<(Instant, Instant)>,
) -> Result<(), String> {
trip(control, report, steady)?;
let mut blocks = Vec::new();
for repeat in 0..HOLD_BLOCKS {
let held = control.call(&format!("bench {BLOCK_SECONDS} raw"))?;
blocks.push(frames_of(&held, "hold", repeat)?);
}
fps_metrics(report, "hold", &blocks);
timing_metrics(report, "hold", &blocks);
report.frames.extend(blocks);
Ok(())
}
struct Plan {
pairs: u32,
warmup: Duration,
}
/// The two values of a switch in B A A B order, a block each, then each arm's passes timed once.
fn ab_blocks(
control: &Control,
switch: &Switch,
plan: &Plan,
report: &mut Report,
steady: &mut Vec<(Instant, Instant)>,
) -> Result<(), String> {
let began = Instant::now();
let mut blocks = Vec::new();
for pair in 0..plan.pairs {
control.call("time 6000")?;
for (n, b) in [(0, pair % 2 == 0), (1, pair % 2 == 1)] {
let (phase, value) = if b {
("ab.b", switch.b)
} else {
("ab.a", switch.a)
};
control.call(&format!("{} {value}", switch.word))?;
thread::sleep(plan.warmup);
let held = control.call(&format!("bench {BLOCK_SECONDS} raw"))?;
blocks.push(frames_of(&held, phase, 2 * pair + n)?);
}
}
steady.push((began, Instant::now()));
for phase in ["ab.a", "ab.b"] {
let arm: Vec<Frames> = blocks
.iter()
.filter(|f| f.phase == phase)
.cloned()
.collect();
fps_metrics(report, phase, &arm);
timing_metrics(report, phase, &arm);
}
let ab = Ab::from_frames(
switch.name,
switch.a,
switch.b,
BLOCK_SECONDS,
plan.warmup.as_secs_f64(),
&blocks,
)?;
if ab.first_block_outside_range {
eprintln!(
"Horizon: benchmark warning: the first B block, measured before any switch, is outside the later B blocks: switching {} may reset what is measured",
switch.name
);
}
report.frames.extend(blocks);
report.ab = Some(ab);
for (scene, value) in [("ab_a", switch.a), ("ab_b", switch.b)] {
control.call(&format!("{} {value}", switch.word))?;
time_passes(control, scene, report)?;
}
Ok(())
}
/// Removes the previous run's world, never one the benchmark didn't create.
@@ -472,11 +831,65 @@ impl RssSampler {
mod tests {
use super::*;
#[test]
fn the_capture_commands_are_the_ones_the_game_parses() {
use crate::client_perf::request::{self, ProfileSide, Request, StartOptions};
let words = |line: &str| request::parse(&line.split(' ').skip(1).collect::<Vec<_>>());
let start = words(START_CAPTURE).unwrap();
assert_eq!(start.side, Some(ProfileSide::Client));
assert_eq!(start.request, Request::Start(StartOptions::default()));
let path = Path::new("/home/a b/game/benchmarks/game-20261108-travel.json");
let stop = words(&stop_capture(path)).unwrap();
assert_eq!(stop.side, Some(ProfileSide::Client));
assert_eq!(
stop.request,
Request::Stop {
bench: Some(path.display().to_string())
}
);
}
#[test]
fn scenarios() {
assert_eq!(scenes("standard").unwrap().len(), 4);
assert_eq!(scenes("ocean").unwrap()[0].0, "ocean");
assert!(scenes("nether").is_none());
let scenes = |name: &str| match Scenario::parse(name) {
Ok(Scenario::Scenes(scenes)) => scenes,
_ => panic!("{name} is not scenes"),
};
assert_eq!(scenes("standard").len(), 4);
assert_eq!(scenes("ocean")[0].0, "ocean");
assert!(matches!(Scenario::parse("travel"), Ok(Scenario::Travel)));
assert!(matches!(
Scenario::parse("ab:gpu-culling"),
Ok(Scenario::Ab(s)) if s.name == "gpu-culling"
));
let Err(e) = Scenario::parse("ab:nope") else {
panic!("nope is no switch")
};
assert!(
e.contains("gpu-culling") && e.contains("section-draws"),
"{e}"
);
let Err(e) = Scenario::parse("nether") else {
panic!("nether is no scenario")
};
assert!(e.contains("travel") && e.contains("mountain"), "{e}");
}
#[test]
fn a_set_names_a_switch_and_one_of_its_two_values() {
assert_eq!(
parse_set("gpu-culling=on"),
Ok(("gpu-culling".into(), "on".into()))
);
assert_eq!(
parse_set("section-draws=indirect"),
Ok(("section-draws".into(), "indirect".into()))
);
let nope = parse_set("gpu-culling=maybe").unwrap_err();
assert!(nope.contains("off or on"), "{nope}");
let unknown = parse_set("fog=on").unwrap_err();
assert!(unknown.contains("gpu-culling") && unknown.contains("section-draws"));
assert!(parse_set("gpu-culling").is_err());
}
#[test]
@@ -588,6 +1001,325 @@ mod tests {
assert!(metrics.is_empty());
}
/// A fake game for the trip: the player stands where `tp` or `fly` put it and counts settle.
fn trip_server(rx: mpsc::Receiver<Request>) -> thread::JoinHandle<Vec<String>> {
thread::spawn(move || {
let (mut lines, mut at, mut polls) = (Vec::new(), [0.0_f64; 3], 0_u64);
while let Ok(request) = rx.recv() {
let words: Vec<&str> = request.line.split(' ').collect();
let reply = match words[0] {
"status" => {
polls += 1;
let mut s: Value =
serde_json::from_str(&status(400 + polls.min(3))).unwrap();
s["player"]["position"] = serde_json::json!(at);
s
}
"bench" | "fly" if words.contains(&"raw") => {
serde_json::json!({
"frames": 200,
"frame_ms": vec![5.0; 200],
"cpu_ms": vec![2.0; 200],
"gpu_ms": vec![4.0; 200],
})
}
_ => serde_json::json!({ "ok": true }),
};
if matches!(words[0], "tp" | "fly") {
let n: Vec<f64> = words[1..4].iter().filter_map(|w| w.parse().ok()).collect();
at = [n[0], n[1], n[2]];
polls = 0;
}
let _ = request.reply.send(reply.to_string());
lines.push(request.line);
}
lines
})
}
#[test]
fn the_trip_flies_512_blocks_then_holds_four_blocks_at_its_end() {
let (tx, rx) = mpsc::channel::<Request>();
let server = trip_server(rx);
let mut report = Report::new(Kind::Game, App::default());
let control = Control(tx);
let mut steady = Vec::new();
travel(&control, &mut report, &mut steady).expect("the trip runs");
drop(control);
let lines: Vec<String> = server
.join()
.unwrap()
.into_iter()
.filter(|l| l != "status")
.map(|l| l.replace("-0.000", "0.000"))
.collect();
// Facing +Z is yaw 0, which the formatter may print as -0.
let mut expected = vec![
"command gamerule advance_time false",
"command gamerule advance_weather false",
"command weather clear",
"time 6000",
"tp -960 150 -176",
"fly -960 150 -176 -960 150 -176 0.1 30",
"look 0.000 30",
"fly -960 150 -176 -960 150 336 32 30 raw",
"fly -960 150 336 -960 150 336 0.1 30",
"look 0.000 30",
];
expected.extend(["bench 4 raw"; 4]);
assert_eq!(lines, expected);
let phases: Vec<_> = report
.frames
.iter()
.map(|f| (f.phase.as_str(), f.repeat))
.collect();
assert_eq!(
phases,
[
("travel", 0),
("hold", 0),
("hold", 1),
("hold", 2),
("hold", 3)
]
);
assert_eq!(steady.len(), 1);
let metric = |name: &str| report.metrics.get(name).map(|s| (s.n, s.mean));
assert_eq!(metric("fps.travel.avg_fps"), Some((1, 200.0)));
assert_eq!(metric("fps.hold.avg_fps"), Some((4, 200.0)));
assert_eq!(metric("cpu.hold.p50_ms"), Some((4, 2.0)));
assert_eq!(metric("gpu.hold.p50_ms"), Some((4, 4.0)));
assert_eq!(metric("gpu.travel.p50_ms"), Some((1, 4.0)));
assert!(metric("chunks.travel.load_ms").is_some());
}
#[test]
fn a_refused_command_ends_the_trip_with_the_games_reason() {
let (tx, rx) = mpsc::channel::<Request>();
let server = thread::spawn(move || {
while let Ok(request) = rx.recv() {
let reply = if request.line.starts_with("command weather") {
serde_json::json!({ "error": "no such weather" })
} else {
serde_json::json!({ "ok": true })
};
let _ = request.reply.send(reply.to_string());
}
});
let mut report = Report::new(Kind::Game, App::default());
let control = Control(tx);
let error = travel(&control, &mut report, &mut Vec::new()).unwrap_err();
assert!(error.contains("no such weather"), "{error}");
assert!(report.frames.is_empty());
drop(control);
server.join().unwrap();
}
#[test]
fn a_refused_switch_is_the_games_reason() {
let (tx, rx) = mpsc::channel::<Request>();
let server = thread::spawn(move || {
let mut lines = Vec::new();
while let Ok(request) = rx.recv() {
let reply = match request.line.as_str() {
"gpu-culling on" => serde_json::json!({ "error": "not offered here" }),
_ => serde_json::json!({ "ok": true }),
};
let _ = request.reply.send(reply.to_string());
lines.push(request.line);
}
lines
});
let control = Control(tx);
apply_set(&control, "section-draws", "indirect").unwrap();
let error = apply_set(&control, "gpu-culling", "on").unwrap_err();
assert!(error.contains("not offered here"), "{error}");
drop(control);
assert_eq!(
server.join().unwrap(),
["section-draws indirect", "gpu-culling on"]
);
}
/// A fake game with GPU culling: `on` is its state; `fail_at` makes that `bench` (counted from 1) fail.
fn culling_server(
rx: mpsc::Receiver<Request>,
mut on: bool,
offered: bool,
fail_at: Option<usize>,
) -> thread::JoinHandle<(Vec<String>, bool)> {
thread::spawn(move || {
let (mut lines, mut benches) = (Vec::new(), 0);
while let Ok(request) = rx.recv() {
let reply = match request.line.as_str() {
"gpu-culling on" if !offered => {
serde_json::json!({ "error": "not offered here" })
}
"gpu-culling on" | "gpu-culling off" => {
on = request.line.ends_with("on");
serde_json::json!({ "gpu_culling": on })
}
"gpu-culling" => serde_json::json!({ "gpu_culling": on }),
"bench 4 raw" => {
benches += 1;
let gpu = if on { 5.0 } else { 3.0 };
if Some(benches) == fail_at {
serde_json::json!({ "error": "the window closed" })
} else {
serde_json::json!({
"frame_ms": vec![gpu + 1.0; 200],
"cpu_ms": vec![1.0; 200],
"gpu_ms": vec![gpu; 200],
})
}
}
"bench 2" => serde_json::json!({
"gpu_passes": {"World": if on { 1.5 } else { 0.5 }},
}),
_ => serde_json::json!({ "ok": true }),
};
let _ = request.reply.send(reply.to_string());
lines.push(request.line);
}
(lines, on)
})
}
const GPU_CULLING: &Switch = &SWITCHES[0];
fn quick(pairs: u32) -> Plan {
Plan {
pairs,
warmup: Duration::ZERO,
}
}
fn vulkan_report() -> Report {
let gpu = horizon_bench::Gpu {
backend: "vulkan".into(),
..Default::default()
};
let hardware = horizon_bench::Hardware {
gpu: Some(gpu),
..Default::default()
};
Report::with_hardware(Kind::Game, App::default(), hardware)
}
#[test]
fn the_blocks_run_b_a_a_b_and_each_arm_is_timed_by_pass() {
let (tx, rx) = mpsc::channel::<Request>();
let server = culling_server(rx, false, true, None);
let mut report = vulkan_report();
let control = Control(tx);
keeping_switch(&control, GPU_CULLING, || {
ab_blocks(
&control,
GPU_CULLING,
&quick(6),
&mut report,
&mut Vec::new(),
)
})
.expect("the blocks run");
drop(control);
let (lines, on) = server.join().unwrap();
let arms: String = lines
.iter()
.take_while(|l| *l != "gpu-passes on")
.filter_map(|l| match l.as_str() {
"gpu-culling on" => Some('B'),
"gpu-culling off" => Some('A'),
_ => None,
})
.collect();
assert_eq!(arms, "BAABBAABBAABA");
assert_eq!(lines.iter().filter(|l| *l == "time 6000").count(), 6);
assert!(!on, "the initial value is back");
assert_eq!(lines.last().unwrap(), "gpu-culling off");
let positions: Vec<_> = report
.frames
.iter()
.map(|f| (f.phase.as_str(), f.repeat))
.collect();
assert_eq!(
positions[..4],
[("ab.b", 0), ("ab.a", 1), ("ab.a", 2), ("ab.b", 3)]
);
let ab = report.ab.as_ref().expect("an A/B run has its result");
let gpu = &ab.result["gpu_p50_ms"];
assert_eq!((gpu.pairs, gpu.a, gpu.b, gpu.diff), (6, 3.0, 5.0, 2.0));
assert_eq!(gpu.verdict, horizon_bench::compare::Verdict::Slower);
assert!(!ab.first_block_outside_range);
let metric = |name: &str| report.metrics.get(name).map(|s| (s.n, s.mean));
assert_eq!(metric("gpu.ab.a.p50_ms"), Some((6, 3.0)));
assert_eq!(metric("gpu.ab.b.p50_ms"), Some((6, 5.0)));
assert_eq!(metric("gpu_pass.ab_a.world_ms"), Some((1, 0.5)));
assert_eq!(metric("gpu_pass.ab_b.world_ms"), Some((1, 1.5)));
}
#[test]
fn a_setting_that_was_on_is_on_again_after_the_run() {
let (tx, rx) = mpsc::channel::<Request>();
let server = culling_server(rx, true, true, None);
let control = Control(tx);
keeping_switch(&control, GPU_CULLING, || {
ab_blocks(
&control,
GPU_CULLING,
&quick(2),
&mut vulkan_report(),
&mut Vec::new(),
)
})
.expect("the blocks run");
drop(control);
assert!(server.join().unwrap().1);
}
#[test]
fn the_setting_is_put_back_when_the_run_fails_midway() {
let (tx, rx) = mpsc::channel::<Request>();
// The 3rd block is A: the setting is off when it fails.
let server = culling_server(rx, true, true, Some(3));
let control = Control(tx);
let mut report = vulkan_report();
let error = keeping_switch(&control, GPU_CULLING, || {
ab_blocks(
&control,
GPU_CULLING,
&quick(6),
&mut report,
&mut Vec::new(),
)
})
.unwrap_err();
assert!(error.contains("the window closed"), "{error}");
assert!(report.frames.is_empty() && report.ab.is_none());
drop(control);
let (lines, on) = server.join().unwrap();
assert!(on, "on was the player's setting");
assert_eq!(lines.last().unwrap(), "gpu-culling on");
}
#[test]
fn a_refused_switch_ends_the_run_with_the_games_reason_and_changes_nothing() {
let (tx, rx) = mpsc::channel::<Request>();
let server = culling_server(rx, false, false, None);
let control = Control(tx);
let error = keeping_switch(&control, GPU_CULLING, || {
apply_set(&control, "gpu-culling", "on")
})
.unwrap_err();
assert!(error.contains("not offered here"), "{error}");
drop(control);
let (lines, on) = server.join().unwrap();
assert_eq!(lines, ["gpu-culling", "gpu-culling on", "gpu-culling"]);
assert!(!on);
}
#[test]
fn yaw_matches_vanilla() {
// South (+Z) is 0, west (-X) 90, east (+X) -90.
@@ -589,12 +589,22 @@ pub const COMMANDS: &[Cmd] = &[
cmd!("mcp", Scripting, "mcp [--paced]", "serve the Model Context Protocol on stdin/stdout, one tool per command",
"Add it to an MCP client as a stdio server: command horizon-ctl, args [\"mcp\"] (with --socket PATH first if needed). The screenshot tool returns the image. --paced stops a single-player world between tool calls: the pause menu opens after each action and closes before the next, so time passes only while an action (or wait) runs, not while the agent thinks; reading tools (state, inventory, entities, events...) leave it paused.",
["horizon-ctl mcp", "horizon-ctl mcp --paced"], &[], mcp: false, local: true),
cmd!("bench", Scripting, "bench SECONDS", "reset frame stats, wait, reply with the summary", "", ["horizon-ctl bench 10"], &[p("seconds", Ty::Num, "how long to measure")], mcp: true, local: false),
cmd!("stats", Scripting, "stats [reset] [raw]", "frame-time summary since the last reset", "", ["horizon-ctl stats"], &[opt("reset", Ty::Bool, "start counting again"), opt("raw", Ty::Bool, "every frame time too (frame_ms, cpu_ms, gpu_ms)")], mcp: true, local: false),
cmd!("gpu-passes", Scripting, "gpu-passes [on|off]", "time Horizon's own render passes on the GPU, shown in F3 and in stats and bench", "", ["horizon-ctl gpu-passes on"], &[opt("mode", Ty::Enum(&["on", "off"]), "on or off; without it, only reports")], mcp: false, local: false),
cmd!("trace", Scripting, "trace [FILE [SECONDS] | stop]", "write every thread's timed spans to FILE as a timeline that ui.perfetto.dev opens, for SECONDS or until stop",
"Each span (ticks by system, streaming, uploads, far terrain, lighting, the renderer) shows on the thread that ran it, nested by time. Without arguments, only reports whether a trace is being written. HORIZON_TRACE=FILE writes one from the start; horizon-trace on a dedicated server's console writes one to reports/. See docs/graphics.md.",
["horizon-ctl trace /tmp/frame.json 10", "horizon-ctl trace stop"], &[], mcp: false, local: false),
cmd!("bench", Scripting, "bench SECONDS [raw]", "reset frame stats, wait, reply with the summary", "", ["horizon-ctl bench 10", "horizon-ctl bench 4 raw"], &[p("seconds", Ty::Num, "how long to measure"), opt("raw", Ty::Bool, "every frame time too (frame_ms, cpu_ms, gpu_ms; alloc_count and alloc_bytes in an alloc-count build)")], mcp: true, local: false),
cmd!("stats", Scripting, "stats [reset] [raw]", "frame-time summary since the last reset", "A build with the alloc-count feature adds alloc: the window thread's allocations and KiB per frame and all threads' allocations per second, else null.", ["horizon-ctl stats"], &[opt("reset", Ty::Bool, "start counting again"), opt("raw", Ty::Bool, "every frame time too (frame_ms, cpu_ms, gpu_ms; alloc_count and alloc_bytes in an alloc-count build)")], mcp: true, local: false),
cmd!("gpu-passes", Scripting, "gpu-passes [on|off|detail]", "time Horizon's own render passes on the GPU, shown in F3 and in stats and bench; detail adds dispatch times and, on Vulkan, fragment and vertex counts per world pass", "", ["horizon-ctl gpu-passes on"], &[opt("mode", Ty::Enum(&["on", "off", "detail"]), "on, off or detail; without it, only reports")], mcp: false, local: false),
cmd!("trace", Scripting, "trace [FILE [SECONDS] | stop]", "write every thread's timed spans to FILE as a .hzperf capture holding a timeline, for SECONDS or until stop",
"Each span (ticks by system, streaming, uploads, far terrain, lighting, the renderer) is recorded on the thread that ran it, with its depth; the timeline stops after 60 seconds. trace stop replies with the file's path once it is whole. Without arguments, only reports whether a trace is being written. HORIZON_TRACE=FILE writes one from the start; horizon-trace on a dedicated server's console writes one to reports/. Open the file in the web UI. See docs/graphics.md.",
["horizon-ctl trace /tmp/frame.hzperf 10", "horizon-ctl trace stop"], &[], mcp: false, local: false),
cmd!("profile", Scripting, "profile [client|server] start [--seconds S] [--hz N] [--over MS] [--threads main|all] [--wall] [--gpu[=detail]] [--no-samples] [--timeline] | stop [--bench REPORT] | status | mark NAME | open FILE | share FILE", "capture the game's performance to a .hzperf file in reports/ in the game home",
"start records the frame times, the health history and the render thread's call trees until stop or --seconds (at most 600). stop replies with the file's path once the file is whole, and with the viewer's address (open_at) only in a build that has a part to open captures; open FILE serves the file once on 127.0.0.1 and opens the viewer on it, share FILE uploads it and replies with the link; both fail in a build without that part and work with every telemetry switch off; status says whether a capture runs, its age and its size; mark NAME puts a label at the current time. F3+O does start and stop from the keyboard; HORIZON_PROFILE=DIR records the first 10 minutes of a run into DIR. --gpu adds each render pass's median GPU time and --gpu=detail the dispatch times and, on Vulkan, fragment and vertex counts per pass; HORIZON_SHADER_PACK_TIMING and HORIZON_GPU_PASS_DETAIL start a capture with them. stop --bench REPORT embeds a benchmark report (benchmarks/*.json) in the capture; --benchmark --profile does this by itself. Read the file with horizon-perf (query gpu). With no side a single-player run captures both sides in one file (the window, and the server thread as its own section); client or server captures one. Connected to a Horizon server, server asks that server to capture its side: it needs operator level 4, only one server capture runs at a time, and the server keeps the file in its reports/ and logs who asked; a refusal is a warning in the window's own capture.",
["horizon-ctl profile start --seconds 30", "horizon-ctl profile mark loaded-the-nether", "horizon-ctl profile stop", "horizon-ctl profile open reports/profile-20251107-143844Z.hzperf"],
&[p("action", Ty::Enum(&["start", "stop", "status", "mark", "open", "share"]), "what to do"), opt("name", Ty::Rest, "mark: the label; open and share: the file"),
flag("side", Ty::Enum(&["client", "server"]), "client or server, for the window or the server (the single-player server thread, or the server joined); both when omitted"), flag("seconds", Ty::Int, "start: stop after this many seconds, 1 to 600"),
flag("hz", Ty::Int, "start: samples per second, 10 to 1000 (default 250)"), flag("over", Ty::Num, "start: sample only frames slower than this many milliseconds"),
flag("threads", Ty::Enum(&["main", "all"]), "start: the render thread (default) or every thread that opted in"), flag("wall", Ty::Bool, "start: also sample on the wall clock, to find waits"),
flag("gpu", Ty::Enum(&["passes", "detail"]), "start: also record each render pass's GPU time (detail: its parts and counters too)"),
flag("no-samples", Ty::Bool, "start: record frame times and health only"), flag("timeline", Ty::Bool, "start: also record every thread's spans, for 60 seconds at most"),
flag("bench", Ty::Str, "stop: the path of a benchmark report to embed in the capture")], mcp: true, local: false),
cmd!("screenshot-sequence", Scripting, "screenshot-sequence FRAMES STEP_MS PATH|cancel", "draw a shader pack's first FRAMES frames on a set clock and write the last as a PNG",
"Needs HORIZON_STILL and a frozen world (docs/agent-control.md). The pack starts over, frame n's clock is n frames and (n - 1) × STEP_MS milliseconds, the PNG is written as screenshot PATH writes it, and the reply comes once it is. Relative paths are taken from horizon-ctl's directory. cancel stops a running sequence.",
["horizon-ctl screenshot-sequence 64 50 frame64.png", "horizon-ctl screenshot-sequence 64 16.666667 frame64.png", "horizon-ctl screenshot-sequence cancel"],
+13 -10
View File
@@ -19,16 +19,16 @@ pub use horizon_client_render::{
effects, environment, extras, fast_hash, fluid_render, fog, font, gfx, graphics,
graphics_style, graphics_ui, hdr_calibration, hud, hud_kit, hud_layout, hud_overlays,
inventory_effects, java_text, jigsaw_block, json_folder, level_loading, light, lod, logging,
menu_blur, menu_style, mesher, minimap, mipmap, mod_browser, mod_list, model_library, models,
multiline_edit, native_emissive, options_file, options_fixes, options_ui, overlay,
pack_browser, pack_formats, pack_hooks, pack_palette, pack_screen, packs, pad_cursor,
pad_glyph, par, particles, poi_sections, polytone_creative_tabs, polytone_post, post_chain,
potent_sulfur, preedit, providers, pvp, pvp_hud, random_source, screen_draw, screen_effects,
sculk, section_mask, server_terrain, settings, shader_options, shader_pack_list, signs, skins,
social, sound_engine, sound_events, sound_preview, spans, split_layout, split_map,
split_profiles, sprites, streaming, structure_block, tab_list, terrain_model, test_block,
test_instance_block, testgen, title_edition, trims, ui, vault, weather, widgets, world,
worldborder, zip,
mark_sprite, menu_blur, menu_style, mesher, minimap, mipmap, mod_browser, mod_list,
model_library, models, multiline_edit, native_emissive, options_file, options_fixes,
options_ui, overlay, pack_browser, pack_formats, pack_hooks, pack_palette, pack_screen, packs,
pad_cursor, pad_glyph, par, particles, poi_sections, polytone_creative_tabs, polytone_post,
post_chain, potent_sulfur, preedit, providers, pvp, pvp_hud, random_source, screen_draw,
screen_effects, sculk, section_mask, server_terrain, settings, shader_options,
shader_pack_list, signs, skins, social, sound_engine, sound_events, sound_preview, spans,
split_layout, split_map, split_profiles, sprites, streaming, structure_block, tab_list,
terrain_model, test_block, test_instance_block, testgen, title_edition, trims, ui, vault,
weather, widgets, world, worldborder, zip,
};
pub mod attack_strength;
pub mod attribute_updates;
@@ -43,6 +43,7 @@ pub mod boss_overlay;
pub mod breathing;
pub mod cauldron;
pub mod chunk_io;
pub mod client_perf;
pub mod clock;
pub mod command_storage;
pub mod confirm_screen;
@@ -107,6 +108,7 @@ pub mod offline_name;
pub mod old_world;
pub mod osk;
pub mod patrol;
pub mod perf_net;
pub mod perf_overlay;
pub mod pick_item;
pub mod player_sync;
@@ -129,6 +131,7 @@ pub mod sound_playback;
pub use horizon_server_files as server;
pub mod controller_notice;
pub mod server_packs;
pub mod server_perf;
pub mod shadows;
pub mod spear;
pub mod spectator_menu;
+321
View File
@@ -0,0 +1,321 @@
//! The client's .hzperf captures (docs/perf-platform.md §4.1): the window's frames feed the always-on health history, and a capture writes it with the samples of the render thread.
use crate::gfx::pack_timing::{GpuReadout, TimedFrames};
use crate::server_perf::traffic::{Conn, Wire};
use crate::server_perf::{
self, CaptureEnd, CaptureOptions, CaptureStatus, MAX_WINDOW_S, Role, SamplerOptions, Saved,
ServerPerf, TickDetail, WorldCounts,
};
use horizon_perf_format::FieldSender;
use horizon_perf_format::v1::{
BenchResult, Machine, Side, Start, StopReason, Trigger, WarningCode,
};
use horizon_protocol::connection::Counters;
use request::{GpuSwitch, StartOptions, Threads};
use std::ffi::OsStr;
use std::path::{Path, PathBuf};
use std::sync::Arc;
use std::sync::atomic::Ordering;
use std::time::{Duration, Instant};
pub mod request;
#[cfg(test)]
mod tests;
/// A frame this slow enters the slow ring when `HORIZON_SPIKES` names no other threshold.
pub const SLOW_FRAME_MS: f32 = 50.0;
/// The variables that start a capture with the GPU passes timed: pass timing, and pass timing with its parts.
pub const ENV_PASS_TIMING: &str = "HORIZON_SHADER_PACK_TIMING";
pub const ENV_PASS_DETAIL: &str = "HORIZON_GPU_PASS_DETAIL";
/// The GPU switch a run's environment asks for; the detail variable wins over the timing one.
pub fn env_gpu(timing: Option<&OsStr>, detail: Option<&OsStr>) -> GpuSwitch {
if detail.is_some() {
GpuSwitch::Detail
} else if timing.is_some() {
GpuSwitch::Passes
} else {
GpuSwitch::Off
}
}
/// Where a client's captures go: `HORIZON_PROFILE` when set, else `reports/` in the game home.
pub fn reports_dir(profile: Option<&OsStr>, home: &Path) -> PathBuf {
match profile.filter(|p| !p.is_empty()) {
Some(dir) => PathBuf::from(dir),
None => home.join("reports"),
}
}
/// What a side on another thread needs to write into the running capture's file.
#[derive(Clone)]
pub struct Join {
pub sender: FieldSender,
pub path: PathBuf,
/// The instant every `t` of the capture counts from.
pub origin: Instant,
/// When the capture's window ends, and why it stops then.
pub ends: Option<(Instant, StopReason)>,
pub options: StartOptions,
pub trigger: Trigger,
}
/// The client's health history, always on, and the capture running from it, if any.
pub struct ClientPerf {
perf: ServerPerf,
epoch: Instant,
cpu_ms: Vec<f32>,
gpu_ms: Vec<f32>,
/// When the running capture's window ends, and why it stops then.
window: Option<(f64, StopReason)>,
/// The running capture's switches, which a side that joins it follows.
running: Option<(StartOptions, Trigger)>,
/// The server connection's byte counts while the window plays on one: the capture's own traffic.
connection: Option<Arc<Counters>>,
}
/// The window's one connection as the traffic recorder's wire list.
fn wire_of(connection: Option<&Counters>) -> Wire {
let order = Ordering::Relaxed;
connection
.map(|c| {
let count = |n: &std::sync::atomic::AtomicI64| n.load(order).max(0) as u64;
vec![Conn {
id: 1,
bytes: (count(&c.bytes_in), count(&c.bytes_out)),
..Conn::default()
}]
})
.unwrap_or_default()
}
impl ClientPerf {
pub fn new(slow_ms: f32) -> Self {
Self {
perf: ServerPerf::with_role(Role::CLIENT, 0.0, slow_ms),
epoch: Instant::now(),
cpu_ms: Vec::new(),
gpu_ms: Vec::new(),
window: None,
running: None,
connection: None,
}
}
/// The connection whose bytes a capture records, or None in a world of the window's own.
pub fn set_connection(&mut self, connection: Option<Arc<Counters>>) {
self.connection = connection;
}
/// Seconds since the game began, the clock every other time here is on.
pub fn now(&self) -> f64 {
self.epoch.elapsed().as_secs_f64()
}
pub fn capturing(&self) -> bool {
self.perf.capturing()
}
pub fn health(&self) -> &horizon_perf_format::HealthRecorder {
self.perf.health()
}
/// One rendered frame that took `wall_ms`, of which `cpu_ms` was the CPU's own work, ending at `t`.
pub fn frame(
&mut self,
t: f64,
wall_ms: f32,
cpu_ms: f32,
gpu_ms: Option<f32>,
detail: Option<TickDetail>,
world: impl FnOnce() -> WorldCounts,
) {
self.frame_with_gpu(t, wall_ms, cpu_ms, gpu_ms, detail, world, || None);
}
/// [`ClientPerf::frame`], where `gpu` is asked for the pass timings only when this frame ends the capture's window.
#[allow(clippy::too_many_arguments)]
pub fn frame_with_gpu(
&mut self,
t: f64,
wall_ms: f32,
cpu_ms: f32,
gpu_ms: Option<f32>,
detail: Option<TickDetail>,
world: impl FnOnce() -> WorldCounts,
gpu: impl FnOnce() -> Option<GpuReadout>,
) {
self.cpu_ms.push(cpu_ms);
self.gpu_ms.extend(gpu_ms);
if self.perf.gauges_due(t) {
self.perf
.set_frame_medians(median(&mut self.cpu_ms), median(&mut self.gpu_ms));
}
let connection = self.connection.as_deref();
self.perf
.tick(t, wall_ms, detail, world, || wire_of(connection));
if let Some((end, reason)) = self.window
&& t >= end
{
self.stop_with_gpu(t, reason, gpu());
}
}
/// The GPU frames timed since the last frame, with this frame's CPU ms, for a capture that times the GPU.
pub fn gpu_frames(&mut self, t: f64, cpu_ms: f32, frames: &TimedFrames) {
self.perf.gpu_frames(t, cpu_ms, frames);
}
/// Opens a capture in `dir`; call it on the render thread, which it samples.
pub fn start(
&mut self,
t: f64,
dir: &Path,
trigger: Trigger,
options: &StartOptions,
start: Start,
block_names: Vec<String>,
) -> std::io::Result<PathBuf> {
let capture = CaptureOptions {
window_s: options.seconds.map(f64::from),
trigger,
start,
samples: options.samples,
sampler: SamplerOptions {
hz: options.hz,
over_ms: options.over_ms,
wall: options.wall,
all_threads: options.threads == Threads::All,
},
block_names,
levels: Vec::new(),
wire: wire_of(self.connection.as_deref()),
gpu: options.gpu.mode(),
timeline: options.timeline.then(Default::default),
file: options.file.clone(),
};
let path = self.perf.start_capture(t, dir, capture)?;
self.window = Some(match options.seconds {
Some(s) => (t + f64::from(s), StopReason::Normal),
None => (t + MAX_WINDOW_S, StopReason::WindowCap),
});
self.running = Some((options.clone(), trigger));
Ok(path)
}
/// What the server thread needs to write its side into the running capture; None when none runs.
pub fn join(&self) -> Option<Join> {
let (sender, path, origin) = self.perf.joinable()?;
let (options, trigger) = self.running.clone()?;
let at = |t: f64| self.epoch + Duration::from_secs_f64(t.max(0.0));
Some(Join {
sender,
path,
origin: at(origin),
ends: self.window.map(|(end, reason)| (at(end), reason)),
options,
trigger,
})
}
/// Ends the running capture; its file is finished off this thread and comes back from [`ClientPerf::take_saved`].
pub fn stop(&mut self, t: f64, reason: StopReason) -> bool {
self.stop_with(t, reason, None, Vec::new())
}
/// [`ClientPerf::stop`], recording `gpu`'s pass timings if the capture asked for them.
pub fn stop_with_gpu(&mut self, t: f64, reason: StopReason, gpu: Option<GpuReadout>) -> bool {
self.stop_with(t, reason, gpu, Vec::new())
}
/// [`ClientPerf::stop_with_gpu`] with benchmark reports embedded in the capture.
pub fn stop_with(
&mut self,
t: f64,
reason: StopReason,
gpu: Option<GpuReadout>,
bench: Vec<BenchResult>,
) -> bool {
self.window = None;
self.running = None;
let end = CaptureEnd {
gpu,
bench,
wire: wire_of(self.connection.as_deref()),
..CaptureEnd::default()
};
self.perf.begin_stop(t, reason, end)
}
/// Whether a stopped capture's file is still being finished or waits to be taken.
pub fn saving(&self) -> bool {
self.perf.finishing()
}
/// The captures whose files are done; with `wait`, every one begun.
pub fn take_saved(&mut self, wait: bool) -> Vec<std::io::Result<Saved>> {
self.perf.take_saved(wait)
}
pub fn mark(&mut self, t: f64, name: &str) -> bool {
self.perf.mark(t, name)
}
/// Records a memory snapshot in the running capture; false when none runs.
pub fn memory_snapshot(&mut self, t: f64, label: &str, rows: &[(String, f64)]) -> bool {
self.perf.memory_snapshot(t, label, rows)
}
/// Records `code` in the running capture; false when none runs.
pub fn warn(&mut self, t: f64, code: WarningCode) -> bool {
self.perf.warn(t, code)
}
pub fn status(&self, t: f64) -> Option<CaptureStatus> {
self.perf.status(t)
}
/// The seconds left in the running capture's window.
pub fn remaining_s(&self, t: f64) -> Option<f64> {
self.window.map(|(end, _)| (end - t).max(0.0))
}
}
/// A benchmark report as the capture embeds it, minus its local-only label and paths; anything but a `horizon-benchmark-report` is refused.
pub fn bench_result(report_json: String) -> Result<BenchResult, String> {
let mut report = horizon_bench::Report::parse(&report_json)?;
report.label = None;
report.local.clear();
Ok(BenchResult {
side: Side::Client as i32,
format: horizon_bench::FORMAT.to_owned(),
report_json: report.to_json(),
corpus_id: String::new(),
})
}
/// The median of the values, which are cleared; None when there were none.
fn median(values: &mut Vec<f32>) -> Option<f32> {
values.sort_by(f32::total_cmp);
let rank = values.len().div_ceil(2);
let m = rank.checked_sub(1).map(|i| values[i]);
values.clear();
m
}
/// `Start` for a client capture: its build and machine; the capture fills the rest.
pub fn start_message(same_process: bool, gpu: Option<&str>) -> Start {
let mut machine: Machine = server_perf::machine(Some(
server_perf::hardware().unwrap_or_else(server_perf::hardware_now),
));
if let Some(gpu) = gpu {
machine.gpu = gpu.to_owned();
}
Start {
build: Some(server_perf::build()),
machine: Some(machine),
same_process,
..Start::default()
}
}
@@ -0,0 +1,519 @@
//! `horizon-ctl profile`: the words the game receives, as typed requests, and the typed replies.
use horizon_perf_format::v1::GpuMode;
use serde::{Deserialize, Serialize};
use std::ops::RangeInclusive;
use std::path::PathBuf;
use std::str::FromStr;
/// The longest name a mark takes; it is the one free text in a capture.
pub const MARK_NAME_MAX: usize = 100;
/// The sampler's rate bounds, per second.
pub const HZ_RANGE: RangeInclusive<u32> = 10..=1000;
/// Which side `profile` addresses; no side means every side the game can reach.
#[derive(Clone, Copy, Debug, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "lowercase")]
pub enum ProfileSide {
Client,
Server,
}
impl FromStr for ProfileSide {
type Err = String;
fn from_str(s: &str) -> Result<Self, String> {
match s {
"client" => Ok(Self::Client),
"server" => Ok(Self::Server),
other => Err(format!("unknown side {other}; use client or server")),
}
}
}
#[derive(Clone, Copy, Debug, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "lowercase")]
pub enum Threads {
Main,
All,
}
impl FromStr for Threads {
type Err = String;
fn from_str(s: &str) -> Result<Self, String> {
match s {
"main" => Ok(Self::Main),
"all" => Ok(Self::All),
other => Err(format!("unknown threads {other}; use main or all")),
}
}
}
/// What `--gpu` records: nothing, the passes' times, or also their parts and counters.
#[derive(Clone, Copy, Debug, Default, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "lowercase")]
pub enum GpuSwitch {
#[default]
Off,
Passes,
Detail,
}
impl FromStr for GpuSwitch {
type Err = String;
fn from_str(s: &str) -> Result<Self, String> {
match s {
"passes" => Ok(Self::Passes),
"detail" => Ok(Self::Detail),
other => Err(format!(
"unknown --gpu value {other}; use --gpu, --gpu=passes or --gpu=detail"
)),
}
}
}
impl GpuSwitch {
pub fn mode(self) -> GpuMode {
match self {
Self::Off => GpuMode::Off,
Self::Passes => GpuMode::Passes,
Self::Detail => GpuMode::Detail,
}
}
}
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
enum Action {
Start,
Stop,
Status,
Mark,
Open,
Share,
}
impl FromStr for Action {
type Err = String;
fn from_str(s: &str) -> Result<Self, String> {
match s {
"start" => Ok(Self::Start),
"stop" => Ok(Self::Stop),
"status" => Ok(Self::Status),
"mark" => Ok(Self::Mark),
"open" => Ok(Self::Open),
"share" => Ok(Self::Share),
other => Err(format!("unknown profile action {other}")),
}
}
}
/// The switches of `profile start`.
#[derive(Clone, Debug, PartialEq)]
pub struct StartOptions {
/// Unset runs until `stop`, or the longest window.
pub seconds: Option<u32>,
pub hz: Option<u32>,
pub over_ms: Option<f32>,
pub threads: Threads,
pub wall: bool,
pub samples: bool,
pub gpu: GpuSwitch,
/// Also record every thread's spans (`--timeline`), for 60 s at most.
pub timeline: bool,
/// Write the capture here instead of into the reports folder: `trace` and `HORIZON_TRACE` only, never a profile switch.
pub file: Option<PathBuf>,
}
impl Default for StartOptions {
fn default() -> Self {
Self {
seconds: None,
hz: None,
over_ms: None,
threads: Threads::Main,
wall: false,
samples: true,
gpu: GpuSwitch::Off,
timeline: false,
file: None,
}
}
}
#[derive(Clone, Debug, PartialEq)]
pub enum Request {
Start(StartOptions),
/// With the path of a benchmark report to embed in the capture.
Stop {
bench: Option<String>,
},
Status,
Mark(String),
/// Open the saved capture in the viewer, through the captures hook.
Open(PathBuf),
/// Upload the saved capture, through the captures hook; the reply is the link.
Share(PathBuf),
}
#[derive(Clone, Debug, PartialEq)]
pub struct Parsed {
pub side: Option<ProfileSide>,
pub request: Request,
}
const USAGE: &str = "profile [client|server] start [--seconds S] [--hz N] [--over MS] [--threads main|all] [--wall] [--gpu[=detail]] [--no-samples] [--timeline] | stop [--bench REPORT] | status | mark NAME | open FILE | share FILE";
/// The words after `profile`; the side comes first or as `--side`, and the switches follow `start`.
pub fn parse(words: &[&str]) -> Result<Parsed, String> {
let (mut side, words) = match words.first().map(|w| w.parse::<ProfileSide>()) {
Some(Ok(side)) => (Some(side), &words[1..]),
_ => (None, words),
};
let Some((action, rest)) = words.split_first() else {
return Err(format!("usage: {USAGE}"));
};
let action: Action = action.parse().map_err(|e| format!("{e}; usage: {USAGE}"))?;
if action == Action::Mark {
return mark(side, rest);
}
if matches!(action, Action::Open | Action::Share) {
let file = rest.join(" ");
if file.is_empty() {
return Err(format!("{} needs a FILE; usage: {USAGE}", words[0]));
}
let file = PathBuf::from(file);
let request = match action {
Action::Open => Request::Open(file),
_ => Request::Share(file),
};
return Ok(Parsed {
side: None,
request,
});
}
let mut options = StartOptions::default();
let mut switched = false;
let mut bench = None;
let mut words = rest.iter().copied().peekable();
while let Some(word) = words.next() {
let mut value = || {
words
.next()
.ok_or_else(|| format!("{word} needs a value; usage: {USAGE}"))
};
match word {
"--side" => side = Some(value()?.parse()?),
"--seconds" => options.seconds = Some(number(word, value()?, 1..=600)?),
"--hz" => options.hz = Some(number(word, value()?, HZ_RANGE)?),
"--over" => {
let ms: f32 = value()?
.parse()
.map_err(|_| "--over must be a number of milliseconds".to_owned())?;
if !ms.is_finite() || ms <= 0.0 {
return Err("--over must be more than 0".into());
}
options.over_ms = Some(ms);
}
"--threads" => options.threads = value()?.parse()?,
"--wall" => options.wall = true,
"--gpu" => {
options.gpu = match words.next_if(|w| w.parse::<GpuSwitch>().is_ok()) {
Some(kind) => kind.parse()?,
None => GpuSwitch::Passes,
}
}
_ if word.starts_with("--gpu=") => options.gpu = word["--gpu=".len()..].parse()?,
"--no-samples" => options.samples = false,
"--timeline" => options.timeline = true,
"--bench" => {
// The line arrives split on whitespace, so a run of spaces or a tab in the path becomes one space.
let path = words.by_ref().collect::<Vec<_>>().join(" ");
if path.is_empty() {
return Err(format!("--bench needs a report path; usage: {USAGE}"));
}
bench = Some(path);
continue;
}
other => return Err(format!("unexpected {other}; usage: {USAGE}")),
}
switched |= word != "--side";
}
let request = match action {
Action::Start if bench.is_some() => {
return Err(format!("only stop takes --bench; usage: {USAGE}"));
}
Action::Start => Request::Start(options),
_ if switched => return Err(format!("only start takes options; usage: {USAGE}")),
Action::Stop => Request::Stop { bench },
_ if bench.is_some() => return Err(format!("only stop takes --bench; usage: {USAGE}")),
_ => Request::Status,
};
Ok(Parsed { side, request })
}
fn mark(mut side: Option<ProfileSide>, mut words: &[&str]) -> Result<Parsed, String> {
if let [flag @ "--side", value, rest @ ..] = words {
let _ = flag;
side = Some(value.parse()?);
words = rest;
}
let name = words.join(" ");
if name.is_empty() {
return Err("mark needs a NAME".into());
}
if name.chars().count() > MARK_NAME_MAX || name.chars().any(char::is_control) {
return Err(format!(
"a mark name is at most {MARK_NAME_MAX} characters without control characters"
));
}
Ok(Parsed {
side,
request: Request::Mark(name),
})
}
fn number(flag: &str, text: &str, range: RangeInclusive<u32>) -> Result<u32, String> {
match text.parse::<u32>() {
Ok(n) if range.contains(&n) => Ok(n),
_ => Err(format!(
"{flag} must be a whole number from {} to {}",
range.start(),
range.end()
)),
}
}
/// The reply to `profile start`.
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)]
pub struct Started {
pub ok: bool,
/// The sides asked for, server first; the server thread's join is confirmed by `status` and `stop`.
pub sides: Vec<ProfileSide>,
pub path: String,
pub seconds: Option<u32>,
pub samples: bool,
pub gpu: GpuSwitch,
}
/// The reply to `profile stop`, once the file is whole.
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)]
pub struct Stopped {
pub ok: bool,
pub sides: Vec<ProfileSide>,
pub path: String,
/// The viewer page, when the build has a part that opens captures.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub open_at: Option<String>,
pub removed: Vec<String>,
}
/// The viewer page a registered captures hook offers.
pub fn viewer() -> Option<String> {
horizon_hooks::engine::captures().map(|c| c.viewer().to_owned())
}
/// The reply to `profile status`.
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)]
pub struct Status {
pub ok: bool,
pub capturing: bool,
pub sides: Vec<ProfileSide>,
pub path: Option<String>,
pub elapsed_s: Option<f64>,
pub bytes: Option<u64>,
}
/// The reply of a deferred `profile stop` that failed.
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)]
pub struct Failed {
pub error: String,
}
/// The reply to `trace`: whether a timeline capture runs, and where it writes.
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)]
pub struct Traced {
pub ok: bool,
pub tracing: bool,
pub file: Option<String>,
}
/// The reply to `profile open`: the viewer the capture was opened in.
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)]
pub struct Opened {
pub ok: bool,
pub viewer: String,
}
/// The reply to `profile share`: the capture's share link.
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)]
pub struct Shared {
pub ok: bool,
pub url: String,
}
/// The reply to `profile mark`.
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)]
pub struct Marked {
pub ok: bool,
pub name: String,
/// Seconds on the game clock, which counts from when the game began, for every side.
pub t: f64,
}
#[cfg(test)]
mod tests {
use super::*;
fn p(line: &str) -> Result<Parsed, String> {
parse(&line.split_whitespace().collect::<Vec<_>>())
}
#[test]
fn open_and_share_take_a_file_with_spaces_kept() {
assert_eq!(
p("open reports/a.hzperf").unwrap().request,
Request::Open(PathBuf::from("reports/a.hzperf"))
);
assert_eq!(
p("share /tmp/my capture.hzperf").unwrap().request,
Request::Share(PathBuf::from("/tmp/my capture.hzperf"))
);
assert!(p("open").is_err());
assert!(p("share").is_err());
}
#[test]
fn the_side_comes_first_or_as_a_flag() {
let a = p("client start --seconds 30").unwrap();
let b = p("start --side client --seconds 30").unwrap();
assert_eq!(a, b);
assert_eq!(a.side, Some(ProfileSide::Client));
assert_eq!(p("start").unwrap().side, None);
assert_eq!(p("server stop").unwrap().side, Some(ProfileSide::Server));
assert_eq!(
p("stop --side client").unwrap().side,
Some(ProfileSide::Client)
);
}
#[test]
fn every_start_switch_lands_in_its_field() {
let Request::Start(o) = p(
"start --hz 500 --over 12.5 --threads all --wall --gpu=detail --no-samples --timeline --seconds 9",
)
.unwrap()
.request
else {
panic!("not a start");
};
assert_eq!(
o,
StartOptions {
seconds: Some(9),
hz: Some(500),
over_ms: Some(12.5),
threads: Threads::All,
wall: true,
samples: false,
gpu: GpuSwitch::Detail,
timeline: true,
file: None,
}
);
}
#[test]
fn a_mark_takes_the_rest_of_the_line_as_its_name() {
assert_eq!(
p("mark loaded the nether").unwrap().request,
Request::Mark("loaded the nether".into())
);
assert_eq!(
p("client mark --wall").unwrap(),
Parsed {
side: Some(ProfileSide::Client),
request: Request::Mark("--wall".into())
}
);
assert_eq!(
p("mark --side client the nether").unwrap(),
Parsed {
side: Some(ProfileSide::Client),
request: Request::Mark("the nether".into())
}
);
assert!(p("mark").is_err());
assert!(p("mark --side client").is_err());
assert!(p(&format!("mark {}", "x".repeat(MARK_NAME_MAX + 1))).is_err());
assert!(p(&format!("mark {}", "x".repeat(MARK_NAME_MAX))).is_ok());
}
#[test]
fn stop_takes_the_rest_of_the_line_as_a_report_path() {
assert_eq!(p("stop").unwrap().request, Request::Stop { bench: None });
let with = p("client stop --bench /home/a b/games/benchmarks/game-1.json").unwrap();
assert_eq!(with.side, Some(ProfileSide::Client));
assert_eq!(
with.request,
Request::Stop {
bench: Some("/home/a b/games/benchmarks/game-1.json".into())
}
);
for line in [
"stop --bench",
"start --bench x.json",
"status --bench x.json",
] {
assert!(p(line).is_err(), "{line:?} was accepted");
}
}
#[test]
fn bad_words_are_refused_with_the_usage() {
for line in [
"",
"client",
"begin",
"start --seconds 0",
"start --seconds 601",
"start --seconds",
"start --hz 5",
"start --over 0",
"start --over nan",
"start --threads some",
"start --gpu=all",
"start --gpu=",
"stop now",
"stop --wall",
"status --hz 100",
"start --side everyone",
] {
assert!(p(line).is_err(), "{line:?} was accepted");
}
assert!(p("start --gpu=all").unwrap_err().contains("--gpu=detail"));
assert!(
p("start --gpu fast")
.unwrap_err()
.contains("usage: profile")
);
}
#[test]
fn gpu_alone_records_the_passes_and_detail_asks_for_more() {
let gpu = |line: &str| match p(line).unwrap().request {
Request::Start(o) => o.gpu,
other => panic!("{other:?}"),
};
assert_eq!(gpu("start"), GpuSwitch::Off);
assert_eq!(gpu("start --gpu"), GpuSwitch::Passes);
assert_eq!(
gpu("client start --seconds 5 --gpu=detail"),
GpuSwitch::Detail
);
assert_eq!(gpu("start --gpu detail --wall"), GpuSwitch::Detail);
assert_eq!(gpu("start --gpu --wall"), GpuSwitch::Passes);
assert!(p("stop --gpu").is_err(), "only start takes options");
}
}
File diff suppressed because it is too large. Load diff
+164 -9
View File
@@ -10,6 +10,7 @@
//! writes `{"transport":"tcp","port":N,"token":"..."}` to the path, readable
//! only by its user; a client's first line must then be `auth TOKEN`.
use crate::block_entity_render::StaticCacheFrameStats;
use horizon_threads::alloc_count::{Counts, ON};
use std::{
io::{self, BufRead, BufReader, Read, Write},
net::{Ipv4Addr, TcpListener, TcpStream},
@@ -20,6 +21,7 @@ use std::{
mpsc,
},
thread,
time::Instant,
};
#[cfg(unix)]
@@ -497,6 +499,7 @@ mod tests {
}
use super::*;
use std::time::Duration;
/// The check agrees with the system at every length around its limit.
#[cfg(unix)]
@@ -602,6 +605,22 @@ mod tests {
assert!(same_token("abc", "abc") && !same_token("abc", "abd") && !same_token("a", "ab"));
}
#[test]
fn the_series_cover_every_frame_the_summary_counts() {
let mut stats = FrameStats::default();
for ms in [4.0, 5.0, 6.0] {
stats.record(ms);
stats.record_cpu(ms / 2.0);
}
let mut summary = stats.summary();
assert!(summary.get("frame_ms").is_none());
stats.add_series(&mut summary);
assert_eq!(summary["frame_ms"], serde_json::json!([4.0, 5.0, 6.0]));
assert_eq!(summary["cpu_ms"], serde_json::json!([2.0, 2.5, 3.0]));
assert_eq!(summary["gpu_ms"], serde_json::json!([]));
assert_eq!(summary["frames"], 3);
}
#[test]
fn frame_summary_reports_block_entity_cache_work() {
let mut stats = FrameStats::default();
@@ -625,6 +644,67 @@ mod tests {
assert_eq!(summary["static_entities_drawn_avg_per_frame"], 4.0);
assert_eq!(summary["dynamic_entities_drawn_avg_per_frame"], 3.0);
}
fn counts(allocs: u64, bytes: u64, frees: u64, freed: u64) -> Counts {
Counts {
allocs,
bytes,
frees,
freed,
}
}
#[test]
fn allocations_are_summarized_per_frame_and_per_second() {
let mut stats = FrameStats::default();
let start = Instant::now();
for (i, a) in [100u64, 200, 300, 400, 1000].into_iter().enumerate() {
// Window: 10 blocks kept; process: twice the blocks, 1 MiB per 100, 20 blocks and 1 MiB kept.
let window = counts(a, a * 1024, a - 10, 0);
let all = counts(
2 * a,
(2 * a / 100) << 20,
2 * a - 20,
((2 * a / 100) - 1) << 20,
);
stats.record_allocs(window, all, start + Duration::from_millis(100 * i as u64));
}
let alloc = stats.alloc_summary();
let window = &alloc["window_thread"];
assert_eq!(window["allocs_per_frame"]["avg"], 400.0);
assert_eq!(window["allocs_per_frame"]["p50"], 300.0);
assert_eq!(window["allocs_per_frame"]["p95"], 1000.0);
assert_eq!(window["allocs_per_frame"]["max"], 1000.0);
assert_eq!(window["kib_per_frame"]["p50"], 300.0);
assert_eq!(window["frees_per_frame_avg"], 390.0);
// The first frame only starts the clock: the rates cover the four after it, 0.4 s.
let process = &alloc["process"];
assert_eq!(process["allocs_per_s"], 9500.0);
assert_eq!(process["frees_per_s"], 9300.0);
assert_eq!(process["mib_per_s"], 95.0);
assert_eq!(process["net_mib"], 4.0);
assert_eq!(stats.alloc_counts(), [100, 200, 300, 400, 1000]);
}
#[test]
fn one_frame_has_counts_but_no_rate() {
let mut stats = FrameStats::default();
stats.record_allocs(counts(5, 64, 5, 64), counts(9, 99, 9, 99), Instant::now());
let alloc = stats.alloc_summary();
assert_eq!(alloc["window_thread"]["allocs_per_frame"]["max"], 5.0);
assert!(alloc["process"]["allocs_per_s"].is_null());
assert!(FrameStats::default().alloc_summary().is_null());
}
#[test]
fn the_summary_has_alloc_only_when_the_game_counts() {
let mut stats = FrameStats::default();
stats.record(10.0);
stats.record_allocs(counts(5, 64, 5, 64), Counts::default(), Instant::now());
assert_eq!(stats.summary()["alloc"].is_null(), !ON);
stats.reset();
assert!(stats.alloc_counts().is_empty() && stats.alloc_summary().is_null());
}
}
/// Frame durations since the last reset, summarized like the Java recorder,
@@ -641,6 +721,15 @@ pub struct FrameStats {
static_bytes_uploaded: u64,
static_entities_drawn: u64,
dynamic_entities_drawn: u64,
/// Window-thread allocations and bytes of each rendered frame (`alloc-count` builds).
allocs: Vec<u32>,
alloc_bytes: Vec<u64>,
/// Window-thread frees summed over the frames in `allocs`.
alloc_frees: u64,
/// When the first and the last frame since the reset were recorded.
alloc_span: Option<(Instant, Instant)>,
/// All threads' counts over the frames after the first, the time base of the rates.
alloc_process: Counts,
}
/// The parts of a frame's CPU time: world streaming and uploads, game ticks,
@@ -717,17 +806,77 @@ impl FrameStats {
self.static_entities_drawn += stats.static_entities_drawn;
self.dynamic_entities_drawn += stats.dynamic_entities_drawn;
}
/// Every recorded frame time, in milliseconds, in order.
pub fn frame_millis(&self) -> &[f32] {
&self.millis
/// Adds every frame's time (`frame_ms`, `cpu_ms`, `gpu_ms`) to a summary.
pub fn add_series(&self, summary: &mut serde_json::Value) {
summary["frame_ms"] = self.millis.clone().into();
summary["cpu_ms"] = self.cpu.clone().into();
summary["gpu_ms"] = self.gpu.clone().into();
}
/// CPU time of every recorded frame, in milliseconds.
pub fn cpu_millis(&self) -> &[f32] {
&self.cpu
/// One frame's allocations: this thread's since the last frame, all threads' since then.
pub fn record_allocs(&mut self, window: Counts, process: Counts, now: Instant) {
if self.allocs.len() < Self::LIMIT {
self.allocs
.push(u32::try_from(window.allocs).unwrap_or(u32::MAX));
self.alloc_bytes.push(window.bytes);
self.alloc_frees += window.frees;
}
match &mut self.alloc_span {
None => self.alloc_span = Some((now, now)),
Some((_, last)) => {
*last = now;
let sum = &mut self.alloc_process;
sum.allocs += process.allocs;
sum.bytes += process.bytes;
sum.frees += process.frees;
sum.freed += process.freed;
}
}
}
/// GPU time of every frame the GPU timed, in milliseconds.
pub fn gpu_millis(&self) -> &[f32] {
&self.gpu
/// Window-thread allocations of every recorded frame, in order.
pub fn alloc_counts(&self) -> &[u32] {
&self.allocs
}
/// Window-thread bytes allocated in every recorded frame, in order.
pub fn alloc_bytes(&self) -> &[u64] {
&self.alloc_bytes
}
/// The `alloc` object of [`summary`](Self::summary), `null` before a frame was recorded.
fn alloc_summary(&self) -> serde_json::Value {
let n = self.allocs.len();
if n == 0 {
return serde_json::Value::Null;
}
let round = |v: f64| (v * 10.0).round() / 10.0;
let stats = |mut values: Vec<f64>| {
values.sort_by(f64::total_cmp);
let pick = |p: f64| values[((n as f64 * p).ceil() as usize).clamp(1, n) - 1];
serde_json::json!({
"avg": round(values.iter().sum::<f64>() / n as f64),
"p50": round(pick(0.5)),
"p95": round(pick(0.95)),
"max": round(values[n - 1]),
})
};
let per_second =
|amount: u64, seconds: f64| (seconds > 0.0).then(|| round(amount as f64 / seconds));
let seconds = self.alloc_span.map_or(0.0, |(first, last)| {
last.duration_since(first).as_secs_f64()
});
let mib = |bytes: u64| bytes as f64 / 1048576.0;
let process = &self.alloc_process;
serde_json::json!({
"window_thread": {
"allocs_per_frame": stats(self.allocs.iter().map(|&a| f64::from(a)).collect()),
"kib_per_frame": stats(self.alloc_bytes.iter().map(|&b| b as f64 / 1024.0).collect()),
"frees_per_frame_avg": round(self.alloc_frees as f64 / n as f64),
},
"process": {
"allocs_per_s": per_second(process.allocs, seconds),
"mib_per_s": (seconds > 0.0).then(|| round(mib(process.bytes) / seconds)),
"frees_per_s": per_second(process.frees, seconds),
"net_mib": round(mib(process.bytes) - mib(process.freed)),
},
})
}
pub fn reset(&mut self) {
self.millis.clear();
@@ -739,6 +888,11 @@ impl FrameStats {
self.static_bytes_uploaded = 0;
self.static_entities_drawn = 0;
self.dynamic_entities_drawn = 0;
self.allocs.clear();
self.alloc_bytes.clear();
self.alloc_frees = 0;
self.alloc_span = None;
self.alloc_process = Counts::default();
}
pub fn summary(&self) -> serde_json::Value {
let mut sorted = self.millis.clone();
@@ -789,6 +943,7 @@ impl FrameStats {
"dynamic_entities_drawn": self.dynamic_entities_drawn,
"static_entities_drawn_avg_per_frame": self.static_entities_drawn as f64 / n as f64,
"dynamic_entities_drawn_avg_per_frame": self.dynamic_entities_drawn as f64 / n as f64,
"alloc": if ON { self.alloc_summary() } else { serde_json::Value::Null },
})
}
}
+154
View File
@@ -8,6 +8,7 @@ use crate::{
java_text,
text::Text,
};
use horizon_client_render::gfx::pack_timing::{Counters, DetailSupport, PartsMode, PassCounters};
use std::time::{Duration, Instant};
/// `extractLines`'s background: ARGB 0x90505050.
@@ -148,6 +149,76 @@ pub fn labelled_pass_lines(label: &str, passes: &[(String, f32)]) -> Vec<String>
lines
}
/// `n` in three digits and an SI suffix: `512`, `83.0k`, `6.30M`, `1.20G`.
pub fn compact_count(n: u64) -> String {
let (mut value, mut unit) = (n as f64, 0);
while value >= 999.5 && unit < 4 {
value /= 1000.0;
unit += 1;
}
let suffix = ["", "k", "M", "G", "T"][unit];
match value {
_ if unit == 0 => n.to_string(),
v if v < 9.995 => format!("{v:.2}{suffix}"),
v if v < 99.95 => format!("{v:.1}{suffix}"),
v => format!("{v:.0}{suffix}"),
}
}
/// F3's detail lines: each pass's parts, then the six passes with the most fragments and their segments.
pub fn pass_detail_lines(
parts: &[(String, Vec<(String, f32)>)],
counters: &[(String, PassCounters)],
support: DetailSupport,
) -> Vec<String> {
const SHOWN: usize = 6;
let tr = |key: &str, args: &[&dyn std::fmt::Display]| Text::tr(key, args).plain();
let mode = match support.parts {
PartsMode::InsidePasses => "horizon.debug.pass_detail.inside_passes",
PartsMode::SeparatePasses => "horizon.debug.pass_detail.separate_passes",
};
let mut lines = vec![tr("horizon.debug.pass_detail.title", &[&tr(mode, &[])])];
for (pass, parts) in parts {
for (part, ms) in parts {
let ms = format!("{ms:.2}");
lines.push(tr("horizon.debug.pass_detail.part", &[pass, part, &ms]));
}
}
if !support.counters {
lines.push(tr("horizon.debug.pass_detail.no_counters", &[]));
return lines;
}
let per_pixel = |fragments: u64, pixels: u64| match support.fragment_unit {
Some(unit) if pixels > 0 => {
format!("{:.2}", unit.pixels_of(fragments) as f64 / pixels as f64)
}
_ => "?".to_owned(),
};
let line = |name: &str, c: &Counters, pixels: u64| {
let (fragments, vertices) = (c.fragment_shader_invocations, c.vertex_shader_invocations);
tr(
"horizon.debug.pass_detail.counters",
&[
&name,
&compact_count(fragments),
&per_pixel(fragments, pixels),
&compact_count(vertices),
&compact_count(c.clipper_primitives_out),
&compact_count(c.clipper_invocations),
],
)
};
let mut busiest: Vec<&(String, PassCounters)> = counters.iter().collect();
busiest.sort_by_key(|c| std::cmp::Reverse(c.1.total.fragment_shader_invocations));
for (pass, c) in busiest.into_iter().take(SHOWN) {
lines.push(line(pass, &c.total, c.pixels));
for (part, counted) in &c.parts {
lines.push(format!(" {}", line(part, counted, c.pixels)));
}
}
lines
}
/// `Mth.wrapDegrees`.
fn wrap_degrees(angle: f32) -> f32 {
let wrapped = angle % 360.0;
@@ -547,6 +618,7 @@ pub fn memory_lines(m: ProcessMemory, rate: i64) -> [String; 3] {
#[cfg(test)]
mod tests {
use super::*;
use horizon_client_render::gfx::pack_timing;
fn lines(v: &[&str]) -> Vec<String> {
v.iter().map(|s| (*s).to_owned()).collect()
@@ -584,6 +656,88 @@ mod tests {
assert_eq!(cache.fresh(start, layout).map(<[_]>::len), Some(0));
}
#[test]
fn counts_read_in_three_digits_with_an_si_suffix() {
for (n, want) in [
(0, "0"),
(999, "999"),
(1000, "1.00k"),
(83_049, "83.0k"),
(999_999, "1.00M"),
(6_300_000, "6.30M"),
(1_200_000_000, "1.20G"),
] {
assert_eq!(compact_count(n), want, "{n}");
}
}
fn counted(fragments: u64, parts: &[(&str, u64)]) -> PassCounters {
let c = |fragment_shader_invocations| Counters {
vertex_shader_invocations: 1000,
clipper_invocations: 400,
clipper_primitives_out: 300,
fragment_shader_invocations,
};
PassCounters {
total: c(fragments),
pixels: 1000,
parts: parts.iter().map(|p| (p.0.to_owned(), c(p.1))).collect(),
}
}
#[test]
fn pass_detail_lines_list_parts_then_the_six_busiest_passes_with_segments() {
let support = DetailSupport {
parts: PartsMode::InsidePasses,
encoder_stamps: true,
counters: true,
fragment_unit: Some(pack_timing::FragmentUnit::Quad),
};
let parts = vec![("GPU culling".to_owned(), vec![("cull".to_owned(), 0.3141)])];
let mut counters: Vec<(String, PassCounters)> = (1..=7)
.map(|i| (format!("P{i}"), counted(i * 100, &[])))
.collect();
counters[2].1 = counted(1_000_000, &[("a", 600_000), ("b", 400_000)]);
let (lines, _) = crate::lang::scoped(crate::lang::Language::load(&[], "en_us"), || {
pass_detail_lines(&parts, &counters, support)
});
assert_eq!(
lines,
[
"Horizon pass detail (timestamps inside passes)",
"GPU culling › cull: 0.31 ms",
"P3: 1.00M fragments, 4000.00 per pixel, 1.00k vertices, 300 of 400 triangles kept",
" a: 600k fragments, 2400.00 per pixel, 1.00k vertices, 300 of 400 triangles kept",
" b: 400k fragments, 1600.00 per pixel, 1.00k vertices, 300 of 400 triangles kept",
"P7: 700 fragments, 2.80 per pixel, 1.00k vertices, 300 of 400 triangles kept",
"P6: 600 fragments, 2.40 per pixel, 1.00k vertices, 300 of 400 triangles kept",
"P5: 500 fragments, 2.00 per pixel, 1.00k vertices, 300 of 400 triangles kept",
"P4: 400 fragments, 1.60 per pixel, 1.00k vertices, 300 of 400 triangles kept",
"P2: 200 fragments, 0.80 per pixel, 1.00k vertices, 300 of 400 triangles kept",
]
);
}
#[test]
fn pass_detail_lines_say_so_when_the_device_has_no_counters() {
let support = DetailSupport {
parts: PartsMode::SeparatePasses,
encoder_stamps: false,
counters: false,
fragment_unit: None,
};
let (lines, _) = crate::lang::scoped(crate::lang::Language::load(&[], "en_us"), || {
pass_detail_lines(&[], &[], support)
});
assert_eq!(
lines,
[
"Horizon pass detail (separate passes, adds GPU time)",
"GPU counters: Vulkan only"
]
);
}
#[test]
fn looking_at_lines_match_vanilla() {
let props = [
-1
View File
@@ -33,7 +33,6 @@ pub fn main_with(mods: crate::mod_api::Mods) {
// The text crates ship (`horizon_hooks::lang`), loaded with the game's own languages.
horizon_client::lang::install_crate_files(horizon_hooks::lang::engine::all());
horizon_client::spans::startup::init(std::time::Instant::now());
horizon_client::spans::trace::start_from_env();
let args: Vec<String> = std::env::args().skip(1).collect();
let game_args = args.iter().take_while(|a| *a != "--dedicated");
let help = game_args
+46 -49
View File
@@ -219,6 +219,7 @@ mod lan_far;
mod lan_idle;
mod lan_mobs;
mod lan_mount_menu;
mod lan_op;
mod lan_particles;
mod lan_pvp;
mod lan_quiet;
@@ -362,6 +363,8 @@ mod paintings;
mod pandas;
mod particle_culling;
mod passengers;
mod perf_link;
mod perf_remote;
mod persist;
mod perspective;
mod physical_trigger;
@@ -384,6 +387,8 @@ mod potent_sulfur;
mod potions;
mod powder_snow;
mod prediction;
mod profile_capture;
mod profile_lines;
mod pumpkin;
mod quick_actions;
mod raids;
@@ -2322,6 +2327,8 @@ struct Game<'a> {
debug: bool,
/// `horizon-ctl gpu-passes`: time Horizon's own passes on the GPU with F3 shut.
gpu_passes: bool,
/// `horizon-ctl gpu-passes detail`: also dispatch times and counters.
gpu_pass_detail: bool,
screen: Screen,
observed_screen: Option<native_hooks::ObservedScreen>,
window: Option<Arc<Window>>,
@@ -2359,9 +2366,6 @@ struct Game<'a> {
/// `SpectatorGui`: the menu a spectator opens with the hotbar keys.
spectator_gui: horizon_client::spectator_menu::SpectatorGui,
previous: Instant,
/// When the last frame's work ended: what runs before the next one starts (`idle`, the
/// event loop) shows in HORIZON_SPIKES as the time between frames.
frame_end: Instant,
/// FramerateLimitTracker.latestInputTime.
last_input: Instant,
/// Controllers (Horizon's own).
@@ -2805,6 +2809,11 @@ struct Game<'a> {
/// When an agent's last command came.
agent_last: Option<Instant>,
frame_stats: horizon_client::control::FrameStats,
/// This thread's and all threads' allocation counts at the last rendered frame.
alloc_mark: Option<(
horizon_threads::alloc_count::Counts,
horizon_threads::alloc_count::Counts,
)>,
/// This play session's frame and tick times, for opt-in performance statistics.
perf: horizon_client::telemetry::Session,
/// Control replies waiting for a rendered frame or a benchmark deadline.
@@ -2813,7 +2822,7 @@ struct Game<'a> {
screenshot_sequence: Option<screenshot_sequence::Sequence>,
/// `render-capture DIR`: replied once the renderer wrote the scene.
pending_render_capture: Option<std::sync::mpsc::Sender<String>>,
pending_bench: Option<(Instant, std::sync::mpsc::Sender<String>)>,
pending_bench: Option<(Instant, std::sync::mpsc::Sender<String>, bool)>,
/// Recent frame times for the performance overlay.
frame_graph: horizon_client::perf_overlay::FrameGraph,
/// Horizon's extras (minimap, auto-reconnect; mods' counters): `game/extras.rs`.
@@ -2869,6 +2878,8 @@ struct Game<'a> {
input_trace: input_trace::InputTrace,
/// A repro recording, export or replay (game/repro.rs).
repro: repro::Repro,
/// The health history and client captures (game/profile_capture.rs).
profile: profile_capture::Profile,
/// `HORIZON_TICK_SCRIPT`'s scenario (game/tick_script.rs).
tick_script: tick_script::TickScript,
/// The overworld's last `sky_darken` and its inputs (time, rain, thunder).
@@ -9762,9 +9773,14 @@ impl Game<'_> {
let debug_passes = self.debug_lists_gpu_passes();
let result = match self.renderer.as_mut() {
Some(renderer) => {
renderer.set_pass_timing(debug_passes || self.gpu_passes);
// `gpu_passes` is a measurement: it takes every sample it can.
renderer.set_pass_timing_gap(if self.gpu_passes {
use horizon_client::client_perf::request::GpuSwitch;
let capture_gpu = self.profile.gpu();
let recording = capture_gpu != GpuSwitch::Off;
renderer.set_pass_timing(debug_passes || self.gpu_passes || recording);
renderer.set_keep_timed_frames(recording);
renderer.set_pass_detail(self.gpu_pass_detail || capture_gpu == GpuSwitch::Detail);
// `gpu_passes` and a capture's `--gpu` are measurements: they take every sample they can.
renderer.set_pass_timing_gap(if self.gpu_passes || recording {
Duration::ZERO
} else {
horizon_client::debug_overlay::GPU_TIMING_GAP
@@ -11047,12 +11063,8 @@ impl Game<'_> {
self.observe_fullscreen();
self.sync_ime();
let now = Instant::now();
let frame_cpu = self
.options
.spikes
.and_then(|_| horizon_client::spans::thread_cpu());
horizon_threads::work::tick_start();
let elapsed = now.duration_since(self.previous);
let between = now.saturating_duration_since(self.frame_end);
self.frame_ticks = elapsed.as_secs_f32() * 20.0;
self.previous = now;
self.studio_fly(elapsed);
@@ -11140,46 +11152,19 @@ impl Game<'_> {
result => result?,
};
horizon_client::spans::time("pr_cursor", || self.apply_cursor_icon());
// The frame's spans, taken every frame they are recorded so they never
// pile up: the slow-frame line's, and the session's (`perf.frame`).
// The frame's spans, taken every frame they are recorded so they never pile up.
let frame_spans = if horizon_client::spans::recording_here() {
horizon_client::spans::take_raw()
} else {
Vec::new()
};
if horizon_client::spans::trace::due() {
horizon_client::spans::trace::stop();
}
if let Some(threshold) = self.options.spikes
&& let Some(r) = self.renderer.as_ref()
{
let cpu = now.elapsed().saturating_sub(r.last_wait).as_secs_f32() * 1000.0;
// Spans timed between frames (`idle`) come first.
let spans = horizon_client::spans::merge(&frame_spans);
let between = between.as_secs_f32() * 1000.0;
if cpu > threshold || between > threshold {
// The render thread's own CPU time in the frame:
// far below the wall time, the frame waited.
let worked = horizon_client::spans::thread_cpu()
.zip(frame_cpu)
.map_or_else(String::new, |(end, start)| {
format!(", cpu {:.1} ms", (end - start).as_secs_f32() * 1000.0)
});
let gap = if between > threshold {
format!(", {between:.1} ms before it")
} else {
String::new()
};
log::info!(
"Horizon: slow frame {cpu:.1} ms{worked}{gap} ({steps} ticks): {}",
horizon_client::spans::describe(&spans)
);
}
}
let mut perf_times = None;
if let Some(r) = self.renderer.as_mut() {
let cpu = now.elapsed().saturating_sub(r.last_wait);
horizon_threads::work::tick_end(cpu.as_nanos() as u64);
if rendered {
let ms = |d: Duration| d.as_secs_f32() * 1000.0;
perf_times = Some((ms(cpu), r.gpu_times.last().copied()));
self.frame_graph.record(
ms(elapsed),
ms(cpu),
@@ -11259,6 +11244,17 @@ impl Game<'_> {
}
if rendered {
self.frame_stats.record(elapsed.as_secs_f32() * 1000.0);
if let Some((cpu, gpu)) = perf_times {
self.record_perf_frame(
elapsed.as_secs_f32() * 1000.0,
cpu,
gpu,
&frame_spans,
);
}
if horizon_threads::alloc_count::ON {
self.record_frame_allocs();
}
self.debug_charts.frame_times.log(elapsed.as_nanos() as i64);
if self.frames == 0 {
// The launcher times Play to this line (launcher/src/perf.rs).
@@ -11299,7 +11295,6 @@ impl Game<'_> {
self.fps_frames = 0;
self.fps_at = Instant::now();
}
self.frame_end = Instant::now();
if self.options.frames.is_some_and(|f| self.frames >= f)
|| self
.options
@@ -11383,6 +11378,9 @@ impl<'a> Game<'a> {
let inventory = vec![None; 36];
let now = Instant::now();
let look = options.look;
let slow_frame_ms = options
.spikes
.unwrap_or(horizon_client::client_perf::SLOW_FRAME_MS);
let particles = horizon_client::particles::ParticleEngine::new(
&root,
&gui.atlas.sprites,
@@ -11499,6 +11497,7 @@ impl<'a> Game<'a> {
pause: false,
debug: false,
gpu_passes: false,
gpu_pass_detail: false,
screen: Screen::Playing,
observed_screen: None,
window: None,
@@ -11521,7 +11520,6 @@ impl<'a> Game<'a> {
started: now,
spectator_gui: Default::default(),
previous: now,
frame_end: now,
last_input: now,
pad: Default::default(),
frames: 0,
@@ -11775,6 +11773,7 @@ impl<'a> Game<'a> {
agent_control_on: false,
agent_last: None,
frame_stats: Default::default(),
alloc_mark: None,
perf: Default::default(),
pending_screenshot: None,
screenshot_sequence: None,
@@ -11785,6 +11784,7 @@ impl<'a> Game<'a> {
events: Default::default(),
input_trace: Default::default(),
repro: Default::default(),
profile: profile_capture::Profile::new(slow_frame_ms),
tick_script: tick_script::TickScript::from_env(),
agent: agent::Agent::default(),
view_frustum: None,
@@ -12279,10 +12279,7 @@ pub fn run_with() -> Result<()> {
horizon_threads::catch_stop_signals();
}
let started = Instant::now();
// A dedicated server writes its slow ticks to the log with the spans
// they spent their time in (`HORIZON_SPIKES`'s line; the variable still
// sets another threshold), so a lag spike players felt can be traced
// from the log alone. Timing the spans costs microseconds a tick.
// A dedicated server times its spans always, so its slow ring holds the steps of a lag spike (microseconds a tick).
if options.spikes.is_none()
&& options.execution_role == execution_role::ExecutionRole::DedicatedServer
{
@@ -15,6 +15,28 @@ use horizon_client::{
use horizon_core::position::BlockPos;
use horizon_world::registry::BlockStateId;
/// The block entity type a sample is attributed to while its ticker runs.
fn ticker_name(entity: &BlockEntity) -> &'static str {
match entity {
BlockEntity::Chest(_) => "hopper",
BlockEntity::Crafter(_) => "crafter",
BlockEntity::Jukebox(_) => "jukebox",
BlockEntity::Beacon(_) => "beacon",
BlockEntity::CreakingHeart(_) => "creaking_heart",
BlockEntity::Spawner(_) => "mob_spawner",
BlockEntity::TrialSpawner(_) => "trial_spawner",
BlockEntity::Vault(_) => "vault",
BlockEntity::Beehive(_) => "beehive",
BlockEntity::Furnace(_) => "furnace",
BlockEntity::Brewing(_) => "brewing_stand",
BlockEntity::DaylightDetector => "daylight_detector",
BlockEntity::Campfire(_) => "campfire",
BlockEntity::Mod(_) => "mod_block_entity",
BlockEntity::PotentSulfur(_) => "potent_sulfur",
_ => "block_entity",
}
}
impl Game<'_> {
pub(super) fn tick_block_entities(&mut self) -> Result<()> {
self.tick_mount_menu();
@@ -65,7 +87,12 @@ impl Game<'_> {
{
continue;
}
match self.world.block_entities.get(&pos) {
horizon_threads::work::set_chunk(pos.0 >> 4, pos.2 >> 4);
let entity = self.world.block_entities.get(&pos);
horizon_threads::work::set_entity(horizon_threads::work::entity_id(
entity.map_or("", ticker_name),
));
match entity {
Some(BlockEntity::Chest(items)) if items.len() == HOPPER_SLOTS => {
self.tick_hopper(pos)
}
@@ -100,6 +127,7 @@ impl Game<'_> {
_ => {}
}
}
horizon_threads::work::clear_work();
// Items move between passes: the columns hold for this one only.
self.item_columns = None;
self.pass_mobs = None;
@@ -224,6 +224,7 @@ impl Game<'_> {
if !debug {
self.tick_scheduled()?;
}
horizon_threads::work::clear_work();
self.with_level(|b, level, ticks| {
// The masks the random ticks will read, built on helper threads first while the
// parallel tick is on (`HORIZON_PARALLEL_TICK`): the draws and ticks don't depend on them.
@@ -271,6 +272,7 @@ impl Game<'_> {
}
});
});
horizon_threads::work::clear_work();
laps = horizon_client::spans::laps();
let sent = self.lan_broadcast_changes();
laps.lap("tb_broadcast");
+77
View File
@@ -1765,6 +1765,82 @@ impl Game<'_> {
pub(super) fn hurt_boat(&mut self, i: usize, damage: f32, creative: bool) -> bool {
self.hurt_boat_result(i, damage, creative).unwrap_or(false)
}
/// Whether a spectator must be refused this boat's menu.
pub(super) fn boat_loot_pending(&self, id: u32) -> bool {
matches!(
self.persist.raw.get(&id),
Some(horizon_world::nbt::Tag::Compound(map)) if map.contains_key(&"LootTable".into())
)
}
/// `ContainerEntity.unpackChestVehicleLootTable`: slot access rolls once;
/// only a player opening the menu fires `CriteriaTriggers.GENERATE_LOOT`, before filling.
pub(super) fn unpack_boat_loot(&mut self, i: usize, player: bool) {
use horizon_world::nbt::Tag;
let id = self.boats[i].id;
let (table, seed) = {
let Some(Tag::Compound(map)) = self.persist.raw.get(&id) else {
return;
};
let Some(table) = crate::save::text(map.get(&"LootTable".into()))
.and_then(|id| horizon_client_core::identifier::normalize_id(&id))
else {
return;
};
let seed = match map.get(&"LootTableSeed".into()) {
Some(Tag::Long(seed)) => Some(*seed),
_ => None,
};
(table, seed)
};
if player {
self.trigger_loot_table(&table);
}
// Advancement rewards can remove the boat or shift its index.
let Some(i) = self.boats.iter().position(|b| b.id == id) else {
return;
};
if let Some(Tag::Compound(map)) = self.persist.raw.get_mut(&id) {
map.remove(&"LootTable".into());
map.remove(&"LootTableSeed".into());
}
let key = (i32::MAX - 15, super::minecarts::CART_KEY_Y, id as i32);
self.store_cart_items(key);
let loot = crate::structure_blocks::loot_data(&self.options.resources, &self.pack_files());
let b = &self.boats[i];
let at = Vec3::new(b.position.x, b.position.y, b.position.z);
let biome = self.biome_name_at(at).unwrap_or_default();
let pos = (
b.position.x.floor() as i32,
b.position.y.floor() as i32,
b.position.z.floor() as i32,
);
crate::structure_blocks::roll_container(
&mut self.boats[i].items,
loot.as_deref(),
table,
seed,
pos,
&biome,
);
if let Some(horizon_client::containers::BlockEntity::Chest(items)) =
self.world.block_entities.get_mut(&key)
{
items.clone_from(&self.boats[i].items);
}
}
/// `AbstractChestBoat.remove`: destroying removals spill the live inventory.
pub(super) fn kill_boat(&mut self, id: u32) {
if let Some(i) = self.boats.iter().position(|b| b.id == id) {
self.release_vehicle_container(id);
self.unpack_boat_loot(i, false);
let b = self.boats.remove(i);
self.drop_vehicle_contents(b.position.components(), &b.items);
self.boat_removed(&b);
}
}
/// None means a canceled hit; Some records whether an accepted hit destroyed it.
pub(super) fn hurt_boat_result(
&mut self,
@@ -1788,6 +1864,7 @@ impl Game<'_> {
self.eject_boat_mobs(i, true);
let id = self.boats[i].id;
self.release_vehicle_container(id);
self.unpack_boat_loot(i, false);
let b = self.boats.remove(i);
// `AbstractChestBoat.remove` spills the chest through
// `Containers.dropContents` for every destroying removal reason
+1 -1
View File
@@ -202,7 +202,7 @@ impl Game<'_> {
}
// SuggestCommand's insertText does nothing on a book; a server's
// text never opens a file.
ClickEvent::SuggestCommand(_) | ClickEvent::OpenFile(_) => {}
ClickEvent::SuggestCommand(_) | ClickEvent::OpenFile(_) | ClickEvent::Profiler(_) => {}
}
}
/// `WritableBookItem.use` / `WrittenBookItem.use` (`Player.openItemGui`).
+50 -1
View File
@@ -33,6 +33,18 @@ pub(super) fn chat_tree(
None => horizon_client::command::vanilla::game_dispatcher(),
}
}
/// The arguments of a typed `/profiler` the client answers itself: only when the chat tree's `profiler` is Horizon's, else the server or mod that owns the name runs it.
fn horizon_profiler_args<'a>(
message: &'a str,
server_tree: Option<&horizon_client::command::Dispatcher>,
) -> Option<&'a str> {
let args = message
.strip_prefix('/')
.and_then(super::profile_capture::profiler_args)?;
server_tree
.is_none_or(|tree| tree.has_horizon_profiler())
.then_some(args)
}
use winit::{
event::KeyEvent,
keyboard::{KeyCode, PhysicalKey},
@@ -155,7 +167,7 @@ impl Game<'_> {
Some(tree) => tree,
None => horizon_client::command::vanilla::game_dispatcher(),
};
with_studio_commands(&with_native_commands(base, &names))
with_studio_commands(&with_native_commands(base, &names)).with_profiler()
}
}
@@ -1403,6 +1415,12 @@ impl Game<'_> {
return;
}
self.ui.chat.add_recent(&message);
if let Some(args) =
horizon_profiler_args(&message, self.remote.as_ref().map(|r| &r.chat_tree))
{
self.profiler_chat(args);
return;
}
// On a server (single player's own included) the server runs it,
// unless a client mod owns the command.
if self.remote.is_some() {
@@ -1530,6 +1548,14 @@ impl Game<'_> {
}
}
Some(ClickEvent::RunCommand(command)) => self.click_command(&command),
Some(ClickEvent::Profiler(command)) => {
if let Some(args) = command
.strip_prefix('/')
.and_then(super::profile_capture::profiler_args)
{
self.profiler_chat(args);
}
}
// Screen.defaultHandleGameClickEvent: the dialog opens over the
// screen, and closes back to it.
Some(ClickEvent::ShowDialog(dialog)) => self.show_dialog_ref(&dialog),
@@ -2100,6 +2126,29 @@ const LINK_COPY: usize = 1;
mod tests {
use super::*;
#[test]
fn a_typed_profiler_is_the_clients_unless_the_server_owns_the_name() {
let vanilla = horizon_client::command::vanilla::game_dispatcher();
let ours = vanilla.with_profiler();
assert_eq!(
super::horizon_profiler_args("/profiler share", Some(&ours)),
Some(" share")
);
assert_eq!(super::horizon_profiler_args("/profiler", None), Some(""));
assert_eq!(
super::horizon_profiler_args("/profilers", Some(&ours)),
None
);
let mut server = vanilla.clone();
let node = server.literal(0, "profiler");
server.executes(node, "server:profiler");
let theirs = server.with_profiler();
assert_eq!(
super::horizon_profiler_args("/profiler share", Some(&theirs)),
None
);
}
/// A mod's text reaches the chat box as typing would leave it: emoji and
/// accents stay, control characters, DEL and the section sign go.
#[test]
+2 -12
View File
@@ -310,7 +310,7 @@ impl Game<'_> {
self.command_source = outer;
ok
}
fn syntax_failure(&mut self, e: &CommandError, command: &str) {
pub(in crate::game) fn syntax_failure(&mut self, e: &CommandError, command: &str) {
self.command_failure(e.message.clone());
let Some(input) = &e.input else { return };
let cursor = e.cursor.min(input.len());
@@ -2487,17 +2487,7 @@ impl Game<'_> {
self.item_frames.retain(|f| f.id != id);
}
Target::Painting(id) => self.paintings.retain(|p| p.id != id),
Target::Boat(id) => {
// AbstractBoat.remove: a killed boat drops its lead.
if let Some(i) = self.boats.iter().position(|b| b.id == id) {
self.release_vehicle_container(id);
let b = self.boats.remove(i);
// AbstractChestBoat.remove, after the live menu writes
// back its slots.
self.drop_vehicle_contents(b.position.components(), &b.items);
self.boat_removed(&b);
}
}
Target::Boat(id) => self.kill_boat(id),
Target::Cart(id) => {
self.release_vehicle_container(id);
if let Some(i) = self.carts.iter().position(|c| c.id == id) {
+167 -39
View File
@@ -1,6 +1,7 @@
//! Commands from the control socket (`horizon-ctl`), run on the game thread.
use super::event_loop::EventLoopCtl;
use super::{Game, Screen};
use horizon_client::gfx::pack_timing;
use horizon_client::held::Source;
use horizon_client::{Result, items::ItemStack};
use horizon_core::geometry::Vec3;
@@ -59,8 +60,9 @@ const HELP: &[&str] = &[
"chat-log [N] the newest N chat messages (default 10)",
"options [PAGE] open the options (video sound controls keybinds mouse chat accessibility language font packs mods graphics graphics-advanced graphics-hdr video-vanilla)",
"stats [reset] [raw] frame-time summary since the last reset (raw: every frame time too)",
"profile [client|server] start|stop|status|mark NAME capture the performance of the window and, in single player, its server thread to reports/ (horizon-ctl profile --help)",
"narration the narrator's status and the last messages it was asked to say",
"bench SECONDS reset stats, wait, then reply with the summary",
"bench SECONDS [raw] reset stats, wait, then reply with the summary (raw: every frame time too)",
"fly X1 Y1 Z1 X2 Y2 Z2 SECONDS [PITCH [raw]] fly the camera along a line at constant speed, then reply with the frame summary (raw: every frame time too)",
"open WORLD [SEED] [creative] open or create a world from the title screen",
"save save the world",
@@ -125,9 +127,7 @@ impl Game<'_> {
summary["tick_ms"] = (flight.tick_ms.iter().sum::<f32>() / n).into();
summary["tick_ms_max"] = flight.tick_ms.iter().copied().fold(0.0, f32::max).into();
if flight.raw {
summary["frame_ms"] = self.frame_stats.frame_millis().into();
summary["cpu_ms"] = self.frame_stats.cpu_millis().into();
summary["gpu_ms"] = self.frame_stats.gpu_millis().into();
self.add_raw_series(&mut summary);
}
let _ = flight.reply.send(summary.to_string());
}
@@ -167,6 +167,14 @@ impl Game<'_> {
.collect(),
);
}
if let Some(r) = self.renderer.as_ref().filter(|r| r.pass_detail())
&& let Some(support) = r.pass_detail_support()
&& let Value::Object(detail) =
pack_timing::detail_json(&r.own_pass_parts(), &r.own_pass_counters(), support)
&& let Some(summary) = summary.as_object_mut()
{
summary.extend(detail);
}
self.add_async_occlusion_stats(&mut summary);
summary
}
@@ -177,6 +185,23 @@ impl Game<'_> {
}
}
/// Adds the allocations since the last rendered frame to the frame stats; the first frame has no mark.
pub(super) fn record_frame_allocs(&mut self) {
use horizon_threads::alloc_count::{here, process};
let now = (here(), process());
if let Some((window, all)) = self.alloc_mark.replace(now) {
self.frame_stats
.record_allocs(now.0.since(window), now.1.since(all), Instant::now());
}
}
/// Every frame's times, and its allocations in builds that count (`raw`).
fn add_raw_series(&self, summary: &mut Value) {
self.frame_stats.add_series(summary);
if horizon_threads::alloc_count::ON {
summary["alloc_count"] = self.frame_stats.alloc_counts().into();
summary["alloc_bytes"] = self.frame_stats.alloc_bytes().into();
}
}
fn reset_frame_stats(&mut self) {
self.frame_stats.reset();
if let Some(culler) = &self.async_occlusion_culler {
@@ -184,16 +209,21 @@ impl Game<'_> {
}
}
pub(super) fn poll_control(&mut self, event_loop: &dyn EventLoopCtl) {
if let Some((deadline, reply)) = self.pending_bench.take() {
if let Some((deadline, reply, raw)) = self.pending_bench.take() {
if Instant::now() >= deadline {
let _ = reply.send(self.summary_with_passes().to_string());
let mut summary = self.summary_with_passes();
if raw {
self.add_raw_series(&mut summary);
}
let _ = reply.send(summary.to_string());
} else {
self.pending_bench = Some((deadline, reply));
self.pending_bench = Some((deadline, reply, raw));
}
}
horizon_client::widgets::end_frame(self.frames);
self.narrate_screen();
self.poll_repro(event_loop);
self.poll_profile();
self.advance_agent();
for request in self.ui_wait.ready() {
self.answer(request, event_loop);
@@ -391,12 +421,21 @@ impl Game<'_> {
if let Some(value) = self.repro_command(&words, line, reply)? {
return Ok(value);
}
if words[0] == "profile" {
return self.profile_command(&words[1..], reply);
}
if words[0] == "trace" {
return self.trace_command(&words[1..], reply);
}
if let Some(value) = self.saves_control(&words, line, reply)? {
return Ok(value);
}
if let Some(value) = self.options_control(&words, line)? {
return Ok(Some(value));
}
if let Some(value) = self.lan_op_control(&words)? {
return Ok(Some(value));
}
if let Some(value) = self.lan_control(&words)? {
return Ok(Some(value));
}
@@ -461,35 +500,17 @@ impl Game<'_> {
Ok(Some(json!({ "ok": true, "cull": r.section_cull() })))
}
"gpu-passes" => {
match words.get(1).copied() {
None => {}
Some("on") => self.gpu_passes = true,
Some("off") => self.gpu_passes = false,
Some(other) => return Err(format!("unknown gpu pass mode {other}").into()),
}
Ok(Some(json!({ "ok": true, "gpu_passes": self.gpu_passes })))
(self.gpu_passes, self.gpu_pass_detail) = gpu_pass_switches(
words.get(1).copied(),
(self.gpu_passes, self.gpu_pass_detail),
)?;
let support = self.renderer.as_ref().and_then(|r| r.pass_detail_support());
Ok(Some(gpu_passes_reply(
self.gpu_passes,
self.gpu_pass_detail,
support,
)))
}
// Every thread's spans as a timeline Perfetto opens (`spans::trace`).
"trace" => match words.get(1).copied() {
None => Ok(Some(
json!({ "ok": true, "tracing": horizon_client::spans::trace::on() }),
)),
Some("stop") => Ok(Some(
json!({ "ok": true, "events": horizon_client::spans::trace::stop() }),
)),
Some(file) => {
let seconds = match words.get(2) {
None => None,
Some(s) => Some(
s.parse::<u32>()
.map_err(|_| "trace [FILE [SECONDS] | stop]")?,
),
};
horizon_client::spans::trace::start(std::path::Path::new(file), seconds)
.map_err(|e| format!("trace {file}: {e}"))?;
Ok(Some(json!({ "ok": true, "tracing": true, "file": file })))
}
},
"render-capture" => match words.get(1).copied() {
None => {
let r = self.renderer.as_ref().ok_or("no renderer")?;
@@ -579,7 +600,7 @@ impl Game<'_> {
"sculk" => Ok(Some(self.sculk_command(&words)?)),
"hello" => Ok(Some(hello())),
"status" => Ok(Some(self.control_status())),
"memory" => Ok(Some(self.memory_report())),
"memory" => Ok(Some(self.memory_value())),
"stall" => {
let millis: u64 = arg(1)?.parse()?;
if millis > 5000 {
@@ -1188,9 +1209,7 @@ impl Game<'_> {
"stats" => {
let mut summary = self.summary_with_passes();
if words[1..].contains(&"raw") {
summary["frame_ms"] = self.frame_stats.frame_millis().into();
summary["cpu_ms"] = self.frame_stats.cpu_millis().into();
summary["gpu_ms"] = self.frame_stats.gpu_millis().into();
self.add_raw_series(&mut summary);
}
if words[1..].contains(&"reset") {
self.reset_frame_stats();
@@ -1210,6 +1229,7 @@ impl Game<'_> {
self.pending_bench = Some((
Instant::now() + Duration::from_secs_f64(seconds),
reply.clone(),
words[2..].contains(&"raw"),
));
Ok(None)
}
@@ -1773,6 +1793,35 @@ fn key_code(name: &str) -> Option<KeyCode> {
})
}
/// The (times, detail) switches `gpu-passes [on|off|detail]` leaves, given the current ones.
fn gpu_pass_switches(word: Option<&str>, now: (bool, bool)) -> Result<(bool, bool)> {
match word {
None => Ok(now),
Some("on") => Ok((true, false)),
Some("detail") => Ok((true, true)),
Some("off") => Ok((false, false)),
Some(other) => Err(format!("unknown gpu pass mode {other}").into()),
}
}
/// The `gpu-passes` reply: the switches and what detail can show on this device.
fn gpu_passes_reply(
passes: bool,
detail: bool,
support: Option<pack_timing::DetailSupport>,
) -> Value {
let shown = support.map(pack_timing::support_json);
let field = |key: &str, none: Value| shown.as_ref().map_or(none, |s| s[key].clone());
json!({
"ok": true,
"gpu_passes": passes,
"detail": detail,
"parts_mode": field("parts_mode", Value::Null),
"counters": field("counters", json!(false)),
"fragment_unit": field("fragment_unit", Value::Null),
})
}
/// The title's own screens (not one opened over it, as Options is) take the
/// text: the world name, the seed, a server's address.
pub(super) fn title_owns_text(screen: &Screen, title_up: bool) -> bool {
@@ -1810,6 +1859,39 @@ mod tests {
assert_eq!(names.len(), features.len(), "a group is named twice");
}
#[test]
fn gpu_passes_detail_replies_detail_and_on_turns_it_off_again() {
let support = pack_timing::DetailSupport {
parts: pack_timing::PartsMode::SeparatePasses,
encoder_stamps: false,
counters: true,
fragment_unit: Some(pack_timing::FragmentUnit::Quad),
};
let (passes, detail) = gpu_pass_switches(Some("detail"), (false, false)).unwrap();
let reply = gpu_passes_reply(passes, detail, Some(support));
assert_eq!(reply["detail"], true);
assert_eq!(reply["gpu_passes"], true);
assert_eq!(reply["parts_mode"], "separate_passes");
assert_eq!(reply["counters"], true);
assert_eq!(reply["fragment_unit"], "quad");
let (passes, detail) = gpu_pass_switches(Some("on"), (passes, detail)).unwrap();
let reply = gpu_passes_reply(passes, detail, None);
assert_eq!(
(reply["gpu_passes"].as_bool(), reply["detail"].as_bool()),
(Some(true), Some(false))
);
assert_eq!(
(reply["parts_mode"].is_null(), reply["counters"].as_bool()),
(true, Some(false))
);
assert_eq!(
gpu_pass_switches(Some("off"), (true, true)).unwrap(),
(false, false)
);
assert_eq!(gpu_pass_switches(None, (true, true)).unwrap(), (true, true));
assert!(gpu_pass_switches(Some("nope"), (false, false)).is_err());
}
#[test]
fn taps_and_ups_always_release_their_key() {
for line in ["key shift tap", "key shift up", "key Shift up"] {
@@ -1825,4 +1907,50 @@ mod tests {
assert_eq!(released_modifier(line), None, "{line}");
}
}
#[test]
fn a_benchmark_profile_only_goes_with_a_benchmark() {
let parse = |args: &[&str]| {
super::super::Options::parse(args.iter().map(|a| a.to_string())).map(|o| o.unwrap())
};
assert!(
parse(&["--benchmark", "village", "--profile"])
.unwrap()
.benchmark_profile
);
assert!(
!parse(&["--benchmark", "village"])
.unwrap()
.benchmark_profile
);
assert!(parse(&["--profile"]).is_err());
}
#[test]
fn a_benchmark_set_only_goes_with_the_trip() {
let parse = |args: &[&str]| {
super::super::Options::parse(args.iter().map(|a| a.to_string())).map(|o| o.unwrap())
};
let set = parse(&["--benchmark", "travel", "--benchmark-set", "gpu-culling=on"]).unwrap();
assert_eq!(set.benchmark_set, Some(("gpu-culling".into(), "on".into())));
let before = parse(&["--benchmark-set", "gpu-culling=on", "--benchmark", "travel"]);
assert!(before.is_ok());
for args in [
&[
"--benchmark",
"standard",
"--benchmark-set",
"gpu-culling=on",
][..],
&["--benchmark-set", "gpu-culling=on"],
&[
"--benchmark",
"travel",
"--benchmark-set",
"gpu-culling=auto",
],
] {
assert!(parse(args).is_err(), "{args:?}");
}
}
}
@@ -333,9 +333,7 @@ impl<M: MobSlot> Group<M> {
};
w.controlling_player_movement = controlling_player_movement;
self.ticked.insert(self.mobs[i].mob().id);
// `horizon-report`'s cost by mob type (tick_profile.rs).
let sampled =
super::super::tick_profile::start().map(|t| (t, self.mobs[i].mob().kind.clone()));
let sampled = super::super::tick_profile::start_mob(&self.mobs[i].mob().kind);
let kept = horizon_client::mobs::riders::tick(
&mut self.mobs,
i,
@@ -343,9 +341,7 @@ impl<M: MobSlot> Group<M> {
&mut self.random,
&mut self.events,
);
if let Some((started, kind)) = sampled {
super::super::tick_profile::add(shared.level_id, &kind, started);
}
super::super::tick_profile::end_mob(shared.level_id, sampled);
let m = self.mobs[i].mob_mut();
if seen_guest.is_none() {
self.player = w.player;
@@ -174,6 +174,19 @@ impl Game<'_> {
if self.debug && !own_lines.is_empty() {
d.group("Horizon passes", own_lines);
}
let detail_lines = self.renderer.as_ref().map_or_else(Vec::new, |r| {
let support = r.pass_detail_support().filter(|_| r.pass_detail());
support.map_or_else(Vec::new, |support| {
overlay::pass_detail_lines(
&r.own_pass_parts(),
&r.own_pass_counters(),
support,
)
})
});
if self.debug && !detail_lines.is_empty() {
d.group("Horizon pass detail", detail_lines);
}
}
"tps" => self.debug_tps_line(d),
// DebugEntryChunkGeneration, where the world is generated
+132 -86
View File
@@ -21,6 +21,7 @@ use horizon_client::{
},
text::Text,
};
use horizon_perf_format::v1::{StopReason, Trigger};
use horizon_protocol::text::translatable;
use horizon_world::nbt::{JavaString, Tag};
use std::{
@@ -37,7 +38,7 @@ use std::{
pub(super) mod events;
mod management;
mod predicate_graph;
mod report;
pub(super) mod report;
mod status;
mod transfer;
@@ -658,9 +659,7 @@ const LEGACY_PRESET_NAMES: [(&str, horizon_client::save::WorldPreset); 2] = [
/// server.properties' line for `server.toml`'s `max_view_distance` (horizon-server writes it).
const FAR_PROPERTY: &str = "horizon-max-view-distance";
/// A dedicated server logs a tick slower than this, in milliseconds, with
/// its spans, unless `HORIZON_SPIKES` names another threshold: five ticks'
/// worth, a stall players feel as rubber-banding.
/// A tick slower than this, in milliseconds, enters the slow ring (five ticks' worth, a stall players feel) unless `HORIZON_SPIKES` names another.
pub(super) const SLOW_TICK_MS: f32 = 250.0;
/// `MinecraftServer.OVERLOADED_WARNING_INTERVAL_NANOS` plus 100 ticks: the
/// "Can't keep up!" warning comes at most this often.
@@ -806,9 +805,19 @@ struct ServerState {
op_level: i32,
/// `horizon-report`: the ticks being recorded.
report: Option<report::Recording>,
/// When the running capture takes its next memory snapshot, and how many it has taken.
memory_due: Option<(f64, u32)>,
/// The always-on health history and slow ring, and the `.hzperf` a report writes from them.
perf: horizon_client::server_perf::ServerPerf,
/// The perf clock's zero: the tick loop's start.
perf_epoch: Instant,
/// The Horizon switches a capture records.
horizon: horizon_perf_format::v1::HorizonOptions,
/// View and simulation distance, and the player limit, for the report.
view: (u32, u32),
max_players: usize,
/// `network-compression-threshold`, for the capture's settings.
compression: i32,
/// The mutable `gamemode` server property (`DedicatedServer.gameMode`).
game_mode: i32,
/// The mutable `force-gamemode` server property (`DedicatedServer.forceGameMode`).
@@ -1235,6 +1244,7 @@ impl Game<'_> {
}
self.sync_ops(&lists);
let anticheat = load_anticheat();
let anticheat_on = anticheat.enabled;
let chat_filter = load_chat_filter();
self.set_pvp_legacy_rules(load_combat());
// Horizon's far ring (docs/hosting.md): server.toml's `max_view_distance`.
@@ -1292,12 +1302,44 @@ impl Game<'_> {
game_mode: props.game_mode,
force_game_mode: props.force_game_mode,
report: None,
memory_due: None,
perf: horizon_client::server_perf::ServerPerf::new(
0.0,
self.options.spikes.unwrap_or(SLOW_TICK_MS),
),
perf_epoch: Instant::now(),
horizon: horizon_client::server_perf::horizon_options(
if self.lan.as_ref().is_some_and(|h| h.hub.far.is_some()) {
far as u32
} else {
0
},
horizon_client::lod_net::distance_property(
props.raw.get(horizon_client::lod_net::PROPERTY),
),
anticheat_on,
),
compression: props.network_compression_threshold,
view: (
props.view_distance.clamp(2, 32) as u32,
props.simulation_distance.clamp(2, 32) as u32,
),
max_players: props.max_players.max(0) as usize,
};
horizon_client::server_perf::warm_hardware();
if let Some(file) = std::env::var_os("HORIZON_TRACE").filter(|f| !f.is_empty()) {
self.record_report(
&mut state,
&report::Plan {
seconds: report::MAX_SECONDS,
samples: false,
timeline: true,
file: Some(file.into()),
trigger: Trigger::Env,
sampler: Default::default(),
},
);
}
let mut next = Instant::now();
// The panic that ended the tick loop, when one did.
let mut crashed: Option<String> = None;
@@ -1361,22 +1403,32 @@ impl Game<'_> {
// MinecraftServer.tickServer: while paused only the connections run.
// `MinecraftServer.runServer` catches what the tick throws, writes the crash
// report and still stops the server (saves). A panic here does the same.
let ticked =
std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| {
match &mut state.report {
Some(recording) => {
let sampled = recording.sample_next();
super::tick_profile::set_sampling(sampled);
let ticked =
self.tick_server_spans(paused, |wall, cpu, spans, recorded| {
recording.add(wall, cpu, spans, recorded, sampled)
});
super::tick_profile::set_sampling(false);
ticked
}
None => self.tick_server_timed(paused),
let slow_ms = state.perf.slow_threshold_ms();
let capturing = state.perf.capturing();
let mut tick_detail = None;
let ticked = std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| {
let mut detail = |timing: super::levels::TickTiming| {
if capturing || timing.wall_ms > slow_ms {
tick_detail = Some(horizon_client::server_perf::TickDetail {
cpu_ms: timing.cpu_ms,
spans: timing.raw,
});
}
}));
};
match &mut state.report {
Some(recording) => {
let sampled = recording.sample_next();
super::tick_profile::set_sampling(sampled);
let ticked = self.tick_server_spans(paused, |timing| {
recording.add(sampled);
detail(timing)
});
super::tick_profile::set_sampling(false);
ticked
}
None => self.tick_server_spans(paused, detail),
}
}));
match ticked {
Ok(result) => result?,
Err(payload) => {
@@ -1384,19 +1436,12 @@ impl Game<'_> {
break;
}
}
if state.report.as_ref().is_some_and(report::Recording::done)
&& let Some(recording) = state.report.take()
{
self.finish_report(recording, state.view, state.max_players);
}
if horizon_client::spans::trace::due()
&& let Some(events) = horizon_client::spans::trace::stop()
{
log(
"INFO",
&format!("Trace written: {events} events (reports/)"),
);
if state.report.as_ref().is_some_and(report::Recording::done) {
self.finish_perf_capture(&mut state, StopReason::Normal);
}
self.serve_perf_requests(&mut state);
let done = report::log_saved(&mut state.perf, false);
self.perf_captures_done(done);
// DedicatedServer.sendLowDiskSpaceWarning: the operators hear of it
// (no window drains the toasts here).
if horizon_client::chunk_io::take().low_disk_space
@@ -1412,6 +1457,14 @@ impl Game<'_> {
}
// MinecraftServer.tickServer records each tick's time, for /tick query (and the status).
self.clock.record(tick_start.elapsed());
state.perf.tick(
state.perf_epoch.elapsed().as_secs_f64(),
tick_start.elapsed().as_secs_f32() * 1000.0,
tick_detail.take(),
|| self.perf_world_counts(),
|| self.wire_bytes(),
);
self.poll_server_memory_snapshot(&mut state);
// The far ring's brake, once a second on the average tick.
if braked.elapsed() >= Duration::from_secs(1)
&& let Some(host) = self.lan.as_mut()
@@ -1437,6 +1490,9 @@ impl Game<'_> {
if let Some(watchdog) = &watchdog {
watchdog.running.store(false, Ordering::Relaxed);
}
self.finish_perf_capture(&mut state, StopReason::Shutdown);
let done = report::log_saved(&mut state.perf, true);
self.perf_captures_done(done);
match crashed {
Some(message) => self.stop_after_crash(&message),
None => self.stop_dedicated(),
@@ -1671,73 +1727,63 @@ impl Game<'_> {
state.lists.lock().unwrap().stop = true;
}
// Horizon's own, on the console only: a performance report (report.rs).
"horizon-report" if rest.first() == Some(&"stop") => match state.report.take() {
Some(recording) => {
log("INFO", "Stopping the performance report: writing it now");
self.finish_report(recording, state.view, state.max_players);
}
None => log("INFO", "No performance report is being recorded"),
},
"horizon-report" if rest.first() == Some(&"status") => match &state.report {
Some(recording) => log("INFO", &recording.status()),
None => log("INFO", "No performance report is being recorded"),
},
"horizon-report" => {
let seconds = match rest.first().map(|s| s.parse::<u32>()) {
None => Ok(60),
Some(Ok(s)) if (1..=report::MAX_SECONDS).contains(&s) => Ok(s),
Some(_) => Err(()),
"horizon-report" | "horizon-trace" => {
let trace = command == "horizon-trace";
let parsed = if trace {
report::parse_trace(&rest)
} else {
report::parse(&rest)
};
match (seconds, &state.report) {
(Err(()), _) => log(
match parsed {
Err(()) if trace => log(
"INFO",
&format!(
"horizon-report [SECONDS | stop | status]: 1 to {} seconds, 60 by default",
"horizon-trace [SECONDS | stop]: 1 to {} seconds, 30 by default",
report::MAX_SECONDS
),
),
(Ok(_), Some(_)) => {
log("INFO", "A performance report is already being recorded")
Err(()) => log(
"INFO",
&format!(
"horizon-report [SECONDS | stop | status] [--no-samples] [--timeline]: 1 to {} seconds, 60 by default",
report::MAX_SECONDS
),
),
Ok(report::Command::Stop) if state.report.is_some() => {
log("INFO", "Stopping the performance report: writing it now");
self.finish_perf_capture(state, StopReason::Normal);
}
(Ok(seconds), None) => {
state.report = Some(report::Recording::start(seconds, self.wire_bytes()));
Ok(report::Command::Stop) => {
log("INFO", "No performance report is being recorded")
}
Ok(report::Command::Status) => match &state.report {
Some(recording) => log("INFO", &recording.status()),
None => log("INFO", "No performance report is being recorded"),
},
Ok(report::Command::Record {
seconds,
samples,
timeline,
}) if state.report.is_none() => {
self.record_report(
state,
&report::Plan {
seconds,
samples,
timeline,
file: None,
trigger: Trigger::Console,
sampler: Default::default(),
},
);
log(
"INFO",
&format!("Recording a performance report for {seconds} s (reports/)"),
);
}
}
}
// Horizon's own, on the console only: every thread's spans as a timeline (report.rs).
"horizon-trace" if rest.first() == Some(&"stop") => {
match horizon_client::spans::trace::stop() {
Some(events) => log("INFO", &format!("Trace stopped: {events} events written")),
None => log("INFO", "No trace is being written"),
}
}
"horizon-trace" => {
let seconds = match rest.first().map(|s| s.parse::<u32>()) {
None => Some(30),
Some(Ok(s)) if (1..=report::TRACE_MAX_SECONDS).contains(&s) => Some(s),
Some(_) => None,
};
match seconds.map(report::start_trace) {
None => log(
"INFO",
&format!(
"horizon-trace [SECONDS | stop]: 1 to {} seconds, 30 by default",
report::TRACE_MAX_SECONDS
),
),
Some(Ok(path)) => log(
"INFO",
&format!(
"Writing a trace for {} s to {} (open it in ui.perfetto.dev)",
seconds.unwrap_or_default(),
path.display()
),
),
Some(Err(e)) => log("ERROR", &format!("Could not start a trace: {e}")),
Ok(report::Command::Record { .. }) => {
log("INFO", "A performance report is already being recorded")
}
}
}
"save-all" => {
File diff suppressed because it is too large. Load diff
+1 -1
View File
@@ -937,7 +937,7 @@ impl Game<'_> {
Some(ClickEvent::CopyToClipboard(value)) => Click::CopyToClipboard(value),
Some(ClickEvent::ShowDialog(dialog)) => Click::ShowDialog(dialog),
Some(ClickEvent::Custom { id, payload }) => Click::Custom { id, payload },
Some(ClickEvent::OpenFile(_)) | None => return Ok(()),
Some(ClickEvent::OpenFile(_) | ClickEvent::Profiler(_)) | None => return Ok(()),
};
let after = open.dialog.after_action;
return self.run_dialog_action(Some(click), after);
+1
View File
@@ -133,6 +133,7 @@ impl Game<'_> {
/// `EvokerFangs.tick`.
pub(super) fn tick_fangs(&mut self) {
horizon_threads::work::clear_work();
let mut list = std::mem::take(&mut self.fangs.list);
list.retain_mut(|f| {
f.warmup = f.warmup.wrapping_sub(1);
+24 -2
View File
@@ -36,6 +36,8 @@ use std::{
time::{Duration, Instant},
};
pub(super) mod capture;
/// How long the window waits for the server thread to answer a `call`.
const CALL_TIMEOUT: Duration = Duration::from_secs(10);
/// The error of a `call` without an answer: the server stopped, or it was
@@ -678,6 +680,7 @@ impl Game<'_> {
opening: Opening,
) -> Result<()> {
let result = self.run_integrated_inner(launch, flags, tasks, ready, opening);
self.finish_server_capture();
if let Err(error) = &result {
self.native_players_leaving();
self.native_lifecycle_close_failed(&error.to_string());
@@ -711,6 +714,13 @@ impl Game<'_> {
let stopped = self.stop_integrated();
return result.and(stopped);
}
let slow_ms = self
.options
.spikes
.unwrap_or(super::dedicated::SLOW_TICK_MS);
self.profile
.server
.get_or_insert_with(|| capture::ServerSide::new(slow_ms));
self.integrated_tick_loop(&flags)
}
@@ -746,11 +756,11 @@ impl Game<'_> {
was_paused = paused;
if paused {
// Only the connections run.
self.tick_server_timed(true)?;
self.tick_server_measured(true)?;
} else {
self.follow_distances(flags);
let started = Instant::now();
self.tick_server_timed(false)?;
self.tick_server_measured(false)?;
// MinecraftServer.tickServer's tick times, for /tick query.
self.clock.record(started.elapsed());
self.debug_charts.log_tick(started, started.elapsed());
@@ -787,7 +797,19 @@ impl Game<'_> {
self.publish_load_view();
self.wait_for_next_server_tick(&mut next, paused);
}
if failure.is_none() {
self.finish_server_capture();
}
if let Some(error) = failure {
let finished = std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| {
self.finish_server_capture();
}));
if let Err(payload) = finished {
log::error!(
"Horizon: the server capture could not be closed after the crash: {}",
super::dedicated::panic_message(payload.as_ref())
);
}
let stopped = std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| {
self.stop_after_crash(&error.to_string())
}));
@@ -0,0 +1,430 @@
//! The server thread's side of a capture (docs/perf-platform.md §4.1): it writes into the window's file, or into one of its own.
use super::super::dedicated::{SLOW_TICK_MS, report::Recording};
use super::super::perf_remote::Done;
use super::super::profile_capture::{reply_text, shown};
use super::super::{Game, tick_profile};
use horizon_client::client_perf::{
Join,
request::{Failed, ProfileSide, StartOptions, Stopped, Threads},
};
use horizon_client::server_perf::{
self, CaptureEnd, CaptureOptions, MAX_WINDOW_S, Role, SamplerOptions, ServerPerf, TickDetail,
};
use horizon_client::spans;
use horizon_perf_format::{
files,
v1::{GpuMode, Start, StopReason, Trigger},
};
use std::io;
use std::path::PathBuf;
use std::sync::{
Arc,
atomic::{AtomicU8, Ordering},
mpsc::Sender,
};
use std::time::Instant;
/// How far the server thread has got with the side of a capture the window asked for.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub(in crate::game) enum Phase {
/// Asked for, not yet picked up between two ticks.
Asked,
Running,
Ended,
/// Refused or failed to begin; nothing of this side is in the file.
Failed,
}
/// A [`Phase`] the window and the server thread share.
pub(in crate::game) struct Live(AtomicU8);
impl Live {
pub(in crate::game) fn new() -> Arc<Self> {
Arc::new(Self(AtomicU8::new(Phase::Asked as u8)))
}
pub(in crate::game) fn get(&self) -> Phase {
match self.0.load(Ordering::Relaxed) {
0 => Phase::Asked,
1 => Phase::Running,
2 => Phase::Ended,
_ => Phase::Failed,
}
}
pub(in crate::game) fn set(&self, phase: Phase) {
self.0.store(phase as u8, Ordering::Relaxed);
}
/// Whether this side is, or is about to be, writing.
pub(in crate::game) fn active(&self) -> bool {
matches!(self.get(), Phase::Asked | Phase::Running)
}
/// Whether the file holds, or will hold, this side.
pub(in crate::game) fn wrote(&self) -> bool {
self.get() != Phase::Failed
}
}
/// Where a server capture is written.
pub(in crate::game) enum Target {
/// The window's file, which the window finishes.
Join(Join),
/// A file of its own, as a server-only capture needs.
Own {
options: StartOptions,
trigger: Trigger,
dir: PathBuf,
},
}
/// The server thread's health history and the capture it writes, if any.
pub(in crate::game) struct ServerSide {
perf: ServerPerf,
epoch: Instant,
recording: Option<Recording>,
/// When the capture's window ends on this thread's clock, and why it stops then.
window: Option<(f64, StopReason)>,
/// Where this side is, for the window to read.
live: Option<Arc<Live>>,
/// Whether the file is this side's to finish.
owner: bool,
/// The `profile stop` waiting for the file, on an owner.
reply: Option<Sender<String>>,
spans_were_on: bool,
}
impl ServerSide {
pub(in crate::game) fn new(slow_ms: f32) -> Self {
Self {
perf: ServerPerf::with_role(Role::INTEGRATED, 0.0, slow_ms),
epoch: Instant::now(),
recording: None,
window: None,
live: None,
owner: false,
reply: None,
spans_were_on: false,
}
}
fn now(&self) -> f64 {
self.epoch.elapsed().as_secs_f64()
}
/// `instant` on this thread's clock, negative when it is before the clock began.
fn at(&self, instant: Instant) -> f64 {
match instant.checked_duration_since(self.epoch) {
Some(after) => after.as_secs_f64(),
None => -self.epoch.duration_since(instant).as_secs_f64(),
}
}
fn ended(&self) {
if let Some(live) = &self.live {
live.set(Phase::Ended);
}
}
}
impl Game<'_> {
/// Begins this side of a capture, between two ticks; the path of the file.
pub(in crate::game) fn start_server_capture(
&mut self,
target: Target,
live: Arc<Live>,
) -> Result<PathBuf, String> {
let slow_ms = self.options.spikes.unwrap_or(SLOW_TICK_MS);
let mut side = self
.profile
.server
.take()
.unwrap_or_else(|| ServerSide::new(slow_ms));
if !side.perf.capturing() && side.perf.finishing() {
let done: Vec<Done> = side
.perf
.take_saved(true)
.into_iter()
.filter_map(|saved| server_saved(&mut side, saved))
.collect();
self.perf_captures_done(done);
}
let began = self.begin_server_capture(&mut side, target);
if began.is_err() {
live.set(Phase::Failed);
} else {
live.set(Phase::Running);
side.live = Some(live);
}
self.profile.server = Some(side);
began
}
fn begin_server_capture(
&self,
side: &mut ServerSide,
target: Target,
) -> Result<PathBuf, String> {
if side.perf.capturing() || side.perf.finishing() {
return Err("a server capture is already running or still being saved".into());
}
let (options, trigger) = match &target {
Target::Join(join) => (join.options.clone(), join.trigger),
Target::Own {
options, trigger, ..
} => (options.clone(), *trigger),
};
let owner = matches!(target, Target::Own { .. });
let capture = self.server_capture_options(&options, trigger);
let t = side.now();
let was_on = spans::recording_here();
spans::record_here(true);
let (began, window) = match target {
Target::Join(join) => {
let origin = side.at(join.origin);
let window = join.ends.map(|(at, reason)| (side.at(at), reason));
let path = join.path.clone();
let began = side
.perf
.join_capture(t, origin, join.sender, join.path, capture);
(began.map(|()| path), window)
}
Target::Own { dir, .. } => {
let window = match options.seconds {
Some(s) => (t + f64::from(s), StopReason::Normal),
None => (t + MAX_WINDOW_S, StopReason::WindowCap),
};
(side.perf.start_capture(t, &dir, capture), Some(window))
}
};
let path = match began {
Ok(path) => path,
Err(e) => {
spans::record_here(was_on);
return Err(e.to_string());
}
};
side.owner = owner;
side.spans_were_on = was_on;
side.recording = Some(Recording::here(options.seconds.unwrap_or(0)));
side.window = window;
log::info!("Horizon: profiling the server thread to {}", path.display());
Ok(path)
}
/// What a capture of this server records as it starts.
fn server_capture_options(&self, options: &StartOptions, trigger: Trigger) -> CaptureOptions {
let (view, simulation, max_players, players) =
self.lan.as_ref().map_or((0, 0, 1, 0), |h| {
let c = &h.hub.config;
(
c.view_distance,
c.simulation_distance,
c.max_players,
h.hub.guests.len(),
)
});
let start = Start {
build: Some(server_perf::build()),
machine: Some(server_perf::machine(Some(
server_perf::hardware().unwrap_or_else(server_perf::hardware_now),
))),
settings: Some(server_perf::settings(
view.max(0) as u32,
simulation.max(0) as u32,
max_players as u32,
-1,
)),
game_rules: Some(server_perf::game_rules(&self.ui.rules)),
same_process: true,
players: players as u32,
..Start::default()
};
CaptureOptions {
window_s: options.seconds.map(f64::from),
trigger,
start,
samples: options.samples,
sampler: SamplerOptions {
hz: options.hz,
over_ms: options.over_ms,
wall: options.wall,
all_threads: options.threads == Threads::All,
},
block_names: self.block_names(),
levels: self.level_counts(),
wire: self.wire_bytes(),
gpu: GpuMode::Unspecified,
timeline: options.timeline.then(Default::default),
file: None,
}
}
/// Ends this side of the running capture, between two ticks; `reply` hears of an owned file once it is whole.
pub(in crate::game) fn stop_server_capture(
&mut self,
reason: StopReason,
reply: Option<Sender<String>>,
) {
let Some(mut side) = self.profile.server.take() else {
return;
};
if side.perf.capturing() {
side.reply = reply.or(side.reply.take());
self.end_server_capture(&mut side, reason);
} else if let Some(reply) = reply {
let _ = reply.send(reply_text(&Failed {
error: "no capture is running".into(),
}));
}
self.profile.server = Some(side);
}
/// Writes the spans, workers and world and hands the file off to finish; false when no capture runs here.
fn end_server_capture(&self, side: &mut ServerSide, reason: StopReason) -> bool {
if !side.perf.capturing() {
return false;
}
// The spans from between ticks are not a tick's, and the price check counts from none.
spans::take_raw();
let end = side
.recording
.take()
.map_or_else(CaptureEnd::default, |r| self.capture_end(&r));
spans::record_here(side.spans_were_on);
side.window = None;
let t = side.now();
side.perf.begin_stop(t, reason, end);
if !side.owner {
side.ended();
}
true
}
/// Labels the moment in the running capture.
pub(in crate::game) fn mark_server_capture(&mut self, name: &str) {
if let Some(side) = &mut self.profile.server {
let t = side.now();
side.perf.mark(t, name);
}
}
/// Whether a server capture runs or is being saved on this thread.
pub(in crate::game) fn server_capture_busy(&self) -> bool {
self.profile
.server
.as_ref()
.is_some_and(|s| s.perf.capturing() || s.perf.finishing())
}
/// One tick of the server thread, timed for the health history and the capture.
pub(super) fn tick_server_measured(&mut self, paused: bool) -> crate::Result<()> {
self.serve_perf_asked();
let Some(mut side) = self.profile.server.take() else {
return self.tick_server_timed(paused);
};
let slow_ms = side.perf.slow_threshold_ms();
let capturing = side.perf.capturing();
let sampled = side.recording.as_ref().is_some_and(Recording::sample_next);
tick_profile::set_sampling(sampled);
let mut detail = None;
let started = Instant::now();
let result = self.tick_server_spans(paused, |timing| {
if capturing || timing.wall_ms > slow_ms {
detail = Some(TickDetail {
cpu_ms: timing.cpu_ms,
spans: timing.raw,
});
}
});
let wall_ms = started.elapsed().as_secs_f32() * 1000.0;
tick_profile::set_sampling(false);
if let Some(recording) = &mut side.recording {
recording.add(sampled);
}
let t = side.now();
side.perf.tick(
t,
wall_ms,
detail,
|| self.perf_world_counts(),
|| self.wire_bytes(),
);
if let Some((end, reason)) = side.window
&& t >= end
{
self.end_server_capture(&mut side, reason);
}
let done: Vec<Done> = side
.perf
.take_saved(false)
.into_iter()
.filter_map(|saved| server_saved(&mut side, saved))
.collect();
self.profile.server = Some(side);
self.perf_captures_done(done);
result
}
/// The server is closing: ends the capture, and waits for the file when it is this side's.
pub(in crate::game) fn finish_server_capture(&mut self) {
let Some(mut side) = self.profile.server.take() else {
return;
};
self.end_server_capture(&mut side, StopReason::Shutdown);
let mut done = Vec::new();
if side.owner {
for saved in side.perf.take_saved(true) {
done.extend(server_saved(&mut side, saved));
}
}
side.ended();
drop(side);
self.perf_captures_done(done);
}
}
/// A capture of this side finished: an owner logs the file and tells the window that waits for it.
fn server_saved(side: &mut ServerSide, saved: io::Result<server_perf::Saved>) -> Option<Done> {
if !side.owner {
if let Err(e) = saved {
log::error!("Horizon: could not write the server side of the capture: {e}");
}
return None;
}
side.ended();
let reply = side.reply.take();
match saved {
Ok(saved) => {
for path in &saved.deleted {
log::info!("Horizon: removed the oldest capture {}", path.display());
}
log::info!("Horizon: {}", files::saved_line(&saved.path));
let removed: Vec<String> = saved.deleted.iter().map(|p| shown(p)).collect();
if let Some(reply) = reply {
let _ = reply.send(reply_text(&Stopped {
ok: true,
sides: vec![ProfileSide::Server],
path: shown(&saved.path),
open_at: horizon_client::client_perf::request::viewer(),
removed: removed.clone(),
}));
}
Some(Done::Saved {
path: shown(&saved.path),
file: saved.path.clone(),
removed,
})
}
Err(e) => {
log::error!("Horizon: could not write the performance capture: {e}");
if let Some(reply) = reply {
let _ = reply.send(reply_text(&Failed {
error: e.to_string(),
}));
}
Some(Done::Failed)
}
}
}
@@ -26,6 +26,7 @@ impl Game<'_> {
}
}
self.tick_dropped_items(&mut round);
horizon_threads::work::clear_work();
self.native_tick_post_batch(round);
}
/// An item that joined during this tick's entity pass: it first ticks on
@@ -57,7 +58,9 @@ impl Game<'_> {
// `ItemUtils.onContainerDestroyed`: what a destroyed item held, and
// where it lay, for adding to the level once its own tick is done.
let mut destroyed: Vec<(ItemStack, Vec3)> = Vec::new();
let item_type = horizon_threads::work::entity_id("minecraft:item");
for i in 0..items.len() {
horizon_threads::work::set_entity(item_type);
if removed[i] {
// Merged into an earlier item before its own tick: it never ticked.
round.not_ticked(items[i].id);
+1 -1
View File
@@ -1310,7 +1310,7 @@ impl Game<'_> {
Ok(Some(json!({ "ok": true })))
}
Some("status") => Ok(Some(self.lan_status())),
_ => Err("lan open|close|status".into()),
_ => Err("lan open|close|status|op".into()),
}
}
+8 -77
View File
@@ -40,7 +40,6 @@ use winit::keyboard::KeyCode;
mod guest_body;
mod horizon_channels;
use horizon_channels::horizon_channel;
#[cfg(test)]
mod crack_feedback_tests;
@@ -1606,7 +1605,7 @@ impl Game<'_> {
);
return false;
}
self.unpack_boat_loot(i);
self.unpack_boat_loot(i, true);
}
// `createMenu` (ChestMenu.threeRows or a HopperMenu), and
// `MenuProvider.getDisplayName`, the vehicle's name key.
@@ -1636,64 +1635,6 @@ impl Game<'_> {
true
}
/// `ContainerEntity.unpackChestVehicleLootTable` for a chest boat: the table
/// its raw NBT still holds rolls into the slots on the first open (an empty
/// boat, as a command leaves one), and `player_generates_container_loot`
/// fires for the player who opened it.
fn boat_loot_pending(&self, id: u32) -> bool {
matches!(
self.persist.raw.get(&id),
Some(horizon_world::nbt::Tag::Compound(map)) if map.contains_key(&"LootTable".into())
)
}
fn unpack_boat_loot(&mut self, i: usize) {
use horizon_world::nbt::Tag;
let id = self.boats[i].id;
let (table, seed) = {
let Some(Tag::Compound(map)) = self.persist.raw.get_mut(&id) else {
return;
};
let Some(table) = crate::save::text(map.get(&"LootTable".into()))
.and_then(|id| horizon_client_core::identifier::normalize_id(&id))
else {
return;
};
let seed = match map.get(&"LootTableSeed".into()) {
Some(Tag::Long(seed)) => Some(*seed),
_ => None,
};
// The table is spent even when nothing can be rolled into it (the
// loot data is not loaded): vanilla clears it before filling.
map.remove(&"LootTable".into());
map.remove(&"LootTableSeed".into());
(table, seed)
};
// `LootTable.fill` sets the slots it rolls and leaves the rest; a boat
// that already holds something keeps it, so no table is applied over it.
if self.boats[i].items.iter().any(Option::is_some) {
return;
}
let loot = crate::structure_blocks::loot_data(&self.options.resources, &self.pack_files());
let b = &self.boats[i];
let at = Vec3::new(b.position.x, b.position.y, b.position.z);
let biome = self.biome_name_at(at).unwrap_or_default();
let pos = (
b.position.x.floor() as i32,
b.position.y.floor() as i32,
b.position.z.floor() as i32,
);
crate::structure_blocks::roll_container(
&mut self.boats[i].items,
loot.as_deref(),
table.clone(),
seed,
pos,
&biome,
);
self.trigger_loot_table(&table);
}
/// The guest's open container is closing (`AbstractContainerMenu.removed`):
/// it no longer counts among the openers of the lid, and the lid's close
/// sound is played for everyone around (`ChestBlockEntity.onClose`, a
@@ -2134,22 +2075,8 @@ impl Game<'_> {
let Some(gi) = host.hub.guests.iter().position(|g| g.conn == conn) else {
return;
};
// Plugin channels a client registers in play (minecraft:register), from a dead player
// too; the packet goes on as before.
if let S::CustomPayload(p) = &packet
&& let Some(register) = horizon_client::lan::channel_registration(&p.channel)
{
self.guest_channels(host, gi, &p.data, register);
}
// A plugin channel's message on a channel that isn't one of Horizon's own (the match below
// takes those) is for the mods, from a dead player too: vanilla's handleCustomPayload doesn't
// look at health.
let packet = match packet {
S::CustomPayload(p) if !horizon_channel(&p.channel) => {
self.guest_custom_payload(host, conn, p.channel, p.data);
return;
}
packet => packet,
let Some(packet) = self.guest_plugin_channels(host, conn, gi, packet) else {
return;
};
// A dead player acts no more, but the death screen still respawns and chats: its
// commands run as anyone's (ServerGamePacketListenerImpl.handleChatCommand doesn't
@@ -2884,6 +2811,9 @@ impl Game<'_> {
S::CustomPayload(p) if p.channel == horizon_client::lod_net::CHANNEL => {
self.guest_far_terrain(host, conn, &p.data)
}
S::CustomPayload(p) if p.channel == horizon_client::perf_net::CHANNEL => {
self.guest_perf(host, conn, &p.data)
}
S::CustomPayload(p) if p.channel == horizon_client::lan::recipes::TRANSFER_CHANNEL => {
self.guest_recipe_transfer(host, conn, &p.data);
}
@@ -6853,7 +6783,7 @@ mod guest_close_listener_tests {
#[cfg(test)]
mod horizon_channel_tests {
use super::horizon_channel;
use super::horizon_channels::horizon_channel;
#[test]
fn horizon_channels_reach_their_handlers_and_others_go_to_the_mods() {
@@ -6865,6 +6795,7 @@ mod horizon_channel_tests {
assert!(horizon_channel(horizon_protocol::quiet::CHANNEL));
assert!(horizon_channel(horizon_protocol::mod_net::CHANNEL));
assert!(horizon_channel(horizon_client::lod_net::CHANNEL));
assert!(horizon_channel(horizon_client::perf_net::CHANNEL));
assert!(!horizon_channel("mymod:hello"));
assert!(!horizon_channel("minecraft:brand"));
}
@@ -1,4 +1,6 @@
//! The plugin channels Horizon handles itself, before the mods.
use super::super::{Game, lan::LanHost};
use horizon_protocol::packets::play::Serverbound as S;
/// Whether a guest's plugin channel message is for one of Horizon's own handlers in
/// `lan_guest_packet` (its `CustomPayload` arms), not for the mods.
@@ -7,7 +9,37 @@ pub(super) fn horizon_channel(channel: &str) -> bool {
horizon_protocol::quiet::CHANNEL,
horizon_protocol::mod_net::CHANNEL,
horizon_client::lod_net::CHANNEL,
horizon_client::perf_net::CHANNEL,
horizon_client::lan::recipes::TRANSFER_CHANNEL,
]
.contains(&channel)
}
impl Game<'_> {
/// A guest's plugin channel registrations and its messages for the mods; the packet when Horizon handles it.
pub(super) fn guest_plugin_channels(
&mut self,
host: &mut LanHost,
conn: u32,
gi: usize,
packet: S,
) -> Option<S> {
// Plugin channels a client registers in play (minecraft:register), from a dead player
// too; the packet goes on as before.
if let S::CustomPayload(p) = &packet
&& let Some(register) = horizon_client::lan::channel_registration(&p.channel)
{
self.guest_channels(host, gi, &p.data, register);
}
// A plugin channel's message on a channel that isn't one of Horizon's own (`lan_guest_packet`'s
// match takes those) is for the mods, from a dead player too: vanilla's handleCustomPayload doesn't
// look at health.
match packet {
S::CustomPayload(p) if !horizon_channel(&p.channel) => {
self.guest_custom_payload(host, conn, p.channel, p.data);
None
}
packet => Some(packet),
}
}
}
+54
View File
@@ -0,0 +1,54 @@
//! `horizon-ctl lan op`: a connected guest's operator level, which remote profiling checks.
use super::Game;
use horizon_client::Result;
impl Game<'_> {
/// `horizon-ctl lan op NAME LEVEL`; `None` for any other command.
pub(super) fn lan_op_control(&mut self, words: &[&str]) -> Result<Option<serde_json::Value>> {
use serde_json::json;
if words.first() != Some(&"lan") || words.get(1) != Some(&"op") {
return Ok(None);
}
let usage = "lan op NAME LEVEL (0 to 4)";
let (Some(name), Some(level)) = (
words.get(2),
words
.get(3)
.and_then(|l| l.parse::<u8>().ok())
.filter(|l| *l <= 4),
) else {
return Err(usage.into());
};
let name = (*name).to_owned();
let found = match &self.integrated {
Some(server) => server
.call(move |g| g.lan_set_op(&name, level))
.ok_or(super::integrated::NO_ANSWER)?,
None => self.lan_set_op(&name, level),
};
if found {
Ok(Some(json!({ "ok": true })))
} else {
Err("no guest of that name is connected".into())
}
}
/// `horizon-ctl lan op`: the connected guest `name` gets operator level `level`; false when none is called that.
pub(super) fn lan_set_op(&mut self, name: &str, level: u8) -> bool {
let Some(host) = self.lan.as_mut() else {
return false;
};
let Some(uuid) = host
.hub
.guests
.iter()
.find(|g| g.name == name)
.map(|g| g.uuid)
else {
return false;
};
host.hub.ops.insert(uuid, level);
host.hub.resend_commands();
true
}
}
@@ -36,7 +36,7 @@ pub(super) struct Options {
pub(super) no_pause: bool,
/// Control socket path (`--control`), for `horizon-ctl`.
pub(super) control: Option<PathBuf>,
/// `HORIZON_SPIKES=MS`: print the timed spans of frames slower than MS.
/// `HORIZON_SPIKES=MS`: the slow ring's threshold, and spans on for every thread; nothing is logged.
pub(super) spikes: Option<f32>,
/// Render into a texture instead of the window (`--offscreen`), so a
/// GPU the display cannot present from can still be benchmarked. The
@@ -63,6 +63,10 @@ pub(super) struct Options {
pub(super) transfers: Option<horizon_client::lan::arrivals::Settings>,
/// `--benchmark SCENARIO`: run the benchmark, write its report and quit.
pub(super) benchmark: Option<String>,
/// `--benchmark-set SWITCH=VALUE`: a setting for the benchmark's run, with `--benchmark travel`.
pub(super) benchmark_set: Option<(String, String)>,
/// `--profile`: with `--benchmark`, record a .hzperf capture of the run that embeds the report.
pub(super) benchmark_profile: bool,
/// `--dev-mod PATH` (repeatable): content mods always on.
pub(super) dev_mods: Vec<PathBuf>,
/// What this Game executes; independent of its current server connection.
@@ -216,6 +220,8 @@ impl Options {
dedicated: None,
transfers: None,
benchmark: None,
benchmark_set: None,
benchmark_profile: false,
dev_mods: Vec::new(),
execution_role: execution_role::ExecutionRole::Client,
fullscreen: false,
@@ -228,7 +234,7 @@ impl Options {
match arg.as_str() {
"--help" | "-h" => {
println!(
"Horizon — Minecraft Java Edition {} in Rust\n\n --resources PATH Prepared resources (horizon-prepare)\n --render-distance N Chunks, 2..128 (default 12)\n --world NAME World folder under saves/ (default \"New World\")\n --seed N World seed for new worlds (default 0)\n --width N, --height N Window size (default 1280 x 720)\n --creative Creative mode (default survival)\n --test-terrain Simple test terrain instead of vanilla generation\n --frames N Exit after N frames (testing)\n --world-frames N Exit after N frames drawn in the world (testing)\n --timeout-seconds N Exit with an error after N seconds (testing; default 120 with --frames, else none)\n --smoke-report PATH Write a JSON report on exit\n --benchmark SCENARIO Run a benchmark (standard, village, forest, ocean, mountain), write its report to benchmarks/ and quit\n --control Accept horizon-ctl commands on a local socket\n --control-socket PATH Control socket path (default $XDG_RUNTIME_DIR/horizon-control.sock)\n --transfers SETTINGS A dedicated server takes a proxy's hand-over requests (server.toml [transfer], docs/server.md)\n --server HOST[:PORT] Join a multiplayer server (offline mode) at start\n --quickPlaySingleplayer WORLD Open this world of saves/ at start, only if it exists (vanilla's)\n --quickPlayMultiplayer HOST[:PORT] The same as --server (vanilla's name)\n --name NAME Player name for multiplayer (default: generated once, kept in horizon-player.txt)\n --session-file PATH Signed-in account from the launcher (read once, then deleted)\n --session-stdin The same account as JSON on standard input\n --game-dir DIR Game folder: saves, options.txt, packs, mods (default: the working directory)\n --home DIR Logs and crash reports (default $HORIZON_HOME, else the platform data folder)\n --player-dir DIR This player's own files when several play in one game folder: options.txt, logs, horizon-player.txt, servers.dat, screenshots\n --fullscreen Open the window fullscreen\n --launch-api Print the Launch API version and options as JSON (docs/api/launch.md)\n --dev-mod PATH A content mod (.hmod or folder) to develop: always on\n --pregen Add Horizon's /pregen command (docs/pregeneration.md)\n\nControls: mouse look, WASD move, Space jump, Left Shift sneak, Left Ctrl sprint,\nleft click break, right click place/use, 1-9/scroll hotbar, E inventory,\nQ drop, F3 debug, Esc pause.",
"Horizon — Minecraft Java Edition {} in Rust\n\n --resources PATH Prepared resources (horizon-prepare)\n --render-distance N Chunks, 2..128 (default 12)\n --world NAME World folder under saves/ (default \"New World\")\n --seed N World seed for new worlds (default 0)\n --width N, --height N Window size (default 1280 x 720)\n --creative Creative mode (default survival)\n --test-terrain Simple test terrain instead of vanilla generation\n --frames N Exit after N frames (testing)\n --world-frames N Exit after N frames drawn in the world (testing)\n --timeout-seconds N Exit with an error after N seconds (testing; default 120 with --frames, else none)\n --smoke-report PATH Write a JSON report on exit\n --benchmark SCENARIO Run a benchmark (standard, village, forest, ocean, mountain, travel, ab:SWITCH), write its report to benchmarks/ and quit\n --profile With --benchmark: also write a performance capture (reports/*.hzperf in the game home) that embeds the report\n --benchmark-set S=V With --benchmark travel: set a graphics switch (gpu-culling, section-cull, section-sides, section-draws) to one of its two values first\n --control Accept horizon-ctl commands on a local socket\n --control-socket PATH Control socket path (default $XDG_RUNTIME_DIR/horizon-control.sock)\n --transfers SETTINGS A dedicated server takes a proxy's hand-over requests (server.toml [transfer], docs/server.md)\n --server HOST[:PORT] Join a multiplayer server (offline mode) at start\n --quickPlaySingleplayer WORLD Open this world of saves/ at start, only if it exists (vanilla's)\n --quickPlayMultiplayer HOST[:PORT] The same as --server (vanilla's name)\n --name NAME Player name for multiplayer (default: generated once, kept in horizon-player.txt)\n --session-file PATH Signed-in account from the launcher (read once, then deleted)\n --session-stdin The same account as JSON on standard input\n --game-dir DIR Game folder: saves, options.txt, packs, mods (default: the working directory)\n --home DIR Logs and crash reports (default $HORIZON_HOME, else the platform data folder)\n --player-dir DIR This player's own files when several play in one game folder: options.txt, logs, horizon-player.txt, servers.dat, screenshots\n --fullscreen Open the window fullscreen\n --launch-api Print the Launch API version and options as JSON (docs/api/launch.md)\n --dev-mod PATH A content mod (.hmod or folder) to develop: always on\n --pregen Add Horizon's /pregen command (docs/pregeneration.md)\n\nControls: mouse look, WASD move, Space jump, Left Shift sneak, Left Ctrl sprint,\nleft click break, right click place/use, 1-9/scroll hotbar, E inventory,\nQ drop, F3 debug, Esc pause.",
horizon_core::VANILLA_VERSION
);
return Ok(None);
@@ -238,6 +244,7 @@ impl Options {
return Ok(None);
}
"--fullscreen" => o.fullscreen = true,
"--profile" => o.benchmark_profile = true,
"--session-stdin" => o.session_stdin = true,
"--dedicated" => {
o.execution_role = execution_role::ExecutionRole::DedicatedServer;
@@ -324,14 +331,13 @@ impl Options {
"--player-dir" => {}
"--dev-mod" => o.dev_mods.push(std::path::absolute(&value)?),
"--benchmark" => {
if horizon_client::benchmark::scenes(&value).is_none() {
return Err(format!(
"unknown benchmark {value}: standard, village, forest, ocean or mountain"
)
.into());
}
horizon_client::benchmark::Scenario::parse(&value)?;
o.benchmark = Some(value);
}
"--benchmark-set" => {
o.benchmark_set =
Some(horizon_client::benchmark::parse_set(&value)?)
}
"--server" | "--quickPlayMultiplayer" => o.server = Some(value),
"--name" => o.player_name = Some(value),
"--session-file" => o.session_file = Some(value),
@@ -362,6 +368,12 @@ impl Options {
}
}
}
if o.benchmark_set.is_some() && o.benchmark.as_deref() != Some("travel") {
return Err("--benchmark-set needs --benchmark travel".into());
}
if o.benchmark_profile && o.benchmark.is_none() {
return Err("--profile needs --benchmark".into());
}
Ok(Some(o))
}
}
+127 -22
View File
@@ -128,6 +128,14 @@ impl Default for Pass {
}
}
/// How long a server tick took and what it spent the time in.
pub(super) struct TickTiming {
pub wall_ms: f32,
pub cpu_ms: Option<f32>,
/// As recorded in the tick, each span after the spans inside it; the ones from between ticks are left out.
pub raw: Vec<horizon_client::spans::Span>,
}
/// The levels not in place, by dimension.
#[derive(Default)]
pub(super) struct Levels {
@@ -162,6 +170,13 @@ impl Levels {
self.parked.values().map(|level| &level.world)
}
/// The streamers of the levels not in place.
pub(super) fn parked_streamings(
&self,
) -> impl Iterator<Item = &horizon_client::streaming::Streaming> {
self.parked.values().map(|level| &level.streaming)
}
/// The world of the level not in place, in `dim`, to change.
pub(super) fn parked_world_mut(&mut self, dim: Dim) -> Option<&mut World> {
self.parked.get_mut(&dim).map(|level| &mut level.world)
@@ -617,6 +632,11 @@ impl Game<'_> {
for (k, dim) in dims.into_iter().enumerate() {
// Each level's share of the tick (the first one also reads the network and runs the
// tick's shared parts): `horizon-report`'s `level_*` spans.
horizon_threads::work::set_dim(match dim {
Dim::Overworld => 1,
Dim::Nether => 2,
Dim::End => 3,
});
let _level_span = horizon_client::spans::guard(match dim {
Dim::Overworld => "level_overworld",
Dim::Nether => "level_the_nether",
@@ -720,22 +740,30 @@ impl Game<'_> {
Ok(())
}
/// `tick_server` as a server thread runs it. With `HORIZON_SPIKES=MS`, a
/// tick longer than MS is logged with its thread CPU time and spans.
/// `tick_server` as a server thread runs it, timed whenever spans are on.
pub(super) fn tick_server_timed(&mut self, paused: bool) -> Result<()> {
self.tick_server_spans(paused, |_, _, _, _| {})
self.tick_server_spans(paused, |_| {})
}
/// `tick_server_timed`, handing the tick's wall time, thread CPU time
/// and spans (milliseconds, summed by name) to `record` while spans are on, with how many
/// spans the tick itself recorded: a dedicated server's performance report turns them on to
/// record its ticks.
/// `tick_server_timed`, handing the tick's timing to `record` while spans are on.
pub(super) fn tick_server_spans(
&mut self,
paused: bool,
record: impl FnOnce(f32, Option<f32>, Vec<horizon_client::spans::Span>, usize),
record: impl FnOnce(TickTiming),
) -> Result<()> {
if self.options.spikes.is_none() && !horizon_client::spans::enabled() {
let started = Instant::now();
horizon_threads::work::tick_start();
let result = self.tick_server_recorded(paused, record);
horizon_threads::work::tick_end(started.elapsed().as_nanos() as u64);
result
}
fn tick_server_recorded(
&mut self,
paused: bool,
record: impl FnOnce(TickTiming),
) -> Result<()> {
if self.options.spikes.is_none() && !horizon_client::spans::recording_here() {
return self.tick_server(paused);
}
// Spans recorded since the last tick ran between ticks (the chunk pump of the wait): they
@@ -749,26 +777,28 @@ impl Game<'_> {
.map(|(end, start)| (end - start).as_secs_f32() * 1000.0);
// Taken every tick: nothing else takes this thread's spans.
let raw = horizon_client::spans::take_raw();
let in_tick = raw.len();
let spans = if between.is_empty() {
horizon_client::spans::merge(&raw)
} else {
horizon_client::spans::merge(&[between, raw].concat())
horizon_client::spans::merge(&[&between[..], &raw[..]].concat())
};
// The spans go to the slow ring; whose packets or events took the tick stays in the owner's log.
let notes = horizon_client::spans::take_notes();
if let Some(threshold) = self.options.spikes
&& wall > threshold
&& !notes.is_empty()
{
let cpu = cpu.map_or_else(String::new, |c| format!(", cpu {c:.1} ms"));
log::info!(
"Horizon: slow tick {wall:.1} ms{cpu} ({} mobs): {}{}",
self.mobs.len(),
horizon_client::spans::describe(&spans),
"Horizon: {wall:.1} ms tick was busy with: {}",
slowest_steps(&notes)
);
}
self.native_server_tick_timed(wall, cpu, &spans);
record(wall, cpu, spans, in_tick);
record(TickTiming {
wall_ms: wall,
cpu_ms: cpu,
raw,
});
result
}
@@ -985,17 +1015,13 @@ impl Game<'_> {
}
}
/// A slow tick's line's end: the steps it noted as its slowest (`spans::note`), whose packets
/// or which player's event, or nothing.
/// The steps a slow tick noted as its slowest (`spans::note`), whose packets or which player's event.
fn slowest_steps(notes: &[(f32, String)]) -> String {
if notes.is_empty() {
return String::new();
}
let list: Vec<String> = notes
.iter()
.map(|(ms, what)| format!("{what} {ms:.1}"))
.collect();
format!("; slowest: {}", list.join(", "))
list.join(", ")
}
#[cfg(test)]
@@ -1054,6 +1080,85 @@ mod tests {
assert_eq!(nether.size(), 100.0);
}
struct LogCapture;
static LOGGED: std::sync::Mutex<Vec<String>> = std::sync::Mutex::new(Vec::new());
impl log::Log for LogCapture {
fn enabled(&self, _: &log::Metadata) -> bool {
true
}
fn log(&self, record: &log::Record) {
LOGGED.lock().unwrap().push(record.args().to_string());
}
fn flush(&self) {}
}
/// 300 slow ticks in ten seconds log no Horizon slow-tick line, and the ring holds the last 256.
#[test]
#[ignore = "needs prepared resources in HORIZON_TEST_RESOURCES"]
fn slow_ticks_enter_the_ring_and_not_the_log() {
use crate::game::{Audio, Options, Progress, Settings, load_resources};
use horizon_client::server_perf::{ServerPerf, TickDetail, WorldCounts};
log::set_logger(&LogCapture).expect("no other logger in this test run");
log::set_max_level(log::LevelFilter::Info);
let root = std::path::PathBuf::from(std::env::var_os("HORIZON_TEST_RESOURCES").unwrap());
let settings = Settings::default();
let loaded = load_resources(
root.clone(),
None,
&root,
&settings,
Instant::now(),
false,
true,
&Progress::new(),
&mut |_, _| unreachable!(),
)
.unwrap();
let mut options = Options::parse(std::iter::empty()).unwrap().unwrap();
options.resources = root.clone();
options.execution_role = crate::game::execution_role::ExecutionRole::IntegratedServer;
options.spikes = Some(0.0);
let mut game = Game::new(
&loaded.resources,
loaded.pack,
loaded.gui.unwrap(),
options,
settings,
Audio::silent(&root),
None,
)
.unwrap();
game.build_registries();
let mut perf = ServerPerf::new(0.0, 0.0);
for i in 0..300 {
let mut detail = None;
let started = Instant::now();
game.tick_server_spans(false, |timing| {
detail = Some(TickDetail {
cpu_ms: timing.cpu_ms,
spans: timing.raw,
});
})
.unwrap();
let wall_ms = started.elapsed().as_secs_f32() * 1000.0;
perf.tick(
f64::from(i) / 30.0,
wall_ms,
detail,
WorldCounts::default,
Vec::new,
);
}
let kept: Vec<u64> = perf.slow_records().map(|r| r.counter).collect();
assert_eq!(kept, (45..=300).collect::<Vec<u64>>());
let logged = LOGGED.lock().unwrap();
assert!(
logged.iter().all(|line| !line.contains("slow tick")),
"{logged:?}"
);
}
/// A game on a scratch world folder, the way `dedicated.rs` opens one
/// (`load_world` with an absolute folder).
fn open_game(
@@ -339,6 +339,8 @@ pub(super) fn start_game<'a>(
fps_cap: None,
offscreen: game.options.offscreen,
game_dir: std::env::current_dir()?,
set: game.options.benchmark_set.clone(),
profile: game.options.benchmark_profile,
},
);
}
+140 -1
View File
@@ -1549,7 +1549,9 @@ impl Game<'_> {
let mut items = if let Some(c) = self.carts.iter_mut().find(|c| c.id == id) {
std::mem::take(&mut c.items)
} else {
std::mem::take(&mut self.boats.iter_mut().find(|b| b.id == id)?.items)
let i = self.boats.iter().position(|b| b.id == id)?;
self.unpack_boat_loot(i, false);
std::mem::take(&mut self.boats[i].items)
};
let key = (i32::MAX - 15, super::minecarts::CART_KEY_Y, id as i32);
let (view, open) = self.lend_items(key, &mut items);
@@ -2243,12 +2245,149 @@ mod mod_container_tests {
#[cfg(test)]
mod hopper_cart_transfer_tests {
use super::super::{
boats::Boat,
minecarts::{CartKind, Minecart},
native_machines::tests::server_game,
};
use super::Game;
use horizon_client::{containers::BlockEntity, items::ItemStack};
use horizon_core::geometry::Vec3;
use horizon_world::nbt::Tag;
#[test]
#[ignore = "needs prepared resources in HORIZON_TEST_RESOURCES"]
fn chest_boat_automation_does_not_award_player_loot() {
use horizon_client::advancements::Tree;
use horizon_world::loot::PackOverlay;
use std::sync::Arc;
let mut game = server_game();
game.options.dedicated = None;
let files = vec![(
"advancement/parity:boat_loot".into(),
vec![br#"{"criteria":{"loot":{"trigger":"minecraft:player_generates_container_loot","conditions":{"loot_tables":"minecraft:chests/spawn_bonus_chest"}}}}"#.to_vec()],
)];
game.adv.tree = Arc::new(Tree::load_with(
&game.options.resources.join("data"),
&PackOverlay { files: &files },
));
game.adv.loaded = true;
let ai = game.adv.tree.by_id["parity:boat_loot"];
for player in [false, true] {
game.next_entity += 1;
let id = game.next_entity;
game.boats
.push(Boat::new(id, "oak".into(), Vec3::new(8.5, 65.0, 8.5), 0.0).with_chest());
game.persist.raw.insert(
id,
Tag::compound([
(
"LootTable",
Tag::String("minecraft:chests/spawn_bonus_chest".into()),
),
("LootTableSeed", Tag::Long(73)),
]),
);
let i = game.boats.len() - 1;
game.unpack_boat_loot(i, player);
assert_eq!(game.adv.progress.done(&game.adv.tree.list[ai]), player);
// Opening a container automation already unpacked awards nothing.
game.unpack_boat_loot(i, true);
assert_eq!(game.adv.progress.done(&game.adv.tree.list[ai]), player);
}
}
#[test]
#[ignore = "needs prepared resources in HORIZON_TEST_RESOURCES"]
fn chest_boat_slot_access_fills_only_empty_slots_once() {
let mut game = server_game();
let sentinel = Some(ItemStack::new("minecraft:diamond", 17));
for open in [false, true] {
for occupied in [1, 24, 27] {
game.next_entity += 1;
let id = game.next_entity;
let mut boat =
Boat::new(id, "oak".into(), Vec3::new(8.5, 65.0, 8.5), 0.0).with_chest();
if occupied > 1 {
boat.items.fill(sentinel.clone());
if occupied == 24 {
for slot in [0, 13, 26] {
boat.items[slot] = None;
}
}
} else {
boat.items[12] = sentinel.clone();
}
let key = (
i32::MAX - 15,
super::super::minecarts::CART_KEY_Y,
id as i32,
);
if open {
game.world
.block_entities
.insert(key, BlockEntity::Chest(boat.items.clone()));
// The menu copy is authoritative, even if the entity is stale.
boat.items.fill(None);
}
game.boats.push(boat);
game.persist.raw.insert(
id,
Tag::compound([
(
"LootTable",
Tag::String("minecraft:chests/spawn_bonus_chest".into()),
),
("LootTableSeed", Tag::Long(73)),
]),
);
let items = game
.with_entity_container(id, |g, _| {
g.world.block_entities.get(&key).unwrap().slots().to_vec()
})
.unwrap();
assert_eq!(items[12], sentinel);
if occupied == 27 {
assert!(items.iter().all(|item| item == &sentinel));
} else if occupied == 24 {
// Pinned 26.3 LootTable.fill, seed 73, only these slots empty.
for (slot, item, count) in
[(0, "stick", 1), (13, "salmon", 1), (26, "acacia_log", 2)]
{
assert_eq!(
items[slot],
Some(ItemStack::new(&format!("minecraft:{item}"), count))
);
}
assert_eq!(items.iter().filter(|item| *item == &sentinel).count(), 24);
} else {
assert!(items.iter().flatten().count() > 1, "pending loot was lost");
}
assert!(game.persist.raw[&id].get("LootTable").is_none());
assert!(game.persist.raw[&id].get("LootTableSeed").is_none());
assert_eq!(game.boats.last().unwrap().items, items);
game.with_entity_container(id, |g, _| {
if let Some(BlockEntity::Chest(items)) = g.world.block_entities.get_mut(&key) {
items.fill(None);
}
})
.unwrap();
game.with_entity_container(id, |g, _| {
assert!(
g.world
.block_entities
.get(&key)
.unwrap()
.slots()
.iter()
.all(Option::is_none)
);
})
.unwrap();
game.release_vehicle_container(id);
}
}
}
/// Issue #8668: a hopper minecart takes one item a tick from the container
/// above. `MinecartHopper.tick` has no transfer cooldown in 26.3: it calls
+682 -93
View File
@@ -1,22 +1,41 @@
//! `horizon-ctl memory`: what holds the game's memory, by category. The
//! process's resident set and the allocator's committed bytes say how much;
//! the census of the client's columns, the integrated server's levels, the
//! renderer's bookkeeping and the GPU allocator's resources by label say
//! where it goes. Bytes the census doesn't reach show as the difference.
//! the census of the columns, the server's entities, players and chunk send
//! caches, the renderer's bookkeeping and the GPU allocator's resources by
//! label say where it goes. `unattributed_mb` is what the named parts leave.
use super::Game;
use horizon_client::chunk::ColumnCensus;
use horizon_client::fast_hash::FxHashSet;
use horizon_client::lan::hub::Hub;
use horizon_client::world::World;
use serde_json::{Value, json};
use serde::Serialize;
use serde_json::Value;
use std::collections::BTreeMap;
use std::mem::size_of;
use std::sync::Arc;
const MB: f64 = 1048576.0;
/// Entity types and players listed apart; the rest only count in the totals.
const TOP: usize = 30;
const TOP_PLAYERS: usize = 50;
fn mb(bytes: impl TryInto<u64>) -> f64 {
let bytes: u64 = bytes.try_into().unwrap_or(0);
(bytes as f64 / MB * 10.0).round() / 10.0
}
fn mb_signed(bytes: i64) -> f64 {
(bytes as f64 / MB * 10.0).round() / 10.0
}
fn vec_bytes<T>(v: &Vec<T>) -> usize {
v.capacity() * size_of::<T>()
}
fn string_bytes(s: &Option<String>) -> usize {
s.as_ref().map_or(0, String::capacity)
}
/// The columns of `worlds`, each section counted once however many columns
/// (or worlds) share it.
fn census<'a>(worlds: impl IntoIterator<Item = &'a World>) -> ColumnCensus {
@@ -30,21 +49,42 @@ fn census<'a>(worlds: impl IntoIterator<Item = &'a World>) -> ColumnCensus {
census
}
fn census_json(c: &ColumnCensus) -> Value {
json!({
"total_mb": mb(c.total_bytes()),
"columns": c.columns,
"sections": c.sections,
"unique_sections": c.unique_sections,
"uniform_block_sections": c.uniform_block_sections,
"light_arrays": c.light_arrays,
"blocks_mb": mb(c.block_bytes),
"biomes_mb": mb(c.biome_bytes),
"light_mb": mb(c.light_bytes),
"heightmaps_mb": mb(c.heightmap_bytes),
"section_records_mb": mb(c.section_record_bytes),
"column_records_mb": mb(c.column_bytes),
})
#[derive(Serialize)]
struct Columns {
total_mb: f64,
columns: usize,
sections: usize,
unique_sections: usize,
uniform_block_sections: usize,
light_arrays: usize,
blocks_mb: f64,
biomes_mb: f64,
light_mb: f64,
heightmaps_mb: f64,
section_records_mb: f64,
column_records_mb: f64,
#[serde(skip)]
bytes: usize,
}
impl Columns {
fn new(c: &ColumnCensus) -> Self {
Self {
total_mb: mb(c.total_bytes()),
columns: c.columns,
sections: c.sections,
unique_sections: c.unique_sections,
uniform_block_sections: c.uniform_block_sections,
light_arrays: c.light_arrays,
blocks_mb: mb(c.block_bytes),
biomes_mb: mb(c.biome_bytes),
light_mb: mb(c.light_bytes),
heightmaps_mb: mb(c.heightmap_bytes),
section_records_mb: mb(c.section_record_bytes),
column_records_mb: mb(c.column_bytes),
bytes: c.total_bytes(),
}
}
}
/// The sections of `worlds` that are also in `others` (by address), and
@@ -69,6 +109,324 @@ fn shared<'a>(
(count, bytes)
}
#[derive(Serialize)]
struct SharedSections {
sections: usize,
mb: f64,
#[serde(skip)]
bytes: usize,
}
#[derive(Serialize)]
struct Generation {
protos: usize,
protos_mb: f64,
proto_blocks_mb: f64,
proto_section_lists_mb: f64,
proto_heightmaps_mb: f64,
proto_other_terrain_mb: f64,
proto_biomes_mb: f64,
proto_records_mb: f64,
proto_sections_by_bits: [usize; 16],
light_views: usize,
light_views_mb: f64,
/// Name, entries and megabytes where the cache can say.
caches: Vec<(&'static str, usize, Option<f64>)>,
stages: Vec<(&'static str, usize)>,
#[serde(skip)]
bytes: usize,
}
impl Generation {
fn of(streaming: &horizon_client::streaming::Streaming) -> Self {
let c = streaming.proto_census();
let [blocks, lists, heightmaps, other, biomes, records] = c.bytes;
let protos: usize = c.bytes.iter().sum();
let cached: usize = c.caches.iter().filter_map(|(_, _, bytes)| *bytes).sum();
Self {
protos: c.protos,
protos_mb: mb(protos),
proto_blocks_mb: mb(blocks),
proto_section_lists_mb: mb(lists),
proto_heightmaps_mb: mb(heightmaps),
proto_other_terrain_mb: mb(other),
proto_biomes_mb: mb(biomes),
proto_records_mb: mb(records),
proto_sections_by_bits: c.sections_by_bits,
light_views: c.light_views,
light_views_mb: mb(c.light_view_bytes),
caches: c
.caches
.iter()
.map(|(name, n, bytes)| (*name, *n, bytes.map(mb)))
.collect(),
stages: streaming.stages().to_vec(),
bytes: protos + c.light_view_bytes + cached,
}
}
}
/// Entities of every level by registry id: how many, and the bytes of their records.
#[derive(Serialize)]
struct Entities {
count: usize,
total_mb: f64,
/// The `TOP` types holding the most bytes: id, count, megabytes.
by_type: Vec<(String, usize, f64)>,
#[serde(skip)]
bytes: usize,
}
/// The connected players: queues and what each is owed or holds, the biggest first.
#[derive(Default, Serialize)]
struct Players {
count: usize,
total_mb: f64,
/// Name, packets and megabytes waiting to be sent.
send_queue: Vec<(String, usize, f64)>,
chunks_waiting: Vec<(String, usize)>,
chunks_sent: Vec<(String, usize)>,
tracked_entities: Vec<(String, usize)>,
#[serde(skip)]
bytes: usize,
}
#[derive(Default, Serialize)]
struct Cached {
entries: usize,
size_mb: f64,
}
/// What the hub keeps to send chunks: encoded sections, heightmaps and compressed frames.
#[derive(Default, Serialize)]
struct ChunkSend {
total_mb: f64,
sections: Cached,
heightmaps: Cached,
frames: Cached,
#[serde(skip)]
bytes: usize,
}
#[derive(Serialize)]
struct Queue {
len: usize,
size_mb: f64,
}
#[derive(Serialize)]
struct Process {
resident_mb: Option<f64>,
allocator_commit_mb: f64,
allocator_peak_commit_mb: f64,
}
#[derive(Serialize)]
struct Allocator {
#[serde(skip_serializing_if = "Option::is_none")]
live_heap_mb: Option<f64>,
commit_mb: f64,
peak_commit_mb: f64,
/// Committed minus the pages in use.
#[serde(skip_serializing_if = "Option::is_none")]
overhead_mb: Option<f64>,
}
#[derive(Serialize)]
struct FarTiles {
active: usize,
cached: usize,
kept: usize,
kept_mb: f64,
}
#[derive(Serialize)]
struct Gpu {
allocated_mb: f64,
reserved_mb: f64,
by_label_mb: Vec<(String, f64)>,
}
#[derive(Serialize)]
struct Renderer {
section_arenas_mb: f64,
section_vertices_mb: f64,
cpu: BTreeMap<String, f64>,
gpu: Option<Gpu>,
}
/// A serving game's own parts: the columns it holds, its generation, entities, players, chunk send caches and queues.
struct ServerSide {
columns: Columns,
addresses: FxHashSet<usize>,
generation: Generation,
entities: Entities,
players: Players,
chunk_send: ChunkSend,
light_queue: Queue,
generation_queue: Queue,
queue_bytes: usize,
}
impl ServerSide {
/// The bytes of every part this side names.
fn bytes(&self) -> usize {
self.columns.bytes
+ self.generation.bytes
+ self.entities.bytes
+ self.players.bytes
+ self.chunk_send.bytes
+ self.queue_bytes
}
}
/// How often a running capture records what holds the memory.
pub(super) const MEMORY_EVERY_S: f64 = 10.0;
/// `horizon-ctl memory`'s report. Every `*_mb` is rounded to a tenth; the parts' bytes are summed unrounded.
#[derive(Serialize)]
pub(super) struct MemoryReport {
process: Process,
allocator: Allocator,
render_distance: i32,
/// The window's columns; a dedicated server has no window and names them `columns`.
#[serde(skip_serializing_if = "Option::is_none")]
client_columns: Option<Columns>,
#[serde(skip_serializing_if = "Option::is_none")]
columns: Option<Columns>,
server_columns: Option<Columns>,
shared_with_server: Option<SharedSections>,
server_generation: Option<Generation>,
entities: Option<Entities>,
players: Option<Players>,
chunk_send: Option<ChunkSend>,
light_queue: Option<Queue>,
generation_queue: Option<Queue>,
far_tiles: Option<FarTiles>,
renderer: Option<Renderer>,
/// A build with the `alloc-census` feature: the live heap by the
/// startup step, span or pool job that allocated it, else its thread.
heap_by_tag_mb: Option<Vec<(String, f64)>>,
/// The columns, generation, entities, players, chunk send caches, queues, renderer CPU data and far tiles together.
named_mb: f64,
/// Resident memory minus `named_mb`: what no census reaches (it can be negative where the parts overlap).
unattributed_mb: Option<f64>,
}
/// `resident - named` in megabytes, None without a resident size.
fn unattributed(resident: Option<u64>, named: usize) -> Option<f64> {
resident.map(|r| mb_signed(r as i64 - named as i64))
}
/// `horizon-ctl memory`'s report as a capture's rows: `a.b` paths, `key[name]` for `[name, value]` lists.
pub(super) fn flatten_report(report: &Value) -> Vec<(String, f64)> {
let mut rows = Vec::new();
flatten(String::new(), report, &mut rows);
rows
}
fn flatten(path: String, value: &Value, rows: &mut Vec<(String, f64)>) {
match value {
Value::Number(n) => rows.extend(n.as_f64().map(|n| (path, n))),
Value::Object(fields) => {
for (key, inner) in fields {
let next = if path.is_empty() {
key.clone()
} else {
format!("{path}.{key}")
};
flatten(next, inner, rows);
}
}
Value::Array(items) => {
for item in items {
let Some([Value::String(name), count, rest @ ..]) =
item.as_array().map(Vec::as_slice)
else {
continue;
};
rows.extend(count.as_f64().map(|n| (format!("{path}[{name}]"), n)));
if let Some(mb) = rest.first().and_then(Value::as_f64) {
rows.push((format!("{path}_mb[{name}]"), mb));
}
}
}
_ => {}
}
}
/// The `limit` rows with the biggest key first, ties by name.
fn biggest<K: Copy + Ord, V>(mut rows: Vec<(String, K, V)>, limit: usize) -> Vec<(String, K, V)> {
rows.sort_by(|a, b| b.1.cmp(&a.1).then_with(|| a.0.cmp(&b.0)));
rows.truncate(limit);
rows
}
/// The hub's players: what waits in each one's send queue and what the hub tracks for it.
fn player_census(hub: &Hub) -> Players {
let set = |len: usize| len * (size_of::<(i32, i32)>() + 1);
let mut rows = Vec::new();
let mut bytes = 0usize;
for g in &hub.guests {
let (packets, queued) = g.sender.queued();
bytes += queued
+ set(g.sent_chunks.capacity())
+ set(g.far_sent_chunks.capacity())
+ vec_bytes(&g.chunk_queue)
+ vec_bytes(&g.far_queue)
+ g.tracker.heap_bytes();
rows.push((
g.name.clone(),
[
queued,
packets,
g.chunk_queue.len() + g.far_queue.len(),
g.sent_chunks.len() + g.far_sent_chunks.len(),
g.tracker.len(),
],
));
}
let top = |by: usize| {
let ranked: Vec<_> = rows
.iter()
.map(|(name, v)| (name.clone(), v[by], v[0]))
.collect();
biggest(ranked, TOP_PLAYERS)
};
Players {
count: hub.guests.len(),
total_mb: mb(bytes),
send_queue: top(0)
.into_iter()
.map(|(name, queued, _)| {
let packets = rows.iter().find(|r| r.0 == name).map_or(0, |r| r.1[1]);
(name, packets, mb(queued))
})
.collect(),
chunks_waiting: top(2).into_iter().map(|(n, v, _)| (n, v)).collect(),
chunks_sent: top(3).into_iter().map(|(n, v, _)| (n, v)).collect(),
tracked_entities: top(4).into_iter().map(|(n, v, _)| (n, v)).collect(),
bytes,
}
}
fn chunk_send_census(hub: &Hub) -> ChunkSend {
let [sections, heights] = hub.encoder.cache_census();
let frames = horizon_client::lan::conn::shared_frame_census();
let cached = |(entries, bytes): (usize, usize)| Cached {
entries,
size_mb: mb(bytes),
};
let bytes = sections.1 + heights.1 + frames.1;
ChunkSend {
total_mb: mb(bytes),
sections: cached(sections),
heightmaps: cached(heights),
frames: cached(frames),
bytes,
}
}
impl Game<'_> {
fn all_worlds(&self) -> impl Iterator<Item = &World> {
std::iter::once(&self.world).chain(self.levels.parked_worlds())
@@ -85,95 +443,326 @@ impl Game<'_> {
.collect()
}
pub(super) fn memory_report(&self) -> Value {
/// The entities of every level by type; a record's bytes are its struct and the buffers its fields own.
fn entity_census(&self) -> Entities {
fn size<T>(_: &[T]) -> usize {
size_of::<T>()
}
let record = [
size(self.dropped.as_slice()),
size(self.orbs.as_slice()),
size(self.creatures.projectiles.as_slice()),
size(self.carts.as_slice()),
size(self.boats.as_slice()),
];
let mut by_type: BTreeMap<String, (usize, usize)> = BTreeMap::new();
let mut level = |mobs: &[horizon_client::mobs::Mob], loose: [usize; 5]| {
for m in mobs.iter().filter(|m| !m.removed) {
let bytes = size_of::<horizon_client::mobs::Mob>()
+ size_of_val(&*m.ai)
+ vec_bytes(&m.outbox)
+ string_bytes(&m.custom_name)
+ string_bytes(&m.standing_on)
+ string_bytes(&m.skin)
+ string_bytes(&m.cape);
let row = by_type.entry(m.kind.id.clone()).or_default();
row.0 += 1;
row.1 += bytes;
}
let loose = super::dedicated::report::LOOSE_ENTITIES
.into_iter()
.zip(loose)
.zip(record);
for ((id, n), size) in loose.filter(|((_, n), _)| *n > 0) {
let row = by_type.entry(id.to_owned()).or_default();
row.0 += n;
row.1 += n * size;
}
};
level(
&self.mobs,
[
self.dropped.len(),
self.orbs.len(),
self.creatures.projectiles.len(),
self.carts.len(),
self.boats.len(),
],
);
for dim in self.levels.parked_dims() {
if let Some(mobs) = self.levels.parked_mobs(dim) {
let loose = self.levels.parked_entity_counts(dim).unwrap_or_default();
level(mobs, loose);
}
}
let count = by_type.values().map(|r| r.0).sum();
let bytes = by_type.values().map(|r| r.1).sum();
let rows = by_type.into_iter().map(|(id, (n, b))| (id, b, n)).collect();
Entities {
count,
total_mb: mb(bytes),
by_type: biggest(rows, TOP)
.into_iter()
.map(|(id, b, n)| (id, n, mb(b)))
.collect(),
bytes,
}
}
/// This game's own server parts: it is a dedicated server, or the single-player server thread.
fn server_side(&self) -> ServerSide {
let (players, chunk_send) = match &self.lan {
Some(host) => (player_census(&host.hub), chunk_send_census(&host.hub)),
None => (Players::default(), ChunkSend::default()),
};
let (mut generation, mut light) = {
let q = self.streaming.queue_census();
(q.generation, q.light)
};
for streaming in self.levels.parked_streamings() {
let q = streaming.queue_census();
generation = (generation.0 + q.generation.0, generation.1 + q.generation.1);
light = (light.0 + q.light.0, light.1 + q.light.1);
}
ServerSide {
columns: Columns::new(&self.level_census()),
addresses: self.section_addresses(),
generation: Generation::of(&self.streaming),
entities: self.entity_census(),
players,
chunk_send,
light_queue: Queue {
len: light.0,
size_mb: mb(light.1),
},
generation_queue: Queue {
len: generation.0,
size_mb: mb(generation.1),
},
queue_bytes: generation.1 + light.1,
}
}
pub(super) fn memory_report(&self) -> MemoryReport {
let (commit, peak_commit) = horizon_threads::allocator_commit();
let client = self.level_census();
let live = horizon_threads::allocator_stats::live_bytes();
let resident = horizon_threads::process_memory::resident_bytes();
let dedicated = self.options.dedicated.is_some();
// The single-player server thread's own levels: a second copy of the
// columns both hold, unless they share sections.
let server = self
.integrated
.as_ref()
.and_then(|i| i.call(|g| (g.level_census(), g.section_addresses())));
// The server thread's generation: proto chunks waiting for their
// neighbours, and the light jobs' views of them.
let generation = self.integrated.as_ref().and_then(|i| {
i.call(|g| {
let c = g.streaming.proto_census();
let [blocks, lists, heightmaps, other, biomes, records] = c.bytes;
json!({
"protos": c.protos,
"protos_mb": mb(c.bytes.iter().sum::<usize>()),
"proto_blocks_mb": mb(blocks),
"proto_section_lists_mb": mb(lists),
"proto_heightmaps_mb": mb(heightmaps),
"proto_other_terrain_mb": mb(other),
"proto_biomes_mb": mb(biomes),
"proto_records_mb": mb(records),
"proto_sections_by_bits": c.sections_by_bits,
"light_views": c.light_views,
"light_views_mb": mb(c.light_view_bytes),
"caches": c.caches.iter().map(|(name, n, bytes)| json!([name, n, bytes.map(mb)])).collect::<Vec<_>>(),
"stages": g.streaming.stages().iter().map(|(k, v)| json!([k, v])).collect::<Vec<_>>(),
})
})
});
let server = if dedicated {
Some(self.server_side())
} else {
self.integrated
.as_ref()
.and_then(|i| i.call(|g| g.server_side()))
};
let window = (!dedicated).then(|| self.level_census());
// The window's sections the server thread holds too (lan/handoff.rs).
let shared = server.as_ref().map(|(_, at)| {
let (sections, bytes) = shared(self.all_worlds(), at);
json!({ "sections": sections, "mb": mb(bytes) })
let shared = server.as_ref().filter(|_| !dedicated).map(|s| {
let (sections, bytes) = shared(self.all_worlds(), &s.addresses);
SharedSections {
sections,
mb: mb(bytes),
bytes,
}
});
let renderer = self.renderer.as_ref().map(|r| {
let [arenas, vertices] = r.section_vertex_bytes();
let cpu: serde_json::Map<String, Value> = r
.cpu_census()
.into_iter()
.map(|(k, v)| (format!("{k}_mb"), mb(v).into()))
.collect();
let cpu = r.cpu_census();
let cpu_bytes: usize = cpu.iter().map(|(_, v)| *v).sum();
let gpu = r
.gpu_memory_by_label()
.map(|(allocated, reserved, labels)| {
json!({
"allocated_mb": mb(allocated),
"reserved_mb": mb(reserved),
"by_label_mb": labels
.iter()
.take(40)
.map(|(label, bytes)| json!([label, mb(*bytes)]))
.collect::<Vec<_>>(),
})
.map(|(allocated, reserved, labels)| Gpu {
allocated_mb: mb(allocated),
reserved_mb: mb(reserved),
by_label_mb: labels
.iter()
.take(40)
.map(|(label, bytes)| (label.to_string(), mb(*bytes)))
.collect(),
});
json!({
"section_arenas_mb": mb(arenas),
"section_vertices_mb": mb(vertices),
"cpu": cpu,
"gpu": gpu,
})
let renderer = Renderer {
section_arenas_mb: mb(arenas),
section_vertices_mb: mb(vertices),
cpu: cpu
.into_iter()
.map(|(k, v)| (format!("{k}_mb"), mb(v)))
.collect(),
gpu,
};
(renderer, cpu_bytes)
});
json!({
"process": {
"resident_mb": horizon_threads::process_memory::resident_bytes().map(mb),
"allocator_commit_mb": mb(commit),
"allocator_peak_commit_mb": mb(peak_commit),
let far = self.lod.as_ref().map(|lod| {
let ((active, cached), (kept, kept_bytes)) = (lod.stats(), lod.kept());
let tiles = FarTiles {
active,
cached,
kept,
kept_mb: mb(kept_bytes),
};
(tiles, kept_bytes)
});
let named = window.as_ref().map_or(0, ColumnCensus::total_bytes)
+ server.as_ref().map_or(0, ServerSide::bytes)
+ renderer.as_ref().map_or(0, |r| r.1)
+ far.as_ref().map_or(0, |f| f.1);
let named = named.saturating_sub(shared.as_ref().map_or(0, |s| s.bytes));
let mut report = MemoryReport {
process: Process {
resident_mb: resident.map(mb),
allocator_commit_mb: mb(commit),
allocator_peak_commit_mb: mb(peak_commit),
},
"render_distance": self.streaming.radius,
"client_columns": census_json(&client),
"server_columns": server.as_ref().map(|(c, _)| census_json(c)),
"shared_with_server": shared,
"server_generation": generation,
"far_tiles": self.lod.as_ref().map(|lod| {
let (active, cached) = lod.stats();
json!({ "active": active, "cached": cached })
}),
"renderer": renderer,
// A build with the `alloc-census` feature: the live heap by the
// startup step, span or pool job that allocated it, else its thread.
"heap_by_tag_mb": horizon_threads::HEAP_CENSUS.then(|| {
allocator: Allocator {
live_heap_mb: live.map(mb),
commit_mb: mb(commit),
peak_commit_mb: mb(peak_commit),
overhead_mb: live.map(|live| mb(commit.saturating_sub(live))),
},
render_distance: self.streaming.radius,
client_columns: window.as_ref().map(Columns::new),
columns: None,
server_columns: None,
shared_with_server: shared,
server_generation: None,
entities: None,
players: None,
chunk_send: None,
light_queue: None,
generation_queue: None,
far_tiles: far.map(|f| f.0),
renderer: renderer.map(|r| r.0),
heap_by_tag_mb: horizon_threads::HEAP_CENSUS.then(|| {
horizon_threads::heap_census::census()
.into_iter()
.take(60)
.map(|(tag, bytes)| json!([tag, mb(bytes.max(0) as u64)]))
.collect::<Vec<_>>()
.map(|(tag, bytes)| (tag.to_string(), mb(bytes.max(0) as u64)))
.collect()
}),
})
named_mb: mb(named),
unattributed_mb: unattributed(resident, named),
};
if let Some(s) = server {
if dedicated {
report.columns = Some(s.columns);
} else {
report.server_columns = Some(s.columns);
}
report.server_generation = Some(s.generation);
report.entities = Some(s.entities);
report.players = Some(s.players);
report.chunk_send = Some(s.chunk_send);
report.light_queue = Some(s.light_queue);
report.generation_queue = Some(s.generation_queue);
}
report
}
/// The report as JSON, for the control socket and the capture's rows.
pub(super) fn memory_value(&self) -> Value {
serde_json::to_value(self.memory_report()).unwrap_or(Value::Null)
}
}
#[cfg(test)]
mod tests {
use super::{biggest, flatten_report, unattributed};
use serde_json::json;
#[test]
fn unattributed_is_resident_minus_the_named_bytes_and_may_be_negative() {
let mib = 1 << 20;
assert_eq!(
unattributed(Some(900 * mib), 600 * mib as usize),
Some(300.0)
);
assert_eq!(unattributed(Some(10 * mib), 25 * mib as usize), Some(-15.0));
assert_eq!(unattributed(None, 5), None);
}
#[test]
fn the_biggest_rows_come_first_with_ties_by_name() {
let rows = vec![
("b".to_string(), 2, ()),
("a".to_string(), 2, ()),
("c".to_string(), 9, ()),
];
let names: Vec<_> = biggest(rows, 2).into_iter().map(|r| r.0).collect();
assert_eq!(names, ["c", "a"]);
}
#[test]
#[ignore = "needs prepared resources in HORIZON_TEST_RESOURCES"]
fn a_servers_report_names_its_entities_and_players_and_leaves_the_rest_unattributed() {
use crate::game::test_pos_guest_tests::{GUEST, game};
use horizon_client::mobs::Mob;
let mut game = game();
game.options.dedicated = Some(Vec::new());
for (id, kind) in [(1, "cow"), (2, "cow"), (3, "pig")] {
let kind = game
.res
.mob_types()
.get(&format!("minecraft:{kind}"))
.unwrap()
.clone();
game.mobs.push(Mob::new(
id,
kind,
horizon_core::geometry::Vec3::new(0.5, 64.0, 0.5),
0.0,
));
}
game.lan.as_ref().unwrap().hub.guests[0]
.sender
.send(vec![0; 3 << 20]);
let report = game.memory_report();
let by_type: std::collections::BTreeMap<_, _> = report
.entities
.as_ref()
.unwrap()
.by_type
.iter()
.map(|(id, n, _)| (id.as_str(), *n))
.collect();
assert_eq!((by_type["minecraft:cow"], by_type["minecraft:pig"]), (2, 1));
let players = report.players.as_ref().unwrap();
assert_eq!(players.count, 1);
assert_eq!(players.send_queue[0].0, GUEST);
assert_eq!(players.send_queue[0].2, 3.0);
assert!(report.client_columns.is_none() && report.columns.is_some());
let resident = report.process.resident_mb.unwrap();
let rest = report.unattributed_mb.unwrap();
assert!(
(rest - (resident - report.named_mb)).abs() <= 0.15,
"{rest} {resident} {}",
report.named_mb
);
}
#[test]
fn a_report_flattens_to_paths_and_named_list_entries() {
let report = json!({
"process": { "resident_mb": 12.5, "note": "skipped" },
"render_distance": 16,
"missing": null,
"by_label_mb": [["chunks", 3.0], ["bare"]],
"caches": [["blocks", 7, 1.5]],
});
let mut rows = flatten_report(&report);
rows.sort_by(|a, b| a.0.cmp(&b.0));
let expected = [
("by_label_mb[chunks]", 3.0),
("caches[blocks]", 7.0),
("caches_mb[blocks]", 1.5),
("process.resident_mb", 12.5),
("render_distance", 16.0),
];
assert_eq!(
rows.iter()
.map(|(p, v)| (p.as_str(), *v))
.collect::<Vec<_>>(),
expected
);
}
}
@@ -1914,6 +1914,7 @@ impl Game<'_> {
continue;
}
let kind = self.carts[i].kind.id();
horizon_threads::work::set_entity(horizon_threads::work::entity_id(kind));
if !self.native_tick_pre_one(&mut round, id, kind) {
continue;
}
@@ -1952,6 +1953,7 @@ impl Game<'_> {
// After the ticked cart is stored: a mod reads and changes the cart as it is now.
self.native_tick_post_one(&round, id, kind);
}
horizon_threads::work::clear_work();
// On the server for every cart, the ridden ones too.
let mut hint = 0;
for id in ids {
@@ -8026,7 +8026,7 @@ impl Game<'_> {
mod_api::dispatch_server_packets(self, mod_api::mods(), player, packets, outbound);
}
/// After each pass of the server tick (paused ones too) while its spans
/// are recorded: always on a dedicated server, whose slow-tick log keeps
/// are recorded: always on a dedicated server, whose slow ring keeps
/// them on, elsewhere while `native_spans(true)`, `HORIZON_SPIKES` or a
/// performance report asks for them. `wall_ms` is the pass's wall time,
/// `cpu_ms` the server thread's CPU time in it (None where the system
@@ -918,6 +918,7 @@ impl<'a> Game<'a> {
}
spawned
};
horizon_threads::work::set_chunk(chunk.0, chunk.1);
spawn_selection::spawn_for_chunk(
&level,
&mut random,
@@ -931,6 +932,7 @@ impl<'a> Game<'a> {
}
scratch.spawn_players = state.into_players();
}
horizon_threads::work::clear_work();
spawn_selection::attempt_counters(true);
laps.lap("sn_attempts");
self.world.random = random;
+453
View File
@@ -0,0 +1,453 @@
//! `profile` against the server this window is connected to (docs/perf-platform.md §4.2): the request over `horizon:perf`, its answer, and the warnings a refusal leaves in the client's capture.
use super::Game;
use super::profile_capture::{reply_text, reports_dir, shown};
use crate::Result;
use horizon_client::client_perf::request::{
Failed, GpuSwitch, ProfileSide, StartOptions, Started, Stopped,
};
use horizon_client::perf_net::{CHANNEL, Outcome, Refusal, Reply, Request, Unsent, Wanted};
use horizon_perf_format::{FieldSender, PartMerger, RawSend, files, v1::WarningCode};
use horizon_protocol::packets::{configuration::CustomPayload, play::Serverbound as S};
use std::fs::File;
use std::io::Write;
use std::path::PathBuf;
use std::sync::mpsc::Sender;
use std::time::{Instant, SystemTime, UNIX_EPOCH};
const MARK_SERVER_STARTED: &str = "server_started";
fn refusal_text(why: Refusal) -> String {
let (key, default) = match why {
Refusal::Denied => (
"horizon.perf.denied",
"a server profile needs operator level 4",
),
Refusal::AlreadyRunning => (
"horizon.perf.already_running",
"the server is already running a profile",
),
Refusal::NotRunning => (
"horizon.perf.not_running",
"no server profile of yours is running",
),
Refusal::Failed => (
"horizon.perf.failed",
"the server could not write the profile; see its log",
),
};
horizon_client::lang::get_or(key, default)
}
fn not_horizon_text() -> String {
horizon_client::lang::get_or(
"horizon.perf.not_horizon",
"this server isn't running Horizon",
)
}
fn unsent_text(why: Unsent) -> String {
match why {
Unsent::NotHorizon => not_horizon_text(),
Unsent::Busy => "a server profile is already running or still being saved".to_owned(),
Unsent::NothingToStop => "no server profile is running".to_owned(),
}
}
fn warning_of(why: Refusal) -> Option<WarningCode> {
match why {
Refusal::Denied => Some(WarningCode::ServerDenied),
Refusal::AlreadyRunning => Some(WarningCode::ServerAlreadyRunning),
Refusal::NotRunning | Refusal::Failed => None,
}
}
/// The window capture's file, which the server's part is written into.
pub(super) struct Sink {
pub(super) sender: FieldSender,
pub(super) origin: Instant,
}
impl Sink {
/// Closes the file's server side as far as it got, and records why it is short.
fn incomplete(&self, merger: &PartMerger) {
for field in merger.cut_short(self.origin.elapsed().as_secs_f64()) {
self.sender.send(field);
}
}
}
enum Dest {
/// Appended to the window's file, whole fields at a time; `held` waits for room in the writer's channel.
Window { sink: Sink, held: Vec<u8> },
/// Saved as it came, as a file of its own.
Own { file: File, path: PathBuf },
}
/// The server's file as it arrives.
pub(super) struct Incoming {
merger: PartMerger,
dest: Dest,
reply: Option<Sender<String>>,
/// The last chunk has arrived and was checked whole.
whole: bool,
}
fn part_text() -> String {
horizon_client::lang::get_or(
"horizon.perf.part_incomplete",
"the server's part of the profile did not arrive whole",
)
}
fn unix_secs() -> i64 {
SystemTime::now()
.duration_since(UNIX_EPOCH)
.map_or(0, |d| d.as_secs() as i64)
}
impl Game<'_> {
/// Whether this window is a client of a server it does not run itself.
pub(super) fn profiles_remote_server(&self) -> bool {
self.integrated.is_none() && self.remote.is_some()
}
fn server_brand(&self) -> &str {
self.remote.as_ref().map_or("", |r| r.brand.as_str())
}
fn send_perf(&self, request: &Request) {
self.send(S::CustomPayload(CustomPayload {
channel: CHANNEL.into(),
data: request.encode(),
}));
}
/// `profile server start` on a server: the answer comes back through `reply`.
pub(super) fn start_remote_only(
&mut self,
options: &StartOptions,
reply: &Sender<String>,
) -> Result<Option<serde_json::Value>> {
if !self.profiles_remote_server() {
return Err("profile server: no server to reach; open a world or join a server".into());
}
if self.profile.perf.capturing() || self.profile.perf.saving() {
return Err("a capture is already running or still being saved".into());
}
let brand = self.server_brand().to_owned();
self.profile
.server_link
.ask(&brand, Instant::now())
.map_err(unsent_text)?;
self.send_perf(&Request::Start(Wanted::from(options)));
self.profile.server_reply = Some(reply.clone());
self.profile.server_joined = false;
Ok(None)
}
/// A window capture on a server also asks the server for its side; a refusal is a warning in the file.
pub(super) fn join_remote_server(&mut self, options: &StartOptions) {
if !self.profiles_remote_server() {
return;
}
let brand = self.server_brand().to_owned();
match self.profile.server_link.ask(&brand, Instant::now()) {
Ok(()) => {
self.send_perf(&Request::Start(Wanted::from(options)));
self.profile.server_joined = true;
}
Err(Unsent::NotHorizon) => self.warn_server(WarningCode::ServerNotHorizon),
Err(_) => {}
}
}
fn warn_server(&mut self, code: WarningCode) {
let t = self.profile.perf.now();
self.profile.perf.warn(t, code);
}
/// `profile server stop` on a server: the file's name comes back through `reply`.
pub(super) fn stop_remote_only(&mut self, reply: &Sender<String>) -> Result<()> {
self.profile
.server_link
.stop(Instant::now())
.map_err(unsent_text)?;
self.send_perf(&Request::Stop);
self.profile.server_reply = Some(reply.clone());
Ok(())
}
/// The window capture ends: so does the server's side that asked with it.
pub(super) fn stop_joined_server(&mut self) {
if self.profile.server_joined && self.profile.server_link.stop(Instant::now()).is_ok() {
self.send_perf(&Request::Stop);
}
}
/// A `horizon:perf` payload from the server.
pub(super) fn remote_perf_reply(&mut self, data: &[u8]) {
let Ok(reply) = Reply::decode(data) else {
log::debug!("Horizon: a bad server profile reply");
return;
};
if let Reply::Chunk(chunk) = reply {
self.part_chunk(&chunk);
return;
}
let waiting = self.profile.server_reply.take();
let joined = self.profile.server_joined;
match self.profile.server_link.answer(reply, Instant::now()) {
Outcome::Started {
path,
seconds,
samples,
} => {
if joined {
let t = self.profile.perf.now();
self.profile.perf.mark(t, MARK_SERVER_STARTED);
self.profile.server_sink = self.profile.perf.join().map(|j| Sink {
sender: j.sender,
origin: j.origin,
});
}
if let Some(waiting) = waiting {
let _ = waiting.send(reply_text(&Started {
ok: true,
sides: vec![ProfileSide::Server],
path,
seconds,
samples,
gpu: GpuSwitch::Off,
}));
}
}
Outcome::Finished { path, bytes, .. } => {
log::info!("Horizon: the server saved its profile to {path}");
self.profile.server_joined = false;
if bytes == 0 {
self.profile.server_reply = waiting;
self.abandon_part();
} else {
self.begin_part(waiting);
}
}
Outcome::TooLarge { bytes } => {
log::warn!("Horizon: the server announced a {bytes}-byte profile; it is not taken");
self.profile.server_joined = false;
self.profile.server_reply = waiting;
self.abandon_part();
}
Outcome::Refused(why) => {
if joined {
if let Some(code) = warning_of(why) {
self.warn_server(code);
}
self.profile.server_joined = false;
self.profiler_client_only(&refusal_text(why));
}
if self.profile.incoming.is_some() || self.profile.server_sink.is_some() {
self.profile.server_reply = waiting;
self.abandon_part();
return;
}
log::warn!(
"Horizon: the server refused the profile: {}",
refusal_text(why)
);
if let Some(waiting) = waiting {
let _ = waiting.send(reply_text(&Failed {
error: refusal_text(why),
}));
}
}
Outcome::Stray => self.profile.server_reply = waiting,
}
}
/// The server's file is about to arrive: into the window's capture if it still holds the file open, else into one of its own.
fn begin_part(&mut self, waiting: Option<Sender<String>>) {
let dest = match self.profile.server_sink.take() {
Some(sink) => Ok(Dest::Window {
sink,
held: Vec::new(),
}),
None => files::create(&reports_dir(), unix_secs())
.map(|(file, path)| Dest::Own { file, path }),
};
match dest {
Ok(dest) => {
self.profile.incoming = Some(Incoming {
merger: PartMerger::new(),
dest,
reply: waiting,
whole: false,
});
}
Err(e) => {
log::warn!("Horizon: the server's profile cannot be saved: {e}");
self.profile.server_link.reset();
if let Some(waiting) = waiting {
let _ = waiting.send(reply_text(&Failed {
error: e.to_string(),
}));
}
}
}
}
/// The next bytes of the server's file.
fn part_chunk(&mut self, chunk: &[u8]) {
use horizon_client::perf_net::Received;
if self.profile.incoming.is_none() {
return;
}
let received = self.profile.server_link.chunk(chunk.len(), Instant::now());
let step = match received {
Received::Stray => return,
Received::TooMuch => Err("the server sent more than it announced".to_owned()),
Received::More | Received::Last => self.take_part(chunk, received == Received::Last),
};
if let Err(why) = step {
log::warn!("Horizon: the server's profile part is unusable: {why}");
self.abandon_part();
}
}
fn take_part(&mut self, chunk: &[u8], last: bool) -> std::result::Result<(), String> {
let Some(incoming) = &mut self.profile.incoming else {
return Ok(());
};
let fields = incoming.merger.push(chunk).map_err(|e| e.to_string())?;
match &mut incoming.dest {
Dest::Window { held, .. } => held.extend_from_slice(&fields),
Dest::Own { file, .. } => file.write_all(chunk).map_err(|e| e.to_string())?,
}
if last {
incoming.merger.finish().map_err(|e| e.to_string())?;
incoming.whole = true;
}
self.settle_part()
}
/// Hands the window's writer what the server's file has yielded, and completes the transfer once all of it is in.
fn settle_part(&mut self) -> std::result::Result<(), String> {
let Some(incoming) = &mut self.profile.incoming else {
return Ok(());
};
if let Dest::Window { sink, held } = &mut incoming.dest
&& !held.is_empty()
{
match sink.sender.try_send_raw(std::mem::take(held)) {
RawSend::Sent => {}
RawSend::Full(back) => *held = back,
RawSend::Closed => return Err("the capture's file is closed".to_owned()),
}
}
let drained = match &incoming.dest {
Dest::Window { held, .. } => held.is_empty(),
Dest::Own { .. } => true,
};
if incoming.whole && drained {
self.part_done();
}
Ok(())
}
/// The server's file is whole: the window's capture can finish, or the file of its own is kept.
fn part_done(&mut self) {
let Some(Incoming { dest, reply, .. }) = self.profile.incoming.take() else {
return;
};
let Dest::Own { file, path } = dest else {
log::info!("Horizon: the server's profile is part of this capture");
return;
};
let kept = file.sync_all().and_then(|()| {
let dir = path.parent().map(PathBuf::from).unwrap_or_default();
files::retain(&dir, &path, files::KEEP_FILES, files::KEEP_BYTES)
});
let text = match kept {
Ok(deleted) => {
log::info!("Horizon: {}", files::saved_line(&path));
reply_text(&Stopped {
ok: true,
sides: vec![ProfileSide::Server],
path: shown(&path),
open_at: horizon_client::client_perf::request::viewer(),
removed: deleted.iter().map(|p| shown(p)).collect(),
})
}
Err(e) => reply_text(&Failed {
error: e.to_string(),
}),
};
if let Some(reply) = reply {
let _ = reply.send(text);
}
}
/// The server's part will not arrive whole: what did arrive stays in the window's capture, with the warning, or the file of its own is removed.
pub(super) fn abandon_part(&mut self) {
let waiting = self.profile.server_reply.take();
let sink = self.profile.server_sink.take();
if let Some(Incoming {
dest,
reply,
merger,
..
}) = self.profile.incoming.take()
{
match dest {
Dest::Window { sink, held } => {
if !held.is_empty() {
let _ = sink.sender.try_send_raw(held);
}
sink.incomplete(&merger);
}
Dest::Own { file, path } => {
drop(file);
let _ = std::fs::remove_file(path);
if let Some(reply) = reply {
let _ = reply.send(reply_text(&Failed { error: part_text() }));
}
}
}
}
if let Some(sink) = sink {
sink.incomplete(&PartMerger::new());
}
if let Some(waiting) = waiting {
let _ = waiting.send(reply_text(&Failed { error: part_text() }));
}
}
/// Once per loop turn: a request the server never answered, or a connection that ended under one.
pub(super) fn poll_server_link(&mut self) {
if self.profile.incoming.is_some() && self.settle_part().is_err() {
self.abandon_part();
}
let now = Instant::now();
let lost = self.remote.is_none() && !self.profile.server_link.is_idle();
let silent = self.profile.server_link.timed_out(now);
let stalled = self.profile.server_link.stalled(now);
if !lost && !silent && !stalled {
return;
}
if lost {
self.profile.server_link.reset();
}
if lost || stalled {
self.abandon_part();
}
self.profile.server_joined = false;
if let Some(waiting) = self.profile.server_reply.take() {
let error = if lost {
"the connection to the server ended".to_owned()
} else {
"the server did not answer; it may not run a Horizon with server profiles"
.to_owned()
};
let _ = waiting.send(reply_text(&Failed { error }));
}
}
}
@@ -0,0 +1,280 @@
//! A guest's `horizon:perf` requests (docs/perf-platform.md §4.2): who may profile this server, the one capture at a time, and the log lines about it.
use super::Game;
use super::integrated::capture::{Live, Target};
use super::lan::LanHost;
use super::profile_capture::{reports_dir, shown};
use horizon_client::perf_net::{
CHANNEL, Held, OPERATOR_LEVEL, Outgoing, Refusal, Reply, Request, Verdict, Wanted, judge,
};
use horizon_perf_format::v1::{StopReason, Trigger};
use horizon_protocol::packets::{configuration::CustomPayload, play};
use std::fs::File;
use std::io::Read;
use std::path::PathBuf;
use std::sync::atomic::Ordering;
use std::time::Instant;
/// The guest whose capture runs on this server.
pub(in crate::game) struct Requester {
conn: u32,
name: String,
/// Their connection ended: the capture stops and its file stays on the server.
left: bool,
}
/// A request that passed the checks and waits for the tick loop that owns the capture state.
pub(in crate::game) enum Asked {
Start {
conn: u32,
name: String,
wanted: Wanted,
},
Stop(StopReason),
}
/// What a guest has asked of this server, for the tick loop.
#[derive(Default)]
pub(in crate::game) struct Served {
requester: Option<Requester>,
asked: Vec<Asked>,
sending: Vec<Sending>,
}
/// A finished capture's file on its way to the guest that asked for it.
struct Sending {
conn: u32,
file: File,
pace: Outgoing,
}
/// How a server capture's file ended.
pub(in crate::game) enum Done {
Saved {
path: String,
file: PathBuf,
removed: Vec<String>,
},
Failed,
}
fn payload(reply: &Reply) -> play::Clientbound {
play::Clientbound::CustomPayload(CustomPayload {
channel: CHANNEL.into(),
data: reply.encode(),
})
}
fn window(wanted: &Wanted) -> String {
wanted
.seconds
.map_or_else(|| "until stopped".to_owned(), |s| format!("{s} s"))
}
impl Game<'_> {
/// A `horizon:perf` payload from `conn`.
pub(super) fn guest_perf(&mut self, host: &mut LanHost, conn: u32, data: &[u8]) {
let Some(guest) = host.hub.guest(conn) else {
return;
};
let name = guest.name.clone();
let Ok(request) = Request::decode(data) else {
log::debug!("Horizon: a bad server profile request from {name}");
return;
};
let served = &mut self.profile.served;
let held = match &served.requester {
Some(r) if r.conn == conn => Held::Asker,
Some(_) => Held::Someone,
None if served
.asked
.iter()
.any(|a| matches!(a, Asked::Start { .. })) =>
{
Held::Someone
}
None => Held::Nothing,
};
match judge(&guest.brand, guest.permission, &request, held) {
Verdict::Ignore => {}
Verdict::Refuse(why) => {
if why == Refusal::Denied {
log::warn!(
"Horizon: {name} asked for a server profile and was refused: operator level {OPERATOR_LEVEL} is required"
);
}
guest.send(&payload(&Reply::Refused(why)));
}
Verdict::Accept => served.asked.push(match request {
Request::Start(wanted) => Asked::Start { conn, name, wanted },
Request::Stop => Asked::Stop(StopReason::Normal),
}),
}
}
/// Sends `reply` to the guest on `conn`, if it is still here.
pub(in crate::game) fn perf_reply(&self, conn: u32, reply: &Reply) {
if let Some(guest) = self.lan.as_ref().and_then(|h| h.hub.guest(conn)) {
guest.send(&payload(reply));
}
}
/// What guests asked since the last tick, and a stop when the guest whose capture runs has left.
pub(in crate::game) fn take_perf_asked(&mut self) -> Vec<Asked> {
let served = &mut self.profile.served;
let mut asked = std::mem::take(&mut served.asked);
if let Some(r) = served.requester.as_mut()
&& !r.left
&& self
.lan
.as_ref()
.is_some_and(|h| h.hub.guest(r.conn).is_none())
{
r.left = true;
log::info!("Horizon: {} left; their server profile stops", r.name);
asked.push(Asked::Stop(StopReason::Disconnected));
}
asked
}
/// The capture `name` asked for runs, in the file at `path`.
pub(in crate::game) fn perf_started(
&mut self,
conn: u32,
name: String,
wanted: &Wanted,
path: &str,
) {
log::info!(
"Horizon: {name} started a server profile ({}, samples {})",
window(wanted),
if wanted.samples { "on" } else { "off" }
);
self.profile.served.requester = Some(Requester {
conn,
name,
left: false,
});
self.perf_reply(
conn,
&Reply::Started {
path: path.to_owned(),
seconds: wanted.seconds,
samples: wanted.samples,
},
);
}
/// The capture `conn` asked for could not begin.
pub(in crate::game) fn perf_refused(&self, conn: u32, why: Refusal) {
self.perf_reply(conn, &Reply::Refused(why));
}
/// Server captures whose files are done: the guest that asked is told, and the log names it.
pub(in crate::game) fn perf_captures_done(&mut self, done: Vec<Done>) {
for done in done {
let Some(Requester { conn, name, left }) = self.profile.served.requester.take() else {
continue;
};
if left {
if let Done::Saved { path, .. } = &done {
log::info!("Horizon: {name}'s server profile saved to {path}");
}
continue;
}
match done {
Done::Saved {
path,
file,
removed,
} => {
log::info!("Horizon: {name}'s server profile saved to {path}");
let (bytes, sending) = match File::open(&file).and_then(|f| {
let len = f.metadata()?.len();
Ok((f, len))
}) {
Ok((file, len)) => (
len,
Some(Sending {
conn,
file,
pace: Outgoing::new(len, Instant::now()),
}),
),
Err(e) => {
log::warn!("Horizon: {name}'s server profile cannot be sent: {e}");
(0, None)
}
};
self.perf_reply(
conn,
&Reply::Finished {
path,
removed,
bytes,
},
);
self.profile.served.sending.extend(sending);
}
Done::Failed => {
log::error!("Horizon: {name}'s server profile could not be written");
self.perf_refused(conn, Refusal::Failed);
}
}
}
}
/// Sends the next chunk of each file that is due, at most [`horizon_client::perf_net::RATE`] a second and not while the guest has a backlog.
pub(in crate::game) fn pump_perf_files(&mut self) {
let now = Instant::now();
let mut sending = std::mem::take(&mut self.profile.served.sending);
sending.retain_mut(|s| {
let Some(guest) = self.lan.as_ref().and_then(|h| h.hub.guest(s.conn)) else {
return false;
};
let queued = guest.sender.pending.load(Ordering::Relaxed);
let Some(n) = s.pace.next(now, queued) else {
return !s.pace.done();
};
let mut chunk = vec![0; n];
match s.file.read_exact(&mut chunk) {
Ok(()) => guest.send(&payload(&Reply::Chunk(chunk))),
Err(e) => {
log::warn!("Horizon: a server profile could not be read for sending: {e}");
guest.send(&payload(&Reply::Refused(Refusal::Failed)));
return false;
}
}
!s.pace.done()
});
sending.append(&mut self.profile.served.sending);
self.profile.served.sending = sending;
}
/// On the server thread, between two ticks: begins and ends the captures guests asked for.
pub(super) fn serve_perf_asked(&mut self) {
self.pump_perf_files();
for asked in self.take_perf_asked() {
match asked {
Asked::Start { conn, name, wanted } => {
if self.server_capture_busy() {
self.perf_refused(conn, Refusal::AlreadyRunning);
continue;
}
let target = Target::Own {
options: wanted.options(),
trigger: Trigger::Remote,
dir: reports_dir(),
};
match self.start_server_capture(target, Live::new()) {
Ok(path) => self.perf_started(conn, name, &wanted, &shown(&path)),
Err(e) => {
log::error!("Horizon: {name}'s server profile did not start: {e}");
self.perf_refused(conn, Refusal::Failed);
}
}
}
Asked::Stop(reason) => self.stop_server_capture(reason, None),
}
}
}
}
File diff suppressed because it is too large. Load diff
@@ -0,0 +1,519 @@
//! What `/profiler` says: a chat line when there is a next step, an action bar notice for progress.
use horizon_client::client_perf::request::ProfileSide;
use horizon_client::mark_sprite::SPRITE as MARK_SPRITE;
use horizon_client::text::{
ClickEvent, DARK_GRAY, GOLD, GRAY, InlineObject, RED, Style, Text, WHITE, YELLOW,
};
use std::path::Path;
/// Where a message goes: the chat history, or the action bar above the hotbar.
#[derive(Debug, PartialEq)]
pub(super) enum Notice {
Chat(Text),
Bar(Text),
}
/// Spark's chat shape: a dim tag round Horizon's mark, then the line in `color`.
fn tagged(body: Text, color: u32) -> Notice {
let mark = Style {
color: Some(WHITE),
inline_object: Some(InlineObject::Atlas {
atlas: "minecraft:gui".into(),
sprite: MARK_SPRITE.into(),
fallback: "H".into(),
}),
..Style::default()
};
Notice::Chat(
Text::colored("[", DARK_GRAY)
.append(Text::styled("\u{fffc}", mark))
.append(Text::colored("] ", DARK_GRAY))
.append(body.with_style(&Style::color(color))),
)
}
fn say(body: Text) -> Notice {
tagged(body, GRAY)
}
fn fail(body: Text) -> Notice {
tagged(body, RED)
}
fn bar(key: &str, args: &[Text]) -> Notice {
Notice::Bar(Text::translate(key, args))
}
fn tr(key: &str, args: &[Text]) -> Text {
Text::translate(key, args)
}
fn button(label: &str, hover: Text, click: ClickEvent) -> Text {
tr(label, &[]).with_style(&Style {
color: Some(GOLD),
click: Some(click),
hover: Some(Box::new(hover)),
..Style::default()
})
}
fn command(label: &str, hover: &str, command: String) -> Text {
button(label, tr(hover, &[]), ClickEvent::Profiler(command))
}
/// Buttons side by side, a single space between.
fn row(buttons: Vec<Text>) -> Text {
let mut out = Text::empty();
for (i, one) in buttons.into_iter().enumerate() {
if i > 0 {
out.push(" ");
}
out.push(one);
}
out
}
fn stop() -> Text {
command(
"horizon.perf.stop_button",
"horizon.perf.stop_hover",
"/profiler stop".into(),
)
}
fn start() -> Text {
command(
"horizon.perf.start_button",
"horizon.perf.start_hover",
"/profiler start".into(),
)
}
fn folder(file: &Path, shown: &str) -> Text {
let dir = file.parent().unwrap_or(file).to_string_lossy().into_owned();
button(
"horizon.perf.folder_button",
tr("horizon.perf.folder_hover", &[Text::literal(shown)]),
ClickEvent::OpenFile(dir),
)
}
fn seconds(secs: f64) -> Text {
Text::literal(format!("{secs:.0}"))
}
/// The shared address as a white, underlined link that drops the scheme from view.
fn link(url: &str) -> Text {
let shown = url
.strip_prefix("https://")
.or_else(|| url.strip_prefix("http://"))
.unwrap_or(url);
Text::styled(
shown,
Style {
color: Some(WHITE),
underlined: Some(true),
click: Some(ClickEvent::OpenUrl(url.to_owned())),
hover: Some(Box::new(tr("horizon.perf.link_hover", &[]))),
..Style::default()
},
)
}
pub(super) fn started(side: Option<ProfileSide>) -> Notice {
let key = match side {
None => "horizon.perf.started",
Some(ProfileSide::Client) => "horizon.perf.started.client",
Some(ProfileSide::Server) => "horizon.perf.started.server",
};
say(tr(key, &[stop()]))
}
/// The answer to a status, a start while one runs, and (idle) a stop or mark with nothing to stop.
pub(super) fn status(sides: &[ProfileSide], elapsed_s: Option<f64>) -> Notice {
let Some(elapsed) = elapsed_s.filter(|_| !sides.is_empty()) else {
return say(tr("horizon.perf.idle", &[start()]));
};
let key = match (
sides.contains(&ProfileSide::Client),
sides.contains(&ProfileSide::Server),
) {
(true, false) => "horizon.perf.running.client",
(false, true) => "horizon.perf.running.server",
_ => "horizon.perf.running.both",
};
say(tr(key, &[seconds(elapsed), stop()]))
}
/// A profile that was asked for both sides and got the client's alone.
pub(super) fn client_only(why: &str) -> Notice {
tagged(
tr("horizon.perf.client_only", &[Text::literal(why)]),
YELLOW,
)
}
/// `file` is the saved profile, `shown` its path as the player sees it, `can_open_and_share` whether this build has the hooks.
pub(super) fn saved(secs: f64, file: &Path, shown: &str, can_open_and_share: bool) -> Notice {
let id = file.file_stem().and_then(|s| s.to_str());
let mut buttons = Vec::new();
if let Some(id) = id.filter(|_| can_open_and_share) {
buttons.push(command(
"horizon.perf.open_button",
"horizon.perf.open_hover",
format!("/profiler open {id}"),
));
buttons.push(share_button(id));
}
buttons.push(folder(file, shown));
say(tr("horizon.perf.saved", &[seconds(secs), row(buttons)]))
}
fn share_button(id: &str) -> Text {
command(
"horizon.perf.share_button",
"horizon.perf.share_hover",
format!("/profiler share {id}"),
)
}
pub(super) fn shared(url: &str) -> Notice {
let copy = button(
"horizon.perf.copy_button",
tr("horizon.perf.copy_hover", &[]),
ClickEvent::CopyToClipboard(url.to_owned()),
);
say(tr("horizon.perf.shared", &[link(url), copy]))
}
pub(super) fn share_missing() -> Notice {
say(tr("horizon.perf.share_missing", &[start()]))
}
pub(super) fn not_found(name: &str) -> Notice {
fail(tr("horizon.perf.not_found", &[Text::literal(name)]))
}
pub(super) fn unavailable() -> Notice {
fail(tr("horizon.perf.unavailable", &[]))
}
pub(super) fn open_failed(error: &str, file: &Path, shown: &str) -> Notice {
fail(tr(
"horizon.perf.open_failed",
&[Text::literal(error), folder(file, shown)],
))
}
/// The retry names the profile, so it shares this one and not whichever is latest.
pub(super) fn upload_failed(error: &str, file: &Path) -> Notice {
let retry = match file.file_stem().and_then(|s| s.to_str()) {
Some(id) => command(
"horizon.perf.retry_button",
"horizon.perf.retry_hover",
format!("/profiler share {id}"),
),
None => Text::empty(),
};
fail(tr(
"horizon.perf.upload_failed",
&[Text::literal(error), retry],
))
}
pub(super) fn error(error: &str) -> Notice {
fail(tr("horizon.perf.error", &[Text::literal(error)]))
}
pub(super) fn saving() -> Notice {
bar("horizon.perf.saving", &[])
}
pub(super) fn uploading() -> Notice {
bar("horizon.perf.uploading", &[])
}
pub(super) fn opened() -> Notice {
bar("horizon.perf.opened", &[])
}
pub(super) fn marked(name: &str) -> Notice {
bar("horizon.perf.marked", &[Text::literal(name)])
}
#[cfg(test)]
mod tests {
use super::*;
use horizon_client::lang::{self, Language};
use std::path::PathBuf;
/// Builds notices in en_us and fails if any key they use is missing from it.
fn english<R>(build: impl FnOnce() -> R) -> R {
let (built, missed) = lang::scoped(Language::load(&[], "en_us"), build);
assert!(missed.is_empty(), "keys missing from en_us: {missed:?}");
built
}
fn chat(notice: Notice) -> Text {
match notice {
Notice::Chat(text) => text,
Notice::Bar(text) => panic!("went to the action bar: {}", text.plain()),
}
}
fn bar_text(notice: Notice) -> Text {
match notice {
Notice::Bar(text) => text,
Notice::Chat(text) => panic!("went to chat: {}", text.plain()),
}
}
/// The span that reads exactly `text`.
fn span<'a>(line: &'a Text, text: &str) -> &'a horizon_client::text::Span {
line.spans
.iter()
.find(|s| s.text == text)
.unwrap_or_else(|| panic!("no span {text:?} in {:?}", line.plain()))
}
fn click<'a>(line: &'a Text, text: &str) -> &'a ClickEvent {
span(line, text)
.style
.click
.as_ref()
.expect("a click event")
}
fn run(command: &str) -> ClickEvent {
ClickEvent::Profiler(command.into())
}
fn file() -> PathBuf {
PathBuf::from("/home/p/reports/profile-20261110-233142Z.hzperf")
}
#[test]
fn a_share_button_sent_back_by_a_server_is_a_plain_command_not_the_local_profiler() {
let line = chat(english(|| saved(12.0, &file(), "reports/x.hzperf", true)));
assert_eq!(
click(&line, "[Share]"),
&run("/profiler share profile-20261110-233142Z")
);
let echoed = english(|| {
horizon_client::component::to_text(&line.component(), &mut |_| Text::empty())
});
assert_eq!(
click(&echoed, "[Share]"),
&ClickEvent::RunCommand("/profiler share profile-20261110-233142Z".into())
);
}
#[test]
fn every_chat_line_starts_with_the_dim_tag_and_the_mark_sprite() {
let line = chat(english(|| started(None)));
assert_eq!(line.plain(), "[H] Profiling the client and server. [Stop]");
assert_eq!(span(&line, "[").style.color, Some(DARK_GRAY));
assert_eq!(span(&line, "] ").style.color, Some(DARK_GRAY));
let Some(InlineObject::Atlas { sprite, .. }) = &span(&line, "\u{fffc}").style.inline_object
else {
panic!("the tag has no sprite");
};
assert_eq!(sprite, MARK_SPRITE);
}
#[test]
fn a_start_names_the_sides_and_offers_stop() {
let (both, client, server) = english(|| {
(
chat(started(None)),
chat(started(Some(ProfileSide::Client))),
chat(started(Some(ProfileSide::Server))),
)
});
assert_eq!(both.plain(), "[H] Profiling the client and server. [Stop]");
assert_eq!(client.plain(), "[H] Profiling the client. [Stop]");
assert_eq!(server.plain(), "[H] Profiling the server. [Stop]");
let stop = span(&both, "[Stop]");
assert_eq!(stop.style.color, Some(GOLD));
assert_eq!(stop.style.underlined, None);
assert_eq!(stop.style.click, Some(run("/profiler stop")));
assert_eq!(
stop.style.hover.as_ref().unwrap().plain(),
"Stop profiling and save the profile."
);
assert_eq!(
span(&both, "Profiling the client and server. ").style.color,
Some(GRAY)
);
}
#[test]
fn a_status_is_one_of_three_lines_and_idle_offers_start() {
let line = |sides: &[ProfileSide], elapsed| chat(english(|| status(sides, elapsed)));
use ProfileSide::{Client, Server};
assert_eq!(
line(&[Server, Client], Some(41.6)).plain(),
"[H] Profiling the client and server for 42 s. [Stop]"
);
assert_eq!(
line(&[Client], Some(3.0)).plain(),
"[H] Profiling the client for 3 s. [Stop]"
);
assert_eq!(
line(&[Server], Some(3.0)).plain(),
"[H] Profiling the server for 3 s. [Stop]"
);
let idle = line(&[], None);
assert_eq!(idle.plain(), "[H] Not profiling. [Start]");
assert_eq!(click(&idle, "[Start]"), &run("/profiler start"));
}
#[test]
fn the_saved_line_carries_the_length_and_the_buttons_for_this_profile() {
let line = chat(english(|| {
saved(
42.2,
&file(),
"reports/profile-20261110-233142Z.hzperf",
true,
)
}));
assert_eq!(
line.plain(),
"[H] Profile saved (42 s). [Open] [Share] [Folder]"
);
assert_eq!(
click(&line, "[Open]"),
&run("/profiler open profile-20261110-233142Z")
);
assert_eq!(
click(&line, "[Share]"),
&run("/profiler share profile-20261110-233142Z")
);
assert_eq!(
click(&line, "[Folder]"),
&ClickEvent::OpenFile("/home/p/reports".into())
);
assert_eq!(
span(&line, "[Folder]")
.style
.hover
.as_ref()
.unwrap()
.plain(),
"Show reports/profile-20261110-233142Z.hzperf in its folder."
);
assert!(line.spans.iter().all(|s| s.style.underlined.is_none()));
let bare = chat(english(|| saved(42.0, &file(), "x", false)));
assert_eq!(bare.plain(), "[H] Profile saved (42 s). [Folder]");
}
#[test]
fn a_shared_profile_is_one_line_with_an_underlined_link_and_copy() {
let url = "https://playhorizon.dev/perf/c/55bwef5st6";
let line = chat(english(|| shared(url)));
assert_eq!(
line.plain(),
"[H] Shared: playhorizon.dev/perf/c/55bwef5st6 [Copy]"
);
let link = span(&line, "playhorizon.dev/perf/c/55bwef5st6");
assert_eq!(link.style.color, Some(WHITE));
assert_eq!(link.style.underlined, Some(true));
assert_eq!(link.style.click, Some(ClickEvent::OpenUrl(url.into())));
assert_eq!(
link.style.hover.as_ref().unwrap().plain(),
"Open the shared profile in your browser."
);
let copy = span(&line, "[Copy]");
assert_eq!(copy.style.color, Some(GOLD));
assert_eq!(
copy.style.click,
Some(ClickEvent::CopyToClipboard(url.into()))
);
assert_eq!(copy.style.underlined, None);
}
#[test]
fn a_failed_share_retries_that_profile_by_name() {
let line = chat(english(|| upload_failed("no network", &file())));
assert_eq!(
line.plain(),
"[H] Could not share the profile: no network. [Retry]"
);
assert_eq!(
click(&line, "[Retry]"),
&run("/profiler share profile-20261110-233142Z")
);
assert_eq!(
span(&line, "Could not share the profile: ").style.color,
Some(RED)
);
}
#[test]
fn failures_are_red_and_say_what_failed_in_words() {
let reds = english(|| {
[
(error("boom"), "[H] Profiler: boom"),
(not_found("old"), "[H] No saved profile named old."),
(
unavailable(),
"[H] Opening and sharing profiles is not available in this build.",
),
(
open_failed("no browser", &file(), "p"),
"[H] Could not open the profile: no browser. [Folder]",
),
]
});
for (notice, words) in reds {
let line = chat(notice);
assert_eq!(line.plain(), words);
let body = &line.spans[3];
assert_eq!(body.style.color, Some(RED), "{words}");
}
}
#[test]
fn a_profile_that_could_not_share_both_sides_is_yellow() {
let line = chat(english(|| {
client_only("a server profile needs operator level 4")
}));
assert_eq!(
line.plain(),
"[H] Profiling the client only: a server profile needs operator level 4"
);
assert_eq!(line.spans[3].style.color, Some(YELLOW));
}
#[test]
fn nothing_saved_yet_is_a_gray_line_with_start() {
let line = chat(english(share_missing));
assert_eq!(
line.plain(),
"[H] No profile saved in this session yet. [Start]"
);
assert_eq!(click(&line, "[Start]"), &run("/profiler start"));
assert_eq!(line.spans[3].style.color, Some(GRAY));
}
#[test]
fn progress_goes_to_the_action_bar_without_a_tag() {
let (saving, uploading, opened, marked) =
english(|| (saving(), uploading(), opened(), marked("boss fight")));
for (notice, words) in [
(saving, "Saving the profile…"),
(uploading, "Uploading the profile…"),
(opened, "Opened in your browser"),
(marked, "Marked boss fight"),
] {
let text = bar_text(notice);
assert_eq!(text.plain(), words);
assert!(
text.spans
.iter()
.all(|s| s.style.inline_object.is_none() && s.style.click.is_none())
);
}
}
}
+12 -4
View File
@@ -71,7 +71,7 @@ impl Game<'_> {
forgotten.insert((p.pos[0], p.pos[1]));
continue;
}
self.forget_remote_columns(&mut forgotten);
horizon_client::spans::time("pr_forget", || self.forget_remote_columns(&mut forgotten));
let chunk = matches!(item, Received::Chunk(_));
if chunk && !overdue && chunks >= CHUNK_PACKETS_PER_FRAME {
events.push_front(item);
@@ -99,7 +99,9 @@ impl Game<'_> {
};
self.native_client_packet(&packet);
if let C::LevelChunkWithLight(p) = *packet {
self.remote_chunk(&p, Some((&format, column)));
horizon_client::spans::time("pr_chunk", || {
self.remote_chunk(&p, Some((&format, column)))
});
}
chunks += started.map_or(Duration::ZERO, |t| t.elapsed());
continue;
@@ -264,7 +266,13 @@ impl Game<'_> {
self.offer_reconnect(retry);
return Ok(());
}
Event::Packet(packet) => self.handle_packet(*packet)?,
Event::Packet(packet) => match *packet {
// The server's answer to a `profile` request.
C::CustomPayload(p) if p.channel == horizon_client::perf_net::CHANNEL => {
self.remote_perf_reply(&p.data);
}
packet => self.handle_packet(packet)?,
},
}
chunks += started.map_or(Duration::ZERO, |t| t.elapsed());
if self.remote.is_none() {
@@ -272,7 +280,7 @@ impl Game<'_> {
}
}
if self.remote.is_some() {
self.forget_remote_columns(&mut forgotten);
horizon_client::spans::time("pr_forget", || self.forget_remote_columns(&mut forgotten));
}
if let Some(r) = self.remote.as_mut() {
r.backlog_since = if events.is_empty() {
@@ -11,10 +11,10 @@ use horizon_client::{
use horizon_protocol::packets::configuration::ClientInformation;
use std::sync::{Arc, atomic::AtomicI32, mpsc};
const GUEST: &str = "H274Guest";
pub(super) const GUEST: &str = "H274Guest";
/// A host on a stone floor with one connected, placed guest (conn 2).
fn game() -> Game<'static> {
pub(super) fn game() -> Game<'static> {
let root = std::path::PathBuf::from(
std::env::var_os("HORIZON_TEST_RESOURCES")
.expect("set HORIZON_TEST_RESOURCES to prepared resources"),
+20 -1
View File
@@ -33,7 +33,7 @@ thread_local! {
static COSTS: RefCell<[FxHashMap<(&'static str, usize), Cost>; 2]> = RefCell::default();
}
/// Who wants the tick's spans recorded besides nobody: `HORIZON_SPIKES` (the slow-tick log),
/// Who wants the tick's spans recorded besides nobody: `HORIZON_SPIKES` (the slow ring's steps),
/// a report's recording, a mod (`native_spans`).
pub(super) const SPANS_SPIKES: u8 = 1;
pub(super) const SPANS_REPORT: u8 = 2;
@@ -50,6 +50,11 @@ pub(super) fn want_spans(user: u8, on: bool) {
horizon_client::spans::enable(users != 0);
}
#[cfg(test)]
pub(super) fn wanted(user: u8) -> bool {
SPAN_USERS.load(Ordering::Relaxed) & user != 0
}
/// Sample (or stop sampling) the mob ticks this thread runs next, for the report.
pub(super) fn set_sampling(on: bool) {
set(REPORT, on);
@@ -76,6 +81,20 @@ pub(super) fn add(dim: &'static str, kind: &MobType, started: Instant) {
add_kind(dim, std::ptr::from_ref(kind) as usize, &kind.id, started);
}
/// A mob of type `kind` starts its tick: the CPU sampler's work, and `horizon-report`'s cost by mob type when sampled.
pub(super) fn start_mob(kind: &MobType) -> Option<(Instant, MobType)> {
horizon_threads::work::set_entity(horizon_threads::work::entity_id(&kind.id));
start().map(|t| (t, kind.clone()))
}
/// The mob [`start_mob`] started ended its tick in level `dim`.
pub(super) fn end_mob(dim: &'static str, sampled: Option<(Instant, MobType)>) {
horizon_threads::work::clear_work();
if let Some((started, kind)) = sampled {
add(dim, &kind, started);
}
}
/// [`add`] for the type at address `key` named `id`.
fn add_kind(dim: &'static str, key: usize, id: &str, started: Instant) {
let nanos = started.elapsed().as_nanos() as u64;
@@ -168,6 +168,7 @@ impl Game<'_> {
if let Err(e) = self.save_for_exit() {
log::warn!("Horizon: failed to save the world on exit: {e}");
}
self.finish_profile();
self.finish_perf();
}
/// winit's `about_to_wait`: what happens between frames (stop requests, the closing
@@ -158,7 +158,7 @@ impl Game<'_> {
self.clear_lod();
self.all_sections_changed();
}
self.prepare_lod();
horizon_client::spans::time("prepare_lod", || self.prepare_lod());
// The columns the single-player server changed in bulk without sending them, which far terrain still drew
// as this window last saw them: summarized again (taken every tick, far terrain on or off, so none waits).
if self.remote.is_some() {
@@ -277,7 +277,8 @@ impl Game<'_> {
if self.remote.is_none() {
horizon_client::spans::time("sync_tick_chunks", || self.sync_tick_chunks());
}
let (far_distance, far_reduced) = self.far_terrain_choice();
let (far_distance, far_reduced) =
horizon_client::spans::time("far_terrain_choice", || self.far_terrain_choice());
// Terrain still on its way from the server (single player's integrated one
// too): the view's outermost columns not all in, or its events not all handled.
let radius = match &self.remote {
@@ -385,6 +386,7 @@ impl Game<'_> {
AUTO_UPLOAD_TIME
};
let started = Instant::now();
let upload_span = horizon_client::spans::guard("upload_queue");
while let Some((_, mesh)) = pending.last() {
if auto && count > 0 && started.elapsed() >= auto_time {
break;
@@ -403,6 +405,7 @@ impl Game<'_> {
count += 1;
bytes += size;
}
drop(upload_span);
self.streaming.free(spent);
}
renderer.remove_columns(unloaded.iter().map(|c| (c.x, c.z)));
+35
View File
@@ -654,6 +654,41 @@ impl ChunkEncoder {
)
}
/// The section entries and the column heightmaps cached, each as (entries, bytes).
pub fn cache_census(&self) -> [(usize, usize); 2] {
use std::mem::size_of;
let (entries, sections, wire) = {
let cache = self.cache.lock().unwrap();
let wire: usize = cache
.values()
.filter_map(|e| e.wire.as_ref())
.map(|(bytes, entities)| bytes.len() + entities.len() * size_of::<(u16, u32)>())
.sum();
(
cache.len(),
cache.capacity() * (size_of::<(usize, Encoded)>() + 1),
wire,
)
};
let heights = self.heights.lock().unwrap();
let maps: usize = heights
.values()
.map(|h| {
h.sections.capacity() * size_of::<std::sync::Weak<Section>>()
+ h.maps
.iter()
.map(|(_, v)| v.capacity() * 8 + 24)
.sum::<usize>()
})
.sum();
[
(entries, sections + wire),
(
heights.len(),
maps + heights.capacity() * (size_of::<((i32, i32), Heights)>() + 1),
),
]
}
/// The cached entry for a section, made with `non_air` when missing.
fn cached<T>(&self, section: &Arc<Section>, read: impl FnOnce(&mut Encoded) -> T) -> T {
read(self.entry(&mut self.cache.lock().unwrap(), section))
+21
View File
@@ -446,6 +446,17 @@ impl PlayerSender {
}
}
}
/// Packets queued for the network thread or held by the tick, and their bytes.
pub fn queued(&self) -> (usize, usize) {
let waiting = match &self.to {
To::Pool(outbox, _) => outbox.queued(),
To::Channel(_) => 0,
};
(
waiting + self.held().len(),
self.pending.load(Ordering::Relaxed),
)
}
/// Marks this sender as a fake player's or not.
pub fn set_fake(&self, fake: bool) {
self.fake.store(fake, Ordering::Relaxed);
@@ -1973,6 +1984,11 @@ impl SharedFrames {
fn cache(&self) -> std::sync::MutexGuard<'_, FrameCache> {
self.frames.lock().unwrap_or_else(PoisonError::into_inner)
}
/// The frames kept and the bytes of their contents.
fn census(&self) -> (usize, usize) {
let cache = self.cache();
(cache.frames.len(), cache.bytes)
}
/// `frame::contents(body, Some(threshold))`, compressed only when no
/// connection has sent the same body lately or compresses it now.
fn contents(&self, body: &[u8], threshold: usize) -> horizon_protocol::Result<Arc<[u8]>> {
@@ -2021,6 +2037,11 @@ impl SharedFrames {
}
}
/// The compressed frames shared by every connection: how many and their bytes.
pub fn shared_frame_census() -> (usize, usize) {
SharedFrames::get().census()
}
/// Frame one packet body into `out`: a large one compressed through
/// [`SharedFrames`], so a chunk column sent to fifty players is compressed
/// once, not fifty times. The bytes on the wire are the same.
+7
View File
@@ -824,6 +824,13 @@ pub struct Tracker {
}
impl Tracker {
/// The bytes of its tables, from their capacities.
pub fn heap_bytes(&self) -> usize {
use std::mem::size_of;
self.seen.capacity() * (size_of::<(i32, Arc<Tracked>)>() + 1)
+ self.events.capacity() * size_of::<TrackEvent>()
+ self.hidden.capacity() * (size_of::<i32>() + 1)
}
pub fn is_tracking(&self, id: i32) -> bool {
self.seen.contains_key(&id)
}
+4
View File
@@ -162,6 +162,10 @@ impl Outbox {
pub(super) fn take_packets(&self) -> [Vec<(i32, usize)>; 2] {
std::mem::take(&mut *self.tally.lock().unwrap_or_else(PoisonError::into_inner))
}
/// The packets queued and not taken by the thread yet.
pub(super) fn queued(&self) -> usize {
self.queue().bodies.len()
}
fn queue(&self) -> MutexGuard<'_, Queue> {
self.queue.lock().unwrap_or_else(PoisonError::into_inner)
}
+2 -5
View File
@@ -1,12 +1,9 @@
//! Horizon: native Rust client for Minecraft Java Edition 26.3 gameplay.
#![forbid(unsafe_code)]
#[cfg(not(any(feature = "system-allocator", feature = "alloc-census")))]
#[cfg(not(feature = "system-allocator"))]
#[global_allocator]
static ALLOCATOR: horizon_threads::Allocator = horizon_threads::ALLOCATOR;
#[cfg(all(feature = "alloc-census", not(feature = "system-allocator")))]
#[global_allocator]
static ALLOCATOR: horizon_threads::CensusAllocator = horizon_threads::CensusAllocator;
static ALLOCATOR: horizon_threads::GameAllocator = horizon_threads::GAME_ALLOCATOR;
/// The mod API level this game serves, kept in the program for the launcher
/// to read without running it (`horizon_mods::api_level::MARK`). `main`
@@ -40,7 +40,8 @@ pub fn attempt_counters(publish: bool) {
}
let counts = ATTEMPT_COUNTS.with(|c| c.replace([0; 2]));
if publish {
crate::spans::counters(&[("sa_attempts", counts[0]), ("sa_spawns", counts[1])]);
horizon_perf_format::counter("natural_spawning.attempts", &[]).add(counts[0]);
horizon_perf_format::counter("natural_spawning.spawns", &[]).add(counts[1]);
}
}
fn attempt_count(index: usize) {
@@ -1554,6 +1555,26 @@ fn top_non_colliding<L: NaturalLevel>(l: &L, entity: &str, x: i32, z: i32) -> [i
mod tests {
use super::*;
/// Attempts and spawns counted while spans are on reach the perf counters once published, and a reset publishes nothing.
#[test]
fn attempt_counters_publish_what_the_tick_counted() {
let attempts = horizon_perf_format::counter("natural_spawning.attempts", &[]);
let spawns = horizon_perf_format::counter("natural_spawning.spawns", &[]);
let before = (attempts.get(), spawns.get());
crate::spans::enable(true);
attempt_counters(false);
for _ in 0..7 {
attempt_count(0);
}
for _ in 0..3 {
attempt_count(1);
}
attempt_counters(true);
attempt_counters(true);
crate::spans::enable(false);
assert_eq!((attempts.get() - before.0, spawns.get() - before.1), (7, 3));
}
/// The grid finds the players whose chunk is within 8 of a chunk on both
/// axes, in their order, as a look at every player does.
#[test]
+589
View File
@@ -0,0 +1,589 @@
//! `horizon:perf`: a client asks a Horizon server to capture its side (docs/perf-platform.md §4.2).
use crate::client_perf::request::{HZ_RANGE, StartOptions, Threads};
use crate::server_perf::MAX_WINDOW_S;
use horizon_perf_format::MAX_PART_BYTES;
use horizon_protocol::{
Error, Result,
buf::{Reader, Writer},
};
use std::time::{Duration, Instant};
pub const CHANNEL: &str = "horizon:perf";
/// The server brand of every Horizon server and LAN host.
pub const HORIZON_BRAND: &str = "horizon";
/// The operator level a server capture needs, as vanilla's `/perf` (`Commands.LEVEL_OWNERS`).
pub const OPERATOR_LEVEL: u8 = 4;
/// How long a request waits for the server before it counts as unanswered.
pub const ANSWER_TIMEOUT: Duration = Duration::from_secs(10);
/// Bytes of a serverbound payload at most (ServerboundCustomPayloadPacket.MAX_PAYLOAD_SIZE).
const MAX_PAYLOAD: usize = 32767;
const MAX_PATH: usize = 256;
/// Bytes of the server's file in one chunk.
pub const CHUNK: usize = 32 * 1024;
/// The most bytes per second the server sends of its file.
pub const RATE: usize = 256 * 1024;
/// Bytes already queued for the guest above which the server holds its next chunk back, so gameplay packets go first.
pub const BUSY_QUEUE: usize = 64 * 1024;
/// How long a client waits for the file to be finished, or for the next chunk, before it gives up on the server's part.
pub const STALL_TIMEOUT: Duration = Duration::from_secs(30);
const MAX_REMOVED: usize = 64;
/// What a capture on the server is asked to record; the switches a server has (no GPU, no file).
#[derive(Clone, Copy, Debug, PartialEq)]
pub struct Wanted {
pub seconds: Option<u32>,
pub hz: Option<u32>,
pub over_ms: Option<f32>,
pub threads: Threads,
pub wall: bool,
pub samples: bool,
pub timeline: bool,
}
impl From<&StartOptions> for Wanted {
fn from(o: &StartOptions) -> Self {
Self {
seconds: o.seconds,
hz: o.hz,
over_ms: o.over_ms,
threads: o.threads,
wall: o.wall,
samples: o.samples,
timeline: o.timeline,
}
}
}
impl Wanted {
pub fn options(&self) -> StartOptions {
StartOptions {
seconds: self.seconds,
hz: self.hz,
over_ms: self.over_ms,
threads: self.threads,
wall: self.wall,
samples: self.samples,
timeline: self.timeline,
..StartOptions::default()
}
}
}
#[derive(Clone, Copy, Debug, PartialEq)]
pub enum Request {
Start(Wanted),
Stop,
}
/// Why the server did not do what was asked.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub enum Refusal {
/// The player is below operator level 4.
Denied,
/// A server capture already runs.
AlreadyRunning,
/// No capture of this player's runs.
NotRunning,
/// The capture could not be started or written; the server's log says why.
Failed,
}
#[derive(Clone, Debug, PartialEq)]
pub enum Reply {
Started {
path: String,
seconds: Option<u32>,
samples: bool,
},
/// The capture ended and its file is whole on the server; `bytes` of it follow as [`Reply::Chunk`]s, none if 0.
Finished {
path: String,
removed: Vec<String>,
bytes: u64,
},
Refused(Refusal),
/// The next bytes of the file, at most [`CHUNK`].
Chunk(Vec<u8>),
}
const START: u8 = 0;
const STOP: u8 = 1;
const WALL: u8 = 1;
const SAMPLES: u8 = 2;
const TIMELINE: u8 = 4;
const THREADS_ALL: u8 = 1;
impl Request {
pub fn encode(&self) -> Vec<u8> {
let mut w = Writer::new();
match self {
Self::Stop => w.u8(STOP),
Self::Start(s) => {
w.u8(START);
w.varint(s.seconds.unwrap_or(0) as i32);
w.varint(s.hz.unwrap_or(0) as i32);
w.f32(s.over_ms.unwrap_or(0.0));
w.u8(match s.threads {
Threads::Main => 0,
Threads::All => THREADS_ALL,
});
let flags = [(s.wall, WALL), (s.samples, SAMPLES), (s.timeline, TIMELINE)]
.into_iter()
.filter(|(on, _)| *on)
.fold(0, |acc, (_, bit)| acc | bit);
w.u8(flags);
}
}
w.data
}
/// A payload from a player; every value is checked against the limits `profile start` has.
pub fn decode(data: &[u8]) -> Result<Self> {
if data.len() > MAX_PAYLOAD {
return Err(Error::new("perf: payload too big"));
}
let mut r = Reader::new(data);
let request = match r.u8()? {
STOP => Self::Stop,
START => {
let seconds = r.varint()?;
let hz = r.varint()?;
let over = r.f32()?;
let threads = match r.u8()? {
0 => Threads::Main,
THREADS_ALL => Threads::All,
other => return Err(Error::new(format!("perf: threads {other}"))),
};
let flags = r.u8()?;
if flags & !(WALL | SAMPLES | TIMELINE) != 0 {
return Err(Error::new(format!("perf: flags {flags}")));
}
let seconds = match u32::try_from(seconds) {
Ok(0) => None,
Ok(s) if f64::from(s) <= MAX_WINDOW_S => Some(s),
_ => return Err(Error::new(format!("perf: {seconds} seconds"))),
};
let hz = match u32::try_from(hz) {
Ok(0) => None,
Ok(h) if HZ_RANGE.contains(&h) => Some(h),
_ => return Err(Error::new(format!("perf: {hz} Hz"))),
};
let over_ms = if over == 0.0 {
None
} else if over.is_finite() && over > 0.0 {
Some(over)
} else {
return Err(Error::new("perf: bad slow threshold"));
};
Self::Start(Wanted {
seconds,
hz,
over_ms,
threads,
wall: flags & WALL != 0,
samples: flags & SAMPLES != 0,
timeline: flags & TIMELINE != 0,
})
}
other => return Err(Error::new(format!("perf: request {other}"))),
};
if r.remaining() > 0 {
return Err(Error::new("perf: trailing bytes"));
}
Ok(request)
}
}
const STARTED: u8 = 0;
const FINISHED: u8 = 1;
const REFUSED: u8 = 2;
const CHUNK_TAG: u8 = 3;
impl Reply {
pub fn encode(&self) -> Vec<u8> {
let mut w = Writer::new();
match self {
Self::Started {
path,
seconds,
samples,
} => {
w.u8(STARTED);
w.string(path);
w.varint(seconds.unwrap_or(0) as i32);
w.bool(*samples);
}
Self::Finished {
path,
removed,
bytes,
} => {
w.u8(FINISHED);
w.string(path);
w.varlong(*bytes as i64);
w.varint(removed.len().min(MAX_REMOVED) as i32);
for name in removed.iter().take(MAX_REMOVED) {
w.string(name);
}
}
Self::Chunk(data) => {
w.u8(CHUNK_TAG);
w.byte_array(data);
}
Self::Refused(why) => {
w.u8(REFUSED);
w.u8(match why {
Refusal::Denied => 0,
Refusal::AlreadyRunning => 1,
Refusal::NotRunning => 2,
Refusal::Failed => 3,
});
}
}
w.data
}
pub fn decode(data: &[u8]) -> Result<Self> {
let mut r = Reader::new(data);
let reply = match r.u8()? {
STARTED => {
let path = r.string(MAX_PATH)?;
let seconds = r.varint()?;
Self::Started {
path,
seconds: u32::try_from(seconds).ok().filter(|s| *s > 0),
samples: r.bool()?,
}
}
FINISHED => {
let path = r.string(MAX_PATH)?;
let bytes = u64::try_from(r.varlong()?)
.map_err(|_| Error::new("perf: negative file size"))?;
let removed = (0..r.count(MAX_REMOVED)?)
.map(|_| r.string(MAX_PATH))
.collect::<Result<_>>()?;
Self::Finished {
path,
removed,
bytes,
}
}
CHUNK_TAG => Self::Chunk(r.byte_array(CHUNK)?),
REFUSED => Self::Refused(match r.u8()? {
0 => Refusal::Denied,
1 => Refusal::AlreadyRunning,
2 => Refusal::NotRunning,
3 => Refusal::Failed,
other => return Err(Error::new(format!("perf: refusal {other}"))),
}),
other => return Err(Error::new(format!("perf: reply {other}"))),
};
if r.remaining() > 0 {
return Err(Error::new("perf: trailing bytes"));
}
Ok(reply)
}
}
/// Who the capture a guest asked of this server belongs to, as the asker sees it.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub enum Held {
Nothing,
/// The asker's own capture runs.
Asker,
/// Another guest's capture runs or waits to begin.
Someone,
}
/// What the server does with a request.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub enum Verdict {
/// Not from a Horizon client: no answer.
Ignore,
Refuse(Refusal),
Accept,
}
/// The server's rules for a guest's request: Horizon clients only, operator level 4, one capture at a time, and only its owner stops it.
pub fn judge(brand: &str, permission: u8, request: &Request, held: Held) -> Verdict {
if !is_horizon(brand) {
return Verdict::Ignore;
}
if permission < OPERATOR_LEVEL {
return Verdict::Refuse(Refusal::Denied);
}
match (request, held) {
(Request::Start(_), Held::Nothing) | (Request::Stop, Held::Asker) => Verdict::Accept,
(Request::Start(_), _) => Verdict::Refuse(Refusal::AlreadyRunning),
(Request::Stop, _) => Verdict::Refuse(Refusal::NotRunning),
}
}
/// Whether a server calls itself Horizon.
pub fn is_horizon(brand: &str) -> bool {
brand == HORIZON_BRAND
}
/// Why a client could not send a request.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub enum Unsent {
NotHorizon,
/// A request is unanswered, or a server capture of ours runs.
Busy,
/// No capture of ours runs on the server.
NothingToStop,
}
/// Where this client's request to the connected server has got.
#[derive(Clone, Debug, Default, PartialEq)]
pub enum Link {
#[default]
Idle,
/// The start request waits for its answer.
Asking { since: Instant },
Running {
path: String,
since: Instant,
seconds: Option<u32>,
},
/// The stop request waits for the file to be whole.
Stopping { path: String, since: Instant },
/// The server's file arrives: `got` of `total` bytes, the last at `since`.
Receiving {
path: String,
since: Instant,
got: u64,
total: u64,
},
}
/// What a chunk of the server's file means for the client.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub enum Received {
/// No file was expected.
Stray,
More,
/// The file is whole.
Last,
/// More bytes arrived than the server announced.
TooMuch,
}
/// The server's pacing of the file it sends: chunks of at most [`CHUNK`], never faster than [`RATE`], and none while the guest's connection has a backlog.
#[derive(Clone, Debug)]
pub struct Outgoing {
remaining: u64,
next_at: Instant,
}
impl Outgoing {
pub fn new(total: u64, now: Instant) -> Self {
Self {
remaining: total,
next_at: now,
}
}
pub fn done(&self) -> bool {
self.remaining == 0
}
/// The size of the chunk to send at `now`, if one is due; `queued` is the bytes already waiting to be sent to the guest.
pub fn next(&mut self, now: Instant, queued: usize) -> Option<usize> {
if self.remaining == 0 || now < self.next_at || queued > BUSY_QUEUE {
return None;
}
let n = self.remaining.min(CHUNK as u64) as usize;
self.remaining -= n as u64;
self.next_at = now + Duration::from_secs_f64(n as f64 / RATE as f64);
Some(n)
}
}
/// What an answer from the server means for the client.
#[derive(Clone, Debug, PartialEq)]
pub enum Outcome {
Started {
path: String,
seconds: Option<u32>,
samples: bool,
},
Finished {
path: String,
removed: Vec<String>,
/// Bytes of the server's file to come; 0 when the server sends none.
bytes: u64,
},
Refused(Refusal),
/// The server announced a file over [`MAX_PART_BYTES`]; none of it is taken.
TooLarge {
bytes: u64,
},
/// The reply fits no request of ours, for example a duplicate.
Stray,
}
impl Link {
pub fn is_idle(&self) -> bool {
matches!(self, Self::Idle)
}
/// Whether the server runs a capture for this client, or is about to.
pub fn running(&self) -> bool {
matches!(self, Self::Running { .. } | Self::Stopping { .. })
}
/// Whether a start is allowed against a server of `brand`; the link waits for the answer.
pub fn ask(&mut self, brand: &str, now: Instant) -> std::result::Result<(), Unsent> {
if !is_horizon(brand) {
return Err(Unsent::NotHorizon);
}
if !self.is_idle() {
return Err(Unsent::Busy);
}
*self = Self::Asking { since: now };
Ok(())
}
/// Whether a stop can be sent; the link waits for the file.
pub fn stop(&mut self, now: Instant) -> std::result::Result<(), Unsent> {
match std::mem::take(self) {
Self::Running { path, .. } => {
*self = Self::Stopping { path, since: now };
Ok(())
}
other => {
*self = other;
Err(Unsent::NothingToStop)
}
}
}
/// The server file's name and the seconds since the capture began, while one runs.
pub fn status(&self, now: Instant) -> Option<(&str, f64)> {
match self {
Self::Running { path, since, .. } | Self::Stopping { path, since } => {
Some((path, now.duration_since(*since).as_secs_f64()))
}
_ => None,
}
}
pub fn answer(&mut self, reply: Reply, now: Instant) -> Outcome {
match (std::mem::take(self), reply) {
(
Self::Asking { .. },
Reply::Started {
path,
seconds,
samples,
},
) => {
*self = Self::Running {
path: path.clone(),
since: now,
seconds,
};
Outcome::Started {
path,
seconds,
samples,
}
}
(
Self::Running { .. } | Self::Stopping { .. },
Reply::Finished {
path,
removed,
bytes,
},
) => {
if bytes > MAX_PART_BYTES {
return Outcome::TooLarge { bytes };
}
if bytes > 0 {
*self = Self::Receiving {
path: path.clone(),
since: now,
got: 0,
total: bytes,
};
}
Outcome::Finished {
path,
removed,
bytes,
}
}
(Self::Receiving { .. }, Reply::Refused(why)) => Outcome::Refused(why),
(Self::Asking { .. }, Reply::Refused(why)) => Outcome::Refused(why),
(Self::Running { .. } | Self::Stopping { .. }, Reply::Refused(why)) => {
Outcome::Refused(why)
}
(before, _) => {
*self = before;
Outcome::Stray
}
}
}
/// Counts `len` bytes of the server's file that arrived.
pub fn chunk(&mut self, len: usize, now: Instant) -> Received {
match std::mem::take(self) {
Self::Receiving {
path, got, total, ..
} => {
let got = got + len as u64;
if got < total {
*self = Self::Receiving {
path,
since: now,
got,
total,
};
Received::More
} else if got == total {
Received::Last
} else {
Received::TooMuch
}
}
other => {
*self = other;
Received::Stray
}
}
}
/// Whether the server's file has been awaited or received too slowly for longer than [`STALL_TIMEOUT`]; the link goes idle.
pub fn stalled(&mut self, now: Instant) -> bool {
match self {
Self::Stopping { since, .. } | Self::Receiving { since, .. }
if now.duration_since(*since) > STALL_TIMEOUT =>
{
*self = Self::Idle;
true
}
_ => false,
}
}
/// Whether the link has waited longer than [`ANSWER_TIMEOUT`] for a start's answer; it goes idle.
pub fn timed_out(&mut self, now: Instant) -> bool {
match self {
Self::Asking { since } if now.duration_since(*since) > ANSWER_TIMEOUT => {
*self = Self::Idle;
true
}
_ => false,
}
}
/// The connection ended: whatever waited will not be answered.
pub fn reset(&mut self) -> Self {
std::mem::take(self)
}
}
#[cfg(test)]
mod tests;
+480
View File
@@ -0,0 +1,480 @@
use super::*;
fn start(seconds: u32) -> Wanted {
Wanted {
seconds: Some(seconds),
hz: None,
over_ms: None,
threads: Threads::Main,
wall: false,
samples: true,
timeline: false,
}
}
fn started(path: &str, seconds: Option<u32>) -> Reply {
Reply::Started {
path: path.into(),
seconds,
samples: true,
}
}
fn finished(path: &str) -> Reply {
finished_with(path, 0)
}
fn finished_with(path: &str, bytes: u64) -> Reply {
Reply::Finished {
path: path.into(),
removed: Vec::new(),
bytes,
}
}
#[test]
fn a_start_request_is_the_bytes_the_wire_says() {
let wanted = Wanted {
seconds: Some(60),
hz: Some(500),
over_ms: Some(2.5),
threads: Threads::All,
wall: true,
samples: false,
timeline: true,
};
let mut expected = vec![0, 60];
expected.extend([0xf4, 0x03]);
expected.extend(2.5f32.to_be_bytes());
expected.extend([1, 0b101]);
assert_eq!(Request::Start(wanted).encode(), expected);
assert_eq!(Request::decode(&expected).unwrap(), Request::Start(wanted));
assert_eq!(Request::Stop.encode(), [1]);
assert_eq!(Request::decode(&[1]).unwrap(), Request::Stop);
}
#[test]
fn unset_switches_stay_unset_across_the_wire() {
let wanted = Wanted {
seconds: None,
..start(1)
};
let back = Request::decode(&Request::Start(wanted).encode()).unwrap();
assert_eq!(back, Request::Start(wanted));
}
#[test]
fn a_request_beyond_the_limits_of_profile_start_is_refused() {
let with = |seconds: i32, hz: i32, over: f32, threads: u8, flags: u8| {
let mut w = Writer::new();
w.u8(0);
w.varint(seconds);
w.varint(hz);
w.f32(over);
w.u8(threads);
w.u8(flags);
w.data
};
assert!(Request::decode(&with(600, 10, 1.0, 0, 2)).is_ok());
for bad in [
with(601, 0, 0.0, 0, 2),
with(-1, 0, 0.0, 0, 2),
with(0, 9, 0.0, 0, 2),
with(0, 1001, 0.0, 0, 2),
with(0, 0, -1.0, 0, 2),
with(0, 0, f32::NAN, 0, 2),
with(0, 0, f32::INFINITY, 0, 2),
with(0, 0, 0.0, 2, 2),
with(0, 0, 0.0, 0, 8),
] {
assert!(Request::decode(&bad).is_err(), "{bad:?}");
}
let mut trailing = Request::Stop.encode();
trailing.push(0);
assert!(Request::decode(&trailing).is_err());
assert!(Request::decode(&[]).is_err());
assert!(Request::decode(&[9]).is_err());
assert!(Request::decode(&vec![1; MAX_PAYLOAD + 1]).is_err());
}
#[test]
fn replies_round_trip_and_refusals_are_distinct_bytes() {
let replies = [
started("reports/profile-20261107-143204Z.hzperf", Some(60)),
Reply::Started {
path: "reports/a.hzperf".into(),
seconds: None,
samples: false,
},
Reply::Finished {
path: "reports/a.hzperf".into(),
removed: vec!["reports/old.hzperf".into(), "reports/older.hzperf".into()],
bytes: 5_000_000_000,
},
Reply::Chunk(vec![0x1f, 0x8b, 8, 0, 255]),
Reply::Chunk(Vec::new()),
];
for reply in replies {
assert_eq!(Reply::decode(&reply.encode()).unwrap(), reply);
}
let bytes: Vec<Vec<u8>> = [
Refusal::Denied,
Refusal::AlreadyRunning,
Refusal::NotRunning,
Refusal::Failed,
]
.map(|why| Reply::Refused(why).encode())
.to_vec();
assert_eq!(bytes, [[2, 0], [2, 1], [2, 2], [2, 3]]);
assert_eq!(
Reply::decode(&[2, 1]).unwrap(),
Reply::Refused(Refusal::AlreadyRunning)
);
assert!(Reply::decode(&[2, 4]).is_err());
assert!(Reply::decode(&[7]).is_err());
}
#[test]
fn the_server_judges_by_brand_then_level_then_what_runs() {
let go = Request::Start(start(5));
let cases = [
("vanilla", 4, &go, Held::Nothing, Verdict::Ignore),
("", 4, &Request::Stop, Held::Asker, Verdict::Ignore),
(
"horizon",
3,
&go,
Held::Nothing,
Verdict::Refuse(Refusal::Denied),
),
(
"horizon",
2,
&Request::Stop,
Held::Asker,
Verdict::Refuse(Refusal::Denied),
),
(
"horizon",
0,
&go,
Held::Someone,
Verdict::Refuse(Refusal::Denied),
),
("horizon", 4, &go, Held::Nothing, Verdict::Accept),
(
"horizon",
4,
&go,
Held::Asker,
Verdict::Refuse(Refusal::AlreadyRunning),
),
(
"horizon",
4,
&go,
Held::Someone,
Verdict::Refuse(Refusal::AlreadyRunning),
),
("horizon", 4, &Request::Stop, Held::Asker, Verdict::Accept),
(
"horizon",
4,
&Request::Stop,
Held::Nothing,
Verdict::Refuse(Refusal::NotRunning),
),
(
"horizon",
4,
&Request::Stop,
Held::Someone,
Verdict::Refuse(Refusal::NotRunning),
),
];
for (brand, level, request, held, expected) in cases {
assert_eq!(
judge(brand, level, request, held),
expected,
"{brand:?} level {level} {request:?} {held:?}"
);
}
}
#[test]
fn only_a_horizon_brand_may_ask() {
let now = Instant::now();
for brand in ["vanilla", "", "horizon-studio", "Horizon"] {
let mut link = Link::default();
assert_eq!(link.ask(brand, now), Err(Unsent::NotHorizon), "{brand:?}");
assert!(link.is_idle());
}
let mut link = Link::default();
assert_eq!(link.ask("horizon", now), Ok(()));
assert_eq!(link.ask("horizon", now), Err(Unsent::Busy));
}
#[test]
fn a_capture_goes_from_asked_to_running_to_finished() {
let t0 = Instant::now();
let mut link = Link::default();
link.ask("horizon", t0).unwrap();
assert!(!link.running());
assert_eq!(link.stop(t0), Err(Unsent::NothingToStop));
assert_eq!(
link.answer(started("reports/p.hzperf", None), t0),
Outcome::Started {
path: "reports/p.hzperf".into(),
seconds: None,
samples: true
}
);
assert!(link.running());
assert_eq!(link.ask("horizon", t0), Err(Unsent::Busy));
assert_eq!(
link.answer(started("reports/p.hzperf", None), t0),
Outcome::Stray
);
assert!(link.running(), "a stray reply leaves the link as it was");
link.stop(t0).unwrap();
assert_eq!(link.stop(t0), Err(Unsent::NothingToStop));
assert!(link.running());
assert!(matches!(
link.answer(finished("reports/p.hzperf"), t0),
Outcome::Finished { .. }
));
assert!(link.is_idle());
}
#[test]
fn a_refusal_frees_the_link_and_a_window_end_finishes_a_running_capture() {
let t0 = Instant::now();
let mut link = Link::default();
link.ask("horizon", t0).unwrap();
assert_eq!(
link.answer(Reply::Refused(Refusal::Denied), t0),
Outcome::Refused(Refusal::Denied)
);
assert!(link.is_idle());
link.ask("horizon", t0).unwrap();
link.answer(started("p", Some(5)), t0);
assert!(matches!(
link.answer(finished("p"), t0),
Outcome::Finished { .. }
));
assert!(link.is_idle());
}
#[test]
fn an_unanswered_start_times_out_but_a_running_capture_does_not() {
let t0 = Instant::now();
let mut link = Link::default();
link.ask("horizon", t0).unwrap();
assert!(!link.timed_out(t0 + ANSWER_TIMEOUT));
assert!(link.timed_out(t0 + ANSWER_TIMEOUT + Duration::from_millis(1)));
assert!(link.is_idle());
link.ask("horizon", t0).unwrap();
link.answer(started("p", None), t0);
assert!(!link.timed_out(t0 + ANSWER_TIMEOUT * 100));
assert!(link.running());
}
#[test]
fn wanted_carries_every_switch_a_server_has_and_never_a_file() {
let options = StartOptions {
seconds: Some(9),
hz: Some(250),
over_ms: Some(40.0),
threads: Threads::All,
wall: true,
samples: false,
timeline: true,
file: Some("/etc/passwd".into()),
..StartOptions::default()
};
let back = Wanted::from(&options).options();
assert_eq!(
back,
StartOptions {
file: None,
..options
}
);
}
#[test]
fn a_chunk_is_a_tag_and_a_length_prefixed_array_and_no_bigger_than_the_limit() {
assert_eq!(Reply::Chunk(vec![9, 8, 7]).encode(), [3, 3, 9, 8, 7]);
let mut big = vec![3, 0x81, 0x80, 0x02];
big.extend(std::iter::repeat_n(0u8, CHUNK + 1));
assert!(Reply::decode(&big).is_err());
assert!(Reply::decode(&[3, 4, 1, 2]).is_err());
assert!(
Reply::decode(&[3, 1, 1, 0]).is_err(),
"bytes after the chunk"
);
}
#[test]
fn a_finished_reply_announces_the_file_size_as_a_varlong() {
let bytes = Reply::Finished {
path: "p".into(),
removed: Vec::new(),
bytes: 300,
}
.encode();
assert_eq!(bytes, [1, 1, b'p', 0xac, 0x02, 0]);
}
/// Everything the server would send, with the poll turns it takes, against a clock.
fn drain(total: u64, step: Duration, queued: impl Fn(Duration) -> usize) -> Vec<(Duration, usize)> {
let t0 = Instant::now();
let mut out = Outgoing::new(total, t0);
let mut sent = Vec::new();
let mut at = Duration::ZERO;
while !out.done() && at < Duration::from_secs(600) {
if let Some(n) = out.next(t0 + at, queued(at)) {
sent.push((at, n));
}
at += step;
}
sent
}
#[test]
fn the_transfer_sends_the_whole_file_in_32_kib_chunks() {
let total = 5 * CHUNK as u64 + 123;
let sent = drain(total, Duration::from_millis(50), |_| 0);
let sizes: Vec<usize> = sent.iter().map(|s| s.1).collect();
assert_eq!(sizes, [32768, 32768, 32768, 32768, 32768, 123]);
assert_eq!(sizes.iter().sum::<usize>() as u64, total);
}
#[test]
fn the_transfer_never_sends_more_than_256_kib_in_any_second() {
let polls = [1, 7, 50, 125, 130];
for poll_ms in polls {
let sent = drain(64 * 1024 * 1024, Duration::from_millis(poll_ms), |_| 0);
let last = sent.last().unwrap().0;
let mut worst = 0usize;
for (start, _) in &sent {
let end = *start + Duration::from_secs(1);
let in_second: usize = sent
.iter()
.filter(|(at, _)| at >= start && *at < end)
.map(|s| s.1)
.sum();
worst = worst.max(in_second);
}
assert!(
worst <= 256 * 1024,
"polled every {poll_ms} ms: {worst} bytes in a second"
);
let seconds = last.as_secs_f64();
let average = (sent.iter().map(|s| s.1).sum::<usize>() as f64) / seconds;
assert!(
average > 200.0 * 1024.0,
"polled every {poll_ms} ms: only {average} bytes/s"
);
}
}
#[test]
fn a_backlog_holds_the_transfer_back_until_gameplay_packets_are_out() {
let busy_until = Duration::from_secs(3);
let sent = drain(4 * CHUNK as u64, Duration::from_millis(50), |at| {
if at < busy_until {
BUSY_QUEUE + 1
} else {
BUSY_QUEUE
}
});
assert_eq!(sent.len(), 4);
assert!(sent.iter().all(|(at, _)| *at >= busy_until));
}
#[test]
fn the_file_arrives_chunk_by_chunk_and_the_last_one_frees_the_link() {
let t0 = Instant::now();
let mut link = Link::default();
link.ask("horizon", t0).unwrap();
link.answer(started("p", None), t0);
link.stop(t0).unwrap();
assert!(matches!(
link.answer(finished_with("p", 70_000), t0),
Outcome::Finished { bytes: 70_000, .. }
));
assert!(!link.is_idle(), "the file is still coming");
assert!(!link.running());
assert_eq!(link.ask("horizon", t0), Err(Unsent::Busy));
assert_eq!(link.chunk(32_768, t0), Received::More);
assert_eq!(link.chunk(32_768, t0), Received::More);
assert_eq!(link.chunk(4_464, t0), Received::Last);
assert!(link.is_idle());
assert_eq!(link.chunk(1, t0), Received::Stray);
}
#[test]
fn a_chunk_beyond_the_announced_size_ends_the_transfer() {
let t0 = Instant::now();
let mut link = Link::default();
link.ask("horizon", t0).unwrap();
link.answer(started("p", None), t0);
link.answer(finished_with("p", 10), t0);
assert_eq!(link.chunk(11, t0), Received::TooMuch);
assert!(link.is_idle());
}
#[test]
fn a_file_announced_over_the_cap_is_refused_and_its_chunks_are_stray() {
let t0 = Instant::now();
let mut link = Link::default();
link.ask("horizon", t0).unwrap();
link.answer(started("p", None), t0);
let bytes = horizon_perf_format::MAX_PART_BYTES + 1;
assert_eq!(
link.answer(finished_with("p", bytes), t0),
Outcome::TooLarge { bytes }
);
assert!(link.is_idle());
assert_eq!(link.chunk(32_768, t0), Received::Stray);
}
#[test]
fn a_server_that_fails_mid_file_frees_the_link() {
let t0 = Instant::now();
let mut link = Link::default();
link.ask("horizon", t0).unwrap();
link.answer(started("p", None), t0);
link.answer(finished_with("p", 100), t0);
assert_eq!(link.chunk(10, t0), Received::More);
assert_eq!(
link.answer(Reply::Refused(Refusal::Failed), t0),
Outcome::Refused(Refusal::Failed)
);
assert!(link.is_idle());
}
#[test]
fn a_silent_server_is_given_up_on_thirty_seconds_after_the_last_sign_of_life() {
let t0 = Instant::now();
let mut link = Link::default();
link.ask("horizon", t0).unwrap();
link.answer(started("p", None), t0);
assert!(
!link.stalled(t0 + STALL_TIMEOUT * 10),
"a running capture may run long"
);
link.stop(t0).unwrap();
assert!(!link.stalled(t0 + STALL_TIMEOUT));
assert!(link.stalled(t0 + STALL_TIMEOUT + Duration::from_millis(1)));
assert!(link.is_idle());
link.ask("horizon", t0).unwrap();
link.answer(started("p", None), t0);
link.stop(t0).unwrap();
link.answer(finished_with("p", 100), t0);
let t1 = t0 + Duration::from_secs(25);
assert_eq!(link.chunk(10, t1), Received::More);
assert!(!link.stalled(t1 + STALL_TIMEOUT));
assert!(link.stalled(t1 + STALL_TIMEOUT + Duration::from_millis(1)));
}
+1
View File
@@ -1301,6 +1301,7 @@ impl Projectile {
/// One tick: returns whether to keep the projectile and what it hit.
pub fn tick(&mut self, w: &impl ProjectileWorld) -> (bool, Hit) {
horizon_threads::work::set_entity(horizon_threads::work::entity_id(self.kind.id()));
self.tick_with(w, true, false)
}
/// One tick of a server's projectile on its client: [`tick`](Self::tick),
+836
View File
@@ -0,0 +1,836 @@
//! A side's .hzperf capture (docs/perf-platform.md §3, §4.1), the dedicated server's or the client's, and the always-on health history and slow ring behind it.
use crate::gfx::pack_timing::{GpuReadout, TimedFrames};
use crate::spans::Span;
use horizon_client_render::perf_gpu;
use horizon_perf_format::files;
use horizon_perf_format::v1::{
BenchResult, GpuMode, Kind, Mark, MemorySnapshot, Options, Overhead, Side, Start, Stop,
StopReason, ThreadSelection, Trigger, Unit, Warning, WarningCode,
};
use horizon_perf_format::{
CounterReporter, Field, FieldSender, Gauges, HealthRecorder, Interner, SlowRecord, SlowRing,
SlowSpan, WriterThread, counter_registry,
};
use std::fs::File;
use std::path::{Path, PathBuf};
use std::thread::{self, JoinHandle};
use std::time::{Instant, SystemTime, UNIX_EPOCH};
#[cfg(test)]
mod tests;
/// Tests that sample threads take turns: a sampler session arms every thread of the name it asks for, the other tests' too.
#[cfg(all(test, target_os = "linux"))]
pub(crate) static SAMPLED_TESTS: std::sync::Mutex<()> = std::sync::Mutex::new(());
mod samples;
pub mod spans;
mod system;
pub mod timeline;
pub mod traffic;
pub mod world;
pub use samples::MAX_WINDOW_S;
use samples::{Sampling, Stopped};
use spans::{SpanRecorder, Tick};
use system::{BackgroundProbe, SystemProbe};
pub use system::{
build, game_rules, hardware, hardware_now, horizon_options, machine, settings, warm_hardware,
};
use timeline::{Drained, TimelineRecorder};
use traffic::{TrafficRecorder, Wire};
use world::{LevelCounts, MobCosts, WorkerJobs};
/// What a capturing side is: which side of the file it writes and the unit and sampled thread it counts in.
#[derive(Clone, Copy)]
pub struct Role {
pub side: Side,
pub kind: Kind,
pub unit: Unit,
pub thread: &'static str,
/// The thread whose units the span recorder sums.
pub span_thread: &'static str,
/// Whether the process and machine probes run off the thread that feeds `tick`.
pub background_probe: bool,
}
impl Role {
pub const DEDICATED: Role = Role {
side: Side::Server,
kind: Kind::Dedicated,
unit: Unit::Tick,
thread: "tick",
span_thread: "Server thread",
background_probe: false,
};
pub const INTEGRATED: Role = Role {
side: Side::Server,
kind: Kind::Integrated,
unit: Unit::Tick,
thread: "tick",
span_thread: "Server thread",
background_probe: true,
};
pub const CLIENT: Role = Role {
side: Side::Client,
kind: Kind::Client,
unit: Unit::Frame,
thread: "render",
span_thread: "Render thread",
background_probe: true,
};
}
/// What a tick spent its time on, gathered by the tick loop only when the tick was slow or a capture runs.
pub struct TickDetail {
pub cpu_ms: Option<f32>,
/// As recorded in the tick, each span after the spans inside it.
pub spans: Vec<Span>,
}
/// The counts the health samples carry, read once a second.
#[derive(Clone, Copy, Default)]
pub struct WorldCounts {
pub players: u32,
pub chunks: u32,
pub entities: u32,
}
/// The sampler's switches (`profile start --hz --over --wall --threads`); the default is the sampler's own.
#[derive(Clone, Copy, Default)]
pub struct SamplerOptions {
pub hz: Option<u32>,
/// Only ticks or frames at least this slow are sampled; the slow ring's threshold when unset.
pub over_ms: Option<f32>,
pub wall: bool,
/// Every thread that opted in, not just the side's main thread.
pub all_threads: bool,
}
/// Switches of one capture.
pub struct CaptureOptions {
/// Unset when the capture runs until stopped, which the file records and the longest window ends.
pub window_s: Option<f64>,
pub trigger: Trigger,
/// Settings, machine and build for `Start`; the capture fills the rest.
pub start: Start,
/// Whether the side's main thread is sampled (where the platform can).
pub samples: bool,
pub sampler: SamplerOptions,
/// Block names by id, for the samples' contexts.
pub block_names: Vec<String>,
/// The levels as the capture begins.
pub levels: Vec<LevelCounts>,
/// The connections as the capture begins.
pub wire: Wire,
/// Whether the GPU's pass timings are recorded: unspecified on a side without a GPU.
pub gpu: GpuMode,
/// Records every thread's spans as `timeline` chunks within these caps, unless another capture already does.
pub timeline: Option<timeline::Caps>,
/// Write the capture at exactly this path, kept by no retention, instead of into the folder given to `start_capture`.
pub file: Option<PathBuf>,
}
/// What the game gathers as a capture ends.
#[derive(Default)]
pub struct CaptureEnd {
pub levels: Vec<LevelCounts>,
pub mob_costs: MobCosts,
pub jobs: Vec<WorkerJobs>,
pub wire: Wire,
/// What timing one span and one mob costs on this machine, in nanoseconds.
pub span_ns: f64,
pub mob_ns: f64,
/// The GPU pass timings, when the capture asked for them.
pub gpu: Option<GpuReadout>,
/// Benchmark reports to embed verbatim; the capture gives them its side.
pub bench: Vec<BenchResult>,
}
/// A finished capture: where it is and the older files retention removed.
pub struct Saved {
pub path: PathBuf,
pub deleted: Vec<PathBuf>,
}
/// Where a running capture is, for `profile status`.
pub struct CaptureStatus {
pub path: PathBuf,
pub elapsed_s: f64,
pub bytes: u64,
}
struct Session {
side: Side,
unit: Unit,
/// Unset when the session joined another side's file, which that side finishes.
writer: Option<WriterThread<File>>,
tx: FieldSender,
path: PathBuf,
/// The folder retention trims; none for a capture written to a path of its own.
dir: Option<PathBuf>,
/// The recorder's clock reading when the capture began.
origin: f64,
cursor: u64,
strings: Interner,
counters: CounterReporter,
sampling: Option<Sampling>,
block_names: Vec<String>,
spans: SpanRecorder,
traffic: TrafficRecorder,
gpu: GpuMode,
/// The GPU timers' frames and the window's CPU frame times, and when their last block was written.
gpu_log: perf_gpu::FrameLog,
gpu_block_t: f64,
timeline: Option<TimelineRecorder>,
}
enum Ending {
Running(JoinHandle<std::io::Result<Saved>>),
Done(std::io::Result<Saved>),
}
/// The server's health tiers and slow ring, always on, and the capture running from them, if any.
pub struct ServerPerf {
role: Role,
base: f64,
health: HealthRecorder,
slow: SlowRing,
ticks: u64,
gauges_at: f64,
system: SystemProbe,
background: Option<BackgroundProbe>,
frame_medians: (Option<f32>, Option<f32>),
session: Option<Session>,
ending: Vec<Ending>,
}
impl ServerPerf {
/// `start` is the clock reading (seconds) every later time is on; `slow_ms` is the slow ring's threshold.
pub fn new(start: f64, slow_ms: f32) -> Self {
Self::with_role(Role::DEDICATED, start, slow_ms)
}
pub fn with_role(role: Role, start: f64, slow_ms: f32) -> Self {
Self {
role,
base: start,
health: HealthRecorder::new(role.side, start),
slow: SlowRing::new(slow_ms),
ticks: 0,
gauges_at: start,
system: SystemProbe::default(),
background: None,
frame_medians: (None, None),
session: None,
ending: Vec::new(),
}
}
pub fn slow_threshold_ms(&self) -> f32 {
self.slow.threshold_ms()
}
/// The slow ring's ticks or frames, oldest first.
pub fn slow_records(&self) -> impl Iterator<Item = &SlowRecord> {
self.slow.records()
}
pub fn capturing(&self) -> bool {
self.session.is_some()
}
pub fn health(&self) -> &HealthRecorder {
&self.health
}
/// Whether the next `tick` reads the gauges, so a client can set the frame medians of the second first.
pub fn gauges_due(&self, t: f64) -> bool {
t - self.gauges_at >= 1.0
}
/// The median CPU and GPU time of the frames since the last second, for the gauges read next.
pub fn set_frame_medians(&mut self, cpu_ms: Option<f32>, gpu_ms: Option<f32>) {
self.frame_medians = (cpu_ms, gpu_ms);
}
/// One tick or frame that took `wall_ms` and ended at `t`; `world` and `wire` are read when the gauges are due.
pub fn tick(
&mut self,
t: f64,
wall_ms: f32,
detail: Option<TickDetail>,
world: impl FnOnce() -> WorldCounts,
wire: impl FnOnce() -> Wire,
) {
let first_background = self.role.background_probe && self.background.is_none();
let due = self.gauges_due(t);
let counts = (first_background || due).then(world);
if let (true, Some(counts)) = (first_background, counts) {
let probe = BackgroundProbe::spawn();
probe.request(t, counts);
self.background = Some(probe);
}
if let (true, Some(counts)) = (due, counts) {
self.gauges_at = t;
let mut g = match &self.background {
Some(probe) => {
probe.request(t, counts);
Gauges {
players: counts.players,
chunks: counts.chunks,
entities: counts.entities,
..probe.latest()
}
}
None => self.system.read(t, counts),
};
(g.frame_cpu_ms_p50, g.frame_gpu_ms_p50) = self.frame_medians;
self.health.set_gauges(g);
if let Some(session) = &mut self.session {
session.traffic.sample(t - session.origin, &wire());
}
}
self.ticks += 1;
self.health.record(t, wall_ms);
if wall_ms > self.slow.threshold_ms() {
let record = SlowRecord {
t,
unit: self.role.unit,
counter: self.ticks,
wall_ms,
cpu_ms: detail.as_ref().and_then(|d| d.cpu_ms).unwrap_or(0.0),
top: detail
.iter()
.flat_map(|d| crate::spans::merge(&d.spans))
.map(|s| SlowSpan {
name: s.name.to_owned(),
wall_ms: s.ms,
cpu_ms: s.cpu_ms,
depth: u32::from(s.depth),
})
.collect(),
};
if self.slow.offer(record)
&& let Some(session) = &mut self.session
&& let Some(kept) = self.slow.records().last()
{
session.slow(kept);
}
}
if let Some(session) = &mut self.session {
session.span_tick(t, self.ticks, wall_ms, detail.as_ref());
session.pump(&self.health, t);
session.timeline_tick(t);
}
}
/// The window's frame that ended at `t`: its CPU ms and the GPU frames timed since the last, written about once a second when the capture times the GPU.
pub fn gpu_frames(&mut self, t: f64, cpu_ms: f32, frames: &TimedFrames) {
let Some(session) = &mut self.session else {
return;
};
if matches!(session.gpu, GpuMode::Unspecified | GpuMode::Off) {
return;
}
let (now, origin) = (Instant::now(), session.origin);
session.gpu_log.cpu_frame(cpu_ms);
session.gpu_log.add(
frames,
|at| t - now.saturating_duration_since(at).as_secs_f64() - origin,
&mut session.strings,
);
if t - origin - session.gpu_block_t >= 1.0 {
session.gpu_block_t = t - origin;
session.gpu_block();
}
}
/// Opens `dir/profile-<time>Z.hzperf` with `Start`, the health tiers and the slow ring; call it on the tick thread, which it samples.
pub fn start_capture(
&mut self,
t: f64,
dir: &Path,
options: CaptureOptions,
) -> std::io::Result<PathBuf> {
if self.session.is_some() {
return Err(std::io::Error::other("a capture is already running"));
}
let now_ms = unix_ms();
let (file, path) = match &options.file {
Some(path) => (files::create_at(path)?, path.clone()),
None => files::create(dir, now_ms.div_euclid(1000))?,
};
let writer = WriterThread::spawn_with(file, profiler_thread)?;
let tx = writer.sender();
self.open(t, t, Some(writer), tx, path.clone(), dir, options);
Ok(path)
}
/// Writes this side into the file another side opened; `origin`, on this side's clock, is the zero of every `t`. Call it on the tick thread.
pub fn join_capture(
&mut self,
t: f64,
origin: f64,
tx: FieldSender,
path: PathBuf,
options: CaptureOptions,
) -> std::io::Result<()> {
if self.session.is_some() {
return Err(std::io::Error::other("a capture is already running"));
}
self.open(t, origin, None, tx, path, Path::new(""), options);
Ok(())
}
/// What another side needs to join the running capture: its sender, its file and the clock reading its `t` is zero at.
pub fn joinable(&self) -> Option<(FieldSender, PathBuf, f64)> {
let s = self.session.as_ref()?;
Some((s.tx.clone(), s.path.clone(), s.origin))
}
#[allow(clippy::too_many_arguments)]
fn open(
&mut self,
t: f64,
origin: f64,
writer: Option<WriterThread<File>>,
tx: FieldSender,
path: PathBuf,
dir: &Path,
options: CaptureOptions,
) {
let now_ms = unix_ms();
let sampling = options
.samples
.then(|| {
Sampling::start(
self.role.thread,
options.window_s.unwrap_or(MAX_WINDOW_S),
options.sampler.over_ms.unwrap_or(self.slow.threshold_ms()),
&options.sampler,
)
})
.flatten();
self.health.advance(t);
let (_, pushed) = self.health.samples_since(u64::MAX, t);
let partial = (t - self.base).fract() > 1e-9;
let timeline = options
.timeline
.and_then(|caps| TimelineRecorder::start(self.role.side, caps));
let mut session = Session {
side: self.role.side,
unit: self.role.unit,
writer,
tx,
path,
dir: options.file.is_none().then(|| dir.to_owned()),
origin,
cursor: pushed + u64::from(partial),
strings: Interner::default(),
counters: CounterReporter::new(self.role.side),
sampling,
block_names: options.block_names,
spans: SpanRecorder::new(self.role.span_thread, self.role.side, self.role.unit),
traffic: TrafficRecorder::new(options.wire),
gpu: options.gpu,
gpu_log: perf_gpu::FrameLog::default(),
gpu_block_t: 0.0,
timeline,
};
let mut start = options.start;
if let Some((used, total)) = system::disk_mb() {
start.world_fs_used_mb = used;
start.world_fs_total_mb = total;
}
start.side = self.role.side as i32;
start.kind = self.role.kind as i32;
start.trigger = options.trigger as i32;
start.started_unix_ms = now_ms;
start.uptime_s = t - self.base;
let sampled = session.sampling.is_some();
let threads = match (sampled, options.sampler.all_threads) {
(false, _) => ThreadSelection::Unspecified,
(true, false) => ThreadSelection::Main,
(true, true) => ThreadSelection::All,
};
start.options = Some(Options {
window_s: options.window_s,
hz: if sampled {
Sampling::hz(&options.sampler)
} else {
0
},
threads: threads as i32,
ticks_over_ms: options.sampler.over_ms.filter(|_| sampled),
wall: sampled && options.sampler.wall,
samples: sampled,
gpu: options.gpu as i32,
timeline: session.timeline.is_some(),
});
if sampled && let Some(build) = &mut start.build {
build.symbols = Sampling::symbols() as i32;
}
session.tx.send(Field::Start(start));
let mut world = world::snapshot("start", 0.0, &options.levels, None, &mut session.strings);
world.side = self.role.side as i32;
session.entries();
session.tx.send(Field::World(world));
for event in self
.slow
.snapshot(self.role.side, origin, &mut session.strings)
{
session.entries();
session.tx.send(Field::SlowHistory(event));
}
session.entries();
for window in self.health.history(origin) {
session.tx.send(Field::History(window));
}
self.session = Some(session);
}
/// Ends the running capture: writes the spans, workers, net and world here and finishes the file off the tick thread; false when none runs.
pub fn begin_stop(&mut self, t: f64, reason: StopReason, end: CaptureEnd) -> bool {
let Some(mut session) = self.session.take() else {
return false;
};
self.health.advance(t);
session.pump(&self.health, t);
let stop_t = t - session.origin;
let overhead = session.close(stop_t, end);
let stopped = session.sampling.take().and_then(Sampling::stop);
// The file's owner waits for the sides that joined it, so it never finishes on the caller's thread.
let ending = if stopped.is_none() && session.writer.is_none() {
Ending::Done(session.finish(None, stop_t, reason, overhead))
} else {
let worker = thread::Builder::new()
.name("horizon-perf-finish".into())
.spawn(move || {
profiler_thread();
session.finish(stopped, stop_t, reason, overhead)
});
match worker {
Ok(handle) => Ending::Running(handle),
Err(e) => Ending::Done(Err(e)),
}
};
self.ending.push(ending);
true
}
/// Whether a stopped capture's file is still being finished or waits in [`ServerPerf::take_saved`].
pub fn finishing(&self) -> bool {
!self.ending.is_empty()
}
/// The captures whose files are done, oldest first; with `wait`, every one that was begun.
pub fn take_saved(&mut self, wait: bool) -> Vec<std::io::Result<Saved>> {
let mut saved = Vec::new();
for ending in std::mem::take(&mut self.ending) {
match ending {
Ending::Done(result) => saved.push(result),
Ending::Running(handle) if wait || handle.is_finished() => {
saved.push(handle.join().unwrap_or_else(|_| {
Err(std::io::Error::other(
"the capture's finishing thread panicked",
))
}))
}
running => self.ending.push(running),
}
}
saved
}
/// Types `name` into the running capture at `t`; false when none runs.
pub fn mark(&mut self, t: f64, name: &str) -> bool {
let Some(session) = &mut self.session else {
return false;
};
session.tx.send(Field::Mark(Mark {
side: session.side as i32,
t: t - session.origin,
name: name.to_owned(),
}));
true
}
/// Records a memory snapshot, rows as flattened paths and values, in the running capture; false when none runs.
pub fn memory_snapshot(&mut self, t: f64, label: &str, rows: &[(String, f64)]) -> bool {
let Some(session) = &mut self.session else {
return false;
};
let label = session.strings.intern(label);
let path = rows
.iter()
.map(|(name, _)| session.strings.intern(name))
.collect();
session.entries();
session.tx.send(Field::Memory(MemorySnapshot {
side: session.side as i32,
t: t - session.origin,
label,
path,
value_mb: rows.iter().map(|(_, value)| *value).collect(),
}));
true
}
/// Records `code` at `t` in the running capture; false when none runs.
pub fn warn(&mut self, t: f64, code: WarningCode) -> bool {
let Some(session) = &mut self.session else {
return false;
};
session.tx.send(Field::Warning(Warning {
side: session.side as i32,
t: t - session.origin,
code: code as i32,
args: Vec::new(),
}));
true
}
/// The running capture's file, age and size so far; None when none runs.
pub fn status(&self, t: f64) -> Option<CaptureStatus> {
let session = self.session.as_ref()?;
Some(CaptureStatus {
path: session.path.clone(),
elapsed_s: t - session.origin,
bytes: std::fs::metadata(&session.path).map_or(0, |m| m.len()),
})
}
/// Ends the running capture and waits for its file; None when none runs.
pub fn stop_capture(
&mut self,
t: f64,
reason: StopReason,
end: CaptureEnd,
) -> Option<std::io::Result<Saved>> {
self.begin_stop(t, reason, end).then(|| {
self.take_saved(true)
.pop()
.unwrap_or_else(|| Err(std::io::Error::other("the capture vanished")))
})
}
}
/// What recording cost the tick, priced at the end of the capture: every span two clocks and two thread-CPU clocks, every sampled mob two clocks.
fn overhead(spans: &SpanRecorder, end: &CaptureEnd) -> Overhead {
let ticks = spans.count().max(1) as f64;
let mspt = spans.mean_wall_ms().max(1e-9);
let spans_ms = spans.spans_recorded() as f64 / ticks * end.span_ns / 1e6;
let mob_samples: u64 = end.mob_costs.costs.iter().map(|c| c.mob_ticks).sum();
let mob_ms = mob_samples as f64 / ticks * end.mob_ns / 1e6;
Overhead {
spans_ms_per_tick: spans_ms as f32,
spans_share: (spans_ms / mspt) as f32,
mob_timing_ms_per_tick: mob_ms as f32,
mob_timing_share: (mob_ms / mspt) as f32,
..Overhead::default()
}
}
impl Session {
fn entries(&mut self) {
for entry in self.strings.take_new(self.side) {
self.tx.send(Field::String(entry));
}
}
fn span_tick(&mut self, t: f64, counter: u64, wall_ms: f32, detail: Option<&TickDetail>) {
let tick = Tick {
t: t - self.origin,
counter,
wall_ms,
cpu_ms: detail.and_then(|d| d.cpu_ms),
spans: detail.map_or(&[], |d| &d.spans),
};
if let Some(block) = self.spans.unit(tick, &mut self.strings) {
self.entries();
self.tx.send(Field::SpanBlock(block));
}
}
/// Writes the spans, workers, net and world as the capture ends; returns what recording cost.
fn close(&mut self, rel: f64, end: CaptureEnd) -> Overhead {
let mut overhead = overhead(&self.spans, &end);
if let Some(timeline) = self.timeline.take() {
let (drained, stopped_t) = timeline.finish(rel, &mut self.strings);
self.send_timeline(drained);
overhead.timeline_stopped_t = stopped_t;
}
self.traffic.sample(rel, &end.wire);
let (last_block, summary) = std::mem::take(&mut self.spans).finish(&mut self.strings);
let mut traffic = std::mem::take(&mut self.traffic).finish(rel);
traffic.side = self.side as i32;
let mut workers = world::workers(&end.jobs, rel, &mut self.strings);
workers.side = self.side as i32;
let mut world = world::snapshot(
"stop",
rel,
&end.levels,
Some(&end.mob_costs),
&mut self.strings,
);
world.side = self.side as i32;
self.entries();
if let Some(block) = last_block {
self.tx.send(Field::SpanBlock(block));
}
self.tx.send(Field::Spans(summary));
self.tx.send(Field::Workers(workers));
self.tx.send(Field::Net(traffic));
self.gpu(rel, end.gpu.as_ref());
self.tx.send(Field::World(world));
for mut bench in end.bench {
bench.side = self.side as i32;
self.tx.send(Field::Bench(bench));
}
overhead
}
/// Writes the GPU timings the capture asked for, or a warning when there were none.
fn gpu(&mut self, rel: f64, readout: Option<&GpuReadout>) {
if matches!(self.gpu, GpuMode::Unspecified | GpuMode::Off) {
return;
}
self.gpu_block();
let sections = readout.map_or_else(Vec::new, |r| {
perf_gpu::gpu_passes(r, &self.gpu_log, self.side, &mut self.strings)
});
self.entries();
if sections.is_empty() {
self.tx.send(Field::Warning(Warning {
side: self.side as i32,
t: rel,
code: WarningCode::GpuUnavailable as i32,
args: Vec::new(),
}));
}
for gpu in sections {
self.tx.send(Field::Gpu(gpu));
}
}
fn gpu_block(&mut self) {
if let Some(block) = self.gpu_log.take_block(self.side) {
self.entries();
self.tx.send(Field::GpuFrames(block));
}
}
fn timeline_tick(&mut self, t: f64) {
let Some(timeline) = self.timeline.as_mut() else {
return;
};
if let Some(drained) = timeline.tick(t - self.origin, &mut self.strings) {
self.send_timeline(drained);
}
}
fn send_timeline(&mut self, drained: Drained) {
self.entries();
for chunk in drained.chunks {
self.tx.send(Field::Timeline(chunk));
}
if let Some(warning) = drained.warning {
self.tx.send(Field::Warning(warning));
}
}
fn slow(&mut self, record: &SlowRecord) {
let event = record.to_proto(self.side, self.origin, &mut self.strings);
self.entries();
self.tx.send(Field::Slow(event));
}
/// Appends the seconds closed since the last call, and counters with them.
fn pump(&mut self, health: &HealthRecorder, t: f64) {
let (samples, next) = health.samples_since(self.cursor, self.origin);
self.cursor = self.cursor.max(next);
if samples.is_empty() {
return;
}
for sample in samples {
self.tx.send(Field::Health(sample));
}
self.counters_now(t);
}
fn counters_now(&mut self, t: f64) {
let registry = counter_registry();
if registry.is_empty() {
return;
}
let report = self
.counters
.report(registry, &mut self.strings, t - self.origin);
self.entries();
for def in report.defs {
self.tx.send(Field::CounterDef(def));
}
self.tx.send(Field::Counters(report.sample));
}
/// Writes the samples (when there are any) and `Stop`, closes the file and applies retention.
fn finish(
mut self,
stopped: Option<Stopped>,
stop_t: f64,
reason: StopReason,
mut overhead: Overhead,
) -> std::io::Result<Saved> {
if let Some(stopped) = stopped {
let done = stopped.finish(self.side, self.unit, &self.block_names, &mut self.strings);
self.entries();
self.tx.send(Field::Samples(done.samples));
overhead = Overhead {
spans_ms_per_tick: overhead.spans_ms_per_tick,
spans_share: overhead.spans_share,
mob_timing_ms_per_tick: overhead.mob_timing_ms_per_tick,
mob_timing_share: overhead.mob_timing_share,
timeline_stopped_t: overhead.timeline_stopped_t,
..done.overhead
};
}
self.tx.send(Field::Stop(Stop {
side: self.side as i32,
t: stop_t,
reason: reason as i32,
overhead: Some(overhead),
}));
let Session {
writer,
tx,
path,
dir,
..
} = self;
drop(tx);
let Some(writer) = writer else {
return Ok(Saved {
path,
deleted: Vec::new(),
});
};
writer.finish()?;
let deleted = match dir {
Some(dir) => files::retain(&dir, &path, files::KEEP_FILES, files::KEEP_BYTES)?,
None => Vec::new(),
};
Ok(Saved { path, deleted })
}
}
fn unix_ms() -> i64 {
SystemTime::now()
.duration_since(UNIX_EPOCH)
.map_or(0, |d| d.as_millis() as i64)
}
/// Keeps the calling thread's allocations out of the heap samples: it is one of the profiler's own.
pub(crate) fn profiler_thread() {
#[cfg(target_os = "linux")]
horizon_threads::heap_sample::ignore_this_thread();
}
@@ -0,0 +1,161 @@
//! The sampler session that runs beside a dedicated server's capture and becomes its `samples` field.
use super::SamplerOptions;
use horizon_perf_format::Interner;
use horizon_perf_format::v1::{Overhead, SampleProfile, Side, Symbols, Unit};
/// The longest window a capture records, and the one a capture without a window gets.
pub const MAX_WINDOW_S: f64 = 600.0;
/// Seconds per time block: 5 s, longer when the window would need more than 32 of them.
fn block_s(window_s: f64) -> f32 {
(window_s / 32.0).ceil().max(5.0) as f32
}
pub struct Finished {
pub samples: SampleProfile,
pub overhead: Overhead,
}
#[cfg(target_os = "linux")]
pub use linux::{Sampling, Stopped};
#[cfg(not(target_os = "linux"))]
pub use other::{Sampling, Stopped};
#[cfg(target_os = "linux")]
mod linux {
use super::*;
use horizon_client_render::perf_capture;
use horizon_threads::heap_sample::{self, HeapProfile};
use horizon_threads::profile::{self, Profile, SymbolQuality};
use horizon_threads::sampler;
/// The CPU sampler, and the process's heap session when this capture was the first to start one.
pub struct Sampling(profile::Session, Option<heap_sample::Session>);
/// A stopped session, not yet converted: the part that can take seconds.
pub struct Stopped(Profile, Option<HeapProfile>);
impl Sampling {
/// Samples the calling thread, which must be the one `thread` names, until [`Sampling::stop`].
pub fn start(
thread: &'static str,
window_s: f64,
over_ms: f32,
options: &SamplerOptions,
) -> Option<Self> {
if !sampler::sampled_thread(thread) {
log::warn!(
"Horizon: this platform cannot sample the {thread} thread; capturing without samples"
);
return None;
}
let config = profile::Config {
sampler: sampler::Config {
hz: Sampling::hz(options),
wall: options.wall,
..sampler::Config::default()
},
threads: if options.all_threads {
Vec::new()
} else {
vec![thread.into()]
},
ticks_over_ms: f64::from(over_ms),
block_s: block_s(window_s),
..profile::Config::default()
};
match profile::start(config) {
Ok(session) => Some(Self(
session,
heap_sample::start(heap_sample::Config::from_env(block_s(window_s))),
)),
Err(e) => {
log::warn!(
"Horizon: could not start the sampler ({e}); capturing without samples"
);
None
}
}
}
pub fn hz(options: &SamplerOptions) -> u32 {
options.hz.unwrap_or(sampler::DEFAULT_HZ)
}
pub fn symbols() -> Symbols {
match profile::symbol_quality() {
SymbolQuality::Dwarf => Symbols::Dwarf,
SymbolQuality::Symtab => Symbols::Symtab,
SymbolQuality::None => Symbols::None,
}
}
/// Cancels the timers and drains what is left; quick.
pub fn stop(self) -> Option<Stopped> {
let heap = self.1.and_then(heap_sample::Session::stop);
match self.0.stop() {
Ok(profile) => Some(Stopped(profile, heap)),
Err(e) => {
log::warn!("Horizon: the sampler failed ({e}); the capture has no samples");
None
}
}
}
}
impl Stopped {
/// Symbolizes and converts; reads the running binary's debug info, so run it off the tick thread.
pub fn finish(
self,
side: Side,
unit: Unit,
block_names: &[String],
strings: &mut Interner,
) -> Finished {
let name = |id: u16| block_names.get(usize::from(id)).cloned();
Finished {
samples: perf_capture::sample_profile(
&self.0,
self.1.as_ref(),
side,
unit,
&name,
strings,
),
overhead: perf_capture::overhead(&self.0, self.1.as_ref()),
}
}
}
}
#[cfg(not(target_os = "linux"))]
mod other {
use super::*;
pub enum Sampling {}
pub enum Stopped {}
impl Sampling {
pub fn start(_: &'static str, _: f64, _: f32, _: &SamplerOptions) -> Option<Self> {
None
}
pub fn hz(_: &SamplerOptions) -> u32 {
0
}
pub fn symbols() -> Symbols {
Symbols::Unspecified
}
pub fn stop(self) -> Option<Stopped> {
match self {}
}
}
impl Stopped {
pub fn finish(self, _: Side, _: Unit, _: &[String], _: &mut Interner) -> Finished {
match self {}
}
}
}
@@ -0,0 +1,348 @@
//! The tick thread's spans as the capture's `span_blocks` (live) and `spans` (at the end).
use crate::fast_hash::FxHashMap;
use crate::spans::Span;
use horizon_perf_format::Interner;
use horizon_perf_format::v1::{
Histogram, Side, SpanBlock, SpanCost, SpanSummary, SpanTotal, Unit, WorstUnit,
};
/// Seconds one `SpanBlock` covers.
pub const BLOCK_S: f32 = 5.0;
/// Ticks kept with their whole span tree.
pub const WORST_KEPT: usize = 16;
/// Upper edges of the tick-time histogram, in ms.
pub const HISTOGRAM_EDGES_MS: [f32; 14] = [
5.0, 10.0, 15.0, 20.0, 25.0, 30.0, 40.0, 50.0, 75.0, 100.0, 150.0, 250.0, 500.0, 1000.0,
];
type Key = (&'static str, &'static str);
/// Wall and CPU milliseconds of one span under one parent, and the units it ran in.
#[derive(Clone, Copy, Default)]
struct Sum {
wall_ms: f64,
cpu_ms: f64,
seen: u64,
depth: u32,
}
struct Worst {
t: f64,
counter: u64,
wall_ms: f32,
cpu_ms: f32,
spans: Vec<Span>,
}
/// The index in `spans` of each span's enclosing span; spans are recorded after the spans inside them, one level deeper.
fn parents(spans: &[Span]) -> Vec<Option<usize>> {
let mut parent = vec![None; spans.len()];
let mut open: Vec<usize> = Vec::new();
for (i, span) in spans.iter().enumerate() {
while let Some(&j) = open.last()
&& spans[j].depth > span.depth
{
open.pop();
parent[j] = Some(i);
}
open.push(i);
}
parent
}
/// One tick's spans merged by (name, parent name), with the tree depth of the first of each.
fn merge_by_parent(spans: &[Span]) -> Vec<(Key, Sum)> {
let parent = parents(spans);
let mut depth = vec![0u32; spans.len()];
for i in (0..spans.len()).rev() {
depth[i] = parent[i].map_or(0, |p| depth[p] + 1);
}
let mut merged: Vec<(Key, Sum)> = Vec::new();
let mut index: FxHashMap<Key, usize> = FxHashMap::default();
for (i, span) in spans.iter().enumerate() {
let key = (span.name, parent[i].map_or("", |p| spans[p].name));
let at = *index.entry(key).or_insert_with(|| {
merged.push((
key,
Sum {
depth: depth[i],
..Sum::default()
},
));
merged.len() - 1
});
merged[at].1.wall_ms += f64::from(span.ms);
merged[at].1.cpu_ms += f64::from(span.cpu_ms);
}
merged
}
/// A tick's whole span tree, parents before children, siblings costliest first; same-named spans under one parent are one node.
fn tree(spans: &[Span], strings: &mut Interner) -> Vec<SpanCost> {
let parent = parents(spans);
struct Node {
name: &'static str,
wall_ms: f32,
cpu_ms: f32,
children: Vec<usize>,
}
let mut nodes: Vec<Node> = Vec::new();
let mut roots: Vec<usize> = Vec::new();
let mut node_of: Vec<usize> = vec![usize::MAX; spans.len()];
let mut by_parent: FxHashMap<(Option<usize>, &'static str), usize> = FxHashMap::default();
for i in (0..spans.len()).rev() {
let up = parent[i].map(|p| node_of[p]);
let at = *by_parent.entry((up, spans[i].name)).or_insert_with(|| {
nodes.push(Node {
name: spans[i].name,
wall_ms: 0.0,
cpu_ms: 0.0,
children: Vec::new(),
});
let id = nodes.len() - 1;
match up {
Some(p) => nodes[p].children.push(id),
None => roots.push(id),
}
id
});
nodes[at].wall_ms += spans[i].ms;
nodes[at].cpu_ms += spans[i].cpu_ms;
node_of[i] = at;
}
let mut out = Vec::with_capacity(nodes.len());
let mut stack: Vec<(usize, u32)> = Vec::new();
let costliest_last = |ids: &mut Vec<usize>, nodes: &[Node]| {
ids.sort_by(|&a, &b| {
nodes[a]
.wall_ms
.total_cmp(&nodes[b].wall_ms)
.then_with(|| nodes[b].name.cmp(nodes[a].name))
});
};
costliest_last(&mut roots, &nodes);
stack.extend(roots.iter().map(|&id| (id, 0)));
while let Some((id, depth)) = stack.pop() {
let node = &nodes[id];
out.push(SpanCost {
name: strings.intern(node.name),
wall_ms: node.wall_ms,
cpu_ms: node.cpu_ms,
depth,
});
let mut children = node.children.clone();
costliest_last(&mut children, &nodes);
stack.extend(children.iter().map(|&c| (c, depth + 1)));
}
out
}
/// One server tick as the recorder takes it.
pub struct Tick<'a> {
/// Seconds into the capture when the tick ended.
pub t: f64,
/// The server's tick counter.
pub counter: u64,
pub wall_ms: f32,
/// The thread's CPU time; None where the platform does not tell it.
pub cpu_ms: Option<f32>,
/// As recorded: each span after the spans inside it.
pub spans: &'a [Span],
}
/// Sums the server thread's ticks into blocks of [`BLOCK_S`] seconds and, at the end, into the capture's totals.
pub struct SpanRecorder {
thread: &'static str,
side: Side,
unit: Unit,
block: u32,
units: u32,
rows: FxHashMap<Key, Sum>,
totals: FxHashMap<Key, Sum>,
count: u64,
wall_total: f64,
cpu_total: f64,
timing: horizon_perf_format::TimingRecorder,
histogram: [u64; HISTOGRAM_EDGES_MS.len() + 1],
worst: Vec<Worst>,
recorded: u64,
}
impl Default for SpanRecorder {
fn default() -> Self {
Self::new("", Side::Server, Unit::Tick)
}
}
impl SpanRecorder {
pub fn new(thread: &'static str, side: Side, unit: Unit) -> Self {
Self {
thread,
side,
unit,
block: 0,
units: 0,
rows: FxHashMap::default(),
totals: FxHashMap::default(),
count: 0,
wall_total: 0.0,
cpu_total: 0.0,
timing: horizon_perf_format::TimingRecorder::default(),
histogram: [0; HISTOGRAM_EDGES_MS.len() + 1],
worst: Vec::new(),
recorded: 0,
}
}
/// Ticks recorded so far.
pub fn count(&self) -> u64 {
self.count
}
/// Spans the ticks recorded themselves, before they were merged: what timing them cost.
pub fn spans_recorded(&self) -> u64 {
self.recorded
}
/// The mean tick time in ms.
pub fn mean_wall_ms(&self) -> f64 {
self.wall_total / self.count.max(1) as f64
}
/// Adds one tick; returns the block it closed, if it began a new one.
pub fn unit(&mut self, tick: Tick, strings: &mut Interner) -> Option<SpanBlock> {
let Tick {
t,
counter,
wall_ms,
cpu_ms,
spans,
} = tick;
let block = (t.max(0.0) / f64::from(BLOCK_S)) as u32;
let closed = if block != self.block {
let closed = self.close_block(strings);
self.block = block;
closed
} else {
None
};
let cpu_ms = cpu_ms.unwrap_or(wall_ms);
self.units += 1;
self.count += 1;
self.timing.record(wall_ms);
self.recorded += spans.len() as u64;
self.wall_total += f64::from(wall_ms);
self.cpu_total += f64::from(cpu_ms);
self.histogram[HISTOGRAM_EDGES_MS.partition_point(|&edge| edge <= wall_ms)] += 1;
for (key, sum) in merge_by_parent(spans) {
for rows in [&mut self.rows, &mut self.totals] {
let row = rows.entry(key).or_insert(Sum {
depth: sum.depth,
..Sum::default()
});
row.wall_ms += sum.wall_ms;
row.cpu_ms += sum.cpu_ms;
row.seen += 1;
}
}
if self.worst.len() < WORST_KEPT || self.worst.iter().any(|w| w.wall_ms < wall_ms) {
if self.worst.len() >= WORST_KEPT {
let cheapest = (0..self.worst.len())
.min_by(|&a, &b| self.worst[a].wall_ms.total_cmp(&self.worst[b].wall_ms))
.unwrap_or(0);
self.worst.swap_remove(cheapest);
}
self.worst.push(Worst {
t,
counter,
wall_ms,
cpu_ms,
spans: spans.to_vec(),
});
}
closed
}
fn close_block(&mut self, strings: &mut Interner) -> Option<SpanBlock> {
let units = std::mem::take(&mut self.units);
let mut rows: Vec<_> = std::mem::take(&mut self.rows).into_iter().collect();
if units == 0 {
return None;
}
rows.sort_by(|a, b| b.1.wall_ms.total_cmp(&a.1.wall_ms).then(a.0.cmp(&b.0)));
let mut block = SpanBlock {
side: self.side as i32,
thread: strings.intern(self.thread),
block: self.block,
t0: f64::from(self.block) * f64::from(BLOCK_S),
block_s: BLOCK_S,
units,
..SpanBlock::default()
};
for ((name, parent), sum) in rows {
block.span.push(strings.intern(name));
block.parent.push(if parent.is_empty() {
0
} else {
strings.intern(parent)
});
block.wall_ms.push(sum.wall_ms as f32);
block.cpu_ms.push(sum.cpu_ms as f32);
block.seen.push(sum.seen as u32);
}
Some(block)
}
/// The block still open and the capture's totals.
pub fn finish(mut self, strings: &mut Interner) -> (Option<SpanBlock>, SpanSummary) {
let last = self.close_block(strings);
let mut totals: Vec<_> = std::mem::take(&mut self.totals).into_iter().collect();
totals.sort_by(|a, b| b.1.wall_ms.total_cmp(&a.1.wall_ms).then(a.0.cmp(&b.0)));
let mut worst = std::mem::take(&mut self.worst);
worst.sort_by(|a, b| {
b.wall_ms
.total_cmp(&a.wall_ms)
.then(a.counter.cmp(&b.counter))
});
let thread = strings.intern(self.thread);
let summary = SpanSummary {
side: self.side as i32,
thread,
unit: self.unit as i32,
count: self.count,
spans: totals
.into_iter()
.map(|((name, parent), sum)| SpanTotal {
name: strings.intern(name),
parent: if parent.is_empty() {
0
} else {
strings.intern(parent)
},
depth: sum.depth,
wall_ms: sum.wall_ms,
cpu_ms: sum.cpu_ms,
seen: sum.seen,
})
.collect(),
histogram: Some(Histogram {
edges_ms: HISTOGRAM_EDGES_MS.to_vec(),
counts: self.histogram.to_vec(),
}),
worst: worst
.into_iter()
.map(|w| WorstUnit {
t: w.t,
counter: w.counter,
wall_ms: w.wall_ms,
cpu_ms: w.cpu_ms,
spans: tree(&w.spans, strings),
})
.collect(),
wall_ms_total: self.wall_total,
cpu_ms_total: self.cpu_total,
timing: self.timing.finish(),
};
(last, summary)
}
}
@@ -0,0 +1,261 @@
use crate::gamerules::GameRules as Rules;
use horizon_perf_format::Gauges;
use horizon_perf_format::v1::{Build, GameRules, HorizonOptions, Machine, Settings};
use std::sync::OnceLock;
use super::WorldCounts;
/// The disk the game runs from, where its worlds are saved: used and total MiB.
pub(super) fn disk_mb() -> Option<(u64, u64)> {
const MIB: u64 = 1 << 20;
let dir = std::env::current_dir().ok()?;
let total = fs4::total_space(&dir).ok()?;
let free = fs4::available_space(&dir).ok()?;
Some((total.saturating_sub(free) / MIB, total / MIB))
}
/// Reads the process and the machine for the health gauges; the CPU figures are rates since the last read.
#[derive(Default)]
pub(super) struct SystemProbe {
last: Option<(f64, f64, Option<(u64, u64)>)>,
}
impl SystemProbe {
pub(super) fn read(&mut self, t: f64, counts: WorldCounts) -> Gauges {
let pid = std::process::id();
let cpu_s = horizon_bench::cpu_seconds(pid);
let stat = std::fs::read_to_string("/proc/stat")
.ok()
.and_then(|s| parse_cpu_stat(&s));
let threads = std::thread::available_parallelism().map_or(1, |n| n.get()) as f64;
let mut g = Gauges {
rss_mb: horizon_bench::rss_bytes(pid).map_or(0.0, |b| (b >> 20) as f32),
players: counts.players,
chunks: counts.chunks,
entities: counts.entities,
..Gauges::default()
};
if let Some((used, swap)) = std::fs::read_to_string("/proc/meminfo")
.ok()
.and_then(|s| parse_meminfo(&s))
{
g.physical_used_mb = used as f32;
g.swap_used_mb = swap as f32;
}
if let Some((last_t, last_cpu, last_stat)) = self.last
&& t > last_t
{
if let Some(cpu) = cpu_s {
g.cpu_process_cores = ((cpu - last_cpu) / (t - last_t)) as f32;
}
if let (Some((busy, total)), Some((last_busy, last_total))) = (stat, last_stat)
&& total > last_total
{
g.cpu_system_cores =
((busy - last_busy) as f64 / (total - last_total) as f64 * threads) as f32;
}
}
self.last = Some((t, cpu_s.unwrap_or(0.0), stat));
g
}
}
/// A `SystemProbe` on a thread of its own, for the sides whose frame must not wait on the probes (macOS runs `ps`).
pub(super) struct BackgroundProbe {
requests: std::sync::mpsc::Sender<(f64, WorldCounts)>,
latest: std::sync::Arc<std::sync::Mutex<Gauges>>,
}
impl BackgroundProbe {
pub(super) fn spawn() -> Self {
let (requests, wanted) = std::sync::mpsc::channel::<(f64, WorldCounts)>();
let latest = std::sync::Arc::new(std::sync::Mutex::new(Gauges::default()));
let shared = std::sync::Arc::clone(&latest);
let worker = std::thread::Builder::new()
.name("horizon-perf-gauges".into())
.spawn(move || {
super::profiler_thread();
let mut probe = SystemProbe::default();
for (t, counts) in wanted {
let gauges = probe.read(t, counts);
*shared.lock().unwrap_or_else(|e| e.into_inner()) = gauges;
}
});
if let Err(e) = worker {
log::warn!("Horizon: no thread for the health gauges ({e})");
}
Self { requests, latest }
}
/// Asks for a reading at `t`; the answer shows in [`BackgroundProbe::latest`] a moment later.
pub(super) fn request(&self, t: f64, counts: WorldCounts) {
let _ = self.requests.send((t, counts));
}
pub(super) fn latest(&self) -> Gauges {
*self.latest.lock().unwrap_or_else(|e| e.into_inner())
}
}
/// `/proc/stat`'s first line as (busy, total) jiffies: everything but idle and iowait is busy.
pub(super) fn parse_cpu_stat(text: &str) -> Option<(u64, u64)> {
let fields: Vec<u64> = text
.lines()
.next()?
.strip_prefix("cpu ")?
.split_whitespace()
.filter_map(|f| f.parse().ok())
.collect();
if fields.len() < 5 {
return None;
}
let total: u64 = fields.iter().take(8).sum();
Some((total - fields[3] - fields[4], total))
}
/// `/proc/meminfo` as (physical MB in use, swap MB in use); in use is total minus available.
pub(super) fn parse_meminfo(text: &str) -> Option<(u64, u64)> {
let kb = |key: &str| -> Option<u64> {
text.lines()
.find_map(|l| l.strip_prefix(key)?.strip_prefix(':'))?
.split_whitespace()
.next()?
.parse()
.ok()
};
let used = kb("MemTotal")?.checked_sub(kb("MemAvailable")?)?;
let swap = kb("SwapTotal")?.saturating_sub(kb("SwapFree")?);
Some((used >> 10, swap >> 10))
}
/// The cgroup v2 `cpu.max` as cores: "200000 100000" is 2; "max 100000" is no limit.
pub(super) fn parse_cgroup_cpu(text: &str) -> Option<f32> {
let mut words = text.split_whitespace();
let quota: f64 = words.next()?.parse().ok()?;
let period: f64 = words.next()?.parse().ok()?;
(period > 0.0).then(|| (quota / period) as f32)
}
/// The cgroup v2 `memory.max` in MB; "max" is no limit.
pub(super) fn parse_cgroup_memory(text: &str) -> Option<u64> {
text.trim().parse::<u64>().ok().map(|b| b >> 20)
}
static HARDWARE: OnceLock<horizon_bench::Hardware> = OnceLock::new();
/// Describes the hardware on a thread of its own: the probes run commands, which the tick must not wait for.
pub fn warm_hardware() {
std::thread::spawn(|| {
HARDWARE.get_or_init(horizon_bench::Hardware::detect);
});
}
/// The hardware, described now if nothing has yet: the probes run commands, so not on a tick.
pub fn hardware_now() -> &'static horizon_bench::Hardware {
HARDWARE.get_or_init(horizon_bench::Hardware::detect)
}
/// The hardware, once `warm_hardware` has described it.
pub fn hardware() -> Option<&'static horizon_bench::Hardware> {
HARDWARE.get()
}
/// What the machine and its container give the process, for `Start.machine`.
pub fn machine(hardware: Option<&horizon_bench::Hardware>) -> Machine {
let cgroup = |name: &str| std::fs::read_to_string(format!("/sys/fs/cgroup/{name}")).ok();
let mut m = Machine {
os: std::env::consts::OS.to_owned(),
arch: std::env::consts::ARCH.to_owned(),
cpu_threads: std::thread::available_parallelism().map_or(0, |n| n.get() as u32),
cgroup_cpu_cores: cgroup("cpu.max").and_then(|t| parse_cgroup_cpu(&t)),
cgroup_memory_mb: cgroup("memory.max").and_then(|t| parse_cgroup_memory(&t)),
..Machine::default()
};
if let Some(h) = hardware {
m.os_version = h.os_version.clone().unwrap_or_default();
m.cpu = h.cpu.clone().unwrap_or_default();
m.cpu_cores = h.cpu_cores.map_or(0, u32::from);
m.ram_mb = h.ram_mb.unwrap_or(0);
}
m
}
/// The build this server runs, for `Start.build`.
pub fn build() -> Build {
Build {
version: crate::crash::version(),
profile: if cfg!(debug_assertions) {
"debug"
} else {
"release"
}
.to_owned(),
target: format!("{}-{}", std::env::consts::ARCH, std::env::consts::OS),
..Build::default()
}
}
/// The server settings a capture records: the typed allowlist, nothing else from `server.properties`.
pub fn settings(view: u32, simulation: u32, max_players: u32, compression: i32) -> Settings {
Settings {
view_distance: Some(view),
simulation_distance: Some(simulation),
max_players: Some(max_players),
network_compression_threshold: Some(compression),
}
}
/// The game rules a capture records: the typed allowlist, nothing else from the world's rule set.
pub fn game_rules(rules: &Rules) -> GameRules {
GameRules {
random_tick_speed: Some(rules.int("random_tick_speed")),
max_entity_cramming: Some(rules.int("max_entity_cramming")),
fire_spread_radius_around_player: Some(rules.int("fire_spread_radius_around_player")),
advance_time: Some(rules.bool("advance_time")),
mob_griefing: Some(rules.bool("mob_griefing")),
spawn_mobs: Some(rules.bool("spawn_mobs")),
spawn_monsters: Some(rules.bool("spawn_monsters")),
spawner_blocks_work: Some(rules.bool("spawner_blocks_work")),
}
}
/// The Horizon switches a capture records: the typed allowlist, never the environment or `server.toml`.
pub fn horizon_options(
max_view_distance: u32,
lod_distance: u32,
anticheat: bool,
) -> HorizonOptions {
HorizonOptions {
max_view_distance: Some(max_view_distance),
lod_distance: Some(lod_distance),
anticheat: Some(anticheat),
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn cpu_stat_busy_excludes_idle_and_iowait() {
// user nice system idle iowait irq softirq steal
let text = "cpu 100 20 30 700 50 5 5 90 0 0\ncpu0 1 1 1 1 1 1 1 1 0 0\n";
assert_eq!(parse_cpu_stat(text), Some((250, 1000)));
assert_eq!(parse_cpu_stat("intr 1 2 3"), None);
}
#[test]
fn meminfo_used_is_total_minus_available() {
let text = "MemTotal: 16384000 kB\nMemFree: 1 kB\nMemAvailable: 8192000 kB\nSwapTotal: 4096000 kB\nSwapFree: 3072000 kB\n";
assert_eq!(parse_meminfo(text), Some((8000, 1000)));
}
#[test]
fn cgroup_limits() {
assert_eq!(parse_cgroup_cpu("250000 100000\n"), Some(2.5));
assert_eq!(parse_cgroup_cpu("max 100000\n"), None);
assert_eq!(parse_cgroup_memory("2147483648\n"), Some(2048));
assert_eq!(parse_cgroup_memory("max\n"), None);
}
}
File diff suppressed because it is too large. Load diff
@@ -0,0 +1,150 @@
use super::*;
use crate::spans;
use horizon_perf_format::v1::{TimelineChunk, WarningCode};
use timeline::Caps;
/// Runs `f` on a thread named `name`, whose spans the timeline files under that name.
fn on_thread(name: &str, f: impl FnOnce() + Send + 'static) {
std::thread::Builder::new()
.name(name.into())
.spawn(f)
.unwrap()
.join()
.unwrap();
}
fn sleep_ms(ms: u64) {
std::thread::sleep(std::time::Duration::from_millis(ms));
}
fn chunks_of<'a>(c: &'a Capture, thread: &str) -> Vec<&'a TimelineChunk> {
c.timeline
.iter()
.filter(|chunk| text(c, chunk.thread) == thread)
.collect()
}
fn timeline_options(caps: Caps) -> CaptureOptions {
CaptureOptions {
timeline: Some(caps),
..options(100.0)
}
}
#[test]
fn a_known_span_sequence_reads_back_with_its_durations_and_depths() {
let _timeline = timeline::test_lock();
let scratch = Scratch::new("timeline-known");
let mut perf = ServerPerf::new(0.0, 250.0);
perf.start_capture(0.0, &scratch.0, timeline_options(Caps::default()))
.unwrap();
on_thread("tl-known", || {
spans::time("tick", || {
sleep_ms(5);
spans::time("tick_mobs", || sleep_ms(10));
spans::time("tick_blocks", || sleep_ms(6));
});
});
for i in 0..40 {
perf.tick(
f64::from(i) * 0.1,
10.0,
None,
WorldCounts::default,
Vec::new,
);
}
let saved = stop(&mut perf, 4.0, R::Normal).unwrap().unwrap();
let c = read(&saved.path);
assert!(c.start[0].options.as_ref().unwrap().timeline);
assert!(c.warnings.is_empty());
assert_eq!(
c.stop[0].overhead.as_ref().unwrap().timeline_stopped_t,
None
);
let chunks = chunks_of(&c, "tl-known");
assert_eq!(
chunks.len(),
1,
"one chunk for the one second they ended in"
);
let chunk = chunks[0];
assert_eq!(chunk.side(), S::Server);
let names: Vec<_> = chunk.name.iter().map(|&id| text(&c, id)).collect();
assert_eq!(names, ["tick", "tick_mobs", "tick_blocks"]);
assert_eq!(chunk.depth, [0, 1, 1]);
let (tick, mobs, blocks) = (0, 1, 2);
assert!(chunk.dur_us[mobs] >= 10_000 && chunk.dur_us[blocks] >= 6_000);
assert!(chunk.dur_us[tick] >= chunk.dur_us[mobs] + chunk.dur_us[blocks] + 5_000);
let end = |i: usize| chunk.start_us[i] + u64::from(chunk.dur_us[i]);
assert!(chunk.start_us[mobs] >= chunk.start_us[tick] + 5_000);
assert!(chunk.start_us[blocks] >= end(mobs));
assert!(end(blocks) <= end(tick));
assert!(
!spans::timeline::on(),
"stopping the capture stops the timeline"
);
}
#[test]
fn the_time_cap_stops_the_timeline_and_says_when() {
let _timeline = timeline::test_lock();
let scratch = Scratch::new("timeline-cap");
let mut perf = ServerPerf::new(0.0, 250.0);
let caps = Caps {
seconds: 3.0,
..Caps::default()
};
perf.start_capture(0.0, &scratch.0, timeline_options(caps))
.unwrap();
for i in 0..12 {
let t = f64::from(i) * 0.5;
on_thread("tl-cap", || spans::time("tick", || ()));
perf.tick(t, 10.0, None, WorldCounts::default, Vec::new);
}
assert!(
!spans::timeline::on(),
"the cap stopped the timeline at t = 3"
);
let saved = stop(&mut perf, 6.0, R::Normal).unwrap().unwrap();
let c = read(&saved.path);
assert_eq!(
c.stop[0].overhead.as_ref().unwrap().timeline_stopped_t,
Some(3.0)
);
assert_eq!(c.warnings.len(), 1);
assert_eq!(c.warnings[0].code(), WarningCode::TimelineCapped);
assert_eq!(c.warnings[0].t, 3.0);
assert_eq!(c.warnings[0].args[0], 3.0);
assert!(c.warnings[0].args[1] > 0.0);
let chunks = chunks_of(&c, "tl-cap");
assert_eq!(chunks.len(), 3, "the drains at 1, 2 and 3 s, none after");
let spans_kept: usize = chunks.iter().map(|chunk| chunk.name.len()).sum();
assert_eq!(
spans_kept, 7,
"a span per half second, up to the one at 3 s"
);
}
#[test]
fn a_second_capture_does_not_take_over_a_running_timeline() {
let _timeline = timeline::test_lock();
let scratch = Scratch::new("timeline-twice");
let (mut first, mut second) = (ServerPerf::new(0.0, 250.0), ServerPerf::new(0.0, 250.0));
first
.start_capture(0.0, &scratch.0, timeline_options(Caps::default()))
.unwrap();
second
.start_capture(0.0, &scratch.0, timeline_options(Caps::default()))
.unwrap();
let second = stop(&mut second, 1.0, R::Normal).unwrap().unwrap();
assert!(
spans::timeline::on(),
"the second capture left the first one's timeline alone"
);
let first = stop(&mut first, 1.0, R::Normal).unwrap().unwrap();
let on = |path: &Path| read(path).start[0].options.as_ref().unwrap().timeline;
assert_eq!((on(&first.path), on(&second.path)), (true, false));
}
@@ -0,0 +1,278 @@
//! The spans of every thread as the capture's `timeline` chunks, within the caps of docs/perf-platform.md §3.7.
use crate::spans::timeline::{self, ThreadEvents};
use horizon_perf_format::v1::{Side, TimelineChunk, Warning, WarningCode};
use horizon_perf_format::{Interner, ProtoMessage};
use std::time::Instant;
/// Seconds between two drains: one chunk per thread each.
pub const CHUNK_S: f64 = 1.0;
/// The timeline stops this long after it began.
pub const MAX_S: f64 = 60.0;
/// The timeline stops when its chunks would pass this many bytes (before compression).
pub const MAX_BYTES: u64 = 256 << 20;
/// When the timeline stops on its own.
#[derive(Clone, Copy, Debug, PartialEq)]
pub struct Caps {
pub seconds: f64,
pub bytes: u64,
}
impl Default for Caps {
fn default() -> Self {
Self {
seconds: MAX_S,
bytes: MAX_BYTES,
}
}
}
/// What a drain produced.
#[derive(Default)]
pub struct Drained {
pub chunks: Vec<TimelineChunk>,
/// Set by the drain that hit a cap.
pub warning: Option<Warning>,
}
/// A capture's claim on the process's timeline: it drains the threads' spans into chunks and enforces the caps.
pub struct TimelineRecorder {
side: Side,
caps: Caps,
drained_at: f64,
bytes: u64,
live: bool,
/// Whether the process's timeline still runs for this recorder.
holds: bool,
stopped_t: Option<f64>,
}
impl Drop for TimelineRecorder {
fn drop(&mut self) {
if self.holds {
timeline::finish();
}
}
}
impl TimelineRecorder {
/// Starts the timeline, timed from now; None when another capture already has it.
pub fn start(side: Side, caps: Caps) -> Option<Self> {
timeline::start(Instant::now()).then(|| Self {
side,
caps,
drained_at: 0.0,
bytes: 0,
live: true,
holds: true,
stopped_t: None,
})
}
/// Seconds into the capture when a cap stopped the timeline.
pub fn stopped_t(&self) -> Option<f64> {
self.stopped_t
}
/// At capture time `t`: a second's chunks when one has passed, and the cap check; None in between or once stopped.
pub fn tick(&mut self, t: f64, strings: &mut Interner) -> Option<Drained> {
if !self.live || (t - self.drained_at < CHUNK_S && t < self.caps.seconds) {
return None;
}
self.drained_at = t;
Some(self.absorb(t, timeline::drain(), strings))
}
/// Ends the timeline at capture time `t` with the spans still held.
pub fn finish(mut self, t: f64, strings: &mut Interner) -> (Drained, Option<f64>) {
let rest = timeline::finish();
self.holds = false;
let drained = if self.live {
self.absorb(t, rest, strings)
} else {
Drained::default()
};
(drained, self.stopped_t)
}
fn absorb(&mut self, t: f64, threads: Vec<ThreadEvents>, strings: &mut Interner) -> Drained {
let mut out = Drained::default();
for thread in threads {
let chunk = chunk(self.side, thread, strings);
let len = chunk.encoded_len() as u64;
if self.bytes + len > self.caps.bytes {
self.cap(t, &mut out);
break;
}
self.bytes += len;
out.chunks.push(chunk);
}
if t >= self.caps.seconds {
self.cap(t, &mut out);
}
out
}
fn cap(&mut self, t: f64, out: &mut Drained) {
if !self.live {
return;
}
self.live = false;
self.stopped_t = Some(t);
out.warning = Some(Warning {
side: self.side as i32,
t,
code: WarningCode::TimelineCapped as i32,
args: vec![t, self.bytes as f64],
});
if self.holds {
self.holds = false;
timeline::finish();
}
}
}
/// One thread's spans as a chunk, in start order, parents before the spans inside them.
fn chunk(side: Side, thread: ThreadEvents, strings: &mut Interner) -> TimelineChunk {
let mut events = thread.events;
events.sort_by_key(|e| (e.start_us, e.depth));
let mut chunk = TimelineChunk {
side: side as i32,
thread: strings.intern(&thread.thread),
..TimelineChunk::default()
};
for e in events {
chunk.name.push(strings.intern(e.name));
chunk.start_us.push(e.start_us);
chunk.dur_us.push(e.dur_us);
chunk.depth.push(u32::from(e.depth));
}
chunk
}
/// The process has one timeline: tests that run one take this first.
#[cfg(test)]
pub(crate) fn test_lock() -> std::sync::MutexGuard<'static, ()> {
static LOCK: std::sync::Mutex<()> = std::sync::Mutex::new(());
LOCK.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner)
}
#[cfg(test)]
mod tests {
use super::*;
use crate::spans::timeline::Event;
use horizon_perf_format::v1::StringEntry;
fn event(name: &'static str, start_us: u64, dur_us: u32, depth: u8) -> Event {
Event {
name,
start_us,
dur_us,
depth,
}
}
fn recorder(caps: Caps) -> TimelineRecorder {
TimelineRecorder {
side: Side::Server,
caps,
drained_at: 0.0,
bytes: 0,
live: true,
holds: false,
stopped_t: None,
}
}
fn thread(name: &str, events: Vec<Event>) -> ThreadEvents {
ThreadEvents {
thread: name.into(),
events,
}
}
fn text(entries: &[StringEntry], id: u32) -> &str {
&entries.iter().find(|e| e.id == id).unwrap().text
}
#[test]
fn a_chunk_holds_its_threads_spans_in_start_order_with_their_durations_and_depths() {
let mut strings = Interner::default();
let events = vec![
event("tick_mobs", 2_000, 700, 1),
event("tick_blocks", 2_800, 150, 1),
event("tick", 2_000, 1_000, 0),
];
let chunk = chunk(Side::Server, thread("Server thread", events), &mut strings);
let entries = strings.take_new(Side::Server);
assert_eq!(chunk.side(), Side::Server);
assert_eq!(text(&entries, chunk.thread), "Server thread");
let names: Vec<_> = chunk.name.iter().map(|&id| text(&entries, id)).collect();
assert_eq!(names, ["tick", "tick_mobs", "tick_blocks"]);
assert_eq!(chunk.start_us, [2_000, 2_000, 2_800]);
assert_eq!(chunk.dur_us, [1_000, 700, 150]);
assert_eq!(chunk.depth, [0, 1, 1]);
}
#[test]
fn the_time_cap_stops_the_timeline_once_and_says_when() {
let mut strings = Interner::default();
let mut recorder = recorder(Caps {
seconds: 60.0,
bytes: MAX_BYTES,
});
let spans = || vec![thread("w", vec![event("x", 0, 10, 0)])];
let before = recorder.absorb(59.0, spans(), &mut strings);
assert_eq!(before.chunks.len(), 1);
assert!(before.warning.is_none() && recorder.stopped_t().is_none());
let at = recorder.absorb(60.25, spans(), &mut strings);
assert_eq!(at.chunks.len(), 1, "the spans up to the cap are kept");
assert_eq!(recorder.stopped_t(), Some(60.25));
let warning = at.warning.unwrap();
assert_eq!(warning.code(), WarningCode::TimelineCapped);
assert_eq!(warning.side(), Side::Server);
assert_eq!(warning.t, 60.25);
assert_eq!(warning.args[0], 60.25);
assert!(recorder.tick(61.0, &mut strings).is_none());
assert!(
recorder
.absorb(62.0, spans(), &mut strings)
.warning
.is_none()
);
}
#[test]
fn the_byte_cap_keeps_the_chunks_that_fit_and_drops_the_rest() {
let mut strings = Interner::default();
let busy = |name: &str, n: u64| {
thread(
name,
(0..n).map(|i| event("span", i * 100, 50, 0)).collect(),
)
};
let first = chunk(Side::Server, busy("a", 100), &mut Interner::default());
let fit = first.encoded_len() as u64;
let mut recorder = recorder(Caps {
seconds: 60.0,
bytes: fit + 5,
});
let drained = recorder.absorb(7.0, vec![busy("a", 100), busy("b", 100)], &mut strings);
assert_eq!(drained.chunks.len(), 1);
assert_eq!(drained.chunks[0].start_us.len(), 100);
assert_eq!(recorder.stopped_t(), Some(7.0));
let warning = drained.warning.unwrap();
assert_eq!(warning.code(), WarningCode::TimelineCapped);
assert_eq!(warning.args, [7.0, fit as f64]);
}
#[test]
fn nothing_drains_before_a_second_has_passed() {
let mut strings = Interner::default();
let mut recorder = recorder(Caps::default());
assert!(recorder.tick(0.4, &mut strings).is_none());
assert!(recorder.tick(0.99, &mut strings).is_none());
}
}
@@ -0,0 +1,136 @@
//! Each player's wire bytes per block of the capture, as `NetTraffic`, with the player's name, client brand and view distance.
use crate::server_perf::spans::BLOCK_S;
use horizon_perf_format::v1::{NetTraffic, PlayerTraffic, Side};
/// One connection as read now: its wire bytes (received, sent) since it opened, and who plays on it.
#[derive(Clone, Debug, Default, PartialEq, Eq)]
pub struct Conn {
pub id: u32,
pub bytes: (u64, u64),
pub name: String,
/// The client's `minecraft:brand`, empty until it sends one.
pub brand: String,
/// The chunk radius this client is sent.
pub view_distance: u32,
}
/// The open connections, read together.
pub type Wire = Vec<Conn>;
struct Player {
conn: u32,
name: String,
brand: String,
view_distance: u32,
joined_t: f64,
left_t: Option<f64>,
last: (u64, u64),
bytes_in: Vec<u64>,
bytes_out: Vec<u64>,
}
/// Turns the running byte counts of the connections, read now and then, into bytes per block; a leaver's bytes since the last read are lost.
#[derive(Default)]
pub struct TrafficRecorder {
players: Vec<Player>,
}
fn block_of(t: f64) -> usize {
(t.max(0.0) / f64::from(BLOCK_S)) as usize
}
impl TrafficRecorder {
/// `wire` is read as the capture starts: those players were on already, and only the bytes after it count.
pub fn new(mut wire: Wire) -> Self {
wire.sort_by_key(|c| c.id);
Self {
players: wire
.into_iter()
.map(|c| Player {
conn: c.id,
name: c.name,
brand: c.brand,
view_distance: c.view_distance,
joined_t: 0.0,
left_t: None,
last: c.bytes,
bytes_in: Vec::new(),
bytes_out: Vec::new(),
})
.collect(),
}
}
/// The connections' counts `t` seconds into the capture; a new connection counts from zero, an unlisted one has left.
pub fn sample(&mut self, t: f64, wire: &[Conn]) {
let block = block_of(t);
for c in wire {
let (now_in, now_out) = c.bytes;
let at = match self.players.iter().position(|p| p.conn == c.id) {
Some(at) => at,
None => {
self.players.push(Player {
conn: c.id,
name: String::new(),
brand: String::new(),
view_distance: 0,
joined_t: t,
left_t: None,
last: (0, 0),
bytes_in: Vec::new(),
bytes_out: Vec::new(),
});
self.players.len() - 1
}
};
let p = &mut self.players[at];
if !c.name.is_empty() {
p.name.clone_from(&c.name);
}
if !c.brand.is_empty() {
p.brand.clone_from(&c.brand);
}
p.view_distance = c.view_distance;
p.bytes_in.resize(block + 1, 0);
p.bytes_out.resize(block + 1, 0);
p.bytes_in[block] += now_in.saturating_sub(p.last.0);
p.bytes_out[block] += now_out.saturating_sub(p.last.1);
p.last = (now_in, now_out);
}
for p in &mut self.players {
if p.left_t.is_none() && !wire.iter().any(|c| c.id == p.conn) {
p.left_t = Some(t);
}
}
}
/// The capture's traffic once it ended `t` seconds in; every player's arrays are as long as the capture.
pub fn finish(self, t: f64) -> NetTraffic {
let blocks = block_of(t) + 1;
NetTraffic {
side: Side::Server as i32,
block_s: BLOCK_S,
players: self
.players
.into_iter()
.enumerate()
.map(|(i, mut p)| {
p.bytes_in.resize(blocks, 0);
p.bytes_out.resize(blocks, 0);
PlayerTraffic {
index: i as u32 + 1,
joined_t: p.joined_t,
left_t: p.left_t,
bytes_in: p.bytes_in,
bytes_out: p.bytes_out,
packets_in: Vec::new(),
packets_out: Vec::new(),
name: p.name,
brand: p.brand,
view_distance: p.view_distance,
}
})
.collect(),
}
}
}
@@ -0,0 +1,107 @@
//! What the capture records of the world: entity and chunk counts, what mobs cost, what the chunk workers spent.
use horizon_perf_format::Interner;
use horizon_perf_format::v1::{
DimensionCounts, EntityCost, Side, TypeCount, WorkerKind, Workers, WorldSnapshot,
};
/// Entity types costed in a snapshot, costliest first.
pub const ENTITY_COSTS_KEPT: usize = 40;
/// One level's chunks and its entities and block entities by registry id.
#[derive(Clone, Debug, Default, PartialEq)]
pub struct LevelCounts {
pub dimension: String,
pub chunks: u32,
pub entities: Vec<(String, u32)>,
pub block_entities: Vec<(String, u32)>,
}
/// The sampled tick time of one entity type in one level: nanoseconds over `mob_ticks` ticks of single mobs.
#[derive(Clone, Debug, PartialEq)]
pub struct MobCost {
pub dimension: String,
pub kind: String,
pub nanos: u64,
pub mob_ticks: u64,
}
/// The mob costs sampled over `sampled_ticks` server ticks, costliest first.
#[derive(Clone, Debug, Default, PartialEq)]
pub struct MobCosts {
pub costs: Vec<MobCost>,
pub sampled_ticks: u64,
}
/// Chunk-worker time of one job kind: nanoseconds over `count` jobs.
#[derive(Clone, Debug, PartialEq)]
pub struct WorkerJobs {
pub kind: String,
pub nanos: u64,
pub count: u64,
}
fn counts(types: &[(String, u32)], strings: &mut Interner) -> Vec<TypeCount> {
let mut sorted: Vec<&(String, u32)> = types.iter().collect();
sorted.sort_by(|a, b| b.1.cmp(&a.1).then_with(|| a.0.cmp(&b.0)));
sorted
.into_iter()
.map(|(kind, count)| TypeCount {
r#type: strings.intern(kind),
count: *count,
})
.collect()
}
/// The world `t` seconds into the capture, named `label`; `costs` is what mobs cost, at the end.
pub fn snapshot(
label: &str,
t: f64,
levels: &[LevelCounts],
costs: Option<&MobCosts>,
strings: &mut Interner,
) -> WorldSnapshot {
let ticks = costs.map_or(1, |c| c.sampled_ticks.max(1)) as f64;
WorldSnapshot {
side: Side::Server as i32,
t,
label: strings.intern(label),
dims: levels
.iter()
.map(|level| DimensionCounts {
dimension: strings.intern(&level.dimension),
chunks: level.chunks,
entities: counts(&level.entities, strings),
block_entities: counts(&level.block_entities, strings),
})
.collect(),
entity_costs: costs
.into_iter()
.flat_map(|c| &c.costs)
.take(ENTITY_COSTS_KEPT)
.map(|c| EntityCost {
dimension: strings.intern(&c.dimension),
entity: strings.intern(&c.kind),
ms_per_tick: (c.nanos as f64 / ticks / 1e6) as f32,
mobs_ticked_per_tick: (c.mob_ticks as f64 / ticks) as f32,
us_per_mob: (c.nanos as f64 / c.mob_ticks.max(1) as f64 / 1e3) as f32,
})
.collect(),
}
}
/// The chunk workers' CPU by job kind over a capture of `window_s` seconds; kinds that ran no job are left out.
pub fn workers(jobs: &[WorkerJobs], window_s: f64, strings: &mut Interner) -> Workers {
Workers {
side: Side::Server as i32,
kinds: jobs
.iter()
.filter(|j| j.count > 0)
.map(|j| WorkerKind {
kind: strings.intern(&j.kind),
jobs: j.count,
cpu_cores: (j.nanos as f64 / 1e9 / window_s.max(1e-9)) as f32,
ms_per_job: (j.nanos as f64 / j.count as f64 / 1e6) as f32,
})
.collect(),
}
}
@@ -244,9 +244,10 @@ pub fn roll_container(
))
});
if let Some(loot) = loot {
// LootTable.fill spreads the items over the container's own size.
let rolled = loot.fill(&table, seed, slots.len(), Some(biome));
for (slot, stack) in slots.iter_mut().zip(rolled) {
// LootTable.fill shuffles only available slots and keeps occupied ones.
let empty = slots.iter().filter(|slot| slot.is_none()).count();
let rolled = loot.fill(&table, seed, empty, Some(biome));
for (slot, stack) in slots.iter_mut().filter(|slot| slot.is_none()).zip(rolled) {
// With what the rolls put on them: enchantments
// (enchant_with_levels in End city chests), potions and an
// ominous bottle's level.
Loaded 100 of 347 files, more files were not shown because too many files have changed in this diff. Show more