mirror of
https://github.com/horizon-rust/horizon.git
synced 2026-11-07 11:06:11 -10:00
Merge pull request #670 from ludovic111/feat/server-bench-equal
Server benchmark against vanilla 26.3 on equal work
This commit is contained in:
9 files changed
+2508
-39
No files matched your search
@@ -0,0 +1,168 @@
|
||||
# Horizon's server against vanilla 26.3, equal work (2026-10-17)
|
||||
|
||||
The same bots on Horizon's dedicated server and on vanilla 26.3's, one server at a time, on the same
|
||||
machine, with the work each server does made the same before the numbers are compared. This replaces the
|
||||
first run in docs/server.md section 9, where the work wasn't equal.
|
||||
|
||||
## What was made equal
|
||||
|
||||
The first run (PR #556) had three differences that made the comparison unfair. Each is fixed in the code
|
||||
that ran here:
|
||||
|
||||
- **Refused moves.** Vanilla pulls a player back when a move is too fast ("moved too quickly") or ends
|
||||
inside a block ("moved wrongly"). Horizon now runs both checks the way vanilla does (PR #586), so a bot
|
||||
that flies too fast is pulled back on both servers.
|
||||
- **Chunk sending.** Vanilla paces chunks by the client's acknowledgements (PlayerChunkSender: one
|
||||
unacknowledged batch at first, then ten, each at most the client's desired chunks per tick). Horizon used
|
||||
to send twice as many per batch; it now paces them the same way (PR #586), and the bots acknowledge
|
||||
batches and ask for a rate the way vanilla's client does, so both servers send at the pace the client
|
||||
asks for. Chunks sent again while the bot still held
|
||||
them (`resent`) are counted: 0 on both servers in every run.
|
||||
- **Movement gated on chunks.** A bot doesn't move while the chunk it stands in hasn't arrived (vanilla's
|
||||
client waits the same way), so a slower server isn't asked to teleport bots into terrain it hasn't sent.
|
||||
|
||||
One difference isn't in Horizon yet: vanilla 26.3's locator bar (waypoints), which costs vanilla O(players²)
|
||||
work per move. Every vanilla run here sets `gamerule locator_bar false`, so neither server does it.
|
||||
|
||||
## Method
|
||||
|
||||
`tools/server_bench.py` runs each configuration on a fresh server:
|
||||
|
||||
- the same `server.properties` on both (seed 12345, view and simulation distance 10, offline, creative,
|
||||
difficulty easy, `sync-chunk-writes=true`), and `gamerule respawn_radius 0`;
|
||||
- the server waits for `Done`, the bots join one a second, a 60-second warm-up, then 300 seconds of
|
||||
measurement;
|
||||
- MSPT (average, P50, P95, P99) comes from each server's own `/tick query`, asked every 5 seconds on the
|
||||
console, so both measure themselves the same way (vanilla's ServerTickRateManager numbers, which Horizon
|
||||
reproduces); TPS is 20 while the average fits in 50 ms, else 1000 / MSPT;
|
||||
- RSS and CPU are the server's whole process tree, once a second (100 % is one core);
|
||||
- chunk counts come from the bots: chunks received per second, chunks no bot had before (new terrain), and
|
||||
pull-backs (position packets that move a bot back).
|
||||
|
||||
The bots are `horizon-bots` (creative, flying 24 blocks above spawn, 20 moves a second):
|
||||
|
||||
- `walk`: flies to random points within 48 blocks of spawn at sprint speed (5.6 blocks a second);
|
||||
- `explore`: flies straight outwards, by default at about 22 blocks a second, as fast as the server sends
|
||||
chunks.
|
||||
|
||||
Machine: the ThinkPad (Intel Core i7-7500U, 2 cores / 4 threads, 32 GB, NixOS), servers under `nice -n 19`
|
||||
because it also runs other services; vanilla on OpenJDK 25.0.4.1 with a 4 GB heap (`-Xms4g -Xmx4g`). The
|
||||
bots ran on the Mac mini and reached the ThinkPad over the tailnet, so they took none of the server's CPU.
|
||||
|
||||
What ran: Horizon from `origin/main` at bf851bfe with #586 (move checks and chunk pacing), the bots and
|
||||
`server_bench.py` from this PR, #623 (sections encoded once, its first commit) and #649 (a tick's packets
|
||||
handed to each connection together). The ThinkPad copy wasn't a git checkout, so the JSON reports have an
|
||||
empty `commit`. Vanilla: the pinned 26.3 server jar.
|
||||
|
||||
## Results
|
||||
|
||||
MSPT is the average milliseconds per tick (50 is the limit for 20 TPS); P95 is the 95th percentile. CPU is
|
||||
the whole process tree. "New chunks/s" are chunks no bot had before.
|
||||
|
||||
### Walk (the same work on both servers)
|
||||
|
||||
| Bots | Server | MSPT | P95 | TPS | RSS | CPU | Chunks/s | New chunks/s |
|
||||
|---:|---|---:|---:|---:|---:|---:|---:|---:|
|
||||
| 1 | Horizon | 5.5 | 12.1 | 20.0 | 1.37 GB | 57 % | 9 | 0.3 |
|
||||
| 1 | Vanilla | 18.3 | 27.3 | 20.0 | 3.35 GB | 66 % | 10 | 0.3 |
|
||||
| 10 | Horizon | 11.5 | 22.1 | 20.0 | 1.41 GB | 124 % | 94 | 2.4 |
|
||||
| 10 | Vanilla | 16.8 | 30.5 | 20.0 | 3.41 GB | 84 % | 100 | 2.8 |
|
||||
| 25 | Horizon | 20.7 | 40.4 | 20.0 | 1.45 GB | 170 % | 236 | 5.7 |
|
||||
| 25 | Vanilla | 16.0 | 26.4 | 20.0 | 3.47 GB | 115 % | 252 | 6.7 |
|
||||
| 50 | Horizon | 31.4 | 79.0 | 20.0 | 1.52 GB | 173 % | 451 | 10.3 |
|
||||
| 50 | Vanilla | 20.5 | 29.7 | 20.0 | 3.55 GB | 194 % | 502 | 12.2 |
|
||||
| 100 | Horizon | 114.5 | 256.2 | 8.7 | 1.33 GB | 181 % | 484 | 30.8 |
|
||||
| 100 | Vanilla | stopped by its watchdog | | | | | | |
|
||||
|
||||
Both servers sent the same chunks (about 10 a second per bot, the churn of flying around) and sent none
|
||||
twice. Vanilla pulled back a few bots for moving too quickly when a tick ran long (5 to 33 per run);
|
||||
Horizon none. At 100 bots vanilla's watchdog stopped the server (a tick ran over 60 s, in
|
||||
`ChunkMap$TrackedEntity.updatePlayer`); Horizon kept running at 8.7 TPS.
|
||||
|
||||
### Explore at a rate both servers keep up with (equal work)
|
||||
|
||||
The bots fly outwards at 0.8 blocks a second (`--explore-speed 0.8`), slow enough that neither server makes
|
||||
them wait for terrain, so both generate and send the same new chunks.
|
||||
|
||||
| Bots | Server | MSPT | P95 | TPS | RSS | CPU | Chunks/s | New chunks/s |
|
||||
|---:|---|---:|---:|---:|---:|---:|---:|---:|
|
||||
| 1 | Horizon | 4.0 | 6.6 | 20.0 | 1.36 GB | 25 % | 2 | 1.3 |
|
||||
| 1 | Vanilla | 18.1 | 27.1 | 20.0 | 3.32 GB | 73 % | 2 | 1.4 |
|
||||
| 10 | Horizon | 13.5 | 24.8 | 20.0 | 1.64 GB | 89 % | 13 | 11.8 |
|
||||
| 10 | Vanilla | 29.2 | 43.0 | 20.0 | 3.52 GB | 143 % | 13 | 11.9 |
|
||||
| 25 | Horizon | 20.2 | 36.8 | 20.0 | 1.74 GB | 121 % | 32 | 28.9 |
|
||||
| 25 | Vanilla | 37.0 | 49.4 | 20.0 | 3.73 GB | 169 % | 32 | 28.9 |
|
||||
|
||||
### Explore as fast as each server sends chunks (not equal work)
|
||||
|
||||
The default 22 blocks a second is faster than either server can generate, so each server sets the pace and
|
||||
a faster server is given more work: from 10 bots Horizon generated 1.5 to 2 times vanilla's new chunks a
|
||||
second. Read it as throughput, not as a like-for-like tick time.
|
||||
|
||||
| Bots | Server | MSPT | P95 | TPS | RSS | CPU | Chunks/s | New chunks/s |
|
||||
|---:|---|---:|---:|---:|---:|---:|---:|---:|
|
||||
| 1 | Horizon | 10.6 | 24.6 | 20.0 | 1.34 GB | 322 % | 39 | 34.6 |
|
||||
| 1 | Vanilla | 9.9 | 21.3 | 20.0 | 3.48 GB | 245 % | 36 | 33.4 |
|
||||
| 10 | Horizon | 30.4 | 52.3 | 20.0 | 2.18 GB | 345 % | 47 | 46.7 |
|
||||
| 10 | Vanilla | 17.0 | 29.6 | 20.0 | 4.04 GB | 242 % | 22 | 22.4 |
|
||||
| 25 | Horizon | 50.7 | 89.3 | 19.7 | 2.30 GB | 286 % | 50 | 49.7 |
|
||||
| 25 | Vanilla | 24.2 | 36.5 | 20.0 | 4.40 GB | 320 % | 34 | 33.7 |
|
||||
| 50 | Horizon | 50.7 | 102.7 | 19.7 | 1.72 GB | 213 % | 97 | 96.5 |
|
||||
| 50 | Vanilla | stopped by its watchdog | | | | | | |
|
||||
| 100 | Horizon | 92.9 | 139.6 | 10.8 | 1.70 GB | 322 % | 114 | 113.6 |
|
||||
| 100 | Vanilla | stopped by its watchdog | | | | | | |
|
||||
|
||||
Vanilla's watchdog stopped it at 50 bots (in `ClientboundLightUpdatePacketData`, copying light for a chunk
|
||||
packet) and at 100 (in `Entity.move` from a move packet).
|
||||
|
||||
## Findings
|
||||
|
||||
- Horizon's RSS is 1.9 to 2.6 times smaller in every run, and it starts in 11 s against vanilla's 15 s.
|
||||
Vanilla's heap is fixed at 4 GB (`-Xms4g`), so part of its RSS is that setting rather than need.
|
||||
- With 1 bot Horizon's tick is 3.3 times shorter walking and 4.5 times shorter exploring at equal work;
|
||||
at 10 bots it is still shorter in both.
|
||||
- At equal exploring work Horizon's tick is about half of vanilla's at 10 and 25 bots.
|
||||
- When the bots only fly around spawn (`walk`), vanilla's tick hardly grows with players (16 to 20 ms from
|
||||
1 to 50 bots) while Horizon's grows by about 0.5 ms per bot, so vanilla is ahead from about 20 bots: 16.0
|
||||
against 20.7 ms at 25 and 20.5 against 31.4 ms at 50. Both keep 20 TPS up to 50.
|
||||
- With 100 bots vanilla's watchdog stops it in every pass, and Horizon keeps running at 9 to 11 TPS.
|
||||
|
||||
### Where Horizon's walk tick goes at 25 bots
|
||||
|
||||
Profiled on the ThinkPad with `perf` (the server thread, 20 s, self time):
|
||||
|
||||
| Share | What | Whose |
|
||||
|---:|---|---|
|
||||
| 18 % | `Streaming::update`: every tick it walks all loaded columns and proto chunks and tests each against every player's area (unloading, dropping protos, finishing and decoration candidates), so the cost grows with players times chunks | streaming / engine tick |
|
||||
| 20 % | `PalettedContainer::get` and `World::block`, mostly from block entity ticks: on the Mac profile `tick_hoppers` looks up the block under every block entity in the loaded world each tick and compares names as strings, and spawners rebuild their display id | engine tick |
|
||||
| 8 % | `light_data` when a chunk is sent: the light layers were built twice per chunk; a follow-up commit on #623 builds them once and keeps each section's packed layers | server (this work) |
|
||||
| 7 % | encoding chunk sections and heightmaps for the chunks each bot newly sees | server |
|
||||
| 4 % | `Hub::tick` (moves, tracking) | server |
|
||||
| 2 % | the moved-wrongly collision check | server |
|
||||
|
||||
The first two grow with the number of players but aren't specific to the server: they are the
|
||||
engine's tick and streaming, reported to perf-server-2. Before #623 and #649, encoding every chunk section
|
||||
once per player and waking each connection's writer thread once per packet cost more than everything above
|
||||
(at 50 walking bots the tick was 281 ms; see the earlier runs).
|
||||
|
||||
## Earlier runs (same machine and method)
|
||||
|
||||
- `run-1-before-chunk-cache.json`: before #623, with vanilla's locator bar on. Horizon's walk tick was
|
||||
27.5, 40.7, 281 and 467 ms at 10, 25, 50 and 100 bots, and vanilla's 33.2 ms at 50 (the locator bar).
|
||||
- `run-2-chunk-cache.json`: Horizon with #623 only: 13.2, 23.5, 31.4 and 84.4 ms walking at 10 to 100
|
||||
bots.
|
||||
|
||||
## Files
|
||||
|
||||
- `horizon.json`, `vanilla.json`: the walk and explore passes (vanilla with `locator_bar=false`).
|
||||
- `explore-equal-rate.json`: the explore pass at 0.8 blocks a second, both servers.
|
||||
- `run-1-before-chunk-cache.json`, `run-2-chunk-cache.json`: the earlier runs.
|
||||
|
||||
To run it again (bots on another machine reached at `--connect`):
|
||||
|
||||
cargo build --release -p horizon-client --bin horizon-client -p horizon-server -p horizon-bots
|
||||
python3 tools/server_bench.py --servers horizon,vanilla --bots 1,10,25,50,100 --behaviour walk,explore \
|
||||
--gamerule locator_bar=false --nice 19 --bots-via 'ssh macmini' --bots-bin /path/to/horizon-bots \
|
||||
--connect <this machine's address> --out results.json
|
||||
python3 tools/server_bench.py --servers horizon,vanilla --bots 1,10,25 --behaviour explore \
|
||||
--explore-speed 0.8 --gamerule locator_bar=false --nice 19 ... --out explore-equal-rate.json
|
||||
@@ -0,0 +1,267 @@
|
||||
{
|
||||
"kind": "horizon-server-benchmark",
|
||||
"date": "2026-10-17T05:47:12+00:00",
|
||||
"machine": {
|
||||
"system": "Linux",
|
||||
"machine": "x86_64",
|
||||
"cpus": 4
|
||||
},
|
||||
"commit": "",
|
||||
"java": [
|
||||
"openjdk 25.0.4.1 2026-08-18"
|
||||
],
|
||||
"settings": {
|
||||
"servers": "horizon,vanilla",
|
||||
"bots": "1,10,25",
|
||||
"behaviour": "explore",
|
||||
"join_every": 1.0,
|
||||
"warmup": 60,
|
||||
"seconds": 300,
|
||||
"seed": 12345,
|
||||
"explore_speed": 0.8,
|
||||
"gamerule": "locator_bar=false",
|
||||
"heap": "4g",
|
||||
"out": "/home/ludovic/scratch/sh2-bench5.json",
|
||||
"nice": 19,
|
||||
"bots_via": "ssh -o BatchMode=yes macmini",
|
||||
"bots_bin": "/tmp/sh2-horizon-bots",
|
||||
"connect": "100.72.51.8",
|
||||
"properties": {
|
||||
"level-seed": "12345",
|
||||
"view-distance": "10",
|
||||
"simulation-distance": "10",
|
||||
"online-mode": "false",
|
||||
"white-list": "false",
|
||||
"gamemode": "creative",
|
||||
"force-gamemode": "true",
|
||||
"spawn-protection": "0",
|
||||
"max-players": "200",
|
||||
"difficulty": "easy",
|
||||
"sync-chunk-writes": "true"
|
||||
}
|
||||
},
|
||||
"runs": [
|
||||
{
|
||||
"server": "horizon",
|
||||
"bots": 1,
|
||||
"behaviour": "explore",
|
||||
"start_seconds": 10.91,
|
||||
"mspt": {
|
||||
"average": 4.02,
|
||||
"p50": 3.75,
|
||||
"p95": 6.63,
|
||||
"p99": 14.55
|
||||
},
|
||||
"mspt_worst_p99": 63.3,
|
||||
"tps": 20.0,
|
||||
"rss_mb": {
|
||||
"mean": 1392.42,
|
||||
"max": 1746.1
|
||||
},
|
||||
"cpu_percent": 24.52,
|
||||
"chunks_per_second": 1.69,
|
||||
"unique_per_second": 1.34,
|
||||
"forgotten_per_second": 1.62,
|
||||
"waited_ticks_per_second": 0.0,
|
||||
"chunks_total": 1048,
|
||||
"chunks_per_bot": {
|
||||
"explore": 1048.0
|
||||
},
|
||||
"unique_chunks_per_bot": {
|
||||
"explore": 952.0
|
||||
},
|
||||
"resent": 0,
|
||||
"pulled_back": 0,
|
||||
"moved_too_quickly": 0,
|
||||
"moved_wrongly": 0,
|
||||
"bots_joined": 1,
|
||||
"bots_failed": [],
|
||||
"tick_queries": 58
|
||||
},
|
||||
{
|
||||
"server": "vanilla",
|
||||
"bots": 1,
|
||||
"behaviour": "explore",
|
||||
"start_seconds": 15.13,
|
||||
"mspt": {
|
||||
"average": 18.12,
|
||||
"p50": 18.78,
|
||||
"p95": 27.07,
|
||||
"p99": 60.81
|
||||
},
|
||||
"mspt_worst_p99": 1397.6,
|
||||
"tps": 20.0,
|
||||
"rss_mb": {
|
||||
"mean": 3397.31,
|
||||
"max": 3430.2
|
||||
},
|
||||
"cpu_percent": 72.99,
|
||||
"chunks_per_second": 1.69,
|
||||
"unique_per_second": 1.38,
|
||||
"forgotten_per_second": 1.62,
|
||||
"waited_ticks_per_second": 0.0,
|
||||
"chunks_total": 1048,
|
||||
"chunks_per_bot": {
|
||||
"explore": 1048.0
|
||||
},
|
||||
"unique_chunks_per_bot": {
|
||||
"explore": 952.0
|
||||
},
|
||||
"resent": 0,
|
||||
"pulled_back": 0,
|
||||
"moved_too_quickly": 0,
|
||||
"moved_wrongly": 0,
|
||||
"bots_joined": 1,
|
||||
"bots_failed": [],
|
||||
"tick_queries": 58
|
||||
},
|
||||
{
|
||||
"server": "horizon",
|
||||
"bots": 10,
|
||||
"behaviour": "explore",
|
||||
"start_seconds": 11.12,
|
||||
"mspt": {
|
||||
"average": 13.49,
|
||||
"p50": 12.09,
|
||||
"p95": 24.81,
|
||||
"p99": 40.9
|
||||
},
|
||||
"mspt_worst_p99": 101.4,
|
||||
"tps": 20.0,
|
||||
"rss_mb": {
|
||||
"mean": 1684.04,
|
||||
"max": 2660.6
|
||||
},
|
||||
"cpu_percent": 89.04,
|
||||
"chunks_per_second": 13.08,
|
||||
"unique_per_second": 11.84,
|
||||
"forgotten_per_second": 13.02,
|
||||
"waited_ticks_per_second": 0.0,
|
||||
"chunks_total": 9277,
|
||||
"chunks_per_bot": {
|
||||
"explore": 927.7
|
||||
},
|
||||
"unique_chunks_per_bot": {
|
||||
"explore": 889.9
|
||||
},
|
||||
"resent": 0,
|
||||
"pulled_back": 3381,
|
||||
"moved_too_quickly": 0,
|
||||
"moved_wrongly": 0,
|
||||
"bots_joined": 10,
|
||||
"bots_failed": [],
|
||||
"tick_queries": 58
|
||||
},
|
||||
{
|
||||
"server": "vanilla",
|
||||
"bots": 10,
|
||||
"behaviour": "explore",
|
||||
"start_seconds": 14.84,
|
||||
"mspt": {
|
||||
"average": 29.18,
|
||||
"p50": 26.76,
|
||||
"p95": 43.02,
|
||||
"p99": 80.16
|
||||
},
|
||||
"mspt_worst_p99": 425.5,
|
||||
"tps": 20.0,
|
||||
"rss_mb": {
|
||||
"mean": 3609.51,
|
||||
"max": 3791.6
|
||||
},
|
||||
"cpu_percent": 142.79,
|
||||
"chunks_per_second": 13.1,
|
||||
"unique_per_second": 11.88,
|
||||
"forgotten_per_second": 13.12,
|
||||
"waited_ticks_per_second": 0.0,
|
||||
"chunks_total": 9245,
|
||||
"chunks_per_bot": {
|
||||
"explore": 924.5
|
||||
},
|
||||
"unique_chunks_per_bot": {
|
||||
"explore": 885.3
|
||||
},
|
||||
"resent": 0,
|
||||
"pulled_back": 3388,
|
||||
"moved_too_quickly": 6,
|
||||
"moved_wrongly": 0,
|
||||
"bots_joined": 10,
|
||||
"bots_failed": [],
|
||||
"tick_queries": 59
|
||||
},
|
||||
{
|
||||
"server": "horizon",
|
||||
"bots": 25,
|
||||
"behaviour": "explore",
|
||||
"start_seconds": 11.02,
|
||||
"mspt": {
|
||||
"average": 20.25,
|
||||
"p50": 18.73,
|
||||
"p95": 36.78,
|
||||
"p99": 55.76
|
||||
},
|
||||
"mspt_worst_p99": 92.3,
|
||||
"tps": 20.0,
|
||||
"rss_mb": {
|
||||
"mean": 1782.26,
|
||||
"max": 2741.9
|
||||
},
|
||||
"cpu_percent": 120.8,
|
||||
"chunks_per_second": 32.22,
|
||||
"unique_per_second": 28.89,
|
||||
"forgotten_per_second": 32.19,
|
||||
"waited_ticks_per_second": 0.0,
|
||||
"chunks_total": 23357,
|
||||
"chunks_per_bot": {
|
||||
"explore": 934.28
|
||||
},
|
||||
"unique_chunks_per_bot": {
|
||||
"explore": 890.32
|
||||
},
|
||||
"resent": 0,
|
||||
"pulled_back": 11580,
|
||||
"moved_too_quickly": 0,
|
||||
"moved_wrongly": 0,
|
||||
"bots_joined": 25,
|
||||
"bots_failed": [],
|
||||
"tick_queries": 58
|
||||
},
|
||||
{
|
||||
"server": "vanilla",
|
||||
"bots": 25,
|
||||
"behaviour": "explore",
|
||||
"start_seconds": 14.54,
|
||||
"mspt": {
|
||||
"average": 37.04,
|
||||
"p50": 34.77,
|
||||
"p95": 49.42,
|
||||
"p99": 101.67
|
||||
},
|
||||
"mspt_worst_p99": 735.2,
|
||||
"tps": 20.0,
|
||||
"rss_mb": {
|
||||
"mean": 3815.46,
|
||||
"max": 4202.0
|
||||
},
|
||||
"cpu_percent": 169.15,
|
||||
"chunks_per_second": 32.34,
|
||||
"unique_per_second": 28.93,
|
||||
"forgotten_per_second": 32.31,
|
||||
"waited_ticks_per_second": 0.0,
|
||||
"chunks_total": 23306,
|
||||
"chunks_per_bot": {
|
||||
"explore": 932.24
|
||||
},
|
||||
"unique_chunks_per_bot": {
|
||||
"explore": 886.96
|
||||
},
|
||||
"resent": 0,
|
||||
"pulled_back": 11539,
|
||||
"moved_too_quickly": 6,
|
||||
"moved_wrongly": 0,
|
||||
"bots_joined": 25,
|
||||
"bots_failed": [],
|
||||
"tick_queries": 59
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,415 @@
|
||||
{
|
||||
"kind": "horizon-server-benchmark",
|
||||
"date": "2026-10-17T05:04:57+00:00",
|
||||
"machine": {
|
||||
"system": "Linux",
|
||||
"machine": "x86_64",
|
||||
"cpus": 4
|
||||
},
|
||||
"commit": "",
|
||||
"java": [
|
||||
"openjdk 25.0.4.1 2026-08-18"
|
||||
],
|
||||
"settings": {
|
||||
"servers": "horizon",
|
||||
"bots": "1,10,25,50,100",
|
||||
"behaviour": "walk,explore",
|
||||
"join_every": 1.0,
|
||||
"warmup": 60,
|
||||
"seconds": 300,
|
||||
"seed": 12345,
|
||||
"explore_speed": 0,
|
||||
"gamerule": "locator_bar=false",
|
||||
"heap": "4g",
|
||||
"out": "/home/ludovic/scratch/sh2-bench4.json",
|
||||
"nice": 19,
|
||||
"bots_via": "ssh -o BatchMode=yes macmini",
|
||||
"bots_bin": "/tmp/sh2-horizon-bots",
|
||||
"connect": "100.72.51.8",
|
||||
"properties": {
|
||||
"level-seed": "12345",
|
||||
"view-distance": "10",
|
||||
"simulation-distance": "10",
|
||||
"online-mode": "false",
|
||||
"white-list": "false",
|
||||
"gamemode": "creative",
|
||||
"force-gamemode": "true",
|
||||
"spawn-protection": "0",
|
||||
"max-players": "200",
|
||||
"difficulty": "easy",
|
||||
"sync-chunk-writes": "true"
|
||||
}
|
||||
},
|
||||
"runs": [
|
||||
{
|
||||
"server": "horizon",
|
||||
"bots": 1,
|
||||
"behaviour": "walk",
|
||||
"start_seconds": 11.02,
|
||||
"mspt": {
|
||||
"average": 5.45,
|
||||
"p50": 4.63,
|
||||
"p95": 12.14,
|
||||
"p99": 30.63
|
||||
},
|
||||
"mspt_worst_p99": 91.5,
|
||||
"tps": 20.0,
|
||||
"rss_mb": {
|
||||
"mean": 1398.5,
|
||||
"max": 1708.4
|
||||
},
|
||||
"cpu_percent": 57.03,
|
||||
"chunks_per_second": 9.17,
|
||||
"unique_per_second": 0.26,
|
||||
"forgotten_per_second": 9.2,
|
||||
"waited_ticks_per_second": 0.0,
|
||||
"chunks_total": 3708,
|
||||
"chunks_per_bot": {
|
||||
"walk": 3708.0
|
||||
},
|
||||
"unique_chunks_per_bot": {
|
||||
"walk": 694.0
|
||||
},
|
||||
"resent": 0,
|
||||
"pulled_back": 0,
|
||||
"moved_too_quickly": 0,
|
||||
"moved_wrongly": 0,
|
||||
"bots_joined": 1,
|
||||
"bots_failed": [],
|
||||
"tick_queries": 58
|
||||
},
|
||||
{
|
||||
"server": "horizon",
|
||||
"bots": 10,
|
||||
"behaviour": "walk",
|
||||
"start_seconds": 11.02,
|
||||
"mspt": {
|
||||
"average": 11.51,
|
||||
"p50": 10.6,
|
||||
"p95": 22.13,
|
||||
"p99": 36.92
|
||||
},
|
||||
"mspt_worst_p99": 71.9,
|
||||
"tps": 20.0,
|
||||
"rss_mb": {
|
||||
"mean": 1440.2,
|
||||
"max": 1832.4
|
||||
},
|
||||
"cpu_percent": 124.19,
|
||||
"chunks_per_second": 94.04,
|
||||
"unique_per_second": 2.38,
|
||||
"forgotten_per_second": 93.95,
|
||||
"waited_ticks_per_second": 0.0,
|
||||
"chunks_total": 37269,
|
||||
"chunks_per_bot": {
|
||||
"walk": 3726.9
|
||||
},
|
||||
"unique_chunks_per_bot": {
|
||||
"walk": 688.9
|
||||
},
|
||||
"resent": 0,
|
||||
"pulled_back": 0,
|
||||
"moved_too_quickly": 0,
|
||||
"moved_wrongly": 0,
|
||||
"bots_joined": 10,
|
||||
"bots_failed": [],
|
||||
"tick_queries": 58
|
||||
},
|
||||
{
|
||||
"server": "horizon",
|
||||
"bots": 25,
|
||||
"behaviour": "walk",
|
||||
"start_seconds": 10.91,
|
||||
"mspt": {
|
||||
"average": 20.74,
|
||||
"p50": 19.27,
|
||||
"p95": 40.42,
|
||||
"p99": 57.12
|
||||
},
|
||||
"mspt_worst_p99": 102.0,
|
||||
"tps": 20.0,
|
||||
"rss_mb": {
|
||||
"mean": 1485.4,
|
||||
"max": 1871.9
|
||||
},
|
||||
"cpu_percent": 169.52,
|
||||
"chunks_per_second": 235.81,
|
||||
"unique_per_second": 5.68,
|
||||
"forgotten_per_second": 235.83,
|
||||
"waited_ticks_per_second": 0.0,
|
||||
"chunks_total": 94456,
|
||||
"chunks_per_bot": {
|
||||
"walk": 3778.24
|
||||
},
|
||||
"unique_chunks_per_bot": {
|
||||
"walk": 692.96
|
||||
},
|
||||
"resent": 0,
|
||||
"pulled_back": 0,
|
||||
"moved_too_quickly": 0,
|
||||
"moved_wrongly": 0,
|
||||
"bots_joined": 25,
|
||||
"bots_failed": [],
|
||||
"tick_queries": 58
|
||||
},
|
||||
{
|
||||
"server": "horizon",
|
||||
"bots": 50,
|
||||
"behaviour": "walk",
|
||||
"start_seconds": 11.12,
|
||||
"mspt": {
|
||||
"average": 31.41,
|
||||
"p50": 25.39,
|
||||
"p95": 78.97,
|
||||
"p99": 110.73
|
||||
},
|
||||
"mspt_worst_p99": 192.9,
|
||||
"tps": 20.0,
|
||||
"rss_mb": {
|
||||
"mean": 1558.34,
|
||||
"max": 1915.3
|
||||
},
|
||||
"cpu_percent": 173.27,
|
||||
"chunks_per_second": 450.85,
|
||||
"unique_per_second": 10.3,
|
||||
"forgotten_per_second": 451.86,
|
||||
"waited_ticks_per_second": 0.0,
|
||||
"chunks_total": 186885,
|
||||
"chunks_per_bot": {
|
||||
"walk": 3737.7
|
||||
},
|
||||
"unique_chunks_per_bot": {
|
||||
"walk": 688.9
|
||||
},
|
||||
"resent": 0,
|
||||
"pulled_back": 0,
|
||||
"moved_too_quickly": 0,
|
||||
"moved_wrongly": 0,
|
||||
"bots_joined": 50,
|
||||
"bots_failed": [],
|
||||
"tick_queries": 58
|
||||
},
|
||||
{
|
||||
"server": "horizon",
|
||||
"bots": 100,
|
||||
"behaviour": "walk",
|
||||
"start_seconds": 18.16,
|
||||
"mspt": {
|
||||
"average": 114.55,
|
||||
"p50": 101.37,
|
||||
"p95": 256.19,
|
||||
"p99": 338.64
|
||||
},
|
||||
"mspt_worst_p99": 442.6,
|
||||
"tps": 8.73,
|
||||
"rss_mb": {
|
||||
"mean": 1357.65,
|
||||
"max": 1358.4
|
||||
},
|
||||
"cpu_percent": 181.21,
|
||||
"chunks_per_second": 484.16,
|
||||
"unique_per_second": 30.76,
|
||||
"forgotten_per_second": 478.77,
|
||||
"waited_ticks_per_second": 0.0,
|
||||
"chunks_total": 212947,
|
||||
"chunks_per_bot": {
|
||||
"walk": 2129.47
|
||||
},
|
||||
"unique_chunks_per_bot": {
|
||||
"walk": 583.02
|
||||
},
|
||||
"resent": 0,
|
||||
"pulled_back": 0,
|
||||
"moved_too_quickly": 0,
|
||||
"moved_wrongly": 0,
|
||||
"bots_joined": 100,
|
||||
"bots_failed": [],
|
||||
"tick_queries": 57
|
||||
},
|
||||
{
|
||||
"server": "horizon",
|
||||
"bots": 1,
|
||||
"behaviour": "explore",
|
||||
"start_seconds": 17.45,
|
||||
"mspt": {
|
||||
"average": 10.62,
|
||||
"p50": 9.2,
|
||||
"p95": 24.61,
|
||||
"p99": 38.18
|
||||
},
|
||||
"mspt_worst_p99": 81.1,
|
||||
"tps": 20.0,
|
||||
"rss_mb": {
|
||||
"mean": 1372.73,
|
||||
"max": 1568.7
|
||||
},
|
||||
"cpu_percent": 322.38,
|
||||
"chunks_per_second": 39.21,
|
||||
"unique_per_second": 34.6,
|
||||
"forgotten_per_second": 38.55,
|
||||
"waited_ticks_per_second": 0.51,
|
||||
"chunks_total": 12549,
|
||||
"chunks_per_bot": {
|
||||
"explore": 12549.0
|
||||
},
|
||||
"unique_chunks_per_bot": {
|
||||
"explore": 11168.0
|
||||
},
|
||||
"resent": 0,
|
||||
"pulled_back": 0,
|
||||
"moved_too_quickly": 0,
|
||||
"moved_wrongly": 0,
|
||||
"bots_joined": 1,
|
||||
"bots_failed": [],
|
||||
"tick_queries": 58
|
||||
},
|
||||
{
|
||||
"server": "horizon",
|
||||
"bots": 10,
|
||||
"behaviour": "explore",
|
||||
"start_seconds": 11.32,
|
||||
"mspt": {
|
||||
"average": 30.38,
|
||||
"p50": 27.84,
|
||||
"p95": 52.31,
|
||||
"p99": 77.89
|
||||
},
|
||||
"mspt_worst_p99": 158.3,
|
||||
"tps": 20.0,
|
||||
"rss_mb": {
|
||||
"mean": 2231.56,
|
||||
"max": 4022.6
|
||||
},
|
||||
"cpu_percent": 344.99,
|
||||
"chunks_per_second": 47.34,
|
||||
"unique_per_second": 46.73,
|
||||
"forgotten_per_second": 44.57,
|
||||
"waited_ticks_per_second": 112.27,
|
||||
"chunks_total": 19870,
|
||||
"chunks_per_bot": {
|
||||
"explore": 1987.0
|
||||
},
|
||||
"unique_chunks_per_bot": {
|
||||
"explore": 1966.5
|
||||
},
|
||||
"resent": 0,
|
||||
"pulled_back": 16130,
|
||||
"moved_too_quickly": 0,
|
||||
"moved_wrongly": 0,
|
||||
"bots_joined": 10,
|
||||
"bots_failed": [],
|
||||
"tick_queries": 58
|
||||
},
|
||||
{
|
||||
"server": "horizon",
|
||||
"bots": 25,
|
||||
"behaviour": "explore",
|
||||
"start_seconds": 11.42,
|
||||
"mspt": {
|
||||
"average": 50.67,
|
||||
"p50": 47.61,
|
||||
"p95": 89.3,
|
||||
"p99": 119.15
|
||||
},
|
||||
"mspt_worst_p99": 222.3,
|
||||
"tps": 19.73,
|
||||
"rss_mb": {
|
||||
"mean": 2358.4,
|
||||
"max": 4273.7
|
||||
},
|
||||
"cpu_percent": 286.32,
|
||||
"chunks_per_second": 50.39,
|
||||
"unique_per_second": 49.67,
|
||||
"forgotten_per_second": 47.45,
|
||||
"waited_ticks_per_second": 311.85,
|
||||
"chunks_total": 28954,
|
||||
"chunks_per_bot": {
|
||||
"explore": 1158.16
|
||||
},
|
||||
"unique_chunks_per_bot": {
|
||||
"explore": 1147.56
|
||||
},
|
||||
"resent": 0,
|
||||
"pulled_back": 36319,
|
||||
"moved_too_quickly": 0,
|
||||
"moved_wrongly": 0,
|
||||
"bots_joined": 25,
|
||||
"bots_failed": [],
|
||||
"tick_queries": 58
|
||||
},
|
||||
{
|
||||
"server": "horizon",
|
||||
"bots": 50,
|
||||
"behaviour": "explore",
|
||||
"start_seconds": 20.06,
|
||||
"mspt": {
|
||||
"average": 50.69,
|
||||
"p50": 47.48,
|
||||
"p95": 102.67,
|
||||
"p99": 133.27
|
||||
},
|
||||
"mspt_worst_p99": 274.4,
|
||||
"tps": 19.73,
|
||||
"rss_mb": {
|
||||
"mean": 1764.01,
|
||||
"max": 2542.0
|
||||
},
|
||||
"cpu_percent": 212.83,
|
||||
"chunks_per_second": 97.33,
|
||||
"unique_per_second": 96.47,
|
||||
"forgotten_per_second": 76.74,
|
||||
"waited_ticks_per_second": 750.49,
|
||||
"chunks_total": 38538,
|
||||
"chunks_per_bot": {
|
||||
"explore": 770.76
|
||||
},
|
||||
"unique_chunks_per_bot": {
|
||||
"explore": 763.84
|
||||
},
|
||||
"resent": 0,
|
||||
"pulled_back": 39880,
|
||||
"moved_too_quickly": 0,
|
||||
"moved_wrongly": 0,
|
||||
"bots_joined": 50,
|
||||
"bots_failed": [],
|
||||
"tick_queries": 58
|
||||
},
|
||||
{
|
||||
"server": "horizon",
|
||||
"bots": 100,
|
||||
"behaviour": "explore",
|
||||
"start_seconds": 11.82,
|
||||
"mspt": {
|
||||
"average": 92.93,
|
||||
"p50": 87.84,
|
||||
"p95": 139.6,
|
||||
"p99": 178.38
|
||||
},
|
||||
"mspt_worst_p99": 272.9,
|
||||
"tps": 10.76,
|
||||
"rss_mb": {
|
||||
"mean": 1736.38,
|
||||
"max": 1855.7
|
||||
},
|
||||
"cpu_percent": 321.53,
|
||||
"chunks_per_second": 113.89,
|
||||
"unique_per_second": 113.56,
|
||||
"forgotten_per_second": 101.68,
|
||||
"waited_ticks_per_second": 1179.4,
|
||||
"chunks_total": 84617,
|
||||
"chunks_per_bot": {
|
||||
"explore": 846.17
|
||||
},
|
||||
"unique_chunks_per_bot": {
|
||||
"explore": 845.01
|
||||
},
|
||||
"resent": 0,
|
||||
"pulled_back": 128104,
|
||||
"moved_too_quickly": 0,
|
||||
"moved_wrongly": 0,
|
||||
"bots_joined": 100,
|
||||
"bots_failed": [],
|
||||
"tick_queries": 58
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,693 @@
|
||||
{
|
||||
"kind": "horizon-server-benchmark",
|
||||
"date": "2026-10-17T00:37:35+00:00",
|
||||
"machine": {
|
||||
"system": "Linux",
|
||||
"machine": "x86_64",
|
||||
"cpus": 4
|
||||
},
|
||||
"commit": "",
|
||||
"java": [
|
||||
"openjdk 25.0.4.1 2026-08-18"
|
||||
],
|
||||
"settings": {
|
||||
"servers": "horizon,vanilla",
|
||||
"bots": "1,10,25,50,100",
|
||||
"behaviour": "walk,explore",
|
||||
"join_every": 1.0,
|
||||
"warmup": 60,
|
||||
"seconds": 300,
|
||||
"seed": 12345,
|
||||
"heap": "4g",
|
||||
"out": "/home/ludovic/scratch/sh2-bench.json",
|
||||
"nice": 19,
|
||||
"bots_via": "ssh -o BatchMode=yes macmini",
|
||||
"bots_bin": "/tmp/sh2-horizon-bots",
|
||||
"connect": "100.72.51.8",
|
||||
"properties": {
|
||||
"level-seed": "12345",
|
||||
"view-distance": "10",
|
||||
"simulation-distance": "10",
|
||||
"online-mode": "false",
|
||||
"white-list": "false",
|
||||
"gamemode": "creative",
|
||||
"force-gamemode": "true",
|
||||
"spawn-protection": "0",
|
||||
"max-players": "200",
|
||||
"difficulty": "easy",
|
||||
"sync-chunk-writes": "true"
|
||||
}
|
||||
},
|
||||
"runs": [
|
||||
{
|
||||
"server": "horizon",
|
||||
"bots": 1,
|
||||
"behaviour": "walk",
|
||||
"start_seconds": 11.12,
|
||||
"mspt": {
|
||||
"average": 7.63,
|
||||
"p50": 5.51,
|
||||
"p95": 17.46,
|
||||
"p99": 36.79
|
||||
},
|
||||
"mspt_worst_p99": 79.6,
|
||||
"tps": 20.0,
|
||||
"rss_mb": {
|
||||
"mean": 1391.94,
|
||||
"max": 1700.7
|
||||
},
|
||||
"cpu_percent": 60.18,
|
||||
"chunks_per_second": 9.15,
|
||||
"unique_per_second": 0.29,
|
||||
"forgotten_per_second": 9.18,
|
||||
"waited_ticks_per_second": 0.0,
|
||||
"chunks_total": 3682,
|
||||
"chunks_per_bot": {
|
||||
"walk": 3682.0
|
||||
},
|
||||
"unique_chunks_per_bot": {
|
||||
"walk": 692.0
|
||||
},
|
||||
"resent": 0,
|
||||
"pulled_back": 0,
|
||||
"moved_too_quickly": 0,
|
||||
"moved_wrongly": 0,
|
||||
"bots_joined": 1,
|
||||
"bots_failed": [],
|
||||
"tick_queries": 58
|
||||
},
|
||||
{
|
||||
"server": "vanilla",
|
||||
"bots": 1,
|
||||
"behaviour": "walk",
|
||||
"start_seconds": 15.35,
|
||||
"mspt": {
|
||||
"average": 18.08,
|
||||
"p50": 18.66,
|
||||
"p95": 28.05,
|
||||
"p99": 46.94
|
||||
},
|
||||
"mspt_worst_p99": 456.4,
|
||||
"tps": 20.0,
|
||||
"rss_mb": {
|
||||
"mean": 3455.72,
|
||||
"max": 3471.7
|
||||
},
|
||||
"cpu_percent": 65.67,
|
||||
"chunks_per_second": 9.89,
|
||||
"unique_per_second": 0.29,
|
||||
"forgotten_per_second": 9.92,
|
||||
"waited_ticks_per_second": 0.0,
|
||||
"chunks_total": 3827,
|
||||
"chunks_per_bot": {
|
||||
"walk": 3827.0
|
||||
},
|
||||
"unique_chunks_per_bot": {
|
||||
"walk": 699.0
|
||||
},
|
||||
"resent": 0,
|
||||
"pulled_back": 0,
|
||||
"moved_too_quickly": 0,
|
||||
"moved_wrongly": 0,
|
||||
"bots_joined": 1,
|
||||
"bots_failed": [],
|
||||
"tick_queries": 58
|
||||
},
|
||||
{
|
||||
"server": "horizon",
|
||||
"bots": 10,
|
||||
"behaviour": "walk",
|
||||
"start_seconds": 11.22,
|
||||
"mspt": {
|
||||
"average": 27.53,
|
||||
"p50": 24.93,
|
||||
"p95": 53.93,
|
||||
"p99": 77.03
|
||||
},
|
||||
"mspt_worst_p99": 111.4,
|
||||
"tps": 20.0,
|
||||
"rss_mb": {
|
||||
"mean": 1433.85,
|
||||
"max": 1831.9
|
||||
},
|
||||
"cpu_percent": 138.65,
|
||||
"chunks_per_second": 95.98,
|
||||
"unique_per_second": 2.38,
|
||||
"forgotten_per_second": 95.78,
|
||||
"waited_ticks_per_second": 0.0,
|
||||
"chunks_total": 38041,
|
||||
"chunks_per_bot": {
|
||||
"walk": 3804.1
|
||||
},
|
||||
"unique_chunks_per_bot": {
|
||||
"walk": 690.8
|
||||
},
|
||||
"resent": 0,
|
||||
"pulled_back": 0,
|
||||
"moved_too_quickly": 0,
|
||||
"moved_wrongly": 0,
|
||||
"bots_joined": 10,
|
||||
"bots_failed": [],
|
||||
"tick_queries": 58
|
||||
},
|
||||
{
|
||||
"server": "vanilla",
|
||||
"bots": 10,
|
||||
"behaviour": "walk",
|
||||
"start_seconds": 14.83,
|
||||
"mspt": {
|
||||
"average": 18.92,
|
||||
"p50": 16.02,
|
||||
"p95": 39.0,
|
||||
"p99": 56.13
|
||||
},
|
||||
"mspt_worst_p99": 180.3,
|
||||
"tps": 20.0,
|
||||
"rss_mb": {
|
||||
"mean": 3479.45,
|
||||
"max": 3535.7
|
||||
},
|
||||
"cpu_percent": 99.39,
|
||||
"chunks_per_second": 99.93,
|
||||
"unique_per_second": 2.94,
|
||||
"forgotten_per_second": 100.04,
|
||||
"waited_ticks_per_second": 0.0,
|
||||
"chunks_total": 38411,
|
||||
"chunks_per_bot": {
|
||||
"walk": 3841.1
|
||||
},
|
||||
"unique_chunks_per_bot": {
|
||||
"walk": 697.1
|
||||
},
|
||||
"resent": 0,
|
||||
"pulled_back": 7,
|
||||
"moved_too_quickly": 7,
|
||||
"moved_wrongly": 0,
|
||||
"bots_joined": 10,
|
||||
"bots_failed": [],
|
||||
"tick_queries": 59
|
||||
},
|
||||
{
|
||||
"server": "horizon",
|
||||
"bots": 25,
|
||||
"behaviour": "walk",
|
||||
"start_seconds": 11.52,
|
||||
"mspt": {
|
||||
"average": 40.69,
|
||||
"p50": 37.28,
|
||||
"p95": 81.16,
|
||||
"p99": 108.42
|
||||
},
|
||||
"mspt_worst_p99": 149.9,
|
||||
"tps": 20.0,
|
||||
"rss_mb": {
|
||||
"mean": 1444.12,
|
||||
"max": 1882.6
|
||||
},
|
||||
"cpu_percent": 176.94,
|
||||
"chunks_per_second": 244.11,
|
||||
"unique_per_second": 6.51,
|
||||
"forgotten_per_second": 243.96,
|
||||
"waited_ticks_per_second": 0.0,
|
||||
"chunks_total": 95219,
|
||||
"chunks_per_bot": {
|
||||
"walk": 3808.76
|
||||
},
|
||||
"unique_chunks_per_bot": {
|
||||
"walk": 695.64
|
||||
},
|
||||
"resent": 0,
|
||||
"pulled_back": 0,
|
||||
"moved_too_quickly": 0,
|
||||
"moved_wrongly": 0,
|
||||
"bots_joined": 25,
|
||||
"bots_failed": [],
|
||||
"tick_queries": 58
|
||||
},
|
||||
{
|
||||
"server": "vanilla",
|
||||
"bots": 25,
|
||||
"behaviour": "walk",
|
||||
"start_seconds": 15.23,
|
||||
"mspt": {
|
||||
"average": 18.81,
|
||||
"p50": 17.26,
|
||||
"p95": 30.16,
|
||||
"p99": 53.9
|
||||
},
|
||||
"mspt_worst_p99": 390.0,
|
||||
"tps": 20.0,
|
||||
"rss_mb": {
|
||||
"mean": 3517.36,
|
||||
"max": 3555.7
|
||||
},
|
||||
"cpu_percent": 148.52,
|
||||
"chunks_per_second": 252.34,
|
||||
"unique_per_second": 7.75,
|
||||
"forgotten_per_second": 251.95,
|
||||
"waited_ticks_per_second": 0.0,
|
||||
"chunks_total": 96645,
|
||||
"chunks_per_bot": {
|
||||
"walk": 3865.8
|
||||
},
|
||||
"unique_chunks_per_bot": {
|
||||
"walk": 698.4
|
||||
},
|
||||
"resent": 0,
|
||||
"pulled_back": 11,
|
||||
"moved_too_quickly": 11,
|
||||
"moved_wrongly": 0,
|
||||
"bots_joined": 25,
|
||||
"bots_failed": [],
|
||||
"tick_queries": 59
|
||||
},
|
||||
{
|
||||
"server": "horizon",
|
||||
"bots": 50,
|
||||
"behaviour": "walk",
|
||||
"start_seconds": 12.72,
|
||||
"mspt": {
|
||||
"average": 281.42,
|
||||
"p50": 278.28,
|
||||
"p95": 343.64,
|
||||
"p99": 384.23
|
||||
},
|
||||
"mspt_worst_p99": 475.1,
|
||||
"tps": 3.55,
|
||||
"rss_mb": {
|
||||
"mean": 1334.32,
|
||||
"max": 1335.4
|
||||
},
|
||||
"cpu_percent": 197.11,
|
||||
"chunks_per_second": 391.5,
|
||||
"unique_per_second": 21.24,
|
||||
"forgotten_per_second": 388.33,
|
||||
"waited_ticks_per_second": 0.0,
|
||||
"chunks_total": 153136,
|
||||
"chunks_per_bot": {
|
||||
"walk": 3062.72
|
||||
},
|
||||
"unique_chunks_per_bot": {
|
||||
"walk": 661.82
|
||||
},
|
||||
"resent": 0,
|
||||
"pulled_back": 0,
|
||||
"moved_too_quickly": 0,
|
||||
"moved_wrongly": 0,
|
||||
"bots_joined": 50,
|
||||
"bots_failed": [],
|
||||
"tick_queries": 57
|
||||
},
|
||||
{
|
||||
"server": "vanilla",
|
||||
"bots": 50,
|
||||
"behaviour": "walk",
|
||||
"start_seconds": 15.74,
|
||||
"mspt": {
|
||||
"average": 33.21,
|
||||
"p50": 29.9,
|
||||
"p95": 53.37,
|
||||
"p99": 109.14
|
||||
},
|
||||
"mspt_worst_p99": 833.3,
|
||||
"tps": 20.0,
|
||||
"rss_mb": {
|
||||
"mean": 3568.07,
|
||||
"max": 3594.9
|
||||
},
|
||||
"cpu_percent": 266.59,
|
||||
"chunks_per_second": 496.3,
|
||||
"unique_per_second": 18.64,
|
||||
"forgotten_per_second": 495.77,
|
||||
"waited_ticks_per_second": 0.0,
|
||||
"chunks_total": 191965,
|
||||
"chunks_per_bot": {
|
||||
"walk": 3839.3
|
||||
},
|
||||
"unique_chunks_per_bot": {
|
||||
"walk": 696.22
|
||||
},
|
||||
"resent": 0,
|
||||
"pulled_back": 184,
|
||||
"moved_too_quickly": 183,
|
||||
"moved_wrongly": 0,
|
||||
"bots_joined": 50,
|
||||
"bots_failed": [],
|
||||
"tick_queries": 58
|
||||
},
|
||||
{
|
||||
"server": "horizon",
|
||||
"bots": 100,
|
||||
"behaviour": "walk",
|
||||
"start_seconds": 11.22,
|
||||
"mspt": {
|
||||
"average": 467.37,
|
||||
"p50": 464.62,
|
||||
"p95": 524.07,
|
||||
"p99": 548.77
|
||||
},
|
||||
"mspt_worst_p99": 581.1,
|
||||
"tps": 2.14,
|
||||
"rss_mb": {
|
||||
"mean": 1353.68,
|
||||
"max": 1354.9
|
||||
},
|
||||
"cpu_percent": 191.96,
|
||||
"chunks_per_second": 377.62,
|
||||
"unique_per_second": 55.8,
|
||||
"forgotten_per_second": 348.88,
|
||||
"waited_ticks_per_second": 0.0,
|
||||
"chunks_total": 165719,
|
||||
"chunks_per_bot": {
|
||||
"walk": 1657.19
|
||||
},
|
||||
"unique_chunks_per_bot": {
|
||||
"walk": 549.8
|
||||
},
|
||||
"resent": 0,
|
||||
"pulled_back": 0,
|
||||
"moved_too_quickly": 0,
|
||||
"moved_wrongly": 0,
|
||||
"bots_joined": 100,
|
||||
"bots_failed": [],
|
||||
"tick_queries": 56
|
||||
},
|
||||
{
|
||||
"server": "vanilla",
|
||||
"bots": 100,
|
||||
"behaviour": "walk",
|
||||
"start_seconds": 14.84,
|
||||
"error": "[Errno 32] Broken pipe"
|
||||
},
|
||||
{
|
||||
"server": "horizon",
|
||||
"bots": 1,
|
||||
"behaviour": "explore",
|
||||
"start_seconds": 11.12,
|
||||
"mspt": {
|
||||
"average": 17.07,
|
||||
"p50": 16.6,
|
||||
"p95": 32.82,
|
||||
"p99": 50.3
|
||||
},
|
||||
"mspt_worst_p99": 113.9,
|
||||
"tps": 20.0,
|
||||
"rss_mb": {
|
||||
"mean": 1391.72,
|
||||
"max": 1566.6
|
||||
},
|
||||
"cpu_percent": 333.8,
|
||||
"chunks_per_second": 39.42,
|
||||
"unique_per_second": 34.77,
|
||||
"forgotten_per_second": 39.34,
|
||||
"waited_ticks_per_second": 0.0,
|
||||
"chunks_total": 13860,
|
||||
"chunks_per_bot": {
|
||||
"explore": 13860.0
|
||||
},
|
||||
"unique_chunks_per_bot": {
|
||||
"explore": 12379.0
|
||||
},
|
||||
"resent": 0,
|
||||
"pulled_back": 192,
|
||||
"moved_too_quickly": 0,
|
||||
"moved_wrongly": 0,
|
||||
"bots_joined": 1,
|
||||
"bots_failed": [],
|
||||
"tick_queries": 58
|
||||
},
|
||||
{
|
||||
"server": "vanilla",
|
||||
"bots": 1,
|
||||
"behaviour": "explore",
|
||||
"start_seconds": 16.03,
|
||||
"mspt": {
|
||||
"average": 9.9,
|
||||
"p50": 8.04,
|
||||
"p95": 20.41,
|
||||
"p99": 63.62
|
||||
},
|
||||
"mspt_worst_p99": 351.1,
|
||||
"tps": 20.0,
|
||||
"rss_mb": {
|
||||
"mean": 3709.92,
|
||||
"max": 3776.2
|
||||
},
|
||||
"cpu_percent": 246.64,
|
||||
"chunks_per_second": 36.06,
|
||||
"unique_per_second": 33.33,
|
||||
"forgotten_per_second": 35.73,
|
||||
"waited_ticks_per_second": 0.0,
|
||||
"chunks_total": 12369,
|
||||
"chunks_per_bot": {
|
||||
"explore": 12369.0
|
||||
},
|
||||
"unique_chunks_per_bot": {
|
||||
"explore": 11559.0
|
||||
},
|
||||
"resent": 0,
|
||||
"pulled_back": 222,
|
||||
"moved_too_quickly": 0,
|
||||
"moved_wrongly": 0,
|
||||
"bots_joined": 1,
|
||||
"bots_failed": [],
|
||||
"tick_queries": 59
|
||||
},
|
||||
{
|
||||
"server": "horizon",
|
||||
"bots": 10,
|
||||
"behaviour": "explore",
|
||||
"start_seconds": 11.42,
|
||||
"mspt": {
|
||||
"average": 33.72,
|
||||
"p50": 31.08,
|
||||
"p95": 61.07,
|
||||
"p99": 89.09
|
||||
},
|
||||
"mspt_worst_p99": 142.6,
|
||||
"tps": 20.0,
|
||||
"rss_mb": {
|
||||
"mean": 2239.97,
|
||||
"max": 4108.8
|
||||
},
|
||||
"cpu_percent": 350.83,
|
||||
"chunks_per_second": 46.49,
|
||||
"unique_per_second": 46.08,
|
||||
"forgotten_per_second": 44.33,
|
||||
"waited_ticks_per_second": 111.28,
|
||||
"chunks_total": 19915,
|
||||
"chunks_per_bot": {
|
||||
"explore": 1991.5
|
||||
},
|
||||
"unique_chunks_per_bot": {
|
||||
"explore": 1974.4
|
||||
},
|
||||
"resent": 0,
|
||||
"pulled_back": 17516,
|
||||
"moved_too_quickly": 0,
|
||||
"moved_wrongly": 0,
|
||||
"bots_joined": 10,
|
||||
"bots_failed": [],
|
||||
"tick_queries": 58
|
||||
},
|
||||
{
|
||||
"server": "vanilla",
|
||||
"bots": 10,
|
||||
"behaviour": "explore",
|
||||
"start_seconds": 15.15,
|
||||
"mspt": {
|
||||
"average": 16.25,
|
||||
"p50": 13.67,
|
||||
"p95": 28.92,
|
||||
"p99": 115.9
|
||||
},
|
||||
"mspt_worst_p99": 1202.1,
|
||||
"tps": 20.0,
|
||||
"rss_mb": {
|
||||
"mean": 4202.72,
|
||||
"max": 4502.7
|
||||
},
|
||||
"cpu_percent": 247.47,
|
||||
"chunks_per_second": 22.3,
|
||||
"unique_per_second": 22.25,
|
||||
"forgotten_per_second": 22.64,
|
||||
"waited_ticks_per_second": 78.89,
|
||||
"chunks_total": 10092,
|
||||
"chunks_per_bot": {
|
||||
"explore": 1009.2
|
||||
},
|
||||
"unique_chunks_per_bot": {
|
||||
"explore": 1007.4
|
||||
},
|
||||
"resent": 0,
|
||||
"pulled_back": 16286,
|
||||
"moved_too_quickly": 12,
|
||||
"moved_wrongly": 0,
|
||||
"bots_joined": 10,
|
||||
"bots_failed": [],
|
||||
"tick_queries": 59
|
||||
},
|
||||
{
|
||||
"server": "horizon",
|
||||
"bots": 25,
|
||||
"behaviour": "explore",
|
||||
"start_seconds": 11.82,
|
||||
"mspt": {
|
||||
"average": 48.03,
|
||||
"p50": 44.99,
|
||||
"p95": 83.06,
|
||||
"p99": 109.63
|
||||
},
|
||||
"mspt_worst_p99": 191.8,
|
||||
"tps": 20.0,
|
||||
"rss_mb": {
|
||||
"mean": 2214.16,
|
||||
"max": 4598.6
|
||||
},
|
||||
"cpu_percent": 356.03,
|
||||
"chunks_per_second": 64.2,
|
||||
"unique_per_second": 63.5,
|
||||
"forgotten_per_second": 60.9,
|
||||
"waited_ticks_per_second": 301.65,
|
||||
"chunks_total": 32247,
|
||||
"chunks_per_bot": {
|
||||
"explore": 1289.88
|
||||
},
|
||||
"unique_chunks_per_bot": {
|
||||
"explore": 1279.4
|
||||
},
|
||||
"resent": 0,
|
||||
"pulled_back": 42159,
|
||||
"moved_too_quickly": 0,
|
||||
"moved_wrongly": 0,
|
||||
"bots_joined": 25,
|
||||
"bots_failed": [],
|
||||
"tick_queries": 58
|
||||
},
|
||||
{
|
||||
"server": "vanilla",
|
||||
"bots": 25,
|
||||
"behaviour": "explore",
|
||||
"start_seconds": 14.73,
|
||||
"mspt": {
|
||||
"average": 25.73,
|
||||
"p50": 22.75,
|
||||
"p95": 37.72,
|
||||
"p99": 201.98
|
||||
},
|
||||
"mspt_worst_p99": 4051.5,
|
||||
"tps": 20.0,
|
||||
"rss_mb": {
|
||||
"mean": 4355.9,
|
||||
"max": 4983.2
|
||||
},
|
||||
"cpu_percent": 309.3,
|
||||
"chunks_per_second": 33.33,
|
||||
"unique_per_second": 33.21,
|
||||
"forgotten_per_second": 34.22,
|
||||
"waited_ticks_per_second": 258.68,
|
||||
"chunks_total": 19426,
|
||||
"chunks_per_bot": {
|
||||
"explore": 777.04
|
||||
},
|
||||
"unique_chunks_per_bot": {
|
||||
"explore": 772.92
|
||||
},
|
||||
"resent": 0,
|
||||
"pulled_back": 53918,
|
||||
"moved_too_quickly": 93,
|
||||
"moved_wrongly": 0,
|
||||
"bots_joined": 25,
|
||||
"bots_failed": [],
|
||||
"tick_queries": 58
|
||||
},
|
||||
{
|
||||
"server": "horizon",
|
||||
"bots": 50,
|
||||
"behaviour": "explore",
|
||||
"start_seconds": 11.01,
|
||||
"mspt": {
|
||||
"average": 108.61,
|
||||
"p50": 107.37,
|
||||
"p95": 151.96,
|
||||
"p99": 177.41
|
||||
},
|
||||
"mspt_worst_p99": 271.6,
|
||||
"tps": 9.21,
|
||||
"rss_mb": {
|
||||
"mean": 1775.98,
|
||||
"max": 1956.9
|
||||
},
|
||||
"cpu_percent": 359.7,
|
||||
"chunks_per_second": 100.32,
|
||||
"unique_per_second": 98.91,
|
||||
"forgotten_per_second": 95.42,
|
||||
"waited_ticks_per_second": 518.15,
|
||||
"chunks_total": 55956,
|
||||
"chunks_per_bot": {
|
||||
"explore": 1119.12
|
||||
},
|
||||
"unique_chunks_per_bot": {
|
||||
"explore": 1111.42
|
||||
},
|
||||
"resent": 0,
|
||||
"pulled_back": 69967,
|
||||
"moved_too_quickly": 0,
|
||||
"moved_wrongly": 0,
|
||||
"bots_joined": 50,
|
||||
"bots_failed": [],
|
||||
"tick_queries": 58
|
||||
},
|
||||
{
|
||||
"server": "vanilla",
|
||||
"bots": 50,
|
||||
"behaviour": "explore",
|
||||
"start_seconds": 14.74,
|
||||
"error": "[Errno 32] Broken pipe"
|
||||
},
|
||||
{
|
||||
"server": "horizon",
|
||||
"bots": 100,
|
||||
"behaviour": "explore",
|
||||
"start_seconds": 11.12,
|
||||
"mspt": {
|
||||
"average": 244.43,
|
||||
"p50": 243.58,
|
||||
"p95": 308.39,
|
||||
"p99": 360.12
|
||||
},
|
||||
"mspt_worst_p99": 470.2,
|
||||
"tps": 4.09,
|
||||
"rss_mb": {
|
||||
"mean": 1806.85,
|
||||
"max": 2084.5
|
||||
},
|
||||
"cpu_percent": 310.46,
|
||||
"chunks_per_second": 186.87,
|
||||
"unique_per_second": 182.99,
|
||||
"forgotten_per_second": 161.75,
|
||||
"waited_ticks_per_second": 1131.65,
|
||||
"chunks_total": 100389,
|
||||
"chunks_per_bot": {
|
||||
"explore": 1003.89
|
||||
},
|
||||
"unique_chunks_per_bot": {
|
||||
"explore": 991.9
|
||||
},
|
||||
"resent": 0,
|
||||
"pulled_back": 56791,
|
||||
"moved_too_quickly": 344,
|
||||
"moved_wrongly": 0,
|
||||
"bots_joined": 100,
|
||||
"bots_failed": [],
|
||||
"tick_queries": 57
|
||||
},
|
||||
{
|
||||
"server": "vanilla",
|
||||
"bots": 100,
|
||||
"behaviour": "explore",
|
||||
"start_seconds": 14.83,
|
||||
"error": "[Errno 32] Broken pipe"
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,413 @@
|
||||
{
|
||||
"kind": "horizon-server-benchmark",
|
||||
"date": "2026-10-17T02:22:59+00:00",
|
||||
"machine": {
|
||||
"system": "Linux",
|
||||
"machine": "x86_64",
|
||||
"cpus": 4
|
||||
},
|
||||
"commit": "",
|
||||
"java": [
|
||||
"openjdk 25.0.4.1 2026-08-18"
|
||||
],
|
||||
"settings": {
|
||||
"servers": "horizon",
|
||||
"bots": "1,10,25,50,100",
|
||||
"behaviour": "walk,explore",
|
||||
"join_every": 1.0,
|
||||
"warmup": 60,
|
||||
"seconds": 300,
|
||||
"seed": 12345,
|
||||
"heap": "4g",
|
||||
"out": "/home/ludovic/scratch/sh2-bench2.json",
|
||||
"nice": 19,
|
||||
"bots_via": "ssh -o BatchMode=yes macmini",
|
||||
"bots_bin": "/tmp/sh2-horizon-bots",
|
||||
"connect": "100.72.51.8",
|
||||
"properties": {
|
||||
"level-seed": "12345",
|
||||
"view-distance": "10",
|
||||
"simulation-distance": "10",
|
||||
"online-mode": "false",
|
||||
"white-list": "false",
|
||||
"gamemode": "creative",
|
||||
"force-gamemode": "true",
|
||||
"spawn-protection": "0",
|
||||
"max-players": "200",
|
||||
"difficulty": "easy",
|
||||
"sync-chunk-writes": "true"
|
||||
}
|
||||
},
|
||||
"runs": [
|
||||
{
|
||||
"server": "horizon",
|
||||
"bots": 1,
|
||||
"behaviour": "walk",
|
||||
"start_seconds": 11.12,
|
||||
"mspt": {
|
||||
"average": 5.41,
|
||||
"p50": 4.63,
|
||||
"p95": 11.61,
|
||||
"p99": 30.5
|
||||
},
|
||||
"mspt_worst_p99": 64.6,
|
||||
"tps": 20.0,
|
||||
"rss_mb": {
|
||||
"mean": 1398.84,
|
||||
"max": 1708.5
|
||||
},
|
||||
"cpu_percent": 56.84,
|
||||
"chunks_per_second": 9.19,
|
||||
"unique_per_second": 0.27,
|
||||
"forgotten_per_second": 9.22,
|
||||
"waited_ticks_per_second": 0.0,
|
||||
"chunks_total": 3713,
|
||||
"chunks_per_bot": {
|
||||
"walk": 3713.0
|
||||
},
|
||||
"unique_chunks_per_bot": {
|
||||
"walk": 694.0
|
||||
},
|
||||
"resent": 0,
|
||||
"pulled_back": 0,
|
||||
"moved_too_quickly": 0,
|
||||
"moved_wrongly": 0,
|
||||
"bots_joined": 1,
|
||||
"bots_failed": [],
|
||||
"tick_queries": 58
|
||||
},
|
||||
{
|
||||
"server": "horizon",
|
||||
"bots": 10,
|
||||
"behaviour": "walk",
|
||||
"start_seconds": 10.92,
|
||||
"mspt": {
|
||||
"average": 13.18,
|
||||
"p50": 11.74,
|
||||
"p95": 26.5,
|
||||
"p99": 44.82
|
||||
},
|
||||
"mspt_worst_p99": 87.6,
|
||||
"tps": 20.0,
|
||||
"rss_mb": {
|
||||
"mean": 1439.83,
|
||||
"max": 1831.6
|
||||
},
|
||||
"cpu_percent": 132.94,
|
||||
"chunks_per_second": 93.87,
|
||||
"unique_per_second": 2.38,
|
||||
"forgotten_per_second": 93.77,
|
||||
"waited_ticks_per_second": 0.0,
|
||||
"chunks_total": 37240,
|
||||
"chunks_per_bot": {
|
||||
"walk": 3724.0
|
||||
},
|
||||
"unique_chunks_per_bot": {
|
||||
"walk": 688.7
|
||||
},
|
||||
"resent": 0,
|
||||
"pulled_back": 0,
|
||||
"moved_too_quickly": 0,
|
||||
"moved_wrongly": 0,
|
||||
"bots_joined": 10,
|
||||
"bots_failed": [],
|
||||
"tick_queries": 58
|
||||
},
|
||||
{
|
||||
"server": "horizon",
|
||||
"bots": 25,
|
||||
"behaviour": "walk",
|
||||
"start_seconds": 10.92,
|
||||
"mspt": {
|
||||
"average": 23.52,
|
||||
"p50": 21.92,
|
||||
"p95": 44.7,
|
||||
"p99": 63.12
|
||||
},
|
||||
"mspt_worst_p99": 119.1,
|
||||
"tps": 20.0,
|
||||
"rss_mb": {
|
||||
"mean": 1485.82,
|
||||
"max": 1872.0
|
||||
},
|
||||
"cpu_percent": 173.82,
|
||||
"chunks_per_second": 235.63,
|
||||
"unique_per_second": 5.64,
|
||||
"forgotten_per_second": 235.72,
|
||||
"waited_ticks_per_second": 0.0,
|
||||
"chunks_total": 94549,
|
||||
"chunks_per_bot": {
|
||||
"walk": 3781.96
|
||||
},
|
||||
"unique_chunks_per_bot": {
|
||||
"walk": 692.8
|
||||
},
|
||||
"resent": 0,
|
||||
"pulled_back": 0,
|
||||
"moved_too_quickly": 0,
|
||||
"moved_wrongly": 0,
|
||||
"bots_joined": 25,
|
||||
"bots_failed": [],
|
||||
"tick_queries": 58
|
||||
},
|
||||
{
|
||||
"server": "horizon",
|
||||
"bots": 50,
|
||||
"behaviour": "walk",
|
||||
"start_seconds": 10.91,
|
||||
"mspt": {
|
||||
"average": 31.36,
|
||||
"p50": 26.08,
|
||||
"p95": 70.6,
|
||||
"p99": 92.83
|
||||
},
|
||||
"mspt_worst_p99": 125.5,
|
||||
"tps": 20.0,
|
||||
"rss_mb": {
|
||||
"mean": 1556.81,
|
||||
"max": 1913.0
|
||||
},
|
||||
"cpu_percent": 192.16,
|
||||
"chunks_per_second": 468.69,
|
||||
"unique_per_second": 10.43,
|
||||
"forgotten_per_second": 468.78,
|
||||
"waited_ticks_per_second": 0.0,
|
||||
"chunks_total": 193547,
|
||||
"chunks_per_bot": {
|
||||
"walk": 3870.94
|
||||
},
|
||||
"unique_chunks_per_bot": {
|
||||
"walk": 692.84
|
||||
},
|
||||
"resent": 0,
|
||||
"pulled_back": 0,
|
||||
"moved_too_quickly": 0,
|
||||
"moved_wrongly": 0,
|
||||
"bots_joined": 50,
|
||||
"bots_failed": [],
|
||||
"tick_queries": 58
|
||||
},
|
||||
{
|
||||
"server": "horizon",
|
||||
"bots": 100,
|
||||
"behaviour": "walk",
|
||||
"start_seconds": 11.02,
|
||||
"mspt": {
|
||||
"average": 84.38,
|
||||
"p50": 81.5,
|
||||
"p95": 132.92,
|
||||
"p99": 168.38
|
||||
},
|
||||
"mspt_worst_p99": 245.5,
|
||||
"tps": 11.85,
|
||||
"rss_mb": {
|
||||
"mean": 1395.64,
|
||||
"max": 1941.3
|
||||
},
|
||||
"cpu_percent": 271.28,
|
||||
"chunks_per_second": 850.05,
|
||||
"unique_per_second": 21.79,
|
||||
"forgotten_per_second": 851.1,
|
||||
"waited_ticks_per_second": 0.0,
|
||||
"chunks_total": 368464,
|
||||
"chunks_per_bot": {
|
||||
"walk": 3684.64
|
||||
},
|
||||
"unique_chunks_per_bot": {
|
||||
"walk": 680.58
|
||||
},
|
||||
"resent": 0,
|
||||
"pulled_back": 0,
|
||||
"moved_too_quickly": 0,
|
||||
"moved_wrongly": 0,
|
||||
"bots_joined": 100,
|
||||
"bots_failed": [],
|
||||
"tick_queries": 58
|
||||
},
|
||||
{
|
||||
"server": "horizon",
|
||||
"bots": 1,
|
||||
"behaviour": "explore",
|
||||
"start_seconds": 11.02,
|
||||
"mspt": {
|
||||
"average": 11.63,
|
||||
"p50": 10.27,
|
||||
"p95": 24.52,
|
||||
"p99": 40.53
|
||||
},
|
||||
"mspt_worst_p99": 99.9,
|
||||
"tps": 20.0,
|
||||
"rss_mb": {
|
||||
"mean": 1413.81,
|
||||
"max": 1596.1
|
||||
},
|
||||
"cpu_percent": 329.84,
|
||||
"chunks_per_second": 39.47,
|
||||
"unique_per_second": 34.81,
|
||||
"forgotten_per_second": 39.42,
|
||||
"waited_ticks_per_second": 0.0,
|
||||
"chunks_total": 13798,
|
||||
"chunks_per_bot": {
|
||||
"explore": 13798.0
|
||||
},
|
||||
"unique_chunks_per_bot": {
|
||||
"explore": 12330.0
|
||||
},
|
||||
"resent": 0,
|
||||
"pulled_back": 185,
|
||||
"moved_too_quickly": 0,
|
||||
"moved_wrongly": 0,
|
||||
"bots_joined": 1,
|
||||
"bots_failed": [],
|
||||
"tick_queries": 58
|
||||
},
|
||||
{
|
||||
"server": "horizon",
|
||||
"bots": 10,
|
||||
"behaviour": "explore",
|
||||
"start_seconds": 11.02,
|
||||
"mspt": {
|
||||
"average": 28.49,
|
||||
"p50": 26.2,
|
||||
"p95": 48.58,
|
||||
"p99": 72.04
|
||||
},
|
||||
"mspt_worst_p99": 125.7,
|
||||
"tps": 20.0,
|
||||
"rss_mb": {
|
||||
"mean": 2231.06,
|
||||
"max": 4077.8
|
||||
},
|
||||
"cpu_percent": 344.95,
|
||||
"chunks_per_second": 46.31,
|
||||
"unique_per_second": 45.88,
|
||||
"forgotten_per_second": 44.14,
|
||||
"waited_ticks_per_second": 111.95,
|
||||
"chunks_total": 19722,
|
||||
"chunks_per_bot": {
|
||||
"explore": 1972.2
|
||||
},
|
||||
"unique_chunks_per_bot": {
|
||||
"explore": 1957.0
|
||||
},
|
||||
"resent": 0,
|
||||
"pulled_back": 18533,
|
||||
"moved_too_quickly": 0,
|
||||
"moved_wrongly": 0,
|
||||
"bots_joined": 10,
|
||||
"bots_failed": [],
|
||||
"tick_queries": 58
|
||||
},
|
||||
{
|
||||
"server": "horizon",
|
||||
"bots": 25,
|
||||
"behaviour": "explore",
|
||||
"start_seconds": 11.12,
|
||||
"mspt": {
|
||||
"average": 40.04,
|
||||
"p50": 37.61,
|
||||
"p95": 62.39,
|
||||
"p99": 83.64
|
||||
},
|
||||
"mspt_worst_p99": 147.0,
|
||||
"tps": 20.0,
|
||||
"rss_mb": {
|
||||
"mean": 2466.85,
|
||||
"max": 4504.5
|
||||
},
|
||||
"cpu_percent": 340.13,
|
||||
"chunks_per_second": 60.61,
|
||||
"unique_per_second": 59.62,
|
||||
"forgotten_per_second": 56.28,
|
||||
"waited_ticks_per_second": 299.57,
|
||||
"chunks_total": 32877,
|
||||
"chunks_per_bot": {
|
||||
"explore": 1315.08
|
||||
},
|
||||
"unique_chunks_per_bot": {
|
||||
"explore": 1300.72
|
||||
},
|
||||
"resent": 0,
|
||||
"pulled_back": 46272,
|
||||
"moved_too_quickly": 0,
|
||||
"moved_wrongly": 0,
|
||||
"bots_joined": 25,
|
||||
"bots_failed": [],
|
||||
"tick_queries": 58
|
||||
},
|
||||
{
|
||||
"server": "horizon",
|
||||
"bots": 50,
|
||||
"behaviour": "explore",
|
||||
"start_seconds": 11.32,
|
||||
"mspt": {
|
||||
"average": 61.82,
|
||||
"p50": 59.21,
|
||||
"p95": 90.46,
|
||||
"p99": 115.58
|
||||
},
|
||||
"mspt_worst_p99": 320.9,
|
||||
"tps": 16.18,
|
||||
"rss_mb": {
|
||||
"mean": 2634.54,
|
||||
"max": 4883.1
|
||||
},
|
||||
"cpu_percent": 356.96,
|
||||
"chunks_per_second": 95.26,
|
||||
"unique_per_second": 94.31,
|
||||
"forgotten_per_second": 85.86,
|
||||
"waited_ticks_per_second": 528.08,
|
||||
"chunks_total": 59251,
|
||||
"chunks_per_bot": {
|
||||
"explore": 1185.02
|
||||
},
|
||||
"unique_chunks_per_bot": {
|
||||
"explore": 1173.96
|
||||
},
|
||||
"resent": 0,
|
||||
"pulled_back": 99493,
|
||||
"moved_too_quickly": 0,
|
||||
"moved_wrongly": 0,
|
||||
"bots_joined": 50,
|
||||
"bots_failed": [],
|
||||
"tick_queries": 58
|
||||
},
|
||||
{
|
||||
"server": "horizon",
|
||||
"bots": 100,
|
||||
"behaviour": "explore",
|
||||
"start_seconds": 10.81,
|
||||
"mspt": {
|
||||
"average": 103.24,
|
||||
"p50": 100.03,
|
||||
"p95": 144.77,
|
||||
"p99": 178.8
|
||||
},
|
||||
"mspt_worst_p99": 258.6,
|
||||
"tps": 9.69,
|
||||
"rss_mb": {
|
||||
"mean": 1784.48,
|
||||
"max": 1922.0
|
||||
},
|
||||
"cpu_percent": 336.94,
|
||||
"chunks_per_second": 115.06,
|
||||
"unique_per_second": 114.86,
|
||||
"forgotten_per_second": 109.43,
|
||||
"waited_ticks_per_second": 1127.57,
|
||||
"chunks_total": 93839,
|
||||
"chunks_per_bot": {
|
||||
"explore": 938.39
|
||||
},
|
||||
"unique_chunks_per_bot": {
|
||||
"explore": 935.28
|
||||
},
|
||||
"resent": 0,
|
||||
"pulled_back": 133404,
|
||||
"moved_too_quickly": 0,
|
||||
"moved_wrongly": 0,
|
||||
"bots_joined": 100,
|
||||
"bots_failed": [],
|
||||
"tick_queries": 58
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,324 @@
|
||||
{
|
||||
"kind": "horizon-server-benchmark",
|
||||
"date": "2026-10-17T03:29:30+00:00",
|
||||
"machine": {
|
||||
"system": "Linux",
|
||||
"machine": "x86_64",
|
||||
"cpus": 4
|
||||
},
|
||||
"commit": "",
|
||||
"java": [
|
||||
"openjdk 25.0.4.1 2026-08-18"
|
||||
],
|
||||
"settings": {
|
||||
"servers": "vanilla",
|
||||
"bots": "1,10,25,50,100",
|
||||
"behaviour": "walk,explore",
|
||||
"join_every": 1.0,
|
||||
"warmup": 60,
|
||||
"seconds": 300,
|
||||
"seed": 12345,
|
||||
"gamerule": "locator_bar=false",
|
||||
"heap": "4g",
|
||||
"out": "/home/ludovic/scratch/sh2-bench3.json",
|
||||
"nice": 19,
|
||||
"bots_via": "ssh -o BatchMode=yes macmini",
|
||||
"bots_bin": "/tmp/sh2-horizon-bots",
|
||||
"connect": "100.72.51.8",
|
||||
"properties": {
|
||||
"level-seed": "12345",
|
||||
"view-distance": "10",
|
||||
"simulation-distance": "10",
|
||||
"online-mode": "false",
|
||||
"white-list": "false",
|
||||
"gamemode": "creative",
|
||||
"force-gamemode": "true",
|
||||
"spawn-protection": "0",
|
||||
"max-players": "200",
|
||||
"difficulty": "easy",
|
||||
"sync-chunk-writes": "true"
|
||||
}
|
||||
},
|
||||
"runs": [
|
||||
{
|
||||
"server": "vanilla",
|
||||
"bots": 1,
|
||||
"behaviour": "walk",
|
||||
"start_seconds": 15.43,
|
||||
"mspt": {
|
||||
"average": 18.32,
|
||||
"p50": 19.26,
|
||||
"p95": 27.34,
|
||||
"p99": 45.48
|
||||
},
|
||||
"mspt_worst_p99": 393.3,
|
||||
"tps": 20.0,
|
||||
"rss_mb": {
|
||||
"mean": 3428.43,
|
||||
"max": 3437.7
|
||||
},
|
||||
"cpu_percent": 66.47,
|
||||
"chunks_per_second": 9.92,
|
||||
"unique_per_second": 0.29,
|
||||
"forgotten_per_second": 9.95,
|
||||
"waited_ticks_per_second": 0.0,
|
||||
"chunks_total": 3850,
|
||||
"chunks_per_bot": {
|
||||
"walk": 3850.0
|
||||
},
|
||||
"unique_chunks_per_bot": {
|
||||
"walk": 699.0
|
||||
},
|
||||
"resent": 0,
|
||||
"pulled_back": 0,
|
||||
"moved_too_quickly": 0,
|
||||
"moved_wrongly": 0,
|
||||
"bots_joined": 1,
|
||||
"bots_failed": [],
|
||||
"tick_queries": 58
|
||||
},
|
||||
{
|
||||
"server": "vanilla",
|
||||
"bots": 10,
|
||||
"behaviour": "walk",
|
||||
"start_seconds": 14.76,
|
||||
"mspt": {
|
||||
"average": 16.8,
|
||||
"p50": 14.55,
|
||||
"p95": 30.52,
|
||||
"p99": 50.01
|
||||
},
|
||||
"mspt_worst_p99": 205.2,
|
||||
"tps": 20.0,
|
||||
"rss_mb": {
|
||||
"mean": 3490.15,
|
||||
"max": 3580.7
|
||||
},
|
||||
"cpu_percent": 83.7,
|
||||
"chunks_per_second": 100.01,
|
||||
"unique_per_second": 2.83,
|
||||
"forgotten_per_second": 100.04,
|
||||
"waited_ticks_per_second": 0.0,
|
||||
"chunks_total": 38623,
|
||||
"chunks_per_bot": {
|
||||
"walk": 3862.3
|
||||
},
|
||||
"unique_chunks_per_bot": {
|
||||
"walk": 697.2
|
||||
},
|
||||
"resent": 0,
|
||||
"pulled_back": 5,
|
||||
"moved_too_quickly": 5,
|
||||
"moved_wrongly": 0,
|
||||
"bots_joined": 10,
|
||||
"bots_failed": [],
|
||||
"tick_queries": 59
|
||||
},
|
||||
{
|
||||
"server": "vanilla",
|
||||
"bots": 25,
|
||||
"behaviour": "walk",
|
||||
"start_seconds": 14.84,
|
||||
"mspt": {
|
||||
"average": 16.01,
|
||||
"p50": 14.55,
|
||||
"p95": 26.39,
|
||||
"p99": 51.26
|
||||
},
|
||||
"mspt_worst_p99": 397.8,
|
||||
"tps": 20.0,
|
||||
"rss_mb": {
|
||||
"mean": 3550.02,
|
||||
"max": 3593.5
|
||||
},
|
||||
"cpu_percent": 114.96,
|
||||
"chunks_per_second": 251.62,
|
||||
"unique_per_second": 6.72,
|
||||
"forgotten_per_second": 251.16,
|
||||
"waited_ticks_per_second": 0.0,
|
||||
"chunks_total": 98701,
|
||||
"chunks_per_bot": {
|
||||
"walk": 3948.04
|
||||
},
|
||||
"unique_chunks_per_bot": {
|
||||
"walk": 698.84
|
||||
},
|
||||
"resent": 0,
|
||||
"pulled_back": 10,
|
||||
"moved_too_quickly": 10,
|
||||
"moved_wrongly": 0,
|
||||
"bots_joined": 25,
|
||||
"bots_failed": [],
|
||||
"tick_queries": 59
|
||||
},
|
||||
{
|
||||
"server": "vanilla",
|
||||
"bots": 50,
|
||||
"behaviour": "walk",
|
||||
"start_seconds": 14.43,
|
||||
"mspt": {
|
||||
"average": 20.52,
|
||||
"p50": 19.55,
|
||||
"p95": 29.72,
|
||||
"p99": 49.04
|
||||
},
|
||||
"mspt_worst_p99": 600.4,
|
||||
"tps": 20.0,
|
||||
"rss_mb": {
|
||||
"mean": 3633.12,
|
||||
"max": 3658.7
|
||||
},
|
||||
"cpu_percent": 193.84,
|
||||
"chunks_per_second": 501.86,
|
||||
"unique_per_second": 12.15,
|
||||
"forgotten_per_second": 501.8,
|
||||
"waited_ticks_per_second": 0.0,
|
||||
"chunks_total": 203429,
|
||||
"chunks_per_bot": {
|
||||
"walk": 4068.58
|
||||
},
|
||||
"unique_chunks_per_bot": {
|
||||
"walk": 698.66
|
||||
},
|
||||
"resent": 0,
|
||||
"pulled_back": 33,
|
||||
"moved_too_quickly": 32,
|
||||
"moved_wrongly": 0,
|
||||
"bots_joined": 50,
|
||||
"bots_failed": [],
|
||||
"tick_queries": 59
|
||||
},
|
||||
{
|
||||
"server": "vanilla",
|
||||
"bots": 100,
|
||||
"behaviour": "walk",
|
||||
"start_seconds": 15.34,
|
||||
"error": "[Errno 32] Broken pipe"
|
||||
},
|
||||
{
|
||||
"server": "vanilla",
|
||||
"bots": 1,
|
||||
"behaviour": "explore",
|
||||
"start_seconds": 15.55,
|
||||
"mspt": {
|
||||
"average": 9.93,
|
||||
"p50": 8.12,
|
||||
"p95": 21.33,
|
||||
"p99": 60.19
|
||||
},
|
||||
"mspt_worst_p99": 373.0,
|
||||
"tps": 20.0,
|
||||
"rss_mb": {
|
||||
"mean": 3564.11,
|
||||
"max": 3655.0
|
||||
},
|
||||
"cpu_percent": 245.18,
|
||||
"chunks_per_second": 35.93,
|
||||
"unique_per_second": 33.35,
|
||||
"forgotten_per_second": 35.64,
|
||||
"waited_ticks_per_second": 0.0,
|
||||
"chunks_total": 12340,
|
||||
"chunks_per_bot": {
|
||||
"explore": 12340.0
|
||||
},
|
||||
"unique_chunks_per_bot": {
|
||||
"explore": 11569.0
|
||||
},
|
||||
"resent": 0,
|
||||
"pulled_back": 261,
|
||||
"moved_too_quickly": 0,
|
||||
"moved_wrongly": 0,
|
||||
"bots_joined": 1,
|
||||
"bots_failed": [],
|
||||
"tick_queries": 59
|
||||
},
|
||||
{
|
||||
"server": "vanilla",
|
||||
"bots": 10,
|
||||
"behaviour": "explore",
|
||||
"start_seconds": 15.03,
|
||||
"mspt": {
|
||||
"average": 16.99,
|
||||
"p50": 14.46,
|
||||
"p95": 29.61,
|
||||
"p99": 132.18
|
||||
},
|
||||
"mspt_worst_p99": 1244.0,
|
||||
"tps": 20.0,
|
||||
"rss_mb": {
|
||||
"mean": 4135.2,
|
||||
"max": 4453.1
|
||||
},
|
||||
"cpu_percent": 242.32,
|
||||
"chunks_per_second": 22.39,
|
||||
"unique_per_second": 22.41,
|
||||
"forgotten_per_second": 22.78,
|
||||
"waited_ticks_per_second": 78.78,
|
||||
"chunks_total": 10177,
|
||||
"chunks_per_bot": {
|
||||
"explore": 1017.7
|
||||
},
|
||||
"unique_chunks_per_bot": {
|
||||
"explore": 1016.2
|
||||
},
|
||||
"resent": 0,
|
||||
"pulled_back": 17314,
|
||||
"moved_too_quickly": 19,
|
||||
"moved_wrongly": 0,
|
||||
"bots_joined": 10,
|
||||
"bots_failed": [],
|
||||
"tick_queries": 59
|
||||
},
|
||||
{
|
||||
"server": "vanilla",
|
||||
"bots": 25,
|
||||
"behaviour": "explore",
|
||||
"start_seconds": 14.84,
|
||||
"mspt": {
|
||||
"average": 24.19,
|
||||
"p50": 21.22,
|
||||
"p95": 36.51,
|
||||
"p99": 174.82
|
||||
},
|
||||
"mspt_worst_p99": 1532.8,
|
||||
"tps": 20.0,
|
||||
"rss_mb": {
|
||||
"mean": 4504.55,
|
||||
"max": 4988.9
|
||||
},
|
||||
"cpu_percent": 319.55,
|
||||
"chunks_per_second": 33.78,
|
||||
"unique_per_second": 33.66,
|
||||
"forgotten_per_second": 34.64,
|
||||
"waited_ticks_per_second": 269.22,
|
||||
"chunks_total": 20522,
|
||||
"chunks_per_bot": {
|
||||
"explore": 820.88
|
||||
},
|
||||
"unique_chunks_per_bot": {
|
||||
"explore": 815.76
|
||||
},
|
||||
"resent": 0,
|
||||
"pulled_back": 46246,
|
||||
"moved_too_quickly": 28,
|
||||
"moved_wrongly": 0,
|
||||
"bots_joined": 25,
|
||||
"bots_failed": [],
|
||||
"tick_queries": 58
|
||||
},
|
||||
{
|
||||
"server": "vanilla",
|
||||
"bots": 50,
|
||||
"behaviour": "explore",
|
||||
"start_seconds": 14.73,
|
||||
"error": "[Errno 32] Broken pipe"
|
||||
},
|
||||
{
|
||||
"server": "vanilla",
|
||||
"bots": 100,
|
||||
"behaviour": "explore",
|
||||
"start_seconds": 14.54,
|
||||
"error": "[Errno 32] Broken pipe"
|
||||
}
|
||||
]
|
||||
}
|
||||
+120
-12
@@ -9,6 +9,11 @@
|
||||
//! horizon-bots --server 127.0.0.1:25565 --bots 25 --behaviour mixed --seconds 360 --report bots.json
|
||||
//!
|
||||
//! It prints one JSON line a second (bots online, chunks received) and writes a summary with `--report`.
|
||||
//! A bot moves only while it holds the chunk it stands in: vanilla's client stops ticking its player in a chunk
|
||||
//! it hasn't received, so a player can't outrun the server's chunk sending, on any server.
|
||||
//! Each bot also keeps the chunks it holds, as a client's chunk cache does, so the report tells chunks that
|
||||
//! entered the view apart from chunks sent again while the bot still held them (`resent`), and counts the
|
||||
//! times the server pulled a bot back (`pulled_back`: a teleport after the first, as for "moved too quickly").
|
||||
|
||||
use horizon_protocol::{
|
||||
PROTOCOL_VERSION,
|
||||
@@ -25,6 +30,7 @@ use horizon_protocol::{
|
||||
};
|
||||
use serde_json::json;
|
||||
use std::{
|
||||
collections::HashSet,
|
||||
sync::{
|
||||
Arc, Mutex,
|
||||
atomic::{AtomicBool, AtomicU64, Ordering},
|
||||
@@ -43,8 +49,9 @@ horizon-bots: load-test bots for a Minecraft Java Edition 26.3 server (offline m
|
||||
--seconds N how long to run after the first bot joins; default 60
|
||||
--view-distance N what the bots ask for; default 10
|
||||
--seed N default 12345
|
||||
--explore-speed B blocks a second an explore bot flies; default 22 (as fast as chunks come)
|
||||
--name PREFIX bot names: PREFIX0, PREFIX1...; default Bot
|
||||
--report FILE a JSON summary at the end
|
||||
--report FILE a JSON summary at the end (- prints it after the per-second lines)
|
||||
";
|
||||
|
||||
#[derive(Clone, Copy, Debug, PartialEq)]
|
||||
@@ -73,6 +80,7 @@ struct Options {
|
||||
seconds: u64,
|
||||
view_distance: i8,
|
||||
seed: u64,
|
||||
explore_speed: f64,
|
||||
name: String,
|
||||
report: Option<String>,
|
||||
}
|
||||
@@ -87,6 +95,7 @@ fn options() -> Result<Options, String> {
|
||||
seconds: 60,
|
||||
view_distance: 10,
|
||||
seed: 12345,
|
||||
explore_speed: 22.0,
|
||||
name: "Bot".into(),
|
||||
report: None,
|
||||
};
|
||||
@@ -110,6 +119,9 @@ fn options() -> Result<Options, String> {
|
||||
"--seconds" => o.seconds = value.parse().map_err(|_| number("seconds"))?,
|
||||
"--view-distance" => o.view_distance = value.parse().map_err(|_| number("2 to 32"))?,
|
||||
"--seed" => o.seed = value.parse().map_err(|_| number("a number"))?,
|
||||
"--explore-speed" => {
|
||||
o.explore_speed = value.parse().map_err(|_| number("blocks a second"))?;
|
||||
}
|
||||
"--name" => o.name = value,
|
||||
"--report" => o.report = Some(value),
|
||||
_ => return Err(format!("unknown option {arg}; --help lists them")),
|
||||
@@ -129,6 +141,17 @@ fn options() -> Result<Options, String> {
|
||||
struct Bot {
|
||||
online: AtomicBool,
|
||||
chunks: AtomicU64,
|
||||
/// Chunks received while the bot already held them (no ForgetLevelChunk in between).
|
||||
resent: AtomicU64,
|
||||
/// Different chunks received over the run.
|
||||
unique: AtomicU64,
|
||||
forgotten: AtomicU64,
|
||||
/// Teleports after the spawn one: the server refused a move.
|
||||
pulled_back: AtomicU64,
|
||||
/// Ticks the bot couldn't move because it didn't hold its chunk yet.
|
||||
waited: AtomicU64,
|
||||
/// The chunks the bot holds (ClientChunkCache).
|
||||
held: Mutex<HashSet<(i32, i32)>>,
|
||||
/// Where the server last put the bot (a teleport), to start from.
|
||||
teleport: Mutex<Option<[f64; 3]>>,
|
||||
error: Mutex<Option<String>>,
|
||||
@@ -268,6 +291,36 @@ fn join(o: &Options, name: &str) -> Result<Connection, String> {
|
||||
}
|
||||
}
|
||||
|
||||
/// ChunkBatchSizeCalculator: the chunks per tick a vanilla client asks for, from how long its batches take
|
||||
/// to arrive.
|
||||
struct BatchSize {
|
||||
nanos_per_chunk: f64,
|
||||
old_samples_weight: u32,
|
||||
started: Instant,
|
||||
}
|
||||
|
||||
impl BatchSize {
|
||||
fn new() -> Self {
|
||||
Self {
|
||||
nanos_per_chunk: 2_000_000.0,
|
||||
old_samples_weight: 1,
|
||||
started: Instant::now(),
|
||||
}
|
||||
}
|
||||
fn finished(&mut self, size: i32) {
|
||||
if size > 0 {
|
||||
let per_chunk = self.started.elapsed().as_nanos() as f64 / f64::from(size);
|
||||
let clamped = per_chunk.clamp(self.nanos_per_chunk / 3.0, self.nanos_per_chunk * 3.0);
|
||||
let weight = f64::from(self.old_samples_weight);
|
||||
self.nanos_per_chunk = (self.nanos_per_chunk * weight + clamped) / (weight + 1.0);
|
||||
self.old_samples_weight = (self.old_samples_weight + 1).min(49);
|
||||
}
|
||||
}
|
||||
fn desired_chunks_per_tick(&self) -> f32 {
|
||||
(7_000_000.0 / self.nanos_per_chunk) as f32
|
||||
}
|
||||
}
|
||||
|
||||
/// Reads the play state: answers what a client must answer, counts chunks, records teleports.
|
||||
fn read(
|
||||
mut reader: horizon_protocol::connection::ConnReader,
|
||||
@@ -277,6 +330,9 @@ fn read(
|
||||
use ids::play::clientbound as cb;
|
||||
use ids::play::serverbound as sb;
|
||||
let send = |body: Vec<u8>| writer.lock().unwrap().send(&body);
|
||||
let mut seen = HashSet::new();
|
||||
let mut batch = BatchSize::new();
|
||||
let mut teleports = 0u64;
|
||||
loop {
|
||||
let (id, body) = match reader.recv() {
|
||||
Ok(packet) => packet,
|
||||
@@ -299,14 +355,38 @@ fn read(
|
||||
},
|
||||
cb::LEVEL_CHUNK_WITH_LIGHT => {
|
||||
bot.chunks.fetch_add(1, Ordering::Relaxed);
|
||||
if let (Ok(x), Ok(z)) = (r.i32(), r.i32()) {
|
||||
if !bot.held.lock().unwrap().insert((x, z)) {
|
||||
bot.resent.fetch_add(1, Ordering::Relaxed);
|
||||
}
|
||||
if seen.insert((x, z)) {
|
||||
bot.unique.fetch_add(1, Ordering::Relaxed);
|
||||
}
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
cb::CHUNK_BATCH_FINISHED => send(
|
||||
play::Serverbound::ChunkBatchReceived(play::ChunkBatchReceived {
|
||||
chunks_per_tick: 25.0,
|
||||
})
|
||||
.encode(),
|
||||
),
|
||||
cb::FORGET_LEVEL_CHUNK => {
|
||||
if let Ok(play::Clientbound::ForgetLevelChunk(p)) =
|
||||
play::Clientbound::decode(id, &body)
|
||||
&& bot.held.lock().unwrap().remove(&(p.pos[0], p.pos[1]))
|
||||
{
|
||||
bot.forgotten.fetch_add(1, Ordering::Relaxed);
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
cb::CHUNK_BATCH_START => {
|
||||
batch.started = Instant::now();
|
||||
Ok(())
|
||||
}
|
||||
cb::CHUNK_BATCH_FINISHED => {
|
||||
batch.finished(r.varint().unwrap_or(0));
|
||||
send(
|
||||
play::Serverbound::ChunkBatchReceived(play::ChunkBatchReceived {
|
||||
chunks_per_tick: batch.desired_chunks_per_tick(),
|
||||
})
|
||||
.encode(),
|
||||
)
|
||||
}
|
||||
cb::PLAYER_POSITION => match play::Clientbound::decode(id, &body) {
|
||||
Ok(play::Clientbound::PlayerPosition(p)) => {
|
||||
let mut at = bot.teleport.lock().unwrap();
|
||||
@@ -320,6 +400,10 @@ fn read(
|
||||
}
|
||||
});
|
||||
*at = Some(pos);
|
||||
teleports += 1;
|
||||
if teleports > 1 {
|
||||
bot.pulled_back.fetch_add(1, Ordering::Relaxed);
|
||||
}
|
||||
send(
|
||||
play::Serverbound::AcceptTeleportation(play::AcceptTeleportation {
|
||||
id: p.teleport,
|
||||
@@ -416,7 +500,10 @@ fn run_bot(
|
||||
if let Some(at) = bot.teleport.lock().unwrap().take() {
|
||||
pos = at;
|
||||
}
|
||||
if pos[1] < cruise {
|
||||
let here = ((pos[0].floor() as i32) >> 4, (pos[2].floor() as i32) >> 4);
|
||||
if !bot.held.lock().unwrap().contains(&here) {
|
||||
bot.waited.fetch_add(1, Ordering::Relaxed);
|
||||
} else if pos[1] < cruise {
|
||||
pos[1] = (pos[1] + 0.5).min(cruise);
|
||||
} else {
|
||||
match behaviour {
|
||||
@@ -440,9 +527,11 @@ fn run_bot(
|
||||
}
|
||||
}
|
||||
Behaviour::Explore => {
|
||||
// Flying fast in creative: about 22 blocks a second, which keeps new chunks coming.
|
||||
pos[0] += heading.cos() * 1.1;
|
||||
pos[2] += heading.sin() * 1.1;
|
||||
// Flying in creative, by default about 22 blocks a second, which keeps new
|
||||
// chunks coming as fast as the server sends them.
|
||||
let step = o.explore_speed / 20.0;
|
||||
pos[0] += heading.cos() * step;
|
||||
pos[2] += heading.sin() * step;
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -520,6 +609,9 @@ fn main() {
|
||||
.filter(|b| b.online.load(Ordering::SeqCst))
|
||||
.count();
|
||||
let chunks: u64 = bots.iter().map(|b| b.chunks.load(Ordering::Relaxed)).sum();
|
||||
let total = |f: fn(&Bot) -> &AtomicU64| -> u64 {
|
||||
bots.iter().map(|b| f(b).load(Ordering::Relaxed)).sum()
|
||||
};
|
||||
let failed = bots
|
||||
.iter()
|
||||
.filter(|b| b.error.lock().unwrap().is_some())
|
||||
@@ -536,6 +628,11 @@ fn main() {
|
||||
"failed": failed,
|
||||
"chunks": chunks,
|
||||
"chunks_per_second": chunks - previous_chunks,
|
||||
"unique": total(|b| &b.unique),
|
||||
"forgotten": total(|b| &b.forgotten),
|
||||
"resent": total(|b| &b.resent),
|
||||
"pulled_back": total(|b| &b.pulled_back),
|
||||
"waited_ticks": total(|b| &b.waited),
|
||||
})
|
||||
);
|
||||
previous_chunks = chunks;
|
||||
@@ -554,19 +651,30 @@ fn main() {
|
||||
"bots": o.bots,
|
||||
"behaviour": o.behaviour,
|
||||
"seed": o.seed,
|
||||
"explore_speed": o.explore_speed,
|
||||
"view_distance": o.view_distance,
|
||||
"seconds": o.seconds,
|
||||
"joined": bots.iter().filter(|b| b.joined_after_ms.load(Ordering::SeqCst) > 0).count(),
|
||||
"chunks": bots.iter().map(|b| b.chunks.load(Ordering::Relaxed)).sum::<u64>(),
|
||||
"resent": bots.iter().map(|b| b.resent.load(Ordering::Relaxed)).sum::<u64>(),
|
||||
"pulled_back": bots.iter().map(|b| b.pulled_back.load(Ordering::Relaxed)).sum::<u64>(),
|
||||
"waited_ticks": bots.iter().map(|b| b.waited.load(Ordering::Relaxed)).sum::<u64>(),
|
||||
"per_bot": bots.iter().enumerate().map(|(i, b)| json!({
|
||||
"name": format!("{}{i}", o.name),
|
||||
"behaviour": behaviour_of(&o.behaviour, i).name(),
|
||||
"joined_after_ms": b.joined_after_ms.load(Ordering::SeqCst),
|
||||
"chunks": b.chunks.load(Ordering::Relaxed),
|
||||
"unique_chunks": b.unique.load(Ordering::Relaxed),
|
||||
"resent": b.resent.load(Ordering::Relaxed),
|
||||
"forgotten": b.forgotten.load(Ordering::Relaxed),
|
||||
"pulled_back": b.pulled_back.load(Ordering::Relaxed),
|
||||
"waited_ticks": b.waited.load(Ordering::Relaxed),
|
||||
"error": *b.error.lock().unwrap(),
|
||||
})).collect::<Vec<_>>(),
|
||||
});
|
||||
if let Some(path) = &o.report {
|
||||
if o.report.as_deref() == Some("-") {
|
||||
println!("{summary:#}");
|
||||
} else if let Some(path) = &o.report {
|
||||
if let Err(e) = std::fs::write(path, format!("{summary:#}\n")) {
|
||||
eprintln!("horizon-bots: {path}: {e}");
|
||||
}
|
||||
|
||||
+17
-3
@@ -90,7 +90,7 @@ not tick cost, so it is in scope here (see section 9).
|
||||
| Management | Only stdin. There is no service, no backups, no status beyond `list`, no restart after a crash, and a server can't be managed while it runs in the background. |
|
||||
| Packaging | The server is the full game client. Resources come from `horizon-prepare`, which needs a JDK and the official launcher's client install, both awkward on a headless VPS. |
|
||||
| Tick times | The dedicated loop never recorded its tick times in the game clock (only the windowed loop did), so `/tick query` on a dedicated server reported 0 ms. Fixed in PR 2b, which the benchmark needs (section 9). |
|
||||
| Scale | Not measured with more than one player. The tick is single-threaded, and each remote player costs the hub one tracking and chunk-sending pass. |
|
||||
| Scale | Measured against vanilla on equal work (section 9, `benchmarks/server-equal-work-2026-10-17/`): 20 TPS up to 50 bots, 9 to 11 TPS at 100, where vanilla's watchdog stops it. Horizon is ahead with few players and when exploring; with 25 to 50 players near spawn vanilla's tick is shorter. |
|
||||
|
||||
### Harnesses on main
|
||||
|
||||
@@ -463,8 +463,22 @@ reproducible by anyone, and show the same gameplay on both sides.
|
||||
- Vanilla's chunk sending yields under load.
|
||||
- Horizon sent each nearby `walk` bot 1.5–4 times as many chunks.
|
||||
|
||||
Horizon sent 2–4 times the chunks per second in every multi-bot run. The next run equalises the work
|
||||
(slower `explore`, a `walk`-only pass) on a quiet machine before anything is published.
|
||||
Horizon sent 2–4 times the chunks per second in every multi-bot run.
|
||||
- **Equal-work run (2026-10-17,** `benchmarks/server-equal-work-2026-10-17/`, on the ThinkPad with the bots
|
||||
on the Mac mini). Horizon now checks moves and paces chunks as vanilla does (#586), the bots wait for the
|
||||
chunk they stand in, an `--explore-speed` pass keeps both servers at the same rate, and vanilla runs with
|
||||
`gamerule locator_bar false` (Horizon has no locator bar yet). Results:
|
||||
- Memory: Horizon's RSS is 1.9 to 2.6 times smaller in every run (vanilla's heap is fixed at 4 GB).
|
||||
- Walking near spawn: Horizon's tick is shorter with 1 and 10 bots (5.5 against 18.3 ms, 11.5 against
|
||||
16.8), and longer with 25 and 50 (20.7 against 16.0, 31.4 against 20.5). Both keep 20 TPS up to 50.
|
||||
- Exploring at the same rate: Horizon's tick is about half of vanilla's (13.5 against 29.2 ms at 10 bots,
|
||||
20.2 against 37.0 at 25).
|
||||
- At 100 bots: vanilla's watchdog stops it in every pass; Horizon runs at 9 to 11 TPS.
|
||||
|
||||
Two server costs were found and fixed on the way: every chunk section was encoded once per player (#623),
|
||||
and every packet woke its connection's writer thread (#649). Together they took the 50-bot walking tick
|
||||
from 281 ms to 31 ms. At 25 walking bots most of what remains is the engine's streaming and block entity
|
||||
ticks, both of which grow with players (the README has the profile).
|
||||
- **Ownership.** perf-server-1 owns making the tick faster. This work only measures it, files what it finds,
|
||||
and fixes startup, RAM and packaging.
|
||||
|
||||
|
||||
+91
-24
@@ -8,10 +8,18 @@ then measures for a while:
|
||||
- MSPT (average, P50, P95, P99) from each server's own `/tick query`, asked every 5 seconds on the console:
|
||||
vanilla's ServerTickRateManager numbers, which Horizon reproduces, so both measure themselves the same way;
|
||||
- RSS and CPU of the server's whole process tree, once a second;
|
||||
- chunks the bots receive per second (new terrain while the `explore` bots fly outwards);
|
||||
- chunks the bots receive per second (new terrain while the `explore` bots fly outwards), chunks sent again
|
||||
while a bot still held them, and the moves the server refused (pull-backs; "moved too quickly" and
|
||||
"moved wrongly" lines);
|
||||
- the start time to `Done`, and every bot that failed to join or was dropped.
|
||||
|
||||
python3 tools/server_bench.py --servers horizon,vanilla --bots 1,10,25 --seconds 120
|
||||
python3 tools/server_bench.py --servers horizon,vanilla --bots 1,10,25 --behaviour walk,explore
|
||||
|
||||
The bots can run on another machine, so they don't take the server's CPU: `--bots-via 'ssh macmini'` with
|
||||
`--bots-bin` naming horizon-bots there and `--connect` the address it reaches this machine at. `--nice 19`
|
||||
runs every server at the lowest priority (a shared machine). The explore bots fly as fast as each server
|
||||
sends them chunks, so a faster server gets more terrain to generate; `--explore-speed` slows them to a rate
|
||||
both servers keep up with, for equal work.
|
||||
|
||||
The vanilla side runs the pinned server jar from tools/lab.py prepare (your own copy). It needs Java 25. The
|
||||
Horizon side runs target/release/horizon-server with the engine beside it; build both first:
|
||||
@@ -186,8 +194,16 @@ def server_command(kind, directory, args):
|
||||
return [server, '--dir', directory, 'run']
|
||||
|
||||
|
||||
def run_once(kind, bots, args, out):
|
||||
directory = out / f'{kind}-{bots}'
|
||||
def bots_command(args):
|
||||
"""horizon-bots here, or through `--bots-via` (for example `ssh macmini`) on another machine."""
|
||||
if args.bots_via:
|
||||
return [*args.bots_via.split(), args.bots_bin or 'horizon-bots']
|
||||
return [args.bots_bin or BIN / f'horizon-bots{EXE}']
|
||||
|
||||
|
||||
def run_once(kind, bots, behaviour, args, out):
|
||||
name = f'{kind}-{behaviour}-{bots}'
|
||||
directory = out / name
|
||||
shutil.rmtree(directory, ignore_errors=True)
|
||||
directory.mkdir(parents=True)
|
||||
port = free_port()
|
||||
@@ -195,19 +211,27 @@ def run_once(kind, bots, args, out):
|
||||
(directory / 'server.properties').write_text(''.join(f'{k}={v}\n' for k, v in props.items()))
|
||||
(directory / 'eula.txt').write_text('eula=true\n')
|
||||
command = server_command(kind, directory, args)
|
||||
if args.nice:
|
||||
command = ['nice', '-n', str(args.nice), *command]
|
||||
started = time.time()
|
||||
server = Server(command, directory, out / f'{kind}-{bots}.log')
|
||||
result = {'server': kind, 'bots': bots}
|
||||
server = Server(command, directory, out / f'{name}.log')
|
||||
result = {'server': kind, 'bots': bots, 'behaviour': behaviour}
|
||||
try:
|
||||
server.wait_for(r'/INFO\]: Done \(', 600)
|
||||
result['start_seconds'] = round(time.time() - started, 2)
|
||||
# Every bot appears on the world spawn itself, not spread around it, so both servers see the same paths.
|
||||
server.console('gamerule respawn_radius 0')
|
||||
for rule in filter(None, args.gamerule.split(',')):
|
||||
server.console('gamerule ' + rule.replace('=', ' '))
|
||||
time.sleep(5)
|
||||
report = out / f'{kind}-{bots}-bots.json'
|
||||
report = out / f'{name}-bots.json'
|
||||
total = bots * args.join_every + args.warmup + args.seconds
|
||||
# The summary comes back on stdout after the per-second lines, so it works over ssh too.
|
||||
bot_process = subprocess.Popen(
|
||||
[BIN / f'horizon-bots{EXE}', '--server', f'127.0.0.1:{port}', '--bots', str(bots),
|
||||
'--behaviour', args.behaviour, '--join-every', str(args.join_every), '--seconds', str(int(total)),
|
||||
'--seed', str(args.seed), '--report', report],
|
||||
[*bots_command(args), '--server', f'{args.connect}:{port}', '--bots', str(bots),
|
||||
'--behaviour', behaviour, '--join-every', str(args.join_every), '--seconds', str(int(total)),
|
||||
'--seed', str(args.seed), '--report', '-',
|
||||
*(['--explore-speed', str(args.explore_speed)] if args.explore_speed else [])],
|
||||
stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True)
|
||||
bot_lines = []
|
||||
threading.Thread(target=lambda: bot_lines.extend(bot_process.stdout), daemon=True).start()
|
||||
@@ -229,11 +253,27 @@ def run_once(kind, bots, args, out):
|
||||
if (t := tick_query(server)) is not None:
|
||||
ticks.append(t)
|
||||
next_query = now + 5
|
||||
measured = [json.loads(l) for l in list(bot_lines) if l.startswith('{')]
|
||||
window = [l for l in measured if l['t'] >= measured[-1]['t'] - args.seconds] if measured else []
|
||||
bot_process.wait(timeout=total + 120)
|
||||
summary = json.loads(report.read_text()) if report.exists() else {}
|
||||
time.sleep(0.5)
|
||||
measured, rest = [], []
|
||||
for line in list(bot_lines):
|
||||
try:
|
||||
sample = json.loads(line)
|
||||
except ValueError:
|
||||
sample = None
|
||||
if isinstance(sample, dict) and 't' in sample:
|
||||
measured.append(sample)
|
||||
else:
|
||||
rest.append(line)
|
||||
window = [l for l in measured if l['t'] >= measured[-1]['t'] - args.seconds] if measured else []
|
||||
summary_text = ''.join(rest)
|
||||
summary = json.loads(summary_text) if summary_text.strip() else {}
|
||||
report.write_text(json.dumps(summary, indent=2) + '\n')
|
||||
mean = lambda xs: round(statistics.fmean(xs), 2) if xs else None
|
||||
per_bot = summary.get('per_bot', [])
|
||||
by_behaviour = {}
|
||||
for b in per_bot:
|
||||
by_behaviour.setdefault(b['behaviour'], []).append(b)
|
||||
result.update({
|
||||
'mspt': {k: mean([t[k] for t in ticks]) for k in ('average', 'p50', 'p95', 'p99')},
|
||||
'mspt_worst_p99': max((t['p99'] for t in ticks), default=None),
|
||||
@@ -242,6 +282,19 @@ def run_once(kind, bots, args, out):
|
||||
'rss_mb': {'mean': mean([r / 1e6 for r in rss]), 'max': round(max(rss) / 1e6, 1) if rss else None},
|
||||
'cpu_percent': mean(cpu),
|
||||
'chunks_per_second': mean([l['chunks_per_second'] for l in window]),
|
||||
# Over the measured window: different chunks reaching the bots, chunks they had dropped and got
|
||||
# again (walking back), and ticks the bots stood still waiting for their own chunk.
|
||||
**({k + '_per_second': round((window[-1][k] - window[0][k]) / max(len(window) - 1, 1), 2)
|
||||
for k in ('unique', 'forgotten', 'waited_ticks')} if len(window) > 1 else {}),
|
||||
'chunks_total': summary.get('chunks'),
|
||||
# Per bot over the whole run: every chunk received, the different ones, and those sent again
|
||||
# while the bot still held them.
|
||||
'chunks_per_bot': {k: mean([b['chunks'] for b in v]) for k, v in by_behaviour.items()},
|
||||
'unique_chunks_per_bot': {k: mean([b['unique_chunks'] for b in v]) for k, v in by_behaviour.items()},
|
||||
'resent': summary.get('resent'),
|
||||
'pulled_back': summary.get('pulled_back'),
|
||||
'moved_too_quickly': sum('moved too quickly!' in l for l in server.since(0)),
|
||||
'moved_wrongly': sum('moved wrongly!' in l for l in server.since(0)),
|
||||
'bots_joined': summary.get('joined'),
|
||||
'bots_failed': [b for b in summary.get('per_bot', []) if b.get('error')],
|
||||
'tick_queries': len(ticks),
|
||||
@@ -257,23 +310,34 @@ def main():
|
||||
parser = argparse.ArgumentParser(description=__doc__.split('\n\n')[0])
|
||||
parser.add_argument('--servers', default='horizon,vanilla')
|
||||
parser.add_argument('--bots', default='1,10,25,50')
|
||||
parser.add_argument('--behaviour', default='mixed')
|
||||
parser.add_argument('--behaviour', default='walk,explore', help='one pass per behaviour (comma-separated)')
|
||||
parser.add_argument('--join-every', type=float, default=1.0)
|
||||
parser.add_argument('--warmup', type=int, default=60)
|
||||
parser.add_argument('--seconds', type=int, default=300)
|
||||
parser.add_argument('--seed', type=int, default=12345)
|
||||
parser.add_argument('--explore-speed', type=float, default=0,
|
||||
help='blocks a second the explore bots fly (default: the bots\' 22, as fast as chunks '
|
||||
'come); a speed both servers keep up with makes the explore pass equal work')
|
||||
parser.add_argument('--gamerule', default='',
|
||||
help="more game rules for both servers, 'name=value,...' (locator_bar=false: Horizon has "
|
||||
"no locator bar yet)")
|
||||
parser.add_argument('--heap', default='4g', help='the Java heap, -Xms and -Xmx (vanilla)')
|
||||
parser.add_argument('--out', type=Path, default=None, help='the report (JSON)')
|
||||
parser.add_argument('--nice', type=int, default=0, help='run the servers under nice -n N')
|
||||
parser.add_argument('--bots-via', default='', help="a command prefix to run the bots elsewhere ('ssh host')")
|
||||
parser.add_argument('--bots-bin', default='', help='horizon-bots (on the --bots-via machine)')
|
||||
parser.add_argument('--connect', default='127.0.0.1', help='the address the bots reach the server at')
|
||||
args = parser.parse_args()
|
||||
out = lab.LOCAL / 'server-bench'
|
||||
out.mkdir(parents=True, exist_ok=True)
|
||||
runs = []
|
||||
for bots in [int(b) for b in args.bots.split(',')]:
|
||||
for kind in args.servers.split(','):
|
||||
print(f'{kind}, {bots} bots...', flush=True)
|
||||
result = run_once(kind, bots, args, out)
|
||||
runs.append(result)
|
||||
print(json.dumps(result), flush=True)
|
||||
for behaviour in args.behaviour.split(','):
|
||||
for bots in [int(b) for b in args.bots.split(',')]:
|
||||
for kind in args.servers.split(','):
|
||||
print(f'{kind}, {bots} {behaviour} bots...', flush=True)
|
||||
result = run_once(kind, bots, behaviour, args, out)
|
||||
runs.append(result)
|
||||
print(json.dumps(result), flush=True)
|
||||
report = {
|
||||
'kind': 'horizon-server-benchmark',
|
||||
'date': datetime.datetime.now(datetime.timezone.utc).isoformat(timespec='seconds'),
|
||||
@@ -286,13 +350,16 @@ def main():
|
||||
path = args.out or out / 'report.json'
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
path.write_text(json.dumps(report, indent=2) + '\n')
|
||||
print(f'\n{"server":8} {"bots":>4} {"start s":>8} {"MSPT":>6} {"P95":>6} {"P99":>6} {"RSS MB":>7} {"CPU %":>6} {"chunks/s":>8} {"joined":>6}')
|
||||
print(f'\n{"server":8} {"pass":8} {"bots":>4} {"start s":>8} {"MSPT":>6} {"P95":>6} {"P99":>6} {"RSS MB":>7} '
|
||||
f'{"CPU %":>6} {"chunks/s":>8} {"/bot":>6} {"resent":>6} {"pulled":>6} {"joined":>6}')
|
||||
for r in runs:
|
||||
if 'error' in r:
|
||||
print(f'{r["server"]:8} {r["bots"]:>4} failed: {r["error"].splitlines()[0]}')
|
||||
print(f'{r["server"]:8} {r["behaviour"]:8} {r["bots"]:>4} failed: {r["error"].splitlines()[0]}')
|
||||
continue
|
||||
print(f'{r["server"]:8} {r["bots"]:>4} {r["start_seconds"]:>8} {r["mspt"]["average"]!s:>6} {r["mspt"]["p95"]!s:>6} '
|
||||
f'{r["mspt"]["p99"]!s:>6} {r["rss_mb"]["mean"]!s:>7} {r["cpu_percent"]!s:>6} {r["chunks_per_second"]!s:>8} '
|
||||
per_bot = r['chunks_per_bot'].get(r['behaviour'])
|
||||
print(f'{r["server"]:8} {r["behaviour"]:8} {r["bots"]:>4} {r["start_seconds"]:>8} {r["mspt"]["average"]!s:>6} '
|
||||
f'{r["mspt"]["p95"]!s:>6} {r["mspt"]["p99"]!s:>6} {r["rss_mb"]["mean"]!s:>7} {r["cpu_percent"]!s:>6} '
|
||||
f'{r["chunks_per_second"]!s:>8} {per_bot!s:>6} {r["resent"]!s:>6} {r["pulled_back"]!s:>6} '
|
||||
f'{r["bots_joined"]!s:>6}')
|
||||
print(f'\nWrote {path}')
|
||||
|
||||
|
||||
Reference in new issue
Block a user