mirror of
https://github.com/horizon-rust/horizon.git
synced 2026-11-07 02:56:17 -10:00
horizon-prepare: the release check's repeat run compares the manifest too; four JVMs with the split
The repeat run's file check skipped horizon-resources.json, so a manifest the repeat rewrote differently passed. The comments that said at most three JVMs run at once now say four with the second server JVM. Co-Authored-By: Claude Opus 5.5 <[email protected]>
This commit is contained in:
1 parent
9cecb86ec2
commit
5b16c46f5d
4 files changed
+14
-12
No files matched your search
@@ -15,7 +15,7 @@ pub struct Jdk {
|
||||
const TIMEOUT: Duration = Duration::from_secs(3600);
|
||||
|
||||
/// Options for every JVM preparing starts (the exporters, the data generator, javac with `-J`).
|
||||
/// They are short batch jobs that run three at a time: the C2 compiler's background work costs
|
||||
/// They are short batch jobs that run side by side: the C2 compiler's background work costs
|
||||
/// more CPU than it wins back, and the serial collector keeps their heaps small. Measured on the
|
||||
/// pinned jars, a fresh preparation took 105 s with the defaults and 63-66 s with these, its
|
||||
/// JVMs together at most 1.25 GB (2 GB with the parallel collector, as fast), same output.
|
||||
|
||||
@@ -754,7 +754,8 @@ type Entries = Vec<(&'static str, Value)>;
|
||||
/// The exporters, the data generator and the asset copy, on four lanes that run at once:
|
||||
/// the server exporter JVM's steps in a fixed order (they share its state), the client
|
||||
/// exporter JVM's then the command registries' JVM, the data generator's then the creative
|
||||
/// inventory's, and the copying, which needs no Java. At most three JVMs run at a time.
|
||||
/// inventory's, and the copying, which needs no Java. At most three JVMs run at a time, four
|
||||
/// with the split below.
|
||||
/// From this many cores (`pool::jobs`) the generator's block behaviour gets its own server JVM.
|
||||
/// It pays a second Bootstrap and ~400 MB, which only pays off when the other lanes leave cores
|
||||
/// idle: a 6-core laptop finished its other JVMs 20 s before the server lane.
|
||||
|
||||
@@ -41,13 +41,13 @@ Fresh release preparation therefore starts five exporter/data-generator JVMs, do
|
||||
(six with 6 cores or more, below), and invokes no compiler. `HORIZON_PREPARE_ISOLATED=1`
|
||||
retains separate exporter JVMs for output comparisons and diagnosis.
|
||||
|
||||
Those JVMs run on four lanes at once, so at most three run at a time: the server worker's
|
||||
steps in their fixed order; the client worker's, then the command registries'; the data
|
||||
generator, then the creative tabs'; and the Rust-only copying (client assets and data, the
|
||||
world generator's bundle), spread over several threads. The world generator's tables, which
|
||||
need the block facts and the reports, and the manifest come after all four. A worker stops
|
||||
when its lane is done. The two sides compile at once too. Imports and checks hash the pinned
|
||||
files on several threads; every file is still checked in full.
|
||||
Those JVMs run on four lanes at once, so at most three run at a time (four with the split
|
||||
below): the server worker's steps in their fixed order; the client worker's, then the command
|
||||
registries'; the data generator, then the creative tabs'; and the Rust-only copying (client
|
||||
assets and data, the world generator's bundle), spread over several threads. The world
|
||||
generator's tables, which need the block facts and the reports, and the manifest come after
|
||||
all four. A worker stops when its lane is done. The two sides compile at once too. Imports and
|
||||
checks hash the pinned files on several threads; every file is still checked in full.
|
||||
|
||||
`HORIZON_PREPARE_JOBS` is how many things run at once: the processors by default, at most 8.
|
||||
`1` runs it all one step after another (for a machine short on memory). From 6, the world
|
||||
|
||||
@@ -146,8 +146,9 @@ def main():
|
||||
times.append(round(time.monotonic() - started, 2))
|
||||
print(f'{name}: {times[-1]}s, prepared', flush=True)
|
||||
verify(results)
|
||||
# Nothing changed since the last run: it starts no JVM and rewrites nothing.
|
||||
before = hashes(resources)
|
||||
# Nothing changed since the last run: it starts no JVM and leaves every file as it was.
|
||||
manifest = resources / 'horizon-resources.json'
|
||||
before = hashes(resources), manifest.read_bytes()
|
||||
trace.write_text('')
|
||||
with (work / 'repeat.log').open('w') as log:
|
||||
subprocess.run([str(binary), '--root', str(folder)], cwd=folder, env=env,
|
||||
@@ -155,7 +156,7 @@ def main():
|
||||
launches = [line for line in trace.read_text().splitlines() if line != '-version']
|
||||
if launches:
|
||||
raise SystemExit(f'A repeat preparation started {len(launches)} JVMs')
|
||||
if hashes(resources) != before:
|
||||
if (hashes(resources), manifest.read_bytes()) != before:
|
||||
raise SystemExit('A repeat preparation changed the resources')
|
||||
print('Repeat preparation: no JVM, same files')
|
||||
print(f'Isolated {times[:2]}s, batched {times[2:]}s')
|
||||
|
||||
Reference in new issue
Block a user