mirror of
https://github.com/horizon-rust/horizon.git
synced 2026-11-07 11:06:11 -10:00
RAM limit: step down before the limit on Windows too, never page the game out
On Windows a RAM limit was a hard working set limit (SetProcessWorkingSetSizeEx with QUOTA_LIMITS_HARDWS_MAX_ENABLE): Windows kept the game under it by moving what it used least to the page file, and the game only stepped down once it was already slow and faulting hard. Players saw a low limit make the game slow instead of making it hold less (#6651, #6652). Windows now does what Linux and macOS do: once a second the game compares its working set with the limit and, past nine tenths of it, turns far terrain off and lowers the render distance in use until it fits. The system cap, the page fault pressure, the frozen-game check and the frame clock that fed it are gone. The limit stays opt-in (no launcher value, no limit), and HORIZON_MEMORY_HARD_MB is unchanged. Co-Authored-By: Claude Opus 5.5 <[email protected]>
This commit is contained in:
1 parent
b33bc70c4f
commit
3f9417521f
5 files changed
+36
-229
No files matched your search
@@ -18394,7 +18394,6 @@ impl Game<'_> {
|
||||
self.frame_stats.record_gpu(&r.gpu_times);
|
||||
r.gpu_times.clear();
|
||||
}
|
||||
horizon_client::ram_limit::frame();
|
||||
self.limit_frame();
|
||||
self.pause = self.pauses_game();
|
||||
if let Some(server) = &self.integrated {
|
||||
|
||||
@@ -27,7 +27,7 @@ impl Game<'_> {
|
||||
return;
|
||||
};
|
||||
let has_far = self.settings.horizon.lod_distance > 0;
|
||||
let pressure = ram_limit::pressure(limit, self.fps);
|
||||
let pressure = ram_limit::pressure(limit);
|
||||
// Every ten seconds, what the limit sees: a report can show the trace.
|
||||
if self.ticks.is_multiple_of(200) {
|
||||
let (commit, _) = horizon_threads::allocator_commit();
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
//! The player's RAM limit: from the launcher's slider at the start (`HORIZON_MEMORY_LIMIT_MB`), and
|
||||
//! `/ramlimit` while the game runs. The window answers that command itself, so it works on any
|
||||
//! server. On Windows the system holds the game under the limit (`horizon_threads::memory_limit`).
|
||||
//! Under pressure, [`Governor`] turns far terrain off and lowers the render distance in use, never
|
||||
//! server. Under pressure, before the game reaches the limit (`horizon_threads::memory_limit`),
|
||||
//! on every system, [`Governor`] turns far terrain off and lowers the render distance in use, never
|
||||
//! the option the player saved, and gives them back when there is room. Only a game that can't run
|
||||
//! under the limit at all ends with a crash report.
|
||||
use horizon_threads::capped;
|
||||
@@ -127,17 +127,8 @@ pub fn used() -> Option<u64> {
|
||||
memory_limit::resident_bytes()
|
||||
}
|
||||
|
||||
/// See [`memory_limit::frame`].
|
||||
pub fn frame() {
|
||||
memory_limit::frame();
|
||||
}
|
||||
|
||||
/// A game this slow, faulting hard under a cap the system enforces, is short of memory.
|
||||
const SLOW_FPS: f32 = 30.;
|
||||
|
||||
/// See [`memory_limit::pressure`]. Under a cap the system enforces the game is already under the
|
||||
/// limit, so nothing the player sees goes unless the game is slow as well.
|
||||
pub fn pressure(limit: u64, fps: f32) -> Pressure {
|
||||
/// See [`memory_limit::pressure`].
|
||||
pub fn pressure(limit: u64) -> Pressure {
|
||||
if capped::limit().is_some() {
|
||||
// A hard ceiling is a budget of its own: what the process commits
|
||||
// decides, and a smooth frame does not excuse holding more. Shedding
|
||||
@@ -149,15 +140,7 @@ pub fn pressure(limit: u64, fps: f32) -> Pressure {
|
||||
memory_limit::pressure_of_commit(commit, limit)
|
||||
});
|
||||
}
|
||||
let pressure = memory_limit::pressure(limit);
|
||||
needed(pressure, memory_limit::system_enforced(), fps)
|
||||
}
|
||||
|
||||
fn needed(pressure: Pressure, enforced: bool, fps: f32) -> Pressure {
|
||||
match pressure {
|
||||
Pressure::Over { .. } if enforced && fps >= SLOW_FPS => Pressure::Steady,
|
||||
pressure => pressure,
|
||||
}
|
||||
memory_limit::pressure(limit)
|
||||
}
|
||||
|
||||
/// See [`memory_limit::can_shrink`].
|
||||
@@ -302,19 +285,6 @@ pub fn run(command: &str) -> Option<String> {
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn under_a_cap_the_system_enforces_only_a_slow_game_gives_anything_up() {
|
||||
const OVER: Pressure = Pressure::Over { keep: 0.8 };
|
||||
assert_eq!(needed(OVER, true, 200.), Pressure::Steady);
|
||||
assert_eq!(needed(OVER, true, 12.), OVER);
|
||||
assert_eq!(
|
||||
needed(OVER, false, 200.),
|
||||
OVER,
|
||||
"nothing else holds this game under"
|
||||
);
|
||||
assert_eq!(needed(Pressure::Room, true, 12.), Pressure::Room);
|
||||
}
|
||||
|
||||
const FAR_OVER: Pressure = Pressure::Over { keep: 0.125 };
|
||||
const OVER: Pressure = Pressure::Over { keep: 0.8 };
|
||||
|
||||
|
||||
@@ -1,16 +1,14 @@
|
||||
//! The player's cap on the memory the game holds, and the last resort behind it.
|
||||
//!
|
||||
//! On Windows the system enforces it: a hard working set limit never lets the game hold more in
|
||||
//! RAM, and moves what it used least out instead of failing, so a low cap costs speed. Elsewhere
|
||||
//! no such limit exists, and the game keeps itself under the cap by holding less
|
||||
//! (`horizon_client::ram_limit`, which reads [`pressure`]).
|
||||
//! On every system the game keeps itself under the cap by holding less, before it reaches it
|
||||
//! (`horizon_client::ram_limit`, which reads [`pressure`]). Nothing asks the system to page the
|
||||
//! game out: on Windows a hard working set limit did, and a low cap made the game slow instead of
|
||||
//! making it hold less.
|
||||
//!
|
||||
//! A watcher thread reads the process's resident memory and, over the cap, gives free pages back.
|
||||
//! Only when the game has nothing left to give up ([`can_shrink`] has stopped being called) and
|
||||
//! stays over for [`GRACE`] does it call the game's `exceeded`. A cap the system enforces can't be
|
||||
//! gone over, so there the sign is a game frozen by it: no [`frame`] for [`STALL`] while it faults
|
||||
//! hard. No thread runs until a cap is set, and there is no lowest cap: one the game can't live
|
||||
//! under ends the game.
|
||||
//! stays over for [`GRACE`] does it call the game's `exceeded`. No thread runs until a cap is set,
|
||||
//! and there is no lowest cap: one the game can't live under ends the game.
|
||||
use std::sync::atomic::{AtomicU64, Ordering};
|
||||
use std::sync::{Once, OnceLock};
|
||||
use std::time::{Duration, Instant};
|
||||
@@ -20,12 +18,6 @@ const EVERY: Duration = Duration::from_secs(1);
|
||||
/// How long the game may stay over the cap with nothing left to give up: a world's first chunks,
|
||||
/// or a burst the last step down has not freed yet, pass within it.
|
||||
pub const GRACE: Duration = Duration::from_secs(20);
|
||||
/// How long a game may draw nothing while faulting hard under a cap the system enforces.
|
||||
pub const STALL: Duration = Duration::from_secs(60);
|
||||
/// Page faults a second that say a game drawing nothing is busy fetching memory back, not just
|
||||
/// minimized or waiting.
|
||||
#[cfg_attr(not(windows), allow(dead_code))]
|
||||
const STALL_FAULTS: f64 = 20_000.;
|
||||
/// How long one [`can_shrink`] counts for.
|
||||
const SHRINK_WORD: Duration = Duration::from_secs(5);
|
||||
|
||||
@@ -34,18 +26,11 @@ static WATCHER: Once = Once::new();
|
||||
static STARTED: OnceLock<Instant> = OnceLock::new();
|
||||
/// Until when, in milliseconds since [`STARTED`], the game says it can still shrink.
|
||||
static SHRINKS_UNTIL: AtomicU64 = AtomicU64::new(0);
|
||||
/// When the last [`frame`] was, in milliseconds since [`STARTED`].
|
||||
static LAST_FRAME: AtomicU64 = AtomicU64::new(0);
|
||||
|
||||
fn since_start() -> u64 {
|
||||
STARTED.get_or_init(Instant::now).elapsed().as_millis() as u64
|
||||
}
|
||||
|
||||
/// The game drew a frame.
|
||||
pub fn frame() {
|
||||
LAST_FRAME.store(since_start(), Ordering::Relaxed);
|
||||
}
|
||||
|
||||
/// The game can still hold less. Said again every second while that is true, so it ends by itself
|
||||
/// when the world closes.
|
||||
pub fn can_shrink() {
|
||||
@@ -57,23 +42,18 @@ pub fn can_shrink() {
|
||||
/// the game can't get under the cap; only the first call's function is kept.
|
||||
pub fn set(limit: Option<u64>, exceeded: fn(u64, u64)) {
|
||||
LIMIT.store(limit.unwrap_or(NONE), Ordering::Relaxed);
|
||||
// A new cap gets the whole of STALL.
|
||||
frame();
|
||||
system_cap(limit);
|
||||
if limit.is_some() {
|
||||
WATCHER.call_once(|| {
|
||||
let watcher = std::thread::Builder::new()
|
||||
.name("Memory limit".into())
|
||||
.spawn(move || {
|
||||
let mut over = Over::default();
|
||||
let mut faults = Faults::default();
|
||||
loop {
|
||||
std::thread::sleep(EVERY);
|
||||
let per_second = faults.per_second();
|
||||
let limit = get();
|
||||
over.follow(limit);
|
||||
if let Some(limit) = limit {
|
||||
check(&mut over, per_second, limit, exceeded);
|
||||
check(&mut over, limit, exceeded);
|
||||
}
|
||||
}
|
||||
});
|
||||
@@ -101,7 +81,7 @@ pub enum Pressure {
|
||||
Steady,
|
||||
}
|
||||
|
||||
/// Against a cap nothing enforces: what the game holds. It should act before it gets there.
|
||||
/// What the game holds against the cap: it should act before it gets there.
|
||||
fn pressure_of_usage(used: u64, limit: u64) -> Pressure {
|
||||
if used > limit / 10 * 9 {
|
||||
Pressure::Over {
|
||||
@@ -114,25 +94,8 @@ fn pressure_of_usage(used: u64, limit: u64) -> Pressure {
|
||||
}
|
||||
}
|
||||
|
||||
/// Under a cap the system enforces the game sits at the cap whether it needs the memory or not, so
|
||||
/// usage says nothing: how often it has to fetch back what was moved out does. Measured on a real
|
||||
/// game loading chunks at 30 frames a second: 30,000-50,000 page faults a second with no cap or any
|
||||
/// cap down to 500 MB, 140,000 at 300 MB, 290,000 at 150 MB. Loading chunks fast faults a lot too,
|
||||
/// so `horizon_client::ram_limit` also asks whether frames are suffering.
|
||||
#[cfg_attr(not(windows), allow(dead_code))]
|
||||
fn pressure_of_faults(per_second: f64) -> Pressure {
|
||||
if per_second > 200_000. {
|
||||
Pressure::Over { keep: 0.8 }
|
||||
} else if per_second < 75_000. {
|
||||
Pressure::Room
|
||||
} else {
|
||||
Pressure::Steady
|
||||
}
|
||||
}
|
||||
|
||||
/// Against a hard ceiling, what the process commits is the budget: the working set sits at its
|
||||
/// cap whatever the game needs, and page faults only say how hard it pages, not how much it
|
||||
/// holds. A ceiling counts every page the process committed, the renderer's own included.
|
||||
/// Against a hard ceiling, what the process commits is the budget, not what stays in RAM: a
|
||||
/// ceiling counts every page the process committed, the renderer's own included.
|
||||
pub fn pressure_of_commit(commit: u64, limit: u64) -> Pressure {
|
||||
if commit > limit / 10 * 9 {
|
||||
Pressure::Over {
|
||||
@@ -145,27 +108,11 @@ pub fn pressure_of_commit(commit: u64, limit: u64) -> Pressure {
|
||||
}
|
||||
}
|
||||
|
||||
/// How tight memory is under `limit`. Reads a difference since the last call: call it once a
|
||||
/// second, from one place.
|
||||
/// How tight memory is under `limit`: what the game holds in RAM against it.
|
||||
pub fn pressure(limit: u64) -> Pressure {
|
||||
#[cfg(windows)]
|
||||
if SYSTEM_CAP.load(Ordering::Relaxed) {
|
||||
// The first reading has nothing to compare with.
|
||||
static FAULTS: std::sync::Mutex<Faults> = std::sync::Mutex::new(Faults::new());
|
||||
let per_second = FAULTS.lock().ok().and_then(|mut f| f.per_second());
|
||||
return per_second.map_or(Pressure::Steady, pressure_of_faults);
|
||||
}
|
||||
resident_bytes().map_or(Pressure::Steady, |used| pressure_of_usage(used, limit))
|
||||
}
|
||||
|
||||
/// Whether the system itself holds the game under the cap.
|
||||
pub fn system_enforced() -> bool {
|
||||
#[cfg(windows)]
|
||||
return SYSTEM_CAP.load(Ordering::Relaxed);
|
||||
#[cfg(not(windows))]
|
||||
false
|
||||
}
|
||||
|
||||
/// How long the game has been over the cap with nothing left to give up.
|
||||
#[derive(Default)]
|
||||
struct Over {
|
||||
@@ -192,12 +139,7 @@ impl Over {
|
||||
}
|
||||
}
|
||||
|
||||
/// Whether a game has frozen under a cap the system enforces.
|
||||
fn frozen(no_frame_for: Duration, faults_per_second: f64) -> bool {
|
||||
no_frame_for >= STALL && faults_per_second >= STALL_FAULTS
|
||||
}
|
||||
|
||||
fn check(state: &mut Over, faults: Option<f64>, limit: u64, exceeded: fn(u64, u64)) {
|
||||
fn check(state: &mut Over, limit: u64, exceeded: fn(u64, u64)) {
|
||||
// A hard ceiling counts what the process committed, the renderer's own
|
||||
// pages included, since that is the number the player set.
|
||||
let hard = crate::capped::limit().is_some();
|
||||
@@ -209,17 +151,6 @@ fn check(state: &mut Over, faults: Option<f64>, limit: u64, exceeded: fn(u64, u6
|
||||
let Some(mut used) = measured else {
|
||||
return;
|
||||
};
|
||||
// A cap the system enforces can't be gone over, so there the sign is a game
|
||||
// frozen by it. A hard ceiling can be gone over, and says so.
|
||||
if system_enforced() && !hard {
|
||||
let no_frame_for = since_start().saturating_sub(LAST_FRAME.load(Ordering::Relaxed));
|
||||
if frozen(Duration::from_millis(no_frame_for), faults.unwrap_or(0.)) {
|
||||
// The report has to be written, which a frozen game can't do under the cap.
|
||||
system_cap(None);
|
||||
exceeded(used, limit);
|
||||
}
|
||||
return;
|
||||
}
|
||||
if used > limit {
|
||||
crate::release_free_memory();
|
||||
used = if hard {
|
||||
@@ -234,47 +165,6 @@ fn check(state: &mut Over, faults: Option<f64>, limit: u64, exceeded: fn(u64, u6
|
||||
}
|
||||
}
|
||||
|
||||
/// Whether the system is enforcing the cap.
|
||||
#[cfg(windows)]
|
||||
static SYSTEM_CAP: std::sync::atomic::AtomicBool = std::sync::atomic::AtomicBool::new(false);
|
||||
|
||||
/// Sets the system's hard limit on the working set, or lifts it. Refused (a cap below what the
|
||||
/// system allows), the watcher treats the game as any other that can't get under its cap.
|
||||
#[cfg(windows)]
|
||||
fn system_cap(limit: Option<u64>) {
|
||||
use windows_sys::Win32::System::Memory::{
|
||||
QUOTA_LIMITS_HARDWS_MAX_DISABLE, QUOTA_LIMITS_HARDWS_MAX_ENABLE,
|
||||
QUOTA_LIMITS_HARDWS_MIN_DISABLE, SetProcessWorkingSetSizeEx,
|
||||
};
|
||||
use windows_sys::Win32::System::Threading::GetCurrentProcess;
|
||||
const MINIMUM: u64 = 16 << 20;
|
||||
// Windows' own defaults for a process, with the limit lifted.
|
||||
let (low, high, flags) = match limit {
|
||||
Some(limit) => (
|
||||
MINIMUM.min(limit / 2) as usize,
|
||||
usize::try_from(limit).unwrap_or(usize::MAX),
|
||||
QUOTA_LIMITS_HARDWS_MAX_ENABLE | QUOTA_LIMITS_HARDWS_MIN_DISABLE,
|
||||
),
|
||||
None => (
|
||||
200 << 10,
|
||||
1380 << 10,
|
||||
QUOTA_LIMITS_HARDWS_MAX_DISABLE | QUOTA_LIMITS_HARDWS_MIN_DISABLE,
|
||||
),
|
||||
};
|
||||
// SAFETY: passes plain values; the current process's pseudo-handle needs no closing.
|
||||
let done = unsafe { SetProcessWorkingSetSizeEx(GetCurrentProcess(), low, high, flags) != 0 };
|
||||
if !done {
|
||||
eprintln!(
|
||||
"Windows refused the memory limit: {}",
|
||||
std::io::Error::last_os_error()
|
||||
);
|
||||
}
|
||||
SYSTEM_CAP.store(done && limit.is_some(), Ordering::Relaxed);
|
||||
}
|
||||
|
||||
#[cfg(not(windows))]
|
||||
fn system_cap(_limit: Option<u64>) {}
|
||||
|
||||
/// The process's memory counters.
|
||||
#[cfg(windows)]
|
||||
fn counters() -> Option<windows_sys::Win32::System::ProcessStatus::PROCESS_MEMORY_COUNTERS> {
|
||||
@@ -292,35 +182,6 @@ fn counters() -> Option<windows_sys::Win32::System::ProcessStatus::PROCESS_MEMOR
|
||||
}
|
||||
}
|
||||
|
||||
/// Page fault rates, between one reading and the next.
|
||||
#[derive(Default)]
|
||||
struct Faults {
|
||||
#[cfg(windows)]
|
||||
last: Option<(Instant, u32)>,
|
||||
}
|
||||
|
||||
impl Faults {
|
||||
#[cfg(windows)]
|
||||
const fn new() -> Self {
|
||||
Faults { last: None }
|
||||
}
|
||||
|
||||
/// Page faults a second since the last call; none on the first, or where they aren't known.
|
||||
#[cfg(windows)]
|
||||
fn per_second(&mut self) -> Option<f64> {
|
||||
let faults = counters()?.PageFaultCount;
|
||||
let now = Instant::now();
|
||||
let before = self.last.replace((now, faults))?;
|
||||
let seconds = now.duration_since(before.0).as_secs_f64();
|
||||
(seconds > 0.).then(|| f64::from(faults.wrapping_sub(before.1)) / seconds)
|
||||
}
|
||||
|
||||
#[cfg(not(windows))]
|
||||
fn per_second(&mut self) -> Option<f64> {
|
||||
None
|
||||
}
|
||||
}
|
||||
|
||||
/// The memory this process holds in RAM, in bytes.
|
||||
#[cfg(windows)]
|
||||
pub fn resident_bytes() -> Option<u64> {
|
||||
@@ -400,40 +261,17 @@ mod tests {
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn page_faults_say_how_tight_a_cap_the_system_enforces_is() {
|
||||
assert_eq!(pressure_of_faults(50_000.), Pressure::Room);
|
||||
assert_eq!(pressure_of_faults(140_000.), Pressure::Steady);
|
||||
assert!(matches!(
|
||||
pressure_of_faults(290_000.),
|
||||
Pressure::Over { .. }
|
||||
));
|
||||
}
|
||||
|
||||
#[cfg(windows)]
|
||||
#[test]
|
||||
fn the_system_holds_the_game_under_a_cap_and_lifting_it_lets_memory_back() {
|
||||
fn every_system_steps_down_before_its_cap() {
|
||||
let _alone = WORKING_SET.lock().unwrap();
|
||||
const SIZE: usize = 256 << 20;
|
||||
const CAP: u64 = 128 << 20;
|
||||
let mut block = vec![1u8; SIZE];
|
||||
system_cap(Some(CAP));
|
||||
block.iter_mut().step_by(4096).for_each(|b| *b = 2);
|
||||
let capped = resident_bytes().unwrap();
|
||||
assert!(capped <= CAP, "{capped} bytes under a cap of {CAP}");
|
||||
system_cap(None);
|
||||
block.iter_mut().step_by(4096).for_each(|b| *b = 3);
|
||||
let lifted = resident_bytes().unwrap();
|
||||
std::hint::black_box(&block);
|
||||
assert!(lifted > CAP, "{lifted} bytes with the cap lifted");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn only_drawing_nothing_while_faulting_hard_counts_as_frozen() {
|
||||
assert!(frozen(STALL, STALL_FAULTS * 10.));
|
||||
// Slow but drawing, as at 1 frame a second.
|
||||
assert!(!frozen(Duration::from_secs(1), STALL_FAULTS * 10.));
|
||||
// Minimized or waiting: no frames, and no memory being fetched back.
|
||||
assert!(!frozen(STALL * 10, 100.));
|
||||
let used = resident_bytes().unwrap();
|
||||
// Under a cap at half what it holds, the game is told to hold less, on Windows as
|
||||
// elsewhere: nothing pages it out to meet the cap instead.
|
||||
let Pressure::Over { keep } = pressure(used / 2) else {
|
||||
panic!("holding {used} bytes is over a cap of half that");
|
||||
};
|
||||
assert!(keep < 0.5, "{keep}");
|
||||
// Far below the cap it may hold more again.
|
||||
assert_eq!(pressure(used.saturating_mul(4)), Pressure::Room);
|
||||
}
|
||||
|
||||
#[test]
|
||||
|
||||
@@ -245,14 +245,14 @@ presentation, so `bench` reports what is on screen.
|
||||
|
||||
### RAM limit
|
||||
|
||||
`HORIZON_MEMORY_LIMIT_MB` (the launcher sets it) and `/ramlimit`, `/ramlimit <MB>`, `/ramlimit off` limit
|
||||
the RAM the game holds; the command lasts until the game closes and works on any server, as the window
|
||||
answers it. On Windows the system enforces the limit with a hard working set limit, so memory the game
|
||||
uses least moves out of RAM and a low limit costs speed. If the game is also slow and faulting hard, far
|
||||
terrain goes off and the render distance in use drops, never the saved option, and both come back when
|
||||
there is room. Linux and macOS have no such limit: there the game steps down before it reaches the limit.
|
||||
The game ends with a crash report only when it can't run under the limit: on Windows no frame for a minute
|
||||
while faulting hard, elsewhere 20 seconds over the limit with nothing left to give up.
|
||||
`HORIZON_MEMORY_LIMIT_MB` (the launcher sets it when the player picks a value; there is none by default)
|
||||
and `/ramlimit`, `/ramlimit <MB>`, `/ramlimit off` limit the RAM the game holds; the command lasts until
|
||||
the game closes and works on any server, as the window answers it. Once a second the game compares what
|
||||
it holds in RAM with the limit and, past nine tenths of it, turns far terrain off and then lowers the
|
||||
render distance in use, never the saved option, until it fits; both come back when there is room. This
|
||||
is the same on Windows, Linux and macOS: nothing asks the system to page the game out to meet the limit.
|
||||
The game ends with a crash report only when it can't run under the limit: 20 seconds over it with nothing
|
||||
left to give up.
|
||||
|
||||
**`HORIZON_MEMORY_HARD_MB` / `/ramlimit hard <MB>`** is the other contract, the one `-Xmx` gives
|
||||
Java: a budget for the memory the process commits, not only for the part that stays in RAM. Nothing is
|
||||
|
||||
Reference in new issue
Block a user