Function JIT: what it learns is kept per guest process

The native runner hosts a whole process tree (a shell, Xwayland, the game
it starts) with one run-time translator, and the function JIT's learned
state was global: builtins by address (memcpy and friends, replaced on every
execve), function starts, never-returning functions, translated entries.
Images of different processes share addresses, not code, so one process's
lesson was applied to another's code. The likely cause of Hollow Knight's
graphics-only --fjit crashes: with worldmachine's glibc 2.44 its ld.so
returned through a smashed frame in _dl_lookup_symbol_x in 4 of 4 graphics
runs (Xwayland and its children, on Debian's ld.so at the same addresses,
share the translator); batch runs, one process, never crashed. Confirmation
runs pending.

Each process half (ftab::Shared) now carries a Process id, unique for the
run and shared with a forked child until it execs; ftab::Request carries it,
Translator::image registers a process's program when its JIT half is made
(replacing the global exec hook and gexc's one-time add_image), and
Translator::forget drops it with the last reference. Dumps
(GEX_FJIT_DUMP/KEEP) are named p<process>-<addr>.

tests/fjit-procs: a parent and the image its child execs, where the child
has a builtin-named function at the address of the parent's probe; fails on
the old gexc ("after probe: 6", memset's return), passes now.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_461fe3ff06374ecba693831148697ba6
This commit is contained in:
velzieandClaude Opus 5.5 committed 2026-10-05 22:43:55 -04:00
1 parent 785372cf52
commit 763fc74c98
8 files changed
+266 -53

No files matched your search

+12 -4
View File
@@ -194,6 +194,11 @@ impl FuncMap {
/// What the host asks a run-time translator (the function JIT, stream F)
/// for when a lookup finds no function at an address.
pub struct Request<'a> {
/// The guest process asking: what a translator learns of one process's
/// code must not be applied to another's (one native runner hosts a whole
/// process tree, whose images share addresses). Unique for the run; a
/// forked child shares its parent's until it `execve`s.
pub process: u64,
/// Guest memory, as the guest sees it.
pub mem: &'a [u8],
/// The address looked up: a function entry, as far as the caller knows.
@@ -229,10 +234,13 @@ pub struct Translated {
/// ([`gex_abi::ftab::FALLBACK`]: tier-1, until the region tier).
pub trait Translator: Send + Sync {
fn translate(&self, req: &Request<'_>) -> Option<Translated>;
/// The process `execve`d `program` (the file `bytes`): what was learned
/// of the image it replaced -- function starts, read-only ranges,
/// builtins by address -- describes addresses the new image now holds.
fn exec(&self, _program: &gex_ld::Program, _bytes: &[u8]) {}
/// Guest process `process` starts running `program` (the file `bytes`,
/// when known): a new process, or one that `execve`d. Whatever was
/// learned of `process`'s old image -- function starts, read-only
/// ranges, builtins by address -- describes addresses the new one holds.
fn image(&self, _process: u64, _program: &gex_ld::Program, _bytes: Option<&[u8]>) {}
/// Guest process `process` is gone (its last reference dropped).
fn forget(&self, _process: u64) {}
}
static TRANSLATOR: std::sync::OnceLock<Box<dyn Translator>> = std::sync::OnceLock::new();
+82 -39
View File
@@ -9,7 +9,10 @@
//! module, the calls among them direct `call`s, summaries over the batch
//! (callees outside it count as unknown). What recovery learns -- function
//! starts, which functions never return, what is translated -- is kept for
//! the next request.
//! the next request, per guest process: the native runner keeps one
//! translator for every process in the tree (a shell, Xwayland and the game
//! it starts), whose images sit at the same addresses with different code, so
//! what one process's code taught must never be applied to another's.
use crate::func::{self, Code, RecoverOptions, StartInfo};
use crate::intrinsic::Intrinsic;
@@ -31,14 +34,14 @@ fn batch_limit() -> usize {
#[derive(Default)]
pub struct Jit {
st: Mutex<State>,
/// Entries tier-1 serves with a host builtin (`memcpy` and friends, by
/// symbol): their functions are the builtin, as in tier-1.
/// Replaced on `execve` ([`Jit::exec`]).
intrinsics: std::sync::RwLock<HashMap<u64, Intrinsic>>,
}
/// What recovery learned of one process's code ([`Jit::image`] starts it over).
#[derive(Default)]
struct State {
struct Learned {
/// Entries tier-1 serves with a host builtin (`memcpy` and friends, by
/// symbol): their functions are the builtin, as in tier-1.
intrinsics: HashMap<u64, Intrinsic>,
/// Every function start seen: a call or jump to one is a call or tail
/// call, and bounds the functions around it.
starts: BTreeMap<u64, StartInfo>,
@@ -50,6 +53,12 @@ struct State {
/// tracked past `mprotect`: data the guest makes writable later counts
/// as read-only still.)
readonly: Vec<(u64, u64)>,
}
#[derive(Default)]
struct State {
/// By process ([`Jit::translate_in`]'s `process`; 0 for a runner with one).
procs: HashMap<u64, Learned>,
/// For `GEX_FJIT_STATS`: translations, functions, time in recovery and
/// emission, module bytes.
modules: u64,
@@ -61,7 +70,22 @@ struct State {
impl Jit {
pub fn new(intrinsics: HashMap<u64, Intrinsic>) -> Jit {
Jit { intrinsics: std::sync::RwLock::new(intrinsics), ..Jit::default() }
let jit = Jit::default();
jit.st.lock().unwrap_or_else(|e| e.into_inner()).procs.entry(0).or_default().intrinsics = intrinsics;
jit
}
/// Process `process` runs a new program (`image`, the file `elf`): it
/// starts with what the loader knows of that image and nothing else --
/// a new process, or an `execve` over an old one.
pub fn image(&self, process: u64, intrinsics: HashMap<u64, Intrinsic>, image: &gex_ld::Image, elf: Option<&[u8]>) {
self.st.lock().unwrap_or_else(|e| e.into_inner()).procs.insert(process, Learned { intrinsics, ..Learned::default() });
self.add_image_in(process, image, elf);
}
/// Process `process` is gone.
pub fn forget(&self, process: u64) {
self.st.lock().unwrap_or_else(|e| e.into_inner()).procs.remove(&process);
}
/// The process replaced its program (`execve`): forget everything
@@ -70,13 +94,7 @@ impl Jit {
/// image now holds (a PIE replacing a PIE lands at the same base) --
/// and start over from `image`.
pub fn exec(&self, intrinsics: HashMap<u64, Intrinsic>, image: &gex_ld::Image, elf: Option<&[u8]>) {
*self.intrinsics.write().unwrap_or_else(|e| e.into_inner()) = intrinsics;
{
let mut st = self.st.lock().unwrap_or_else(|e| e.into_inner());
let State { modules, functions, recover_us, emit_us, bytes, .. } = std::mem::take(&mut *st);
*st = State { modules, functions, recover_us, emit_us, bytes, ..State::default() };
}
self.add_image(image, elf);
self.image(0, intrinsics, image, elf);
}
/// What the loader knows of an image the program maps: its function
@@ -84,9 +102,14 @@ impl Jit {
/// file, for `.eh_frame` and ELF32 symbols), with their extents, which
/// bound functions and computed-goto tables as they do ahead of time.
pub fn add_image(&self, image: &gex_ld::Image, elf: Option<&[u8]>) {
self.add_image_in(0, image, elf);
}
fn add_image_in(&self, process: u64, image: &gex_ld::Image, elf: Option<&[u8]>) {
let code = Code::with_elf(image, elf);
let (starts, _) = func::seeds(&code, elf, false);
let mut st = self.st.lock().unwrap_or_else(|e| e.into_inner());
let mut guard = self.st.lock().unwrap_or_else(|e| e.into_inner());
let st = guard.procs.entry(process).or_default();
for (a, info) in starts {
if info.strong() {
st.starts.insert(a, info);
@@ -113,6 +136,23 @@ impl Jit {
volatile: &dyn Fn(u64) -> bool,
page_gen: &dyn Fn(u64) -> u32,
class32: bool,
) -> Option<Vec<u8>> {
self.translate_in(0, mem, addr, executable, read_only, volatile, page_gen, class32)
}
/// [`Jit::translate`] for guest process `process`, whose learned state
/// ([`Jit::image`]) is its own.
#[allow(clippy::too_many_arguments)]
pub fn translate_in(
&self,
process: u64,
mem: &[u8],
addr: u64,
executable: &dyn Fn(u64) -> bool,
read_only: &dyn Fn(u64, u64) -> bool,
volatile: &dyn Fn(u64) -> bool,
page_gen: &dyn Fn(u64) -> u32,
class32: bool,
) -> Option<Vec<u8>> {
if gex_abi::PPC || (class32 && !gex_abi::X86) {
return None;
@@ -124,8 +164,10 @@ impl Jit {
if std::env::var("GEX_FJIT_STATS").as_deref() == Ok("2") {
eprintln!("[fjit] translate {addr:#x}: bytes {:02x?}", &mem[addr as usize..addr as usize + 16]);
}
let intrinsics = self.intrinsics.read().unwrap_or_else(|e| e.into_inner());
let mut st = self.st.lock().unwrap_or_else(|e| e.into_inner());
let mut guard = self.st.lock().unwrap_or_else(|e| e.into_inner());
let State { procs, modules, functions: n_functions, recover_us, emit_us, bytes } = &mut *guard;
let st = procs.entry(process).or_default();
let intrinsics = st.intrinsics.clone();
let readonly = st.readonly.clone();
let ro = |a: u64, n: u64| readonly.iter().any(|&(lo, hi)| a >= lo && a.saturating_add(n) <= hi) || read_only(a, n);
let code = Code::memory(mem, executable, &ro);
@@ -141,7 +183,7 @@ impl Jit {
if batch.contains_key(&a) || (a != addr && st.translated.contains(&a)) || !executable(a) {
continue;
}
let State { starts, noreturn, .. } = &mut *st;
let Learned { starts, noreturn, .. } = &mut *st;
let f = func::recover_one(&code, a, starts, noreturn, &opts);
if let Err(e) = f.verify().and_then(|_| block_local(&f)) {
if std::env::var_os("GEX_FJIT_STATS").is_some() {
@@ -223,7 +265,7 @@ impl Jit {
// The batch as GIR text (`gex_ir::text`), to replay offline.
let _ = std::fs::create_dir_all(&dir);
let text: String = functions.iter().map(gex_ir::text::print).collect::<Vec<_>>().join("\n");
let _ = std::fs::write(format!("{dir}/{addr:x}.gir"), text);
let _ = std::fs::write(format!("{dir}/p{process}-{addr:x}.gir"), text);
}
// A bug in the emitter costs the batch its translation, not the
// process: the fallback runs it.
@@ -243,7 +285,7 @@ impl Jit {
st.translated.extend(entries);
if let Ok(dir) = std::env::var("GEX_FJIT_KEEP") {
let _ = std::fs::create_dir_all(&dir);
let _ = std::fs::write(format!("{dir}/{addr:x}.wasm"), &wasm);
let _ = std::fs::write(format!("{dir}/p{process}-{addr:x}.wasm"), &wasm);
}
Some(wasm)
}
@@ -258,13 +300,13 @@ impl Jit {
None
}
};
st.modules += 1;
st.functions += n as u64;
st.recover_us += recovered;
st.emit_us += now_us().saturating_sub(started).saturating_sub(recovered);
st.bytes += out.as_ref().map_or(0, |w| w.len() as u64);
if std::env::var_os("GEX_FJIT_STATS").is_some() && st.modules.is_power_of_two() {
eprintln!("[fjit] {}", st.line());
*modules += 1;
*n_functions += n as u64;
*recover_us += recovered;
*emit_us += now_us().saturating_sub(started).saturating_sub(recovered);
*bytes += out.as_ref().map_or(0, |w| w.len() as u64);
if std::env::var_os("GEX_FJIT_STATS").is_some() && modules.is_power_of_two() {
eprintln!("[fjit] {}", stats_line(*modules, *n_functions, *recover_us, *emit_us, *bytes));
}
out
}
@@ -342,21 +384,22 @@ pub use gex_cgen::func::ret_pops;
impl State {
fn line(&self) -> String {
let n = self.functions.max(1);
format!(
"{} modules, {} functions; recovery {:.1} ms ({} us/function), emission {:.1} ms ({} us/function), {} wasm bytes ({}/function)",
self.modules,
self.functions,
self.recover_us as f64 / 1e3,
self.recover_us / n,
self.emit_us as f64 / 1e3,
self.emit_us / n,
self.bytes,
self.bytes / n
)
stats_line(self.modules, self.functions, self.recover_us, self.emit_us, self.bytes)
}
}
fn stats_line(modules: u64, functions: u64, recover_us: u64, emit_us: u64, bytes: u64) -> String {
let n = functions.max(1);
format!(
"{modules} modules, {functions} functions; recovery {:.1} ms ({} us/function), emission {:.1} ms ({} us/function), {bytes} wasm bytes ({}/function)",
recover_us as f64 / 1e3,
recover_us / n,
emit_us as f64 / 1e3,
emit_us / n,
bytes / n
)
}
/// Microseconds since the first call, for the translation statistics. 0 on
/// wasm32-unknown-unknown, where `std::time::Instant` panics (the browser's
+4
View File
@@ -61,6 +61,9 @@ pub struct Options {
/// behind them, so the translator emits that instead of lifting the
/// function's own code. See [`intrinsic`].
pub intrinsics: HashMap<u64, Intrinsic>,
/// The file the program was loaded from, for the function JIT's view of
/// its image (`.eh_frame` function starts): set per process.
pub image_elf: Option<std::sync::Arc<[u8]>>,
/// Extra guest addresses to treat as block entry points.
pub extra_entries: Vec<u64>,
/// Scan data for values that look like code pointers.
@@ -139,6 +142,7 @@ impl Default for Options {
Options {
limits: BlockLimits::default(),
intrinsics: HashMap::new(),
image_elf: None,
extra_entries: Vec::new(),
scan_pointers: true,
// Off by default now that runtime translation covers the gaps;
+11 -6
View File
@@ -538,12 +538,16 @@ struct FjitTranslator(gex_xlate::fjit::Jit);
impl gex_kernel::ftab::Translator for FjitTranslator {
fn translate(&self, req: &gex_kernel::ftab::Request<'_>) -> Option<gex_kernel::ftab::Translated> {
self.0
.translate(req.mem, req.addr, req.executable, req.read_only, req.volatile, req.page_gen, req.class32)
.translate_in(req.process, req.mem, req.addr, req.executable, req.read_only, req.volatile, req.page_gen, req.class32)
.map(|wasm| gex_kernel::ftab::Translated { wasm })
}
fn exec(&self, program: &gex_ld::Program, bytes: &[u8]) {
self.0.exec(gex_xlate::intrinsic::program_intrinsics(program), &program.exe, Some(bytes));
fn image(&self, process: u64, program: &gex_ld::Program, bytes: Option<&[u8]>) {
self.0.image(process, gex_xlate::intrinsic::program_intrinsics(program), &program.exe, bytes);
}
fn forget(&self, process: u64) {
self.0.forget(process);
}
}
@@ -669,9 +673,10 @@ fn cmd_run(args: &[String]) -> Result<ExitCode> {
.with_context(|| format!("loading {}", ra.path))?;
ra.xlate.intrinsics = gex_xlate::intrinsic::program_intrinsics(&program);
if ra.xlate.no_blocks {
let jit = gex_xlate::fjit::Jit::new(ra.xlate.intrinsics.clone());
jit.add_image(&program.exe, Some(&bytes));
gex_kernel::ftab::set_translator(Box::new(FjitTranslator(jit)));
// The program's image is registered per process when it starts
// (`Translator::image`, from the native runner).
gex_kernel::ftab::set_translator(Box::new(FjitTranslator(gex_xlate::fjit::Jit::new(Default::default()))));
ra.xlate.image_elf = Some(std::sync::Arc::from(bytes.as_slice()));
}
// argv[0] is the path as invoked, then whatever followed. Under a sysroot
+30 -1
View File
@@ -49,6 +49,29 @@ pub(crate) struct Shared {
/// overwrites ~120 lua51.dll entry points; pewasm, whose lua51 is
/// native too, ignores those writes the same way.
pub providers: std::collections::HashSet<u32>,
/// Which guest process this is, for the run-time translator
/// (`ftab::Request::process`): one per process image, shared with a
/// forked child until it `execve`s.
pub process: Process,
}
/// A guest process's identity for the run-time translator, unique for the
/// run; the translator forgets the process when the last reference goes.
pub(crate) struct Process(pub u64);
impl Default for Process {
fn default() -> Process {
static NEXT: std::sync::atomic::AtomicU64 = std::sync::atomic::AtomicU64::new(1);
Process(NEXT.fetch_add(1, std::sync::atomic::Ordering::Relaxed))
}
}
impl Drop for Process {
fn drop(&mut self) {
if let Some(t) = gex_kernel::ftab::translator() {
t.forget(self.0);
}
}
}
/// One store's half, inside [`Jit`].
@@ -637,6 +660,7 @@ pub(crate) fn background() -> bool {
/// can do it: the mappings as they are now, and the retirement epoch they
/// belong to.
struct Job {
process: u64,
sh: Arc<Mutex<JitShared>>,
memory: gex_mem::GuestMemory,
engine: wasmtime::Engine,
@@ -686,8 +710,12 @@ fn job(caller: &mut Caller<'_, HostState>, sh: &Arc<Mutex<JitShared>>, memory: &
if !exec.iter().any(|&(s, e)| addr >= s && addr < e) {
return None;
}
let epoch = sh.lock().unwrap_or_else(|e| e.into_inner()).ftab.epoch;
let (epoch, process) = {
let s = sh.lock().unwrap_or_else(|e| e.into_inner());
(s.ftab.epoch, s.ftab.process.0)
};
Some(Job {
process,
sh: Arc::clone(sh),
memory: memory.clone(),
engine: caller.engine().clone(),
@@ -725,6 +753,7 @@ fn run_job(j: &Job) -> Result<bool> {
g
};
let req = gex_kernel::ftab::Request {
process: j.process,
mem: memory.bytes(),
addr,
executable: &executable,
+12 -3
View File
@@ -1104,6 +1104,14 @@ fn run_program_with(
limits: xlate.limits,
ftab: None,
});
// (An NT program's images are the NT layer's to describe: as before,
// the translator starts from nothing for it.)
if let Some(t) = gex_kernel::ftab::translator().filter(|_| nt_setup.is_none()) {
let process = store.data().jit.as_ref().map(|j| j.shared.lock().unwrap_or_else(|e| e.into_inner()).ftab.process.0);
if let Some(process) = process {
t.image(process, program, xlate.image_elf.as_deref());
}
}
}
if !aot_shared.is_empty() {
aot::install_shared(&mut store, aot_shared)?;
@@ -2074,9 +2082,10 @@ fn exec_program(
// one holds now.
let mut xlate = xlate.clone();
xlate.intrinsics = gex_xlate::intrinsic::program_intrinsics(&program);
if let Some(t) = gex_kernel::ftab::translator() {
t.exec(&program, &bytes);
}
// The run-time translator hears of the new image when its process half
// is made (`Translator::image`, in `run_program_with`): a new identity,
// so nothing learned of the old image, or of another process, applies.
xlate.image_elf = Some(Arc::from(bytes.as_slice()));
// Descriptors marked close-on-exec go now; the rest survive, which is how
// a shell hands a pipe to the command it starts.
kernel.close_on_exec();
+89
View File
@@ -0,0 +1,89 @@
/* Two images at the same addresses under one function JIT (run.sh).
*
* Built twice from this file: the parent image (no CHILD) and the child image
* (-DCHILD). The first function each defines, after the same C runtime and
* page-aligned, sits at the same address in both: the parent's `probe`, which
* returns a value, and the child's `__memset_gexprobe`, a name the function
* JIT replaces with a host memset; the child's `probe` never returns. The
* parent forks, the child execs the child image and runs. Only then does the
* parent translate `after`, which calls its own `probe` and prints the result.
* A translator that applied what it learned of the child image (builtins by
* address, functions that never return) to the parent gets it wrong. */
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <sys/wait.h>
#ifdef CHILD
/* A name the function JIT serves with a host builtin (gex_xlate::intrinsic:
* glibc's private `__memset_*` shape), at the address of the parent's
* `probe`. */
__attribute__((noinline, aligned(4096))) void *__memset_gexprobe(void *d, int c, unsigned long n)
{
volatile unsigned char *p = d;
while (n--)
*p++ = (unsigned char)c;
return d;
}
__attribute__((noinline, noreturn)) void probe(int x)
{
char buf[16];
__memset_gexprobe(buf, 'a', sizeof buf);
if (x >= 0 && buf[15] == 'a')
_exit(x & 7);
for (;;)
;
}
#else
__attribute__((noinline, aligned(4096))) int probe(int x)
{
volatile int y = x * 3 + 1;
return y;
}
#endif
__attribute__((noinline)) int after(int x)
{
#ifdef CHILD
probe(x);
#else
int r = probe(x);
printf("after probe: %d\n", r + 1);
return r;
#endif
}
int main(int argc, char **argv)
{
/* Never true: keeps the read-only data the same in both images. */
if (argc == 12345)
printf("after probe: %d\n", argc);
if (argc > 1 && !strcmp(argv[1], "child")) {
printf("child running\n");
fflush(stdout);
int (*volatile later)(int) = after;
later(argc + 1);
return 1;
}
if (argc < 2) {
fprintf(stderr, "usage: procs CHILD-IMAGE\n");
return 2;
}
fflush(stdout);
pid_t p = fork();
if (p == 0) {
execl(argv[1], argv[1], "child", (char *)0);
_exit(99);
}
int st = 0;
waitpid(p, &st, 0);
printf("child exit %d\n", WIFEXITED(st) ? WEXITSTATUS(st) : -1);
/* Through a pointer: a direct call would put `after` and `probe` in the
batch translated with `main`, before the child ran. */
int (*volatile later)(int) = after;
int r = later(5);
printf("parent done %d\n", r);
return 0;
}
+26
View File
@@ -0,0 +1,26 @@
#!/bin/sh
# The function JIT keeps what it learns per guest process: a parent and the
# image its child execs share addresses, not code (procs.c). Output under
# gexc --fjit must match native.
#
# tests/fjit-procs/run.sh [gexc] (needs zig for the musl target)
set -u
HERE=$(cd "$(dirname "$0")" && pwd)
GEXC=${1:-target/x86/release/gexc}
W=${WORK:-$(mktemp -d)}
mkdir -p "$W"
zig cc -target x86_64-linux-musl -O1 -static -o "$W/procs" "$HERE/procs.c" || exit 1
zig cc -target x86_64-linux-musl -O1 -static -DCHILD -o "$W/procs-child" "$HERE/procs.c" || exit 1
pa=$(nm "$W/procs" | awk '$3=="probe"{print $1}'); ca=$(nm "$W/procs-child" | awk '$3=="__memset_gexprobe"{print $1}')
[ "$pa" = "$ca" ] || { echo "FAIL: probe at $pa in the parent but __memset_gexprobe at $ca in the child (the test needs them equal)"; exit 1; }
native=$("$W/procs" "$W/procs-child")
fail=0
for flags in "" "--fjit"; do
gex=$("$GEXC" run $flags "$W/procs" "$W/procs-child" 2>&1)
if [ "$native" = "$gex" ]; then
echo "ok procs ${flags:-t1}"
else
echo "FAIL procs ${flags:-t1}"; printf 'native:\n%s\ngex:\n%s\n' "$native" "$gex"; fail=1
fi
done
exit $fail