mirror of
https://github.com/HeyPuter/gex.git
synced 2026-11-13 20:12:43 -10:00
Merge merge/w3 4b022cb7: Windows child processes without the session shared memory (read_session_data), delay imports, per-module CRT binding, childgui test
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_d2453757b8a84fdbb4ee7c9e093141df
This commit is contained in:
commit
4f0426370b
7 files changed
+181
-4
No files matched your search
+108
-3
@@ -1204,6 +1204,104 @@ crypt (no provider registered), pipe (the x64 architecture check).
|
||||
build tree must not mix objects from before and after a plugin change.
|
||||
- **No Wine fork patch** for GL.
|
||||
|
||||
**Child processes without the session's shared memory (2026-10-07).**
|
||||
The user's decision: no cross-process shared memory in gex-eject; Wine is
|
||||
patched instead. Scope, as found:
|
||||
- win32u reads the session in six places: the **user handle table**
|
||||
(`get_user_entry*`, `is_valid_entry`: every HWND check, IsWindow,
|
||||
GetWindowThreadProcessId, the handle-table walks), **window** objects
|
||||
(`get_shared_window`: class, DPI, wndproc, GetWindowLong's extra bytes),
|
||||
**class** objects (class.c: names, GetClassLong), the **desktop**
|
||||
(`get_shared_desktop`: async key state, cursor position and clip, the
|
||||
monitor serial), the **queue** (`get_shared_queue`: wake and changed
|
||||
bits, hook counts) and the thread's **input** (`get_shared_input`:
|
||||
foreground, active, focus, capture, caret, key state). user32 reads
|
||||
none directly.
|
||||
- Wine 11.15 still has requests for part of it (get_key_state for one key,
|
||||
get_queue_status, set_cursor's position and clip, get_window_info and
|
||||
get_class_info for one offset, get_window_rectangles), but the forms
|
||||
that returned the rest were cut when shared memory came in (9.0's
|
||||
get_key_state returned the whole key array, get_thread_input the
|
||||
foreground/focus/capture/caret handles, get_window_info the pid, tid and
|
||||
atom). A fallback per site would re-add those replies across six files.
|
||||
- So one generic request instead: `read_session_data(offset, size)`
|
||||
returns the bytes of the session mapping, and a child keeps a local copy
|
||||
laid out like the mapping; each read sequence refreshes the bytes it is
|
||||
about to read (a seqlock acquire: the object and up to 512 extra bytes; a
|
||||
user entry), so the seqlock check that follows passes as it would on the
|
||||
real mapping. ~60 lines of hand-written change in win32u, server and
|
||||
protocol.def (128 with the generated protocol files). The process hosting
|
||||
the server keeps its direct read.
|
||||
- Two more fork patches came with it: a child process is a copy of the
|
||||
first, in-process server state included, so `wineserver_session_data`
|
||||
answers only in the process the session was created in (a child got a
|
||||
frozen snapshot, and a seqlock caught odd spun for ever); and one
|
||||
desktop thread per session (a named mutant): with children able to read
|
||||
the session each started its own, and both waited for the other's shell
|
||||
window.
|
||||
- Toolchain fixes found on the way (merge/w3-boot): pelink no longer makes
|
||||
trapping stubs for clang's other `main` names (crt_main references
|
||||
them weakly; services.exe and wineboot.exe called a trap), gex defines
|
||||
them null in the PIC module; and a native module now loads its **delay
|
||||
imports** at load time (wineboot delay-imports user32 and gdi32: nothing
|
||||
loaded them, win32u's syscall table was missing in its process and
|
||||
every NtUser call returned STATUS_INVALID_SYSTEM_SERVICE).
|
||||
- More, found by the tests (fork, `wasm32-w3`): a child's seqlock wait
|
||||
refreshes its copy (a copy caught mid-write would spin for ever); a Unix
|
||||
call with no library handle returns STATUS_DLL_NOT_FOUND instead of
|
||||
trapping (wineps.drv's DllMain failure path, during DefaultInstall).
|
||||
- Tests: `tests/nt` **childgui** (user_win as a child: CreateProcess, wait;
|
||||
`GEX_NT_CHILDREN=1`, native Wine only) passes all 34 checks in the child,
|
||||
5 runs of 5. The launcher calls GetDesktopWindow first, as a GUI
|
||||
launcher has a desktop: the desktop thread then lives in the process
|
||||
hosting the server.
|
||||
- **Known limit: a desktop hosted in a child** (a console launcher whose
|
||||
first GUI is its child) is not reliable: about one run in three, a
|
||||
thread of the child's desktop process (the display settings restorer)
|
||||
holds the USER lock into a message callback (`user_check_not_lock`
|
||||
asserts) and the child's first window hangs. Not yet understood.
|
||||
- wineboot's full install with its steps as children
|
||||
(`$W/w3tests/install-as-children.sh`: the launcher runs `wineboot.exe
|
||||
-u`, which runs rundll32 for PreInstall and DefaultInstall) **completes
|
||||
the install**: the registry has the crypto providers (16 keys), and
|
||||
tests/nt `crypt` passes on that prefix (6 of 6). wineboot itself then
|
||||
does not finish in a child: after the install it calls into wininet
|
||||
(`initialize_internet`), and wininet's code (wininet+0x1d594) calls
|
||||
TlsGetValue millions of times, each through gex's function-table lookup
|
||||
(the probe cache never holds it: 3.5 million lookups of one address,
|
||||
`GEX_FTAB_TRACE=1`). InternetOpenW alone in a child works, so it is
|
||||
that state of wineboot's process; not yet found. The test is judged by
|
||||
the registry, not wineboot's exit. As the first process wineboot
|
||||
finishes.
|
||||
- **Fixed: wineboot's garbage wine.inf mtime** (5242148, so every first
|
||||
process reinstalled the prefix): pelink bound wineboot's `_wopen` to
|
||||
ucrtbase and its `_fstat32` to msvcrt, two descriptor tables; a module's
|
||||
calls now go to its own C runtime first. `.update-timestamp` now holds
|
||||
wine.inf's real mtime.
|
||||
- The registry-fixture idea is dropped.
|
||||
- Cost, on edger through bench.sh (native gexc; a Windows server call,
|
||||
GetExitCodeThread, is 74.5 us there): in a child, IsWindow 97.8 us (0.30
|
||||
us in the process hosting the server), GetAsyncKeyState 572 us (0.76 us:
|
||||
it refreshes the queue and the desktop object, ~1 KiB each), a frame
|
||||
polling all 256 keys 157 ms (0.19 ms). The transport alone, as a
|
||||
pipe-and-epoll round trip like a server request's: 24.7 us between
|
||||
threads natively and 45.7 us in the browser runtime (Node,
|
||||
`run-node.mjs`: the page's shim and glue, Workers as worker_threads);
|
||||
between processes 13.5 us natively, 27.8 us in the browser runtime.
|
||||
(`$W/rtt/bench-rtt.sh`, `$W/rtt/bench3.txt`.) Not yet measured in a
|
||||
Chromium page: W3 does not run in the page yet, and no page harness
|
||||
runs a wasm32 program by itself.
|
||||
- Who would feel it: only a **child** process. A game started directly is
|
||||
the process hosting the server and reads directly. A game started by a
|
||||
launcher (launcher.exe → game.exe, `start /wait`) would be a child: per
|
||||
frame, GetAsyncKeyState over many keys (Unity's and SDL's input polling,
|
||||
GameMaker's keyboard_check over the key range) costs ~0.6 ms a key, so a
|
||||
full 256-key poll is unplayable (157 ms) and a few keys a frame is
|
||||
~2-5 ms; IsWindow/GetWindowLong on hot paths ~0.1 ms each. Next steps
|
||||
if a launcher-started game needs it: refresh only the bytes a read uses
|
||||
(the key byte, not the object), and cache the desktop key state per
|
||||
keystate_serial.
|
||||
|
||||
#### Fork-patch inventory (W3, branch `wasm32-w3` over `wasm32`)
|
||||
|
||||
Every change is behind `__wasm32__` (or only built for it). What, why, and
|
||||
@@ -1229,8 +1327,12 @@ what would let it go:
|
||||
| user callbacks return through gex's pad | gex watches call-out returns by dispatch slot | never (port) |
|
||||
| **win32u reads the session from the in-process server** (`wineserver_session_data`) | a wasm memory cannot map the server's pages twice | coherent shared memory (below) |
|
||||
| **explorer's desktop as a thread** (`gexp_desktop_thread`, win32u starts it once, suspended so it knows the thread, which does not wait for itself; callers wait for the shell window; no shell in that thread; returns, no ExitProcess) | a second Windows process cannot see the session (§3.5) | coherent shared memory across gex processes, so explorer can be its own process again |
|
||||
| **a child process reads the session through requests** (`read_session_data`, server/mapping.c; win32u keeps a local copy laid out as the mapping, refreshed per seqlock acquire and per user handle entry; the handle-table walk skips entries the process has no object for) | a child has its own wasm memory and cannot map the session the server thread writes (user 2026-10-07: gex-eject never ships cross-process shared memory) | never under that decision; the process hosting the server keeps its direct read |
|
||||
| `wineserver_session_data` answers only in the process the session was created in | a child process starts as a copy of the first, server state included: it read a frozen snapshot | with the request path (never, under the no-shared-memory decision) |
|
||||
| one desktop thread per session (a named mutant held by its host process); the wait for its shell window is 60 s | with children reading the session, each process started a desktop thread of its own | explorer as its own process (needs cross-process shared memory) |
|
||||
| ntdll's Unix-call dispatcher returns STATUS_DLL_NOT_FOUND for a null library handle | a DLL whose Unix library is not linked into the closed-world program still makes calls (wineps.drv's DllMain failure path) and trapped | every such Unix library linked in |
|
||||
| winebuild (wasm32 export list): a renamed `-import` export is a forward | pelink resolves a call by the export list (user32's ClipCursor is win32u's NtUserClipCursor) | never (tool side of the port) |
|
||||
| win32u `kbd_tables_wchar_to_vkey` skips an SGCAPS pair's Caps Lock row | VkKeyScan swapped Shift for letters under winewayland (every letter an SGCAPS pair) | the fix upstream (it is not wasm32-specific): upstream-ready, ungated, in `docs/wine-upstream/0001-win32u-VkKeyScan-skip-SGCAPS-caps-row.patch`, **not submitted** (needs the user's OK) |
|
||||
| win32u `kbd_tables_wchar_to_vkey` skips an SGCAPS pair's Caps Lock row | VkKeyScan swapped Shift for letters under winewayland (every letter an SGCAPS pair) | the fix upstream (it is not wasm32-specific): **not being upstreamed (user 2026-10-07)**: kept as a fork patch; the ungated patch stays in `docs/wine-upstream/0001-win32u-VkKeyScan-skip-SGCAPS-caps-row.patch` for reference |
|
||||
|
||||
**Future option for the last two (and the server's 16 MiB block)**, not
|
||||
built: a SharedArrayBuffer-backed extra memory (wasm multi-memory) holding
|
||||
@@ -1743,10 +1845,13 @@ Heads:
|
||||
compositor (`$W/pfx-w3wl`). Output modes (Wine's EmulateModeset) and VkKeyScan done; W3.4 GL: opengl32 on gex-gl through winewayland, glcompat and glfast pass (34 of 40 on `$W/pfx-w3wl`). Wine fork head 74de38ed1ba (mw).
|
||||
**gate2 landed** (master 36f55bb6), and w3-gate1 (wp_viewporter, master
|
||||
1b4bd047, merged into merge/w3). **merge/w3 gate head** sent after
|
||||
`$W/check-w3-gate.sh` (output `$W/w3chk.out`): crates green, elf-prov 5/5
|
||||
`$W/check-w3-gate.sh` (output `$W/w3chk.out`; the child-session head on
|
||||
master ac269692: `$W/w3chk2.out`, the same counts plus childgui and the
|
||||
install-as-children leg): crates green, elf-prov 5/5
|
||||
native and in the browser, x86-programs 82, nt --no-cgen 83/83 on x86 and
|
||||
pfx-gui, cgen-prefix 83/83, wasm32 4/4 + busybox both, browser nt 49/49,
|
||||
W3 i386 32 of 40 (null driver) and 34 of 40 (Wayland + GL). The old merge/elf-prov-gate is deleted. To
|
||||
W3 i386 32 of 40 (null driver; 33 of 41 with childgui) and 34 of 40
|
||||
(Wayland + GL); wineboot's install as children leaves the full registry. The old merge/elf-prov-gate is deleted. To
|
||||
rebuild after new dev commits: same path split (`git checkout <dev
|
||||
head> -- <paths>` per commit), rerun the checks, merge master once
|
||||
integ-g lands, send the head.
|
||||
|
||||
@@ -344,6 +344,28 @@ fn instantiate(
|
||||
}
|
||||
}
|
||||
jit_stats::INSTALLS.fetch_add(1, std::sync::atomic::Ordering::Relaxed);
|
||||
// clang's three names for a C main (`main`, `__main_argc_argv`,
|
||||
// `__original_main`): Wine's crt_main references all of them weakly and a
|
||||
// program defines one. In a position-independent module the others stay
|
||||
// imports, their address through `GOT.func`: null, as a static link
|
||||
// leaves them, and the direct calls behind the null check a trap.
|
||||
const MAIN_NAMES: [&str; 3] = ["main", "__main_argc_argv", "__original_main"];
|
||||
for imp in module.imports() {
|
||||
let (m, name) = (imp.module(), imp.name());
|
||||
if !MAIN_NAMES.contains(&name) || l.get(&mut c, m, name).is_ok() {
|
||||
continue;
|
||||
}
|
||||
if m == "GOT.func" {
|
||||
let g = i32c(&mut c, 0u64, wasmtime::Mutability::Var)?;
|
||||
l.define(&mut c, m, name, g)?;
|
||||
} else if let (true, wasmtime::ExternType::Func(ft)) = (m == f::PIC_MODULE, imp.ty()) {
|
||||
let label = name.to_string();
|
||||
let func = wasmtime::Func::new(&mut c, ft, move |_, _, _| {
|
||||
Err(wasmtime::Error::msg(format!("{label}: a main the program does not define was called")))
|
||||
});
|
||||
l.define(&mut c, m, name, func)?;
|
||||
}
|
||||
}
|
||||
let inst = l.instantiate(&mut c, module).context("instantiating a function-table module")?;
|
||||
if let Some(x) = stateful {
|
||||
crate::libprov::relocate_once(&mut c, &inst, x)?;
|
||||
|
||||
Executable
BIN
Binary file not shown.
Executable
BIN
Binary file not shown.
@@ -108,6 +108,10 @@ for arch in x64 x86; do
|
||||
check $arch/icall 0 "icall: 1284803844 -1238658015 2 3 4" "$b/icall.exe"
|
||||
# N4: USER without a display.
|
||||
check $arch/user_win 0 "$allok" "$b/user_win.exe"
|
||||
# The same as a child process (CreateProcess, waited for): native Wine
|
||||
# (W3) only, where a child reads the session through requests; the NT
|
||||
# layer has no child processes. GEX_NT_CHILDREN=1 runs it.
|
||||
[ -n "${GEX_NT_CHILDREN:-}" ] && check $arch/childgui 0 "${allok}${nl}child exit 0" "$b/childgui.exe"
|
||||
check $arch/gdi 0 "$allok" "$b/gdi.exe"
|
||||
check $arch/input 0 "$allok" "$b/input.exe"
|
||||
check $arch/filemap 0 "$allok" "$b/filemap.exe"
|
||||
|
||||
@@ -0,0 +1,27 @@
|
||||
/* A launcher: a GUI test (user_win.exe, beside this one) as a child process,
|
||||
* waited for, its exit code passed on. Under native Wine on gex (W3) the
|
||||
* child cannot map the session's shared memory and reads it through
|
||||
* wineserver requests (docs/wine-native.md). Its stdout is inherited.
|
||||
* As a GUI launcher is, this one is a user32 program that has the desktop
|
||||
* before it starts the child (GetDesktopWindow): the desktop then lives in
|
||||
* the process hosting the server.
|
||||
* gex-nt-cc: clang --target=ARCH-w64-mingw32 -fuse-ld=lld */
|
||||
#include <windows.h>
|
||||
#include <stdio.h>
|
||||
int main(int argc, char **argv) {
|
||||
char cmd[MAX_PATH + 16];
|
||||
STARTUPINFOA si = {sizeof si};
|
||||
PROCESS_INFORMATION pi;
|
||||
DWORD code = 99;
|
||||
snprintf(cmd, sizeof cmd, "%s", argc > 1 ? argv[1] : "user_win.exe");
|
||||
GetDesktopWindow();
|
||||
fflush(stdout);
|
||||
if (!CreateProcessA(0, cmd, 0, 0, TRUE, 0, 0, 0, &si, &pi)) {
|
||||
printf("FAIL CreateProcess %lu\n", GetLastError());
|
||||
return 98;
|
||||
}
|
||||
WaitForSingleObject(pi.hProcess, INFINITE);
|
||||
GetExitCodeProcess(pi.hProcess, &code);
|
||||
printf("child exit %lu\n", code);
|
||||
return code;
|
||||
}
|
||||
@@ -143,6 +143,12 @@ def main():
|
||||
is_crt = lambda n: n.startswith(('msvcr', 'ucrt')) or n == 'crtdll'
|
||||
crt = next((i for i in eager if is_crt(i)), None if is_crt(mod.split('.')[0]) else 'ucrtbase')
|
||||
eager += [c for c in (crt, 'kernel32', 'ntdll') if c and c not in eager]
|
||||
# and its delay imports, after them: a native module's calls are
|
||||
# bound in the one module, so nothing would load a delay-loaded
|
||||
# DLL on first call, and a DLL never loaded never ran its DllMain
|
||||
# (wineboot's user32 and gdi32: win32u's syscall table missing,
|
||||
# every NtUser call STATUS_INVALID_SYSTEM_SERVICE)
|
||||
eager += [i.lstrip('-') for i in makefile_var(mk, 'DELAYIMPORTS') if i.lstrip('-') not in eager]
|
||||
EAGER[mod] = [i if '.' in i else i + '.dll' for i in eager]
|
||||
if not dlls:
|
||||
sys.exit('no DLL objects found (build with winegcc for wasm32)')
|
||||
@@ -255,7 +261,11 @@ def link(dlls, out, gexrt, others=None):
|
||||
owners = [o for o in owners if (name, 'defined') in sigs[o]] or owners
|
||||
if mod in owners:
|
||||
continue # its own: already resolved inside it
|
||||
pick = next((o for o in imports if o in owners), owners[0])
|
||||
# (its own C runtime first, which its import list does not
|
||||
# name: ucrtbase's _wopen and msvcrt's _fstat32 are two fd
|
||||
# tables, and wineboot's fstat of a file it had just opened
|
||||
# failed)
|
||||
pick = next((o for o in EAGER.get(mod, []) + imports if o in owners), owners[0])
|
||||
# a caller declaring the function otherwise than its
|
||||
# definition (an adapter from the .spec: user32's
|
||||
# NotifyWinEvent is void, win32u's stub returns a word) goes
|
||||
@@ -413,6 +423,15 @@ def link(dlls, out, gexrt, others=None):
|
||||
for (n, k), v in sigs[mod].items():
|
||||
if k == 'undefined' and n in missing:
|
||||
callers.setdefault(n, v)
|
||||
# Only weakly referenced (msvcrt's crt_main asks whether the program
|
||||
# defined __main_argc_argv or __original_main): left undefined, so the
|
||||
# reference stays null. A trapping stub would make it non-null, and a
|
||||
# program with the other form of main would call the trap.
|
||||
strong = set()
|
||||
for mod, ro, ex, imports in dlls:
|
||||
nm = subprocess.run(['llvm-nm', '--undefined-only', ro], capture_output=True, text=True).stdout
|
||||
strong |= {l.split()[-1] for l in nm.splitlines() if l.split()[:1] == ['U']}
|
||||
callers = {n: v for n, v in callers.items() if n in strong}
|
||||
gaps = sorted(callers.items())
|
||||
if gaps:
|
||||
ctype = {'i': 'int', 'I': 'long long', 'f': 'float', 'd': 'double', 'v': 'void'}
|
||||
|
||||
Reference in new issue
Block a user