FEX/ThunkLibs
Ryan Houdek 180d16af7a Thunks: Fixes missing vulkan definitions
A couple of games were hitting these. Not sure how they were missed in
PR #3159 but adds the missing one.

Small rearrangement to make this easier as well. Hopefully thunk stuff
lands sooner rather than later to automate this for Vulkan.

Maybe `-isystem` instead of `-I` needs to be used unlike what #2076,
might depend on what is installed on the host system.
2023-10-11 01:49:42 -07:00
..
Generator unittests/ThunkLibs: Specify clang resource directory when compiling test code 2023-10-02 22:18:23 +02:00
GuestLibs Merge pull request #3176 from neobrain/fix_thunks_unused_artifacts 2023-10-04 07:07:18 -07:00
HostLibs Thunks/gen: Add detection logic for data layout differences 2023-10-02 22:18:22 +02:00
include/common Thunks: Simplify PackedArguments invocation code 2023-09-07 13:56:53 +02:00
libasound Thunks/gen: Consolidate all generated code to one file per library per platform 2022-09-05 15:03:49 +02:00
libdrm Thunks/gen: Consolidate all generated code to one file per library per platform 2022-09-05 15:03:49 +02:00
libEGL Thunks: Fixes missing thunk librarie so versions 2022-10-24 20:59:54 -07:00
libfex_malloc Thunks/gen: Consolidate all generated code to one file per library per platform 2022-09-05 15:03:49 +02:00
libfex_malloc_loader Thunks: Adds FEX malloc libraries 2021-10-02 18:12:07 -07:00
libfex_malloc_symbols Thunks: Adds FEX malloc libraries 2021-10-02 18:12:07 -07:00
libfex_thunk_test unittests: Add test thunk library 2023-09-18 16:53:35 +02:00
libGL Thunks: Fixes missing thunk librarie so versions 2022-10-24 20:59:54 -07:00
libSDL2 Thunks/gen: Consolidate all generated code to one file per library per platform 2022-09-05 15:03:49 +02:00
libVDSO VDSO: Add sigreturn functions to VDSO 2023-02-04 10:35:06 -08:00
libvulkan Thunks: Fixes missing vulkan definitions 2023-10-11 01:49:42 -07:00
libwayland-client Thunks/wayland: Add more message signatures required by Super Meat Boy with zink 2023-09-19 17:33:24 +02:00
libX11 Remove inline assembly 2023-07-20 16:05:25 -07:00
libxcb Thunks/xcb: Fixes typo in version check. 2023-07-18 04:16:11 -07:00
libxcb-dri2 Thunks: Fixes missing thunk librarie so versions 2022-10-24 20:59:54 -07:00
libxcb-dri3 Thunks: Fixes missing thunk librarie so versions 2022-10-24 20:59:54 -07:00
libxcb-glx Thunks: Fixes missing thunk librarie so versions 2022-10-24 20:59:54 -07:00
libxcb-present Thunks: Fixes missing thunk librarie so versions 2022-10-24 20:59:54 -07:00
libxcb-randr Thunks: Fixes missing thunk librarie so versions 2022-10-24 20:59:54 -07:00
libxcb-shm Thunks: Fixes missing thunk librarie so versions 2022-10-24 20:59:54 -07:00
libxcb-sync Thunks/gen: Consolidate all generated code to one file per library per platform 2022-09-05 15:03:49 +02:00
libxcb-xfixes Thunks: Fixes missing thunk librarie so versions 2022-10-24 20:59:54 -07:00
libXext ThunkLibs: X11/Xext: Removes two functions that don't exist on 32-bit 2022-09-24 09:51:40 -07:00
libXfixes Thunks/gen: Consolidate all generated code to one file per library per platform 2022-09-05 15:03:49 +02:00
libXrender Thunks/gen: Consolidate all generated code to one file per library per platform 2022-09-05 15:03:49 +02:00
libxshmfence Thunks/gen: Consolidate all generated code to one file per library per platform 2022-09-05 15:03:49 +02:00
README.md Thunks/gen: Drop now unneeded callback_unpacks file 2022-08-08 16:08:46 +02:00

FEX Library Thunking (Thunklibs)

FEX supports special guest libraries that call out to host code for speed and compatibility.

We support both guest->host thunks, as well as host->guest callbacks

Building and using

The thunked libraries can be built via the guest-libs and host-libs targets of the main FEX project. The outputs are in $BUILDDIR/Guest and $BUILDDIR/Host

After that, a guest rootfs is needed with the guest-libs installed. Typically this is done with symlinks that replace the native guest libraries. eg

# Unlink original guest lib
unlink $ROOTFS/lib/x86_64-linux-gnu/libX11.so.6
# Make it point to thunked version
ln -s $BUILDDIR/Guest/libX11-guest.so $ROOTFS/lib/x86_64-linux-gnu/libX11.so.6

Finally, FEX needs to be told where to look for the matching host libraries with -t /Host/Libs/Path. eg FEXLoader -c irjit -n 500 -R $ROOTFS -t $BUILDDIR/Host -- /PATH/TO/ELF

We currently don't have any unit tests for the guest libraries, only for OP_THUNK.

Implementation outline

There are several parts that make this possible. This is a rough outline.

In FEX

  • Opcode 0xF 0x3F (IR::OP_THUNK) is used for the Guest -> Host transition. Register RSI (arg0 in guest) is passed as arg0 in host. Thunks are identified by a string in the form library:function that directly follows the Guest opcode.
  • Context::HandleCallback does the Host -> Guest transition, and returns when the Guest function returns.
  • A special thunk, fex:loadlib is used to load and initialize a matching host lib. For more details, look in ThunkHandler_impl::LoadLib
  • ThunkHandler_impl::CallCallback is provided to the host libs, so they can call callbacks. It prepares guest arguments and uses Context::HandleCallback

ThunkLibs, Library loading

  • In Guest code, when a thunking library is loaded it has a constructor that calls the fex:loadlib thunk, with the library name and callback unpackers, if any.
  • In FEX, a matching host library is loaded using dlopen, fexthunks_exports_$libname(CallCallbackPtr, GuestUnpackers) is called to initialize the host library.
  • In Host code, the real host library is loaded using dlopen and dlsym (see ldr generation)

ThunkLibs, Guest -> Host

  • In Guest code (guest packer), a packer takes care of packing the arguments & return value into a struct in Guest stack. The packer is usually exported as a symbol from the Guest library.
  • In Guest code (guest thunk), a thunk does the Guest -> Host transition via OP_THUNK, and passes the struct pointer as an argument
  • FEX handles OP_THUNK and looks up the Host function from the opcode argument
  • In Host code (host unpacker), an unpacker takes the arguments from the struct, and calls a function pointer with the implementation of that function. It also stores the return value, if any, to the struct.
  • In Host code (host unpacker), the unpacker returns, and we do an implicit Host -> Guest transition
  • In Guest code (guest packer), the return value is loaded from the struct and returned, if needed

ThunkLibs, Host -> Guest. This is only possible while handling a Guest -> Host call (ie, callbacks).

  • In Host code (host packer), a packer packs the arguments & return value to a struct in Host stack.
  • In Host code (host packer), ThunkHandler_impl::CallCallback is called with the Guest unpacker, and Guest function as arguments
  • In Guest code (guest unpacker), the arguments are unpacked, the Guest function is called, and the return value is stored to the struct
  • In Guest code (guest unpacker), the unpacker returns and we do an implicit Guest -> Host transition
  • In host code (host packer), the return value is loaded from the struct and returned, if needed

Boilerplate code is automated using a dedicated code generator tool, which parses a C++ source file (libX_interface.cpp) that specializes a templated fex_gen_config struct for each thunked function. The generator will pull all required function signatures from the original library's header files and emit the appropriate boilerplate (guest->host thunks, argument packers/unpackers, host library loader, ...).

In most cases, an empty fex_gen_config specialization is sufficient, but if needed the generator behavior can be customized on a function-by-function basis using an annotation-syntax: Binary properties are toggled by inheriting from a fixed set of tag types (e.g. fexgen::custom_host_impl), whereas complicated properties are customized by defining struct members/aliases with a magic name detected by the generator (e.g. using uniform_va_type = char).

For each thunked library, the generator outputs the following files:

  • thunks.inl: Guest -> Host transition functions that use 0xF 0x3F
  • function_packs.inl: Guest argument packers / rv handling, private to the SO. These are used to solve symbol resolution issues with glxGetProc*, etc.
  • function_packs_public.inl: Guest argument packers / rv handling, exported from the SO. These are identical to the function_packs, but exported from the SO
  • function_unpacks.inl: Host argument unpackers / rv handling
  • ldr.inl: Host loader that dlopens/dlsyms the "real" host library for the implementation functions.
  • ldr_ptrs.inl: Host loader pointer declarations, used by ldr and function_unpacks
  • tab_function_unpacks.inl: Host function unpackers list, passed to FEX after Host library init so it can resolve the Guest Thunks to Host functions

Adding a new library

There are two kinds of libs, simpler ones with no callbacks, and complex ones with callbacks. You can see how libX11 is implemented for a callbacks example, and libasound for a non-callbacks example.

Getting started

  • Create libName/libName_interface.cpp and customize the fex_gen_config template for each thunked function. See some existing lib for details.
  • Create libName/libName_Guest.cpp and libName/libName_Host.cpp. Copy & rename from some existing lib is the way to go.
  • Edit GuestLibs/CMakeLists.txt and HostLibs/CMakeLists.txt to add the new targets, similar to how other libs are done.

Now the host and the guest libs should be built as part of guest-libs and host-libs