1701 Commits

Author SHA1 Message Date
Ryan Houdek
e3e7f0279c
Merge pull request #3644 from alyssarosenzweig/clang-format/left
clang-format: left-align escaped newlines
2024-05-20 07:12:50 -07:00
Alyssa Rosenzweig
a10f984b1c clang-format: left-align escaped newlines
alternative to #3638. this is theoretically better for side-by-side diffs. in
practice it may make other diffs worse since all the \'s change when part of the
macro change.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-05-20 09:47:21 -04:00
Ryan Houdek
ac1a096bae
InstCountCI: Hardcode the offset to load tests into
Depending on where the assembly was getting loaded in to memory it was
causing slight code generation differences.

Map the entire file to the same fixed offset as our ASM tests to ensure
consistency and removing flakes in CI.
2024-05-18 17:00:28 -07:00
Paulo Matos
5bee17bee1 NFC: Fix typo 2024-05-15 15:10:00 +02:00
Ryan Houdek
d19b57a52e
FEXCore: Get rid of DeferredSignalFaultAddress and use the InterruptFaultPage
Arm64ec introduced the InterruptFaultPage which is lower overhead since
instead of ldr+str it just turns in to a single str. We were already
allocating the space, FEXCore and the frontend signal delegator just
needed to be updated to understand the new location.

We can additionally use this in the future if we want to make deferred
async signals INSIDE the JIT only cost a single str as well.
2024-05-10 15:31:28 -07:00
Ryan Houdek
ef6d640a8c
Merge pull request #3612 from Sonicadvance1/threadmanager_move
FEXLoader: Changes frontend thread management to wrap FEXCore thread objects
2024-05-09 09:27:11 -07:00
Ryan Houdek
55d1d6bcd4
Merge pull request #3615 from bylaws/wow64-fix
Fix WOW64 frontend with recent wine versions
2024-05-07 22:35:32 -07:00
Billy Laws
c521d2b48d WOW64: Support unwinding past FEX from within syscall handlers
This is required by recent wine changes to use longjmp for user
callbacks. Switch to saving the context at every simulate call and
setting the unwind SP/PC to that context with a small SEH trampoline
for the syscall handler.
2024-05-06 15:26:36 +00:00
Billy Laws
9ed8165405 WOW64: Dynamically allocate unixcall/syscall entrypoints
Removes the requirement that FEX needs to be loaded as part of the lower
32-bit address space.
2024-05-06 14:55:59 +00:00
Ryan Houdek
5099b2b5dc
FEXServer: Removes temporary variable allocation
Was causing unnecessary memory allocation churn when a FEXInterpreter
was asking for the rootfs folder path.
2024-05-05 14:11:26 -07:00
Ryan Houdek
d372552593
FEXLoader: Changes frontend thread management to wrap FEXCore thread objects
A bit of refactoring necessary before we can move the remaining Linux
specific code to the frontend.

Most of this taken from #3535 but attempting to be NFC as much as
possible.
2024-05-05 07:43:09 -07:00
Ryan Houdek
729e32ccc2
Linux: Move ThreadManager to its own header 2024-05-05 06:32:59 -07:00
Teoh Han Hui
7519af2819
Pass compulsory mode argument to open when O_CREAT is used
From `man 2 open`:

> The mode argument must be supplied if O_CREAT or O_TMPFILE is
> specified in flags; if it is not supplied, some arbitrary bytes
> from the stack will be applied as the file mode.
2024-05-03 03:16:29 +08:00
Ryan Houdek
6463054fa3
Arm64: Adds another TSO hack to disable half-barrier TSO
A feature of FEX's JIT is that when an unaligned atomic load/store
operation occurs, the instructions will be backpatched in to a barrier
plus a non-atomic memory instruction. This is the half-barrier technique
that still ensures correct visibility of loadstores in an unaligned
context.

The problem with this approach is that the dmb instructions are HEAVY,
because they effectively stop the world until all memory operations in
flight are visible. But it is a necessary evil since unaligned atomics
aren't a thing on ARM processors. FEAT_LSE only gives you unaligned
atomics inside of a 16-byte granularity, which doesn't match x86
behaviour of cacheline size (effectively always 64B).

This adds a new TSO option to disable the half-barrier on unaligned
atomic and instead only convert it to a regular loadstore instruction,
ommiting the half-barrier. This gives more insight in to how well a
CPU's LRCPC implementation is by not stalling on DMB instructions when
possible.

Originally implemented as a test to see if this makes Sonic Adventure 2
run full speed with TSO enabled (but all available TSO options disabled)
on NVIDIA Orin. Unfortunately this basically makes the code no longer
stall on dmb instructions and instead just showing how bad the LRCPC
implementation is, since the stalls show up on `ldapur` instructions
instead.

Tested Sonic Adventure 2 on X13s and it ran at 60FPS there without the
hack anyway.
2024-04-24 13:09:00 -07:00
Billy Laws
a7f4e99278 InvalidationTracker: Always invalidate all regions of a section on unmap
Unmapping a section will unmap the whole size initially allocated,
irrespective of how their protections are changed afterwards. Make sure
to follow this logic for invalidation too.
2024-04-18 15:16:28 +00:00
Billy Laws
7391456e48 Windows: Don't redefine existing MinGW ntdll exports 2024-04-18 15:15:11 +00:00
Billy Laws
a6d061b711 InvalidationTracker: Invalidate code across all threads
When thread management was moved to the frontend, invalidation moved
from being a global operation to per-thread but the WOW64 backend wasn't
updated to account for this. Now for any invalidation event loop over
all threads tracked by the frontend and invalidate the appropriate
range.
2024-04-18 15:14:31 +00:00
Billy Laws
d92580bccf WOW64: Keep track of all created threads on the frontend
This is necessary so that code can be invalidated across all threads
rather than just the initiator on any event that triggers invalidation.
2024-04-18 15:00:24 +00:00
Paulo Matos
905aa935f5 Reformat until fixed-point
Followup to 2b4ec88daebd35fefb5bf5c73d7fc2b4155771ed.
Some files needed a couple of calls to clang-format 16.0.6 to
reach a fixed point.
2024-04-15 09:40:00 +02:00
Paulo Matos
2b4ec88dae Whole-tree reformat
This follows discussions from #3413.
Followup commits add clang-format file, script and blame ignore lists.
2024-04-12 16:26:02 +02:00
Ryan Houdek
904646e93b
FEXCore: Fixes priority of FEX_APP_CONFIG
This environment variable had an incorrect priority on the configuration
system. The expectation was higher priority than most other layers.

Now the only layer that has higher priority is the environment
variables.
2024-04-05 13:10:43 -07:00
Ryan Houdek
4214d9bda0
Merge pull request #3538 from pmatos/OffsetofOoB
Fix reference to out of bounds address in offsetof
2024-04-01 19:41:57 -07:00
Ryan Houdek
5c29c9d464
Merge pull request #3527 from Sonicadvance1/move_type_defines
Moves FHU TypeDefines to FEXCore includes
2024-04-01 08:57:22 -07:00
Ryan Houdek
3bed305660
Merge pull request #3526 from Sonicadvance1/move_codeloader
FEXCore: Moves CodeLoader to frontend
2024-04-01 07:52:02 -07:00
Paulo Matos
96087a69fa Fix reference to OoB address in offsetof and remove rflags printout
Adjust static array size to match new size.
Remove rflags from printing code and adjust offsets - fixes
printing off-by-one error.
2024-04-01 13:13:17 +02:00
Ryan Houdek
d11a36eaea
Moves FHU TypeDefines to FEXCore includes
FEXCore includes was including an FHU header which would result in
compilation failure for external projects trying to link to libFEXCore.

Moves it over to fix this, it was the only FHU usage in FEXCore/include
NFC
2024-03-29 02:54:54 -07:00
Ryan Houdek
f46e88ebdb
FEXCore: Moves CPUBackend definition internal
This is no longer necessary to be part of the public API. Moves the
header internally.

Needed to pass through `IsAddressInCodeBuffer` from CPUBackend through
the Context object, but otherwise no functional change.
2024-03-29 02:27:29 -07:00
Ryan Houdek
20eb338644
FEXCore: Moves CodeLoader to frontend
FEXCore no longer has a need for this since a bunch of related code was
already moved to the frontend. Move the CodeLoader now.
2024-03-29 02:24:53 -07:00
Mai
3d31291c3d
Merge pull request #3510 from Sonicadvance1/fix_pthread_memleak
Linux/Threads: Fixes a stack memory leak for pthreads
2024-03-27 21:38:44 -04:00
Ryan Houdek
624bc3fce5
Merge pull request #3520 from Sonicadvance1/sleep_process
FEXLoader: Add a way to sleep a process on startup
2024-03-27 18:35:06 -07:00
Ryan Houdek
24fd28ed9e
Merge pull request #3511 from Sonicadvance1/more_tso_levers
FEXCore: Adds more TSO control levers
2024-03-26 23:23:41 -07:00
Ryan Houdek
79454ed8a6
Merge pull request #3507 from Sonicadvance1/fd_tracking_check
FEXLoader: Add some debug-only tracking for FEX owned FDs
2024-03-26 23:22:29 -07:00
Ryan Houdek
7f90ca53f7
Merge pull request #3505 from Sonicadvance1/telemetry_noncanonical
Telemetry: Adds tracker for non-canonical memory access crash
2024-03-26 23:21:32 -07:00
Ryan Houdek
ade0c46845
FEXLoader: Add a way to sleep a process on startup
I find myself reimplementing this nearly monthly. Actually codify it so
I can stop reimplementing it.
2024-03-26 07:48:09 -07:00
Ryan Houdek
6f29e75f67
FEXCore: Removes vestigial mman SMC checking
This wasn't actually wired up to anything ever since some refactoring
occured two years ago.
2024-03-26 02:56:26 -07:00
Ryan Houdek
c1c797bcba
FEXConfig: Add new TSO levers
Nice and convenient when testing applications.
2024-03-26 02:50:54 -07:00
Ryan Houdek
ea31363221
Linux/Threads: Fixes a stack memory leak for pthreads
Same situation as the last stack leak memory fix, this is fairly tricky
since it is dealing with stack pivoting. Fixes the memory leak around
pthread stack allocations, making memory usage lower for applications
that constantly spin-up and destroy threads (Like Steam).

We need to let glibc allocate a minimum sized stack (128KB and we can't
control it) to work around a race condition with DTV/TLS regions. This
means we need to do a stack pivot once the thread starts executing.

We also need to be careful because the `PThread` object is deleted
inside of the execution thread, which was resulting in a use-after-free
bug.

There are definitely some more memory leaks that I'm still fighting, and I have
noticed in my abusive thread creation program that we might want to
change some jemalloc options to more aggressively cut down on residency.
This is just one out of many.
2024-03-24 05:22:22 -07:00
Ryan Houdek
50f62663ac
ELFParser: Stop using a VLA
Clang-18 complains about this, use a vector instead.
2024-03-22 22:51:57 -07:00
Ryan Houdek
60755acef0
FEXLoader: Add some debug-only tracking for FEX owned FDs
I remember seeing some application last year where they closed a FEX
owned FD but now I don't remember what it was. This can really mess us
up so add some debug tracking so we can try and find it again.

Might be something specifically around flatpack, appimage, or chrome's
sandbox. I have some ideas about how to work around these problems if
they crop up but need to find the problem applications again.
2024-03-22 22:49:26 -07:00
Ryan Houdek
5a35e119fe
Telemetry: Adds tracker for non-canonical memory access crash
This may be useful for tracking TSO faulting when it manages to fetch
stale data. While most TSO crashes are due to nullptr dereferences, this
can still check for the corruption case.
2024-03-21 20:47:36 -07:00
Ryan Houdek
45ea0cd782
Removes false termux support
This was a funny joke that this was here, but it is fundamentally
incompatible with what we're doing. All those users are running proot
anyway because of how broken running under termux directly is.

Just remove this from here.
2024-03-20 22:04:32 -07:00
Ryan Houdek
d8202335e0
Linux: Expose support for v6.8
The new syscalls for futexes are the most interesting part
2024-03-10 15:48:55 -07:00
Ryan Houdek
9ec20c4bef
Linux/Ioctls: Update ioctl emulation for v6.8
- v3d added an ioctl
- drm base added a new ioctl
- pvr and xe are new drivers in v6.8
2024-03-10 15:46:21 -07:00
Ryan Houdek
8a607135fd
Linux: Update syscalls for v6.8 2024-03-10 15:22:51 -07:00
Ryan Houdek
aeb41e9ae2
IoctlEmulation: Add missing nouveau ioctl
The NVIF ioctl isn't publicly described in the nouveau headers and it is
required for anything to work with Nouveau.

Pass the ioctl command through without modification and hope that this
ioctl is architecture agnostic.
2024-03-05 16:05:13 -08:00
Ryan Houdek
2f9449cb5a
Merge pull request #3465 from alyssarosenzweig/icci/pa
InstCountCI: enable preserve_all
2024-02-27 16:39:46 -08:00
Ryan Houdek
139367d248
Merge pull request #3463 from Sonicadvance1/update_xxhash
Update xxhash to v0.8.2
2024-02-27 16:39:38 -08:00
Alyssa Rosenzweig
b50292493a InstCountCI: enable preserve_all ABI
This is what we'll actually ship (I hope), so that's the config we want to
track long-term. It's also a lot more managable resulting asm.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-02-27 12:03:58 -04:00
Ryan Houdek
151e2279af
Linux: Converts passthrough syscalls to direct passthrough handlers
Reimagining of #3355 without any json generators or new concepts.

Fixes some mislabeling of system calls. Some getting inlined when they
shouldn't be, a lot not getting inlined when they can be.

This really cleans up the syscall implementation, all syscalls that can
be passthrough implementations require a very small two line
declaration.
Additionally cleans up a bit of implementation cruft where some
passthrough syscalls were using the glibc syscall handler, and some were
using the glibc implementation. We have had multiple issues in the past
where the glibc implementation does something subtly different than the
raw syscall and breaks things. Now all passthrough handlers do a system
call directly, removing at least one indirection and some ambiguity.

This makes it significantly easier to add new passthrough syscalls as
well. Only need to do a version check and add the three lines per
syscall. Which there are new syscalls incoming that we will want to add.

Tangible improvements:
- Syscalls are lower overhead than ever.
- When I'm adding more syscalls I have less chance of mucking it up.
2024-02-27 02:40:53 -08:00
Ryan Houdek
93ada89708
Linux: Move unimplement ustat and sysfs
AArch64 doesn't implement these and will return ENOSYS.
Moving them to NotImplemented so we can get a log if an application
tries to use these.
2024-02-27 02:39:36 -08:00