Commit Graph

121 Commits

Author SHA1 Message Date
Ryan Houdek
84379b5fdf CMake: Check for binfmt_misc conflicts before install
Check for qemu and box binfmt_misc file conflicts before the
`binfmt_misc` install command.

This ensures if you're building from source that you won't inadvertently
install conflicting binfmt_misc files, breaking program execution.
2022-07-01 13:42:45 -07:00
Ryan Houdek
8b35275ec1 unittests: Classify CPU based on CPU features
Instead of relying on runner features, classify based on CPU features.

This fixes an annoying issue where if running unit tests locally without
it set then you get an unexpected failure.

Fixes #1807
2022-06-30 13:55:38 -07:00
Stefanos Kornilios Misis Poiitidis
19fbc10c16 TestHarnessRunner: Don't setup a FEX Context when running host tests 2022-06-16 19:41:42 +00:00
Ryan Houdek
c0a8984799
Merge pull request #1764 from Sonicadvance1/fix_xxhash
FEXRootFSFetcher: Update and fix xxhash file hashing
2022-06-10 04:57:09 -07:00
Stefanos Kornilios Misis Poiitidis
40ec910108 unittests: Add FEXLinuxTests, with a few signal and smc tests 2022-06-10 08:34:40 +03:00
Ryan Houdek
bc2840e4a7 FEXRootFSFetcher: Update and fix xxhash file hashing
The final tail of the file reading was incorrect, so our hashing was
"correct" but it was using stale data from the previous block size read.

Noticed this while wiring up the CI rootfs fetching since the hashing is
a lot simpler there.

Now instead of reading a tail, just attempt to read the full block size
and use the resulting data size instead. Confirmed it matches expected
results now.

In the process we are going to need to update hyperlinks and hashes
anyway, change the hash to XXH3 so it is faster to run.
2022-06-09 20:20:43 -07:00
Ryan Houdek
d2a57f6231 Scripts: Add CI rootfs fetch script
This will allow our CI system to automatically pull their rootfs.
2022-06-09 20:08:24 -07:00
Ryan Houdek
952e157770 Scripts: Allow user override on tagged version
In the case of a missing month or a minor version, need to allow user
defined overrides.
2022-06-04 18:06:28 -07:00
Ryan Houdek
82319c9deb Scripts: Updates generate syscall numbers to support renaming
Instead of manually renaming the three syscalls each time, let the
script do it automatically.
2022-04-23 11:56:33 -07:00
Ryan Houdek
fba698cb74 Scripts: Updates AArch64 fit for Clang 14
Clang now supports these latest ARMv9 CPUs
2022-04-01 18:08:02 -07:00
Ryan Houdek
9911fe68d4 Scripts: Stop using deprecated Distutils
According to PEP 386: https://www.python.org/dev/peps/pep-0386/

distutils is deprecated and will be removed in an upcoming python
version.

Switch over to pkg_resources for version parsing and comparison
2022-03-01 07:10:41 -08:00
Ryan Houdek
2af23d9bec
Merge pull request #1591 from Sonicadvance1/new_cpus_in_native_fit
Scripts: Updates CPU fitting script for latest CPUs
2022-02-28 07:05:45 -08:00
Ryan Houdek
bb7fa84fb8 Scripts: Updates CPU fitting script for latest CPUs
Clang-13 doesn't yet understand the latest ARM CPUs so just document them
and set to the closest thing.
2022-02-26 02:14:52 -08:00
Ryan Houdek
2b8f60c108 TestHarness: Support for asm files having the option to set config options
Allows some something like the following:
"Env": {
  "FEX_MAXINST": "500"
}

Not that I would recommend overriding MAXINST in the asm tests, as
command line overrides that
2022-02-21 14:53:27 -08:00
Ryan Houdek
228aed98c7 unittests: Fixes ROOTFS needing to be defined prior to cmake
cmake will bake in the environment variable in to the build scripts.
Instead have the guest_test_runner fetch it at runtime.

This means if you forget to set ROOTFS prior to running cmake, you can
now set it afterwards and rerun with just ctest instead of a cmake
dance.

Fixes #315
2022-01-09 01:56:13 -08:00
Ryan Houdek
f836ff0897 Scripts: Adds a python script that can hand hold a user through FEX install
This is definitely a bit divisive but overall this is a win.
This is a user pattern that is emerging in a bunch of projects.
Allow an officially sourced script that lets you pipe a script directly
in to python/bash and setup the environment entirely.

This only supports Ubuntu {20.04, 21.04, 21.10, 22.04} which matches
exactly what we expose in the PPA.

Once this is in the repo, and our PPA is updated to the latest release
tag you can run this script like:
`curl --silent <Direct Github raw link> | python3`

Once the PPA is updated, the README and Wiki will be updated with Quick
Start guides to use this path.

The script steps
1) Checks if ARMv8, or fail
2) Checks if supported Ubuntu, or fail
3) Checks if PPA installed
3a) Install PPA if not, or fail
4) Check if packages are installed
4a) Install non-installed packages, or fail
5) Check if RootFS is configured/exists
5a) Run through FEXRootFSFetcher to get/setup rootfs, or fail
6) Attempt to run emulated uname -a through FEX, or fail
7) Provide some examples for how to use FEX
8) Exit with success!
2022-01-03 14:20:09 -08:00
Ryan Houdek
33be4fa98d Scripts: Updates syscall definition extractor
We were missing one of the definition types which means we missed
definitions
2021-11-22 14:03:28 -08:00
Ryan Houdek
aae1dd4d81 Adds new script to generate syscall enums
Lets us define our own enums for syscall mapping.
Necessary to get all the syscall IDs in a sane way
2021-11-15 14:29:49 -08:00
Ryan Houdek
0bfc1bbe70 Arm64: Don't fall back to native
In the case of Arm64, make sure not to fallback to native if we hit an
unsupported CPU.
Can cause issues depending on system configuration.
2021-10-21 20:39:49 -07:00
Ryan Houdek
2b757a9b9a
Merge pull request #1284 from Sonicadvance1/fix_struct_match
StructVerifier: Fix struct match and minor fixes
2021-10-02 11:07:51 -07:00
Ryan Houdek
1b0d2bbf9f Scripts: Adds a new script for extracting function definitions
This is very useful for extracting function definitions for thunks.
Keep it in upstream to not get lost.

Sometimes it can munge a definition but it is usually fine.
2021-09-30 18:35:43 -07:00
Ryan Houdek
5b2d944886 Linux: Minor struct verifier and types fixes
fex-match annotation was actually not doing anything due to python typo.
Fixes the minor warnings that cropped up. Nothing actually broken
2021-09-30 18:08:05 -07:00
Ryan Houdek
083d3a464a StructPackVerifier: Add a couple missing defines 2021-09-03 16:46:16 -07:00
Ryan Houdek
fd6d33d197
Merge pull request #996 from FEX-Emu/skmp/aotirgen
AOTIR: Offline Generation
2021-05-03 15:58:57 -07:00
Stefanos Kornilios Mitsis Poiitidis
be41f14452 Fix generation script 2021-05-03 18:16:43 +03:00
Stefanos Kornilios Mitsis Poiitidis
cf5bd202d4 AOTIR: Update gen script to default to global fex installation 2021-05-03 06:15:25 +03:00
Ryan Houdek
8d34222556 Add vardecl type matching for ioctl verification 2021-05-01 07:39:14 -07:00
Stefanos Kornilios Mitsis Poiitidis
9a4fc758c1 AOTIR: FEXUpdateAOTIRCache.sh now passes compilation modifiers 2021-05-01 09:40:50 +03:00
Stefanos Kornilios Mitsis Poiitidis
4bd997adaa AOT: Postfix ir files with .aotir 2021-05-01 09:40:50 +03:00
Stefanos Kornilios Mitsis Poiitidis
244ef94e9b AOTIR: Generate list of files with code, add Scripts/FEXUpdateAOTIRCache.sh to generate all files 2021-05-01 09:40:50 +03:00
Ryan Houdek
b06ebf486f Work around M1 Parallels hypervisor not showing CPU type
Parallels claims that the CPU is of implementor 0x41 and part number 0.
Easy enough to work around before real numbers get exposed
2021-04-27 22:26:35 -07:00
Ryan Houdek
8adb1ca3a2 Fixes ordering problem of tag generation in release script
Wasn't actually creating an annotated tag with this order
2021-04-05 18:35:09 -07:00
Ryan Houdek
26d9ddef9f Updates release script to match what I want it to be
Generates the current and previous tag automatically.
Checks to ensure the previous tag exists and that the current tag does not
2021-04-02 11:49:19 -07:00
Stefanos Kornilios Mitsis Poiitidis
15537f8c8c Scripts: Also add unittests folder for doc outline 2021-03-30 11:58:18 +03:00
Stefanos Kornilios Mitsis Poiitidis
1fdd6fbb14 Scripts: Add a documentation comment in changelog_generator.py 2021-03-30 11:55:39 +03:00
Stefanos Kornilios Mitsis Poiitidis
458bebf598 Scripts: Add a documentation comment in doc_outline_generator.py 2021-03-30 11:53:11 +03:00
Stefanos Kornilios Mitsis Poiitidis
2b10b9792b Scripts: Fix generate_release, don't use markdown for changelogs 2021-03-30 11:23:00 +03:00
Stefanos Kornilios Mitsis Poiitidis
0dd02da57c Docs: Update outline scripts 2021-03-30 11:23:00 +03:00
Stefanos Kornilios Mitsis Poiitidis
309139e203 Scripts: Add generate_release script 2021-03-30 11:23:00 +03:00
Stefanos Kornilios Mitsis Poiitidis
350c33ada4 Docs: Update outline script 2021-03-30 11:23:00 +03:00
Stefanos Kornilios Mitsis Poiitidis
bdd35e5743 docs: Update outline generator 2021-03-30 11:23:00 +03:00
Stefanos Kornilios Mitsis Poiitidis
ceb7082e37 Docs/Versioning: Add Changelog Generator 2021-03-30 11:23:00 +03:00
Stefanos Kornilios Mitsis Poiitidis
2d6dc80039 Docs: Adds tag-based outline generator w/ glossary support 2021-03-30 11:22:56 +03:00
Ryan Houdek
de890e7387 Have unit tests check for runner label 2021-03-22 09:03:48 -07:00
Ryan Houdek
aacb0f6891 Adds new struct_verifier ctest to cmake
Currently only testing 32bit syscall struct definitions
2021-03-15 15:24:49 -07:00
Ryan Houdek
e0c5840f2c Adds libclang based struct verifier
This requires multiarch on the targets to work.
Will run a header through multiple architectures and ensure that the struct packing all works
2021-03-15 06:58:06 -07:00
Stefanos Kornilios Mitsis Poiitidis
29a8fd0a4a Tests: Add gcc-target-tests for 32 and 64 bit 2021-01-26 12:00:45 +02:00
Ryan Houdek
921867de7e Works around Clang failing to identify new Kryo CPUs
Some of the newer CPU cores in LLVM's source claim to be a Cortex-A73,
which means they become limited to an ARMv8.0 feature set.

This is what you get if you compile FEX with -mcpu=native

To work around this issue, manually parse /proc/cpuinfo ourselves and
pull out the CPU type to pass to clang directly.
This also fixes the issue that we were using -march on AArch64, which no
longer works on newer clang versions. We instead need to use mcpu or
mtune.

Should improve all atomic op performance outside of the JITs, where they
were turning in to loadstore exclusive pairs.
2021-01-19 03:21:10 -08:00
Stefanos Kornilios Mitsis Poiitidis
311aa2b811 Tests: Cleanup scripts, review feedback 2021-01-10 18:06:02 +02:00
Stefanos Kornilios Mitsis Poiitidis
278e156a8a Tests: Move timeout to ctest from python + timeout 2021-01-09 19:26:15 +02:00
Stefanos Kornilios Mitsis Poiitidis
fea7ac1f83 Tests: Update posix_test_runner, update gvisor and posix test lists 2021-01-09 18:57:19 +02:00
Ryan Houdek
7c14619346 Support disabling ASM and IR tests per runner type
Allows us to disable specific tests for interpreter, jit, or host.
No architecture specific disabling yet
2020-09-25 21:17:54 -07:00
Scott Mansell
cb6dcf62eb Add MemoryData to IR and ASM Tests 2020-08-21 01:29:37 +12:00
Ryan Houdek
d23b049dd0 Changes posix test timeout wrapper to use SIGKILL
Since we now capture all signals we need to be more aggressive.
2020-08-11 00:33:49 +01:00
Ryan Houdek
f613d670c7 Increase timeout of test runner 2020-07-19 04:03:21 -07:00
Stefanos Kornilios Mitsis Poiitidis
edd6d5ec4b posixtests: Add support for comments in the failures/outputs/disabled lists 2020-06-30 13:11:45 +03:00
Ryan Houdek
b262b59916 Fixes up prefix handling for 32bit
Moves REX to a 64bit only table entry. This allows the missing INC/DEC
instructions to exist in 32bit space
2020-06-10 16:49:06 -07:00
Ryan Houdek
2474ca1437 Adds support for disabled tests in the IR and ASM runners 2020-05-28 15:09:51 -07:00
Stefanos Kornilios Mitsis Poiitidis
f10891eb11 posixtests: Move disabled list to file from filenames 2020-05-26 17:39:22 +03:00
Stefanos Kornilios Mitsis Poiitidis
5ae047e031 posix_test_runner.py: More debug output 2020-05-23 19:25:29 +03:00
Stefanos Kornilios Mitsis Poiitidis
430ff3e2a6 Move posixtest to ctest, add known failures 2020-05-22 16:23:07 +03:00
Ryan Houdek
6cdd3916c0 Adds IR based unit test support.
Now that everything is punched through that is necessary, we can now
have IR based unit tests that will be used to ensure the IR is working
correctly.
Specifically this is necessary for quick iterating on RA testing.
2020-05-09 12:19:36 -07:00
Ryan Houdek
31afb62a2a Split python unit test ASM config parser code out to an independent file
This will be needed once we have multiple file type configs getting
parsed
2020-05-09 12:14:59 -07:00
Ryan Houdek
6305b40a33 Adds known failure support to unit test runner
This will allow us to add unit tests that have known failures like
unsupported features

Additionally if the test passes and is on the known failure list then it
still fails since its state has changed
2020-03-16 12:35:09 -07:00
Scott Mansell
d4cffa5354 Add x87/MMX registers to test harness 2020-03-06 09:08:47 +02:00
Ryan Houdek
369686c992 Initial Commit 2020-03-06 09:08:13 +02:00
Stefanos Kornilios Mitsis Poiitidis
e9ea4cbb76 FEXCore: Move to Externals/FEXCore 2020-03-06 09:07:37 +02:00
Ryan Houdek
218f38ff3d Adds new IR::GetArgs function
This returns SSA args that will eventually be register allocated.
2020-03-06 07:55:20 +02:00
Ryan Houdek
199cfd76d8 Refactor IR and other changes that are hard to split
I had to change how blocks are represented to make it easier to parse
This required a fairly substantial refactor that makes it so blocks are
represented differently and we can walk them sequentially.

This will make future analysis easier to deal with.
Had to rewrite the passes and core's parsing of the IR afterwards.

Moved RA in to a optimization pass to be shared between the JIT backends
This works because x86-64 and AArch64 RA can be identical.

Still doesn't support PHI nodes or spilling correctly, this is the first
step in the process of getting there.
2020-03-06 07:55:13 +02:00
Ryan Houdek
40cad15da9 Cleanup old IR emitter functions
Argumentless IR emitter functions were prone to generating invalid code.
Remove them from the python emitter and change the branch instructions
that were using them to a new version instead.
Adds NumUse tracking as well.
2020-03-06 07:48:41 +02:00
Ryan Houdek
7252050e42 Initial commit 2020-03-06 07:48:39 +02:00