mirror of
https://github.com/libretro/MesenCE.git
synced 2026-11-07 23:02:54 -10:00
hizzlekizzle
d2e771806a
this was breaking the CI process -_- simplify libretro makefile (#1) only build the components needed by libretro default to building with GCC see if this can clear some CI hurdles let runners override toolchain libretro: build fixes (#2) libretro: more apple build fixes (#4) libretro: apple build fixes (#5) buildfix for iOS/tvOS libretro: frontend VFS support + Android/Windows/libnx build fixes (#3) * libretro: route all file access through the frontend VFS RetroArch can hand the core paths that don't exist on the real filesystem - the common case being Android SAF content:// URIs, which only the frontend can open. The core sets need_fullpath, so it opens the content itself and every one of those loads currently fails. All of the core's file I/O goes through utf8::ifstream/utf8::ofstream, so routing those two through a VFS-backed streambuf covers ROMs, patches, firmware, battery saves, save states, screenshots and recordings: - Utilities/VfsFile.{h,cpp}: VfsStreamBuf, a std::streambuf on top of retro_vfs_interface with a stdio fallback for when no frontend VFS is available, plus VfsIo helpers for stat/mkdir/opendir/remove/rename. Note that RetroArch's seek() returns 0 rather than the new offset, so the position is always read back with tell(). - UTF8Util.h: in LIBRETRO builds utf8::ifstream/ofstream are backed by VfsStreamBuf (the _WIN32 wide-path classes stay for other builds). - retro_set_environment asks for VFS v3 and falls back to v2, so the directory calls are used when the frontend has them. - FolderUtilities uses the VFS for CreateFolder/GetFolders/ GetFilesInFolder when v3 is available, std::filesystem otherwise. - ZipWriter builds archives in memory and writes them out through the stream layer - miniz's file API goes straight to stdio. - The remaining direct std::ifstream/std::ofstream/rename/remove calls in libretro.cpp and sha1.cpp now go through the same layer. Verified with a harness that only accepts saf://-prefixed paths (a core still on stdio fails to load): GB and SNES content loads, and the RecentGames archive is written back through the VFS. A second harness that refuses GET_VFS_INTERFACE checks the stdio fallback. * libretro: fix the Android, Windows and libnx builds Android (No rule to make target '.../Core/APU.cpp'): Libretro/jni/ Android.mk pulled its file list from Libretro/Makefile.common, a leftover from the Mesen 1 libretro port that lists sources this tree doesn't have. Android.mk now derives the sources the same way Makefile.libretro does (and builds as C++17, not C++11); the two stale Mesen 1 makefiles are removed. Windows x64: two failures. - Lua/unixdgram.c needs sys/un.h. The unix-domain luasocket modules (unix.c, unixdgram.c, unixstream.c, serial.c) aren't registered by the core and aren't in the MSVC project either, so they're excluded. - The core is cross-compiled from Linux, so uname reported Linux and the build produced a .so with ELF-only link flags. The OS is now taken from the target triple, and the Windows build gets --no-undefined plus the winsock/UPnP import libraries. libnx: the job builds with platform=libnx and then moves mesen2_libretro_libnx.a, but Makefile.libretro had no such target, so it built a host .so and the job died on the move. It now includes switch_rules, compiles with the devkitA64 flags and archives a static library. Untested - this needs a CI run on the libnx image to confirm the core itself builds for the Switch. Also fixes the compiler detection: `$(CXX) --version` matched 'gcc|g++' by name, so a plain `make -f Makefile.libretro` (CXX=c++) took the clang path and passed -flto=thin to GCC. It now looks for clang instead. * libretro: use _wfopen_s in the MSVC build The Mesen solution compiles with C4996 as an error, so the deprecated _wfopen broke the standalone Windows build. The stdio fallback now has a single open helper that uses _wfopen_s on MSVC. * libretro: keep the core changes behind LIBRETRO, restore the old makefiles Review feedback on libretro/MesenCE#3: the changes outside the libretro glue should be conditional so this stays a soft fork, and the deleted files should come back. Everything the VFS work touches in shared code is now behind #ifdef LIBRETRO - the WaveRecorder stream member, sha1's file hash, ZipWriter's archive writing and FolderUtilities' directory calls all build exactly as before without it. VfsFile.{h,cpp} is a libretro-only translation unit now (empty otherwise), so it is out of Utilities.vcxproj again; Makefile.libretro and Android.mk pick it up by globbing. The macro is LIBRETRO, which is what Makefile.libretro and Android.mk define, rather than __LIBRETRO__. Libretro/Makefile and Libretro/Makefile.common are restored. Android.mk no longer includes Makefile.common, so the stale Mesen 1 file list is only dead weight rather than a broken build. Re-verified after the change: linux-x64 core builds, the saf:// harness loads GB and SNES content and writes the RecentGames archive through the VFS, and the no-VFS harness still loads content through stdio. * libretro: confine the VFS to the libretro layer Review feedback on libretro/MesenCE#3: keep the VFS out of the core and wrap the core's existing file access instead. VfsFile.{h,cpp} moves to Libretro/, so it is not part of Utilities at all any more. The interception point is utf8::ifstream/ofstream in UTF8Util.h - the one place the core already routes file access through - which is the only file in Core/ or Utilities/ that mentions the VFS. The four sites that used to bypass utf8:: now go through it, which drops their #ifdef LIBRETRO entirely (and incidentally fixes non-ASCII paths on Windows, where utf8:: is what the rest of the codebase already uses): - sha1.cpp opened a raw std::ifstream - WaveRecorder held a std::ofstream, now a utf8::ofstream opened in place rather than move-assigned - ZipWriter used miniz's stdio API; it now drives mz_zip_writer_init with a write callback backed by an ofstream, and reads added files through an ifstream. Still streamed, so no in-memory archive - FolderUtilities keeps three guarded early-returns; the directory walk itself moved to VfsIo::GetEntries/CreateFolderIfSupported, which return false when the frontend has no v3 folder calls so the core falls back to std::filesystem Makefile.libretro and Android.mk compile Libretro/VfsFile.cpp explicitly. libretro: fix the Windows and libnx buildbot jobs (#6) * libretro: lowercase the Windows headers for the MinGW cross-build The buildbot's libretro-build-windows-x64 job is an MXE MinGW cross-build running on Linux, where the include lookup is case-sensitive. mingw-w64 ships these headers all-lowercase, so <Windows.h> and <Ws2tcpip.h> are not found and the job dies on Utilities/PlatformUtilities.cpp: fatal error: Windows.h: No such file or directory MSVC is unaffected either way, since the Windows filesystem matches the name regardless of case. UTF8Util.cpp keeps its <Windows.h>: that include sits behind #ifdef _MSC_VER, so the MinGW build never reaches it. * libretro: don't require IN6ADDR_ANY_INIT on libnx libretro-build-libnx-aarch64 dies on luasocket: Lua/inet.c:374:39: error: 'IN6ADDR_ANY_INIT' undeclared devkitA64's newlib declares in6_addr, sockaddr_in6 and AF_INET6 - the rest of the IPv6 code around it compiles - but not that macro. Fall back to zeroing the address, which is what the wildcard address is by definition and does not depend on how in6_addr is laid out. libretro doesn't need upnp mapping hopefully fix win64 builds remove LUAOBJ from libnx hopefully get it building on CI fix typo in ifdef libretro: fix the remaining Windows and libnx buildbot failures (#7) * libretro: link winmm on Windows PlatformUtilities asks for a 1ms timer resolution while a game runs, and nothing in LIBS provides timeBeginPeriod/timeEndPeriod, so the MXE link ends on __imp_timeBeginPeriod / __imp_timeEndPeriod. Those two are the only symbols left unresolved in the buildbot's windows-x64 job. * libretro: define _DEFAULT_SOURCE on libnx Utilities/Socket.cpp does not compile there: u_long, fd_set, FD_ZERO, FD_SET, select and struct timeval are all undeclared. They are all in newlib, behind __MISC_VISIBLE and __BSD_VISIBLE - and newlib sets both from _DEFAULT_SOURCE alone, which it auto-defines only when __STRICT_ANSI__ is absent. CXXFLAGS passes --std=c++17, which defines it. glibc turns _DEFAULT_SOURCE on whatever the dialect, so the same source builds everywhere else and only the Switch job sees this. re-add LUAOBJ to libnx libretro: fix the windows-x64 link and get libnx past Lua (#8) * libretro: give the VFS streams a key function The windows-x64 job links with -flto and stops on multiple definitions of the vtable, VTT and construction vtables for utf8::ifstream and utf8::ofstream, one pair per translation unit that includes UTF8Util.h: BsxCart.o: multiple definition of `vtable for utf8::ifstream'; libretro.o: first defined here std::istream and std::ostream inherit virtually, so a class derived from them gets construction vtables and a VTT as well as its own vtable. With no key function all of it is emitted everywhere into .gnu.linkonce comdats, and on PE/COFF the group keys come out inconsistent between objects - the same section is keyed on the vtable in one and on the construction vtable in another - so the linker keeps both instead of folding them. Declaring the destructors in the header and defining them in the .cpp makes those the key functions, so one object emits the lot. The VFS streams are the only classes here that derive from a virtual base, which is why this is the job that noticed. * libretro: build libnx as gnu++17 instead of patching around __STRICT_ANSI__ declarations back for Utilities/Socket.cpp. That was the narrower of the two options and it only covered newlib: the job now reaches Lua and stops there, because gcc's own limits.h leaves LLONG_MAX undefined in C++ mode when __STRICT_ANSI__ is set, and luaconf.h reads that as "no long long": Lua/luaconf.h:572: error: #error "Compiler does not support 'long long'. Use option '-DLUA_32BITS' or '-DLUA_C89_NUMBERS'" Both come from the same place, so take the other option instead. MESENFLAGS lands after the global --std=c++17 in CXXFLAGS, so this stays scoped to libnx; every other target keeps the dialect it had. * Delete mesen2_libretro.so libretro: give libnx the ipv6_mreq luasocket needs (#9) libretro-build-libnx-aarch64 is the last red job on the buildbot, and it dies in luasocket: Lua/options.c: In function 'opt_ip6_setmembership': Lua/options.c:399:22: error: storage size of 'val' isn't known 399 | struct ipv6_mreq val; devkitA64's newlib declares in6_addr, sockaddr_in6 and the IPV6_ADD_MEMBERSHIP / IPV6_DROP_MEMBERSHIP constants - the two callers a few lines up compile, and the IN6ADDR_ANY_INIT fallback in inet.c already leans on in6_addr being there - but it never declares the struct those two options take. There is no macro to test a struct tag for, so the shim keys off __SWITCH__, and it declares luasocket_ipv6_mreq rather than filling in struct ipv6_mreq so that a devkitA64 that grows the real one later cannot collide with it. socket.h is where it goes because options.c is built on every platform and only reaches usocket.h off Windows. The layout is the one RFC 3493 fixes; checked against glibc, both are 20 bytes with the fields at 0 and 16, so the buffer handed to setsockopt() is unchanged. Whether the Switch network stack honours IPv6 multicast joins at runtime is a separate question - this only stops the option from breaking the build for every other libnx user. set core option to default to 44.1 khz this was making everything 55 FPS instead of 60 libretro: give libnx the LLONG_MAX luaconf.h expects (#10) With the ipv6_mreq shim from #9 in, the libnx job gets past Lua/options.c and dies further along, in the first C++ translation unit that pulls in lua.hpp: In file included from Lua/lua.h:16, from Lua/lua.hpp:6, from Core/Debugger/LuaCallHelper.h:3, from Core/Debugger/LuaCallHelper.cpp:2: Lua/luaconf.h:572:2: error: #error "Compiler does not support 'long long'." luaconf.h includes <limits.h> itself and then gates its LUA_INT_LONGLONG branch on LLONG_MAX. devkitA64's newlib does not make that macro visible to C++ translation units, so the branch falls through to the #error - while the C files in the same build, which the -std=gnu++17 in CFLAGS does not apply to, see it and compile. The error suggests LUA_32BITS or LUA_C89_NUMBERS, but both change Lua's integer type, and only on the Switch. GCC always predefines __LONG_LONG_MAX__, so fill LLONG_MAX, LLONG_MIN and ULLONG_MAX from it instead, guarded on __SWITCH__ and on the macro actually being absent. Checked against glibc's limits.h: all three expressions compare equal to the real LLONG_MAX, LLONG_MIN and ULLONG_MAX, and luaconf.h still parses from both C and C++ with the block inert off-Switch.