Compare commits

...
676 Commits
Author SHA1 Message Date
Justine Tunney c9152b6f14 Release Cosmopolitan v3.8.0
This change switches c++ exception handling from sjlj to standard dwarf.
It's needed because clang for aarch64 doesn't support sjlj. It turns out
that libunwind had a bare-metal configuration that made this easy to do.

This change gets the new experimental cosmocc -mclang flag in a state of
working so well that it can now be used to build all of llamafile and it
goes 3x faster in terms of build latency, without trading away any perf.

The int_fast16_t and int_fast32_t types are now always defined as 32-bit
in the interest of having more abi consistency between cosmocc -mgcc and
-mclang mode.
2024-08-30 20:14:07 -07:00
Justine Tunney 5b9862907c Delete superfluous function 2024-08-29 23:51:22 -07:00
Justine Tunney c2420860e6 Fix --ftrace 2024-08-29 23:51:05 -07:00
Gabriel Ravier 6baf6cdb10 Fix vfprintf and derived functions badly handling +/` flag conflict (#1269) 2024-08-29 19:07:05 -07:00
Gabriel Ravier 06a1193b4d Make it so the test harness closes fds up to 100 (#1268) 2024-08-29 16:12:53 -07:00
Justine Tunney 884d89235f Harden against aba problem 2024-08-26 20:01:55 -07:00
Justine Tunney 610c951f71 Fix the build 2024-08-26 16:44:05 -07:00
Justine Tunney 12ecaf8650 Modernize ipv4.games server
The server was originally written before I implemented support for POSIX
thread cancelation. We now use standard pthreads APIs instead of talking
directly to *NSYNC. This means we're no longer using *NSYNC notes, which
aren't as good as the POSIX thread cancelation support I added to *NSYNC
which was only made possible by making *NSYNC part of libc. I believe it
will solve a crash we observed recently with ipv4.games, courtesy of the
individual who goes by the hacker alias Lambro.
2024-08-26 16:05:23 -07:00
Justine Tunney 185e957696 Detect implicit function declarations
This was suppressed recently and it's the worst possible idea when doing
greenfield software development with C. I'm so sorry it slipped through.
If the C standards committee was smart they would change the standard so
that implicit int becomes implicit long. Then problems such as this will
never occur and we could even use traditional C safely if we wanted too.
2024-08-26 15:34:08 -07:00
Justine Tunney ebe1cbb1e3 Add crash proofing to ipv4.games server 2024-08-26 12:57:28 -07:00
Justine Tunney e7b586e7f8 Add preliminary support for cosmocc -mclang
C++ code compiles very slowly with cosmocc, possibly because we're using
LLVM LIBCXX with GCC, and LLVM doesn't work as hard to make GCC go fast.
Therefore, it should be possible, to ask cosmocc to favor Clang over GCC
under the hood. On llamafile, my intention's to use this to make certain
files, e.g. llama.cpp/common.cpp, go from taking 17 seconds to 5 seconds

This new -mclang flag isn't ready for production yet since there's still
the question of how to get Clang to generate SJLJ exception code. If you
use this, then it's recommended you also pass -fno-exceptions.

The tradeoff is we're adding a 121mb binary to the cosmocc distribution.
There are no plans as of yet to fully migrate to Clang since GCC is very
good and has always treated us well.
2024-08-26 12:33:35 -07:00
Justine Tunney 111ec9a989 Fix bug we added to *NSYNC a while ago
This is believed to fix a crash, that's possible in nsync_waiter_free_()
when you call pthread_cond_timedwait(), or nsync_cv_wait_with_deadline()
where an assertion can fail. Thanks ipv4.games for helping me find this!
2024-08-26 12:25:50 -07:00
Justine Tunney f3ce684aef Fix getpeername() bug on Windows
The WIN32 getpeername() function returns ENOTCONN when it uses connect()
the SOCK_NONBLOCK way. So we simply store the address, provided earlier.
2024-08-25 11:28:53 -07:00
Justine Tunney 908b7a82ca Add VSCode settings 2024-08-25 11:02:31 -07:00
Justine Tunney bb06230f1e Avoid linker conflicts on DescribeFoo symbols
These symbols belong to the user. It caused a confusing error for Blink.
2024-08-24 18:10:22 -07:00
Justine Tunney 38cc4b3c68 Get rid of some legacy code 2024-08-24 17:53:30 -07:00
Justine Tunney 37ca1badaf Make Emacs load 2x faster 2024-08-23 20:08:05 -07:00
Justine Tunney 1a9f82bc9f Romanize Hindi, Yiddish, Arabic, Cyrillic, etc. 2024-08-20 16:27:16 -07:00
Justine Tunney df1aee7ce5 Upgrade superconfigure and monorepo toolchain
See #1260
2024-08-20 08:46:21 -07:00
Justine Tunney 2d44142444 Get Meson builds working
See #917
2024-08-19 08:40:18 -07:00
Justine Tunney 4bbc16e2cc Add helpful error messages 2024-08-19 07:28:49 -07:00
Justine Tunney 863c704684 Add string similarity function 2024-08-17 16:45:07 -07:00
Justine Tunney 60e697f7b2 Move LoadZipArgs() to cosmo.h 2024-08-17 12:06:27 -07:00
Justine Tunney 4389f4709a Expose wmempcpy() to _GNU_SOURCE 2024-08-17 08:13:22 -07:00
Justine Tunney ca2c30c977 Release redbean v3.0.0 2024-08-17 06:45:35 -07:00
Justine Tunney eb6e96f036 Change InfoZIP to not auto-append .zip to pathname 2024-08-17 06:45:23 -07:00
Justine Tunney 77be460290 Make Windows REPLs great again 2024-08-17 06:32:10 -07:00
Justine Tunney 8e14b27749 Make fread() more consistent with glibc 2024-08-17 02:57:22 -07:00
Justine Tunney 1d532ba3f8 Disable some anti-Musl Lua tests 2024-08-17 02:20:08 -07:00
Justine Tunney b2a1811c01 Add missing pragma 2024-08-16 21:49:28 -07:00
Justine Tunney 2eda50929b Add stdfloat header
Fixes #1260
2024-08-16 21:38:00 -07:00
Justine Tunney 098638cc6c Fix pthread_kill_test flake on qemu 2024-08-16 21:18:26 -07:00
Justine Tunney 732554ce3a Release Cosmopolitan v3.7.1 2024-08-16 11:56:47 -07:00
Gavin Hayes 914d521090 Fix relative Windows path normalization (#1261)
Fixes #1223
2024-08-16 11:55:49 -07:00
Justine Tunney 11d9fb521d Make atomics faster on aarch64
This change implements the compiler runtime for ARM v8.1 ISE atomics and
gets rid of the mandatory -mno-outline-atomics flag. It can dramatically
speed things up, on newer ARM CPUs, as indicated by the changed lines in
test/libc/thread/footek_test.c. In llamafile dispatching on hwcap atomic
also shaved microseconds off synchronization barriers.
2024-08-16 11:14:46 -07:00
Justine Tunney de0cde8def Release Cosmopolitan v3.7.0 2024-08-16 07:43:10 -07:00
Justine Tunney 5bd22aef12 Experiment with supporting Windows Arm64 natively
So far I haven't found any way to run native Arm64 code on Windows Arm64
without using MSVC. When I build a PE binary from scratch that should be
a valid Windows Arm64 program, the OS refuses to run it. Possibly due to
requiring additional content like XML manifests or relocation or control
flow integrity data that isn't normally required on x64. I've also tried
using VirtualAlloc2() to JIT an Arm64 native function, but VirtualAlloc2
always fails with invalid parameter. I tried using MSVC to create an ARM
DLL that my x64 emulated program can link at runtime, to pass a function
pointer with ARM code, but LoadLibrary() rejects ARM DLLs as invalid exe

The only option left, is likely to write a new program like ape/ape-m1.c
which can be compiled by MSVC to load and run an AARCH64 ELF executable.
The emulated x64 binary would detect emulation using IsWow64Process2 and
then drop the loader executable in a temporary folder, and re-launch the
original executable, using the Arm64 segments of the cosmocc fat binary.
2024-08-16 06:43:59 -07:00
Justine Tunney 1671283f1a Avoid clobbering errno 2024-08-15 23:54:14 -07:00
Justine Tunney 0a79c6961f Make malloc scalable on all platforms
It turns out sched_getcpu() didn't work on many platforms. So the system
call now has tests and is well documented. We now employ new workarounds
on platforms where it isn't supported in our malloc() implementation. It
was previously the case that malloc() was only scalable on Linux/Windows
for x86-64. Now the other platforms are scalable too.
2024-08-15 23:32:53 -07:00
Justine Tunney 3fd275f59f Import optimized routines changes to exp10 2024-08-15 18:37:33 -07:00
Justine Tunney 2045e87b7c Fix build issues 2024-08-15 18:37:33 -07:00
Jacob Hummer 24666e121d add nightly cosmocc (artifact) builds (#1254)
Using https://nightly.link/ with GitHub actions artifacts you can have a
nightly build (but not a _release_ -- there's no releasing or
pre-releasing happening) of cosmocc.

Example URL if this PR were merged:


https://nightly.link/jart/cosmopolitan/workflows/nightly-cosmocc/master/cosmocc.zip

Or you can just download it directly from the GitHub "Actions"
https://github.com/jart/cosmopolitan/actions workflow summary page of a
particular run

example from my own fork:


![image](https://github.com/user-attachments/assets/8ba708dd-8289-4f8b-932c-cf535ee86f62)
could download by clicking on the artifact

or by using third-party service to provide a link for unauthenticated
requests (like wget or curl)

https://nightly.link/jcbhmr/cosmopolitan/workflows/tool-cosmocc-package-sh/master/cosmocc.zip

this would be useful for users who don't want to or can't figure out how
to build cosmocc themselves (like Windows) but still want to use a
nightly build since a fix hasn't been released as a release version yet.

this would also be a good way to test the release process but instead of
pushing the `cosmocc.zip` to _wherever it goes now_ you publish it as a
github actions artifact for the very few nightly bleeding edge users to
use & test.

you don't have to use https://nightly.link or recommend it or anything;
i just know its a cool way to wget or curl the URLs instead of
downloading it via your browser web UI. particularly useful for
remote/ssh/web-ide development.
2024-08-15 00:47:40 -04:00
Justine Tunney ff1a0d1c40 Upgrade to superconfigure z0.0.51 2024-08-04 14:59:53 -07:00
Justine Tunney 7f0db3e3b9 Add last commit to .git-blame-ignore-revs 2024-08-04 12:54:30 -07:00
Justine Tunney 31194165d2 Remove .internal from more header filenames 2024-08-04 12:52:25 -07:00
Justine Tunney c265c17d54 Fix the build 2024-08-04 07:08:28 -07:00
Justine Tunney 7499367060 Ignore -Wimplicit-function-declaration in cosmocc 2024-08-03 21:36:36 -07:00
Justine Tunney 3f26dfbb31 Share file offset across execve() on Windows
This is a breaking change. It defines the new environment variable named
_COSMO_FDS_V2 which is used for inheriting non-stdio file descriptors on
execve() or posix_spawn(). No effort has been spent thus far integrating
with the older variable. If a new binary launches the older ones or vice
versa they'll only be able to pass stdin / stdout / stderr to each other
therefore it's important that you upgrade all your cosmo binaries if you
depend on this functionality. You'll be glad you did because inheritance
of file descriptors is more aligned with the POSIX standard than before.
2024-08-03 17:48:00 -07:00
Justine Tunney 761c6ad615 Share file offset across processes
This change ensures that if a file descriptor for an open disk file gets
shared by multiple processes within a process tree, then lseek() changes
will be visible across processes, and read() / write() are synchronized.
Note this only applies to Windows, because UNIX kernels already do this.
2024-08-03 01:39:11 -07:00
Justine Tunney a80ab3f8fe Implement bf16 compiler runtime library 2024-08-02 02:04:53 -07:00
Gautham 9ebacb7892 Convert GCC 14 errors back to warnings (#1247)
https://gcc.gnu.org/gcc-14/porting_to.html#warnings-as-errors

Changing these to warnings helps build code with `cosmocc`. Perhaps this
can be a patch to `cosmocc` or skipped entirely.
2024-08-01 21:42:40 -07:00
Justine Tunney 6ac3d3b804 Add precompiled header support to cosmocc 2024-07-31 06:04:41 -07:00
Justine Tunney f8cfc89eba Allow -c to be specified with -E in cosmocc 2024-07-31 02:09:15 -07:00
Justine Tunney 4ed4a1095a Improve build latency 2024-07-31 01:21:27 -07:00
Justine Tunney 8d8aecb6d9 Avoid legacy instruction penalties on x86 2024-07-31 01:02:38 -07:00
Justine Tunney 1fba310e22 Delete mislocated headers 2024-07-31 01:01:00 -07:00
Justine Tunney bb815eafaf Update Musl Libc code
We now have implement all of Musl's localization code, the same way that
Musl implements localization. You may need setlocale(LC_ALL, "C.UTF-8"),
just in case anything stops working as expected.
2024-07-30 22:51:29 -07:00
Justine Tunney d0360bf4bd Introduce cosmoranlib 2024-07-29 19:18:52 -07:00
Justine Tunney 8cdb3e136b Check in ruler summation experiments 2024-07-29 18:02:16 -07:00
Justine Tunney 3dab207351 Remove mkfifo() prototype 2024-07-29 07:42:54 -07:00
Justine Tunney 1092d9b7e8 Remove old usages of %n 2024-07-29 01:39:36 -07:00
Justine Tunney d40acc60b1 Detect more x86 features 2024-07-29 00:16:29 -07:00
Justine Tunney cf1559c448 Remove __threaded variable 2024-07-28 23:43:30 -07:00
Justine Tunney 01b09bc817 Support printf %n directive 2024-07-28 22:27:06 -07:00
Justine Tunney c1a0b017e9 Fix the build 2024-07-28 21:02:04 -07:00
Justine Tunney 77d3a07ff2 Fix std::filesystem
This change makes a second pass, at fixing the errno issue with libcxx's
filesystem code. Previously, 89.01% of LLVM's test suite was passing and
now 98.59% of their tests pass. Best of all, it's now possible for Clang
to be built as a working APE binary that can to compile the Cosmopolitan
repository. Please note it has only been vetted so far for some objects,
and more work would obviously need to be done in cosmo, to fix warnings.
2024-07-28 17:31:21 -07:00
Justine Tunney 0d7c272d3f Don't use sendfile() in libcxx 2024-07-28 17:31:21 -07:00
Justine Tunney 18964e5d76 Fix remove() directory on Windows 2024-07-28 17:31:21 -07:00
Justine Tunney e18fe1e112 Freshen build/bootstrap/cocmd
See https://news.ycombinator.com/item?id=41055121
2024-07-27 23:22:11 -07:00
Justine Tunney 8621034d42 Release Cosmopolitan v3.6.2 2024-07-27 20:20:54 -07:00
Justine Tunney 32292aee84 Fix build determinism issue 2024-07-27 19:03:32 -07:00
Justine Tunney 81dd9639ed Fix bug in tool/decode/zip 2024-07-27 18:34:21 -07:00
Justine Tunney f147d3dde9 Fix some static analysis issues 2024-07-27 09:16:54 -07:00
Justine Tunney fb54604b31 Upgrade to superconfigure z0.0.48
Our cosmocc binaries are now built with GCC 14.1, using the Cosmo commit
efb3a34608 from yesterday.

GCC is now configured using --enable-analyzer so you can use -fanalyzer.
2024-07-27 08:35:36 -07:00
Justine Tunney cdfcee51ca Properly serialize fork() operations
This change solves an issue where many threads attempting to spawn forks
at once would cause fork() performance to degrade with the thread count.
Things got real nasty on NetBSD, which slowed down the whole test fleet,
because there's no vfork() and we're forced to use fork() in our server.

   threads      count task
         1       1062 fork+exit+wait
         2        668 fork+exit+wait
         4         66 fork+exit+wait
         8         19 fork+exit+wait
        16         22 fork+exit+wait
        32         16 fork+exit+wait

Things are now much less bad on NetBSD, but not great, since it does not
have futexes; we rely on its semaphore file descriptors to do conditions

   threads      count task
         1       1085 fork+exit+wait
         2        842 fork+exit+wait
         4        532 fork+exit+wait
         8        400 fork+exit+wait
        16        276 fork+exit+wait
        32         66 fork+exit+wait

With OpenBSD which also lacks vfork(), things were just as bad as NetBSD

   threads      count task
         1        584 fork+exit+wait
         2        687 fork+exit+wait
         4        206 fork+exit+wait
         8         24 fork+exit+wait
        16         33 fork+exit+wait
        32         26 fork+exit+wait

But since OpenBSD has futexes fork() works terrifically thanks to *NSYNC

   threads      count task
         1        525 fork+exit+wait
         2        580 fork+exit+wait
         4        451 fork+exit+wait
         8        479 fork+exit+wait
        16        408 fork+exit+wait
        32        373 fork+exit+wait

This issue would most likely only manifest itself, when pthread_atfork()
callers manage to slip a spin lock into the outermost position of fork's
list of locks. Since fork() is very slow, a spin lock can be devastating

Needless to say vfork() rules and anyone who says differently is kidding
themselves. Look at what a FreeBSD 14.1 virtual machine with equal specs
can do over the course of three hundred milliseconds.

   threads      count task
         1       2559 vfork+exit+wait
         2       5389 vfork+exit+wait
         4      34933 vfork+exit+wait
         8      43273 vfork+exit+wait
        16      49648 vfork+exit+wait
        32      40247 vfork+exit+wait

So it's a shame that so few OSes support vfork(). It creates an unsavory
situation, where someone wanting to build a server that spawns processes
would be better served to not use threads and favor a multiprocess model
2024-07-27 08:23:44 -07:00
Justine Tunney 18a620cc1a Make some improvements of little consequence 2024-07-27 08:20:18 -07:00
Justine Tunney 690d3df66e Expand the virtual address space on Windows 2024-07-27 08:19:05 -07:00
Justine Tunney efb3a34608 Fix package.sh build error 2024-07-26 06:57:24 -07:00
Justine Tunney 642e9cb91a Introduce cosmocc flags -mdbg -mtiny -moptlinux
The cosmocc.zip toolchain will now include four builds of the libcosmo.a
runtime libraries. You can pass the -mdbg flag if you want to debug your
cosmopolitan runtime. You can pass the -moptlinux flag if you don't want
windows code lurking in your binary. See tool/cosmocc/README.md for more
details on how these flags may be used and their important implications.
2024-07-26 05:10:25 -07:00
Justine Tunney 59692b0882 Make spinlocks faster (take two)
This change is green on x86 and arm test fleet.
2024-07-26 00:45:24 -07:00
Justine Tunney 02e1cbcd00 Revert "Make spin locks go faster"
This reverts commit c8e25d811c.
2024-07-25 22:24:32 -07:00
Justine Tunney 0679cfeb41 Fix build 2024-07-25 22:12:08 -07:00
Justine Tunney edd5e2c8e3 Expose gethostbyname() 2024-07-25 19:21:35 -07:00
Justine Tunney c8e25d811c Make spin locks go faster 2024-07-25 17:37:11 -07:00
Justine Tunney a31d5ea399 Remove cosmoc++ compiler warning 2024-07-25 15:20:41 -07:00
Justine Tunney 7d88343973 Release Cosmopolitan v3.6.1 2024-07-25 13:34:02 -07:00
Justine Tunney 3915ca0f71 Have cosmocc define -Wno-implicit-int
GCC 13+ changed its policies to be very aggressive about breaking builds
that have anything resembling K&R C. I very strongly disagree with these
decisions. Users who think their compiler should should also be a linter
are perfectly welcome to opt-in to -Wimplicit-int.
2024-07-25 05:51:57 -07:00
Justine Tunney 2c4b88753b Add special errno handling to libcxx 2024-07-25 01:23:02 -07:00
Justine Tunney 0f486a13c8 Fix fdlibm license 2024-07-24 20:42:08 -07:00
Justine Tunney 1020dd41cc bzero() should be defined without special defines 2024-07-24 16:15:30 -07:00
Justine Tunney d3a13e8d70 Improve lock hierarchy
- NetBSD no longer needs a spin lock to create semaphores
- Windows fork() now locks process manager in correct order
2024-07-24 16:05:48 -07:00
Justine Tunney 7ba9a73840 Remove more _Atomic keywords from public headers
It's been thirteen years and C++ still hasn't implemented this wonderful
simple builtin keyword. In C++23 a solution was provided for making this
work in C++ which is libcxx's stdatomic.h. Including that header schleps
in literally 253 unique header files!! Many of the header files it needs
are libc header files like pthread.h where we need to have the _Atomic()
keyword, but since <atomic> depends on pthreads we can't have it include
the <stdatomic.h> header that defines _Atomic for C++ users, and instead
we simply make the type non-atomic, hoping and praying only C code shall
use those internal data structures. This just shows how STL clowns can't
be trusted to define the innermost primitives of a language. They should
instead be focusing on being the best at algorithms and data structures.
2024-07-24 13:56:03 -07:00
Justine Tunney 4a1ae86124 Make some code build faster 2024-07-24 12:26:52 -07:00
Justine Tunney 5dd7ddb9ea Remove bad defines from early days of project
These definitions were causing issues with building LLVM. It is possible
they also caused crashes we've seen with our MacOS ARM64 OpenMP support.
2024-07-24 12:11:21 -07:00
Justine Tunney f25fbbaaeb Use libcxx abi v1 2024-07-24 09:49:48 -07:00
Justine Tunney fbc4b03d4c Restore support for AMD K8 2024-07-24 08:59:29 -07:00
Justine Tunney e398f3887c Make more improvements to threads and mappings
- NetBSD should now have faster synchronization
- POSIX barriers may now be shared across processes
- An edge case with memory map tracking has been fixed
- Grand Central Dispatch is no longer used on MacOS ARM64
- POSIX mutexes in normal mode now use futexes across processes
2024-07-24 01:19:54 -07:00
Justine Tunney 2187d6d2dd Fix MODE=optlinux for GitHub Actions 2024-07-23 04:17:22 -07:00
Justine Tunney 0602ff6bab Fix MODE=optlinux and MODE=tiny builds 2024-07-23 04:04:19 -07:00
Justine Tunney 5660ec4741 Release Cosmopolitan v3.6.0
This release is an atomic upgrade to GCC 14.1.0 with C23 and C++23
2024-07-23 03:28:19 -07:00
Justine Tunney 62ace3623a Release Cosmopolitan v3.5.9 2024-07-22 21:02:40 -07:00
Justine Tunney 6e809ee49b Add unit test for process shared conditions 2024-07-22 18:48:54 -07:00
Justine Tunney 61c36c1dd6 Allow pthread_condattr_setpshared() to set shared 2024-07-22 18:41:45 -07:00
Justine Tunney 0a9a6f86bb Support process shared condition variables 2024-07-22 16:35:29 -07:00
Justine Tunney 3de6632be6 Graduate some clock_gettime() constants to #define
- CLOCK_THREAD_CPUTIME_ID
- CLOCK_PROCESS_CPUTIME_ID

Cosmo now supports the above constants universally across supported OSes
therefore it's now safe to let programs detect their presence w/ #ifdefs
2024-07-22 07:14:35 -07:00
Justine Tunney c83ec5fdd9 Fix more issues 2024-07-22 02:28:41 -07:00
Justine Tunney 23611cd854 Talk more about alignment 2024-07-22 02:11:09 -07:00
Justine Tunney 62a97c919f Fix typos in APE specification
Fixes #1244
2024-07-22 01:41:44 -07:00
Justine Tunney 16d244614e Mention red zone in APE ABI specification 2024-07-21 21:18:12 -07:00
Justine Tunney 41cc053419 Add more content to APE specification 2024-07-21 20:47:24 -07:00
Justine Tunney 5d2d9e9640 Add back missing TlsAlloc() call
Cosmopolitan Libc once called this important function although somewhere
along the way, possibly in a refactoring, it got removed and __tls_alloc
has always been zero ever since.
2024-07-21 20:45:27 -07:00
Justine Tunney e08a4cd99e Release Cosmopolitan v3.5.8 2024-07-21 17:01:33 -07:00
Justine Tunney 7ebaff34c6 Fix ctype.h and wctype.h 2024-07-21 15:54:17 -07:00
Justine Tunney e7be5a5e2b Upgrade to latest superconfigure 2024-07-21 15:30:47 -07:00
Justine Tunney 30afd6ddbb Improve multithreading 2024-07-21 14:40:45 -07:00
Justine Tunney d3167126aa Fix regression with last commit 2024-07-20 16:43:48 -07:00
Justine Tunney d3f87f4c64 Upgrade to cosmocc v3.5.7 2024-07-20 11:21:26 -07:00
Justine Tunney 29ce25c767 Start writing formal specification for APE 2024-07-20 10:04:22 -07:00
Justine Tunney 7996bf67b5 Release Cosmopolitan v3.5.7 2024-07-20 03:48:57 -07:00
Justine Tunney 626a5d02ee Add missing lock statement 2024-07-20 03:47:22 -07:00
Justine Tunney 527aaa41eb Prevent MODE=tiny ShowCrashReports() looping 2024-07-20 03:34:37 -07:00
Justine Tunney 421a819d88 Fix bug in munmap_test 2024-07-20 03:23:37 -07:00
Justine Tunney 3374cbba73 Release Cosmopolitan v3.5.6 2024-07-20 02:43:10 -07:00
Justine Tunney 2018cac11f Use better memory strategy on Windows
Rather than using the the rollo global to pick addresses, we select them
randomly now using a conservative vaspace.
2024-07-20 02:20:03 -07:00
Justine Tunney 6a5d4ed65b Fix bug with disabling sigaltstack() 2024-07-20 01:00:16 -07:00
Justine Tunney 493ffc9b7f Release Cosmopolitan v3.5.5 2024-07-19 22:33:17 -07:00
Justine Tunney 101fb3d9b3 Make some new Windows 10 memory APIs available 2024-07-19 22:26:49 -07:00
Justine Tunney f6ba270ff3 Add last commit to .git-blame-ignore-revs 2024-07-19 19:38:53 -07:00
Justine Tunney 86d884cce2 Get rid of .internal.h convention in LIBC_INTRIN 2024-07-19 19:38:00 -07:00
Justine Tunney 0ed916ad5c Fix a bug in example code 2024-07-19 19:11:28 -07:00
Justine Tunney 1029dcc597 Reduce default stack size from 256kb to 81kb
This is the same as Musl Libc. Please note it only applies to threads.
2024-07-19 14:18:06 -07:00
Ikko Eltociear Ashimine c697133a2d Fix typo in accept4-sysv.c (#1235) 2024-07-19 05:46:29 -07:00
Justine Tunney 1ff037df3c Add some documentation 2024-07-19 04:46:26 -07:00
Justine Tunney 567d8fe32d Create variables for page size 2024-07-18 21:16:53 -07:00
Justine Tunney 23dfb79d33 Fix minor suboptimalities in memory manager 2024-07-18 19:19:51 -07:00
Justine Tunney 76cea6c687 Squeeze more performance out of memory manager 2024-07-08 03:08:42 -07:00
Justine Tunney 63065cdd70 Make a test less intensive by default 2024-07-07 19:35:09 -07:00
Justine Tunney 3f2a1b696e Fix greenbean example
The memory leak detector was crashing. When using gc() you shouldn't use
the CheckForMemoryLeaks() function from inside the same function, due to
how it runs the atexit handlers.
2024-07-07 17:52:33 -07:00
Justine Tunney f590e96abd Work around QEMU bugs 2024-07-07 15:42:46 -07:00
Justine Tunney f7780de24b Make realloc() go 100x faster on Linux/NetBSD
Cosmopolitan now supports mremap(), which is only supported on Linux and
NetBSD. First, it allows memory mappings to be relocated without copying
them; this can dramatically speed up data structures like std::vector if
the array size grows larger than 256kb. The mremap() system call is also
10x faster than munmap() when shrinking large memory mappings.

There's now two functions, getpagesize() and getgransize() which help to
write portable code that uses mmap(MAP_FIXED). Alternative sysconf() may
be called with our new _SC_GRANSIZE. The madvise() system call now has a
better wrapper with improved documentation.
2024-07-07 12:40:30 -07:00
Justine Tunney 196942084b Recomment out accidental code 2024-07-06 19:57:47 -07:00
Justine Tunney 6be030cd7c Fix MODE=tinylinux build 2024-07-06 01:51:08 -07:00
Justine Tunney 8c645fa1ee Make mmap() scalable
It's now possible to create thousands of thousands of sparse independent
memory mappings, without any slowdown. The memory manager is better with
tracking memory protection now, particularly on Windows in a precise way
that can be restored during fork(). You now have the highest quality mem
manager possible. It's even better than some OSes like XNU, where mmap()
is implemented as an O(n) operation which means sadly things aren't much
improved over there. With this change the llamafile HTTP server endpoint
at /tokenize with a prompt of 50 tokens is now able to handle 2.6m r/sec
2024-07-05 23:26:00 -07:00
Justine Tunney 3756870635 Implement new red-black tree 2024-07-05 12:56:03 -07:00
Justine Tunney fc65422660 Remove __mmap() and __munmap() 2024-07-05 12:55:46 -07:00
Justine Tunney 01587de761 Simplify memory manager 2024-07-05 05:47:15 -07:00
Justine Tunney 5a9a08d1cf Fix regression in elf2pe program 2024-07-04 04:02:20 -07:00
Justine Tunney bd6d9ff99a Get deathstar demo working again on metal 2024-07-04 03:44:17 -07:00
Justine Tunney 15ea0524b3 Reduce code size of mandatory runtime
This change reduces o/tiny/examples/life from 44kb to 24kb in size since
it avoids linking mmap() when unnecessary. This is important, to helping
cosmo not completely lose touch with its roots.
2024-07-04 02:50:20 -07:00
Justine Tunney fdab49b30e Demonstrate signal safety of recursive mutexes 2024-07-04 02:47:52 -07:00
Justine Tunney 135d538b1d Make ctl::set use 30% less memory than libcxx 2024-07-04 02:46:27 -07:00
Justine Tunney 6dbc3fba18 Add AMD cache sizes to o//tool/viz/cpuid 2024-07-04 02:45:45 -07:00
Justine Tunney 70f77aad33 Release Cosmopolitan v3.5.4 2024-07-01 07:17:57 -07:00
Justine Tunney d0cd719375 Make more CTL fixes 2024-07-01 07:17:57 -07:00
Justine Tunney 61370983e1 Complete the Windows TLS fix made in e437bed00 2024-07-01 07:17:57 -07:00
Terror 72511ff0ac [Redbean] Add UuidV7 method (#1213)
To Complete #1140 add UUID version 7 to Redbean
2024-07-01 06:06:56 -07:00
Justine Tunney c1f8d0678c Mark ctl::to_string() noexcept 2024-07-01 05:54:59 -07:00
Justine Tunney e627bfa359 Introduce ctl::to_string() 2024-07-01 05:40:38 -07:00
Justine Tunney acbabedf27 Make CTL definitions less ambiguous 2024-07-01 03:48:28 -07:00
Justine Tunney 239f8ce76e Release Cosmopolitan v3.5.3 2024-07-01 02:07:56 -07:00
Justine Tunney ca4cf67eb8 Include more programs in cosmocc
The Cosmopolitan Compiler Collection now includes the following programs

- `ar.ape` is a faster alternative to `ar rcsD` for creating determistic
  static archives. It's ~10x faster than GNU because it isn't quadratic.
  It'll even outperform LLVM ar by 2x, thanks to writev/copy_file_range.

- `sha256sum.ape` is a faster alternative to the `sha256sum` command. It
  goes 2x faster since it leverages vectorized assembly implementations.

- `resymbol` is a brand new program we invented, like objcopy, that lets
  you rename all the global symbols in a .o file to have a new suffix or
  prefix. In the future, this will be used by cosmocc automatically when
  building -O3 math kernels, that need to be vectorized for all hardware

- `gzip.ape` is a faster version of the `gzip` command, that is included
  by most Linux distros. It gains better performance using Chromium Zlib
  which, once again, includes highly optimized assembly, that Mark Adler
  won't merge into the official MS-DOS compatible zlib codebase.

- `cocmd` is the cosmopolitan shell. It can function as a faster `sh -c`
  alternative than bash and dash as the `SHELL = /opt/cosmocc/bin/cocmd`
  at the top of your Makefile. Please note you should be using the cosmo
  fork of GNU make (already included), since normal make won't recognize
  this as a bourne-compatible shell and remove the execve() optimization
  which makes things slower. In some ways that's true. This doesn't have
  a complete POSIX shell implementation. However it's enough for cosmo's
  mono repo. It also implements faster behaviors in some respects.

The following programs are also introduced, which aren't as interesting.
The main reason why they're here is so Cosmopolitan's mono repo shall be
able to remove build/bootstrap/ in future editions. That way we can keep
build utilities better up to date, without bloating the git history much

- `chmod.ape` for hermeticity
- `cp.ape` for hermeticity
- `echo.ape` for hermeticity
- `objbincopy` is an objcopy-like tool that's used to build ape loader
- `package.ape` is used for strict dependency checking of object graph
- `rm.ape` for hermeticity
- `touch.ape` for hermeticity
2024-07-01 02:05:25 -07:00
Justine Tunney 78d3b86ec7 Fix Android support
Thanks to @aj47 (techfren.net) the new Cosmo memory manager is confirmed
to be working on Android!! The only issue turned out to be forgetting to
update the program address in the linker script. We now know w/ absolute
certainty that APE binaries as complex as llamafile, now work correctly.
2024-07-01 01:06:47 -07:00
Justine Tunney 44191b3f50 Add more type traits to CTL 2024-06-30 20:59:38 -07:00
Justine Tunney e437bed006 Fix crash caused when Windows needs a lot of TLS 2024-06-30 20:53:43 -07:00
Justine Tunney 76957983cf Make POSIX threads improvements
- Ensure SIGTHR isn't blocked in newly created threads
- Use TIB rather than thread_local for thread atexits
- Make POSIX thread keys atomic within thread
- Don't bother logging prctl() to --strace
- Log thread destructor names to --strace
2024-06-30 15:38:59 -07:00
Justine Tunney 387310c659 Fix issue with ctl::vector constructor 2024-06-30 02:26:38 -07:00
Justine Tunney 4cb5e21ba8 Introduce pthread_decimate_np() api
This is useful with CheckForMemoryLeaks().
2024-06-30 02:26:06 -07:00
Justine Tunney 1bf2d8e308 Further improve mmap() locking story
The way to use double linked lists, is to remove all the things you want
to work on, insert them into a new list on the stack. Then once you have
all the work items, you release the lock, do your work, and then lock it
again, to add the shelled out items back to a global freelist.
2024-06-29 17:12:43 -07:00
Justine Tunney 98e684622b Add iostream to CTL 2024-06-29 15:45:09 -07:00
Justine Tunney 617ddfee93 Release Cosmopolitan v3.5.2 2024-06-29 10:58:47 -07:00
Justine Tunney 464858dbb4 Fix bugs with new memory manager
This fixes a regression in mmap(MAP_FIXED) on Windows caused by a recent
revision. This change also fixes ZipOS so it no longer needs a MAP_FIXED
mapping to open files from the PKZIP store. The memory mapping mutex was
implemented incorrectly earlier which meant that ftrace and strace could
cause cause crashes. This lock and other recursive mutexes are rewritten
so that it should be provable that recursive mutexes in cosmopolitan are
asynchronous signal safe.
2024-06-29 10:53:57 -07:00
Justine Tunney 6de12c1032 Upgrade to superconfigure z0.0.44 2024-06-29 05:07:25 -07:00
Justine Tunney a16eb76f5e Fix build break 2024-06-29 04:34:27 -07:00
Justine Tunney 021c53ba32 Add more CTL content 2024-06-28 19:09:54 -07:00
Justine Tunney 38921dc46b Introduce more CTL content
This change introduces accumulate, addressof, advance, all_of, distance,
array, enable_if, allocator_traits, back_inserter, bad_alloc, is_signed,
any_of, copy, exception, fill, fill_n, is_same, is_same_v, out_of_range,
lexicographical_compare, is_integral, uninitialized_fill_n, is_unsigned,
numeric_limits, uninitialized_fill, iterator_traits, move_backward, min,
max, iterator_tag, move_iterator, reverse_iterator, uninitialized_move_n

This change experiments with rewriting the ctl::vector class to make the
CTL design more similar to the STL. So far it has not slowed things down
to have 42 #include lines rather than 2, since it's still almost nothing
compared to LLVM's code. In fact the closer we can flirt with being just
like libcxx, the better chance we might have of discovering exactly what
makes it so slow to compile. It would be an enormous discovery if we can
find one simple trick to solving the issue there instead.

This also fixes a bug in `ctl::string(const string &s)` when `s` is big.
2024-06-27 22:42:32 -07:00
Steven Dee (Jōshin) 054da021d0 ctl::string benchmarking code (#1200) 2024-06-26 21:30:05 -04:00
Justine Tunney 199662071a Make std::random_device use getentropy() 2024-06-24 07:32:07 -07:00
Justine Tunney 572ac7d100 Release Cosmopolitan v3.5.1 2024-06-24 06:54:15 -07:00
Justine Tunney d461c6f47d Do more quality assurance work 2024-06-24 06:53:49 -07:00
Justine Tunney 67b19ae733 Release Cosmopolitan v3.5.0 2024-06-23 22:45:14 -07:00
Justine Tunney c4c812c154 Introduce ctl::set and ctl::map
We now have a C++ red-black tree implementation that implements standard
template library compatible APIs while compiling 10x faster than libcxx.
It's not as beautiful as the red-black tree implementation in Plinko but
this will get the job done and the test proves it upholds all invariants

This change also restores CheckForMemoryLeaks() support and fixes a real
actual bug I discovered with Doug Lea's dlmalloc_inspect_all() function.
2024-06-23 22:27:11 -07:00
Justine Tunney 388e236360 Revert misguided dlmalloc optimization 2024-06-22 09:55:02 -07:00
Justine Tunney f2c8ddbbe3 Fix --strace use-after-free in pthread_join() 2024-06-22 06:05:52 -07:00
Justine Tunney d1d4388201 Delete ASAN
It hasn't been helpful enough to be justify the maintenance burden. What
actually does help is mprotect(), kprintf(), --ftrace and --strace which
can always be counted upon to work correctly. We aren't losing much with
this change. Support for ASAN on AARCH64 was never implemented. Applying
ASAN to the core libc runtimes was disabled many months ago. If there is
some way to have an ASAN runtime for user programs that is less invasive
we can potentially consider reintroducing support. But now is premature.
2024-06-22 05:45:49 -07:00
Justine Tunney 6ffed14b9c Rewrite memory manager
Actually Portable Executable now supports Android. Cosmo's old mmap code
required a 47 bit address space. The new implementation is very agnostic
and supports both smaller address spaces (e.g. embedded) and even modern
56-bit PML5T paging for x86 which finally came true on Zen4 Threadripper

Cosmopolitan no longer requires UNIX systems to observe the Windows 64kb
granularity; i.e. sysconf(_SC_PAGE_SIZE) will now report the host native
page size. This fixes a longstanding POSIX conformance issue, concerning
file mappings that overlap the end of file. Other aspects of conformance
have been improved too, such as the subtleties of address assignment and
and the various subtleties surrounding MAP_FIXED and MAP_FIXED_NOREPLACE

On Windows, mappings larger than 100 megabytes won't be broken down into
thousands of independent 64kb mappings. Support for MAP_STACK is removed
by this change; please use NewCosmoStack() instead.

Stack overflow avoidance is now being implemented using the POSIX thread
APIs. Please use GetStackBottom() and GetStackAddr(), instead of the old
error-prone GetStackAddr() and HaveStackMemory() APIs which are removed.
2024-06-22 05:45:11 -07:00
Justine Tunney 7f6d0b8709 Support -x LANG flag in cosmocc 2024-06-20 23:55:46 -07:00
Steven Dee (Jōshin) d7b1919b29 ctl::unique_ptr improvements and cleanup (#1221)
Explicitly value-initializes the deleter, even though I have not found a
way to get the deleter to act like it’s been default-initialized in unit
tests so far.

Uses auto in reset. The static cast is apparently not needed (unless I’m
missing some case I didn’t think of.)

Implements the general move constructor - turns out that the reason this
didn’t work before was that default_delete<U> was not move constructible
from default_delete<T>.

Drop inline specifiers from functions defined entirely inside the struct
definition since they are implicitly inline.

* Cleans up reset to match spec

Remove the variants from the T[] specialization. Also follow the spec on
the order of operations in reset, which may matter if we are deleting an
object that has a reference to the unique_ptr that is being reset. (?)

* Tests Base/Derived reset.

* Adds some constexpr declarations.

* Adds default_delete specialization for T[].

* Makes parameters const.
2024-06-20 18:44:31 -04:00
Steven Dee (Jōshin) f86e6f8eb0 Make new.cc definitions weak (#1233)
The STL says that these should be replaceable by user code.

new.cc now defines only a few direct functions (including a free wrapper
that perplexingly is needed since g++ didn’t want to alias "free".) Now,
all of the operators are weak references to those functions.
2024-06-20 15:20:54 -04:00
Steven Dee (Jōshin) 7e780e57d4 More ctl::string optimization (#1232)
Moves some isbig checks into string.h, enabling smarter optimizations to
be made on small strings. Also we no longer zero out our string prior to
calling the various constructors, buying back the performance we lost on
big strings when we made the small-string optimization. We further add a
little optimization to the big_string copy constructor: if the string is
using half or more of its capacity, then we don’t recompute capacity and
just take the old string’s. As well, the copy constructor always makes a
small string when it will fit, even if copied from a big string that got
truncated.

This also reworks the test to follow the idiom adopted elsewhere re stl,
and adds a helper function to tell if a string is small based on data().
2024-06-20 14:52:12 -04:00
Steven Dee (Jōshin) 9a5a13854d CTL: utility.h, use ctl::swap in string (#1227)
* Add ctl utility.h

Implements forward, move, swap, and declval. This commit also adds a def
for nullptr_t to cxx.inc. We need it now because the CTL headers stopped
including anything from libc++, so we no longer get their basic types.

* Use ctl::swap in string

The STL spec says that swap is located in the string_view header anyawy.
Performance-wise this is a noop, but it’s slightly cleaner.
2024-06-19 01:00:59 -04:00
Steven Dee (Jōshin) a795017416 Fix c.inc _Atomic define for C++ (#1231)
c.inc (AFAICT erroneously) defined _Atomic(t) as `volatile t *`, when it
should have just said `volatile t`, when __STDC_VERSION__ was too small.
This happens when we’re compiling C++, but in C++11, _Atomic is a define
supplied by the STL rather than a keyword supplied by the compiler. Wait
though, it gets better: in C++11, _Atomic hooks you into the morass that
is stdatomic.h, and ultimately refers everything back to std::atomic<T>.

The gory, horrifying details are in libcxx's __atomic/cxx_atomic_impl.h.
The tldr is that for our purposes it’s fine to just say volatile and use
the normal libc/intrin/atomic.h functions.
2024-06-17 21:12:02 -07:00
Brian 8e3b361aeb Remove testing label from labeler.yml (#1220)
#1219 had an issue with noisy testing label. Investigated if there is a
syntax to make it more exclusive, but turns out there isn't one yet. So
let's remove an manually add it in as needed.

Fixes #1219.
2024-06-16 11:45:57 -04:00
Steven Dee (Jōshin) f9dd5683a4 Implement ctl::unique_ptr (#1216)
The way unique_ptr is supposed to work is as a purely compile-time check
that your raw pointers are getting deleted when they go out of scope. It
should ideally emit the same exact machine code as if you were using raw
pointers with manual deletes.

Part of what this means is that under normal circumstances, a unique_ptr
shouldn’t take up more space than a raw pointer - in other words, sizeof
unique_ptr<T> should == sizeof(T*).

The present PR doesn’t bother with the specialization for array types. I
also left a couple other parts of the STL API unimplemented. I’d love to
see someone else implement these, or I’ll get to them at some point.
2024-06-15 22:54:52 -04:00
Steven Dee (Jōshin) e38a6e7996 ctl string const/value tweaks (#1218)
The mangled name of a C++ function will typically not vary by const-ness
of a by-value parameter; in other words, there is no meaning to a const-
qualified by-value parameter in a function prototype. However, the const
keyword _does_ matter at function _definition_ time, like it does with a
variable declared in the body. So for prototypes, we strip out const for
by-value parameters; but for definitions, we leave them alone.

At function definition (as opposed to prototype), we add const to values
in parameters by default, unless we’re going to mutate them.

This commit also changes a couple of const string_view& to be simply by-
value string_view. A string_view is only two words; it rarely ever makes
sense to pass one by reference if it’s not going to be mutated.
2024-06-15 18:09:30 -07:00
Steven Dee (Jōshin) ddfaf3fee0 Explain the clang-format off in new.h 2024-06-15 17:17:04 -07:00
Michael Lenaghan 0dbf01bf1d Bring Lua to 5.4.6. (#1214)
This essentially re-does the work of #875 on top of master.

This is what I did to check that Cosmo's Lua extensions still worked:

```
$ build/bootstrap/make MODE=aarch64 o/aarch64/third_party/lua/lua
$ ape o/aarch64/third_party/lua/lua
>: 10
10
>: 010
8
>: 0b10
2
>: string.byte("\e")
27
>: "Hello, %s" % {"world"}
Hello, world
>: "*" * 3
***
```

`luaL_traceback2` was used to show the stack trace with parameter
values; it's used in `LuaCallWithTrace`, which is used in Redbean to run
Lua code. You should be able to see the extended stack trace by running
something like this: `redbean -e "function a(b)c()end a(2)"` (with
"params" indicating the extended stack trace):

```
stack traceback:
 [string "function a(b)c()end a(2)"]:1: in function 'a', params: b = 2;
 [string "function a(b)c()end a(2)"]:1: in main chunk
```
@pkulchenko confirmed that I get the expected result with the updated
code.

This is what I did to check that Lua itself still worked:

```
$ cd third_party/lua/test/
$ ape ../../../o/aarch64/third_party/lua/lua all.lua
```

There's one test failure, in `files.lua`:

```
***** FILE 'files.lua'*****
testing i/o
../../../o/aarch64/third_party/lua/lua: files.lua:84: assertion failed!
stack traceback:
[C]: in function 'assert'
files.lua:84: in main chunk
(...tail calls...)
all.lua:195: in main chunk
[C]: in ?
.>>> closing state <<<
```

That isn't a result of these changes; the same test is failing in
master.

The failure is here:

```lua
if not _port then   -- invalid seek
  local status, msg, code = io.stdin:seek("set", 1000)
  assert(not status and type(msg) == "string" and type(code) == "number")
end
```

The test expects a seek to offset 1,000 on stdin to fail — but it
doesn't. `status` ends up being the new offset rather than `nil`.

If I comment out that one test, the remaining tests succeed.
2024-06-15 20:13:08 -04:00
Jōshin 3a599bfbe1 Update .git-blame-ignore-revs 2024-06-15 13:36:07 -07:00
Jōshin 89fc95fefd Rerun clang-format on the repo (#1217)
🚨 clang-format changes output per version!

This is with version 19.0.0. The modifications seem to be fixing the old
version’s errors - mainly involving omitted whitespace around binary ops
and inserted whitespace between goto labels and colons (if followed by a
curly brace.)

Also fixes a few mistakes made by e.g. someone (ahem) forgetting to pass
his ctl/string.h modifications through it.

We should add this to .git-blame-ignore-revs once we have its final hash
on master.
2024-06-15 16:34:48 -04:00
Jōshin 8e37ee2598 ctl::string cleanup (#1215)
We now fully initialize a ctl::string’s memory, so that it is always set
to a well-defined value, thus making it always safe to memcpy out of it.
This incidentally makes our string::swap function legal, which it wasn’t
before. This also saves us a store in string::reserve.

Now that we have made both big_string and small_string POD, I believe it
is safe to elide the launder calls, and have done so, thus cleaning up a
lot of the blob-related code.

I also got rid of set_big_capacity and replaced it with a set_big_string
that leaves us in a well-defined state afterwards. This function also is
able to be somewhat simpler; rather than delicate bit-twiddling, it just
reaches straight into blob and rewrites it wholesale.

Overall, this shaves about 1–2ns off of most benchmarks, and adds 1ns to
only one of them - creating a string from a char *.
2024-06-15 10:49:52 -07:00
Jōshin d9b4f647d8 Uncomment swap test (#1210) 2024-06-10 21:51:19 -07:00
Jōshin 0dde3a0e70 Get rid of preprocessor stuff in test (#1202)
Also it's a bit more idiomatic to say s.npos rather than string::npos.
2024-06-10 07:00:37 -07:00
Jōshin 32643e9fa7 Decouple swap from std (#1211)
This allows you to implement your own swap function without it having to
be part of the std namespace. std::swap is still used if it's available.
2024-06-10 03:40:17 -07:00
Jōshin 0a92c78035 Remove erroneous ctl:: prefixes 2024-06-08 15:07:39 -07:00
Jōshin 118db71121 Provide a minimal new.h for CTL (#1205)
This replaces the STL <new> header. Mainly, it defines a global operator
new and operator delete, as well as the placement versions of these. The
placement versions are required to not get compile errors when trying to
write a placement new statement.

Each of these operators is defined with many, many different variants. A
glance at new.cc is recommended followed by a chaser of the Alexandrescu
talk "std::allocator is to Allocation as std::vector is to Vexation". We
must provide a global-namespace source-level definition of each operator
and it is illegal for any of them to be marked inline, so here we are.

The upshot is that we no longer need to include <new>, and our optional/
vector headers are self-contained.
2024-06-08 15:05:38 -07:00
Alkis Evlogimenos a0410f0170 Make big_string pod (#1204)
`big_string` is not pod which means it needs to be properly constructed
and destroyed. Instead make it POD and destroy it manually in `string`
destructor.
2024-06-08 10:02:33 -07:00
Alkis Evlogimenos d44a7dc603 Fix bugs in in ctl::optional (#1203)
Manually manage the lifetime of `value_` by using an anonymous
`union`. This fixes a bunch of double-frees and double-constructs.

Additionally move the `present_` flag last. When `T` has padding
`present_` will be placed there saving `alignof(T)` bytes from
`sizeof(optional<T>)`.
2024-06-07 20:47:24 -04:00
Jōshin 2ba6b0158f Fix some memory issues with ctl::string (#1201)
There were a few errors in how capacity and memory was being handled for
small strings. The capacity errors meant that small strings would become
big strings too soon, and the memory error introduced undefined behavior
that was caught by CheckMemoryLeaks in our test file but only sometimes.

The crucial change is in reserve: we only copy n bytes into p2, and then
we manually set the null terminator instead of expecting it to have been
there already. (E.g. it might not be there for an empty small string.)

We also fix one other doozy in append when we were exactly at the small-
to-big string boundary: we set the last byte (i.e., the remainder field)
to 0, then decremented it, giving us size_t max. Whoops. We boneheadedly
fix this by setting the 0 byte after we've fixed up the remainder, so it
is at worst a no-op.

Otherwise, capacity now works the same for small strings as it does with
big strings: it's the amount of space available including the null byte.

We test all of this with a new test that only gets included if our class
under test is not std::string (presumably meaning it's ctl::string.) The
test manually verifies that the small string optimization behaves how we
expect.

Since this test checks against std::string, we go ahead and include that
other header from the STL.

Also modifies the new test we introduced to also run on std::string, but
it just does the append without expecting anything about how its data is
stored. We also check that the string has the right value afterwards.
2024-06-07 01:15:37 -04:00
Jōshin f3effcb703 One more SSO erratum from #1199
Making a string_view from a string appears to take about 1.3ns no matter
what. 100% definitely no point deviating from the STL API over that.
2024-06-06 18:01:26 -07:00
Jōshin 03b476f943 Minor small-string errata from #1199
These commits were sitting on a local branch that I neglected to push
before merging. :(

* Use memcpy for string::reserve

* Remove fence comments
2024-06-06 17:56:30 -07:00
Jōshin 8b3e368e9a ctl::string small-string optimization (#1199)
A small-string optimization is a way of reusing inline storage space for
sufficiently small strings, rather than allocating them on the heap. The
current approach takes after an old Facebook string class: it reuses the
highest-order byte for flags and small-string size, in such a way that a
maximally-sized small string will have its last byte zeroed, making it a
null terminator for the C string.

The only flag we have is in the highest-order bit, that says whether the
string is big (set) or small (cleared.) Most of the logic switches based
on the value of this bit; e.g. data() returns big()->p if it's set, else
small()->buf if it's cleared. For a small string, the capacity is always
fixed at sizeof(string) - 1 bytes; we store the length in the last byte,
but we store it as the number of remaining bytes of capacity, so that at
max size, the last byte will read zero and serve as our null terminator.

Morally speaking, our class's storage is a union over two POD C structs.
For now I gravitated towards a slightly more obtuse approach: the string
class itself contains a blob of the right size, and we alias that blob's
pointer for the two structs, taking some care not to run afoul of object
lifetime rules in C++. If anyone wants to improve on this, contributions
are welcome.

This commit also introduces the `ctl::__` namespace. It can't be legally
spelled by library users, and serves as our version of boost's "detail".

We introduced a string::swap function, and we now use that in operator=.
operator= now takes its argument by value, so we never need to check for
the case where the pointers are equal and can just swap the entire store
of the argument with our own, leaving the C++ destructor to free our old
storage afterwards.

There are probably still a few places where our capacity is slightly off
and we grow too fast, although there don't appear to be any where we are
too slow. I will leave these to be fixed in future changes.
2024-06-06 20:50:51 -04:00
Brian df6b384e31 github: add labeler action (#1196)
Bit easier to do this as everything seems to be sorted into logical
folders. You may need to add new labels to support this however.
2024-06-06 05:51:36 -07:00
Brian 280bdec817 github: add issue template (#1195)
copy paste and adapted from llamafile but added a research template as
well for the more technical research tickets
2024-06-06 05:51:06 -07:00
Jōshin 2c5e7ec547 Add terminating :vi on some modelines
Noticed because the settings they specified weren't getting picked up by
editor sessions in those files.
2024-06-05 20:36:55 -07:00
Jōshin fdcb8b2f7e add formatting commit 2024-06-05 16:36:34 -07:00
Jōshin 04c6bc478e vim C++ filetype is still spelled "cpp" 2024-06-05 16:34:47 -07:00
Justine Tunney cc2c1893c5 Fix some nits 2024-06-05 04:05:49 -07:00
Justine Tunney 3093f0e467 Release Cosmopolitan v3.4.0 2024-06-05 03:07:03 -07:00
Justine Tunney 3609f65de3 Make malloc() go 200x faster
If pthread_create() is linked into the binary, then the cosmo runtime
will create an independent dlmalloc arena for each core. Whenever the
malloc() function is used it will index `g_heaps[sched_getcpu() / 2]`
to find the arena with the greatest hyperthread / numa locality. This
may be configured via an environment variable. For example if you say
`export COSMOPOLITAN_HEAP_COUNT=1` then you can restore the old ways.
Your process may be configured to have anywhere between 1 - 128 heaps

We need this revision because it makes multithreaded C++ applications
faster. For example, an HTTP server I'm working on that makes extreme
use of the STL went from 16k to 2000k requests per second, after this
change was made. To understand why, try out the malloc_test benchmark
which calls malloc() + realloc() in a loop across many threads, which
sees a a 250x improvement in process clock time and 200x on wall time

The tradeoff is this adds ~25ns of latency to individual malloc calls
compared to MODE=tiny, once the cosmo runtime has transitioned into a
fully multi-threaded state. If you don't need malloc() to be scalable
then cosmo provides many options for you. For starters the heap count
variable above can be set to put the process back in single heap mode
plus you can go even faster still, if you include tinymalloc.inc like
many of the programs in tool/build/.. are already doing since that'll
shave tens of kb off your binary footprint too. Theres also MODE=tiny
which is configured to use just 1 plain old dlmalloc arena by default

Another tradeoff is we need more memory now (except in MODE=tiny), to
track the provenance of memory allocation. This is so allocations can
be freely shared across threads, and because OSes can reschedule code
to different CPUs at any time.
2024-06-05 02:02:14 -07:00
Justine Tunney 9906f299bb Refactor and improve CTL and other code 2024-06-04 05:45:48 -07:00
Justine Tunney 1d8f37a2f0 Fix the MODE=tiny builds 2024-06-03 10:36:38 -07:00
Justine Tunney e677460d14 Delete .vscode folder
It hasn't been maintained in years. I'm tired of the root level of our
project having an advertisement for Microsoft Visual Studio Code. Your
preferred editor should be Emacs or Vim.
2024-06-03 09:40:45 -07:00
Justine Tunney 4937843f70 Introduce Cosmopolitan Templates Library (CTL) 2024-06-03 09:21:59 -07:00
Justine Tunney b003888696 Make __demangle() heap 10% more compact 2024-06-02 16:18:55 -07:00
Justine Tunney 2ca491dc56 Write more __demangle() tests 2024-06-02 07:37:15 -07:00
Justine Tunney 9aa353d88b Document __demangle() and fix a const func ptr bug 2024-06-02 04:15:48 -07:00
Justine Tunney c67faf61df Delete some unintentional code 2024-06-01 20:36:58 -07:00
Justine Tunney 165c6b37e2 Add C++ demangling to privileged runtime
Cosmo will now print C++ symbols correctly in --ftrace logs and
backtraces. Doing this required reducing the memory requirement
of the __demangle() function by 3x. This was accomplished using
16-bit indices and 16-bit malloc granularity. That puts a limit
on the longest symbol we can successfully decode, which I think
would be around 6553 characters long, given a 65536-byte buffer
2024-06-01 20:10:58 -07:00
Jōshin dcd626edf8 Add .git-blame-ignore-revs
If you follow the directions in that file then git blame will ignore the
listed commits. A commit should only go in that file if its only changes
were to formatting, particularly on a large part of the codebase (like a
change to .clang-format getting applied to the repo.)

Cribbed from here:

https://www.stefanjudis.com/today-i-learned/how-to-exclude-commits-from-git-blame/
2024-06-01 20:10:48 -07:00
Jōshin f032b5570b Run clang-format (#1197) 2024-06-01 16:30:43 -04:00
Justine Tunney ea081b262c Add some noexcept annotations 2024-06-01 03:19:53 -07:00
Justine Tunney fae1c32267 Encode ±INFINITY as ±1e5000
The V8 behavior of encoding infinity as null doesn't make sense to me.
Using ±1e5000 is better, because JSON.parse decodes it as INFINITY and
the information is preserved. This could be a breaking change for some
2024-06-01 03:19:50 -07:00
Justine Tunney 9b6718ac99 Improve backtraces
We're now able to rewind the instruction pointer in x86 backtraces. This
helps ensure addr2line cannot print information about unrelated adjacent
code. I've restored -fno-schedule-insns2 in most cases because it really
does cause unpredictable breakage for backtraces.
2024-05-30 15:23:11 -07:00
Justine Tunney cd672e251f Improve crash signal reporting on Windows
This change fixes a bug where exiting a crash signal handler on Windows
after adding the signal to uc_sigmask, but not correcting the CPU state
would cause the signal handler to loop infinitely, causing process hang

Another issue is that very tiny programs, that don't link posix signals
would not have their SIGILL / SIGSEGV / etc. status reported to Cosmo's
bash shell when terminating on crash. That's fixed by a tiny handler in
WinMain() that knows how to map WIN32 crash codes to the POSIX flavors.
2024-05-30 14:04:10 -07:00
Justine Tunney 500a47bc2f Fix undefined behavior in unit test
Fixes #1194
2024-05-29 20:31:46 -07:00
Justine Tunney e4d25d68e4 Drop support for Windows 8
Microsoft caused some very gentle breakages for Cosmopolitan. They
removed the version information from the PEB which caused uname to
report WINDOWS 0.0.0. We should have called GetVersionExW but that
doesn't really exist anymore either. Windows policy is now to give
whatever version we used in ape/ape.S. Windows8 has been EOL since
2023-01-10 so lets avoid our modern executables being relegated to
legacy infrastructure. Requiring Windows 10+ going forward lets us
remove runtime compatibility bloat from the codebase. Further note
Cosmopolitan maintains a Windows Vista branch on GitHub, so anyone
preferring the older versions, can still have a future with Cosmo.

Another neat thing this fixes is UTF-8 support in the console. The
changes Microsoft made broke the if statement that enabled UTF8 in
terminals. This explains why bug reports had broken arrows. In the
future this should be less of an issue, since the PEB code is gone
which means we more strictly conform to only Microsoft's WIN32 API
2024-05-29 19:37:47 -07:00
Justine Tunney f31a98d50a Fix bug with realpath() on Windows 2024-05-29 18:47:01 -07:00
Justine Tunney 2816df59b2 Increase tinymalloc granularity 2024-05-29 18:26:01 -07:00
Justine Tunney a05ce3ad9d Support avx512f + vpclmulqdq crc32() acceleration
Cosmo's _Cz_crc32() function now goes 73 GiB/s on Threadripper. This
will significantly improve the performance of the PKZIP file format.
This algorithm is also used by apelink, to create deterministic ids.
2024-05-29 10:13:37 -07:00
Justine Tunney 7c8df05042 Improve -march=native micro-architecture detection 2024-05-29 10:12:49 -07:00
Justine Tunney 4c77acdfcf Add LoadZipArgs() to <cosmo.h> 2024-05-29 10:12:20 -07:00
Justine Tunney b74b974cfd Introduce #include <tinygetopt.h>
The normal getopt() function is bloated because it links printf(). This
change exports the original authentic bsd getopt function, that cosmo's
always used internally so cosmocc users don't need to include internals
2024-05-29 10:11:17 -07:00
Justine Tunney 07cef612c3 Make dlmalloc 2.4x faster for multithreading
This change adds a TLS freelist for small dynamic memory allocations.
Cosmopolitan's TIB is now 512 bytes in size. Single-threaded malloc()
performance isn't impacted by this, until pthread_create() is called.
Single-threaded programs may also want to consider using:

    #include "libc/mem/tinymalloc.inc"

Which will shave 30k off the executable size and sometimes go faster.
2024-05-28 11:18:34 -07:00
Justine Tunney deaef81463 Favor siginfo_t over struct siginfo 2024-05-28 02:34:17 -07:00
Justine Tunney c638eabfe0 Fix compiler warning 2024-05-27 02:23:24 -07:00
Justine Tunney 8e68384e15 Upgrade to 2022-era LLVM LIBCXX 2024-05-27 02:12:27 -07:00
Justine Tunney 2f4ca71f26 Release Cosmopolitan v3.3.10 2024-05-26 22:13:45 -07:00
Justine Tunney 07004ebf04 Upgrade to superconfigure z0.0.42 2024-05-26 22:12:25 -07:00
Justine Tunney 086d7006da Improve crash handler on XNU
This avoids an issue where a crash signal could cause the MacOS process
to freeze and consume all CPU rather than dying as it rightfully should
2024-05-26 18:42:09 -07:00
Gavin Hayes 0a51241f7a ntspawn: fix initializing NtStartupInfoEx (#1190) 2024-05-26 20:54:09 -04:00
Justine Tunney c68f6599e5 Fix definition of getpeername on FreeBSD
We were using the COMPAT magic number, which was recently removed.
2024-05-26 17:03:22 -07:00
Justine Tunney af3f62a71a Ensure io requests are always capped at 0x7ffff000
This gives us the Linux behavior across platforms.

Fixes #1189
2024-05-26 16:53:13 -07:00
Justine Tunney 6cf9b9e0fc Release Cosmopolitan v3.3.9 2024-05-26 15:28:03 -07:00
Justine Tunney 1d4b452839 Refactor some code 2024-05-26 06:03:50 -07:00
Justine Tunney c2db3b703a Introduce --timelog=FILE flag to GNU Make 2024-05-25 14:50:20 -07:00
Justine Tunney edb03b89d8 Make stdin unbuffered when appropriate 2024-05-25 07:57:13 -07:00
Justine Tunney 7724664b13 Release Cosmopolitan v3.3.8 2024-05-25 05:59:20 -07:00
Justine Tunney 1df4296208 Fix stdio for character device regression
Caused by ed93fc3dd7
2024-05-25 05:58:09 -07:00
Justine Tunney ce9aeb2aed Release Cosmopolitan v3.3.7 2024-05-24 19:37:21 -07:00
Justine Tunney ed93fc3dd7 Fix fread() with 2gb+ sizes 2024-05-24 19:28:23 -07:00
Justine Tunney 5f61d273e4 Add hwap constants to sys/auxv.h 2024-05-24 11:44:44 -07:00
Justine Tunney bf3531de81 Make crash reports reliable in multithreaded case 2024-05-24 11:44:44 -07:00
Justine Tunney 3b0ea2db4d Remove -fno-schedule-insns2
This was originally added when we were using the mcount hooking
technique, to prevent the nop from floating above the frame pointer
creation. Now we use a better codegen technique for hooking so it's no
longer needed to sacrifice performance by using this flag.
2024-05-24 11:44:44 -07:00
Justine Tunney f029375d39 Introduce MAP_HUGETLB 2024-05-24 11:44:44 -07:00
Justine Tunney 9b87dd2b87 Refactor some code 2024-05-24 11:44:44 -07:00
Jōshin 787b04f752 Run all BLAKE2B256 test vectors (#1185) 2024-05-24 10:59:23 -07:00
Jōshin 0768807935 Rename python -> python3 (closes #1144) (#1187)
When we removed the com suffix from ape binaries, we broke the build for
ape's python for any case-insensitive file system, i.e. Windows and XNU,
because there is a third_party/python/Python that gets mirrored in the o
directory with the python object files and clashes with the binary name.
This patch hacks around this by renaming the binary to "python3" so that
it no longer clashes with that directory.
2024-05-24 10:56:33 -07:00
Justine Tunney 0b59f01b43 Put confstr() in unistd.h
Fixes #1184
2024-05-21 15:35:06 -07:00
Justine Tunney cf70a44756 Support shebang on Windows
Fixes #1010
2024-05-20 22:11:42 -07:00
Justine Tunney a52792c59f Add some missing build dependencies 2024-05-20 17:23:25 -07:00
Jōshin 65c9b28e99 Fix buffer overflow in os.tmpname (#1180)
At least on macOS, `strlen(getenv("TMPDIR"))` is 50. We now allow a /tmp
that takes up to 120 or so bytes to spell. Instead of overflowing, we do
a bounds check and the function fails successfully on even longer /tmps.

Fixes #1108 (os.tmpname crashes redbean)
2024-05-20 03:46:27 -04:00
Jōshin 4292348707 Import libbsd readpassphrase (#1182)
Included from:

https://gitlab.freedesktop.org/libbsd/libbsd

At commit 04a24db27ad1572f766bad772cdd9c146e6d9cf0.
2024-05-20 03:37:30 -04:00
Jōshin 2ec20b6c4c Fix #922 (Duplicated line) (#1178) 2024-05-20 00:33:50 -07:00
Jōshin 47183551d6 Fix sleb64 (#1179)
Fixes #920
2024-05-20 00:33:17 -07:00
Jōshin 42891e82bb Fix #959 (typo in cosmocc --version) 2024-05-18 15:20:27 -07:00
Gavin Hayes 624119ea38 Fix NT accept/connect not initializing with SO_UPDATE_*_CONTEXT (#1164) 2024-05-17 02:45:30 -07:00
Justine Tunney 2f3c6e7cc3 Revert "Remove zlib namespacing (#1142)"
This reverts commit 5488f0b2ca which was a
good experiment to try, that didn't work out due to #1176

Fixes #1176
2024-05-14 20:45:23 -07:00
Justine Tunney 13b9ecd537 Remove outdated restriction on -Werror on cosmocc
This was a good idea back when we were only using it to build various
open source projects. However it no longer makes sense that many more
people are depending on cosmocc, to develop new software. Our tooling
shouldn't be making these kinds of decisions for the user.
2024-05-09 14:03:51 -07:00
Justine Tunney 6659981457 Release Cosmopolitan v3.3.6 2024-05-08 04:20:22 -07:00
Justine Tunney ae2a7ac844 Fix thread-local storage bugs on aarch64
This change fixes an issue where .tbss memory might not be initialized.
2024-05-08 04:20:22 -07:00
Justine Tunney 793393a341 Polyfill SA_RESETHAND on MacOS ARM64
This change solves the XNU crash loop mystery. Apple's documentation
claims to support this feature, but they only define the constant in
their header files. The kernel acknowledges thi SA_RESETHAND bit, by
clearing it from the sa_flags state, returns zero, and does nothing.
2024-05-08 04:20:22 -07:00
Justine Tunney 952b9009e8 Avoid crash looping on AARCH64 2024-05-08 04:20:22 -07:00
Jōshin 8f18b3ad65 Reapply apeinstall.sh binfmt flags (#1171)
7d31fc3 made it safe to register ape with the binfmt P flag. Older files
will still get their paths passed via argv[0] and everything should just
work. This also restores the F flag that was rolled back alongside the P
flag; this seems like a good idea to me now. It makes it so /usr/bin/ape
is kind of part of the kernel; simply replacing the file will not change
the loader used. The binfmt will have to be reregistered as well.

ape/apeinstall.sh will already nag you if there's an existing binfmt for
ape. It might be nice to make it actually replace the registration.

This reverts commit df648fb174.
2024-05-08 01:48:48 -04:00
Jōshin 56e0fd9d1d Use latest cosmocc (#1175) 2024-05-08 00:08:26 -04:00
Justine Tunney df68a6362b Release Cosmopolitan v3.3.5 2024-05-07 18:18:10 -07:00
Justine Tunney 19c81863a3 Improve crash backtrace reliability
We're now able to pretty print a C++ backtrace upon crashing in pretty
much any runtime execution scenario. The default pledge sandbox policy
on Linux is now to return EPERM. If you call pledge and have debugging
functions linked (e.g. GetSymbolTable) then the symbol table shall get
loaded before any security policy is put in place. This change updates
build/bootstrap/fixupobj too and fixes some other sneaky build errors.
2024-05-07 18:10:28 -07:00
Jōshin 7d31fc311a Loaders rewrite argv[0] for old binaries (#1170)
For this to work, a loader has to be able to tell the difference between
an ‘old’ and a ‘new’ binary. This is achieved via a repurposing of ELF’s
e_flags field. We previously tried to use the padding in e_ident for it,
but binutils was resetting it to zero in e.g. strip.

This introduces one new ELF flag for cosmopolitan binaries. It is called
`EF_APE_MODERN`. We choose 0x101ca75, "lol cat 5".

It should now be safe to install the ape loader binfmt registration with
the `P` flag.
2024-05-07 20:42:18 -04:00
Justine Tunney 57c0b065c8 Make old C++ demangler asynchronous signal safe
It's now possible to safely print C++ backtraces from signal handlers.
This symbol demangler doesn't need malloc, tls, or even static memory.
Additionally, this change makes it 2x faster and adds test cases. It's
almost as performant and accurate as the libcxxabi implementation now.
2024-05-07 03:41:33 -07:00
Justine Tunney a6ecbb747d Introduce libc/mem/tinymalloc.inc
This allocator shaves ~20kb off single-threaded tool programs and is
slightly faster than proper malloc for simple non-demanding programs
2024-05-07 03:19:49 -07:00
Justine Tunney 3bf95ae7ec Invent tool for renaming ELF object symbols 2024-05-07 03:19:49 -07:00
Justine Tunney 5fd7b07fac Improve AVX512 feature detection 2024-05-07 03:19:49 -07:00
Jōshin 317c8bc312 Update MODE=tiny time zone list (#1167)
I took one canonical IANA zone ID from each of the different colored
regions in this article, except those that do not observe DST and do
not have a Google office. See the "Time in Europe" Wikipedia article.

As to which canonical ID to use, this was somewhat arbitrary. Brussels
was obvious, as the de facto capital of the EU. For the rest, I mostly
just went with lexicographic ordering of the most recognizable options.

I've sorted the American zones. This Keeps the U.S. ones together but
does everything alphabetically otherwise. I've added the remaining
Canadian zones These have DST (and Newfoundland is off by a half-
hour from a UTC interval) so they cannot use Etc/. The Pacific/ zones
are sort of sorted. The Chathan Islands have been added. This is the
last of the zones I believe with a non-integer hour offset from UTC.
2024-05-06 16:48:49 -07:00
Jōshin 2a1f352622 Tell vim not to mess with ape.S 2024-05-06 15:36:35 -07:00
Justine Tunney 06d916b449 Add VirtualAlloc2 WIN32 API 2024-05-04 23:26:40 -07:00
Justine Tunney 9ea64725b6 Fix build error 2024-05-04 23:23:56 -07:00
Justine Tunney f9fc7eb49f Fix MODE=dbg build errors 2024-05-04 23:20:12 -07:00
Justine Tunney b0df6c1fce Implement proper time zone support
Cosmopolitan now supports 104 time zones. They're embedded inside any
binary that links the localtime() function. Doing so adds about 100kb
to the binary size. This change also gets time zones working properly
on Windows for the first time. It's not needed to have /etc/localtime
exist on Windows, since we can get this information from WIN32. We're
also now updated to the latest version of Paul Eggert's TZ library.
2024-05-04 23:06:37 -07:00
Justine Tunney d5ebb1fa5b Add MapViewOfFile3 WIN32 API 2024-05-04 12:25:41 -07:00
Justine Tunney 130fd66f9e Release Cosmopolitan v3.3.4 2024-05-03 09:25:02 -07:00
Justine Tunney 8a44f913ae Delete flaky tests
Signals are extremely difficult to unit test reliably. This is why
functions like sigsuspend() exist. When testing something else and
portably it becomes impossible without access to kernel internals.

OpenMP flakes in QEMU on one of my workstations. I don't think the
support is production worthy, because there's been issues on MacOS
additionally. It works great for every experiment I've used it for
though. However a flaky test is worse than no test at all. So it's
removed until someone takes an interest in productionizing it.
2024-05-03 09:11:04 -07:00
Gautham 5488f0b2ca Remove zlib namespacing (#1142)
We have an optimized version of zlib from the Chromium project.
We need it for a lot of our libc services. It would be nice to export
this to user applications if we can, since projects like llamafile are
already depending on it under the private namespace, to avoid
needing to link zlib twice.
2024-05-03 08:07:25 -07:00
Gavin Hayes deff138e7e recvfrom: don't convert address if addrsize is 0 (#1153) 2024-05-03 08:03:57 -07:00
Gavin Hayes b6e40a3a58 Add /dev/(u)random on NT (#1163) 2024-05-03 07:59:51 -07:00
Cadence Ember 8f6bc9dabc Let signals interrupt fgets unless SA_RESTART set (#1152) 2024-05-03 07:49:41 -07:00
Justine Tunney 181cd4cbe8 Add sysctlbyname() for MacOS 2024-05-02 23:21:43 -07:00
Justine Tunney 5c6877b02b Introduce support for trapping math
The feenableexcept() and fedisableexcept() APIs are now provided which
let you detect when NaNs appear the moment it happens from anywhere in
your program. Tests have also been added for the mission critical math
functions expf() and erff(), whose perfect operation has been assured.
See examples/trapping.c to see how to use this powerful functionality.
2024-04-30 13:38:43 -07:00
Justine Tunney 403bc25412 Make MXCSR editable by signal handlers on Windows
It's now possible to recover from floating point exception traps.
2024-04-30 13:38:43 -07:00
Matt Colyer 3bcd40be12 Fix regression in apeinstall.sh (#1161)
This should have been a part of a6baba1.
2024-04-29 20:40:38 -07:00
Paul Kulchenko fcfdd26b72 Fix wrong parameter type in redbean documentation (#1162) 2024-04-29 20:09:25 -04:00
Justine Tunney 2b00ee8723 Fix errno.h documentation typos
Closes #1156
2024-04-28 22:57:21 -07:00
Justine Tunney ff2b6fc0b0 Add munlock() and mlock2() to pledge 2024-04-28 16:53:23 -07:00
Justine Tunney 0eef971494 Add much of C11 threads.h API 2024-04-28 07:04:08 -07:00
Gavin Hayes 2bfd6b37c1 Various paginate improvements (#1148)
* start on improving __paginate

* make __paginate more robust

* add __paginate_file

* cleanup __paginate unlinking
2024-04-26 23:12:25 -04:00
Gavin Hayes 69db501c68 Fix fork locking on win32 (#1141)
* Fix fork locking on win32

- __enable_threads / set __threaded in __proc_setup as threads are required for
  win32 subprocess management
- move mmi/fds locking out of pthread_atfork.c into fork.c so it's done anytime
  __threaded is set instead of being dependent of pthreads
- explicitly yoink _pthread_onfork_prepare, _pthread_onfork_parent, and
  _pthread_onfork_child in pthread_create.c so they are linked in in-case they
  are separated from _pthread_atfork

Big Thanks to @dfyz for help with locating the issue, testing, and devising a fix!

* fix child processes not being able to open files, initialize all necessary locks on fork
2024-04-25 23:01:27 -04:00
Jōshin 6e6fc38935 Apply clang-format update to repo (#1154)
Commit bc6c183 introduced a bunch of discrepancies between what files
look like in the repo and what clang-format says they should look like.
However, there were already a few discrepancies prior to that. Most of
these discrepancies seemed to be unintentional, but a few of them were
load-bearing (e.g., a #include that violated header ordering needing
something to have been #defined by a 'later' #include.)

I opted to take what I hope is a relatively smooth-brained approach: I
reverted the .clang-format change, ran clang-format on the whole repo,
reapplied the .clang-format change, reran clang-format again, and then
reverted the commit that contained the first run. Thus the full effect
of this PR should only be to apply the changed formatting rules to the
repo, and from skimming the results, this seems to be the case.

My work can be checked by applying the short, manual commits, and then
rerunning the command listed in the autogenerated commits (those whose
messages I have prefixed auto:) and seeing if your results agree.

It might be that the other diffs should be fixed at some point but I'm
leaving that aside for now.

fd '\.c(c|pp)?$' --print0| xargs -0 clang-format -i
2024-04-25 10:38:00 -07:00
Jōshin 342d0c81e5 vim spells the c++ filetype 'cpp' 2024-04-24 13:56:37 -07:00
Jōshin cec4c04aa1 Remove .com logic from GetProgramExecutableName (#1134)
The `com` parameter to `TryPath` was always 1, so there was no reason to
have it. This patch changes the logic to be as though `com` was 0, which
provides a possible answer to the TODO question -- the answer is no.

If we never care about appending `.com`, then `CopyWithCwd` doesn't need
to return anything beyond a boolean success value.
2024-04-23 22:07:19 -04:00
Alexey Izbyshev f8c0186221 Fix calling __dns_parse with potentially too large rlen
__res_send returns the full answer length even if it didn't fit the
buffer, but __dns_parse expects the length of the filled part of the
buffer.

Analogous to Musl commit 77327ed064bd57b0e1865cd0e0364057ff4a53b4 which
fixed the only other __dns_parse call site.
2024-04-23 09:36:07 -07:00
Quentin Rameau 6992d8c195 Remove arbitrary limit from DNS result parsing
The name resolution would abort when getting more than 63 records per
request, due to what seems to be a left-over from the original code.
This check was non-breaking but spurious prior to TCP fallback
support, since any 512-byte packet with more than 63 records was
necessarily malformed. But now, it wrongly rejects valid results.

Reported by Daniel Stefanik in Alpine Linux aports issue 15320.
2024-04-23 09:33:02 -07:00
Justine Tunney 1a6b4ab627 Import mntent bug fixes from Musl Libc
f314e133929b6379eccc632bef32eaebb66a7335
Author: Rich Felker <[email protected]>
Date:   Thu Nov 16 12:55:21 2023 -0500

    mntent: fields are delimited only by tabs or spaces, not general whitespace

    this matters because the kernel-provided mtab only escapes tabs,
    spaces, newlines, and backslashes. it leaves carriage returns, form
    feeds, and vertical tabs literal.

commit ee1d39bc1573c1ae49ee6b658938b56bbef95a6c
Author: q66 <[email protected]>
Date:   Thu Nov 9 20:48:44 2023 +0100

    mntent: unescape octal sequences

    As entries in mtab are delimited by spaces, whitespace characters
    are escaped as octal sequences. When reading them out, we have to
    unescape these sequences to get the proper string.
2024-04-23 09:29:56 -07:00
Justine Tunney 9e848abad9 Add missing Musl license headers 2024-04-23 09:29:46 -07:00
Justine Tunney 223eb358c4 Import compiler warning fix from Musl 2024-04-23 09:29:28 -07:00
Szabolcs Nagy ea3cc4ea0b math: fix fma(x,y,0) when x*y rounds to -0
if x!=0, y!=0, z==0 then

  fma(x,y,z) == x*y

in all rounding modes, while adding z can ruin the sign of 0 if x*y
rounds to -0.
2024-04-23 09:22:33 -07:00
Paul Kulchenko 01267ea0f5 Update GetRemoteAddr documentation to clarify that it may return nil. (#1149) 2024-04-22 01:55:08 -04:00
Justine Tunney bc6c18368f Update .clang-format 2024-04-19 17:11:08 -07:00
Jōshin 06839ab301 Change loop bound on uuidv4 test (#1143)
A half second (on my machine) is too long for a unit test. 1000
iterations is probably still overkill, but 0.01 seconds is fine.

Also made `y` local.
2024-04-12 12:22:34 -04:00
BONNAURE Olivier 39dde41516 [Redbean] Add UuidV4 method (#1140) 2024-04-12 11:10:27 -04:00
Jōshin 3e16e59f72 Move store after __asan_unpoison in __zipos_alloc (#1135)
Previously, the atomic store looked like it was happening while the
struct's memory was still poisoned. I was unable to observe any issues
with this, but this change seems to make the code more obviously correct
(at the cost of a redundant atomic store to zeroed space in case the map
needed to be extended.)
2024-04-10 23:04:03 -04:00
Justine Tunney cf9a1f7f33 Fix MODE=optlinux build 2024-04-06 21:02:19 -07:00
Justine Tunney 49a32136f8 Upgrade the One True Awk 2024-04-06 19:21:48 -07:00
Justine Tunney b9d6e6e348 Standardize on -fsigned-char
MaGuess on Discord pointed out the fact that cosmocc contradicts itself
on the signedness of `char`. It's up to each platform to choose one, so
the cosmo platform shall choose signed. The rationale is it makes the C
language syntax more internally similar. `char` should be `signed char`
for the same reason `int` means `signed int`. It's recommended that you
still assume `char` could go either way since that's portable thinking.
But if you want to assume we'll always have signed char, that's ok too.
2024-03-30 21:46:08 -07:00
Justine Tunney b431ca4449 Upgrade to cosmocc v3.3.3 2024-03-30 20:17:14 -07:00
Justine Tunney 98d06b23cc Release Cosmopolitan v3.3.3 2024-03-30 19:24:04 -07:00
Justine Tunney 9ff65cff63 Fix ZipOS extraction of ZIP64 assets 2024-03-30 19:00:40 -07:00
Justine Tunney bef2cb05ff Fix AVX512_BF16 2024-03-26 00:00:46 -07:00
mierenhoop 43885a76e4 Fix binary formatting for integers 2 and 3 (#1123) 2024-03-25 21:11:09 -07:00
Justine Tunney 74a85087de Support -S in cosmocross
You can now run commands like `x86_64-unknown-cosmo-c++ -S` when using
your cosmocc toolchain. Please note the S flag isn't supported for the
cosmocc command itself.
2024-03-24 15:49:55 -07:00
Justine Tunney 045632a743 Make fixupobj work on Windows
This fixes a regression with recent cosmocc toolchain releases that
causes fixupobj to fail with "ifunc ftruncate #1 failed" on Windows
2024-03-24 03:14:25 -07:00
Justine Tunney 9a10adac35 Remove .com from README 2024-03-22 19:38:38 -07:00
Justine Tunney 0c12c26e1d Update Emacs config 2024-03-22 19:27:11 -07:00
Justine Tunney 40b7da8422 Speed up fmaf() on x86 2024-03-22 19:26:56 -07:00
Justine Tunney 640668931d Clarify MacOS supported versions
We technically support versions much earlier, possibly dating back to
2018 or earlier, but they're not officially supported since I have no
way of testing them.

Closes #1129
2024-03-22 19:25:07 -07:00
Justine Tunney 1c34088ba0 Ignore -pie and -fpie in cosmocc
Fixes #1126
2024-03-22 03:55:01 -07:00
Justine Tunney bb92347158 Add deprecated stubs for _bsf / _bsr 2024-03-05 09:45:51 -08:00
Justine Tunney 8bfd56b59e Rename _bsr/_bsf to bsr/bsf
Now that these functions are behind _COSMO_SOURCE there's no reason for
having the ugly underscore anymore. To use these functions, you need to
pass -mcosmo to cosmocc.
2024-03-04 17:33:26 -08:00
Justine Tunney a6baba1b07 Stop using .com extension in monorepo
The WIN32 CreateProcess() function does not require an .exe or .com
suffix in order to spawn an executable. Now that we have Cosmo bash
we're no longer so dependent on the cmd.exe prompt.
2024-03-03 03:12:19 -08:00
Justine Tunney c8383f25b4 Release Cosmopolitan v3.3.2 2024-02-27 09:15:59 -08:00
Justine Tunney 64a9e6fe56 Fix compiler runtime for _Float16 type 2024-02-27 09:06:23 -08:00
Justine Tunney 0ef36489c8 Walk back most uses of __STRICT_ANSI__ 2024-02-27 04:09:49 -08:00
Justine Tunney 7c7bf4bb90 Release Cosmopolitan v3.3.1 2024-02-27 00:31:36 -08:00
Justine Tunney 592f6ebc20 Make quality improvements
- Write some more unit tests
- memcpy() on ARM is now faster
- Address the Musl complex math FIXME comments
- Some libm funcs like pow() now support setting errno
- Import the latest and greatest math functions from ARM
- Use more accurate atan2f() and log1pf() implementations
- atoi() and atol() will no longer saturate or clobber errno
2024-02-25 19:08:43 -08:00
Justine Tunney af8f2bd19f Shave 4kb off each binary 2024-02-25 11:11:34 -08:00
Justine Tunney 38bceaeb8f Give up on making clock_nanosleep() precise
Multiple projects I care about make the assumption that this isn't a
system call that sleeps for a particular number of nanonseconds, but
rather a function that parks processes on kernel scheduler quantums.
Anyone who wants the old behavior should use cosmo_clock_nanosleep()
2024-02-24 10:17:47 -08:00
Ivan Komarov 99f0491f04 Fix invalid XNU binaries generated by apelink in some edge cases (#1106)
* Fix `if...fi` generation in the generated APE shell script

A shell will fail with a syntax error on an empty `if` or `else` body.
That is, neither of these is allowed:

    # Empty `if`
    if [ ... ]; then
    fi

    # Empty `else`
    if [ ... ]; then
    ...
    else
    fi

There were two places where `apelink` could generate problematic `if`'s:

1. The XNU shell generation for aarch64 binaries when no loaders (either
   binary or source) are provided. They can't assimilate, so the resulting
   `else` body becomes empty.
   There is actually a code path guarded by the `gotsome` variable that
   inserts an extra `true` in this case, but the variable was never
   initialized, so in practice this code path didn't activate in my
   tests. This is fixed by initializing the variable.
2. The loader extraction code when no loaders are provided and XNU
   support is requested. This is fixed by adding a simliar code path
   that prevents an empty body from being generated.

* Update the apelink manual after commit d53c335

The `-s` option changed its meaning, but the docs weren't updated.
2024-02-23 08:11:27 -08:00
Justine Tunney e72a88ea70 Make fixups for libcrypt 2024-02-23 07:39:44 -08:00
Ivan Komarov 9b4701b3bc Don't include <crypt.h> from <stdlib.h> (#1112) 2024-02-23 07:18:35 -08:00
Ivan Komarov f7ff515961 *scanf() fixes to make TeX work (#1109)
* Fix reading the same symbol twice when using `{f,}scanf()`

PR #924 appears to use `unget()` subtly incorrectly when parsing
floating point numbers. The rest of the code only uses `unget()`
immediately followed by `goto Done;` to return back the symbol that
can't possibly belong to the directive we're processing.

With floating-point, however, the ungot characters could very well
be valid for the *next* directive, so we will essentially read them
twice. It can't be seen in `sscanf()` tests because `unget()` is a
no-op there, but the test I added for `fscanf()` fails like this:

        ...
        EXPECT_EQ(0xDEAD, i1)
                need 57005 (or 0xdead) =
                 got 908973 (or 0x000ddead)
        ...
        EXPECT_EQ(0xBEEF, i2)
                need 48879 (or 0xbeef) =
                 got 769775 (or 0x000bbeef)

This means we read 0xDDEAD instead of 0xDEAD and 0xBBEEF instead of
0xBEEF. I checked that both musl and glibc read 0xDEAD/0xBEEF, as
expected.

Fix the failing test by removing the unneeded `unget()` calls.

* Don't read invalid floating-point numbers in `*scanf()`

Currently, we just ignore any errors from `strtod()`. They can
happen either because no valid float can be parsed at all, or
because the state machine recognizes only a prefix of a valid
floating-point number.

Fix this by making sure `strtod()` parses everything we recognized,
provided it's non-empty. This requires to pop the last character
off the FP buffer, which is supposed to be parsed by the next
`*scanf()` directive.

* Make `%c` parsing in `*scanf()` respect the C standard

Currently, `%c`-style directives always succeed even if there
are actually fewer characters in the input than requested.

Before the fix, the added test fails like this:

        ...
        EXPECT_EQ(2, sscanf("ab", "%c %c %c", &c2, &c3, &c4))
                need 2 (or 0x02 or '\2' or ENOENT) =
                 got 3 (or 0x03 or '\3' or ESRCH)
        ...
        EXPECT_EQ(0, sscanf("abcd", "%5c", s2))
                need 0 (or 0x0 or '\0') =
                 got 1 (or 0x01 or '\1' or EPERM)

musl and glibc pass this test.
2024-02-23 07:15:30 -08:00
Justine Tunney 3afe3a3646 Fix broken link 2024-02-23 07:11:44 -08:00
Justine Tunney b4faaa7c57 Relax style guide 2024-02-23 06:55:37 -08:00
Justine Tunney c4991e53ac Document copyright policy exceptions 2024-02-23 06:53:46 -08:00
Justine Tunney ad3944a3b6 Support any HTTP method
It's now possible to use redbean Fetch() with arbitrary HTTP methods,
e.g. LIST which is used by Hashicorp. There's an eight char limit and
uppercase canonicalization still happens. This change also includes a
better function for launching a browser tab, that won't deadlock on a
headless workstation running Debian.

Closes #1107
2024-02-22 14:12:18 -08:00
Justine Tunney 29eac8e2a2 Fix regression in logger functions
The recent release broke things so that the logger wasn't logging by
default. This ensures that `__log_file` initializes in correct order
2024-02-22 14:08:20 -08:00
Justine Tunney a5a3e78fcf Make ftrace/strace C APIs work with cosmocc 2024-02-22 12:13:16 -08:00
Justine Tunney e9a756e78b Make --ftrace more crash proof with foreign code
Sometimes we need to interact with code that wasn't compiled using
`-fno-omit-frame-pointer`. For example, if a function pointer gets
passed and called by a foreign function, linked by cosmo_dlopen().
Function call tracing will now detect backtrace pointer corruption
and simply reduce the indentation level back to zero, as a result.
2024-02-22 11:06:31 -08:00
Justine Tunney 38af54a2dd Fix GCC version in cosmocc wrapper script
See #1111
2024-02-21 19:16:39 -08:00
Justine Tunney 77a92f517b Introduce getcpu() system call from glibc 2024-02-21 18:17:20 -08:00
Justine Tunney 3eb405e0e2 Resurrect <windows.h> as <windowsesque.h> 2024-02-21 16:41:11 -08:00
Justine Tunney 68b9479f0c Make cosmo.h work a little better 2024-02-21 16:40:09 -08:00
Justine Tunney d9df41c4d5 Add example email to CONTRIBUTING.md 2024-02-21 16:36:52 -08:00
Justine Tunney fc3c536ec2 Make --ftrace work better with cosmocc
This change causes cosmocc to use -fno-inline-functions-called-once by
default, unless -Os or -finline-functions-called-once is defined. This
is important since I believe it generally makes code go faster, and it
most importantly makes --ftrace output much more understandable, since
the trace will be more likely to reflect the actual shape of the code.
We've always used this flag in the mono repo when ftracing is enabled,
but it slipped my mind to incorporate this into the cosmocc toolchain.
2024-02-21 13:20:21 -08:00
Justine Tunney 77ddf4cb07 Fix fixupobj regression on MacOS ARM64 2024-02-21 12:13:54 -08:00
Justine Tunney 4d018306b3 Fix MODE=optlinux build 2024-02-20 14:24:56 -08:00
Justine Tunney b3bb93d1d9 Fix MODE=opt build 2024-02-20 14:12:12 -08:00
Justine Tunney 980af27bbf Fix documentation build 2024-02-20 13:40:46 -08:00
Justine Tunney 957c61cbbf Release Cosmopolitan v3.3
This change upgrades to GCC 12.3 and GNU binutils 2.42. The GNU linker
appears to have changed things so that only a single de-duplicated str
table is present in the binary, and it gets placed wherever the linker
wants, regardless of what the linker script says. To cope with that we
need to stop using .ident to embed licenses. As such, this change does
significant work to revamp how third party licenses are defined in the
codebase, using `.section .notice,"aR",@progbits`.

This new GCC 12.3 toolchain has support for GNU indirect functions. It
lets us support __target_clones__ for the first time. This is used for
optimizing the performance of libc string functions such as strlen and
friends so far on x86, by ensuring AVX systems favor a second codepath
that uses VEX encoding. It shaves some latency off certain operations.
It's a useful feature to have for scientific computing for the reasons
explained by the test/libcxx/openmp_test.cc example which compiles for
fifteen different microarchitectures. Thanks to the upgrades, it's now
also possible to use newer instruction sets, such as AVX512FP16, VNNI.

Cosmo now uses the %gs register on x86 by default for TLS. Doing it is
helpful for any program that links `cosmo_dlopen()`. Such programs had
to recompile their binaries at startup to change the TLS instructions.
That's not great, since it means every page in the executable needs to
be faulted. The work of rewriting TLS-related x86 opcodes, is moved to
fixupobj.com instead. This is great news for MacOS x86 users, since we
previously needed to morph the binary every time for that platform but
now that's no longer necessary. The only platforms where we need fixup
of TLS x86 opcodes at runtime are now Windows, OpenBSD, and NetBSD. On
Windows we morph TLS to point deeper into the TIB, based on a TlsAlloc
assignment, and on OpenBSD/NetBSD we morph %gs back into %fs since the
kernels do not allow us to specify a value for the %gs register.

OpenBSD users are now required to use APE Loader to run Cosmo binaries
and assimilation is no longer possible. OpenBSD kernel needs to change
to allow programs to specify a value for the %gs register, or it needs
to stop marking executable pages loaded by the kernel as mimmutable().

This release fixes __constructor__, .ctor, .init_array, and lastly the
.preinit_array so they behave the exact same way as glibc.

We no longer use hex constants to define math.h symbols like M_PI.
2024-02-20 13:27:59 -08:00
BONNAURE Olivier d3ff48c63f [Redbean] Feature / OnError(status, message) hook (#1103) 2024-02-14 01:55:50 -08:00
Justine Tunney 2ab9e9f7fd Make improvements
- Introduce portable sched_getcpu() api
- Support GCC's __target_clones__ feature
- Make fma() go faster on x86 in default mode
- Remove some asan checks from core libraries
- WinMain() now ensures $HOME and $USER are defined
2024-02-12 10:23:00 -08:00
Ivan Komarov d5225a693b libc headers: make <complex.h> work, add struct ipv6_mreq (#1100)
1. `libc/isystem/complex.h` (included when you do `#include <complex.h>`)
   defines `_COMPLEX_H`, and then proceeds to include `libc/complex.h`,
   which contains the actual complex-related declarations. However, they
   are *also* guarded by `_COMPLEX_H` and hence effectively ignored.
   Fix this by changing `_COMPLEX_H` to `COSMOPOLITAN_LIBC_COMPLEX_H_`,
   which is consistent with what the other headers (such as `math.h`) do.
2. Cosmopolitan could only support IPv4 multicast requests for sockets,
   since a declaration for `struct ipv6_mreq` was missing. Add support
   for IPv6, too, by adding the missing declaration.
2024-02-05 17:22:56 -05:00
Paul Kulchenko c72904b2f6 Fix redbean Fetch redirect with relative URL (#1034) 2024-02-02 10:02:29 -05:00
Justine Tunney 616717fa82 Fine tune OpenMP some more 2024-01-30 06:30:24 -08:00
Justine Tunney 369aebfc48 Make improvements
- Let OpenMP be usable via cosmocc
- Let libunwind be usable via cosmocc
- Make X86_HAVE(AVXVNNI) work correctly
- Avoid using MAP_GROWSDOWN on qemu-aarch64
- Introduce in6addr_any and in6addr_loopback
- Have thread stacks use MAP_GROWSDOWN by default
- Ask OpenMP to not use filesystem to manage threads
- Make NI_MAXHOST and NI_MAXSERV available w/o _GNU_SOURCE
2024-01-29 16:31:58 -08:00
Justine Tunney 5f8e9f14c1 Add OpenMP support 2024-01-28 22:39:02 -08:00
Justine Tunney c1e18e7903 Restore MODE=dbg support
We recently broke MODE=dbg support when we added C++ exception support.
This change adds the missing UBSAN interfaces, needed to get it working
again. Some of the ASAN checking in the SJLJ guts needed to be disabled
since I doubt anyone's combined the two features until now.
2024-01-26 23:07:18 -08:00
Justine Tunney f27808c4d2 Remove feature for embedding blink in ape scripts
Embedding Blink builds in Cosmo executables was a failed experiment. It
turned out to be easier than expected to let the mono repo have support
for multiple architectures. Blink still works great; it's supported and
recommended; just please use it as a separate program. For example, you
can use Blink to run Cosmo binaries on architectures like i486 / s390x.
2024-01-26 22:30:56 -08:00
Justine Tunney 2719080986 Add program for printing Windows network dev info 2024-01-26 22:29:23 -08:00
Trung Nguyen 33418f6742 libc: Fix __cxa_thread_atexit prototype (#1088)
The first parameter should be a function pointer.

Otherwise, this would conflict with the definition provided by
`libcxxabi`.
2024-01-22 10:23:28 -08:00
Trung Nguyen 83a8686c06 tool/cosmocc: Properly handle dependency output (#1091)
a2753de contains some regressions, causing `fixupobj` to be
inappropriately suppressed when `-MD` or `-MMD` is passed.

This commit reverts most changes by a2753de, and:
- Treats all invocations of the compiler with `-M` and `-MM` as with the
`cpp` intent, since these flags imply `-E`.
- Handle the dependency output path specified by `-MF`.
  + This is trivial for `cosmocross` since the script does not throw
  objects to and from temporary directories.
  + For `cosmocc`, the file names are calculated based on the `-MF`
  value provided by the user. If this flag is not specified, the script
  generates the file name based on the output file using GCC rules.
  Then, before calling the real compilers, an additional `-MF` flag is
  passed to override the dependency outputs with mangled file names.
2024-01-22 10:22:16 -08:00
Justine Tunney 8ab3a545c6 Increase build memory quota
If you install qemu-user from apt then glibc links a lot of address
space bloat that causes pthread_create() to ENOMEM (a.k.a. EAGAIN).
Boosting the virtual memory quota from 512m to 2048m will hopefully
future proof the build for the future, as Linux distros get fatter.
Please note this only applies to MODE=aarch64 on x86_64 builds when
you're using QEMU from Debian/Ubuntu rather than installing the one
cosmo provides in third_party/qemu/qemu-aarch64.gz. This change may
also be useful to people who are using the host compiler toolchain.
2024-01-22 10:02:30 -08:00
Justine Tunney 8ebe2e9020 Fix assertion error in pthread_create() 2024-01-22 10:02:30 -08:00
BONNAURE Olivier 51cd83f674 Add Curve25519 method in help.txt + add new specs file (#1096) 2024-01-21 06:58:48 -08:00
Justine Tunney 39b0a9c03e Fix scanf() %n off by one at eof
Fixes #1094
2024-01-20 15:06:16 -08:00
Justine Tunney d50064a779 Add Curve25519() API to Redbean 2024-01-20 01:06:19 -08:00
Justine Tunney 1226eb7a5e Fix gcc warning 2024-01-18 16:48:02 -08:00
Trung Nguyen 8834dde0c2 libcxx: Add missing implementation source files (#1089)
Added the implementation for `std::bad_any_cast` from upstream
`any.cpp`, and `std::bad_variant_access` from upstream `variant.cpp`.

This fixes missing `vtable` and `typeinfo` symbols when trying to link
code referencing these exception types.
2024-01-18 08:20:25 -08:00
Justine Tunney 1ef63eb206 Retire third_party/quickjs/
QuickJS cosmocc binaries are now being distributed on
https://bellard.org/quickjs/
2024-01-17 12:35:46 -08:00
Brian 08793aa143 Make ... optional in .args (#1086) 2024-01-15 16:20:50 -08:00
Trung Nguyen 6db1200a7e libc: Fix cxxabi.h header (#1087)
- `__cxa_*` runtime functions are expected to be in the `abi` namespace,
which is currently an alias for `__cxxabiv1`.
- Rely on the header provided by `libcxxabi` for functions that we do
not implement ourselves anymore.
2024-01-15 11:50:38 -08:00
Trung Nguyen a2753de7fd tool/cosmocc: Do not run fixupobj for text output (#1084)
Some compiler flags (such as -E or -MM) instruct GCC to only run the
preprocessor and produce certain text files.

In this case, we do not want to run `fixupobj` and make the tool fail
because the input is not an ELF64 binary.
2024-01-15 07:16:13 -08:00
Jōshin 6715b670b1 Skip the CAS on new_pos < 0 (#1080) 2024-01-12 21:08:28 -08:00
Justine Tunney fd75fd1467 Make std::pair trivial 2024-01-09 09:51:59 -08:00
Justine Tunney 5d80e8dbf6 Add more Intel microarchitectures 2024-01-09 01:54:26 -08:00
Justine Tunney eeb20775d2 Add dontthrow attribute to most libc functions
This will help C++ code that uses exceptions to be tinier. For example,
this change shaves away 1000 lines of assembly code from LLVM's libcxx,
which is 0.7% of all assembly instructions in the entire library.
2024-01-09 01:26:03 -08:00
Justine Tunney cb19e172da Release Cosmopolitan v3.2.4 2024-01-08 19:37:59 -08:00
Jōshin df648fb174 Revert apeinstall.sh binfmt flags (#1072)
The P flag breaks backwards compatibility with older binaries. The idea
is to revert this commit after that break has been resolved.
2024-01-08 14:21:21 -08:00
Justine Tunney 9a6af92bed Fix libcxxabi tests in MODE=tiny and MODE=rel
See #1076
2024-01-08 13:49:06 -08:00
Justine Tunney 6ab01716ce Fix aarch64 setjmp abi
We now store values in jmp_buf where the compiler wants them to be. This
fixes code that calls __builtin_setjmp() and __builtin_longjmp() such as
libunwind. All libcxxabi tests are now passing on ARM64.

See #1076
2024-01-08 13:26:28 -08:00
Justine Tunney 21093044c0 Rebuild build/bootstrap/chmod.com 2024-01-08 12:22:12 -08:00
Justine Tunney b8a9fbbd01 Review catch_multi_level_pointer test failure
This test was was failing on GitHub Actions because GA uses Linux and
Linux supports resource usage accounting. Cosmo's compile.com program
imposes CPU, memory and file size limits on both the compiler and the
test programs themselves.

See #1076
2024-01-08 12:10:29 -08:00
Justine Tunney 81ce2e4cbc Call thread finalizers on exit()
While we don't call POSIX thread key destructors from exit(), we do need
to call these, since C++ uses it for TLS object destructors.

See #1076
2024-01-08 11:45:02 -08:00
Justine Tunney 07db3004d6 Disable libcxxabi/test/guard_threaded_test
It works, but exceeds the CPU quota granted to tests, by a lot.
2024-01-08 11:19:09 -08:00
Trung Nguyen b0566348b2 third_party/libcxxabi: Add test suite (#1076)
Added the `libcxxabi` test suite as found in LLVM 17.0.6.

Some tests that do not apply to the current configuration of
comsopolitan are not added. These include:
- `backtrace_test`, `forced_unwind*`: Use unwind function unsupported in
SjLj mode.
- `noexception*`: Designed to test `libcxxabi` in no exceptions mode.

Some tests are added but not enabled due to bugs specific to GCC or
cosmopolitan. These are clearly indicated in the `BUILD.mk` file.
2024-01-08 10:50:06 -08:00
Justine Tunney 8fb24a8f88 Turn exceptions back off by default 2024-01-08 10:47:42 -08:00
Justine Tunney a4b455185b Bring back gc() function
Renaming gc() to _gc() was a mistake since the better thing to do is put
it behind the _COSMO_SOURCE macro. We need this change because I haven't
wanted to use my amazing garbage collector ever since we renamed it. You
now need to define _COSMO_SOURCE yourself when using amalgamation header
and cosmocc users need to pass the -mcosmo flag to get the gc() function

Some other issues relating to cancelation have been fixed along the way.
We're also now putting cosmocc in a folder named `.cosmocc` so it can be
more safely excluded by grep --exclude-dir=.cosmocc --exclude-dir=o etc.
2024-01-08 10:26:28 -08:00
Jōshin 6cb0354e19 mkofs improvements
- Always use $PWD/o. if it doesn't exist, say how to create it.

- return 2 on unsupported platforms.
2024-01-08 17:23:09 +00:00
Trung Nguyen b580080af1 tool/cosmocc: Allow exceptions and rtti (#1074)
With `libunwind` and `libcxxabi` included in `libcosmo`, we can now
allow users to build C++ applications with exceptions and RTTI enabled.

The default is still disabling these two to avoid bloating the binary.

Closes #1065
2024-01-08 08:47:53 -08:00
Trung Nguyen 8b33204f37 Add LLVM libcxxabi (#1063)
* third_party: Add libcxxabi

Added libcxxabi from LLVM 17.0.6
The library implements the Itanium C++ exception handling ABI.

* third_party/libcxxabi: Enable __cxa_thread_atexit

Enable `__cxa_thread_atexit` from libcxxabi.
`__cxa_thread_atexit_impl` is still implemented by the cosmo libc.
The original `__cxa_thread_atexit` has been removed.

* third_party/libcxx: Build with exceptions

Build libcxx with exceptions enabled.

- Removed `_LIBCPP_NO_EXCEPTIONS` from `__config`.
- Switched the exception implementation to `libcxxabi`. These two files
are taken from the same `libcxx` version as mentioned in `README.cosmo`.
- Removed `new_handler_fallback` in favor of `libcxxabi` implementation.
- Enable `-fexceptions` and `-frtti` for `libcxx`.
- Removed `THIRD_PARTY_LIBCXX` dependency from `libcxxabi` and
`libunwind`. These libraries do not use any runtime `libcxx` functions,
just headers.

* libc: Remove remaining redundant cxa functions

- `__cxa_pure_virtual` in `libcxxabi` is also a stub similar to the
existing one.
- `__cxa_guard_*` from `libcxxabi` is used instead of the ones from
Android.

Now there should be no more duplicate implementations.
`__cxa_thread_atexit_impl`, `__cxa_atexit`, and related supporting
functions, are still left to other libraries as in `libcxxabi`.

`libcxxabi` is also now added to `cosmopolitan.a` to make up for the
removed functions.

Affected in-tree libraries (`third_party/double-conversion`) have been
updated.
2024-01-08 08:45:10 -08:00
Trung Nguyen 94bab1618d Add fixes to libunwind (#1069) 2024-01-08 08:31:13 -08:00
Jōshin cafea9a0a5 Flesh out tool/zsh/mkofs
It now prevents double-mounting.

Also adds vim filetype modelines.
2024-01-08 05:18:18 +00:00
Jōshin 16099801cc zsh mkofs tool 2024-01-08 03:43:09 +00:00
Justine Tunney 4cd02c29ed Release Cosmopolitan v3.2.3 2024-01-07 12:39:09 -08:00
Justine Tunney 5ae2554c10 Have cosmo_dlopen() request -z execstack on Linux
The AMD HIP SDK for Linux ships prebuilt DSOs with an RWX PT_GNU_STACK
since old versions of GCC made it nearly impossible to build artifacts
where that wasn't the case, however modern glibc systems will flat out
refuse to link RWX DSOs from an execuatble that uses PT_GNU_STACK = RW
2024-01-07 11:37:18 -08:00
Justine Tunney 531ac85923 Disable Python test_resource
It fails strangely on some Linux systems due to constraints imposed by
the build system. We'll try to learn more about this later on.
2024-01-07 11:36:31 -08:00
Jōshin 8d9fcb5e5a Fix ape-m1.c usage
It's not about `$0` anymore.
2024-01-07 10:35:50 -05:00
Jōshin aa37a327ea Make $prog.ape more reliable on Apple Silicon (#1071)
Now it doesn't matter what argv `$prog.ape` is invoked with. We just get
our executable path the Apple way.
2024-01-07 07:13:20 -08:00
Justine Tunney c4205f8305 Remove lingering dlopen thunk 2024-01-07 02:14:41 -08:00
Justine Tunney b3fb6cff43 Add /dev/fd support to Windows
GNU bash needs this functionality, otherwise it can't do <(cmd...).
2024-01-06 18:20:21 -08:00
Jōshin 2e5f662dfe mmake: different cosmocc regime
Now that cosmocc is unpacked into a version-specific directory under
cosmocc/, it makes more sense to put the versions in /opt/cosmocc and
maintain a symlink to the currently active one.
2024-01-06 19:14:13 -05:00
Justine Tunney 5e7137097d Make breaking change to cosmo_dlsym()
The cosmo_dlsym() function now returns the raw function address. You
need to call cosmo_dltramp() on the result, to make it safe to call.
This change is important, because cosmo_dltramp() magic can't always
work; for some tricky functions, you need to translate ABIs by hand.
2024-01-06 15:45:26 -08:00
Justine Tunney f224a55d57 Fix MODE=dbg build
Fixes #1067
2024-01-06 15:22:19 -08:00
Justine Tunney 0c51995b2b Add missing sha256sum.c file 2024-01-06 14:47:14 -08:00
Jōshin a9f23ccd17 Fix download-cosmocc.sh on Mac (#1066)
shasum behaves enough like sha256sum to work in this case.
2024-01-06 14:46:31 -08:00
Justine Tunney 57d1bf7a9c Clarify CONTRIBUTING.md 2024-01-06 14:18:01 -08:00
Justine Tunney ce17ed60df Support -MF and -MT in cosmocc 2024-01-06 12:10:06 -08:00
Jōshin d27a47b0e2 Bugfix: ape --help should exit 0 (#1060) 2024-01-06 12:07:32 -08:00
Jōshin 390335eb45 apeinstall/uninstall.sh can use doas (#1062) 2024-01-06 12:06:21 -08:00
Jōshin 492a8f4f53 Fix make o/aarch64/ape on aarch64 (#1064) 2024-01-06 11:42:39 -08:00
Jōshin 636bc4007b Enable argv[0] tests in more places (#1061)
Now we do them for assimilated binaries (except on OpenBSD or XNU
non-Silicon), for XnuSilicon, and for binaries with the preserve-
argv[0] auxv flag set. We check whether to pass the argv[0] value
at the test site rather than the Child site. We move a lot of the
test initialization into Child in the non-child case, in order to
get at the pre-init value of `__program_executable_name`. Finally,
we print out info about what we are skipping.
2024-01-06 11:42:03 -08:00
Trung Nguyen b09096691a third_party: Add libunwind (#1053)
Added libunwind from LLVM 17.0.6.
The library includes functions required for C++ exception handling.
2024-01-06 15:04:30 +07:00
Justine Tunney 91de6f1f5d Release Cosmopolitan 3.2.2 2024-01-05 21:00:22 -08:00
Justine Tunney fad1279c61 Make cosmo_dlopen() safer and faster
If cosmo_dlopen() is linked on AMD64 then the runtime will switch to
using %gs for thread-local storage. This eliminates the need for the
imported symbol trampoline. It's now safer to pass function pointers
back and forth with imported libraries. Your program gets recompiled
at runtime to make it happen and the overhead is a few milliseconds.
2024-01-05 20:42:19 -08:00
Jōshin c0eacf2eb1 clang-format 2024-01-05 23:21:52 -05:00
Justine Tunney c60b150fcf Release Cosmopolitan v3.2.1 2024-01-05 15:36:57 -08:00
Jōshin 80ec6c9283 Try to detect kernel version for P flag (#1059) 2024-01-05 15:19:46 -08:00
Justine Tunney 2d93788ce3 Fix --ftrace with cosmo_dlopen()
This change ensures function call logging won't crash the process when
cosmo_dlopen() is called.
2024-01-05 15:13:07 -08:00
Jōshin 15548b523c Cleanup apeinstall.sh (#1057) 2024-01-05 13:45:14 -08:00
Jōshin 412a200ae4 Support binfmt_misc P flag in APE loader (#1058)
This allows ape to automatically preserve `argv[0]` [as of Linux kernel
5.12][0] if the [binfmt_misc][1] registration contains the P flag.

This also removes may_path_search, which was identical to the literally
flag in usage. As a result, FindCommand is subsumed into Commandv.

[0]: https://patchew.org/QEMU/[email protected]/
[1]: https://www.kernel.org/doc/html/latest/admin-guide/binfmt-misc.html
2024-01-05 12:35:01 -08:00
Justine Tunney 44a463e4d2 Invent *cosmo_dltramp() function 2024-01-05 09:11:51 -08:00
Justine Tunney 0de6a08988 Upgrade mono repo to cosmocc 3.2
The toolchain will now be downloaded going forward from multiple pinned
URLs which have shasums. Either wget or curl must be installed.

This change unblocks #1053
2024-01-05 08:02:04 -08:00
Paul Kulchenko a334f9cc33 Update redbean ProgramDirectory to return a list of previously set directories (#1021) 2024-01-05 02:17:55 -08:00
Jōshin 3315b6ef11 Store StrlenDevFd in n (#1056) 2024-01-04 23:17:37 -05:00
Justine Tunney e93c8c4f8f Fix typo 2024-01-04 13:59:36 -08:00
Justine Tunney 0e49bed660 Support 40 cosmo_dlopen() function parameters
Our dynamic linking implementation is now able to support functions with
dozens of parameters. In addition to having extra integral arguments you
can now pass vector registers using intrinsic types. Lastly, you can now
return multiple values, which is useful for functions returning structs.
2024-01-04 13:41:26 -08:00
Justine Tunney a3deef70c2 Release Cosmopolitan v3.2 2024-01-04 09:39:48 -08:00
Justine Tunney 873069fcd7 Fix -Xaarch64 in cosmocc
It's now possible to pass flags like -Xaarch64-march=armv8.2-a+dotprod
so that cosmocc will use newer ARM ISAs. For AMD64 there's another one
worth mentioning, which looks like this: -Xx86_64-mssse3
2024-01-04 07:22:05 -08:00
Justine Tunney 3864f78b88 Give GNU Make limitless stacks 2024-01-04 03:26:26 -08:00
Justine Tunney 796148790f Remove hard coded paths from APE bootloader
This increases risk of fork bomb but is needed to support the NixOS.
Upstream dependencies of APE (uname, mkdir, dd, chmod, gzip, and mv)
will be removed from releases, and deleted from the cosmo.zip server

See #12
2024-01-03 17:55:19 -08:00
Justine Tunney 34ed1fcbea Fix bugs with DNS library on Windows 2024-01-03 17:39:57 -08:00
Justine Tunney 1bb52c223b Add missing build dependency 2024-01-03 17:04:21 -08:00
Justine Tunney 31dcdefaa8 Update bootstrap binaries
It's now possible to build the monorepo on FreeBSD ARM64.
2024-01-03 17:03:51 -08:00
Jōshin 2902b76168 mmake gets smarter about locating make
- Check `$COSMOCC`, defaulting to `/opt/cosmocc`.

- Try to get the full path of the repo `make.com`. I'm not aware of
  a way of getting the path that defines a zsh function, so the best
  fallback available is `$PWD`.
2024-01-01 23:34:35 -05:00
Justine Tunney 307823ae00 Upgrade to superconfigure z0.0.29 2024-01-01 19:31:58 -08:00
Jōshin 9801f2d1db Remove COSMOPOLITAN_PROGRAM_EXECUTABLE (#1047) 2024-01-01 07:25:16 -08:00
Jōshin 68dbe5312f Prepend getcwd to exename early in init (#1048) 2024-01-01 07:23:23 -08:00
Justine Tunney 2f89c2482a Delete some dead code 2024-01-01 00:13:16 -08:00
Justine Tunney 9e6faa5256 Fix --ftrace on Windows 2024-01-01 00:00:42 -08:00
Jōshin b02d13ccde Re-enable/fix non-argv[0] exe name tests (#1046) 2023-12-31 11:53:15 -08:00
Justine Tunney 81949f038e Mint APE Loader v1.10 2023-12-31 11:43:13 -08:00
Jōshin c9550afe5e Fix loader usage, shave off a few bytes (#1016)
* Remove -f from loader usage

-f was removed in 1.5. As there is now only one flag, a couple more
bytes can be shaved off as well.

* Further loader golf

Shaves off a few bytes, paying for the cost of `RealPath` and then some
on x86_64 and offsetting some of the cost to aarch64.

* Shave off a few more bytes

Removes `-h` and flags from usage. Keeps flag-parsing logic the same,
i.e. still accepts `-h` / `--help`. Only difference is what fd and rc
the usage uses.

Still over 1k north of 8192.
2023-12-31 11:33:42 -08:00
Jōshin 10b4693e37 Sanity-check set-id interpreter script paths (#1029) 2023-12-31 07:46:27 -08:00
Jōshin 14fe83facd aarch64 loader passes os (#1042)
* Reorder Launch arguments, pass aarch64 os

Third and fourth arguments are now identical between cosmo and Launch.
By passing sp as argument 4, we save a bit of register juggling.

Fourth argument (os) is now always passed by the loader on aarch64. It
is not yet processed by cosmo. Pushing this change separately, as the
cosmo side turns out to be somewhat more involved.

* cosmo2 receives os from loader

FreeBSD aarch64 now traps early rather than pretending to be Linux.
o/aarch64/examples/env.com still works on Linux and Xnu.
2023-12-31 06:42:36 -08:00
Justine Tunney d8ad34686a Implement issetugid() on NetBSD 2023-12-30 14:58:16 -08:00
Justine Tunney 83107f78ed Introduce FreeBSD ARM64 support
It's 100% passing test fleet. Solid as a rock.
2023-12-29 20:14:02 -08:00
Justine Tunney 43fe5956ad Use DNS implementation from Musl Libc
Now that our socket system call polyfills are good enough to support
Musl's DNS library we should be using that rather than the barebones
domain name system implementation we rolled on our own. There's many
benefits to making this change. So many, that I myself wouldn't feel
qualified to enumerate them all. The Musl DNS code had to be changed
in order to support Windows of course, which looks very solid so far
2023-12-28 23:04:35 -08:00
Justine Tunney 1a28e35c62 Use good locks in dlmalloc
Using mere spin locks causes runitd.com to go painstakingly slow on
NetBSD for reasons that aren't clear yet.
2023-12-28 04:57:36 -08:00
Justine Tunney 80fca1f7c3 Make getprogramexecutablename_test pass fleet 2023-12-28 04:57:36 -08:00
Justine Tunney 8b0e42a31b Make zipos_test hermetic 2023-12-28 04:57:36 -08:00
Paul Kulchenko 6be9477b9e Fix redbean to only check additional content types when Lua is present (#1033)
Closes #971
2023-12-28 01:08:19 -08:00
Justine Tunney ce0143e2a1 Fix madvise() on Windows 2023-12-27 22:41:46 -08:00
Justine Tunney f51fd97644 Fix MODE=dbg build break 2023-12-27 22:24:27 -08:00
Jōshin 5c35863524 Rename __zipos_free -> __zipos_drop (#1043)
Removes the separate decref function, uses keep/drop in the internal
API.
2023-12-26 12:08:57 -05:00
Jōshin fd772b9b2a mmake uses $MAKE if available 2023-12-26 10:27:04 -05:00
Jōshin 4c6f928fbf Disable argv[0] tests on non-XnuSilicon (#1015)
Adds a TODO explaining the correct condition.
2023-12-24 19:43:32 -05:00
Jōshin 25266b037b Save a redundant load in zipos read/seek (#1037)
When h->pos has changed to something other than SIZE_MAX, we don't need
the extra atomic load.
2023-12-24 19:40:18 -05:00
Jōshin bb2602a524 Minor mmake cleanup 2023-12-23 11:09:50 -05:00
mataha 1bc48bc8e4 Update stb (#885)
This commit and, by extension, PR attempts to update `stb` in the most
straightforward way possible as well as include fixes from main repo's
unmerged PRs for cases rearing their ugly heads during everyday usage:

 - stb#1299: stb_rect_pack: Make rect_height_compare a stable sort
 - stb#1402: stb_image: Fix "unused invalid_chunk" with STBI_FAILURE_USERMSG
 - stb#1404: stb_image: Fix gif two_back memory address
 - stb#1420: stb_image: Improve error reporting if file operations fail
   within *_from_file functions
 - stb#1445: stb_vorbis: Few static analyzers fixes
 - stb#1487: stb_vorbis: Fix residue classdata bounding for
   f->temp_memory_required
 - stb#1490: stb_vorbis: Fix broken clamp in codebook_decode_deinterleave_repeat
 - stb#1496: stb_image: Fix pnm only build
 - stb#1497: stb_image: Fix memory leaks if stbi__convert failed
 - stb#1498: stb_vorbis: Fix memory leaks in stb_vorbis
 - stb#1499: stb_vorbis: Minor change to prevent the undefined behavior -
   left shift of a negative value
 - stb#1500: stb_vorbis: Fix signed integer overflow

Includes additional small fixes that I felt didn't warrant a separate PR.
2023-12-22 21:39:27 -08:00
Jōshin 7faffde303 Fix buffer overflow in TryPath (#1035)
Missed this when changing the code back to be like the old version.
com is now a parameter.

The only plausible way to trigger this would be to pass a loader
pathname close to MAX_PATH characters long, and then remove that
path prior to the first sys_faccessat.
2023-12-18 23:02:16 -05:00
Jōshin 85e4c020a3 mmake: formatting + aarch64 tab-completion helper
o/aarch64/* is turned into o/$mode/* in a way analogous to o//, so
aarch64 systems can use tab complete for alternate modes just like
x86_64 can.
2023-12-18 21:29:32 -05:00
Jōshin 1280797db2 Fix indentation, make llvm happy (#1032)
clang says "label at end of switch statement is a C2x extension."
2023-12-18 20:48:33 -05:00
Jōshin 58b620d8f7 Add missing bash build dependency (#1031) 2023-12-18 17:07:34 -05:00
Jōshin 2a11a09d98 Remove realpath/getcwd from loaders (#1024)
This implements proposals 1 and 2a from this gist:

https://gist.github.com/mrdomino/2222cab61715fd527e82e036ba4156b1

The only reason to use realpath from the loader was to try to prevent a
TOCTOU between the loader and the binary. But this is only a real issue
in set-id contexts, and in those cases there is already a canonical way
to do it: `/dev/fd`, passed by the kernel to the loader, so all we have
to do is pass that along to the binary.

Aside from realpath, there is no reason to absolutize the path we supply
to the binary, since it can call `getcwd` as well as we can, and on non-
M1 the binary is in a much better position to make that call.

Since we no longer absolutize the path, the binary does need to do this,
so we make its argv-parsing code generic and apply that to the different
possible places the path could come from. This means that `_` is finally
usable as a relative path, as a nice side benefit.

The M1 realpath code had a significant bug - it uses the wrong offset to
truncate the `.ape` in the `$prog.ape` case.

This PR also fixes a regression in `ape $progname` out of `$PATH` on the
two BSDs (Free and Net) that did not implement `RealPath`.
2023-12-18 15:01:16 -05:00
Jōshin f73576ab8a mmake knows about some .PHONY targets 2023-12-17 17:04:57 -05:00
Jōshin dd57c685aa mmake gets smarter
`o/$mode/*` is passed through as-is. `o/*` other than `$mode` has
`$mode` inserted. `*` has `o/$mode/` prepended.

Really leveraging zsh default tab completion here; if you have built
things with `MODE=` you can leverage that for perfect tab completion
in other modes.
2023-12-17 15:42:27 -05:00
Jōshin c11c105270 mmake doesn't even run nproc unless it needs to 2023-12-17 15:09:41 -05:00
Jōshin 3654650c52 mmake only autoloads nproc if needed 2023-12-17 15:07:10 -05:00
Jōshin 77facc4f7b Cover more cases for detecting ape loader (#1022)
Fixes a regression in GetProgramExecutableName on Linux against old
loaders. In the loader case, /proc/self/exe gives the loader's path.
We tried to detect this by checking for `/usr/bin/ape`. But that is
only one of the possible places the loader could be.
2023-12-17 10:30:22 -05:00
Jōshin b08b92248b mmake: -j$(nproc), passthrough o//
Now `mmake o/ape/ape.elf` works.
2023-12-17 01:55:52 -05:00
Jōshin 2b315626f3 Revert retabbing of net/http and tinymath (#1020) 2023-12-16 23:59:11 -05:00
Jōshin 3a8e01a77a more modeline errata (#1019)
Somehow or another, I previously had missed `BUILD.mk` files.

In the process I found a few straggler cases where the modeline was
different from the file, including one very involved manual fix where a
file had been treated like it was ts=2 and ts=8 on separate occasions.

The commit history in the PR shows the gory details; the BUILD.mk was
automated, everything else was mostly manual.
2023-12-16 23:07:10 -05:00
Jōshin 60813003a3 tool/zsh/mmake cleanup
- collects all flags into their own array to support `mmake $target -j8`

- prints out the raw make invocation to be run
2023-12-16 19:22:17 -05:00
Jōshin c0bbc45268 Add mmake zsh helper function (#1017) 2023-12-16 08:47:38 -08:00
Jōshin f94c11d978 Loader path security (#1012)
The ape loader now passes the program executable name directly as a
register. `x2` is used on aarch64, `%rdx` on x86_64. This is passed
as the third argument to `cosmo()` (M1) or `Launch` (non-M1) and is
assigned to the global `__program_executable_name`.

`GetProgramExecutableName` now returns this global's value, setting
it if it is initially null. `InitProgramExecutableName` first tries
exotic, secure methods: `KERN_PROC_PATHNAME` on FreeBSD/NetBSD, and
`/proc` on Linux. If those produce a reasonable response (i.e., not
`"/usr/bin/ape"`, which happens with the loader before this change),
that is used. Otherwise, if `issetugid()`, the empty string is used.
Otherwise, the old argv/envp parsing code is run.

The value returned from the loader is always the full absolute path
of the binary to be executed, having passed through `realpath`. For
the non-M1 loader, this necessitated writing `RealPath`, which uses
`readlinkat` of `"/proc/self/fd/[progfd]"` on Linux, `F_GETPATH` on
Xnu, and the `__realpath` syscall on OpenBSD. On FreeBSD/NetBSD, it
punts to `GetProgramExecutableName`, which is secure on those OSes.

With the loader, all platforms now have a secure program executable
name. With no loader or an old loader, everything still works as it
did, but setuid/setgid is not supported if the insecure pathfinding
code would have been needed.

Fixes #991.
2023-12-15 12:23:58 -05:00
Jōshin 8a10ccf9c4 Fix ZipOS deadlock/segfault (#1011)
This change adds a new stress test for ZipOS which helped
us improve the locking semantics in open() and close().
2023-12-14 19:59:20 -08:00
Justine Tunney 897fa6ac00 Add some Emacs keywords for CUDA 2023-12-14 19:55:56 -08:00
Jōshin 8dbfb77890 tolower() on terminfo directories (#1009)
Fixes the ncurses build on my M1 Mac. Required a make clean.
2023-12-13 12:26:12 -05:00
Jōshin 7a05fb43ac Assume __zipos_close when closing a zipos fd 2023-12-13 10:07:32 -05:00
Jōshin 3b302e6379 Cleanup zipos vfork (#1004) 2023-12-13 01:36:44 -08:00
Jōshin 6cedbc746c Fix type of refs 2023-12-13 03:03:25 -05:00
Jōshin 2fc507c98f Fix more vi modelines (#1006)
* modelines: tw -> sw

shiftwidth, not textwidth.

* space-surround modelines

* fix irregular modelines

* Fix modeline in titlegen.c
2023-12-13 02:28:11 -05:00
Justine Tunney bd10cf9d38 Whip up find command for PE binaries
Using this shell script:

    #!/bin/sh
    mkdir -p exe
    for f in $(findpe); do
      if [ -e exe/${f##*/}.exe ]; then
        cp $f exe/${f##*/}-$(rand64).exe
      else
        cp $f exe/${f##*/}.exe
      fi
    done
    rm -f /mnt/videos/microsoft.zip
    zip -rj6 /mnt/videos/microsoft.zip exe
    echo /mnt/videos/microsoft.zip

Helps file reports with Microsoft about incorrect AV detections.

See #1003
2023-12-12 10:29:03 -08:00
Justine Tunney ae85fb74b2 Update README.md
Fixes #1002
2023-12-12 07:53:23 -08:00
Justine Tunney f4a2a65cc8 Add bash 5.2 to third_party 2023-12-11 07:38:17 -08:00
Justine Tunney e1f65d5f2b Add readline 8.2 to third_party 2023-12-11 05:59:20 -08:00
Justine Tunney cbe72c4339 Fix ncurses yoinks 2023-12-10 13:11:45 -08:00
Justine Tunney 6744de659e Remove awful terminfo definitions
These files collide on MacOS and Windows due to case insensitivity.
2023-12-10 11:28:22 -08:00
Justine Tunney 4db6b74de7 Fix build warning error 2023-12-10 08:26:36 -08:00
Justine Tunney 128be2bd2e Add less 643 2023-12-10 08:01:37 -08:00
Justine Tunney 0ed7309fdb Add ncurses 6.4 2023-12-10 07:33:37 -08:00
Justine Tunney 6ee5580adc Delete GGML and RadPajama
Please use https://github.com/mozilla-Ocho/llamafile which is better,
newer, and built on cosmocc. If you need the RadPajama model, file an
issue with llamafile asking for support.
2023-12-10 06:08:54 -08:00
Justine Tunney 4a63e4a27e Reduce header complexity of PCRE2 2023-12-10 06:08:25 -08:00
Justine Tunney b27e9fe845 Add PCRE2 library to third_party 2023-12-10 05:59:18 -08:00
Justine Tunney 4f66d7f2dd Add WIN32 pseudo console APIs
See #999
2023-12-10 01:29:25 -08:00
Justine Tunney 1bd69867d1 Explain how to build FOSS in cosmocc README 2023-12-09 06:58:24 -08:00
Joseph Battelle b0cced855d Move tcgetpgrp and tcsetpgrp decls to unistd.h (#996) (#997) 2023-12-08 22:54:46 -08:00
Justine Tunney 076e0bf9a9 Re-disable WIN32 call tracing 2023-12-08 20:06:16 -08:00
Justine Tunney 1a96de6eda Add libresolv from Musl Libc
Locally modified to get nameservers from Windows Registry when
`\etc\resolv.conf` isn't defined.
2023-12-08 20:04:10 -08:00
Justine Tunney 8874a37abc Add <link.h> for absl 2023-12-08 20:04:10 -08:00
Zoey Greer a11016e590 Use gid_t and uid_t in passwd.h (#995) 2023-12-08 18:27:41 -08:00
Jōshin e16a7d8f3b flip et / noet in modelines
`et` means `expandtab`.

```sh
rg 'vi: .* :vi' -l -0 | \
  xargs -0 sed -i '' 's/vi: \(.*\) et\(.*\)  :vi/vi: \1 xoet\2:vi/'
rg 'vi: .*  :vi' -l -0 | \
  xargs -0 sed -i '' 's/vi: \(.*\)noet\(.*\):vi/vi: \1et\2  :vi/'
rg 'vi: .*  :vi' -l -0 | \
  xargs -0 sed -i '' 's/vi: \(.*\)xoet\(.*\):vi/vi: \1noet\2:vi/'
```
2023-12-07 22:17:11 -05:00
Justine Tunney 90c53f3064 Fix some references to stdio/internal.h 2023-12-07 16:30:49 -08:00
Justine Tunney f8ea02d4d1 Import /etc/shadow support from Musl for Linux
Fixes #992
2023-12-07 16:26:04 -08:00
Justine Tunney 06ace4e7b4 Always use compiler builtin for offsetof()
We have received multiple reports of GCC breaking builds when compiler
flags like `-std=c11` were being passed. The workaround until the next
release is to simply not define `__STRICT_ANSI__` which is a bad idea.
2023-12-07 14:20:12 -08:00
Justine Tunney e36283f1d9 Support double parameters in dlsym() imports 2023-12-06 06:49:35 -08:00
Justine Tunney 4772796cd8 Improve cosmo_dlopen() documentation 2023-12-06 06:33:50 -08:00
Justine Tunney 7c39818c13 Simplify GNU Make build config 2023-12-06 03:25:16 -08:00
Jōshin 394d998315 Fix vi modelines (#989)
At least in neovim, `│vi:` is not recognized as a modeline because it
has no preceding whitespace. After fixing this, opening a file yields
an error because `net` is not an option. (`noet`, however, is.)
2023-12-05 14:37:54 -08:00
Justine Tunney 221a27836b Define max_align_t in <stddef.h>
It was previously only being defined in <cstddef>

See oilshell/oil#1760
2023-12-05 00:52:33 -08:00
Justine Tunney 536dfb947a Make getprogramexecutablename_test report status
Fixes #988
2023-12-04 23:19:28 -08:00
Justine Tunney cd52c59552 Allow MAP_POPULATE under pledge() 2023-12-04 23:15:41 -08:00
Jōshin 53357aa26a Fix __zipos_close (#984) 2023-12-04 20:24:26 -08:00
Jōshin 577bb180b7 benchmark, getenv, test cleanup (#987) 2023-12-04 20:01:52 -08:00
Jōshin da8baf2aa5 ape-m1 minor formatting cleanup (#986) 2023-12-04 19:58:32 -08:00
Jōshin ed8fadea37 Keep argv[0], add COSMOPOLITAN_PROGRAM_EXECUTABLE (#980)
* Introduce env.com

Handy tool for debugging environment issues.

* Inject path as COSMOPOLITAN_PROGRAM_EXECUTABLE

`argv[0]` was previously being used as a communication channel between
the loader and the binary, giving the binary its full path for use e.g.
in `GetProgramExecutableName`. But `argv[0]` is not a good channel for
this; much of what made 2a3813c6 so gross is due to that.

This change fixes the issue by preserving `argv[0]` and establishing a
new communication channel: `COSMOPOLITAN_PROGRAM_EXECUTABLE`.

The M1 loader will always set this as the first variable. Linux should
soon follow. On the other side, `GetProgramExecutableName` checks that
variable first. If it sees it, it trusts it as-is.

A lot of the churn in `ape/ape-m1.c` in this change is actually backing
out hacks introduced in 2a3813c6; the best comparison is:

    git diff 2a3813c6^..
2023-12-04 12:45:46 -08:00
Jōshin 2a3813c6cf $prog.ape support (#977)
* ape loader: $prog.ape + login shell support

If the ape loader is invoked with `$0 = $prog.ape`, then it searches for
a `$prog` in the same directory as it and loads that. In particular, the
loader searches the `PATH` for an executable named `$prog.ape`, then for
an executable named `$prog` in the same directory. If the former but not
the latter is found, the search terminates with an error.

It also handles the special case of getting started as `-$SHELL`, which
getty uses to indicate that the shell is a login shell. The path is not
searched in this case, and the program location is read straight out of
the `SHELL` variable.

It is now possible to have `/usr/local/bin/zsh.ape` act as a login shell
for a `/usr/local/bin/zsh` αpε, insofar as the program will get started
with the 'correct' args. Unfortunately, many things break if `$0` is not
the actual full path of the executable being run; for example, backspace
does not update the display properly.

To work around the brokenness introduced by not having `$0` be the full
path of the binary, we cut the leading `-` out of `argv[0]` if present.
This gets the loader's behavior with `$prog.ape` up to par, but doesn't
tell login shells that they are login shells.

So we introduce a hack to accomplish that: if ape is run as `-$prog.ape`
and the shell is `$prog`, the binary that is loaded has a `-l` flag put
into its first argument.

As of this commit, αpε binaries can be used as login shells on OSX.

* if islogin, execfn = shell

Prior to this, execfn was not being properly set for login shells that
did not receive `$_`, which was the case for iTerm2 on Mac. There were
no observable consequences of this, but fixing it seems good anyway.

* Fix auxv location calculation

In the non-login-shell case, it was leaving a word of uninitialized
memory at `envp[i] + 1`. This reuses the previous calculation based
on `envp`.
2023-12-03 19:39:32 -08:00
Jōshin 8dd348067b refs is atomic_size_t (#976)
The unassert covers all invalid values rather than a third of them.
2023-12-02 16:48:58 -08:00
Justine Tunney 450d9b1d43 Release Cosmopolitan v3.1.3 2023-12-01 10:29:59 -08:00
Justine Tunney 69ba516466 Revert "Re-enable apelink size optimization"
This reverts commit 7b3d7ee07b.

It broke llava-v1.5-7b-q4-server.llamafile on Windows.
2023-12-01 10:26:17 -08:00
Justine Tunney 9e9d2c0006 Release Cosmopolitan v3.1.2 2023-12-01 06:47:57 -08:00
Jōshin a0f599ce12 Fix memory_order_release in __zipos_seek (#975) 2023-12-01 02:28:15 -08:00
Jōshin d95d61b1af Better zipos refcounts and atomic reads/seeks (#973)
* Better refcounting

Cribbed from [Rust Arc][1] and the [Boost docs][2]:

"""
Increasing the reference counter can always be done with
memory_order_relaxed: New references to an object can only be formed
from an existing reference, and passing an existing reference from one
thread to another must already provide any required synchronization.

It is important to enforce any possible access to the object in one
thread (through an existing reference) to happen before deleting the
object in a different thread. This is achieved by a "release" operation
after dropping a reference (any access to the object through this
reference must obviously happened before), and an "acquire" operation
before deleting the object.

It would be possible to use memory_order_acq_rel for the fetch_sub
operation, but this results in unneeded "acquire" operations when the
reference counter does not yet reach zero and may impose a performance
penalty.
"""

[1] https://moshg.github.io/rust-std-ja/src/alloc/arc.rs.html
[2] https://www.boost.org/doc/libs/1_55_0/doc/html/atomic/usage_examples.html

* Make ZiposHandle's pos atomic

Implements a somewhat stronger guarantee than POSIX specifies: reads and
seeks are atomic. They may be arbitrarily reordered between threads, but
each one happens all the way and leaves the fd in a consistent state.

This is achieved by "locking" pos in __zipos_read by storing SIZE_MAX to
pos during the operation, so only one can be in-flight at a time. Seeks,
on the other hand, just update pos in one go, and rerun if it changed in
the meantime.

I used `LIKELY` / `UNLIKELY` to pessimize the concurrent case; hopefully
that buys back some performance.
2023-12-01 01:01:03 -08:00
Jōshin f0bfabba07 Revert "Remove noop __zipos_postdup" (#974)
`close` does not modify the fd table, so `rc` can potentially have a
stale zipos object sitting on it. As such, we need `__zipos_postdup`
there.
2023-12-01 00:51:01 -08:00
Jōshin 7845495a93 Get tool/cosmocc/package.sh working on M1 (#970)
On aarch64 hosts, MODE= is changed to MODE=aarch64, so o// targets don't
work. So On aarch64, get apelink.com out of o/aarch64/. Also prepend ape
when calling it. And finally, fetch with curl when wget isn't installed.
2023-12-01 00:08:55 -08:00
Jōshin d1a745c17c Implement __zipos_dup (#972)
* Implement __zipos_dup

Makes ZiposHandle reference-counted by an `rc` field in a union with its
freelist `next` pointer. The functions `__zipos_free` and `__zipos_keep`
function as incref/decref for it. Adds `__zipos_postdup` to fix metadata
on file descriptors after dup-like operations, and adds zipos support to
`sys_dup_nt` + `sys_close_nt`.

* Remove noop __zipos_postdup

rc is never a zipos file because it is always a previously unused file
descriptor. fd is never a zipos file because that case has been handled
above by __zipos_fcntl.
2023-12-01 00:08:30 -08:00
Justine Tunney 6556dd2673 Add $(uniq token...) native function to Make
This reduces mono repo `make` latency from 336ms to 226ms.
2023-11-30 21:02:16 -08:00
Justine Tunney 14bf57180f Import GNU Make 4.4.1
Landlock Make hasn't been working well on AARCH64 systems. Let's do this
over the right way, using our new build tools.
2023-11-30 20:54:52 -08:00
Justine Tunney 9315ebbfd9 Fix MODE=tinylinux build 2023-11-30 15:38:08 -08:00
mataha 879bb84244 Update quickjs (#890)
Includes additional fixes from main repo's unmerged PRs:

 - quickjs#132: Fix undefined behavior: shift 32 bits for uint32_t in bf_set_ui
 - quickjs#171: Fix locale-aware representation of hours in Date class
 - quickjs#182: Fix stack overflow in CVE-2023-31922
2023-11-30 10:51:16 -08:00
Justine Tunney 4b7ba9a4c5 Fix some bugs with dup2() and ZipOS
On UNIX if dup2(newfd) was a ZipOS file descriptor, then its resources
weren't being released, and the newly created file descriptor would be
mistaken for ZipOS due to its memory not being cleared. On Windows, an
issue also existed relating to newfd resources not being released.
2023-11-30 10:10:02 -08:00
Justine Tunney 26c70e08bf Make it possible to package.sh cosmocc w/ modes 2023-11-30 07:12:15 -08:00
Justine Tunney 20938a45e6 Add appropriate chmod to README instructions
See Mozilla-Ocho/llamafile#13
2023-11-30 03:35:17 -08:00
Justine Tunney 7b3d7ee07b Re-enable apelink size optimization
I'm no longer able to reproduce the PE import table corruption that was
previously observed. Since this optimization shaves up to 64kb off each
fat APE binary we build, it's worth turning this back on again, to wait
and see if something breaks, and if so, fix it. At least until the next
release is shipped.

See #965
2023-11-29 05:41:23 -08:00
Justine Tunney ff955aaa01 Make memcmp() and memchr() go fast again
Readahead within the specified size is legal, even if it overlaps a page
boundary; it's the fault of the caller if that causes a segfault.
2023-11-29 05:17:21 -08:00
Justine Tunney 70155df7a9 Avoid linking win32 signals impl
This shaves ~4kb off o/tiny/examples/hello2.com by avoiding linking the
WIN32 signals polyfill unless sigaction() is being used.

See #965
2023-11-29 04:09:31 -08:00
Justine Tunney 2b960bb249 Exclude strace from MODE=tiny builds
This change gets o/tinylinux/examples/hello2.com back down to 8kb in
size which had been unintentionally bloated to 40kb in recent months

See #965
2023-11-29 03:45:54 -08:00
Justine Tunney b7e1dc81c2 Release Cosmopolitan v3.1.1 2023-11-29 02:19:56 -08:00
Justine Tunney 41c06c6325 Make cosmocc 100x faster on MacOS
This change upgrades to superconfigure z0.0.23 which fixes an issue
where the compiler had harmless /home/... paths baked-in, which are
normally only present in the build environment, and usually skipped
over. Sadly on MacOS calling fstatat() on these paths would lead to
cloud file system ops that caused system calls to take a long time.
That's problematic, since cosmocc needs to be a 100% local command.
2023-11-29 01:48:17 -08:00
Justine Tunney 4427581a05 Cleanup inline locking code 2023-11-29 00:36:22 -08:00
Justine Tunney 96185e1ac0 Fix MODE=aarch64 cross-compiling
This change fixes a regression that happened some time ago when building
for AARCH64 using the vendored toolchain rather than cosmocc. The errors
that would show up `Relocations in generic ELF (EM: 62)` have been fixed
2023-11-28 22:40:06 -08:00
Justine Tunney ca5e361919 Don't show (win32 error 0) 2023-11-28 21:30:20 -08:00
Justine Tunney 9cad66aa11 Freshen bootstrap binaries for MacOS + SIP 2023-11-28 21:18:31 -08:00
Justine Tunney 03de761890 Update cosmocc to use superconfigure z0.0.23 2023-11-28 20:04:15 -08:00
Justine Tunney 43de12db55 Introduce forkpty() 2023-11-28 19:59:57 -08:00
Justine Tunney fa20edc44d Reduce header complexity
- Remove most __ASSEMBLER__ __LINKER__ ifdefs
- Rename libc/intrin/bits.h to libc/serialize.h
- Block pthread cancelation in fchmodat() polyfill
- Remove `clang-format off` statements in third_party
2023-11-28 14:39:42 -08:00
Justine Tunney 96f979dfc5 Rename makefiles BUILD.mk
This way they appear at the top of directory listings.
2023-11-28 11:21:08 -08:00
Paul Kulchenko 0bffd09433 Allow redbean to accept --assimilate option without complaining (#916)
This allows using --assimilate on Windows or already assimilated binary.
Ref #356.
2023-11-19 19:02:30 -08:00
Stephen Gregoratto cc5c5319bf Linux: Add cachestat, fchmodat2 syscalls (#958) 2023-11-19 19:01:20 -08:00
Justine Tunney 69faf1b403 Release Cosmpolitan v3.1 2023-11-18 20:40:35 -08:00
Justine Tunney e4dea37b8e Make clock_nanosleep() cancel faster 2023-11-18 18:12:09 -08:00
Justine Tunney 0c89516ac5 Fix unzip warning with apelink generated symtabs 2023-11-18 17:15:22 -08:00
Justine Tunney 3e6d536822 Correct misunderstanding with zip64 extra records 2023-11-18 14:35:57 -08:00
Justine Tunney dbd8176ea8 Add CPU_COUNT_S() 2023-11-18 12:38:30 -08:00
Justine Tunney 545a8f4cb0 Add more sched.h content 2023-11-18 08:08:15 -08:00
Justine Tunney f7cfe03888 Fix dlopen() for FreeBSD and NetBSD 2023-11-18 04:35:48 -08:00
Justine Tunney 2c1efd3d78 Fix Python build break 2023-11-18 02:32:01 -08:00
Matheus Moreira 3ac473df3b Floating point parsing support for scanf family (#924) 2023-11-18 02:25:36 -08:00
Justine Tunney 8caf1b48a9 Improve time/sleep accuracy on Windows
It's now almost as good as Linux thanks to a Windows 8+ API.
2023-11-18 01:57:44 -08:00
Justine Tunney 72ac5f18d9 Fix dash compatible syntax issue 2023-11-17 16:47:47 -08:00
Justine Tunney 2c9d2943d6 Introduce AddDllDirectory() 2023-11-17 10:35:34 -08:00
Justine Tunney 32b97f2d25 Improve execve() path argument munging
Munging of paths passed inside the system() interpreter command is no
longer supported. You have to pass your paths to posix_spawn() or the
execve() family of functions if you want them to be munged. The first
three characters must match `^/[a-z]/` in which case, it'll be turned
into a DOS-style drive path with backslashes.
2023-11-17 09:59:03 -08:00
Justine Tunney 529cb4817c Improve dlopen() on Apple Silicon
- Introduce MAP_JIT which is zero on other platforms
- Invent __jit_begin() and __jit_end() which wrap Apple's APIs
- Runtime dispatch to sys_icache_invalidate() in __clear_cache()
2023-11-17 02:33:14 -08:00
Justine Tunney 7a9e176ecf Improve debug binary location detection 2023-11-17 00:07:39 -08:00
Justine Tunney 68c7c9c1e0 Clean up some code
- Use good ELF technique in cosmo_dlopen()
- Make strerror() conform more to other libc impls
- Introduce __clear_cache() and use it in cosmo_dlopen()
- Remove libc/fmt/fmt.h header (trying to kill off LIBC_FMT)
2023-11-16 17:31:07 -08:00
tkchia 7010a8081e Fix regression when building deathstar.com for aarch64 (#957)
It turns out my earlier commit ddc08dc974 caused a build with
MODE=aarch64 to fail.  The commit changed deathstar.c to link
in code to support a VGA console, but this is not implemented
yet for AArch64.  Thanks to @ahgamut for spotting this issue.
2023-11-16 12:13:17 -08:00
tkchia 665747a319 Make test/libc/calls/pledge_test.c work with "nonstandard" umask (#956)
The test was failing if the process's umask happened to be
0077, for example.  The file `foo` was then created with a
file mode of 0100600, rather than the expected 0100644.
2023-11-16 00:47:21 -08:00
Justine Tunney e05933c641 Mention zsh and fish in readme 2023-11-15 23:25:47 -08:00
Justine Tunney 8f5e516b39 Remove sync_file_range()
After hearing horror stories from a trusted colleague, I don't think
this is the kind of API we want to be supporting. Also SQLite wisdom
regarding fdatasync() has been added to the documentation.
2023-11-15 23:21:22 -08:00
Justine Tunney 8318d67503 Fix regression in cosmocc toolchain 2023-11-15 21:41:53 -08:00
Justine Tunney 1351d3cede Remove bool from public headers 2023-11-15 20:58:46 -08:00
Justine Tunney dffee606cf Run host compiler to create dlopen helper 2023-11-15 20:58:46 -08:00
tkchia ddc08dc974 [metal] Make deathstar.com draw onto both VGA and serial consoles (#955)
In addition, deathstar.com can now be booted up via UEFI.
2023-11-15 19:40:54 -08:00
Justine Tunney e4584ace81 Get cosmo_dlopen() working better on System Five
Imported functions are now aspected with a trampoline that blocks
signals and changes the thread-local storage register. This means
bigger more complicated libraries can now be imported even though
the whole technique remains fundamentally unsafe.
2023-11-15 10:56:30 -08:00
Justine Tunney 3a470ed356 Remove some exploratory some tests
These were reported as failing on some Apple Arm64 systems.
2023-11-14 21:21:22 -08:00
Justine Tunney 9f9aec013a Do better job documenting platform issues 2023-11-14 18:15:58 -08:00
tkchia eea601f346 [metal] Some minor fixes and tweaks (#933)
* [metal] Ensure DF is clear when calling C from exception handler
* [metal] Mark some internal routines and declarations as `@internal`
* [metal] Fix crash under UEFI when command line string is NULL
* [metal] Fix argc & argv[] setting, & VM page freeing, for UEFI

Part of the memory occupied by the argv[] contents was
erroneously used for page tables & then later erroneously
freed.  The symptom was that argv[0] would show up as an
empty string ("").
2023-11-14 15:26:59 -08:00
Justine Tunney 1c2e7c1333 Introduce SIP_DISABLED compile option for ape-m1.c
Systems that don't use SIP can now build APE Loader with this flag to
get a performance speedup.
2023-11-13 22:04:05 -08:00
Justine Tunney 0283f2772c Fix build in MODE=dbg 2023-11-13 15:13:45 -08:00
Justine Tunney 751d20d98d Fix nsync_mu_unlock_slow_() on Apple Silicon
We torture test dlmalloc() in test/libc/stdio/memory_test.c. That test
was crashing on occasion on Apple M1 microprocessors when dlmalloc was
using *NSYNC locks. It was relatively easy to spot the cause, which is
this one particular compare and swap operation, which needed to change
to use sequentially-consistent ordering rather than an acquire barrier
2023-11-13 11:07:13 -08:00
Justine Tunney 3b15d31247 Fix #include <shared_mutex> 2023-11-13 08:30:17 -08:00
Justine Tunney f047172bd3 Fix uname assignment in Makefile 2023-11-13 05:53:10 -08:00
mattx 5d5b282d12 Inform user when wsl2 interop causes build issues (#952) 2023-11-13 05:29:25 -08:00
Justine Tunney 25a892d6ae Add some platform notes to the README file 2023-11-13 02:26:34 -08:00
Justine Tunney d1e764b3cf Modernize README.md 2023-11-12 22:04:07 -08:00
Paul Kulchenko 5c1fdc9819 Fix session changeset procesing (reverting some of 0d748ad5 changes) (#949)
This removes strdup from Lua SQLite wrapper, as it's applied to binary changeset
strings, thus cutting them short (because of strlen).
2023-11-12 14:25:50 -08:00
Justine Tunney f25beb341c Add architecture flag to zipobj 2023-11-12 09:35:59 -08:00
Justine Tunney bed77186c3 Use simple locks in dlmalloc 2023-11-12 09:00:49 -08:00
Justine Tunney bcf268adf8 Don't modify argument block on MacOS Arm64
Some dynamic library had access to this information somehow and was
crashing when it didn't have the expected structure.
2023-11-12 05:59:03 -08:00
Justine Tunney 0863427b3a Make development more pleasant on MacOS Arm64 2023-11-12 05:42:17 -08:00
Justine Tunney f7cad70da1 Introduce posix tests package 2023-11-12 05:41:48 -08:00
Justine Tunney 48bd3d85df Fix sigaction() return handler address on XNU/BSD 2023-11-12 05:40:06 -08:00
Justine Tunney bd56a9cf51 Rename dlopen() to cosmo_dlopen() 2023-11-12 01:19:04 -08:00
Justine Tunney c6d3802d3a Add more fixes for new cosmocc toolchain
We now have an `#include <cxxabi.h>` header which defines all the APIs
Cosmopolitan's implemented so far. The `cosmocc` README.md file is now
greatly expanded with documentation.
2023-11-11 23:28:19 -08:00
Gautham 95124cacbe Fixes for building superconfigure (#948) 2023-11-11 22:11:11 -08:00
Justine Tunney b320321319 Add build/bootstrap/rollup.com 2023-11-11 19:58:45 -08:00
Justine Tunney 291103ad8d Redesign cosmocc toolchain
The `cosmocc` compiler is now being distributed as a self-contained
toolchain that's path-agnostic and it no longer requires you clone the
Cosmop repo to use it. The bin/ folder has been deleted from the mono
repo. The `fatcosmocc` command has been renamed to `cosmocc`. MacOS
support now works very well.
2023-11-11 14:18:08 -08:00
Justine Tunney 3802428026 Fix missing .ok file make warning 2023-11-11 02:32:22 -08:00
Justine Tunney d0ad2694ed Iterate more on recent changes 2023-11-11 00:28:22 -08:00
Paul Kulchenko 7138399f96 Close Lua state before exiting redbean worker to allow GC to collect (#941) 2023-11-10 18:37:00 -08:00
Justine Tunney cdbd10c9d0 Introduce X86_CHECK() macro 2023-11-10 15:41:55 -08:00
Justine Tunney afbef6c361 Freshen build/bootstrap/mkdeps.com 2023-11-10 12:39:52 -08:00
Justine Tunney 3d3cdcf276 Fix APE uname direction
Fixes #942
2023-11-10 12:34:25 -08:00
Justine Tunney d2f49ca175 Improve mkdeps
Our makefile generator now accepts badly formatted include lines. It's
now more hermetic with better error checking in the cosmo repo, and it
can be configured to not be hermetic at all.
2023-11-10 04:14:27 -08:00
Justine Tunney 241f949540 Use dynamic memory for *NSYNC waiters 2023-11-10 01:42:06 -08:00
Justine Tunney 15af5c2d7e Fix ucontext_t alignment in strict ansi mode
Using `cosmocc -std=c11` was causing `ucontext_t` to become misaligned.
This change also adds the GNU constants on x86_64 for accessing general
registers, so you will not need `#ifdef`s to support both Cosmo and GNU
2023-11-09 12:48:29 -08:00
Justine Tunney b8d1377ae1 Give new wait4() another review pass 2023-11-09 11:23:30 -08:00
Justine Tunney ea28f93a26 Support <isystem> includes in monorepo 2023-11-08 19:14:56 -08:00
Justine Tunney 5330442d85 Avoid APE fork bomb
This change addresses a $PATH resolution issue where APE depends on
uname and uname is an APE program. So sorry to anyone this impacted
we'll get a release out soon.
2023-11-08 11:37:42 -08:00
mataha cc3e3de044 Make args replace ... only at the end of file (#909)
So that the string '...' can be used verbatim elsewhere in the file.
2023-11-08 10:00:52 -08:00
Justine Tunney ac125d3e1f Fix console copy/paste issue
Fixes #936
2023-11-08 09:29:45 -08:00
Justine Tunney 956e68be59 Revert "Use %gs as TLS register when dlopen() is linked"
This reverts commit d71da7fc72.
2023-11-08 01:33:01 -08:00
Justine Tunney d71da7fc72 Use %gs as TLS register when dlopen() is linked
Fixes #938
2023-11-08 01:11:17 -08:00
Justine Tunney e961385e55 Put more thought into i/o polyfills
wait4() is now solid enough to run `make -j100` on Windows. You can now
use MSG_DONTWAIT on Windows. There was a handle leak in accept() that's
been fixed. Our WIN32 overlapped i/o code has been simplified. Priority
class now inherits into subprocesses, so the verynice command will work
and the signal mask will now be inherited by execve() and posix_spawn()
2023-11-07 18:32:35 -08:00
Justine Tunney 736fdb757a Implement raise() with getcontext() / setcontext() 2023-11-05 18:04:36 -08:00
Justine Tunney dd83db9567 Improve makefile for uname 2023-11-05 16:21:09 -08:00
Justine Tunney 23d812f116 Release Cosmopolitan v3.0.2
It's recommended that all users, especially Apple Arm, re-run:

    ape/apeinstall.sh

Within the Cosmopolitan monorepo to refresh your APE loader.
2023-11-05 15:54:47 -08:00
Justine Tunney c39eb244d4 Disable an mprotect() test for now due to SIP 2023-11-05 15:38:16 -08:00
Justine Tunney f63c4d4f52 Use /usr/local/bin/ape on Apple Silicon 2023-11-05 14:52:27 -08:00
Paul Kulchenko b893479417 Add database and statement readonly checks to sqlite in redbean (#914) 2023-11-05 13:29:09 -08:00
Justine Tunney d02cba2451 Mint APE Loader v1.9 2023-11-05 13:11:34 -08:00
Justine Tunney 4cf987ddb1 Fix usleep() definition 2023-11-05 13:06:15 -08:00
Justine Tunney 20c794a353 Add strace to aarch64 vfork() 2023-11-05 13:06:15 -08:00
Gregory Haerr 4f100d2aad Fix stack alignment and auxv for macho APE loader (#931) 2023-11-05 13:06:10 -08:00
Justine Tunney a12ad17291 Get APE Loader working on MacOS with SIP enabled 2023-11-05 01:36:47 -08:00
Justine Tunney 653c7b2f87 Fatten build/bootstrap/echo.com 2023-11-04 21:01:43 -07:00
Justine Tunney d7917ea076 Make win32 i/o signals atomic and longjmp() safe 2023-11-04 20:33:29 -07:00
Justine Tunney 585c86e2a4 Support \n in /zip/.args files 2023-11-04 07:25:20 -07:00
Justine Tunney 48e260e653 Introduce NO_ADDRESS constant 2023-11-03 13:56:17 -07:00
Justine Tunney aa106c9356 Fix Lua test suite regression 2023-11-03 07:01:46 -07:00
Justine Tunney 5e8c928f1a Introduce dlopen() support
Every program built using Cosmopolitan is statically-linked. However
there are some cases, e.g. GUIs and video drivers, where linking the
host platform libraries is desirable. So what we do in such cases is
launch a stub executable using the host platform's libc, and longjmp
back into this executable. The stub executable passes back to us the
platform-specific dlopen() implementation, which we shall then wrap.

Here's the list of platforms that are supported so far:

- x86-64 Linux w/ Glibc
- x86-64 Linux w/ Musl Libc
- x86-64 FreeBSD
- x86-64 Windows
- aarch64 Linux w/ Glibc
- aarch64 MacOS

What this means is your Cosmo programs can call foreign functions on
your host operating system. However, it's important to note that any
foreign library you link won't have the ability to call functions in
your Cosmopolitan program. For example it's now technically possible
that Lua can load a module, however that almost certainly won't work
since the Lua module won't have access to Cosmo's Lua API.

Kudos to @jacereda for figuring out how to do this.
2023-11-03 06:37:18 -07:00
Justine Tunney 1eb6484c9c Rewrite getcwd()
This change addresses a bug that was reported in #923 where bash on
Windows behaved strangely. It turned out that our weak linking of
malloc() caused bash's configure script to favor its own getcwd()
function, which is implemented in the most astonishing way, using
opendir() and readdir() to recursively construct the current path.

This change moves getcwd() into LIBC_STDIO so it can strongly link
malloc(). A new __getcwd() function is now introduced, so all the
low-level runtime services can still use the actual system call. It
provides the Linux Kernel API convention across platforms, and is
overall a higher-quality implementation than what we had before.

In the future, we should probably take a closer look into why bash's
getcwd() polyfill wasn't working as intended on Windows, since there
might be a potential opportunity there to improve our readdir() too.
2023-11-02 13:16:42 -07:00
Justine Tunney a46ec61787 Make cosmocc -dumpmachine work correctly 2023-11-02 09:52:25 -07:00
Justine Tunney 024be3b009 Introduce getifaddrs()
This function was invented by the BSDs (it's not in POSIX.1). It
provides a high-level interface into ioctl(SIOCGIFCONF) which is
comparatively clumsy to use. We already made the ioctls portable
across our entire support vector back in 2021, so this interface
is portable too. See o//tool/viz/getifaddrs.com for our demo app
2023-11-02 08:33:03 -07:00
Justine Tunney 6ca5ab4680 Mention cosmocc-0.0.16 in Makefile 2023-11-01 12:08:50 -07:00
Justine Tunney 55d5d6bbc0 Release Cosmopolitan v3.0.1 2023-11-01 09:23:43 -07:00
Justine Tunney d259e573b6 Remove *NSYNC WIN32 semaphores
This implementation doesn't work as well as WIN32 futexes. This code
path was only added back when we were having issues with set context
however that's been solved so we can go back to the much better code
2023-11-01 07:18:58 -07:00
Justine Tunney 7b284f6bda Fix bugs and regressions in the pledge command
This change gets the pledge (formerly pledge.com) command back in tip
top shape for a 3.0.1 cosmos release. It now runs on all platforms, even
though it's mostly a no-op on ones that lack the kernel security stuff.
The binary footprint is now smaller, since it no longer needs to link
malloc. It's also now able to be built as a fat binary.
2023-11-01 06:08:58 -07:00
Justine Tunney b0e3d89942 Work around qemu-aarch64 bug
Qemu appears to define O_LARGEFILE as having its x86-64 value, which is
an easy mistake to make since this is one of the few magic numbers that
Linux special-cases for AARCH64.
2023-11-01 00:24:48 -07:00
Justine Tunney 0b1acce680 Introduce shm_open() and shm_unlink() 2023-10-31 23:57:52 -07:00
Justine Tunney fadb64a2bf Introduce pthread_rwlock_try{rd,wr}lock
This also changes recursive mutexes to favor cpu over scheduler yield.
2023-10-31 22:13:08 -07:00
Justine Tunney a1e1e821cb Introduce GNU/BSD timeval macros 2023-10-31 21:58:49 -07:00
Justine Tunney 2af7c802b6 Stub out getutline and pututline 2023-10-31 21:58:39 -07:00
Justine Tunney 3b791d2f44 Introduce lgammal, tgammal, erfl, and erfcl
git://git.musl-libc.org/musl
79bdacff83a6bd5b70ff5ae5eb8b6de82c2f7c30
2023-10-31 21:58:29 -07:00
Justine Tunney b4084dd6c6 Introduce libcxx <shared_mutex>
[email protected]:llvm-mirror/libcxx.git
78d6a7767ed57b50122a161b91f59f19c9bd0d19
2023-10-31 21:57:28 -07:00
Justine Tunney ee82f90bba Introduce __cxa_thread_atexit() 2023-10-31 20:04:31 -07:00
Justine Tunney a699cda5c8 Make cosmocc -Wl,--version do the right thing
Our `cosmocc` wrapper script will now report the linker version, the
same way as the `cc` command. The `fatcosmocc` command won't do this
because we wouldn't want programs to parse out the path of the amd64
linker command and use it explicitly.

Fixes #917
2023-10-31 18:20:28 -07:00
Justine Tunney f86ffa1a25 Fix aarch64 build breakage caused by ed17d3008 2023-10-31 18:17:32 -07:00
Justine Tunney a09f62141d Fix bug in login_tty() 2023-10-31 18:03:13 -07:00
tkchia ed17d3008b [metal] Add a uprintf() routine, for non-emergency boot logging (#905)
* [metal] Add a uprintf() routine, for non-emergency boot logging
* [metal] _Really_ push forward timing of VGA TTY initialization
* [metal] Do something useful with uprintf()
* [metal] Locate some ACPI tables, for later hardware detection

Specifically the code now tries to find the ACPI RSDP,
RSDT/XSDT, FADT, & MADT tables, whether in legacy BIOS
bootup mode or in a UEFI bootup.  These are useful for
figuring out how to (re)enable asynchronous interrupts
in legacy 8259 PIC mode.
2023-10-25 14:32:20 -07:00
Justine Tunney 062b2d776e Mention cosmocc v0.0.14 in Makefile
https://justine.lol/cosmocc-0.0.14.zip
https://github.com/ahgamut/superconfigure/releases/tag/z0.0.14
2023-10-16 00:07:33 -07:00
13645 changed files with 972467 additions and 407881 deletions

No files matched your search

+2
View File
@@ -6,6 +6,8 @@ AlignConsecutiveMacros: true
AlignConsecutiveDeclarations: false
AlwaysBreakBeforeMultilineStrings: false
AllowShortFunctionsOnASingleLine: false
AllowShortLoopsOnASingleLine: false
AllowShortIfStatementsOnASingleLine: false
KeepEmptyLinesAtTheStartOfBlocks: true
ConstructorInitializerAllOnOneLineOrOnePerLine: true
IncludeBlocks: Merge
+31
View File
@@ -0,0 +1,31 @@
# Run this command to always ignore formatting commits in git blame
# git config blame.ignoreRevsFile .git-blame-ignore-revs
# Upgraded clang-format to 19.0.0git
89fc95fefd40413706ff1bdaac2e8d98b328dbf1
# vim c++ filetype in modelines
04c6bc478e082263d67c41bedbd033dde2d429eb
# Ran clang-format
f032b5570b4cd87c6bb4abb54c0b98e69c939955
# Applied clang-format update to repo
6e6fc38935054db0534d5af4fb99c6193305b946
# revert retabbing
2b315626f3af765cdfbc61114647412cdb798b3a
# more modeline errata
3a8e01a77a7c97af0b16fb1651b230cee7f7d4c6
# fix more vi modelines
2fc507c98f53a76718f61f9a36602f86b5ac0cc9
# flip et/noet in modelines
e16a7d8f3b8f906c3ef76e79f57f3adfc7f25186
# fix vi modelines
394d998315f613a888cc6b6c051d4163bdf5cd6f
# clang-format
c0eacf2eb1e1c0b3bd4f71f12fef258f5b249c3f
# ape-m1 formatting cleanup
da8baf2aa5ce93b958aca90a0ae69f537806324b
# Run clang-format on most sources
369f9740de4534c28d0e81ab2afc99decbb9a3e6
# Get rid of .internal.h convention in LIBC_INTRIN
86d884cce24d773e298a2714c1e3d91ecab9be45
# Remove .internal from more header filenames
31194165d2afca36c2315a6e7ca2f0797dde09e3
+59
View File
@@ -0,0 +1,59 @@
name: Low Severity Bugs
description: Used to report low severity bugs in cosmopolitan (e.g. cosmetic issues, non critical UI glitches)
title: "Bug: "
labels: ["bug", "low severity"]
body:
- type: markdown
attributes:
value: |
Thanks for taking the time to fill out this bug report!
Please include information about your system, the steps to reproduce the bug,
and the version of cosmopolitan that you are using.
If possible, please provide a minimal code example that reproduces the bug.
You may also consider using function call tracing `--ftrace` or the lighter system call tracing `--strace`
for additional technical logging that may allow us to narrow down where the fault occurred.
- type: input
id: contact
attributes:
label: Contact Details
description: How can we get in touch with you if we need more info?
placeholder: ex. [email protected]
validations:
required: false
- type: textarea
id: what-happened
attributes:
label: What happened?
description: Also tell us, what did you expect to happen?
placeholder: Tell us what you see!
validations:
required: true
- type: textarea
id: version
attributes:
label: Version
description: What version of our software are you running? (use `--version` to get a version string)
placeholder: "cosmocc (GCC) 12.3.0"
validations:
required: true
- type: dropdown
id: operating-system
attributes:
label: What operating system are you seeing the problem on?
multiple: true
options:
- Linux
- Mac
- Windows
- FreeBSD
- OpenBSD
- NetBSD
- BIOS
validations:
required: false
- type: textarea
id: logs
attributes:
label: Relevant log output
description: Please copy and paste any relevant log output. This will be automatically formatted into code, so no need for backticks.
render: shell
+59
View File
@@ -0,0 +1,59 @@
name: Medium Severity Bug
description: Used to report medium severity bugs in cosmopolitan (e.g. Malfunctioning Features but generally still useable)
title: "Bug: "
labels: ["bug", "medium severity"]
body:
- type: markdown
attributes:
value: |
Thanks for taking the time to fill out this bug report!
Please include information about your system, the steps to reproduce the bug,
and the version of cosmopolitan that you are using.
If possible, please provide a minimal code example that reproduces the bug.
You may also consider using function call tracing `--ftrace` or the lighter system call tracing `--strace`
for additional technical logging that may allow us to narrow down where the fault occurred.
- type: input
id: contact
attributes:
label: Contact Details
description: How can we get in touch with you if we need more info?
placeholder: ex. [email protected]
validations:
required: false
- type: textarea
id: what-happened
attributes:
label: What happened?
description: Also tell us, what did you expect to happen?
placeholder: Tell us what you see!
validations:
required: true
- type: textarea
id: version
attributes:
label: Version
description: What version of our software are you running? (use `--version` to get a version string)
placeholder: "cosmocc (GCC) 12.3.0"
validations:
required: true
- type: dropdown
id: operating-system
attributes:
label: What operating system are you seeing the problem on?
multiple: true
options:
- Linux
- Mac
- Windows
- FreeBSD
- OpenBSD
- NetBSD
- BIOS
validations:
required: false
- type: textarea
id: logs
attributes:
label: Relevant log output
description: Please copy and paste any relevant log output. This will be automatically formatted into code, so no need for backticks.
render: shell
+59
View File
@@ -0,0 +1,59 @@
name: High Severity Bug
description: Used to report high severity bugs in cosmopolitan (e.g. Malfunctioning features hindering important common workflow)
title: "Bug: "
labels: ["bug", "high severity"]
body:
- type: markdown
attributes:
value: |
Thanks for taking the time to fill out this bug report!
Please include information about your system, the steps to reproduce the bug,
and the version of cosmopolitan that you are using.
If possible, please provide a minimal code example that reproduces the bug.
You may also consider using function call tracing `--ftrace` or the lighter system call tracing `--strace`
for additional technical logging that may allow us to narrow down where the fault occurred.
- type: input
id: contact
attributes:
label: Contact Details
description: How can we get in touch with you if we need more info?
placeholder: ex. [email protected]
validations:
required: false
- type: textarea
id: what-happened
attributes:
label: What happened?
description: Also tell us, what did you expect to happen?
placeholder: Tell us what you see!
validations:
required: true
- type: textarea
id: version
attributes:
label: Version
description: What version of our software are you running? (use `--version` to get a version string)
placeholder: "cosmocc (GCC) 12.3.0"
validations:
required: true
- type: dropdown
id: operating-system
attributes:
label: What operating system are you seeing the problem on?
multiple: true
options:
- Linux
- Mac
- Windows
- FreeBSD
- OpenBSD
- NetBSD
- BIOS
validations:
required: false
- type: textarea
id: logs
attributes:
label: Relevant log output
description: Please copy and paste any relevant log output. This will be automatically formatted into code, so no need for backticks.
render: shell
@@ -0,0 +1,59 @@
name: Critical Severity Bug
description: Used to report critical severity bugs in cosmopolitan (e.g. Crashing, Corrupted, Dataloss)
title: "Bug: "
labels: ["bug", "critical severity"]
body:
- type: markdown
attributes:
value: |
Thanks for taking the time to fill out this bug report!
Please include information about your system, the steps to reproduce the bug,
and the version of cosmopolitan that you are using.
If possible, please provide a minimal code example that reproduces the bug.
You may also consider using function call tracing `--ftrace` or the lighter system call tracing `--strace`
for additional technical logging that may allow us to narrow down where the fault occurred.
- type: input
id: contact
attributes:
label: Contact Details
description: How can we get in touch with you if we need more info?
placeholder: ex. [email protected]
validations:
required: false
- type: textarea
id: what-happened
attributes:
label: What happened?
description: Also tell us, what did you expect to happen?
placeholder: Tell us what you see!
validations:
required: true
- type: textarea
id: version
attributes:
label: Version
description: What version of our software are you running? (use `--version` to get a version string)
placeholder: "cosmocc (GCC) 12.3.0"
validations:
required: true
- type: dropdown
id: operating-system
attributes:
label: What operating system are you seeing the problem on?
multiple: true
options:
- Linux
- Mac
- Windows
- FreeBSD
- OpenBSD
- NetBSD
- BIOS
validations:
required: false
- type: textarea
id: logs
attributes:
label: Relevant log output
description: Please copy and paste any relevant log output. This will be automatically formatted into code, so no need for backticks.
render: shell
+51
View File
@@ -0,0 +1,51 @@
name: Enhancement template
description: Used to request enhancements for cosmopolitan
title: "Feature Request: "
labels: ["enhancement"]
body:
- type: markdown
attributes:
value: |
[Please post your idea first in Discussion if there is not yet a consensus for this enhancement request. This will help to keep this issue tracker focused on enhancements that the community has agreed needs to be implemented.](https://github.com/jart/cosmopolitan/discussions/categories/ideas)
- type: checkboxes
id: prerequisites
attributes:
label: Prerequisites
description: Please confirm the following before submitting your enhancement request.
options:
- label: I am running the latest code. Mention the version if possible as well.
required: true
- label: I carefully followed the [README.md](https://github.com/jart/cosmopolitan/blob/master/README.md).
required: true
- label: I searched using keywords relevant to my issue to make sure that I am creating a new issue that is not already open (or closed).
required: true
- label: I reviewed the [Discussions](https://github.com/jart/cosmopolitan/discussions), and have a new and useful enhancement to share.
required: true
- type: textarea
id: feature-description
attributes:
label: Feature Description
description: Please provide a detailed written description of what you were trying to do, and what you expected `cosmopolitan` to do as an enhancement.
placeholder: Detailed description of the enhancement
validations:
required: true
- type: textarea
id: motivation
attributes:
label: Motivation
description: Please provide a detailed written description of reasons why this feature is necessary and how it is useful to `cosmopolitan` users.
placeholder: Explanation of why this feature is needed and its benefits
validations:
required: true
- type: textarea
id: possible-implementation
attributes:
label: Possible Implementation
description: If you have an idea as to how it can be implemented, please write a detailed description. Feel free to give links to external sources or share visuals that might be helpful to understand the details better.
placeholder: Detailed description of potential implementation
validations:
required: false
+52
View File
@@ -0,0 +1,52 @@
name: Research
description: Track new technical research area
title: "Research: "
labels: ["research"]
body:
- type: markdown
attributes:
value: |
Don't forget to check for any [duplicate research issue tickets](https://github.com/jart/cosmopolitan/issues?q=is%3Aopen+is%3Aissue+label%3A%22research+%F0%9F%94%AC%22)
- type: checkboxes
id: research-stage
attributes:
label: Research Stage
description: Track general state of this research ticket
options:
- label: Background Research (Let's try to avoid reinventing the wheel)
- label: Hypothesis Formed (How do you think this will work and it's effect?)
- label: Strategy / Implementation Forming
- label: Analysis of results
- label: Debrief / Documentation (So people in the future can learn from us)
- type: textarea
id: background
attributes:
label: Previous existing literature and research
description: Whats the current state of the art and whats the motivation for this research?
- type: textarea
id: hypothesis
attributes:
label: Hypothesis
description: How do you think this will work and it's effect?
- type: textarea
id: implementation
attributes:
label: Implementation
description: Got an approach? e.g. a PR ready to go?
- type: textarea
id: analysis
attributes:
label: Analysis
description: How does the proposed implementation behave?
- type: textarea
id: logs
attributes:
label: Relevant log output
description: Please copy and paste any relevant log output. This will be automatically formatted into code, so no need for backticks.
render: shell
+28
View File
@@ -0,0 +1,28 @@
name: Refactor (Maintainers)
description: Used to track refactoring opportunities
title: "Refactor: "
labels: ["refactor"]
body:
- type: markdown
attributes:
value: |
Don't forget to [check for existing refactor issue tickets](https://github.com/jart/cosmopolitan/issues?q=is%3Aopen+is%3Aissue+label%3Arefactoring) in case it's already covered.
Also you may want to check [Pull request refactor label as well](https://github.com/jart/cosmopolitan/pulls?q=is%3Aopen+is%3Apr+label%3Arefactoring) for duplicates too.
- type: textarea
id: background-description
attributes:
label: Background Description
description: Please provide a detailed written description of the pain points you are trying to solve.
placeholder: Detailed description behind your motivation to request refactor
validations:
required: true
- type: textarea
id: possible-approaches
attributes:
label: Possible Refactor Approaches
description: If you have some idea of possible approaches to solve this problem. You may want to make it a todo list.
placeholder: Your idea of possible refactoring opportunity/approaches
validations:
required: false
+15
View File
@@ -0,0 +1,15 @@
blank_issues_enabled: true
contact_links:
- name: FAQ
url: https://github.com/jart/cosmopolitan/wiki/FAQ
about: Is your question a common one? You may want to check here first.
- name: Got an idea?
url: https://github.com/jart/cosmopolitan/discussions/categories/ideas
about: Pop it there. It may then become an enhancement ticket.
- name: Got a question?
url: https://github.com/jart/cosmopolitan/discussions/categories/q-a
about: Ask a question there!
- name: Want to contribute?
url: https://github.com/jart/cosmopolitan/wiki/contribute
about: Head to the contribution guide page of the wiki for areas you can help with
+49
View File
@@ -0,0 +1,49 @@
# https://github.com/actions/labeler
documentation:
- changed-files:
- any-glob-to-any-file:
- README.md
- LICENSE
- CONTRIBUTING.md
- libc/README.md
- tool/cosmocc/README.md
- third_party/getopt/README.txt
build:
- changed-files:
- any-glob-to-any-file:
- build/**
- Makefile
- '*/*.mk'
examples:
- changed-files:
- any-glob-to-any-file: examples/**
devops:
- changed-files:
- any-glob-to-any-file:
- .github/**
- .clang-format
dsp:
- changed-files:
- any-glob-to-any-file:
- dsp/**
ape:
- changed-files:
- any-glob-to-any-file:
- ape/**
libc:
- changed-files:
- any-glob-to-any-file:
- libc/**
net:
- changed-files:
- any-glob-to-any-file:
- net/**
third_party:
- changed-files:
- any-glob-to-any-file:
- third_party/**
tool:
- changed-files:
- any-glob-to-any-file:
- tool/**
+17
View File
@@ -0,0 +1,17 @@
name: "Pull Request Labeler"
on:
- pull_request_target
jobs:
labeler:
permissions:
contents: read
pull-requests: write
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
repository: "jart/cosmopolitan"
- uses: actions/labeler@v5
with:
configuration-path: '.github/labeler.yml'
+24
View File
@@ -0,0 +1,24 @@
name: Nightly cosmocc
on:
schedule:
# https://crontab.guru/#37_4_*_*_*
- cron: "37 4 * * *"
workflow_dispatch:
concurrency:
group: ${{ github.workflow }}
cancel-in-progress: true
jobs:
build-cosmocc:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- run: |
sudo cp build/bootstrap/ape.elf /usr/bin/ape
sudo sh -c "echo ':APE:M::MZqFpD::/usr/bin/ape:' >/proc/sys/fs/binfmt_misc/register"
- run: tool/cosmocc/package.sh
# https://github.com/actions/upload-artifact/issues/590
- uses: actions/[email protected]
with:
name: cosmocc
path: cosmocc
compression-level: 9
+2 -1
View File
@@ -1,7 +1,8 @@
# -*- conf -*-
/o
/.prompt.jtlp
/cosmocc
/.cosmocc
# TODO: Find some way to have Python write to o/
__pycache__
-109
View File
@@ -1,109 +0,0 @@
{
"configurations": [
{
"name": "Win32",
"includePath": [
"${workspaceFolder}"
],
"cStandard": "c11",
"cppStandard": "c++11",
"forcedInclude": ["${workspaceFolder}/.vscode/vscode.h"],
"defines": [
"libcesque=",
"pureconst=",
"paramsnonnull(x)=",
"alignas(x)",
"alignof(x)",
"artificial=",
"__wur=",
"mayalias=",
"forceinline=",
"forcealign(x)=",
"scanfesque(x)=",
"strftimeesque(x)=",
"wontreturn=",
"textreal=",
"mallocesque=",
"callocesque=",
"vallocesque=",
"reallocesque=",
"strlenesque=",
"memcpyesque=",
"hasatleast=",
"noinline=",
"textexit=",
"returnstwice=",
"textwindows=",
"privileged=",
"dontinstrument=",
"nodebuginfo=",
"interruptfn=",
"optimizespeed=",
"forcealignargpointer=",
"dontasan=",
"dontubsan=",
"donothing=",
"nosideeffect=",
"unreachable=",,
"notpossible=",
"thatispacked=",
"dontthrow=",
"nocallback=",
"relegated=",
"hidden=",
"textstartup=",
"initarray=",
"returnsnonnull=",
"returnspointerwithnoaliases=",
"printfesque(x)=",
"attributeallocsize(x)=",
"returnsaligned(x)=",
"attributeallocalign(x)=",
"nullterminated(x)="
]
},
{
"name": "Linux",
"includePath": [
"${workspaceFolder}"
],
"cStandard": "gnu17",
"compilerPath": "${workspaceFolder}/o/third_party/gcc/bin/x86_64-linux-musl-gcc",
"compilerArgs": [
"-Wall",
"-Werror",
"-fdebug-prefix-map=${workspaceFolder}=",
"-frecord-gcc-switches",
"-Wa,-W",
"-Wa,-I.",
"-Wa,--noexecstack",
"-Og",
"-g",
"-gdescribe-dies",
"-msse3",
"-mno-red-zone",
"-fno-math-errno",
"-fno-trapping-math",
"-fno-fp-int-builtin-inexact",
"-fno-ident",
"-fno-common",
"-fno-gnu-unique",
"-fstrict-aliasing",
"-fstrict-overflow",
"-fno-semantic-interposition",
"-mno-omit-leaf-frame-pointer",
"-fno-jump-tables",
"-nostdinc",
"-iquote."
],
"forcedInclude": [
"libc/integral/normalize.inc"
],
"defines": [
"COSMO",
"MODE="
]
}
],
"version": 4
}
+36
View File
@@ -0,0 +1,36 @@
{
"C_Cpp.default.compilerPath": ".cosmocc/3.8.0/bin/aarch64-linux-cosmo-c++",
"C_Cpp.default.compilerArgs": [
"-nostdinc",
"-nostdlib",
"-iquote.",
"-isystemlibc/isystem",
"-isystemthird_party/libcxx",
"-includelibc/integral/normalize.inc",
"-D_COSMO_SOURCE",
"-D__aarch64__"
],
"[c]": {
"editor.tabSize": 2,
"editor.insertSpaces": true
},
"[cpp]": {
"editor.tabSize": 2,
"editor.insertSpaces": true
},
"[makefile]": {
"editor.tabSize": 8,
"editor.insertSpaces": false
},
"[make]": {
"editor.tabSize": 8,
"editor.insertSpaces": false
},
"[assembly]": {
"editor.tabSize": 8,
"editor.insertSpaces": true
},
"files.associations": {
"log.h": "c"
}
}
-2
View File
@@ -1,2 +0,0 @@
#define __VSCODE_INTELLISENSE__ 1
#include "libc/integral/normalize.inc"
+69 -45
View File
@@ -2,23 +2,77 @@
We'd love to accept your patches! Please read this guide first.
## Identity Disclosure
This project does not accept anonymous contributions. Justine Tunney
won't merge pull requests from strangers. In order to change the Cosmo
codebase, and have your changes be upstreamed, she has to know who you
are. You're encouraged to disclose your full name and email address to
the public too, by including them in your git commit messages; however
that's not a requirement; as we're happy to respect the wishes of
contributors who prefer to remain anonymous to the public.
## Copyright Assignment
Please send an email to Justine Tunney <jtunney@gmail.com> stating that
you intend to assign her the copyright to the changes you contribute to
Cosmopolitan. Please use the same email address you use for git commits
which should only contain original source code from you or other people
who are also assigning copyright. Please note that, if you're employed,
you may need to get your employer's approval beforehand. If you can not
assign copyright due to local laws, then you may alternatively consider
disclaiming it using the language in [Unlicense](https://unlicense.org)
or [CC-0](http://creativecommons.org/share-your-work/public-domain/cc0)
The first time you send a pull request, you need to send an email to
Justine Tunney <jtunney@gmail.com> stating that you intend to assign her
the copyright to the changes you contribute to Cosmopolitan. It only
needs to happen once. This only applies to the code you *choose* to
contribute. The email should be sent from an email address associated
with your identity. Your email should link to your pull request.
This is important because we can't produce 12kb single-file executables
that comply with license requirements if we have to embed lots of them.
Although that's less of an issue depending on the purpose of the files.
For example, ownership is much less of a concern in the unit test files
so you're encouraged to put your copyright on those, provided it's ISC.
To make things easy, here's an example of a good email you can use:
> **From**: YOUR NAME ([email protected])
> **To**: Justine Tunney ([email protected])
> **Subject**: Cosmopolitan Copyright Assignment for YOUR NAME
>
> Hi Justine,
>
> I made my first contribution to Cosmopolitan in
> https://github.com/jart/cosmopolitan/pull/XXXX could you please take a
> look? I intend to assign you the copyright to the changes I contribute
> to Cosmopolitan.
>
> Thanks!
Please note that in order to give Justine the copyright, it has to be
yours to give in the first place. If you're employed, then you should
get your employer's approval to do this beforehand. Even with big
companies like Google, this process is quick and painless. Usually we
see employers granting authorization in less than one day.
If you live in a country that doesn't recognize one's ability to assign
copyright, then you may alternatively consider disclaiming it using the
language in [Unlicense](https://unlicense.org) or
[CC-0](http://creativecommons.org/share-your-work/public-domain/cc0).
If you're checking-in third party code, then you need to have headers at
the top of each source file (but never header files) documenting its
owners and the code should go in the `third_party/` folder. Every third
party project should have a `README.cosmo` file that documents its
provenance as well as any local changes you've made.
## Copyright Policy Exceptions
### Tests
You're encoraged to claim ownership of your test code. If you add a new
file under the `test/` directory, then you should put your name in the
ISC license header at the top of the file. If you add new test cases to
an existing unit test file, then you're encouraged to append a line with
your name to the existing copyright header of that file.
### Exceptional Features
Let's say you discovered a faster better way to implement `log10()` and
you want to give it to Cosmopolitan. In cases like this, it really isn't
appropriate for Justine to own your code. What you could do instead, is
write your own new and improved `log10.c` from scratch, put your name on
the top with the ISC license, and then add a `__notice()` directive so
that your name will be embedded inside every executable that links the
`log10()` function. This will help you get your name out there. Please
note you need get approval from Justine each time you want to do this.
## Style Guide
@@ -29,34 +83,4 @@ clang-format -i -style=file tool/net/redbean.c
```
If you use Emacs this can be automated on save for Cosmopolitan using
[tool/emacs/cosmo-format.el]([tool/emacs/cosmo-format.el]).
### Source Files
- Must use include paths relative to the root of the repository
- Must have comment at top of file documenting copyright and license
- Must have notice embedding if not owned by Justine (exception: tests)
- May use language extensions that are supported by both GCC and Clang
- Should use Google indentation (otherwise use `/* clang-format off */`)
- Should use asm() instead of compiler APIs (exception: ctz, clz, memcpy)
### Header Files
- Must not have copyright or license comments
- Must have once guards (otherwise change `.h` to `.inc`)
- Must be ANSI C89 compatible to be included in the amalgamation header
- Must include its dependencies (exception: libc/integral/normalize.inc)
- Must not define objects (i.e. `cc -c -xc foo.h` will produce empty `.o`)
- Should not use typedefs
- Should not use forward declarations
- Should not include documentation comments
- Should not include parameter names in prototypes
- Should not pose problems if included by C++ or Assembly sources
- Should not declare non-ANSI code, at all, when the user requests ANSI
### Build Config
- Must not write files outside `o/`
- Must not communicate with Internet
- Must not depend on system libraries
- Must not depend on system commands (exception: sh, make, gzip, zip)
[tool/emacs/cosmo-format.el](tool/emacs/cosmo-format.el).
+305 -204
View File
@@ -1,5 +1,5 @@
#-*-mode:makefile-gmake;indent-tabs-mode:t;tab-width:8;coding:utf-8-*-┐
#───vi: set et ft=make ts=8 tw=8 fenc=utf-8 :vi───────────────────────┘
#── vi: set et ft=make ts=8 sw=8 fenc=utf-8 :vi ──────────────────────┘
#
# SYNOPSIS
#
@@ -21,23 +21,23 @@
# make -j8 -O MODE=tiny
#
# # build individual target
# make -j8 -O o//examples/hello.com
# o//examples/hello.com
# make -j8 -O o//examples/hello
# o//examples/hello
#
# # view source
# less examples/hello.c
#
# # view binary
# o//tool/viz/bing.com o//examples/hello.com |
# o//tool/viz/fold.com
# o//tool/viz/bing o//examples/hello |
# o//tool/viz/fold
#
# # view transitive closure of legalese
# o//tool/viz/bing.com -n o//examples/hello.com |
# o//tool/viz/fold.com
# o//tool/viz/bing -n o//examples/hello |
# o//tool/viz/fold
#
# # basic debugging
# make -j8 -O MODE=dbg o/dbg/examples/crashreport.com
# o/examples/crashreport.com
# make -j8 -O MODE=dbg o/dbg/examples/crashreport
# o/examples/crashreport
# less examples/crashreport.c
#
# # extremely tiny binaries
@@ -51,7 +51,7 @@
#
# TROUBLESHOOTING
#
# make -j8 -O V=1 o//examples/hello.com
# make -j8 -O V=1 o//examples/hello
# make o//examples/life.elf -pn |& less
# etc.
#
@@ -59,7 +59,7 @@
#
# build/config.mk
SHELL = build/bootstrap/cocmd.com
SHELL = build/bootstrap/cocmd
MAKEFLAGS += --no-builtin-rules
.SUFFIXES:
@@ -73,22 +73,50 @@ MODE := $(m)
endif
endif
UNAME_M = $(shell uname -m)
UNAME_S = $(shell uname -s)
COMMA := ,
PWD := $(shell pwd)
# detect wsl2 running cosmopolitan binaries on the host by checking whether:
# - user ran build/bootstrap/make, in which case make's working directory is in wsl
# - user ran make, in which case cocmd's working directory is in wsl
ifneq ($(findstring //wsl.localhost/,$(CURDIR) $(PWD)),)
$(warning wsl2 interop is enabled)
$(error you need to run sudo sh -c 'echo -1 > /proc/sys/fs/binfmt_misc/WSLInterop')
endif
UNAME_M := $(shell uname -m)
UNAME_S := $(shell uname -s)
# apple still distributes a 17 year old version of gnu make
ifeq ($(MAKE_VERSION), 3.81)
$(error please use build/bootstrap/make.com)
$(error please use build/bootstrap/make)
endif
# provide instructions to non-linux users on unbundling gcc
ifeq ($(TOOLCHAIN),) # if TOOLCHAIN isn't defined
ifeq ("$(wildcard o/third_party/gcc/bin/x86_64-linux-cosmo-*)","") # if our gcc isn't unbundled
ifneq ($(UNAME_M)-$(UNAME_S), x86_64-Linux) # if this is not amd64 linux
$(error you need to download https://justine.lol/cosmocc-0.0.12.zip and unzip it inside the cosmo directory)
endif
endif
endif
LC_ALL = C
SOURCE_DATE_EPOCH = 0
ARFLAGS = rcsD
ZFLAGS ?=
XARGS ?= xargs -P4 -rs8000
DOT ?= dot
CLANG = clang
TMPDIR = o/tmp
AR = $(BOOTSTRAP)/ar.ape
CP = $(BOOTSTRAP)/cp.ape
RM = $(BOOTSTRAP)/rm.ape -f
GZIP = $(BOOTSTRAP)/gzip.ape
ECHO = $(BOOTSTRAP)/echo.ape
CHMOD = $(BOOTSTRAP)/chmod.ape
TOUCH = $(BOOTSTRAP)/touch.ape
PKG = $(BOOTSTRAP)/package.ape
MKDEPS = $(BOOTSTRAP)/mkdeps
ZIPOBJ = $(BOOTSTRAP)/zipobj
ZIPCOPY = $(BOOTSTRAP)/zipcopy
PECHECK = $(BOOTSTRAP)/pecheck
FIXUPOBJ = $(BOOTSTRAP)/fixupobj
OBJBINCOPY = $(BOOTSTRAP)/objbincopy
MKDIR = $(BOOTSTRAP)/mkdir.ape -p
COMPILE = $(BOOTSTRAP)/compile.ape -V9 -M2048m -P8192 $(QUOTA)
# the default build modes is empty string
# on x86_64 hosts, MODE= is the same as MODE=x86_64
@@ -102,11 +130,52 @@ MODE := aarch64
endif
endif
ifneq ($(findstring aarch64,$(MODE)),)
ARCH = aarch64
HOSTS ?= pi pi5 studio freebsdarm
else
ARCH = x86_64
HOSTS ?= freebsd rhel7 xnu openbsd netbsd win10
endif
ZIPOBJ_FLAGS += -a$(ARCH)
export ADDR2LINE
export LC_ALL
export MKDIR
export MODE
export SOURCE_DATE_EPOCH
export TMPDIR
COSMOCC = .cosmocc/3.8.0
BOOTSTRAP = $(COSMOCC)/bin
TOOLCHAIN = $(COSMOCC)/bin/$(ARCH)-linux-cosmo-
DOWNLOAD := $(shell build/download-cosmocc.sh $(COSMOCC) 3.8.0 813c6b2f95062d2e0a845307a79505424cb98cb038e8013334f8a22e3b92a474)
IGNORE := $(shell $(MKDIR) $(TMPDIR))
AS = $(TOOLCHAIN)as
CC = $(TOOLCHAIN)gcc
CXX = $(TOOLCHAIN)g++
CXXFILT = $(TOOLCHAIN)c++filt
LD = $(TOOLCHAIN)ld.bfd
NM = $(TOOLCHAIN)nm
GCC = $(TOOLCHAIN)gcc
STRIP = $(TOOLCHAIN)strip
OBJCOPY = $(TOOLCHAIN)objcopy
OBJDUMP = $(TOOLCHAIN)objdump
ifneq ($(wildcard $(PWD)/$(TOOLCHAIN)addr2line),)
ADDR2LINE = $(PWD)/$(TOOLCHAIN)addr2line
else
ADDR2LINE = $(TOOLCHAIN)addr2line
endif
# primary build rules
all: o
o: o/$(MODE)
o/$(MODE): \
o/$(MODE)/ape \
o/$(MODE)/ctl \
o/$(MODE)/dsp \
o/$(MODE)/net \
o/$(MODE)/libc \
@@ -115,7 +184,12 @@ o/$(MODE): \
o/$(MODE)/examples \
o/$(MODE)/third_party
# check if we're using o//third_party/make/make.com
# TODO(jart): Make Emacs `C-c C-c` shortcut not need this.
.PHONY: o/$(MODE)/ o/$(MODE)/.
o/$(MODE)/: o/$(MODE)
o/$(MODE)/.: o/$(MODE)
# check if we're using o//third_party/make/make
# we added sandboxing to guarantee cosmo's makefile is hermetic
# it also shaves away 200ms of startup latency with native $(uniq)
ifneq ($(LANDLOCKMAKE_VERSION),)
@@ -125,7 +199,7 @@ $(warning please run ape/apeinstall.sh if you intend to use landlock make)
$(shell sleep .5)
endif
endif
ifeq ($(USE_SYSTEM_TOOLCHAIN),)
ifneq ($(TOOLCHAIN),)
.STRICT = 1
endif
endif
@@ -134,10 +208,9 @@ endif
.UNVEIL += \
libc/integral \
libc/stdbool.h \
libc/disclaimer.inc \
rwc:/dev/shm \
rx:.cosmocc \
rx:build/bootstrap \
rx:o/third_party/gcc \
r:build/portcosmo.h \
/proc/stat \
rw:/dev/null \
@@ -158,149 +231,159 @@ include build/definitions.mk # ├──META
include build/config.mk # │ You can build
include build/rules.mk # │ You can topologically order
include build/online.mk #─┘
include libc/nexgen32e/nexgen32e.mk #─┐
include libc/sysv/sysv.mk # ├──SYSTEM SUPPORT
include libc/nt/nt.mk # │ You can do math
include libc/intrin/intrin.mk # │ You can use the stack
include third_party/compiler_rt/compiler_rt.mk # │ You can manipulate arrays
include libc/tinymath/tinymath.mk # │ You can issue raw system calls
include libc/str/str.mk # │
include third_party/xed/xed.mk # │
include third_party/puff/puff.mk # │
include libc/elf/elf.mk # │
include ape/ape.mk # │
include libc/fmt/fmt.mk # │
include libc/vga/vga.mk #─┘
include libc/calls/calls.mk #─┐
include third_party/nsync/nsync.mk # │
include libc/runtime/runtime.mk # ├──SYSTEMS RUNTIME
include third_party/double-conversion/dc.mk # │ You can issue system calls
include libc/crt/crt.mk # │
include third_party/dlmalloc/dlmalloc.mk #─┘
include libc/mem/mem.mk #─┐
include third_party/gdtoa/gdtoa.mk # ├──DYNAMIC RUNTIME
include third_party/nsync/mem/mem.mk # │ You can now use stdio
include libc/proc/proc.mk # │ You can now use threads
include libc/thread/thread.mk # │ You can now use processes
include tool/hello/hello.mk # │ You can finally call malloc()
include third_party/zlib/zlib.mk # │
include libc/stdio/stdio.mk # │
include libc/time/time.mk # │
include net/net.mk # │
include third_party/vqsort/vqsort.mk # │
include libc/log/log.mk # │
include third_party/getopt/getopt.mk # │
include third_party/bzip2/bzip2.mk # │
include dsp/core/core.mk # │
include third_party/musl/musl.mk # │
include libc/x/x.mk # │
include third_party/stb/stb.mk # │
include dsp/scale/scale.mk # │
include dsp/mpeg/mpeg.mk # │
include dsp/dsp.mk # │
include third_party/zlib/gz/gz.mk # │
include third_party/intel/intel.mk # │
include third_party/aarch64/aarch64.mk # │
include libc/libc.mk #─┘
include libc/sock/sock.mk #─┐
include dsp/tty/tty.mk # ├──ONLINE RUNTIME
include libc/dns/dns.mk # │ You can communicate with the network
include net/http/http.mk # │
include third_party/mbedtls/mbedtls.mk # │
include third_party/libcxx/libcxx.mk # │
include third_party/ggml/ggml.mk # │
include third_party/radpajama/radpajama.mk # │
include net/https/https.mk # │
include third_party/regex/regex.mk #─┘
include third_party/tidy/tidy.mk
include third_party/third_party.mk
include third_party/nsync/testing/testing.mk
include libc/testlib/testlib.mk
include tool/viz/lib/vizlib.mk
include tool/args/args.mk
include test/tool/args/test.mk
include third_party/linenoise/linenoise.mk
include third_party/maxmind/maxmind.mk
include net/finger/finger.mk
include third_party/double-conversion/test/test.mk
include third_party/lua/lua.mk
include third_party/tree/tree.mk
include third_party/zstd/zstd.mk
include third_party/tr/tr.mk
include third_party/sed/sed.mk
include third_party/awk/awk.mk
include third_party/hiredis/hiredis.mk
include third_party/make/make.mk
include third_party/ctags/ctags.mk
include third_party/finger/finger.mk
include third_party/argon2/argon2.mk
include third_party/smallz4/smallz4.mk
include third_party/sqlite3/sqlite3.mk
include third_party/mbedtls/test/test.mk
include third_party/quickjs/quickjs.mk
include third_party/lz4cli/lz4cli.mk
include third_party/zip/zip.mk
include third_party/xxhash/xxhash.mk
include third_party/unzip/unzip.mk
include tool/build/lib/buildlib.mk
include third_party/chibicc/chibicc.mk
include third_party/chibicc/test/test.mk
include third_party/python/python.mk
include tool/build/build.mk
include tool/curl/curl.mk
include third_party/qemu/qemu.mk
include examples/examples.mk
include examples/pyapp/pyapp.mk
include examples/pylife/pylife.mk
include tool/decode/lib/decodelib.mk
include tool/decode/decode.mk
include tool/lambda/lib/lib.mk
include tool/lambda/lambda.mk
include tool/plinko/lib/lib.mk
include tool/plinko/plinko.mk
include test/tool/plinko/test.mk
include tool/net/net.mk
include tool/viz/viz.mk
include tool/tool.mk
include net/turfwar/turfwar.mk
include test/libc/tinymath/test.mk
include test/libc/intrin/test.mk
include test/libc/mem/test.mk
include test/libc/nexgen32e/test.mk
include test/libc/runtime/test.mk
include test/libc/thread/test.mk
include test/libc/sock/test.mk
include test/libc/str/test.mk
include test/libc/log/test.mk
include test/libc/str/test.mk
include test/libc/calls/test.mk
include test/libc/x/test.mk
include test/libc/xed/test.mk
include test/libc/fmt/test.mk
include test/libc/dns/test.mk
include test/libc/time/test.mk
include test/libc/proc/test.mk
include test/libc/stdio/test.mk
include test/libc/release/test.mk
include test/libc/test.mk
include test/net/http/test.mk
include test/net/https/test.mk
include test/net/finger/test.mk
include test/net/test.mk
include test/tool/build/lib/test.mk
include test/tool/build/test.mk
include test/tool/viz/lib/test.mk
include test/tool/viz/test.mk
include test/tool/net/test.mk
include test/tool/test.mk
include test/dsp/core/test.mk
include test/dsp/scale/test.mk
include test/dsp/tty/test.mk
include test/dsp/test.mk
include examples/package/lib/build.mk
include examples/package/build.mk
include libc/nexgen32e/BUILD.mk #─┐
include libc/sysv/BUILD.mk # ├──SYSTEM SUPPORT
include libc/nt/BUILD.mk # │ You can do math
include libc/intrin/BUILD.mk # │ You can use the stack
include third_party/compiler_rt/BUILD.mk # │ You can manipulate arrays
include libc/tinymath/BUILD.mk # │ You can issue raw system calls
include libc/str/BUILD.mk # │
include third_party/xed/BUILD.mk # │
include third_party/puff/BUILD.mk # │
include libc/elf/BUILD.mk # │
include ape/BUILD.mk # │
include libc/fmt/BUILD.mk # │
include libc/vga/BUILD.mk # │
include libc/irq/BUILD.mk #─┘
include libc/calls/BUILD.mk #─┐
include libc/irq/BUILD.mk # ├──SYSTEMS RUNTIME
include third_party/nsync/BUILD.mk # │ You can issue system calls
include libc/runtime/BUILD.mk # │
include libc/crt/BUILD.mk # │
include third_party/dlmalloc/BUILD.mk #─┘
include libc/mem/BUILD.mk #─┐
include third_party/gdtoa/BUILD.mk # ├──DYNAMIC RUNTIME
include third_party/nsync/mem/BUILD.mk # │ You can now use stdio
include libc/proc/BUILD.mk # │ You can now use threads
include libc/dlopen/BUILD.mk # │ You can now use processes
include libc/thread/BUILD.mk # │ You can finally call malloc()
include third_party/zlib/BUILD.mk # │
include libc/stdio/BUILD.mk # │
include tool/hello/BUILD.mk # │
include third_party/tz/BUILD.mk # │
include net/BUILD.mk # │
include third_party/vqsort/BUILD.mk # │
include libc/log/BUILD.mk # │
include third_party/getopt/BUILD.mk # │
include third_party/bzip2/BUILD.mk # │
include dsp/core/BUILD.mk # │
include third_party/zlib/gz/BUILD.mk # │
include third_party/intel/BUILD.mk # │
include third_party/aarch64/BUILD.mk # │
include libc/BUILD.mk #─┘
include libc/sock/BUILD.mk #─┐
include net/http/BUILD.mk # ├──ONLINE RUNTIME
include third_party/musl/BUILD.mk # │ You can communicate with the network
include libc/x/BUILD.mk # │
include dsp/scale/BUILD.mk # │
include dsp/mpeg/BUILD.mk # │
include dsp/tty/BUILD.mk # │
include dsp/BUILD.mk # │
include third_party/stb/BUILD.mk # │
include third_party/mbedtls/BUILD.mk # │
include third_party/ncurses/BUILD.mk # │
include third_party/readline/BUILD.mk # │
include third_party/libunwind/BUILD.mk # |
include third_party/libcxxabi/BUILD.mk # |
include third_party/double-conversion/BUILD.mk # │
include ctl/BUILD.mk # │
include third_party/libcxx/BUILD.mk # │
include third_party/openmp/BUILD.mk # │
include third_party/pcre/BUILD.mk # │
include third_party/less/BUILD.mk # │
include net/https/BUILD.mk # │
include third_party/regex/BUILD.mk #─┘
include third_party/tidy/BUILD.mk
include third_party/BUILD.mk
include third_party/nsync/testing/BUILD.mk
include libc/testlib/BUILD.mk
include tool/viz/lib/BUILD.mk
include tool/args/BUILD.mk
include test/math/BUILD.mk
include test/posix/BUILD.mk
include test/ctl/BUILD.mk
include test/libcxx/BUILD.mk
include test/tool/args/BUILD.mk
include third_party/linenoise/BUILD.mk
include third_party/maxmind/BUILD.mk
include net/finger/BUILD.mk
include third_party/double-conversion/test/BUILD.mk
include third_party/lua/BUILD.mk
include third_party/tree/BUILD.mk
include third_party/zstd/BUILD.mk
include third_party/tr/BUILD.mk
include third_party/sed/BUILD.mk
include third_party/awk/BUILD.mk
include third_party/hiredis/BUILD.mk
include third_party/make/BUILD.mk
include third_party/ctags/BUILD.mk
include third_party/finger/BUILD.mk
include third_party/argon2/BUILD.mk
include third_party/smallz4/BUILD.mk
include third_party/sqlite3/BUILD.mk
include third_party/mbedtls/test/BUILD.mk
include third_party/lz4cli/BUILD.mk
include third_party/zip/BUILD.mk
include third_party/xxhash/BUILD.mk
include third_party/unzip/BUILD.mk
include tool/build/lib/BUILD.mk
include third_party/chibicc/BUILD.mk
include third_party/chibicc/test/BUILD.mk
include third_party/python/BUILD.mk
include tool/build/BUILD.mk
include tool/curl/BUILD.mk
include third_party/qemu/BUILD.mk
include third_party/libcxxabi/test/BUILD.mk
include examples/BUILD.mk
include examples/pyapp/BUILD.mk
include examples/pylife/BUILD.mk
include tool/decode/lib/BUILD.mk
include tool/decode/BUILD.mk
include tool/lambda/lib/BUILD.mk
include tool/lambda/BUILD.mk
include tool/plinko/lib/BUILD.mk
include tool/plinko/BUILD.mk
include test/tool/plinko/BUILD.mk
include tool/net/BUILD.mk
include tool/viz/BUILD.mk
include tool/BUILD.mk
include net/turfwar/BUILD.mk
include test/libc/tinymath/BUILD.mk
include test/libc/intrin/BUILD.mk
include test/libc/mem/BUILD.mk
include test/libc/nexgen32e/BUILD.mk
include test/libc/runtime/BUILD.mk
include test/libc/thread/BUILD.mk
include test/libc/sock/BUILD.mk
include test/libc/str/BUILD.mk
include test/libc/log/BUILD.mk
include test/libc/str/BUILD.mk
include test/libc/calls/BUILD.mk
include test/libc/x/BUILD.mk
include test/libc/xed/BUILD.mk
include test/libc/fmt/BUILD.mk
include test/libc/time/BUILD.mk
include test/libc/proc/BUILD.mk
include test/libc/stdio/BUILD.mk
include test/libc/BUILD.mk
include test/net/http/BUILD.mk
include test/net/https/BUILD.mk
include test/net/finger/BUILD.mk
include test/net/BUILD.mk
include test/tool/build/lib/BUILD.mk
include test/tool/build/BUILD.mk
include test/tool/viz/lib/BUILD.mk
include test/tool/viz/BUILD.mk
include test/tool/net/BUILD.mk
include test/tool/BUILD.mk
include test/dsp/core/BUILD.mk
include test/dsp/scale/BUILD.mk
include test/dsp/tty/BUILD.mk
include test/dsp/BUILD.mk
include examples/package/lib/BUILD.mk
include examples/package/BUILD.mk
#-φ-examples/package/new.sh
include test/test.mk
include test/BUILD.mk
OBJS = $(foreach x,$(PKGS),$($(x)_OBJS))
SRCS := $(foreach x,$(PKGS),$($(x)_SRCS))
@@ -326,7 +409,7 @@ o/$(MODE)/hdrs.txt: o/$(MODE)/.x $(MAKEFILES) $(call uniq,$(foreach x,$(HDRS) $(
o/$(MODE)/incs.txt: o/$(MODE)/.x $(MAKEFILES) $(call uniq,$(foreach x,$(INCS) $(INCS),$(dir $(x)))) $(INCS) $(INCS)
$(file >$@,$(INCS))
o/$(MODE)/depend: o/$(MODE)/.x o/$(MODE)/srcs.txt o/$(MODE)/hdrs.txt o/$(MODE)/incs.txt $(SRCS) $(HDRS) $(INCS)
$(COMPILE) -AMKDEPS -L320 $(MKDEPS) -o $@ -r o/$(MODE)/ @o/$(MODE)/srcs.txt @o/$(MODE)/hdrs.txt @o/$(MODE)/incs.txt
$(COMPILE) -AMKDEPS -L320 $(MKDEPS) -o $@ -s -r o/$(MODE)/ @o/$(MODE)/srcs.txt @o/$(MODE)/hdrs.txt @o/$(MODE)/incs.txt
o/$(MODE)/srcs-old.txt: o/$(MODE)/.x $(MAKEFILES) $(call uniq,$(foreach x,$(SRCS),$(dir $(x))))
$(file >$@) $(foreach x,$(SRCS),$(file >>$@,$(x)))
@@ -334,35 +417,39 @@ o/$(MODE)/hdrs-old.txt: o/$(MODE)/.x $(MAKEFILES) $(call uniq,$(foreach x,$(HDRS
$(file >$@) $(foreach x,$(HDRS) $(INCS),$(file >>$@,$(x)))
TAGS: private .UNSANDBOXED = 1
TAGS: o/$(MODE)/srcs-old.txt $(SRCS) #o/$(MODE)/third_party/ctags/ctags.com
TAGS: o/$(MODE)/srcs-old.txt $(SRCS) #o/$(MODE)/third_party/ctags/ctags
@$(RM) $@
@o/$(MODE)/third_party/ctags/ctags.com $(TAGSFLAGS) -L $< -o $@
@o/$(MODE)/third_party/ctags/ctags $(TAGSFLAGS) -L $< -o $@
HTAGS: private .UNSANDBOXED = 1
HTAGS: o/$(MODE)/hdrs-old.txt $(filter-out third_party/libcxx/%,$(HDRS)) #o/$(MODE)/third_party/ctags/ctags.com
HTAGS: o/$(MODE)/hdrs-old.txt $(filter-out third_party/libcxx/%,$(HDRS)) #o/$(MODE)/third_party/ctags/ctags
@$(RM) $@
@build/htags o/$(MODE)/third_party/ctags/ctags.com -L $< -o $@
@build/htags o/$(MODE)/third_party/ctags/ctags -L $< -o $@
loc: private .UNSANDBOXED = 1
loc: o/$(MODE)/tool/build/summy.com
loc: o/$(MODE)/tool/build/summy
find -name \*.h -or -name \*.c -or -name \*.S | \
$(XARGS) wc -l | grep total | awk '{print $$1}' | $<
# PLEASE: MAINTAIN TOPOLOGICAL ORDER
# FROM HIGHEST LEVEL TO LOWEST LEVEL
COSMOPOLITAN_OBJECTS = \
CTL \
THIRD_PARTY_DOUBLECONVERSION \
THIRD_PARTY_OPENMP \
TOOL_ARGS \
NET_HTTP \
LIBC_DNS \
LIBC_SOCK \
LIBC_NT_WS2_32 \
LIBC_NT_IPHLPAPI \
LIBC_X \
THIRD_PARTY_GETOPT \
LIBC_LOG \
LIBC_TIME \
THIRD_PARTY_TZ \
THIRD_PARTY_MUSL \
THIRD_PARTY_ZLIB_GZ \
THIRD_PARTY_LIBCXXABI \
THIRD_PARTY_LIBUNWIND \
LIBC_STDIO \
THIRD_PARTY_GDTOA \
THIRD_PARTY_REGEX \
@@ -371,9 +458,11 @@ COSMOPOLITAN_OBJECTS = \
THIRD_PARTY_NSYNC_MEM \
LIBC_MEM \
THIRD_PARTY_DLMALLOC \
LIBC_DLOPEN \
LIBC_RUNTIME \
THIRD_PARTY_NSYNC \
LIBC_ELF \
LIBC_IRQ \
LIBC_CALLS \
LIBC_SYSV_CALLS \
LIBC_VGA \
@@ -395,16 +484,17 @@ COSMOPOLITAN_OBJECTS = \
LIBC_STR \
LIBC_SYSV \
LIBC_INTRIN \
LIBC_NT_BCRYPTPRIMITIVES \
LIBC_NT_KERNEL32 \
LIBC_NEXGEN32E
COSMOPOLITAN_HEADERS = \
COSMOPOLITAN_H_PKGS = \
APE \
LIBC \
LIBC_CALLS \
LIBC_DNS \
LIBC_ELF \
LIBC_FMT \
LIBC_DLOPEN \
LIBC_INTRIN \
LIBC_LOG \
LIBC_MEM \
@@ -419,7 +509,6 @@ COSMOPOLITAN_HEADERS = \
LIBC_STR \
LIBC_SYSV \
LIBC_THREAD \
LIBC_TIME \
LIBC_TINYMATH \
LIBC_X \
LIBC_VGA \
@@ -429,30 +518,43 @@ COSMOPOLITAN_HEADERS = \
THIRD_PARTY_GDTOA \
THIRD_PARTY_GETOPT \
THIRD_PARTY_MUSL \
THIRD_PARTY_ZLIB \
THIRD_PARTY_ZLIB_GZ \
THIRD_PARTY_REGEX
COSMOCC_PKGS = \
$(COSMOPOLITAN_H_PKGS) \
CTL \
THIRD_PARTY_AARCH64 \
THIRD_PARTY_LIBCXX \
THIRD_PARTY_LIBCXXABI \
THIRD_PARTY_LIBUNWIND \
THIRD_PARTY_OPENMP \
THIRD_PARTY_INTEL
o/$(MODE)/cosmopolitan.a: \
$(foreach x,$(COSMOPOLITAN_OBJECTS),$($(x)_A_OBJS))
o/cosmopolitan.h: \
o/$(MODE)/tool/build/rollup.com \
libc/integral/normalize.inc \
$(foreach x,$(COSMOPOLITAN_HEADERS),$($(x)_HDRS)) \
$(foreach x,$(COSMOPOLITAN_HEADERS),$($(x)_INCS))
$(file >$(TMPDIR)/$(subst /,_,$@),libc/integral/normalize.inc $(foreach x,$(COSMOPOLITAN_HEADERS),$($(x)_HDRS)))
@$(ECHO) '#ifndef __STRICT_ANSI__' >$@
@$(ECHO) '#define _COSMO_SOURCE' >>$@
@$(ECHO) '#endif' >>$@
@$(COMPILE) -AROLLUP -T$@ o/$(MODE)/tool/build/rollup.com @$(TMPDIR)/$(subst /,_,$@) >>$@
COSMOCC_HDRS = \
$(wildcard libc/integral/*) \
$(foreach x,$(COSMOCC_PKGS),$($(x)_HDRS)) \
$(foreach x,$(COSMOCC_PKGS),$($(x)_INCS))
o/cosmocc.h.txt: Makefile libc $(MAKEFILES) $(call uniq,$(foreach x,$(HDRS) $(INCS),$(dir $(x)))) $(HDRS) $(INCS)
$(file >$@, $(call uniq,$(COSMOCC_HDRS)))
COSMOPOLITAN_H_ROOT_HDRS = \
libc/integral/normalize.inc \
$(foreach x,$(COSMOPOLITAN_H_PKGS),$($(x)_HDRS))
o/cosmopolitan.html: private .UNSANDBOXED = 1
o/cosmopolitan.html: \
o/$(MODE)/third_party/chibicc/chibicc.com.dbg \
o/$(MODE)/third_party/chibicc/chibicc.dbg \
$(filter-out %.s,$(foreach x,$(COSMOPOLITAN_OBJECTS),$($(x)_SRCS))) \
$(SRCS) \
$(filter-out %.cpp,$(filter-out %.cc,$(SRCS))) \
$(HDRS)
$(file >$(TMPDIR)/$(subst /,_,$@),$(filter-out %.s,$(foreach x,$(COSMOPOLITAN_OBJECTS),$($(x)_SRCS))))
o/$(MODE)/third_party/chibicc/chibicc.com.dbg -J \
$(file >$(TMPDIR)/$(subst /,_,$@),$(filter-out %.cpp,$(filter-out %.cc,$(filter-out %.s,$(foreach x,$(COSMOPOLITAN_OBJECTS),$($(x)_SRCS))))))
o/$(MODE)/third_party/chibicc/chibicc.dbg -J \
-fno-common -include libc/integral/normalize.inc -o $@ \
-DCOSMO @$(TMPDIR)/$(subst /,_,$@)
@@ -466,7 +568,6 @@ $(SRCS): \
ifeq ($(ARCH), x86_64)
TOOLCHAIN_ARTIFACTS = \
o/cosmopolitan.h \
o/$(MODE)/ape/ape.lds \
o/$(MODE)/libc/crt/crt.o \
o/$(MODE)/ape/ape.elf \
@@ -475,13 +576,13 @@ TOOLCHAIN_ARTIFACTS = \
o/$(MODE)/ape/ape-no-modify-self.o \
o/$(MODE)/cosmopolitan.a \
o/$(MODE)/third_party/libcxx/libcxx.a \
o/$(MODE)/tool/build/march-native.com \
o/$(MODE)/tool/build/ar.com \
o/$(MODE)/tool/build/mktemper.com \
o/$(MODE)/tool/build/fixupobj.com \
o/$(MODE)/tool/build/zipcopy.com \
o/$(MODE)/tool/build/apelink.com \
o/$(MODE)/tool/build/pecheck.com
o/$(MODE)/tool/build/march-native \
o/$(MODE)/tool/build/ar \
o/$(MODE)/tool/build/mktemper \
o/$(MODE)/tool/build/fixupobj \
o/$(MODE)/tool/build/zipcopy \
o/$(MODE)/tool/build/apelink \
o/$(MODE)/tool/build/pecheck
else
TOOLCHAIN_ARTIFACTS = \
o/$(MODE)/ape/ape.elf \
@@ -489,9 +590,9 @@ TOOLCHAIN_ARTIFACTS = \
o/$(MODE)/libc/crt/crt.o \
o/$(MODE)/cosmopolitan.a \
o/$(MODE)/third_party/libcxx/libcxx.a \
o/$(MODE)/tool/build/march-native.com \
o/$(MODE)/tool/build/fixupobj.com \
o/$(MODE)/tool/build/zipcopy.com
o/$(MODE)/tool/build/march-native \
o/$(MODE)/tool/build/fixupobj \
o/$(MODE)/tool/build/zipcopy
endif
.PHONY: toolchain
+112 -454
View File
@@ -12,29 +12,25 @@ possible performance and the tiniest footprint imaginable.
## Background
For an introduction to this project, please read the [αcτµαlly pδrταblε
εxεcµταblε](https://justine.lol/ape.html) blog post and [cosmopolitan
For an introduction to this project, please read the [actually portable
executable](https://justine.lol/ape.html) blog post and [cosmopolitan
libc](https://justine.lol/cosmopolitan/index.html) website. We also have
[API documentation](https://justine.lol/cosmopolitan/documentation.html).
[API
documentation](https://justine.lol/cosmopolitan/documentation.html).
## Getting Started
It's recommended that Cosmopolitan be installed to `/opt/cosmo` and
`/opt/cosmos` on your computer. The first has the monorepo. The second
contains your non-monorepo artifacts.
You can start by obtaining a release of our `cosmocc` compiler from
<https://cosmo.zip/pub/cosmocc/>.
```sh
sudo mkdir -p /opt
sudo chmod 1777 /opt
git clone https://github.com/jart/cosmopolitan /opt/cosmo
export PATH="/opt/cosmo/bin:/opt/cosmos/bin:$PATH"
echo 'PATH="/opt/cosmo/bin:/opt/cosmos/bin:$PATH"' >>~/.profile
ape-install # optionally install a faster systemwide ape loader
cosmocc --update # pull cosmo and rebuild toolchain
mkdir -p cosmocc
cd cosmocc
wget https://cosmo.zip/pub/cosmocc/cosmocc.zip
unzip cosmocc.zip
```
You've now successfully installed your very own cosmos. Now let's build
an example program:
Here's an example program we can write:
```c
// hello.c
@@ -45,26 +41,11 @@ int main() {
}
```
To compile the program, you can run the `cosmocc` command. It's
important to give it an output path that ends with `.com` so the output
format will be Actually Portable Executable. When this happens, a
concomitant debug binary is created automatically too.
It can be compiled as follows:
```sh
cosmocc -o hello.com hello.c
./hello.com
./hello.com.dbg
```
You can use the `cosmocc` toolchain to build conventional open source
projects which use autotools. This strategy normally works:
```sh
export CC=cosmocc
export CXX=cosmoc++
./configure --prefix=/opt/cosmos
make -j
make install
cosmocc -o hello hello.c
./hello
```
The Cosmopolitan Libc runtime links some heavyweight troubleshooting
@@ -72,445 +53,108 @@ features by default, which are very useful for developers and admins.
Here's how you can log system calls:
```sh
./hello.com --strace
./hello --strace
```
Here's how you can get a much more verbose log of function calls:
```sh
./hello.com --ftrace
./hello --ftrace
```
If you don't want rich runtime features like the above included, and you
just want libc, and you want smaller simpler programs. In that case, you
can consider using `MODE=tiny`, which is preconfigured by the repo in
[build/config.mk](build/config.mk). Using this mode is much more
effective at reducing binary footprint than the `-Os` flag alone. You
can change your build mode by doing the following:
You can use the Cosmopolitan's toolchain to build conventional open
source projects which use autotools. This strategy normally works:
```sh
export MODE=tiny
cosmocc --update
```
We can also make our program slightly smaller by using the system call
interface directly, which is fine, since Cosmopolitan polyfills these
interfaces across platforms, including Windows. For example:
```c
// hello2.c
#include <unistd.h>
int main() {
write(1, "hello world\n", 12);
}
```
Once compiled, your APE binary should be ~36kb in size.
```sh
export MODE=tiny
cosmocc -Os -o hello2.com hello2.c
./hello2.com
```
But let's say you only care about your binaries running on Linux and you
don't want to use up all this additional space for platforms like WIN32.
In that case, you can try `MODE=tinylinux` for example which will create
executables more on the order of 8kb (similar to Musl Libc).
```sh
export MODE=tinylinux
cosmocc --update
cosmocc -Os -o hello2.com hello2.c
./hello2.com # <-- actually an ELF executable
```
## ARM
Cosmo supports cross-compiling binaries for machines with ARM
microprocessors. For example:
```sh
make -j8 m=aarch64 o/aarch64/third_party/ggml/llama.com
make -j8 m=aarch64-tiny o/aarch64-tiny/third_party/ggml/llama.com
```
That'll produce ELF executables that run natively on two operating
systems: Linux Arm64 (e.g. Raspberry Pi) and MacOS Arm64 (i.e. Apple
Silicon), thus giving you full performance. The catch is you have to
compile these executables on an x86_64-linux machine. The second catch
is that MacOS needs a little bit of help understanding the ELF format.
To solve that, we provide a tiny APE loader you can use on M1 machines.
```sh
scp ape/ape-m1.c macintosh:
scp o/aarch64/third_party/ggml/llama.com macintosh:
ssh macintosh
xcode-install
cc -o ape ape-m1.c
sudo cp ape /usr/local/bin/ape
```
You can run your ELF AARCH64 executable on Apple Silicon as follows:
```sh
ape ./llama.com
```
If you want to run the `MODE=aarch64` unit tests, you need to have
qemu-aarch64 installed as a binfmt_misc interpreter. It needs to be a
static binary if you want it to work with Landlock Make's security. You
can use the build included in our `third_party/qemu/` folder.
```
doas cp o/third_party/qemu/qemu-aarch64 /usr/bin/qemu-aarch64
doas sh -c "echo ':qemu-aarch64:M::\x7fELF\x02\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\xb7\x00:\xff\xff\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff\xff:/usr/bin/qemu-aarch64:CF' > /proc/sys/fs/binfmt_misc/register"
make -j8 m=aarch64
```
Please note that the qemu-aarch64 binfmt_misc interpreter installation
process is *essential* for being able to use the `aarch64-unknown-cosmo`
toolchain to build fat APE binaries on your x86-64 machine.
## AMD64 + ARM64 fat APE binaries
If you've setup the qemu binfmt_misc interpreter, then you can can use
cosmo's toolchains to build fat ape binaries. It works by compiling your
program twice, so you can have a native build for both architectures in
the same file. The two programs are merged together by apelink.com which
also embeds multiple copies of APE loader and multiple symbols tables.
The easiest way to build fat APE is using `fatcosmocc`. This compiler
works by creating a concomitant `.aarch64/foo.o` for every `foo.o` you
compile. The only exception is the C preprocessor mode, which actually
runs x86-64 GCC except with macros like `__x86_64__` undefined.
This toolchain works great for C projects that are written in a portable
way and don't produce architecture-specific artifacts. One example of a
large project that can be easily built is GNU coreutils.
```sh
cd coreutils
fatcosmocc --update ||exit
./configure CC=fatcosmocc \
AR=fatcosmoar \
INSTALL=$(command -v fatcosmoinstall) \
--prefix=/opt/cosmos \
--disable-nls \
--disable-dependency-tracking \
--disable-silent-rules
make -j8
```
You'll then have a bunch of files like `src/ls` which are fat ape
binaries. If you want to run them on Windows, then you simply need to
rename the file so that it has the `.com` suffix. Better yet, consider
making that a symlink (a.k.a. reparse point). The biggest gotcha with
`fatcosmocc` though is ensuring builds don't strip binaries. For
example, Linux's `install -s` command actually understands Windows'
Portable Executable format well enough to remove the MS-DOS stub, which
is where the APE shell script is stored. You need to ensure that
`fatcosmoinstall` is used instead. Especially if your project needs to
install the libraries built by `fatacosmoar` into `/opt/cosmos`.
## Advanced Fat APE Builds
Once you get seriously involved in creating fat APE builds of software
you're going to eventually outgrow `fatcosmocc`. One example is Emacs
which is trickier to build, because it produces architecture-specific
files, and it also depends on shared files, e.g. zoneinfo. Since we like
having everything in a neat little single-file executable container that
doesn't need an "installation wizard", this tutorial will explain how we
manage to accomplish that.
What you're going to do is, instead of using `fatcosmocc`, you're going
to use both the `x86_64-unknown-cosmo-cc` and `aarch64-unknown-cosmo-cc`
toolchains independently, and then run `apelink` and `zip` to manually
build the final files. But there's a few tricks to learn first.
The first trick is to create a symlink on your system called `/zip`.
Cosmopolitan Libc normally uses that as a synthetic folder that lets you
access the assets in your zip executable. But since that's a read-only
file system, your build system should use the normal one.
```sh
doas ln -sf /opt/cosmos /zip
```
Now create a file named `rebuild-fat.sh` which runs the build twice:
```sh
#!/bin/sh
set -ex
export MODE=aarch64
export COSMOS=/opt/cosmos/aarch64
rebuild-cosmos.sh aarch64
export MODE=
export COSMOS=/opt/cosmos/x86_64
rebuild-cosmos.sh x86_64
wall.com 'finished building'
```
Then create a second file `rebuild-cosmos.sh` which runs your build:
```sh
#!/bin/bash
set -ex
ARCH=${1:-x86_64}
export COSMO=${COSMO:-/opt/cosmo}
export COSMOS=${COSMOS:-/opt/cosmos/$ARCH}
export AS=$(command -v $ARCH-unknown-cosmo-as) || exit
export CC=$(command -v $ARCH-unknown-cosmo-cc) || exit
export CXX=$(command -v $ARCH-unknown-cosmo-c++) || exit
export AR=$(command -v $ARCH-unknown-cosmo-ar) || exit
export STRIP=$(command -v $ARCH-unknown-cosmo-strip) || exit
export INSTALL=$(command -v $ARCH-unknown-cosmo-install) || exit
export OBJCOPY=$(command -v $ARCH-unknown-cosmo-objcopy) || exit
export OBJDUMP=$(command -v $ARCH-unknown-cosmo-objdump) || exit
export ADDR2LINE=$(command -v $ARCH-unknown-cosmo-addr2line) || exit
$CC --update
export COSMOPOLITAN_DISABLE_ZIPOS=1
cd ~/vendor/zlib
./configure --prefix=$COSMOS --static
make clean
make -j
make install
cd ~/vendor/ncurses-6.4
./configure --prefix=$COSMOS --sysconfdir=/zip --datarootdir=/zip/share --exec-prefix=/zip/$ARCH --disable-shared
make clean
make -j
make install
cd ~/vendor/readline-8.2
./configure --prefix=$COSMOS --sysconfdir=/zip --datarootdir=/zip/share --exec-prefix=/zip/$ARCH --disable-shared
make uninstall || true
make clean
make -j
make install
# NOTES:
# 1. You'll need to patch enum { FOO = x } that fails to build into a #define FOO
# 2. You'll need to patch configure.ac so it DOES NOT define USABLE_FIONREAD to 1
# 2. You'll need to patch configure.ac so it DOES NOT define INTERRUPT_INPUT to 1
cd ~/vendor/emacs-28.2
./configure --prefix=$COSMOS --sysconfdir=/zip --datarootdir=/zip/share --exec-prefix=/zip/$ARCH \
--without-x --with-threads --without-gnutls --disable-silent-rules --with-file-notification=no
make uninstall || true
make clean
export CC=x86_64-unknown-cosmo-cc
export CXX=x86_64-unknown-cosmo-c++
./configure --prefix=/opt/cosmos/x86_64
make -j
make install
```
Once you've completed this build process, you'll have the ELF files
`/opt/cosmos/x86_64/bin/emacs` and `/opt/cosmos/aarch64/bin/emacs`. Your
next move is to combine them into a single pristine `emacs.com` file.
## Cosmopolitan Source Builds
Cosmopolitan can be compiled from source on any of our supported
platforms. The Makefile will download cosmocc automatically.
It's recommended that you install a systemwide APE Loader. This command
requires `sudo` access to copy the `ape` command to a system folder and
register with binfmt_misc on Linux, for even more performance.
```sh
cd /zip
COSMO=${COSMO:-/opt/cosmo}
mkdir -p /opt/cosmos/bin
apelink \
-o /opt/cosmos/bin/emacs.com \
-l "$COSMO/o//ape/ape.elf" \
-l "$COSMO/o/aarch64/ape/ape.elf" \
-M "$COSMO/ape/ape-m1.c" \
/opt/cosmos/x86_64/bin/emacs \
/opt/cosmos/aarch64/bin/emacs
cd /zip
zip -r /opt/cosmos/bin/emacs.com \
aarch64/libexec \
x86_64/libexec \
share/terminfo \
$(find share/emacs -type f |
grep -v '\.el.gz$' |
grep -v refcards |
grep -v images)
ape/apeinstall.sh
```
You can now scp your `emacs.com` build to seven operating systems for
two distinct kinds of microprocessors without any dependencies. All the
LISP, zoneinfo, and termcap files it needs are stored inside the ZIP
structure of the binary, which has performance that's equivalent to the
Linux filesystem (even though it decompresses artifacts on the fly!) For
this reason, you might actually find that fat APE Emacs goes faster if
you're using an operating system like Windows where files are go slow.
If you like to use Vim instead of Emacs, then you can build that too.
However Vim's build system makes it a bit harder, since it's configured
to always strip binaries. The `apelink` program needs the symbol tables
to still be there when it creates the fat version. Otherwise tools like
`--ftrace` won't work.
## Monolithic Source Builds
Cosmopolitan can be compiled from source on any Linux distro. First, you
need to download or clone the repository.
You can now build the mono repo with any modern version of GNU Make. To
make life easier, we've included one in the cosmocc toolchain, which is
guaranteed to be compatible and furthermore includes our extensions for
doing build system sandboxing.
```sh
wget https://justine.lol/cosmopolitan/cosmopolitan.tar.gz
tar xf cosmopolitan.tar.gz # see releases page
cd cosmopolitan
```
This will build the entire repository and run all the tests:
```sh
build/bootstrap/make.com
o//examples/hello.com
find o -name \*.com | xargs ls -rShal | less
```
If you get an error running make.com then it's probably because you have
WINE installed to `binfmt_misc`. You can fix that by installing the the
APE loader as an interpreter. It'll improve build performance too!
```sh
bin/ape-install
build/bootstrap/make -j8
o//examples/hello
```
Since the Cosmopolitan repository is very large, you might only want to
build a particular thing. Cosmopolitan's build config does a good job at
having minimal deterministic builds. For example, if you wanted to build
only hello.com then you could do that as follows:
build one particular thing. Here's an example of a target that can be
compiled relatively quickly, which is a simple POSIX test that only
depends on core LIBC packages.
```sh
build/bootstrap/make.com o//examples/hello.com
rm -rf o//libc o//test
build/bootstrap/make o//test/posix/signal_test
o//test/posix/signal_test
```
Sometimes it's desirable to build a subset of targets, without having to
list out each individual one. You can do that by asking make to build a
directory name. For example, if you wanted to build only the targets and
subtargets of the chibicc package including its tests, you would say:
list out each individual one. For example if you wanted to build and run
all the unit tests in the `TEST_POSIX` package, you could say:
```sh
build/bootstrap/make.com o//third_party/chibicc
o//third_party/chibicc/chibicc.com --help
build/bootstrap/make o//test/posix
```
Cosmopolitan provides a variety of build modes. For example, if you want
really tiny binaries (as small as 12kb in size) then you'd say:
```sh
build/bootstrap/make.com m=tiny
build/bootstrap/make m=tiny
```
Here's some other build modes you can try:
You can furthermore cut out the bloat of other operating systems, and
have Cosmopolitan become much more similar to Musl Libc.
```sh
build/bootstrap/make.com m=dbg # asan + ubsan + debug
build/bootstrap/make.com m=asan # production memory safety
build/bootstrap/make.com m=opt # -march=native optimizations
build/bootstrap/make.com m=rel # traditional release binaries
build/bootstrap/make.com m=optlinux # optimal linux-only performance
build/bootstrap/make.com m=fastbuild # build 28% faster w/o debugging
build/bootstrap/make.com m=tinylinux # tiniest linux-only 4kb binaries
build/bootstrap/make m=tinylinux
```
For further details, see [//build/config.mk](build/config.mk).
## Cosmopolitan Amalgamation
## Debugging
Another way to use Cosmopolitan is via our amalgamated release, where
we've combined everything into a single static archive and a single
header file. If you're doing your development work on Linux or BSD then
you need just five files to get started. Here's what you do on Linux:
To print a log of system calls to stderr:
```sh
wget https://justine.lol/cosmopolitan/cosmopolitan-amalgamation-3.0.zip
unzip cosmopolitan-amalgamation-3.0.zip
printf 'main() { printf("hello world\\n"); }\n' >hello.c
gcc -g -Os -static -nostdlib -nostdinc -fno-pie -no-pie -mno-red-zone \
-fno-omit-frame-pointer -pg -mnop-mcount -mno-tls-direct-seg-refs -gdwarf-4 \
-o hello.com.dbg hello.c -fuse-ld=bfd -Wl,-T,ape.lds -Wl,--gc-sections \
-Wl,-z,common-page-size=0x1000 -Wl,-z,max-page-size=0x1000 \
-include cosmopolitan.h crt.o ape-no-modify-self.o cosmopolitan.a
objcopy -S -O binary hello.com.dbg hello.com
cosmocc -o hello hello.c
./hello --strace
```
You now have a portable program.
To print a log of function calls to stderr:
```sh
./hello.com
bash -c './hello.com' # older zsh/fish workaround (patched in zsh 5.9 and fish 3.3.0)
cosmocc -o hello hello.c
./hello --ftrace
```
If `./hello.com` executed on Linux throws an error about not finding an
interpreter, it should be fixed by running the following command (although
note that it may not survive a system restart):
Both strace and ftrace use the unbreakable kprintf() facility, which is
able to be sent to a file by setting an environment variable.
```sh
sudo sh -c "echo ':APE:M::MZqFpD::/bin/sh:' >/proc/sys/fs/binfmt_misc/register"
export KPRINTF_LOG=log
./hello --strace
```
If the same command produces puzzling errors on WSL or WINE when using
Redbean 2.x, they may be fixed by disabling binfmt_misc:
```sh
sudo sh -c 'echo -1 >/proc/sys/fs/binfmt_misc/status'
```
Since we used the `ape-no-modify-self.o` bootloader (rather than
`ape.o`) your executable will not modify itself when it's run. What
it'll instead do, is extract a 4kb program (the [APE loader](https://justine.lol/apeloader/))
to `${TMPDIR:-${HOME:-.}}` that maps your program into memory without
needing to copy it. The APE loader must be in an executable location
(e.g. not stored on a `noexec` mount) for it to run. See below for
alternatives:
It's possible to install the APE loader systemwide as follows.
```sh
# System-Wide APE Install
# for Linux, Darwin, and BSDs
# 1. Copies APE Loader to /usr/bin/ape
# 2. Registers w/ binfmt_misc too if Linux
bin/ape-install
# System-Wide APE Uninstall
# for Linux, Darwin, and BSDs
bin/ape-uninstall
```
It's also possible to convert APE binaries into the system-local format
by using the `--assimilate` flag. Please note that if binfmt_misc is in
play, you'll need to unregister it temporarily before doing this, since
the assimilate feature is part of the shell script header.
```sh
$ file hello.com
hello.com: DOS/MBR boot sector
./hello.com --assimilate
$ file hello.com
hello.com: ELF 64-bit LSB executable
```
Now that you're up and running with Cosmopolitan Libc and APE, here's
some of the most important troubleshooting tools APE offers that you
should know, in case you encounter any issues:
```sh
./hello.com --strace # log system calls to stderr
./hello.com --ftrace # log function calls to stderr
```
Do you love tiny binaries? If so, you may not be happy with Cosmo adding
heavyweight features like tracing to your binaries by default. In that
case, you may want to consider using our build system:
```sh
make m=tiny
```
Which will cause programs such as `hello.com` and `life.com` to shrink
from 60kb in size to about 16kb. There's also a prebuilt amalgamation
online <https://justine.lol/cosmopolitan/cosmopolitan-tiny.zip> hosted
on our download page <https://justine.lol/cosmopolitan/download.html>.
## GDB
Here's the recommended `~/.gdbinit` config:
@@ -535,46 +179,60 @@ end
src
```
You normally run the `.com.dbg` file under gdb. If you need to debug the
`.com` file itself, then you can load the debug symbols independently as
You normally run the `.dbg` file under gdb. If you need to debug the
`` file itself, then you can load the debug symbols independently as
```sh
gdb foo.com -ex 'add-symbol-file foo.com.dbg 0x401000'
gdb foo -ex 'add-symbol-file foo.dbg 0x401000'
```
## Alternative Development Environments
## Platform Notes
### MacOS
### Shells
If you're developing on MacOS you can install the GNU compiler
collection for x86_64-elf via homebrew:
If you use zsh and have trouble running APE programs try `sh -c ./prog`
or simply upgrade to zsh 5.9+ (since we patched it two years ago). The
same is the case for Python `subprocess`, old versions of fish, etc.
### Linux
Some Linux systems are configured to launch MZ executables under WINE.
Other distros configure their stock installs so that APE programs will
print "run-detectors: unable to find an interpreter". For example:
```sh
brew install x86_64-elf-gcc
jart@ubuntu:~$ wget https://cosmo.zip/pub/cosmos/bin/dash
jart@ubuntu:~$ chmod +x dash
jart@ubuntu:~$ ./dash
run-detectors: unable to find an interpreter for ./dash
```
Then in the above scripts just replace `gcc` and `objcopy` with
`x86_64-elf-gcc` and `x86_64-elf-objcopy` to compile your APE binary.
### Windows
If you're developing on Windows then you need to download an
x86_64-pc-linux-gnu toolchain beforehand. See the [Compiling on
Windows](https://justine.lol/cosmopolitan/windows-compiling.html)
tutorial. It's needed because the ELF object format is what makes
universal binaries possible.
Cosmopolitan officially only builds on Linux. However, one highly
experimental (and currently broken) thing you could try, is building the
entire cosmo repository from source using the cross9 toolchain.
You can fix that by registering APE with `binfmt_misc`:
```sh
mkdir -p o/third_party
rm -rf o/third_party/gcc
wget https://justine.lol/linux-compiler-on-windows/cross9.zip
unzip cross9.zip
mv cross9 o/third_party/gcc
build/bootstrap/make.com
sudo wget -O /usr/bin/ape https://cosmo.zip/pub/cosmos/bin/ape-$(uname -m).elf
sudo chmod +x /usr/bin/ape
sudo sh -c "echo ':APE:M::MZqFpD::/usr/bin/ape:' >/proc/sys/fs/binfmt_misc/register"
sudo sh -c "echo ':APE-jart:M::jartsr::/usr/bin/ape:' >/proc/sys/fs/binfmt_misc/register"
```
You should be good now. APE will not only work, it'll launch executables
400µs faster now too. However if things still didn't work out, it's also
possible to disable `binfmt_misc` as follows:
```sh
sudo sh -c 'echo -1 > /proc/sys/fs/binfmt_misc/cli' # remove Ubuntu's MZ interpreter
sudo sh -c 'echo -1 > /proc/sys/fs/binfmt_misc/status' # remove ALL binfmt_misc entries
```
### WSL
It's normally unsafe to use APE in a WSL environment, because it tries
to run MZ executables as WIN32 binaries within the WSL environment. In
order to make it safe to use Cosmopolitan software on WSL, run this:
```sh
sudo sh -c "echo -1 > /proc/sys/fs/binfmt_misc/WSLInterop"
```
## Discord Chatroom
@@ -584,16 +242,16 @@ server. You're welcome to join us! <https://discord.gg/FwAVVu7eJ4>
## Support Vector
| Platform | Min Version | Circa |
| :--- | ---: | ---: |
| AMD | K8 Venus | 2005 |
| Intel | Core | 2006 |
| Linux | 2.6.18 | 2007 |
| Windows | 8 [1] | 2012 |
| Mac OS X | 15.6 | 2018 |
| OpenBSD | 7 | 2021 |
| FreeBSD | 13 | 2020 |
| NetBSD | 9.2 | 2021 |
| Platform | Min Version | Circa |
| :--- | ---: | ---: |
| AMD | K8 | 2003 |
| Intel | Core | 2006 |
| Linux | 2.6.18 | 2007 |
| Windows | 8 [1] | 2012 |
| Darwin (macOS) | 23.1.0+ | 2023 |
| OpenBSD | 7 | 2021 |
| FreeBSD | 13 | 2020 |
| NetBSD | 9.2 | 2021 |
[1] See our [vista branch](https://github.com/jart/cosmopolitan/tree/vista)
for a community supported version of Cosmopolitan that works on Windows
+287
View File
@@ -0,0 +1,287 @@
#-*-mode:makefile-gmake;indent-tabs-mode:t;tab-width:8;coding:utf-8-*-┐
#── vi: set noet ft=make ts=8 sw=8 fenc=utf-8 :vi ────────────────────┘
#
# OVERVIEW
#
# αcτµαlly pδrταblε εxεcµταblε
#
# DESCRIPTION
#
# This file defines the libraries, runtimes, and build rules needed to
# create executables from your Linux workstation that'll run anywhere.
# Loading this package will make certain systemic modifications to the
# build like turning off the System V "Red Zone" optimization, because
# αcτµαlly pδrταblε εxεcµταblεs need to be able to run in kernelspace.
PKGS += APE
APE_FILES := $(wildcard ape/*.*)
APE_HDRS = $(filter %.h,$(APE_FILES))
APE_INCS = $(filter %.inc,$(APE_FILES))
ifeq ($(ARCH), aarch64)
APE = o/$(MODE)/ape/aarch64.lds
APELINK = \
$(COMPILE) \
-ALINK.ape \
$(LINK) \
$(LINKARGS) \
$(OUTPUT_OPTION) && \
$(COMPILE) \
-AFIXUP.ape \
-wT$@ \
$(FIXUPOBJ) \
$@
APE_SRCS = ape/ape.S ape/start.S ape/launch.S ape/systemcall.S
APE_OBJS = o/$(MODE)/ape/ape.o
APE_NO_MODIFY_SELF = $(APE)
APE_COPY_SELF = $(APE)
.PHONY: o/$(MODE)/ape
o/$(MODE)/ape: $(APE)
o/$(MODE)/ape/aarch64.lds: \
ape/aarch64.lds \
libc/zip.h \
libc/thread/tls.h \
libc/calls/struct/timespec.h \
libc/macros.h \
libc/str/str.h
APE_LOADER_LDFLAGS = \
-pie \
-static \
-nostdlib \
--no-dynamic-linker \
-z norelro \
-z common-page-size=0x4000 \
-z max-page-size=0x4000
APE_LOADER_FLAGS = \
-DNDEBUG \
-iquote. \
-Wall \
-Wextra \
-fpie \
-Os \
-ffreestanding \
-mgeneral-regs-only \
-fno-asynchronous-unwind-tables \
-fno-stack-protector \
-fno-ident \
-fno-gnu-unique \
-c \
$(OUTPUT_OPTION) \
$<
o/$(MODE)/ape/ape.elf: o/$(MODE)/ape/ape.elf.dbg
@$(COMPILE) -AOBJCOPY -T$@ $(OBJCOPY) -g $< $@
@$(COMPILE) -AFIXUPOBJ -wT$@ $(FIXUPOBJ) $@
o/$(MODE)/ape/ape.elf.dbg: \
o/$(MODE)/ape/start.o \
o/$(MODE)/ape/loader.o \
o/$(MODE)/ape/launch.o \
o/$(MODE)/ape/systemcall.o
@$(COMPILE) -ALINK.elf $(LD) $(APE_LOADER_LDFLAGS) -o $@ $(patsubst %.lds,-T %.lds,$^)
o/$(MODE)/ape/loader.o: ape/loader.c ape/ape.h
@$(COMPILE) -AOBJECTIFY.c $(CC) -DSUPPORT_VECTOR=33 -g $(APE_LOADER_FLAGS)
o/$(MODE)/ape/start.o: ape/start.S
@$(COMPILE) -AOBJECTIFY.S $(OBJECTIFY.S) $(OUTPUT_OPTION) -c $<
o/$(MODE)/ape/launch.o: ape/launch.S
@$(COMPILE) -AOBJECTIFY.S $(OBJECTIFY.S) $(OUTPUT_OPTION) -c $<
o/$(MODE)/ape/systemcall.o: ape/systemcall.S
@$(COMPILE) -AOBJECTIFY.S $(OBJECTIFY.S) $(OUTPUT_OPTION) -c $<
.PHONY: o/$(MODE)/ape
o/$(MODE)/ape: o/$(MODE)/ape/ape.elf
else
APE = o/$(MODE)/ape/ape.o \
o/$(MODE)/ape/ape.lds
APELINK = \
$(COMPILE) \
-ALINK.ape \
$(LINK) \
$(LINKARGS) \
$(OUTPUT_OPTION) && \
$(COMPILE) \
-AFIXUP.ape \
-wT$@ \
$(FIXUPOBJ) \
$@
APE_NO_MODIFY_SELF = \
o/$(MODE)/ape/ape.lds \
o/$(MODE)/ape/ape-no-modify-self.o
APE_COPY_SELF = \
o/$(MODE)/ape/ape.lds \
o/$(MODE)/ape/ape-copy-self.o
APE_LOADER_FLAGS = \
-DNDEBUG \
-iquote. \
-Wall \
-Wextra \
-Werror \
-pedantic-errors \
-fpie \
-Os \
-ffreestanding \
-mgeneral-regs-only \
-fno-stack-protector \
-fno-ident \
-fno-gnu-unique \
-c \
$(OUTPUT_OPTION) \
$<
APE_SRCS_C = ape/loader.c
APE_SRCS_S = $(filter %.S,$(APE_FILES))
APE_SRCS = $(APE_SRCS_C) $(APE_SRCS_S)
APE_OBJS = $(APE_SRCS_S:%.S=o/$(MODE)/%.o)
APE_CHECKS = $(APE_HDRS:%=o/%.ok)
o/ape/idata.inc: \
ape/idata.internal.h \
ape/relocations.h
o/$(MODE)/ape/ape-no-modify-self.o: \
ape/ape.S \
ape/ape.h \
ape/macros.internal.h \
ape/relocations.h \
ape/ape.internal.h \
libc/dce.h \
libc/elf/def.h \
libc/thread/tls.h \
libc/macho.h \
libc/macros.h \
libc/nexgen32e/uart.internal.h \
libc/calls/metalfile.internal.h \
libc/nt/pedef.internal.h \
libc/runtime/e820.internal.h \
libc/runtime/mman.internal.h \
libc/runtime/pc.internal.h \
libc/sysv/consts/prot.h \
o/$(MODE)/ape/ape.elf
@$(COMPILE) \
-AOBJECTIFY.S \
$(OBJECTIFY.S) \
$(OUTPUT_OPTION) \
-DAPE_NO_MODIFY_SELF \
-DAPE_LOADER='"o/$(MODE)/ape/ape.elf"' $<
o/$(MODE)/ape/ape-copy-self.o: \
ape/ape.S \
ape/ape.h \
ape/macros.internal.h \
ape/relocations.h \
ape/ape.internal.h \
libc/dce.h \
libc/elf/def.h \
libc/thread/tls.h \
libc/macho.h \
libc/macros.h \
libc/nexgen32e/uart.internal.h \
libc/calls/metalfile.internal.h \
libc/nt/pedef.internal.h \
libc/runtime/e820.internal.h \
libc/runtime/mman.internal.h \
libc/runtime/pc.internal.h \
libc/sysv/consts/prot.h
@$(COMPILE) \
-AOBJECTIFY.S \
$(OBJECTIFY.S) \
$(OUTPUT_OPTION) \
-DAPE_NO_MODIFY_SELF $<
o/$(MODE)/ape/loader.o: ape/loader.c
@$(COMPILE) -AOBJECTIFY.c $(CC) -DSUPPORT_VECTOR=121 -g $(APE_LOADER_FLAGS)
o/$(MODE)/ape/loader-gcc.asm: ape/loader.c
@$(COMPILE) -AOBJECTIFY.c $(CC) -DSUPPORT_VECTOR=121 -S -g0 $(APE_LOADER_FLAGS)
o/$(MODE)/ape/loader-clang.asm: ape/loader.c
@$(COMPILE) -AOBJECTIFY.c $(CLANG) -DSUPPORT_VECTOR=121 -S -g0 $(APE_LOADER_FLAGS)
o/$(MODE)/ape/loader-xnu.o: ape/loader.c
@$(COMPILE) -AOBJECTIFY.c $(CC) -DSUPPORT_VECTOR=8 -g $(APE_LOADER_FLAGS)
o/$(MODE)/ape/loader-xnu-gcc.asm: ape/loader.c
@$(COMPILE) -AOBJECTIFY.c $(CC) -DSUPPORT_VECTOR=8 -S -g0 $(APE_LOADER_FLAGS)
o/$(MODE)/ape/loader-xnu-clang.asm: ape/loader.c
@$(COMPILE) -AOBJECTIFY.c $(CLANG) -DSUPPORT_VECTOR=8 -S -g0 $(APE_LOADER_FLAGS)
o/$(MODE)/ape/ape.elf: o/$(MODE)/ape/ape.elf.dbg
@$(COMPILE) -AOBJBINCOPY -w $(OBJBINCOPY) -f -o $@ $<
o/$(MODE)/ape/ape.macho: o/$(MODE)/ape/ape.elf.dbg
@$(COMPILE) -AOBJBINCOPY -w $(OBJBINCOPY) -fm -o $@ $<
APE_LOADER_LDFLAGS = \
-static \
-nostdlib \
--no-dynamic-linker \
-z separate-code \
-z common-page-size=0x1000 \
-z max-page-size=0x10000
o/$(MODE)/ape/ape.elf.dbg: \
o/$(MODE)/ape/loader-macho.o \
o/$(MODE)/ape/start.o \
o/$(MODE)/ape/loader.o \
o/$(MODE)/ape/launch.o \
o/$(MODE)/ape/systemcall.o \
ape/loader.lds
@$(COMPILE) -ALINK.elf $(LD) $(APE_LOADER_LDFLAGS) -o $@ $(patsubst %.lds,-T %.lds,$^)
.PHONY: o/$(MODE)/ape
o/$(MODE)/ape: $(APE_CHECKS) \
o/$(MODE)/ape/ape.o \
o/$(MODE)/ape/ape.lds \
o/$(MODE)/ape/ape.elf \
o/$(MODE)/ape/ape.macho \
endif
# these assembly files are safe to build on aarch64
o/$(MODE)/ape/ape.o: \
ape/ape.S \
ape/ape.h \
libc/dce.h \
libc/elf/def.h \
ape/relocations.h \
libc/thread/tls.h \
ape/ape.internal.h \
ape/macros.internal.h \
libc/macho.h \
libc/macros.h \
libc/sysv/consts/prot.h \
libc/nt/pedef.internal.h \
libc/runtime/pc.internal.h \
libc/runtime/e820.internal.h \
libc/runtime/mman.internal.h \
libc/nexgen32e/uart.internal.h \
libc/calls/metalfile.internal.h
@$(COMPILE) -AOBJECTIFY.S $(OBJECTIFY.S) $(OUTPUT_OPTION) -c $<
o/$(MODE)/ape/ape.lds: \
ape/ape.lds \
ape/macros.internal.h \
ape/relocations.h \
ape/ape.internal.h \
libc/thread/tls.h \
libc/calls/struct/timespec.h \
libc/thread/thread.h \
libc/dce.h \
libc/elf/def.h \
libc/elf/pf2prot.internal.h \
libc/macros.h \
libc/nt/pedef.internal.h \
libc/str/str.h \
libc/zip.h
+22 -18
View File
@@ -1,6 +1,7 @@
/*-*- mode: ld-script; indent-tabs-mode: nil; tab-width: 2; coding: utf-8 -*-│
│vi: set et sts=2 tw=2 fenc=utf-8 :vi│
│ vi: set et sts=2 sw=2 fenc=utf-8 :vi │
╚─────────────────────────────────────────────────────────────────────────────*/
#include "libc/sysv/consts/prot.h"
#include "libc/thread/tls.h"
ENTRY(_start)
@@ -11,7 +12,7 @@ OUTPUT_FORMAT("elf64-littleaarch64",
SECTIONS {
. = SEGMENT_START("text-segment", 0x010000000000);
. = SEGMENT_START("text-segment", 0x000800000000);
__executable_start = .;
. += SIZEOF_HEADERS;
@@ -89,10 +90,12 @@ SECTIONS {
*(.ubsan.data)
}
.comment : {
__comment_start = .;
KEEP(*(.comment))
.notice : {
__notices = .;
KEEP(*(.notice))
BYTE(0);
BYTE(10);
BYTE(10);
}
.eh_frame_hdr : {
@@ -100,10 +103,8 @@ SECTIONS {
*(.eh_frame_entry .eh_frame_entry.*)
}
.eh_frame : ONLY_IF_RO {
KEEP(*(.eh_frame))
*(.eh_frame.*)
}
__eh_frame_hdr_start = SIZEOF(.eh_frame_hdr) > 0 ? ADDR(.eh_frame_hdr) : 0;
__eh_frame_hdr_end = SIZEOF(.eh_frame_hdr) > 0 ? . : 0;
.gcc_except_table : ONLY_IF_RO {
*(.gcc_except_table .gcc_except_table.*)
@@ -124,9 +125,11 @@ SECTIONS {
. += CONSTANT(MAXPAGESIZE);
. = DATA_SEGMENT_ALIGN(CONSTANT(MAXPAGESIZE), CONSTANT(COMMONPAGESIZE));
.eh_frame : ONLY_IF_RW {
.eh_frame : {
__eh_frame_start = .;
KEEP(*(.eh_frame))
*(.eh_frame.*)
__eh_frame_end = .;
}
.gnu_extab : ONLY_IF_RW {
@@ -157,8 +160,11 @@ SECTIONS {
.init_array : {
__init_array_start = .;
KEEP(*(SORT_BY_INIT_PRIORITY(.init_array.*) SORT_BY_INIT_PRIORITY(.ctors.*)))
KEEP(*(.init_array EXCLUDE_FILE(*crtbegin.o *crtbegin?.o *crtend.o *crtend?.o ) .ctors))
KEEP(*(.preinit_array))
KEEP(*(SORT_BY_INIT_PRIORITY(.init_array.*)
SORT_BY_INIT_PRIORITY(.ctors.*)))
KEEP(*(.init_array))
KEEP(*(.ctors))
__init_array_end = .;
}
@@ -280,15 +286,13 @@ SECTIONS {
ape_stack_vaddr = DEFINED(ape_stack_vaddr) ? ape_stack_vaddr : 0x700000000000;
ape_stack_memsz = DEFINED(ape_stack_memsz) ? ape_stack_memsz : 8 * 1024 * 1024;
ape_stack_align = DEFINED(ape_stack_align) ? ape_stack_align : 16;
ape_stack_round = -ape_stack_align;
ape_stack_prot = PROT_READ | PROT_WRITE;
_tls_size = _tbss_end - _tdata_start;
_tdata_size = _tdata_end - _tdata_start;
_tbss_size = _tbss_end - _tbss_start;
_tbss_offset = _tbss_start - _tdata_start;
_tls_content = (_tdata_end - _tdata_start) + (_tbss_end - _tbss_start);
_tls_align = MAX(ALIGNOF(.tdata), ALIGNOF(.tbss));
ASSERT(ALIGNOF(.tdata) <= TLS_ALIGNMENT && ALIGNOF(.tbss) <= TLS_ALIGNMENT,
"_Thread_local _Alignof can't exceed TLS_ALIGNMENT");
_tdata_align = ALIGNOF(.tdata);
_tbss_align = ALIGNOF(.tbss);
_tls_align = MAX(TLS_ALIGNMENT, MAX(ALIGNOF(.tdata), ALIGNOF(.tbss)));
+529 -453
View File
File diff suppressed because it is too large. Load diff
+29 -60
View File
@@ -1,5 +1,5 @@
/*-*- mode:unix-assembly; indent-tabs-mode:t; tab-width:8; coding:utf-8 -*-│
│vi: set et ft=asm ts=8 tw=8 fenc=utf-8 :vi│
│ vi: set noet ft=asm ts=8 sw=8 fenc=utf-8 nofixeol :vi │
╞══════════════════════════════════════════════════════════════════════════════╡
│ Copyright 2020 Justine Alexandra Roberts Tunney │
│ │
@@ -33,12 +33,11 @@
│ αcτµαlly pδrταblε εxεcµταblε § program header │
╚─────────────────────────────────────────────────────────────────────────────*/
#include "ape/macros.internal.h"
#include "ape/notice.inc"
#include "ape/relocations.h"
#include "libc/calls/metalfile.internal.h"
#include "libc/dce.h"
#include "libc/elf/def.h"
#include "libc/macho.internal.h"
#include "libc/macho.h"
#include "libc/nexgen32e/uart.internal.h"
#include "libc/nt/pedef.internal.h"
#include "libc/runtime/pc.internal.h"
@@ -197,7 +196,7 @@ ape_mz:
.quad ape_elf_entry // 18: e_entry
.quad ape_elf_phoff // 20: e_phoff
.quad ape_elf_shoff // 28: e_shoff
.long 0 // 30: e_flags
.long 0x101ca75 // 30: ape e_flags
.short 64 // 34: e_ehsize
.short 56 // 36: e_phentsize
.short ape_elf_phnum // 38: e_phnum
@@ -592,7 +591,7 @@ ape_disk:
#ifdef APE_IS_SHELL_SCRIPT
apesh: .ascii "\n@\n#'\"\n" // sixth edition shebang
.ascii "m=\"$(uname -m)\"\n"
.ascii "m=$(uname -m 2>/dev/null) || m=x86_64\n"
.ascii "if [ \"$m\" = x86_64 ] || [ \"$m\" = amd64 ]; then\n"
// Until all operating systems can be updated to support APE,
@@ -670,7 +669,7 @@ apesh: .ascii "\n@\n#'\"\n" // sixth edition shebang
.shstub ape_elf_entry,8 // 18: e_entry
.shstub ape_elf_phoff,8 // 20: e_phoff
.shstub ape_elf_shoff,8 // 28: e_shoff
.ascii "\\0\\0\\0\\0" // 30: e_flags
.ascii "\\165\\312\\1\\1" // 30: ape e_flags
.ascii "\\100\\0" // 34: e_ehsize
.ascii "\\070\\0" // 36: e_phentsize
.shstub ape_elf_phnum,2 // 38: e_phnum
@@ -700,36 +699,9 @@ apesh: .ascii "\n@\n#'\"\n" // sixth edition shebang
#endif /* APE_NO_MODIFY_SELF */
.ascii "exit $?\n"
.ascii "fi\n" // x86_64
// ...
// decentralized section (.apesh)
// ...
.ascii "PHDRS='' <<'@'\n"
.endobj apesh
// elf program headers get inserted here
// because they need to be in the first 4096 bytes
.section .emushprologue,"a",@progbits
emush: .ascii "\n@\n#'\"\n"
.ascii "s=\"$(uname -s)\"\n"
// our script is running on a non-x86_64 architecture
// 1. `dd` out the appropriate blink vm blob
// 2. gunzip the blink virtual machine executable
// 3. relaunch this program inside the blink vm
.ascii "o=\"$(command -v \"$0\")\"\n"
.ascii "e=\"${TMPDIR:-${HOME:-.}}/.ape-blink-1.0.0\"\n"
.previous
// ...
// decentralized section (.emush)
// - __static_yoink("blink_linux_aarch64"); // for raspberry pi
// - __static_yoink("blink_xnu_aarch64"); // is apple silicon
// ...
.section .emushepilogue,"a",@progbits
.ascii "echo \"$0: this ape binary lacks $m support\" >&2\n"
.rept 16
.ascii "exit 127\n"
.endr
.ascii "echo error: this ape binary only supports x86_64 >&2\n"
.ascii "exit 1\n"
.previous
.endobj apesh
#ifdef APE_LOADER
.section .ape.loader,"a",@progbits
@@ -748,6 +720,8 @@ ape_loader_end:
#if SupportsSystemv() || SupportsMetal()
.section .elf.phdrs,"a",@progbits
.globl ape_phdrs
ape_phdrs:
.long PT_LOAD
.long PF_R|PF_X
.stub ape_cod_offset,quad
@@ -787,7 +761,7 @@ ape_loader_end:
.quad 0
.quad 0
.quad 0
.stub ape_stack_align,quad
.quad 16
#if SupportsOpenbsd() || SupportsNetbsd()
.long PT_NOTE
@@ -1062,7 +1036,7 @@ ape_pe: .ascin "PE",4
.quad ape_pe_base // ImageBase
.long ape_pe_sectionalignment // SectionAlignment
.long ape_pe_filealignment // FileAlignment
.short v_ntversion // MajorOperatingSystemVersion
.short 10 // MajorOperatingSystemVersion
.short 0 // MinorOperatingSystemVersion
.short 0 // MajorImageVersion
.short 0 // MinorImageVersion
@@ -1797,49 +1771,31 @@ kernel: movabs $ape_stack_vaddr,%rsp
.type ape_text_nops,@object
.type __test_end,@object
.section .commentprologue,"a",@progbits
.globl __comment_start
.type __comment_start,@object
.hidden __comment_start
__comment_start:/*
...
decentralized content
...
*/.previous
.section .commentepilogue,"a",@progbits
.byte 0
.previous
.section .ape.pad.head,"a",@progbits
.type ape_pad_head,@object
.hidden ape_pad_head
ape_pad_head:
.previous
.section .ape.pad.text,"a",@progbits
.type ape_pad_text,@object
.hidden ape_pad_text
ape_pad_text:
.previous
.section .ape.pad.privileged,"a",@progbits
.type ape_pad_privileged,@object
.hidden ape_pad_privileged
ape_pad_privileged:
.previous
.section .ape.pad.data,"a",@progbits
.type ape_pad_data,@object
.hidden ape_pad_data
ape_pad_data:
.previous
#if SupportsWindows()
.section .idata.ro,"a",@progbits
.type ape_idata_ro,@object
.hidden ape_idata_ro
ape_idata_ro:
.previous
#endif /* SupportsWindows() */
.section .dataprologue,"aw",@progbits
@@ -1847,32 +1803,45 @@ ape_idata_ro:
.globl __data_start
.hidden __data_start
__data_start:
.previous
.section .dataepilogue,"aw",@progbits
.type __data_end,@object
.globl __data_end
.hidden __data_end
__data_end:
.previous
.section .bssprologue,"aw",@nobits
.type __bss_start,@object
.globl __bss_start
.hidden __bss_start
__bss_start:
.previous
.section .bssepilogue,"aw",@nobits
.type __bss_end,@object
.globl __bss_end
.hidden __bss_end
__bss_end:
.previous
.section .fstls,"awT",@nobits
.align TLS_ALIGNMENT
.previous
.section .notice,"aR",@progbits
.asciz "\n\n\
Cosmopolitan\n\
Copyright 2024 Justine Alexandra Roberts Tunney\n\
\n\
Permission to use, copy, modify, and/or distribute this software for\n\
any purpose with or without fee is hereby granted, provided that the\n\
above copyright notice and this permission notice appear in all copies.\n\
\n\
THE SOFTWARE IS PROVIDED \"AS IS\" AND THE AUTHOR DISCLAIMS ALL\n\
WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED\n\
WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE\n\
AUTHOR BE LIABLE FOR ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL\n\
DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR\n\
PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER\n\
TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR\n\
PERFORMANCE OF THIS SOFTWARE."
.end

+1 -1
View File
@@ -2,7 +2,7 @@
#define COSMOPOLITAN_APE_APE_H_
#define APE_VERSION_MAJOR 1
#define APE_VERSION_MINOR 8
#define APE_VERSION_MINOR 10
#define APE_VERSION_STR APE_VERSION_STR_(APE_VERSION_MAJOR, APE_VERSION_MINOR)
#define APE_VERSION_NOTE APE_VERSION_NOTE_(APE_VERSION_MAJOR, APE_VERSION_MINOR)
+36 -51
View File
@@ -1,5 +1,5 @@
/*-*- mode: ld-script; indent-tabs-mode: nil; tab-width: 2; coding: utf-8 -*-│
│vi: set et sts=2 tw=2 fenc=utf-8 :vi│
│ vi: set et sts=2 sw=2 fenc=utf-8 :vi │
╞══════════════════════════════════════════════════════════════════════════════╡
│ Copyright 2020 Justine Alexandra Roberts Tunney │
│ │
@@ -229,7 +229,6 @@ SECTIONS {
/* Real Mode */
KEEP(*(.head))
KEEP(*(.apesh))
KEEP(*(.text.head))
/* Executable & Linkable Format */
@@ -238,10 +237,6 @@ SECTIONS {
KEEP(*(.elf.phdrs))
ape_phdrs_end = .;
KEEP(*(.emushprologue))
KEEP(*(.emush))
KEEP(*(.emushepilogue))
/* OpenBSD */
. = ALIGN(. != 0 ? __SIZEOF_POINTER__ : 0);
ape_note = .;
@@ -287,12 +282,9 @@ SECTIONS {
KEEP(*(SORT_BY_NAME(.init.*)))
KEEP(*(.init))
KEEP(*(.initepilogue))
KEEP(*(.pltprologue))
*(.plt)
KEEP(*(.pltepilogue))
KEEP(*(.pltgotprologue))
*(.plt.got)
KEEP(*(.pltgotepilogue))
*(.iplt)
*(.text.startup .text.startup.*)
*(.text.exit .text.exit.*)
*(.text.unlikely .text.*_unlikely .text.unlikely.*)
@@ -301,7 +293,6 @@ SECTIONS {
KEEP(*(.textwindowsprologue))
*(.text.windows)
KEEP(*(.textwindowsepilogue))
KEEP(*(.blink))
*(SORT_BY_ALIGNMENT(.text.modernity))
*(SORT_BY_ALIGNMENT(.text.modernity.*))
*(SORT_BY_ALIGNMENT(.text.hot))
@@ -319,7 +310,7 @@ SECTIONS {
. = ALIGN(__privileged_end > __privileged_start ? CONSTANT(COMMONPAGESIZE) : 0);
/*END: morphable code */
__privileged_start = .;
*(.privileged)
*(.privileged .privileged.*)
__privileged_end = .;
KEEP(*(.ape.pad.text))
@@ -329,7 +320,7 @@ SECTIONS {
/*BEGIN: Read Only Data */
.rodata . : {
.rodata ALIGN(CONSTANT(COMMONPAGESIZE)) : {
KEEP(*(.rodata.pytab.0));
KEEP(*(.rodata.pytab.1));
KEEP(*(.rodata.pytab.2));
@@ -338,12 +329,16 @@ SECTIONS {
*(.ubsan.types)
*(.ubsan.data)
__eh_frame_hdr_start_actual = .;
*(.eh_frame_hdr)
__eh_frame_hdr_end_actual = .;
/* Legal Notices */
#if !defined(IM_FEELING_NAUGHTY) || defined(EMBED_NOTICES)
KEEP(*(.commentprologue))
KEEP(*(.comment))
KEEP(*(.commentepilogue))
#endif
__notices = .;
KEEP(*(.notice))
BYTE(0);
BYTE(10);
BYTE(10);
/*BEGIN: read-only data that's only needed for initialization */
@@ -399,26 +394,28 @@ SECTIONS {
/*BEGIN: NT FORK COPYING */
KEEP(*(.dataprologue))
*(.data .data.*)
*(.gnu_extab)
*(.gcc_except_table .gcc_except_table.*)
*(.exception_ranges*)
*(.PyRuntime) /* for python */
*(.subrs) /* for emacs */
KEEP(*(SORT_BY_NAME(.sort.data.*)))
. += . > 0 ? CODE_GRANULE : 0;
KEEP(*(.gotprologue))
. = ALIGN(. != 0 ? __SIZEOF_POINTER__ : 0);
__got_start = .;
*(.got)
KEEP(*(.gotepilogue))
__got_end = .;
KEEP(*(.gotpltprologue))
*(.got.plt)
KEEP(*(.gotpltepilogue))
. = ALIGN(. != 0 ? __SIZEOF_POINTER__ : 0);
__init_array_start = .;
KEEP(*(.preinit_array))
KEEP(*(SORT_BY_INIT_PRIORITY(.init_array.*)
SORT_BY_INIT_PRIORITY(.ctors.*)))
KEEP(*(.ctors))
KEEP(*(.init_array))
KEEP(*(.preinit_array))
KEEP(*(.ctors))
__init_array_end = .;
. = ALIGN(. != 0 ? __SIZEOF_POINTER__ : 0);
@@ -429,13 +426,20 @@ SECTIONS {
KEEP(*(.dtors))
__fini_array_end = .;
__eh_frame_start = .;
KEEP(*(.eh_frame))
*(.eh_frame.*)
__eh_frame_end = .;
/*BEGIN: Post-Initialization Read-Only */
. = ALIGN(. != 0 ? __SIZEOF_POINTER__ : 0);
KEEP(*(SORT_BY_NAME(.piro.relo.sort.*)))
. = ALIGN(. != 0 ? __SIZEOF_POINTER__ : 0);
KEEP(*(SORT_BY_NAME(.piro.data.sort.*)))
KEEP(*(.piro.pad.data))
*(.igot.plt)
KEEP(*(.dataepilogue))
. = ALIGN(. != 0 ? CONSTANT(COMMONPAGESIZE) : 0);
/*END: NT FORK COPYING */
_edata = .;
@@ -533,7 +537,9 @@ SECTIONS {
*(.piro.data.sort.iat.*)
#endif
*(__patchable_function_entries)
*(.note.gnu.property)
*(__mcount_loc)
*(.rela.dyn)
*(.discard)
*(.yoink)
}
@@ -556,7 +562,9 @@ _tdata_size = _tdata_end - _tdata_start;
_tbss_size = _tbss_end - _tbss_start;
_tbss_offset = _tbss_start - _tdata_start;
_tls_content = (_tdata_end - _tdata_start) + (_tbss_end - _tbss_start);
_tls_align = 1;
_tdata_align = ALIGNOF(.tdata);
_tbss_align = ALIGNOF(.tbss);
_tls_align = MAX(TLS_ALIGNMENT, MAX(ALIGNOF(.tdata), ALIGNOF(.tbss)));
ape_cod_offset = 0;
ape_cod_vaddr = ADDR(.head);
@@ -589,8 +597,6 @@ ape_stack_vaddr = DEFINED(ape_stack_vaddr) ? ape_stack_vaddr : 0x700000000000;
ape_stack_paddr = ape_ram_paddr + ape_ram_filesz;
ape_stack_filesz = 0;
ape_stack_memsz = DEFINED(ape_stack_memsz) ? ape_stack_memsz : 8 * 1024 * 1024;
ape_stack_align = DEFINED(ape_stack_align) ? ape_stack_align : 16;
ape_stack_round = -ape_stack_align;
ape_note_offset = ape_cod_offset + (ape_note - ape_cod_vaddr);
ape_note_filesz = ape_note_end - ape_note;
@@ -604,6 +610,9 @@ ape_text_memsz = ape_text_filesz;
ape_text_align = CONSTANT(COMMONPAGESIZE);
ape_text_rva = RVA(ape_text_vaddr);
__eh_frame_hdr_start = __eh_frame_hdr_end_actual > __eh_frame_hdr_start_actual ? __eh_frame_hdr_start_actual : 0;
__eh_frame_hdr_end = __eh_frame_hdr_end_actual > __eh_frame_hdr_start_actual ? __eh_frame_hdr_end_actual : 0;
/* we roundup here because xnu wants the file load segments page-aligned */
/* but we don't want to add the nop padding to the ape program, so we'll */
/* let ape.S dd read past the end of the file into the wrapping binaries */
@@ -613,29 +622,6 @@ SHSTUB2(ape_loader_dd_count,
? ROUNDUP(ape_loader_end - ape_loader, CONSTANT(COMMONPAGESIZE)) / 64
: 0);
#if defined(APE_IS_SHELL_SCRIPT) && !IsTiny()
#define IDENTITY(X) X
#define APE_DECLARE_FIXED_DECIMAL(F, X) \
X##_quad = DEFINED(X) ? ((F(X) < 1000000000 ? 32 : F(X) / 1000000000 % 10 + 48) << 000 | \
(F(X) < 100000000 ? 32 : F(X) / 100000000 % 10 + 48) << 010 | \
(F(X) < 10000000 ? 32 : F(X) / 10000000 % 10 + 48) << 020 | \
(F(X) < 1000000 ? 32 : F(X) / 1000000 % 10 + 48) << 030 | \
(F(X) < 100000 ? 32 : F(X) / 100000 % 10 + 48) << 040 | \
(F(X) < 10000 ? 32 : F(X) / 10000 % 10 + 48) << 050 | \
(F(X) < 1000 ? 32 : F(X) / 1000 % 10 + 48) << 060 | \
(F(X) < 100 ? 32 : F(X) / 100 % 10 + 48) << 070) : 0; \
X##_short = DEFINED(X) ? ((F(X) < 10 ? 32 : F(X) / 10 % 10 + 48) << 000 | \
(F(X) % 10 + 48) << 010) : 0
APE_DECLARE_FIXED_DECIMAL(RVA, blink_linux_aarch64);
APE_DECLARE_FIXED_DECIMAL(IDENTITY, blink_linux_aarch64_size);
APE_DECLARE_FIXED_DECIMAL(RVA, blink_xnu_aarch64);
APE_DECLARE_FIXED_DECIMAL(IDENTITY, blink_xnu_aarch64_size);
#endif /* APE_IS_SHELL_SCRIPT */
#if SupportsMetal()
v_ape_realsectors = MIN(0x70000 - IMAGE_BASE_REAL, ROUNDUP(RVA(_ezip), 512)) / 512;
v_ape_realbytes = v_ape_realsectors * 512;
@@ -739,7 +725,6 @@ ape_idataz = LINK_WINDOWS ? RVA(ape_idata_iat) : 0;
ape_idata_iatsize = LINK_WINDOWS ? ape_idata_iatend - ape_idata_iat : 0;
ape_idata = LINK_WINDOWS ? RVA(ape_idata_idt) : 0;
ape_idata_idtsize = LINK_WINDOWS ? ape_idata_idtend - ape_idata_idt : 0;
v_ntversion = LINK_WINDOWS ? 6 : 1;
v_ntdllchar = LINK_WINDOWS ? 288 : 0;
v_ntsubversion = LINK_WINDOWS ? 6 : 5;
v_ntsubsystem = (LINK_WINDOWS
-287
View File
@@ -1,287 +0,0 @@
#-*-mode:makefile-gmake;indent-tabs-mode:t;tab-width:8;coding:utf-8-*-┐
#───vi: set et ft=make ts=8 tw=8 fenc=utf-8 :vi───────────────────────┘
#
# OVERVIEW
#
# αcτµαlly pδrταblε εxεcµταblε
#
# DESCRIPTION
#
# This file defines the libraries, runtimes, and build rules needed to
# create executables from your Linux workstation that'll run anywhere.
# Loading this package will make certain systemic modifications to the
# build like turning off the System V "Red Zone" optimization, because
# αcτµαlly pδrταblε εxεcµταblεs need to be able to run in kernelspace.
PKGS += APE
APE_FILES := $(wildcard ape/*.*)
APE_HDRS = $(filter %.h,$(APE_FILES))
APE_INCS = $(filter %.inc,$(APE_FILES))
ifeq ($(ARCH), aarch64)
APE = o/$(MODE)/ape/aarch64.lds
APELINK = \
$(COMPILE) \
-ALINK.ape \
$(LINK) \
$(LINKARGS) \
$(OUTPUT_OPTION) && \
$(COMPILE) \
-AFIXUP.ape \
-wT$@ \
$(FIXUPOBJ) \
$@
APE_SRCS = ape/ape.S ape/start.S ape/launch.S ape/systemcall.S
APE_OBJS = o/$(MODE)/ape/ape.o
APE_NO_MODIFY_SELF = $(APE)
APE_COPY_SELF = $(APE)
.PHONY: o/$(MODE)/ape
o/$(MODE)/ape: $(APE)
o/$(MODE)/ape/aarch64.lds: \
ape/aarch64.lds \
libc/zip.internal.h \
libc/intrin/bits.h \
libc/thread/tls.h \
libc/calls/struct/timespec.h \
libc/macros.internal.h \
libc/str/str.h
APE_LOADER_LDFLAGS = \
-pie \
-static \
-nostdlib \
--no-dynamic-linker \
-z norelro \
-z common-page-size=0x4000 \
-z max-page-size=0x4000
APE_LOADER_FLAGS = \
-DNDEBUG \
-iquote. \
-Wall \
-Wextra \
-fpie \
-Os \
-ffreestanding \
-mgeneral-regs-only \
-fno-asynchronous-unwind-tables \
-fno-stack-protector \
-fno-ident \
-fno-gnu-unique \
-c \
$(OUTPUT_OPTION) \
$<
o/$(MODE)/ape/ape.elf: o/$(MODE)/ape/ape.elf.dbg
$(COMPILE) -AOBJCOPY -T$@ $(OBJCOPY) -g $< $@
o/$(MODE)/ape/ape.elf.dbg: \
o/$(MODE)/ape/start.o \
o/$(MODE)/ape/loader.o \
o/$(MODE)/ape/launch.o \
o/$(MODE)/ape/systemcall.o
@$(COMPILE) -ALINK.elf $(LD) $(APE_LOADER_LDFLAGS) -o $@ $(patsubst %.lds,-T %.lds,$^)
o/$(MODE)/ape/loader.o: ape/loader.c ape/ape.h
@$(COMPILE) -AOBJECTIFY.c $(CC) -DSUPPORT_VECTOR=1 -g $(APE_LOADER_FLAGS)
o/$(MODE)/ape/start.o: ape/start.S
@$(COMPILE) -AOBJECTIFY.S $(OBJECTIFY.S) $(OUTPUT_OPTION) -c $<
o/$(MODE)/ape/launch.o: ape/launch.S
@$(COMPILE) -AOBJECTIFY.S $(OBJECTIFY.S) $(OUTPUT_OPTION) -c $<
o/$(MODE)/ape/systemcall.o: ape/systemcall.S
@$(COMPILE) -AOBJECTIFY.S $(OBJECTIFY.S) $(OUTPUT_OPTION) -c $<
.PHONY: o/$(MODE)/ape
o/$(MODE)/ape: o/$(MODE)/ape/ape.elf
else
APE = o/$(MODE)/ape/ape.o \
o/$(MODE)/ape/ape.lds
APELINK = \
$(COMPILE) \
-ALINK.ape \
$(LINK) \
$(LINKARGS) \
$(OUTPUT_OPTION) && \
$(COMPILE) \
-AFIXUP.ape \
-wT$@ \
$(FIXUPOBJ) \
$@
APE_NO_MODIFY_SELF = \
o/$(MODE)/ape/ape.lds \
o/$(MODE)/ape/ape-no-modify-self.o
APE_COPY_SELF = \
o/$(MODE)/ape/ape.lds \
o/$(MODE)/ape/ape-copy-self.o
APE_LOADER_FLAGS = \
-DNDEBUG \
-iquote. \
-Wall \
-Wextra \
-Werror \
-pedantic-errors \
-fpie \
-Os \
-ffreestanding \
-mgeneral-regs-only \
-fno-stack-protector \
-fno-ident \
-fno-gnu-unique \
-c \
$(OUTPUT_OPTION) \
$<
APE_SRCS_C = ape/loader.c
APE_SRCS_S = $(filter %.S,$(APE_FILES))
APE_SRCS = $(APE_SRCS_C) $(APE_SRCS_S)
APE_OBJS = $(APE_SRCS_S:%.S=o/$(MODE)/%.o)
APE_CHECKS = $(APE_HDRS:%=o/%.ok)
o/ape/idata.inc: \
ape/idata.internal.h \
ape/relocations.h
o/$(MODE)/ape/ape-no-modify-self.o: \
ape/ape.S \
ape/ape.h \
ape/macros.internal.h \
ape/notice.inc \
ape/relocations.h \
ape/ape.internal.h \
libc/dce.h \
libc/elf/def.h \
libc/thread/tls.h \
libc/intrin/asancodes.h \
libc/macho.internal.h \
libc/macros.internal.h \
libc/nexgen32e/uart.internal.h \
libc/calls/metalfile.internal.h \
libc/nexgen32e/vidya.internal.h \
libc/nt/pedef.internal.h \
libc/runtime/e820.internal.h \
libc/runtime/mman.internal.h \
libc/runtime/pc.internal.h \
libc/sysv/consts/prot.h \
ape/blink-linux-aarch64.gz \
ape/blink-xnu-aarch64.gz \
o/$(MODE)/ape/ape.elf
@$(COMPILE) \
-AOBJECTIFY.S \
$(OBJECTIFY.S) \
$(OUTPUT_OPTION) \
-DAPE_NO_MODIFY_SELF \
-DAPE_LOADER='"o/$(MODE)/ape/ape.elf"' $<
o/$(MODE)/ape/ape-copy-self.o: \
ape/ape.S \
ape/ape.h \
ape/macros.internal.h \
ape/notice.inc \
ape/relocations.h \
ape/ape.internal.h \
libc/dce.h \
libc/elf/def.h \
libc/thread/tls.h \
libc/intrin/asancodes.h \
libc/macho.internal.h \
libc/macros.internal.h \
libc/nexgen32e/uart.internal.h \
libc/calls/metalfile.internal.h \
libc/nexgen32e/vidya.internal.h \
libc/nt/pedef.internal.h \
libc/runtime/e820.internal.h \
libc/runtime/mman.internal.h \
libc/runtime/pc.internal.h \
libc/sysv/consts/prot.h \
ape/blink-linux-aarch64.gz \
ape/blink-xnu-aarch64.gz
@$(COMPILE) \
-AOBJECTIFY.S \
$(OBJECTIFY.S) \
$(OUTPUT_OPTION) \
-DAPE_NO_MODIFY_SELF $<
o/$(MODE)/ape/loader.o: ape/loader.c
@$(COMPILE) -AOBJECTIFY.c $(CC) -DSUPPORT_VECTOR=121 -g $(APE_LOADER_FLAGS)
o/$(MODE)/ape/loader-gcc.asm: ape/loader.c
@$(COMPILE) -AOBJECTIFY.c $(CC) -DSUPPORT_VECTOR=121 -S -g0 $(APE_LOADER_FLAGS)
o/$(MODE)/ape/loader-clang.asm: ape/loader.c
@$(COMPILE) -AOBJECTIFY.c $(CLANG) -DSUPPORT_VECTOR=121 -S -g0 $(APE_LOADER_FLAGS)
o/$(MODE)/ape/loader-xnu.o: ape/loader.c
@$(COMPILE) -AOBJECTIFY.c $(CC) -DSUPPORT_VECTOR=8 -g $(APE_LOADER_FLAGS)
o/$(MODE)/ape/loader-xnu-gcc.asm: ape/loader.c
@$(COMPILE) -AOBJECTIFY.c $(CC) -DSUPPORT_VECTOR=8 -S -g0 $(APE_LOADER_FLAGS)
o/$(MODE)/ape/loader-xnu-clang.asm: ape/loader.c
@$(COMPILE) -AOBJECTIFY.c $(CLANG) -DSUPPORT_VECTOR=8 -S -g0 $(APE_LOADER_FLAGS)
o/$(MODE)/ape/ape.elf: o/$(MODE)/ape/ape.elf.dbg
@$(COMPILE) -AOBJBINCOPY -w build/bootstrap/objbincopy.com -f -o $@ $<
o/$(MODE)/ape/ape.macho: o/$(MODE)/ape/ape.elf.dbg
@$(COMPILE) -AOBJBINCOPY -w build/bootstrap/objbincopy.com -fm -o $@ $<
APE_LOADER_LDFLAGS = \
-static \
-nostdlib \
--no-dynamic-linker \
-z separate-code \
-z common-page-size=0x1000 \
-z max-page-size=0x10000
o/$(MODE)/ape/ape.elf.dbg: \
o/$(MODE)/ape/loader-macho.o \
o/$(MODE)/ape/start.o \
o/$(MODE)/ape/loader.o \
o/$(MODE)/ape/launch.o \
o/$(MODE)/ape/systemcall.o \
ape/loader.lds
@$(COMPILE) -ALINK.elf $(LD) $(APE_LOADER_LDFLAGS) -o $@ $(patsubst %.lds,-T %.lds,$^)
.PHONY: o/$(MODE)/ape
o/$(MODE)/ape: $(APE_CHECKS) \
o/$(MODE)/ape/ape.o \
o/$(MODE)/ape/ape.lds \
o/$(MODE)/ape/ape.elf \
o/$(MODE)/ape/ape.macho \
o/$(MODE)/ape/ape-copy-self.o \
o/$(MODE)/ape/ape-no-modify-self.o
endif
# these assembly files are safe to build on aarch64
o/$(MODE)/ape/ape.o: ape/ape.S
@$(COMPILE) -AOBJECTIFY.S $(OBJECTIFY.S) $(OUTPUT_OPTION) -c $<
o/$(MODE)/ape/ape.o: \
ape/blink-linux-aarch64.gz \
ape/blink-xnu-aarch64.gz
o/$(MODE)/ape/ape.lds: \
ape/ape.lds \
ape/macros.internal.h \
ape/relocations.h \
ape/ape.internal.h \
libc/intrin/bits.h \
libc/thread/tls.h \
libc/calls/struct/timespec.h \
libc/thread/thread.h \
libc/dce.h \
libc/elf/def.h \
libc/elf/pf2prot.internal.h \
libc/macros.internal.h \
libc/nt/pedef.internal.h \
libc/str/str.h \
libc/zip.internal.h
-1
View File
@@ -1 +0,0 @@
../bin/ape-install
+180
View File
@@ -0,0 +1,180 @@
#!/bin/sh
PROG=${0##*/}
MODE=${MODE:-$m}
TMPDIR=${TMPDIR:-/tmp}
COSMO=${COSMO:-/opt/cosmo}
COSMOS=${COSMOS:-/opt/cosmos}
if [ ! -f ape/loader.c ]; then
cd "$COSMO" || exit
fi
if [ -x build/bootstrap/make ]; then
MAKE=build/bootstrap/make
else
MAKE=make
fi
if [ "$(id -u)" -eq 0 ]; then
SUDO=
elif command -v sudo >/dev/null 2>&1; then
SUDO=sudo
elif command -v doas >/dev/null 2>&1; then
SUDO=doas
else
echo "need root or sudo" >&2
exit
fi
if command -v install >/dev/null 2>&1; then
if [ x"$(uname -s)" = xLinux ]; then
INSTALL="install -o root -g root -m 755"
else
INSTALL="install -o root -g wheel -m 755"
fi
else
INSTALL="cp -f"
fi
echo "Actually Portable Executable (APE) Installer" >&2
echo "Author: Justine Tunney <[email protected]>" >&2
# special installation process for apple silicon
if [ x"$(uname -s)" = xDarwin ] && [ x"$(uname -m)" = xarm64 ]; then
echo "cc -O -o $TMPDIR/ape.$$ ape/ape-m1.c" >&2
cc -O -o "$TMPDIR/ape.$$" ape/ape-m1.c || exit
trap 'rm "$TMPDIR/ape.$$"' EXIT
if [ ! -d /usr/local/bin ]; then
echo "$SUDO mkdir -p /usr/local/bin" >&2
$SUDO mkdir -p /usr/local/bin || exit
fi
echo "$SUDO $INSTALL $TMPDIR/ape.$$ /usr/local/bin/ape" >&2
$SUDO $INSTALL "$TMPDIR/ape.$$" /usr/local/bin/ape || exit
exit
fi
if [ x"$(uname -m)" = xarm64 ] || [ x"$(uname -m)" = xaarch64 ]; then
MODE=aarch64
EXT=elf
BEXT=aarch64
elif [ x"$(uname -m)" = xx86_64 ]; then
MODE=
if [ x"$(uname -s)" = xDarwin ]; then
EXT=macho
else
EXT=elf
fi
BEXT=$EXT
else
echo "unsupported architecture $(uname -m)" >&2
exit
fi
################################################################################
# INSTALL APE LOADER SYSTEMWIDE
if [ -f o/$MODE/depend ] && $MAKE -j8 o/$MODE/ape; then
echo "successfully recompiled ape loader" >&2
elif [ -x o/$MODE/ape/ape.$EXT ]; then
echo "using ape loader you compiled earlier" >&2
elif [ -d build/bootstrap ]; then
# if make isn't being used then it's unlikely the user changed the sources
# in that case the prebuilt binaries should be completely up-to-date
echo "using prebuilt ape loader from cosmo repo" >&2
mkdir -p o/$MODE/ape || exit
cp -af build/bootstrap/ape.$BEXT o/$MODE/ape/ape.$EXT || exit
else
echo "no cosmopolitan libc repository here" >&2
echo "fetching ape loader from justine.lol" >&2
mkdir -p o/$MODE/ape || exit
if command -v wget >/dev/null 2>&1; then
wget -qO o/$MODE/ape/ape.$EXT https://justine.lol/ape.$BEXT || exit
else
curl -Rso o/$MODE/ape/ape.$EXT https://justine.lol/ape.$BEXT || exit
fi
chmod +x o/$MODE/ape/ape.$EXT || exit
fi
if ! [ /usr/bin/ape -nt o/$MODE/ape/ape.$EXT ]; then
echo >&2
echo "installing o/$MODE/ape/ape.$EXT to /usr/bin/ape" >&2
echo "$SUDO $INSTALL o/$MODE/ape/ape.$EXT /usr/bin/ape" >&2
$SUDO $INSTALL o/$MODE/ape/ape.$EXT /usr/bin/ape || exit
echo "done" >&2
fi
################################################################################
# REGISTER APE LOADER WITH BINFMT_MISC TOO (LINUX-ONLY)
if [ x"$(uname -s)" = xLinux ]; then
if [ -e /proc/sys/fs/binfmt_misc/APE ]; then
echo >&2
echo it looks like APE is already registered with binfmt_misc >&2
echo To reinstall please run ape/apeuninstall.sh first >&2
echo please check that it is mapped to ape not /bin/sh >&2
echo cat /proc/sys/fs/binfmt_misc/APE >&2
cat /proc/sys/fs/binfmt_misc/APE >&2
exit
fi
if ! [ -e /proc/sys/fs/binfmt_misc ]; then
echo >&2
echo loading binfmt_misc into your kernel >&2
echo you may need to edit configs to persist across reboot >&2
echo $SUDO modprobe binfmt_misc >&2
$SUDO modprobe binfmt_misc || exit
echo done >&2
fi
if ! [ -e /proc/sys/fs/binfmt_misc/register ]; then
echo >&2
echo mounting binfmt_misc into your kernel >&2
echo you may need to edit configs to persist across reboot >&2
echo $SUDO mount -t binfmt_misc none /proc/sys/fs/binfmt_misc >&2
$SUDO mount -t binfmt_misc none /proc/sys/fs/binfmt_misc || exit
echo done >&2
fi
uname_r="$(uname -r)"
if printf '%s\n%s\n' 5.12 "$uname_r" | sort -CV; then
FLAGS=FP
else
FLAGS=F
fi
echo >&2
echo registering APE with binfmt_misc >&2
echo you may need to edit configs to persist across reboot >&2
echo '$SUDO sh -c "echo '"'"':APE:M::MZqFpD::/usr/bin/ape:'"$FLAGS'"' >/proc/sys/fs/binfmt_misc/register"' >&2
$SUDO sh -c "echo ':APE:M::MZqFpD::/usr/bin/ape:$FLAGS' >/proc/sys/fs/binfmt_misc/register" || exit
echo '$SUDO sh -c "echo '"'"':APE-jart:M::jartsr::/usr/bin/ape:'"$FLAGS'"' >/proc/sys/fs/binfmt_misc/register"' >&2
$SUDO sh -c "echo ':APE-jart:M::jartsr::/usr/bin/ape:$FLAGS' >/proc/sys/fs/binfmt_misc/register" || exit
echo done >&2
if [ x"$(cat /proc/sys/fs/binfmt_misc/status)" = xdisabled ]; then
echo >&2
echo enabling binfmt_misc >&2
echo you may need to edit configs to persist across reboot >&2
echo $SUDO sh -c 'echo 1 >/proc/sys/fs/binfmt_misc/status' >&2
$SUDO sh -c 'echo 1 >/proc/sys/fs/binfmt_misc/status' || exit
echo done >&2
fi
fi
################################################################################
{
echo
echo "------------------------------------------------------------------"
echo
echo "APE INSTALL COMPLETE"
echo
echo "If you decide to uninstall APE later on"
echo "you may do so using ape/apeuninstall.sh"
echo
echo "Enjoy your APE loader (>'.')>"
echo
} >&2
-1
View File
@@ -1 +0,0 @@
../bin/ape-uninstall
+68
View File
@@ -0,0 +1,68 @@
#!/bin/sh
PROG=${0##*/}
MODE=${MODE:-$m}
COSMO=${COSMO:-/opt/cosmo}
COSMOS=${COSMOS:-/opt/cosmos}
if [ ! -f ape/loader.c ]; then
cd "$COSMO" || exit
fi
if [ "$UID" = "0" ]; then
SUDO=
elif command -v sudo >/dev/null 2>&1; then
SUDO=sudo
elif command -v doas >/dev/null 2>&1; then
SUDO=doas
else
echo "need root or sudo" >&2
exit
fi
{
echo
echo "APE Uninstaller intends to run (in pseudo-shell)"
echo
echo " sudo echo -1 into /proc/sys/fs/binfmt_misc/APE*"
echo " sudo rm -f /usr/bin/ape ~/.ape /tmp/.ape # etc."
echo
echo "You may then use ape/apeinstall.sh to reinstall it"
echo
} >&2
set -ex
for f in /proc/sys/fs/binfmt_misc/APE*; do
if [ -f $f ]; then
$SUDO sh -c "echo -1 >$f" || exit
fi
done
# system installation
if [ -f /usr/bin/ape ]; then
$SUDO rm -f /usr/bin/ape
fi
if [ -f /usr/local/bin/ape ]; then
$SUDO rm -f /usr/local/bin/ape
fi
# legacy installations
rm -f o/tmp/ape /tmp/ape "${TMPDIR:-/tmp}/ape"
# ad-hoc installations
for x in .ape \
.ape-1.1 \
.ape-1.3 \
.ape-1.4 \
.ape-1.5 \
.ape-1.6 \
.ape-1.7 \
.ape-1.8 \
.ape-1.9 \
.ape-1.10; do
rm -f \
~/$x \
/tmp/$x \
o/tmp/$x \
"${TMPDIR:-/tmp}/$x"
done
Binary file not shown.
Binary file not shown.
+1 -1
View File
@@ -1,5 +1,5 @@
/*-*- mode:unix-assembly; indent-tabs-mode:t; tab-width:8; coding:utf-8 -*-│
│vi: set et ft=asm ts=8 sw=8 fenc=utf-8 :vi│
│ vi: set noet ft=asm ts=8 sw=8 fenc=utf-8 :vi │
╞══════════════════════════════════════════════════════════════════════════════╡
│ Copyright 2020 Justine Alexandra Roberts Tunney │
│ │
+6 -8
View File
@@ -1,5 +1,5 @@
/*-*- mode:unix-assembly; indent-tabs-mode:t; tab-width:8; coding:utf-8 -*-│
│vi: set et ft=asm ts=8 tw=8 fenc=utf-8 :vi│
│ vi: set noet ft=asm ts=8 sw=8 fenc=utf-8 :vi │
╞══════════════════════════════════════════════════════════════════════════════╡
│ Copyright 2023 Justine Alexandra Roberts Tunney │
│ │
@@ -16,7 +16,7 @@
│ TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR │
│ PERFORMANCE OF THIS SOFTWARE. │
╚─────────────────────────────────────────────────────────────────────────────*/
#include "libc/macros.internal.h"
#include "libc/macros.h"
// Calls _start() function of loaded program.
//
@@ -31,17 +31,16 @@
//
// @param rdi is passed through as-is
// @param rsi is address of entrypoint (becomes zero)
// @param rdx is stack pointer (becomes zero)
// @param rdx is passed through as-is
// @param rcx is passed through as-is
// @param r8 is stack pointer (becomes zero)
// @noreturn
Launch:
#ifdef __aarch64__
mov x16,x1
mov sp,x2
mov sp,x4
mov x1,0
mov x2,0
mov x3,0
mov x4,0
mov x5,0
mov x6,0
@@ -71,6 +70,7 @@ Launch:
#else
mov %r8,%rsp
xor %r8d,%r8d
xor %r9d,%r9d
xor %r10d,%r10d
@@ -79,8 +79,6 @@ Launch:
xor %r13d,%r13d
xor %r14d,%r14d
xor %r15d,%r15d
mov %rdx,%rsp
xor %edx,%edx
push %rsi
xor %esi,%esi
xor %ebp,%ebp
+4 -4
View File
@@ -1,5 +1,5 @@
/*-*- mode:unix-assembly; indent-tabs-mode:t; tab-width:8; coding:utf-8 -*-│
│vi: set et ft=asm ts=8 tw=8 fenc=utf-8 :vi│
│ vi: set noet ft=asm ts=8 sw=8 fenc=utf-8 :vi │
╞══════════════════════════════════════════════════════════════════════════════╡
│ Copyright 2021 Justine Alexandra Roberts Tunney │
│ │
@@ -16,13 +16,13 @@
│ TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR │
│ PERFORMANCE OF THIS SOFTWARE. │
╚─────────────────────────────────────────────────────────────────────────────*/
#include "libc/macho.internal.h"
#include "libc/macho.h"
#include "libc/sysv/consts/prot.h"
#include "libc/dce.h"
#include "libc/macros.internal.h"
#include "libc/macros.h"
// Apple Mach-O Executable Headers
// Fixups are applied by objbincopy.com
// Fixups are applied by objbincopy
// There must exist a MAC_LC_SEGMENT_64 for every PT_LOAD
.section .macho,"a",@progbits
+201 -157
View File
@@ -1,5 +1,5 @@
/*-*- mode:c;indent-tabs-mode:nil;c-basic-offset:2;tab-width:8;coding:utf-8 -*-│
│vi: set net ft=c ts=2 sts=2 sw=2 fenc=utf-8 :vi│
│ vi: set et ft=c ts=2 sts=2 sw=2 fenc=utf-8 :vi │
╞══════════════════════════════════════════════════════════════════════════════╡
│ Copyright 2021 Justine Alexandra Roberts Tunney │
│ │
@@ -87,6 +87,8 @@
#define MIN(X, Y) ((Y) > (X) ? (X) : (Y))
#define MAX(X, Y) ((Y) < (X) ? (X) : (Y))
#define PATH_MAX 1024 /* XXX verify */
#define SupportsLinux() (SUPPORT_VECTOR & LINUX)
#define SupportsXnu() (SUPPORT_VECTOR & XNU)
#define SupportsFreebsd() (SUPPORT_VECTOR & FREEBSD)
@@ -111,41 +113,47 @@
#define EXTERN_C
#endif
#define O_RDONLY 0
#define PROT_NONE 0
#define PROT_READ 1
#define PROT_WRITE 2
#define PROT_EXEC 4
#define MAP_SHARED 1
#define MAP_PRIVATE 2
#define MAP_FIXED 16
#define MAP_ANONYMOUS (IsLinux() ? 32 : 4096)
#define MAP_NORESERVE (IsLinux() ? 16384 : 0)
#define ELFCLASS32 1
#define ELFDATA2LSB 1
#define EM_NEXGEN32E 62
#define EM_AARCH64 183
#define ET_EXEC 2
#define ET_DYN 3
#define PT_LOAD 1
#define PT_DYNAMIC 2
#define PT_INTERP 3
#define EI_CLASS 4
#define EI_DATA 5
#define PF_X 1
#define PF_W 2
#define PF_R 4
#define AT_PHDR 3
#define AT_PHENT 4
#define AT_PHNUM 5
#define AT_PAGESZ 6
#define AT_EXECFN_LINUX 31
#define AT_EXECFN_NETBSD 2014
#define X_OK 1
#define XCR0_SSE 2
#define XCR0_AVX 4
#define PR_SET_MM 35
#define PR_SET_MM_EXE_FILE 13
#define O_RDONLY 0
#define PROT_NONE 0
#define PROT_READ 1
#define PROT_WRITE 2
#define PROT_EXEC 4
#define MAP_SHARED 1
#define MAP_PRIVATE 2
#define MAP_FIXED 16
#define MAP_ANONYMOUS (IsLinux() ? 32 : 4096)
#define MAP_NORESERVE (IsLinux() ? 16384 : 0)
#define ELFCLASS32 1
#define ELFDATA2LSB 1
#define EM_NEXGEN32E 62
#define EM_AARCH64 183
#define ET_EXEC 2
#define ET_DYN 3
#define PT_LOAD 1
#define PT_DYNAMIC 2
#define PT_INTERP 3
#define EI_CLASS 4
#define EI_DATA 5
#define PF_X 1
#define PF_W 2
#define PF_R 4
#define AT_PHDR 3
#define AT_PHENT 4
#define AT_PHNUM 5
#define AT_PAGESZ 6
#define AT_FLAGS 8
#define AT_FLAGS_PRESERVE_ARGV0_BIT 0
#define AT_FLAGS_PRESERVE_ARGV0 (1 << AT_FLAGS_PRESERVE_ARGV0_BIT)
#define AT_EXECFN_LINUX 31
#define AT_EXECFN_NETBSD 2014
#define X_OK 1
#define XCR0_SSE 2
#define XCR0_AVX 4
#define PR_SET_MM 35
#define PR_SET_MM_EXE_FILE 13
#define EF_APE_MODERN 0x101ca75
#define EF_APE_MODERN_MASK 0x1ffffff
#define READ32(S) \
((unsigned)(255 & (S)[3]) << 030 | (unsigned)(255 & (S)[2]) << 020 | \
@@ -161,13 +169,6 @@
(unsigned long)(255 & (S)[1]) << 010 | \
(unsigned long)(255 & (S)[0]) << 000)
#define DEBUG(VAR) \
{ \
char ibuf[19] = {0}; \
Utox(ibuf, VAR); \
Print(os, 2, ibuf, " " #VAR, "\n", 0l); \
}
struct ElfEhdr {
unsigned char e_ident[16];
unsigned short e_type;
@@ -212,39 +213,43 @@ struct PathSearcher {
const char *name;
const char *syspath;
unsigned long namelen;
char path[1024];
char path[PATH_MAX];
};
struct ApeLoader {
union ElfPhdrBuf phdr;
struct PathSearcher ps;
char path[1024];
char path[PATH_MAX];
};
EXTERN_C long SystemCall(long, long, long, long, long, long, long, int);
EXTERN_C void Launch(void *, long, void *, int) __attribute__((__noreturn__));
EXTERN_C void Launch(void *, long, void *, int, void *)
__attribute__((__noreturn__));
extern char __executable_start[];
extern char _end[];
static unsigned long StrLen(const char *s) {
unsigned long n = 0;
while (*s++) ++n;
while (*s++)
++n;
return n;
}
static int StrCmp(const char *l, const char *r) {
unsigned long i = 0;
while (l[i] == r[i] && r[i]) ++i;
while (l[i] == r[i] && r[i])
++i;
return (l[i] & 255) - (r[i] & 255);
}
static const char *BaseName(const char *s) {
int c;
const char *b = "";
#if 0
static const char *StrRChr(const char *s, int c) {
const char *b = 0;
if (s) {
while ((c = *s++)) {
if (c == '/') {
for (; *s; ++s) {
if (*s == c) {
b = s;
}
}
@@ -252,6 +257,13 @@ static const char *BaseName(const char *s) {
return b;
}
static const char *BaseName(const char *s) {
const char *b = StrRChr(s, '/');
return b ? b + 1 : s;
}
#endif
static void Bzero(void *a, unsigned long n) {
long z;
char *p, *e;
@@ -326,24 +338,7 @@ static char *GetEnv(char **p, const char *s) {
return 0;
}
static char *Utox(char p[19], unsigned long x) {
int i;
if (x) {
*p++ = '0';
*p++ = 'x';
i = (__builtin_clzl(x) ^ (sizeof(long) * 8 - 1)) + 1;
i = (i + 3) & -4;
do {
*p++ = "0123456789abcdef"[(x >> (i -= 4)) & 15];
} while (i);
} else {
*p++ = '0';
}
*p = 0;
return p;
}
static char *Utoa(char p[21], unsigned long x) {
static char *Utoa(char p[20], unsigned long x) {
char t;
unsigned long i, a, b;
i = 0;
@@ -363,7 +358,8 @@ static char *Utoa(char p[21], unsigned long x) {
}
static char *Itoa(char p[21], long x) {
if (x < 0) *p++ = '-', x = -(unsigned long)x;
if (x < 0)
*p++ = '-', x = -(unsigned long)x;
return Utoa(p, x);
}
@@ -372,7 +368,8 @@ __attribute__((__noinline__)) static long CallSystem(long arg1, long arg2,
long arg5, long arg6,
long arg7, int numba,
char os) {
if (IsXnu()) numba |= 0x2000000;
if (IsXnu())
numba |= 0x2000000;
return SystemCall(arg1, arg2, arg3, arg4, arg5, arg6, arg7, numba);
}
@@ -520,10 +517,62 @@ static long Print(int os, int fd, const char *s, ...) {
return Write(fd, b, n, os);
}
static long Printf(int os, int fd, const char *fmt, ...) {
int i;
char c;
int k = 0;
unsigned u;
char b[512];
const char *s;
unsigned long d;
__builtin_va_list va;
__builtin_va_start(va, fmt);
for (;;) {
switch ((c = *fmt++)) {
case '\0':
__builtin_va_end(va);
return Write(fd, b, k, os);
case '%':
switch ((c = *fmt++)) {
case 's':
for (s = __builtin_va_arg(va, const char *); s && *s; ++s) {
if (k < 512)
b[k++] = *s;
}
break;
case 'd':
d = __builtin_va_arg(va, unsigned long);
for (i = 16; i--;) {
u = (d >> (i * 4)) & 15;
if (u < 10) {
c = '0' + u;
} else {
u -= 10;
c = 'a' + u;
}
if (k < 512)
b[k++] = c;
}
break;
default:
if (k < 512)
b[k++] = c;
break;
}
break;
default:
if (k < 512)
b[k++] = c;
break;
}
}
}
static void Perror(int os, const char *thing, long rc, const char *reason) {
char ibuf[21];
ibuf[0] = 0;
if (rc) Itoa(ibuf, -rc);
if (rc)
Itoa(ibuf, -rc);
Print(os, 2, "ape error: ", thing, ": ", reason,
rc ? " failed w/ errno " : "", ibuf, "\n", 0l);
}
@@ -535,8 +584,10 @@ __attribute__((__noreturn__)) static void Pexit(int os, const char *c, int rc,
}
static char AccessCommand(struct PathSearcher *ps, unsigned long pathlen) {
if (pathlen + 1 + ps->namelen + 1 > sizeof(ps->path)) return 0;
if (pathlen && ps->path[pathlen - 1] != '/') ps->path[pathlen++] = '/';
if (pathlen + 1 + ps->namelen + 1 > sizeof(ps->path))
return 0;
if (pathlen && ps->path[pathlen - 1] != '/')
ps->path[pathlen++] = '/';
MemMove(ps->path + pathlen, ps->name, ps->namelen);
ps->path[pathlen + ps->namelen] = 0;
return !Access(ps->path, X_OK, ps->os);
@@ -561,45 +612,25 @@ static char SearchPath(struct PathSearcher *ps) {
}
}
static char FindCommand(struct PathSearcher *ps) {
ps->path[0] = 0;
/* paths are always 100% taken literally when a slash exists
$ ape foo/bar.com arg1 arg2 */
if (MemChr(ps->name, '/', ps->namelen)) {
return AccessCommand(ps, 0);
}
/* we don't run files in the current directory
$ ape foo.com arg1 arg2
unless $PATH has an empty string entry, e.g.
$ expert PATH=":/bin"
$ ape foo.com arg1 arg2
however we will execute this
$ ape - foo.com foo.com arg1 arg2
because cosmo's execve needs it */
if (ps->literally && AccessCommand(ps, 0)) {
return 1;
}
/* otherwise search for name on $PATH */
return SearchPath(ps);
}
static char *Commandv(struct PathSearcher *ps, int os, const char *name,
static char *Commandv(struct PathSearcher *ps, int os, char *name,
const char *syspath) {
if (!(ps->namelen = StrLen((ps->name = name))))
return 0;
if (ps->literally || MemChr(ps->name, '/', ps->namelen))
return name;
ps->os = os;
ps->syspath = syspath ? syspath : "/bin:/usr/local/bin:/usr/bin";
if (!(ps->namelen = StrLen((ps->name = name)))) return 0;
if (ps->namelen + 1 > sizeof(ps->path)) return 0;
if (FindCommand(ps)) {
if (ps->namelen + 1 > sizeof(ps->path))
return 0;
ps->path[0] = 0;
if (SearchPath(ps)) {
return ps->path;
} else {
return 0;
}
}
__attribute__((__noreturn__)) static void Spawn(int os, const char *exe, int fd,
__attribute__((__noreturn__)) static void Spawn(int os, char *exe, int fd,
long *sp, unsigned long pagesz,
struct ElfEhdr *e,
struct ElfPhdr *p) {
@@ -647,7 +678,8 @@ __attribute__((__noreturn__)) static void Spawn(int os, const char *exe, int fd,
Pexit(os, exe, 0, "ELF segments overlap your APE loader");
}
for (j = i + 1; j < e->e_phnum; ++j) {
if (p[j].p_type != PT_LOAD) continue;
if (p[j].p_type != PT_LOAD)
continue;
c = p[j].p_vaddr & -pagesz;
d = (p[j].p_vaddr + p[j].p_memsz + (pagesz - 1)) & -pagesz;
if (MAX(a, c) < MIN(b, d)) {
@@ -680,7 +712,8 @@ __attribute__((__noreturn__)) static void Spawn(int os, const char *exe, int fd,
if (e->e_type == ET_DYN) {
rc = Mmap(0, virtmax - virtmin, PROT_NONE,
MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0, os);
if (rc < 0) Pexit(os, exe, rc, "pie mmap");
if (rc < 0)
Pexit(os, exe, rc, "pie mmap");
dynbase = rc;
if (dynbase & (pagesz - 1)) {
Pexit(os, exe, 0, "OS mmap incongruent w/ AT_PAGESZ");
@@ -696,14 +729,18 @@ __attribute__((__noreturn__)) static void Spawn(int os, const char *exe, int fd,
for (i = 0; i < e->e_phnum; ++i) {
void *addr;
unsigned long size;
if (p[i].p_type != PT_LOAD) continue;
if (p[i].p_type != PT_LOAD)
continue;
/* configure mapping */
prot = 0;
flags = MAP_FIXED | MAP_PRIVATE;
if (p[i].p_flags & PF_R) prot |= PROT_READ;
if (p[i].p_flags & PF_W) prot |= PROT_WRITE;
if (p[i].p_flags & PF_X) prot |= PROT_EXEC;
if (p[i].p_flags & PF_R)
prot |= PROT_READ;
if (p[i].p_flags & PF_W)
prot |= PROT_WRITE;
if (p[i].p_flags & PF_X)
prot |= PROT_EXEC;
if (p[i].p_filesz) {
/* load from file */
@@ -730,17 +767,21 @@ __attribute__((__noreturn__)) static void Spawn(int os, const char *exe, int fd,
addr = (void *)(dynbase + (p[i].p_vaddr & -pagesz));
size = (p[i].p_vaddr & (pagesz - 1)) + p[i].p_filesz;
rc = Mmap(addr, size, prot1, flags, fd, p[i].p_offset & -pagesz, os);
if (rc < 0) Pexit(os, exe, rc, "prog mmap");
if (wipe) Bzero((void *)(dynbase + a), wipe);
if (rc < 0)
Pexit(os, exe, rc, "prog mmap");
if (wipe)
Bzero((void *)(dynbase + a), wipe);
if (prot2 != prot1) {
rc = Mprotect(addr, size, prot2, os);
if (rc < 0) Pexit(os, exe, rc, "prog mprotect");
if (rc < 0)
Pexit(os, exe, rc, "prog mprotect");
}
/* allocate extra bss */
if (c > b) {
flags |= MAP_ANONYMOUS;
rc = Mmap((void *)(dynbase + b), c - b, prot, flags, -1, 0, os);
if (rc < 0) Pexit(os, exe, rc, "extra bss mmap");
if (rc < 0)
Pexit(os, exe, rc, "extra bss mmap");
}
} else {
/* allocate pure bss */
@@ -748,7 +789,8 @@ __attribute__((__noreturn__)) static void Spawn(int os, const char *exe, int fd,
size = (p[i].p_vaddr & (pagesz - 1)) + p[i].p_memsz;
flags |= MAP_ANONYMOUS;
rc = Mmap(addr, size, prot, flags, -1, 0, os);
if (rc < 0) Pexit(os, exe, rc, "bss mmap");
if (rc < 0)
Pexit(os, exe, rc, "bss mmap");
}
}
@@ -757,11 +799,11 @@ __attribute__((__noreturn__)) static void Spawn(int os, const char *exe, int fd,
Msyscall(dynbase + code, codesize, os);
/* call program entrypoint */
Launch(IsFreebsd() ? sp : 0, dynbase + e->e_entry, sp, os);
Launch(IsFreebsd() ? sp : 0, dynbase + e->e_entry, exe, os, sp);
}
static const char *TryElf(struct ApeLoader *M, union ElfEhdrBuf *ebuf,
const char *exe, int fd, long *sp, long *auxv,
char *exe, int fd, long *sp, long *auxv,
unsigned long pagesz, int os) {
long i, rc;
unsigned size;
@@ -769,7 +811,8 @@ static const char *TryElf(struct ApeLoader *M, union ElfEhdrBuf *ebuf,
struct ElfPhdr *p;
/* validate page size */
if (!pagesz) pagesz = 4096;
if (!pagesz)
pagesz = 4096;
if (pagesz & (pagesz - 1)) {
Pexit(os, exe, 0, "AT_PAGESZ isn't two power");
}
@@ -794,6 +837,10 @@ static const char *TryElf(struct ApeLoader *M, union ElfEhdrBuf *ebuf,
return "couldn't find ELF header with x86-64 machine type";
}
#endif
if ((e->e_flags & EF_APE_MODERN_MASK) != EF_APE_MODERN && sp[0] > 0) {
/* change argv[0] to resolved path for older binaries */
((char **)(sp + 1))[0] = exe;
}
if (e->e_phentsize != sizeof(struct ElfPhdr)) {
Pexit(os, exe, 0, "e_phentsize is wrong");
}
@@ -804,8 +851,10 @@ static const char *TryElf(struct ApeLoader *M, union ElfEhdrBuf *ebuf,
/* read program headers */
rc = Pread(fd, M->phdr.buf, size, e->e_phoff, os);
if (rc < 0) return "failed to read ELF program headers";
if (rc != size) return "truncated read of ELF program headers";
if (rc < 0)
return "failed to read ELF program headers";
if (rc != size)
return "truncated read of ELF program headers";
/* bail on recoverable program header errors */
p = &M->phdr.phdr;
@@ -885,18 +934,13 @@ __attribute__((__noreturn__)) static void ShowUsage(int os, int fd, int rc) {
"NAME\n"
"\n"
" actually portable executable loader version " APE_VERSION_STR "\n"
" copyright 2023 justine alexandra roberts tunney\n"
" copyrights 2024 justine alexandra roberts tunney\n"
" https://justine.lol/ape.html\n"
"\n"
"USAGE\n"
"\n"
" ape [FLAGS] PROG [ARGV1,ARGV2,...]\n"
" ape [FLAGS] - PROG [ARGV0,ARGV1,...]\n"
"\n"
"FLAGS\n"
"\n"
" -h show this help\n"
" -f force loading of program (do not use execve)\n"
" ape PROG [ARGV1,ARGV2,...]\n"
" ape - PROG [ARGV0,ARGV1,...]\n"
"\n",
0l);
Exit(rc, os);
@@ -906,16 +950,16 @@ EXTERN_C __attribute__((__noreturn__)) void ApeLoader(long di, long *sp,
char dl) {
int rc, n;
unsigned i;
char literally;
const char *ape;
int c, fd, os, argc;
struct ApeLoader *M;
char arg0, literally;
unsigned long pagesz;
union ElfEhdrBuf *ebuf;
long *auxv, *ap, *endp, *sp2;
char *p, *pe, *exe, *prog, **argv, **envp;
(void)Utox;
(void)Printf;
/* detect freebsd */
if (SupportsXnu() && dl == XNU) {
@@ -940,20 +984,30 @@ EXTERN_C __attribute__((__noreturn__)) void ApeLoader(long di, long *sp,
/* determine ape loader program name */
ape = argv[0];
if (!ape) ape = "ape";
if (!ape)
ape = "ape";
/* detect openbsd */
if (SupportsOpenbsd() && !os && !auxv[0]) {
os = OPENBSD;
}
/* xnu passes auxv as an array of strings */
if (os == XNU) {
*auxv = 0;
}
/* detect netbsd and find end of words */
pagesz = 0;
arg0 = 0;
for (ap = auxv; ap[0]; ap += 2) {
if (ap[0] == AT_PAGESZ) {
pagesz = ap[1];
} else if (SupportsNetbsd() && !os && ap[0] == AT_EXECFN_NETBSD) {
os = NETBSD;
} else if (SupportsLinux() && ap[0] == AT_FLAGS) {
// TODO(mrdomino): maybe set/insert this when we are called as "ape -".
arg0 = !!(ap[1] & AT_FLAGS_PRESERVE_ARGV0);
}
}
if (!pagesz) {
@@ -966,22 +1020,13 @@ EXTERN_C __attribute__((__noreturn__)) void ApeLoader(long di, long *sp,
os = LINUX;
}
/* parse flags */
while (argc > 1) {
if (argv[1][0] != '-') break; /* normal argument */
if (!argv[1][1]) break; /* hyphen argument */
if (!StrCmp(argv[1], "-h") || !StrCmp(argv[1], "--help")) {
ShowUsage(os, 1, 0);
} else {
Print(os, 2, ape, ": invalid flag (pass -h for help)\n", 0l);
Exit(1, os);
}
*++sp = --argc;
++argv;
}
/* we can load via shell, shebang, or binfmt_misc */
if ((literally = argc >= 3 && !StrCmp(argv[1], "-"))) {
if (arg0) {
literally = 1;
prog = (char *)sp[2];
argc = sp[2] = sp[0] - 2;
argv = (char **)((sp += 2) + 1);
} else if ((literally = argc >= 3 && !StrCmp(argv[1], "-"))) {
/* if the first argument is a hyphen then we give the user the
power to change argv[0] or omit it entirely. most operating
systems don't permit the omission of argv[0] but we do, b/c
@@ -990,9 +1035,13 @@ EXTERN_C __attribute__((__noreturn__)) void ApeLoader(long di, long *sp,
argc = sp[3] = sp[0] - 3;
argv = (char **)((sp += 3) + 1);
} else if (argc < 2) {
Print(os, 2, ape, ": missing command name (pass -h for help)\n", 0l);
Exit(1, os);
ShowUsage(os, 2, 1);
} else {
if (argv[1][0] == '-') {
rc = !((argv[1][1] == 'h' && !argv[1][2]) ||
!StrCmp(argv[1] + 1, "-help"));
ShowUsage(os, 1 + rc, rc);
}
prog = (char *)sp[2];
argc = sp[1] = sp[0] - 1;
argv = (char **)((sp += 1) + 1);
@@ -1008,7 +1057,8 @@ EXTERN_C __attribute__((__noreturn__)) void ApeLoader(long di, long *sp,
grows down the alloc by poking the guard pages */
n = (endp - sp + 1) * sizeof(long);
sp2 = (long *)__builtin_alloca(n);
if ((long)sp2 & 15) ++sp2;
if ((long)sp2 & 15)
++sp2;
for (; n > 0; n -= pagesz) {
((char *)sp2)[n - 1] = 0;
}
@@ -1037,12 +1087,6 @@ EXTERN_C __attribute__((__noreturn__)) void ApeLoader(long di, long *sp,
}
pe = ebuf->buf + rc;
/* change argv[0] to resolved path if it's ambiguous */
if (argc > 0 && ((*prog != '/' && *exe == '/' && !StrCmp(prog, argv[0])) ||
!StrCmp(BaseName(prog), argv[0]))) {
argv[0] = exe;
}
/* ape intended behavior
1. if ape, will scan shell script for elf printf statements
2. shell script may have multiple lines producing elf headers
+1 -1
View File
@@ -1,5 +1,5 @@
/*-*- mode: ld-script; indent-tabs-mode: nil; tab-width: 2; coding: utf-8 -*-│
│vi: set et sts=2 tw=2 fenc=utf-8 :vi│
│ vi: set et sts=2 sw=2 fenc=utf-8 :vi │
╞══════════════════════════════════════════════════════════════════════════════╡
│ Copyright 2023 Justine Alexandra Roberts Tunney │
│ │
+23 -146
View File
@@ -1,5 +1,5 @@
/*-*- mode:unix-assembly; indent-tabs-mode:t; tab-width:8; coding:utf-8 -*-│
│vi: set et ft=asm ts=8 sw=8 fenc=utf-8 :vi│
│ vi: set noet ft=asm ts=8 sw=8 fenc=utf-8 :vi │
╞══════════════════════════════════════════════════════════════════════════════╡
│ Copyright 2020 Justine Alexandra Roberts Tunney │
│ │
@@ -18,123 +18,10 @@
╚─────────────────────────────────────────────────────────────────────────────*/
#ifndef APE_MACROS_H_
#define APE_MACROS_H_
#include "libc/macros.internal.h"
#include "libc/macros.h"
#ifdef __ASSEMBLER__
/* clang-format off */
/**
* @fileoverview Macros relevant to αcτµαlly pδrταblε εxεcµταblε.
*/
// Calls near (i.e. pc+pcrel<64kB) FUNCTION.
// @mode long,legacy,real
// @cost 9 bytes overhead
.macro rlcall function:req
.byte 0x50 # push %[er]ax
.byte 0xb8,0,0 # mov $?,%[e]ax
jmp 911f
.byte 0x58 # pop %[er]ax
.byte 0xe8 # call Jvds
.long \function\()-.-4
jmp 912f
911: .byte 0x58 # pop %[er]ax
.byte 0xe8 # call Jvds
.short \function\()-.-2
912:
.endm
// Loads far (i.e. <1mb) abs constexpr ADDRESS into ES:DI+EDX+RDX.
// @mode long,legacy,real
.macro movesdi address:req
.byte 0xbf # mov $0x????xxxx,%[e]di
.short \address>>4
.byte 0x8e,0xc7 # mov %di,%es
.byte 0xbf # mov $0x????xxxx,%[e]di
.short \address&0xf
jmp 297f
.byte 0xbf # mov $0x????xxxx,%edi
.long \address
297:
.endm
// Loads 16-bit CONSTEXPR into Qw-register w/ optional zero-extend.
// @mode long,legacy,real
.macro bbmov constexpr:req abcd abcd.hi:req abcd.lo:req
.ifnb \abcd
.if (\constexpr)<128 && (\constexpr)>=0
pushpop \constexpr,\abcd
.exitm
.endif
.endif
movb $(\constexpr)>>8&0xff,\abcd.hi
movb $(\constexpr)&0xff,\abcd.lo
.endm
// Compares 16-bit CONSTEXPR with Qw-register.
// @mode long,legacy,real
.macro bbcmp constexpr:req abcd.hi:req abcd.lo:req
cmpb $(\constexpr)>>8&0xff,\abcd.hi
jnz 387f
cmpb $(\constexpr)&0xff,\abcd.lo
387:
.endm
// Adds 16-bit CONSTEXPR to Qw-register.
// @mode long,legacy,real
.macro bbadd constexpr:req abcd.hi:req abcd.lo:req
addb $(\constexpr)&0xff,\abcd.lo
.if (\constexpr) != 0
adcb $(\constexpr)>>8&0xff,\abcd.hi
.endif
.endm
// Subtracts 16-bit CONSTEXPR from Qw-register.
// @mode long,legacy,real
.macro bbsub constexpr:req abcd.hi:req abcd.lo:req
subb $(\constexpr)&0xff,\abcd.lo
.if (\constexpr) != 0
sbbb $(\constexpr)>>8&0xff,\abcd.hi
.endif
.endm
// Ands Qw-register with 16-bit CONSTEXPR.
// @mode long,legacy,real
.macro bband constexpr:req abcd.hi:req abcd.lo:req
.if ((\constexpr)&0xff) != 0xff || ((\constexpr)>>8&0xff) == 0xff
andb $(\constexpr)&0xff,\abcd.lo
.endif
.if ((\constexpr)>>8&0xff) != 0xff
andb $(\constexpr)>>8&0xff,\abcd.hi
.endif
.endm
// Ors Qw-register with 16-bit CONSTEXPR.
// @mode long,legacy,real
.macro bbor constexpr:req abcd.hi:req abcd.lo:req
.if ((\constexpr)&0xff) != 0 || ((\constexpr)>>8&0xff) != 0
orb $(\constexpr)&0xff,\abcd.lo
.endif
.if ((\constexpr)>>8&0xff) != 0
orb $(\constexpr)>>8&0xff,\abcd.hi
.endif
.endm
// Performs ACTION only if in real mode.
// @mode long,legacy,real
.macro rlo clobber:req action:vararg
990: mov $0,\clobber
.if .-990b!=3
.error "bad clobber or assembler mode"
.endif
991: \action
.rept 2-(.-991b)
nop
.endr
.if .-991b!=2
.error "ACTION must be 1-2 bytes"
.endif
.endm
// Initializes real mode stack.
// The most holiest of holy code.
// @mode real
@@ -197,9 +84,10 @@
/* clang-format on */
#elif defined(__LINKER__)
#define BCX_NIBBLE(X) ((((X)&0xf) > 0x9) ? ((X)&0xf) + 0x37 : ((X)&0xf) + 0x30)
#define BCX_OCTET(X) ((BCX_NIBBLE((X) >> 4) << 8) | (BCX_NIBBLE((X) >> 0) << 0))
#define BCX_INT16(X) ((BCX_OCTET((X) >> 8) << 16) | (BCX_OCTET((X) >> 0) << 0))
#define BCX_NIBBLE(X) \
((((X) & 0xf) > 0x9) ? ((X) & 0xf) + 0x37 : ((X) & 0xf) + 0x30)
#define BCX_OCTET(X) ((BCX_NIBBLE((X) >> 4) << 8) | (BCX_NIBBLE((X) >> 0) << 0))
#define BCX_INT16(X) ((BCX_OCTET((X) >> 8) << 16) | (BCX_OCTET((X) >> 0) << 0))
#define BCXSTUB(SYM, X) \
HIDDEN(SYM##_bcx0 = BCX_INT16((X) >> 48)); \
HIDDEN(SYM##_bcx1 = BCX_INT16((X) >> 32)); \
@@ -211,12 +99,12 @@
*
* <p>This allows linker scripts to generate printf commands.
*/
#define BCO_OCTET(X) (((X)&0x7) + 0x30)
#define BCO_OCTET(X) (((X) & 0x7) + 0x30)
#define BCOB_UNIT(X) \
((BCO_OCTET((X) >> 0) << 24) | (BCO_OCTET((X) >> 3) << 16) | \
(BCO_OCTET(((X)&0xff) >> 6) << 8) | 0x5c)
(BCO_OCTET(((X) & 0xff) >> 6) << 8) | 0x5c)
#define PFBYTE(SYM, X, I) HIDDEN(SYM##_bcs##I = BCOB_UNIT((X) >> ((I)*8)))
#define PFBYTE(SYM, X, I) HIDDEN(SYM##_bcs##I = BCOB_UNIT((X) >> ((I) * 8)))
#define PFSTUB2(SYM, X) \
HIDDEN(SYM = (X)); \
PFBYTE(SYM, X, 0); \
@@ -253,23 +141,16 @@
: (X) < 10000000 ? BCD_RIGHT(BCD_SMEAR((X) / 1000)) \
: (X) < 100000000 ? BCD_RIGHT(BCD_SMEAR((X) / 10000)) \
: 0xffffffffffffffff)
#define BCD_RIGHT(X) \
(((X)) < 10000 ? 0x20202020 \
: (X) < 100000 ? 0x20202030 + \
(X) % 10 \
: (X) < 1000000 ? 0x20203030 + \
((X) / 10) % 10 + \
(X) % 10 * 0x100 \
: (X) < 10000000 ? 0x20303030 + \
((X) / 100) % 10 + \
((X) / 10) % 10 * 0x100 + \
(X) % 10 * 0x10000 \
: (X) < 100000000 ? 0x30303030 + \
((X) / 1000) % 10 + \
((X) / 100) % 10 * 0x100 + \
((X) / 10) % 10 * 0x10000 + \
(X) % 10 * 0x1000000 \
: 0xffffffffffffffff)
#define BCD_RIGHT(X) \
(((X)) < 10000 ? 0x20202020 \
: (X) < 100000 ? 0x20202030 + (X) % 10 \
: (X) < 1000000 ? 0x20203030 + ((X) / 10) % 10 + (X) % 10 * 0x100 \
: (X) < 10000000 ? 0x20303030 + ((X) / 100) % 10 + \
((X) / 10) % 10 * 0x100 + (X) % 10 * 0x10000 \
: (X) < 100000000 \
? 0x30303030 + ((X) / 1000) % 10 + ((X) / 100) % 10 * 0x100 + \
((X) / 10) % 10 * 0x10000 + (X) % 10 * 0x1000000 \
: 0xffffffffffffffff)
/**
* Laying out the GDT entries for a TSS for bare metal operation.
@@ -278,15 +159,11 @@
HIDDEN(SYM##_desc_ent0 = TSSDESC_ENT0(BASE, LIM)); \
HIDDEN(SYM##_desc_ent1 = TSSDESC_ENT1(BASE)); \
ASSERT((LIM) >= 0 && (LIM) <= 0xffff, "bare metal TSS is suspiciously fat")
#define TSSDESC_ENT0(BASE, LIM) \
(((LIM) << 0 & 0x000000000000ffff) | \
((BASE) << 16 & 0x000000ffffff0000) | \
0x89 << 40 | \
((LIM) >> 16 << 48 & 0x000f000000000000) | \
0x2 << 52 | \
#define TSSDESC_ENT0(BASE, LIM) \
(((LIM) << 0 & 0x000000000000ffff) | ((BASE) << 16 & 0x000000ffffff0000) | \
0x89 << 40 | ((LIM) >> 16 << 48 & 0x000f000000000000) | 0x2 << 52 | \
((BASE) >> 24 << 56 & 0xff00000000000000))
#define TSSDESC_ENT1(BASE) \
((BASE) >> 32 << 0 & 0x00000000ffffffff)
#define TSSDESC_ENT1(BASE) ((BASE) >> 32 << 0 & 0x00000000ffffffff)
#endif /* __ASSEMBLER__ */
#endif /* APE_MACROS_H_ */
+10 -7
View File
@@ -1,9 +1,8 @@
#ifndef COSMOPOLITAN_APE_SECTIONS_INTERNAL_H_
#define COSMOPOLITAN_APE_SECTIONS_INTERNAL_H_
#if !(__ASSEMBLER__ + __LINKER__ + 0)
COSMOPOLITAN_C_START_
extern const char __comment_start[] __attribute__((__weak__));
extern const char __notices[] __attribute__((__weak__));
extern unsigned char __executable_start[] __attribute__((__weak__));
extern unsigned char __privileged_start[] __attribute__((__weak__));
extern unsigned char _ehead[] __attribute__((__weak__));
@@ -17,13 +16,17 @@ extern unsigned char _tdata_end[] __attribute__((__weak__));
extern unsigned char _tbss_start[] __attribute__((__weak__));
extern unsigned char _tbss_end[] __attribute__((__weak__));
extern unsigned char _tls_align[] __attribute__((__weak__));
extern unsigned char _tdata_align[] __attribute__((__weak__));
extern unsigned char _tbss_align[] __attribute__((__weak__));
extern unsigned char __test_start[] __attribute__((__weak__));
extern unsigned char __ro[] __attribute__((__weak__));
extern uint8_t __data_start[] __attribute__((__weak__));
extern uint8_t __data_end[] __attribute__((__weak__));
extern uint8_t __bss_start[] __attribute__((__weak__));
extern uint8_t __bss_end[] __attribute__((__weak__));
extern unsigned char __data_start[] __attribute__((__weak__));
extern unsigned char __data_end[] __attribute__((__weak__));
extern unsigned char __bss_start[] __attribute__((__weak__));
extern unsigned char __bss_end[] __attribute__((__weak__));
extern unsigned long __got_start[] __attribute__((__weak__));
extern unsigned long __got_end[] __attribute__((__weak__));
extern unsigned char ape_phdrs[] __attribute__((__weak__));
COSMOPOLITAN_C_END_
#endif /* !(__ASSEMBLER__ + __LINKER__ + 0) */
#endif /* COSMOPOLITAN_APE_SECTIONS_INTERNAL_H_ */
+703
View File
@@ -0,0 +1,703 @@
# Actually Portable Executable Specification v0.1
Actually Portable Executable (APE) is an executable file format that
polyglots the Windows Portable Executable (PE) format with a UNIX Sixth
Edition style shell script that doesn't have a shebang. This makes it
possible to produce a single file binary that executes on the stock
installations of the many OSes and architectures.
## Supported OSes and Architectures
- AMD64
- Linux
- MacOS
- Windows
- FreeBSD
- OpenBSD
- NetBSD
- BIOS
- ARM64
- Linux
- MacOS
- FreeBSD
- Windows (non-native)
## File Header
APE defines three separate file magics, all of which are 8 characters
long. Any file that starts with one of these magic values can be
considered an APE program.
### (1) APE MZ Magic
- ASCII: `MZqFpD='`
- Hex: 4d 5a 71 46 70 44 3d 27
This is the canonical magic used by almost all APE programs. It enables
maximum portability between OSes. When interpreted as a shell script, it
is assigning a single quoted string to an unused variable. The shell
will then ignore subsequent binary content that's placed inside the
string.
It is strongly recommended that this magic value be immediately followed
by a newline (\n or hex 0a) character. Some shells, e.g. FreeBSD SH and
Zsh impose a binary safety check before handing off files that don't
have a shebang to `/bin/sh`. That check applies to the first line, which
can't contain NUL characters.
The letters were carefully chosen so as to be valid x86 instructions in
all operating modes. This makes it possible to store a BIOS bootloader
disk image inside an APE binary. For example, simple CLI programs built
with Cosmopolitan Libc will boot from BIOS into long mode if they're
treated as a floppy disk image.
The letters also allow for the possibility of being treated on x86-64 as
a flat executable, where the PE / ELF / Mach-O executable structures are
ignored, and execution simply begins at the beginning of the file,
similar to how MS-DOS .COM binaries work.
The 0x4a relative offset of the magic causes execution to jump into the
MS-DOS stub defined by Portable Executable. APE binaries built by Cosmo
Libc use tricks in the MS-DOS stub to check the operating mode and then
jump to the appropriate entrypoint, e.g. `_start()`.
#### Decoded as i8086
```asm
dec %bp
pop %dx
jno 0x4a
jo 0x4a
```
#### Decoded as i386
```asm
push %ebp
pop %edx
jno 0x4a
jo 0x4a
```
#### Decoded as x86-64
```asm
rex.WRB
pop %r10
jno 0x4a
jo 0x4a
```
### (2) APE UNIX-Only Magic
- ASCII: `jartsr='`
- Hex: 6a 61 72 74 73 72 3d 27
Being a novel executable format that was first published in 2020, the
APE file format is less understood by industry tools compared to the PE,
ELF, and Mach-O executable file formats, which have been around for
decades. For this reason, APE programs that use the MZ magic above can
attract attention from Windows AV software, which may be unwanted by
developers who aren't interested in targeting the Windows platform.
Therefore the `jartsr='` magic is defined which enables the creation of
APE binaries that can safely target all non-Windows platforms. Even
though this magic is less common, APE interpreters and binfmt-misc
installations MUST support this.
It is strongly recommended that this magic value be immediately followed
by a newline (\n or hex 0a) character. Some shells, e.g. FreeBSD SH and
Zsh impose a binary safety check before handing off files that don't
have a shebang to `/bin/sh`. That check applies to the first line, which
can't contain NUL characters.
The letters were carefully chosen so as to be valid x86 instructions in
all operating modes. This makes it possible to store a BIOS bootloader
disk image inside an APE binary. For example, simple CLI programs built
with Cosmopolitan Libc will boot from BIOS into long mode if they're
treated as a floppy disk image.
The letters also allow for the possibility of being treated on x86-64 as
a flat executable, where the PE / ELF / Mach-O executable structures are
ignored, and execution simply begins at the beginning of the file,
similar to how MS-DOS .COM binaries work.
The 0x78 relative offset of the magic causes execution to jump into the
MS-DOS stub defined by Portable Executable. APE binaries built by Cosmo
Libc use tricks in the MS-DOS stub to check the operating mode and then
jump to the appropriate entrypoint, e.g. `_start()`.
#### Decoded as i8086 / i386 / x86-64
```asm
push $0x61
jb 0x78
jae 0x78
```
### (3) APE Debug Magic
- ASCII: `APEDBG='`
- Hex: 41 50 45 44 42 47 3d 27
While APE files must be valid shell scripts, in practice, UNIX systems
will oftentimes be configured to provide a faster safer alternative to
loading an APE binary through `/bin/sh`. The Linux Kernel can be patched
to have execve() recognize the APE format and directly load its embedded
ELF header. Linux systems can also use binfmt-misc to recognize APE's MZ
and jartsr magic, and pass them to a userspace program named `ape` that
acts as an interpreter. In such environments, the need sometimes arises
to be able to test that the `/bin/sh` is working correctly, in which
case the `APEDBG='` magic is RECOMMENDED.
APE interpreters, execve() implementations, and binfmt-misc installs
MUST ignore this magic. If necessary, steps can be taken to help files
with this magic be passed to `/bin/sh` like a normal shebang-less shell
script for execution.
## Embedded ELF Header
APE binaries MAY embed an ELF header inside them. Unlike conventional
executable file formats, this header is not stored at a fixed offset.
It's instead encoded as octal escape codes in a shell script `printf`
statement. For example:
```
printf '\177ELF\2\1\1\011\0\0\0\0\0\0\0\0\2\0\076\0\1\0\0\0\166\105\100\000\000\000\000\000\060\013\000\000\000\000\000\000\000\000\000\000\000\000\000\000\165\312\1\1\100\0\070\0\005\000\0\0\000\000\000\000'
```
This `printf` statement MUST appear in the first 8192 bytes of the APE
executable, so as to limit how much of the initial portion of a file an
interpreter must load.
Multiple such `printf` statements MAY appear in the first 8192 bytes, in
order to specify multiple architectures. For example, fat binaries built
by the `apelink` program (provided by Cosmo Libc) will have two encoded
ELF headers, for AMD64 and ARM64, each of which point into the proper
file offsets for their respective native code. Therefore, kernels and
interpreters which load the APE format directly MUST check the
`e_machine` field of the `Elf64_Ehdr` that's decoded from the octal
codes, before accepting a `printf` shell statement as valid.
These printf statements MUST always use only unescaped ASCII characters
or octal escape codes. These printf statements MUST NOT use space saving
escape codes such as `\n`. For example, rather than saying `\n` it would
be valid to say `\012` instead. It's also valid to say `\12` but only if
the encoded characters that follow aren't an octal digit.
For example, the following algorithm may be used for parsing octal:
```c
static int ape_parse_octal(const unsigned char page[8192], int i, int *pc)
{
int c;
if ('0' <= page[i] && page[i] <= '7') {
c = page[i++] - '0';
if ('0' <= page[i] && page[i] <= '7') {
c *= 8;
c += page[i++] - '0';
if ('0' <= page[i] && page[i] <= '7') {
c *= 8;
c += page[i++] - '0';
}
}
*pc = c;
}
return i;
}
```
APE aware interpreters SHOULD only take `e_machine` into consideration.
It is the responsibility of the `_start()` function to detect the OS.
Therefore, multiple `printf` statements are only embedded in the shell
script for different CPU architectures.
The OS ABI field of an APE embedded `Elf64_Ehdr` SHOULD be set to
`ELFOSABI_FREEBSD`, since it's the only UNIX OS APE supports that
actually checks the field. However different values MAY be chosen for
binaries that don't intend to have FreeBSD in their support vector.
Counter-intuitively, the ARM64 ELF header is used on the MacOS ARM64
platform when loading from fat binaries.
## Embedded Mach-O Header (x86-64 only)
APE shell scripts that support MacOS on AMD64 must use the `dd` command
in a very specific way to specify how the embedded binary Macho-O header
is copied backward to the start of the file. For example:
```
dd if="$o" of="$o" bs=8 skip=433 count=66 conv=notrunc
```
These `dd` statements have traditionally been generated by the GNU as
and ld.bfd programs by encoding ASCII into 64-bit linker relocations,
which necessitated a fixed width for integer values. It took several
iterations over APE's history before we eventually got it right:
- `arg=" 9293"` is how we originally had ape do it
- `arg=$(( 9293))` b/c busybox sh disliked quoted space
- `arg=9293 ` is generated by modern apelink program
Software that parses the APE file format, which needs to extract the
Macho-O x86-64 header SHOULD support the old binaries that use the
previous encodings. To make backwards compatibility simple the following
regular expression may be used, which generalizes to all defined
formats:
```c
regcomp(&rx,
"bs=" // dd block size arg
"(['\"] *)?" // #1 optional quote w/ space
"(\\$\\(\\( *)?" // #2 optional math w/ space
"([[:digit:]]+)" // #3
"( *\\)\\))?" // #4 optional math w/ space
"( *['\"])?" // #5 optional quote w/ space
" +" //
"skip=" // dd skip arg
"(['\"] *)?" // #6 optional quote w/ space
"(\\$\\(\\( *)?" // #7 optional math w/ space
"([[:digit:]]+)" // #8
"( *\\)\\))?" // #9 optional math w/ space
"( *['\"])?" // #10 optional quote w/ space
" +" //
"count=" // dd count arg
"(['\"] *)?" // #11 optional quote w/ space
"(\\$\\(\\( *)?" // #12 optional math w/ space
"([[:digit:]]+)", // #13
REG_EXTENDED);
```
For further details, see the canonical implementation in
`cosmopolitan/tool/build/assimilate.c`.
## Static Linking
Actually Portable Executables are always statically linked. This
revision of the specification does not define any facility for storing
code in dynamic shared objects.
Cosmopolitan Libc provides a solution that enables APE binaries have
limited access to dlopen(). By manually loading a platform-specific
executable and asking the OS-specific libc's dlopen() to load
OS-specific libraries, it becomes possible to use GPUs and GUIs. This
has worked great for AI projects like llamafile.
There is no way for an Actually Portable Executable to interact with
OS-specific dynamic shared object extension modules to programming
languages. For example, a Lua interpreter compiled as an Actually
Portable Executable would have no way of linking extension libraries
downloaded from the Lua Rocks package manager. This is primarily because
different OSes define incompatible ABIs.
While it was possible to polyglot PE+ELF+MachO to create multi-OS
executables, it simply isn't possible to do that same thing for
DLL+DYLIB+SO. Therefore, in order to have DSOs, APE would need to either
choose one of the existing formats or invent one of its own, and then
develop its own parallel ecosystem of extension software. In the future,
the APE specification may expand to encompass this. However the focus to
date has been exclusively on executables with limited dlopen() support.
## Application Binary Interface (ABI)
APE binaries use the System V ABI, as defined by:
- [System V ABI - AMD64 Architecture Processor Supplement](https://gitlab.com/x86-psABIs/x86-64-ABI)
- AARCH64 has a uniform consensus defined by ARM Limited
There are however a few changes we've had to make.
### No Red Zone
Actually Portable Executables that have Windows and/or bare metal in
their support vector MUST be compiled using `-mno-red-zone`. This is
because, on Windows, DLLs and other software lurking in the va-space
might use tricks like SetThreadContext() to take control of a thread
whereas on bare metal, it's also generally accepted that kernel-mode
code cannot assume a red zone either due to hardware interrupts that
pull the exact same kinds of stunts.
APE software that only has truly System V ABI conformant OSes (e.g.
Linux) in their support vector MAY use the red zone optimization.
### Thread Local Storage
#### aarch64
Here's the TLS memory layout on aarch64:
```
x28
%tpidr_el0
│
│ _Thread_local
┌───┼───┬──────────┬──────────┐
│tib│dtv│ .tdata │ .tbss │
├───┴───┴──────────┴──────────┘
│
__get_tls()
```
The ARM64 code in actually portable executables use the `x28` register
to store the address of the thread information block. All aarch64 code
linked into these executables SHOULD be compiled with `-ffixed-x28`
which is supported by both Clang and GCC.
The runtime library for an actually portable executables MAY choose to
use `tpidr_el0` instead, if OSes like MacOS aren't being targeted. For
example, if the goal is to create a Linux-only fat binary linker program
for Musl Libc, then choosing to use the existing `tpidr_el0` convention
would be friction-free alternative.
It's not possible for an APE runtime that targets the full range of OSes
to use the `tpidr_el0` register for TLS because Apple won't allow it. On
MacOS ARM64 systems, this register can only be used by a runtime to
implement the `sched_getcpu()` system call. It's reserved by MacOS.
#### x86-64
Here's the TLS memory layout on x86_64:
```
__get_tls()
│
%fs OpenBSD/NetBSD
_Thread_local │
┌───┬──────────┬──────────┼───┐
│pad│ .tdata │ .tbss │tib│
└───┴──────────┴──────────┼───┘
│
Linux/FreeBSD/Windows/Mac %gs
```
Quite possibly the greatest challenge in Actually Portable Executable
working, has been overcoming the incompatibilities between OSes in how
thread-local storage works on x86-64. The AMD64 architecture defines two
special segment registers. Every OS uses one of these segment registers
to implement TLS support. However not all OSes agree on which register
to use. Some OSes grant userspace the power to define either of these
registers to hold any value that is desired. Some OSes only effectively
allow a single one of them to be changed. Lastly, some OSes, e.g.
Windows, claim ownership of the memory layout these registers point
towards too.
Here's a breakdown on how much power is granted to userspace runtimes by
each OS when it comes to changing amd64 segment registers.
| | %fs | %gs |
|---------|--------------|--------------|
| Linux | unrestricted | unrestricted |
| MacOS | inaccessible | unrestricted |
| Windows | inaccessible | restricted |
| FreeBSD | unrestricted | unrestricted |
| NetBSD | unrestricted | broken |
| OpenBSD | unrestricted | inaccessible |
Therefore, regardless of which register one we choose, some OSes are
going to be incompatible.
APE binaries are always built with a Linux compiler. So another issue
arises in the fact that our Linux-flavored GCC and Clang toolchains
(which are used to produce cross-OS binaries) are also only capable of
producing TLS instructions that use the %fs convention.
To solve these challenges, the `cosmocc` compiler will rewrite binary
objects after they've been compiled by GCC, so that the `%gs` register
is used, rather than `%fs`. Morphing x86-64 binaries after they've been
compiled is normally difficult, due to the complexity of the machine
instruction language. However GCC provides `-mno-tls-direct-seg-refs`
which greatly reduces the complexity of this task. This flag forgoes
some optimizations to make the generated code simpler. Rather than doing
clever arithmetic with `%fs` prefixes, the compiler will always generate
the thread information block address load as a separate instruction.
```c
// Change AMD code to use %gs:0x30 instead of %fs:0
// We assume -mno-tls-direct-seg-refs has been used
static void ChangeTlsFsToGs(unsigned char *p, size_t n) {
unsigned char *e = p + n - 9;
while (p <= e) {
// we're checking for the following expression:
// 0144 == p[0] && // %fs
// 0110 == (p[1] & 0373) && // rex.w (and ignore rex.r)
// (0213 == p[2] || // mov reg/mem → reg (word-sized)
// 0003 == p[2]) && // add reg/mem → reg (word-sized)
// 0004 == (p[3] & 0307) && // mod/rm (4,reg,0) means sib → reg
// 0045 == p[4] && // sib (5,4,0) → (rbp,rsp,0) → disp32
// 0000 == p[5] && // displacement (von Neumann endian)
// 0000 == p[6] && // displacement
// 0000 == p[7] && // displacement
// 0000 == p[8] // displacement
uint64_t w = READ64LE(p) & READ64LE("\377\373\377\307\377\377\377\377");
if ((w == READ64LE("\144\110\213\004\045\000\000\000") ||
w == READ64LE("\144\110\003\004\045\000\000\000")) &&
!p[8]) {
p[0] = 0145; // change %fs to %gs
p[5] = 0x30; // change 0 to 0x30
p += 9;
} else {
++p;
}
}
}
```
By favoring `%gs` we've now ensured friction-free compatibility for the
APE runtime on MacOS, Linux, and FreeBSD which are all able to conform
easily to this convention. However additional work needs to be done at
runtime when an APE program is started on Windows, OpenBSD, and NetBSD.
On these platforms, all executable pages must be faulted and morphed to
fixup the TLS instructions.
On OpenBSD and NetBSD, this is as simple as undoing the example
operation above. Earlier at compile-time we turned `%fs` into `%gs`.
Now, at runtime, `%gs` must be turned back into `%fs`. Since the
executable is morphing itself, this is easier said than done.
OpenBSD for example enforces a `W^X` invariant. Code that's executing
can't modify itself at the same time. The way Cosmopolitan solves this
is by defining a special part of the binary called `.text.privileged`.
This section is aligned to page boundaries. A GNU ld linker script is
used to ensure that code which morphs code is placed into this section,
through the use of a header-defined cosmo-specific keyword `privileged`.
Additionally, the `fixupobj` program is used by the Cosmo build system
to ensure that compiled objects don't contain privileged functions that
call non-privileged functions. Needless to say, `mprotect()` needs to be
a privileged function, so that it can be used to disable the execute bit
on all other parts of the executable except for the privileged section,
thereby making it writable. Once this has been done, code can change.
On Windows the displacement bytes of the TLS instruction are changed to
use the `%gs:0x1480+i*8` ABI where `i` is a number assigned by the WIN32
`TlsAlloc()` API. This avoids the need to call `TlsGetValue()` which is
implemented this exact same way under the hood. Even though 0x1480 isn't
explicitly documented by MSDN, this ABI is believed to be stable because
MSVC generates binaries that use this offset directly. The only caveat
is that `TlsAlloc()` must be called as early in the runtime init as
possible, to ensure an index less than 64 is returned.
### Thread Information Block (TIB)
The Actually Portable Executable Thread Information Block (TIB) is
defined by this version of the specification as follows:
- The 64-bit TIB self-pointer is stored at offset 0x00.
- The 64-bit TIB self-pointer is also stored at offset 0x30.
- The 32-bit `errno` value is stored at offset 0x3c.
All other parts of the thread information block should be considered
unspecified and therefore reserved for future specifications.
The APE thread information block is aligned on a 64-byte boundary.
Cosmopolitan Libc v3.5.8 (c. 2024-07-21) currently implements a thread
information block that's 512 bytes in size.
### Foreign Function Calls
Even though APE programs always use the System V ABI, there arises the
occasional need to interface with foreign functions, e.g. WIN32. The
`__attribute__((__ms_abi__))` annotation introduced by GCC v6 is used
for this purpose.
The ability to change a function's ABI on a case-by-case basis is
surprisingly enough supported by GCC, Clang, NVCC, and even the AMD HIP
compilers for both UNIX systems and Windows. All of these compilers
support both the System V ABI and the Microsoft x64 ABI.
APE binaries will actually favor the Microsoft ABI even when running on
UNIX OSes for certain dlopen() use-cases. For example, if we control the
code to a CUDA module, which we compile on each OS separately from our
main APE binary, then any function that's inside the APE binary whose
pointer may be passed into a foreign module SHOULD be compiled to use
the Microsoft ABI. This is because in practice the OS-specific module
may need to be compiled by MSVC, where MS ABI is the *only* ABI, which
forces our UNIX programs to partially conform. Thankfully, all UNIX
compilers support doing it on a case-by-case basis.
### Char Signedness
Actually Portable Executable defines `char` as signed.
Therefore conformant APE software MUST use `-fsigned-char` when building
code for aarch64, as well as any other architecture that (unlike x86-64)
would otherwise define `char` as being `unsigned char` by default.
This decision was one of the cases where it made sense to offer a more
consistent runtime experience for fat multi-arch binaries. However you
SHOULD still write code to assume `char` can go either way. But if all
you care about is using APE, then you CAN assume `char` is signed.
### Long Double
On AMD64 platforms, APE binaries define `long double` as 80-bit.
On ARM64 platforms, APE binaries define `long double` as 128-bit.
We accept inconsistency in this case, because hardware acceleration is
far more valuable than stylistic consistency in the case of mathematics.
One challenge arises on AMD64 for supporting `long double` across OSes.
Unlike UNIX systems, the Windows Executive on x86-64 initializes the x87
FPU to have double (64-bit) precision rather than 80-bit. That's because
code compiled by MSVC treats `long double` as though it were `double` to
prefer always using the more modern SSE instructions. However System V
requires genuine 80-bit `long double` support on AMD64.
Therefore, if an APE program detects that it's been started on a Windows
x86-64 system, then it SHOULD use the following assembly to initialize
the x87 FPU in System V ABI mode.
```asm
fldcw 1f(%rip)
.rodata
.balign 2
// 8087 FPU Control Word
// IM: Invalid Operation ───────────────┐
// DM: Denormal Operand ───────────────┐│
// ZM: Zero Divide ───────────────────┐││
// OM: Overflow ─────────────────────┐│││
// UM: Underflow ───────────────────┐││││
// PM: Precision ──────────────────┐│││││
// PC: Precision Control ───────┐ ││││││
// {float,∅,double,long double}│ ││││││
// RC: Rounding Control ──────┐ │ ││││││
// {even, →-∞, →+∞, →0} │┌┤ ││││││
// ┌┤││ ││││││
// d││││rr││││││
1: .short 0b00000000000000000001101111111
.previous
```
## Executable File Alignment
Actually Portable Executable is a statically-linked flat executable file
format that is, as a thing in itself, agnostic to file alignments. For
example, the shell script payload at the beginning of the file and its
statements have no such requirements. Alignment requirements are however
imposed by the executable formats that APE wraps.
1. ELF requires that file offsets be congruent with virtual addresses
modulo the CPU page size. So when we add a shell script to the start
of an executable, we need to round up to the page size in order to
maintain ELF's invariant. Although no such roundup is required on the
program segments once the invariant is restored. ELF loaders will
happily map program headers from arbitrary file intervals (which may
overlap) onto arbitrarily virtual intervals (which don't need to be
contiguous). In order to do that, the loaders will generally use
UNIX's mmap() function which is more restrictive and only accepts
addresses and offsets that are page aligned. To make it possible to
map an unaligned ELF program header that could potentially start and
stop at any byte, ELF loaders round-out the intervals, which means
adjacent unrelated data might also get mapped, which may need to be
explicitly zero'd. Thanks to the cleverness of ELF, it's possible to
have an executable file be very tiny, without needing any alignment
bytes, and it'll be loaded into a properly aligned virtual space
where segments can be as sparse as we want them to be.
2. PE doesn't care about congruence and instead defines two separate
kinds of alignment. First, PE requires that the layout of segment
memory inside the file be aligned on at minimum the classic 512 byte
MS-DOS page size. This means that, unlike ELF, some alignment padding
may need to be encoded into the file, making it slightly larger. Next
PE imposes an alignment restriction on segments once they've been
mapped into the virtual address space, which must be rounded to the
system page size. Like ELF, PE segments need to be properly ordered
but they're allowed to drift apart once mapped in a non-contiguous
sparsely mapped way. When inserting shell script content at the start
of a PE file, the most problematic thing is the need to round up to
the 64kb system granularity, which results in a lot of needless bytes
of padding being inserted by a naive second-pass linker.
3. Apple's Mach-O format is the strictest of them all. While both ELF
and PE are defined in such a way that invites great creativity, XNU
will simply refuse to an executable that does anything creative with
alignment. All loaded segments need to both start and end on a page
aligned address. XNU also wants segments to be contiguous similar to
portable executable, except it applies to both the file and virtual
spaces, which must follow the same structure.
Actually Portable Executables must conform to the strictest requirements
demanded by the support vector. Therefore an APE binary that has headers
for all three of the above executable formats MUST conform to the Apple
way of doing things. GNU ld linker scripts aren't very good at producing
ELF binaries that rigidly conform to this simple naive layout. There are
so many ways things can go wrong, where third party code might slip its
own custom section name in-between the linker script sections that are
explicitly defined, thereby causing ELF's powerful features to manifest
and the resulting content overlapping. The best `ld` flag that helps is
`--orphan-handling=error` which can help with explaining such mysteries.
While Cosmopolitan was originally defined to just use stock GNU tools,
this proved intractable over time, and the project has been evolving in
the direction of building its own. Inventing the `apelink` program was
what enabled the project to achieve multi-architecture binaries whereas
previously it was only possible to do multi-OS binaries. In the future,
our hope is that a fast power linker like Mold can be adapted to produce
fat APE binaries directly from object files in one pass.
## Position Independent Code
APE doesn't currently support position independent executable formats.
This is because APE was originally written for the GNU linker, where PIC
and PIE were after-thoughts and never fully incorporated with the older
more powerful linker script techniques upon which APE relies. Future
iterations of this specification are intended to converge on modern
standards, as our tooling becomes developed enough to support it.
However this only applies to the wrapped executable formats themselves.
While our convention to date has been to always load ELF programs at the
4mb mark, this is not guaranteed across OSes and architectures. Programs
should have no expectations that a program will be loaded to any given
address. For example, Cosmo currently implements APE on AARCH64 as
loading executables to a starting address of 0x000800000000. This
address occupies a sweet spot of requirements.
## Address Space
In order to create a single binary that supports as many platforms as
possible without needing to be recompiled, there's a very narrow range
of addresses that can be used. That range is somewhere between 32 bits
and 39 bits.
- Embedded devices that claim to be 64-bit will oftentimes only support
a virtual address space that's 39 bits in size.
- We can't load executable images on AARCH64 beneath 0x100000000 (4gb)
because Apple forbids doing that, possibly in an effort to enforce a
best practice for spotting 32-bit to 64-bit transition bugs. Please
note that this restriction only applies to Apple ARM64 systems. The
x86-64 version of XNU will happily load APE binaries to 0x00400000.
- The AMD64 architecture on desktops and servers can usually be counted
upon to provide a 47-bit address space. The Linux Kernel for instance
grants each userspace program full dominion over addresses 0x00200000
through 0x00007fffffffffff provided the hardware supports this. On
modern workstations supporting Intel and AMD's new PML5T feature which
virtualizes memory using a radix trie that's five layers deep, Linux
is able to offer userspace its choice of fixed addresses from
0x00200000 through 0x00ffffffffffffff. The only exception to this rule
we've encountered so far is that Windows 7 and Windows Vista behaved
similar to embedded devices in reducing the number of va bits.
## Page Size
APE software MUST be page size agnostic. For many years the industry had
converged on a strong consensus of having a page size that's 4096 bytes.
However this convention was never guaranteed. New computers have become
extremely popular, such as Apple Silicon, that use a 16kb page size.
By convention, Cosmopolitan Libc currently generates ELF headers for
x86-64 that are strictly aligned on a 4096-byte page size. On ARM64
Cosmopolitan is currently implemented to always generate ELF headers
aligned on a 16kb page size.
In addition to being page size agnostic, APE software that cares about
working correctly on Windows needs to be aware of the concept of
allocation granularity. While the page size on Windows is generally 4kb
in size, memory mappings can only be created on addresses that aligned
to the system allocation granularity, which is generally 64kb. If you
use a function like mmap() with Cosmopolitan Libc, then the `addr` and
`offset` parameters need to be aligned to `sysconf(_SC_GRANSIZE)` or
else your software won't work on Windows. Windows has other limitations
too, such as lacking the ability to carve or punch holes in mappings.
+4 -2
View File
@@ -1,5 +1,5 @@
/*-*- mode:unix-assembly; indent-tabs-mode:t; tab-width:8; coding:utf-8 -*-│
│vi: set et ft=asm ts=8 tw=8 fenc=utf-8 :vi│
│ vi: set noet ft=asm ts=8 sw=8 fenc=utf-8 :vi │
╞══════════════════════════════════════════════════════════════════════════════╡
│ Copyright 2023 Justine Alexandra Roberts Tunney │
│ │
@@ -18,11 +18,12 @@
╚─────────────────────────────────────────────────────────────────────────────*/
#include "libc/dce.h"
#include "ape/ape.h"
#include "libc/macros.internal.h"
#include "libc/macros.h"
#ifdef __aarch64__
_start: mov x1,sp
and sp,x1,#-16
mov x29,0
bl ApeLoader
.endfn _start,globl
@@ -33,6 +34,7 @@ XnuEntrypoint:
mov $_HOSTXNU,%dl // xnu's not unix!
ElfEntrypoint:
mov %rsp,%rsi // save real stack
andq $-16,%rsp // force SSE alignment
call ApeLoader
.endfn ElfEntrypoint,globl
.endfn XnuEntrypoint,globl
+2 -2
View File
@@ -1,5 +1,5 @@
/*-*- mode:unix-assembly; indent-tabs-mode:t; tab-width:8; coding:utf-8 -*-│
│vi: set et ft=asm ts=8 tw=8 fenc=utf-8 :vi│
│ vi: set noet ft=asm ts=8 sw=8 fenc=utf-8 :vi │
╞══════════════════════════════════════════════════════════════════════════════╡
│ Copyright 2023 Justine Alexandra Roberts Tunney │
│ │
@@ -16,7 +16,7 @@
│ TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR │
│ PERFORMANCE OF THIS SOFTWARE. │
╚─────────────────────────────────────────────────────────────────────────────*/
#include "libc/macros.internal.h"
#include "libc/macros.h"
// Invokes system call.
//
-19
View File
@@ -1,19 +0,0 @@
#!/bin/sh
MODE=${MODE:-${m:-aarch64}}
COSMO=${COSMO:-/opt/cosmo}
COSMOS=${COSMOS:-/opt/cosmos/aarch64}
if [ -x "$COSMO/o/third_party/gcc/bin/aarch64-linux-musl-gcc" ]; then
BRAND=musl
else
BRAND=cosmo
fi
TOOL="$COSMO/o/third_party/gcc/bin/aarch64-linux-$BRAND-addr2line"
if [ ! -x "$TOOL" ]; then
echo "$0: you need to run: aarch64-unknown-cosmo-cc --update" >&2
exit 1
fi
exec "$TOOL" "$@"
-1
View File
@@ -1 +0,0 @@
apecopy
-21
View File
@@ -1,21 +0,0 @@
#!/bin/sh
PROG=${0##*/}
MODE=${MODE:-${m:-aarch64}}
COSMO=${COSMO:-/opt/cosmo}
fatal_error() {
echo "$PROG: $1" >&2
exit 1
}
if [ -x "$COSMO/o/third_party/gcc/bin/aarch64-linux-musl-gcc" ]; then
BRAND=musl
else
BRAND=cosmo
fi
if [ ! -x "$COSMO/o/third_party/gcc/bin/aarch64-linux-$BRAND-ar" ]; then
fatal_error "you need to run: aarch64-unknown-cosmo-cc --update"
fi
exec "$COSMO/o/third_party/gcc/bin/aarch64-linux-$BRAND-ar" "$@"
-19
View File
@@ -1,19 +0,0 @@
#!/bin/sh
MODE=${MODE:-${m:-aarch64}}
COSMO=${COSMO:-/opt/cosmo}
COSMOS=${COSMOS:-/opt/cosmos/aarch64}
if [ -x "$COSMO/o/third_party/gcc/bin/aarch64-linux-musl-gcc" ]; then
BRAND=musl
else
BRAND=cosmo
fi
TOOL="$COSMO/o/third_party/gcc/bin/aarch64-linux-$BRAND-as"
if [ ! -x "$TOOL" ]; then
echo "$0: you need to run: aarch64-unknown-cosmo-cc --update" >&2
exit 1
fi
exec "$TOOL" "$@"
-19
View File
@@ -1,19 +0,0 @@
#!/bin/sh
MODE=${MODE:-${m:-aarch64}}
COSMO=${COSMO:-/opt/cosmo}
COSMOS=${COSMOS:-/opt/cosmos/aarch64}
if [ -x "$COSMO/o/third_party/gcc/bin/aarch64-linux-musl-gcc" ]; then
BRAND=musl
else
BRAND=cosmo
fi
TOOL="$COSMO/o/third_party/gcc/bin/aarch64-linux-$BRAND-nm"
if [ ! -x "$TOOL" ]; then
echo "$0: you need to run: aarch64-unknown-cosmo-cc --update" >&2
exit 1
fi
exec "$TOOL" "$@"
-19
View File
@@ -1,19 +0,0 @@
#!/bin/sh
MODE=${MODE:-${m:-aarch64}}
COSMO=${COSMO:-/opt/cosmo}
COSMOS=${COSMOS:-/opt/cosmos/aarch64}
if [ -x "$COSMO/o/third_party/gcc/bin/aarch64-linux-musl-gcc" ]; then
BRAND=musl
else
BRAND=cosmo
fi
TOOL="$COSMO/o/third_party/gcc/bin/aarch64-linux-$BRAND-objcopy"
if [ ! -x "$TOOL" ]; then
echo "$0: you need to run: aarch64-unknown-cosmo-cc --update" >&2
exit 1
fi
exec "$TOOL" "$@"
-19
View File
@@ -1,19 +0,0 @@
#!/bin/sh
MODE=${MODE:-${m:-aarch64}}
COSMO=${COSMO:-/opt/cosmo}
COSMOS=${COSMOS:-/opt/cosmos/aarch64}
if [ -x "$COSMO/o/third_party/gcc/bin/aarch64-linux-musl-gcc" ]; then
BRAND=musl
else
BRAND=cosmo
fi
TOOL="$COSMO/o/third_party/gcc/bin/aarch64-linux-$BRAND-objdump"
if [ ! -x "$TOOL" ]; then
echo "$0: you need to run: aarch64-unknown-cosmo-cc --update" >&2
exit 1
fi
exec "$TOOL" "$@"
-19
View File
@@ -1,19 +0,0 @@
#!/bin/sh
MODE=${MODE:-${m:-aarch64}}
COSMO=${COSMO:-/opt/cosmo}
COSMOS=${COSMOS:-/opt/cosmos/aarch64}
if [ -x "$COSMO/o/third_party/gcc/bin/aarch64-linux-musl-gcc" ]; then
BRAND=musl
else
BRAND=cosmo
fi
TOOL="$COSMO/o/third_party/gcc/bin/aarch64-linux-$BRAND-strip"
if [ ! -x "$TOOL" ]; then
echo "$0: you need to run: aarch64-unknown-cosmo-cc --update" >&2
exit 1
fi
exec "$TOOL" "$@"
-134
View File
@@ -1,134 +0,0 @@
#!/bin/sh
PROG=${0##*/}
MODE=${MODE:-$m}
COSMO=${COSMO:-/opt/cosmo}
COSMOS=${COSMOS:-/opt/cosmos}
if [ ! -f ape/loader.c ]; then
cd "$COSMO" || exit
fi
if [ "$(id -u)" -eq 0 ]; then
SUDO=
else
SUDO=sudo
fi
echo "Actually Portable Executable (APE) Installer" >&2
echo "Author: Justine Tunney <[email protected]>" >&2
################################################################################
# INSTALL APE LOADER SYSTEMWIDE
if [ -f o/depend ] && make -j8 o//ape; then
echo "successfully recompiled ape loader" >&2
elif [ -x o//ape/ape.elf ] && [ -x o//ape/ape.macho ]; then
echo "using ape loader you compiled earlier" >&2
elif [ -d build/bootstrap ]; then
# if make isn't being used then it's unlikely the user changed the sources
# in that case the prebuilt binaries should be completely up-to-date
echo "using prebuilt ape loader from cosmo repo" >&2
mkdir -p o//ape || exit
cp -af build/bootstrap/ape.elf o//ape/ape.elf || exit
cp -af build/bootstrap/ape.macho o//ape/ape.macho || exit
else
echo "no cosmopolitan libc repository here" >&2
echo "fetching ape loader from justine.lol" >&2
mkdir -p o//ape || exit
if command -v wget >/dev/null 2>&1; then
wget -qO o//ape/ape.elf https://justine.lol/ape.elf || exit
wget -qO o//ape/ape.macho https://justine.lol/ape.macho || exit
else
curl -Rso o//ape/ape.elf https://justine.lol/ape.elf || exit
curl -Rso o//ape/ape.macho https://justine.lol/ape.macho || exit
fi
chmod +x o//ape/ape.elf || exit
chmod +x o//ape/ape.macho || exit
fi
if [ "$(uname -s)" = "Darwin" ]; then
if ! [ /usr/bin/ape -nt o//ape/ape.macho ]; then
echo >&2
echo "installing o//ape/ape.elf to /usr/bin/ape" >&2
echo "$SUDO mv -f o//ape/ape.elf /usr/bin/ape" >&2
$SUDO cp -f o//ape/ape.macho /usr/bin/ape || exit
echo "done" >&2
fi
else
if ! [ /usr/bin/ape -nt o//ape/ape.elf ]; then
echo >&2
echo "installing o//ape/ape.elf to /usr/bin/ape" >&2
echo "$SUDO mv -f o//ape/ape.elf /usr/bin/ape" >&2
$SUDO cp -f o//ape/ape.elf /usr/bin/ape || exit
echo "done" >&2
fi
fi
################################################################################
# REGISTER APE LOADER WITH BINFMT_MISC TOO (LINUX-ONLY)
if [ x"$(uname -s)" = xLinux ]; then
if [ -e /proc/sys/fs/binfmt_misc/APE ]; then
echo >&2
echo it looks like APE is already registered with binfmt_misc >&2
echo To reinstall please run ape/apeuninstall.sh first >&2
echo please check that it is mapped to ape not /bin/sh >&2
echo cat /proc/sys/fs/binfmt_misc/APE >&2
cat /proc/sys/fs/binfmt_misc/APE >&2
exit
fi
if ! [ -e /proc/sys/fs/binfmt_misc ]; then
echo >&2
echo loading binfmt_misc into your kernel >&2
echo you may need to edit configs to persist across reboot >&2
echo $SUDO modprobe binfmt_misc >&2
$SUDO modprobe binfmt_misc || exit
echo done >&2
fi
if ! [ -e /proc/sys/fs/binfmt_misc/register ]; then
echo >&2
echo mounting binfmt_misc into your kernel >&2
echo you may need to edit configs to persist across reboot >&2
echo $SUDO mount -t binfmt_misc none /proc/sys/fs/binfmt_misc >&2
$SUDO mount -t binfmt_misc none /proc/sys/fs/binfmt_misc || exit
echo done >&2
fi
echo >&2
echo registering APE with binfmt_misc >&2
echo you may need to edit configs to persist across reboot >&2
echo '$SUDO sh -c "echo '"'"':APE:M::MZqFpD::/usr/bin/ape:'"'"' >/proc/sys/fs/binfmt_misc/register"' >&2
$SUDO sh -c "echo ':APE:M::MZqFpD::/usr/bin/ape:' >/proc/sys/fs/binfmt_misc/register" || exit
echo '$SUDO sh -c "echo '"'"':APE-jart:M::jartsr::/usr/bin/ape:'"'"' >/proc/sys/fs/binfmt_misc/register"' >&2
$SUDO sh -c "echo ':APE-jart:M::jartsr::/usr/bin/ape:' >/proc/sys/fs/binfmt_misc/register" || exit
echo done >&2
if [ x"$(cat /proc/sys/fs/binfmt_misc/status)" = xdisabled ]; then
echo >&2
echo enabling binfmt_misc >&2
echo you may need to edit configs to persist across reboot >&2
echo $SUDO sh -c 'echo 1 >/proc/sys/fs/binfmt_misc/status' >&2
$SUDO sh -c 'echo 1 >/proc/sys/fs/binfmt_misc/status' || exit
echo done >&2
fi
fi
################################################################################
{
echo
echo "------------------------------------------------------------------"
echo
echo "APE INSTALL COMPLETE"
echo
echo "If you decide to uninstall APE later on"
echo "you may do so using ape/apeuninstall.sh"
echo
echo "Enjoy your APE loader (>'.')>"
echo
} >&2
-60
View File
@@ -1,60 +0,0 @@
#!/bin/sh
PROG=${0##*/}
MODE=${MODE:-$m}
COSMO=${COSMO:-/opt/cosmo}
COSMOS=${COSMOS:-/opt/cosmos}
if [ ! -f ape/loader.c ]; then
cd "$COSMO" || exit
fi
if [ "$UID" = "0" ]; then
SUDO=
else
SUDO=sudo
fi
{
echo
echo "APE Uninstaller intends to run (in pseudo-shell)"
echo
echo " sudo echo -1 into /proc/sys/fs/binfmt_misc/APE*"
echo " sudo rm -f /usr/bin/ape ~/.ape /tmp/.ape # etc."
echo
echo "You may then use ape/apeinstall.sh to reinstall it"
echo
} >&2
set -ex
for f in /proc/sys/fs/binfmt_misc/APE*; do
if [ -f $f ]; then
$SUDO sh -c "echo -1 >$f" || exit
fi
done
# system installation
if [ -f /usr/bin/ape ]; then
$SUDO rm -f /usr/bin/ape
fi
# legacy installations
rm -f o/tmp/ape /tmp/ape "${TMPDIR:-/tmp}/ape"
# ad-hoc installations
for x in .ape \
.ape-1.1 \
.ape-1.3 \
.ape-1.4 \
.ape-1.5 \
.ape-1.6 \
.ape-1.7 \
.ape-1.8 \
.ape-blink-0.9.2 \
.ape-blink-1.0.0; do
rm -f \
~/$x \
/tmp/$x \
o/tmp/$x \
"${TMPDIR:-/tmp}/$x"
done
-102
View File
@@ -1,102 +0,0 @@
#!/bin/sh
PROG=${0##*/}
COSMO=${COSMO:-/opt/cosmo}
if [ x"$1" = x"--version" ]; then
cat <<EOF
$PROG 0.1
Copyright (c) 2023 Justine Alexandra Roberts Tunney
Cosmopolitan Libc and LLVM libcxx/compiler-rt are subject to non-GPL
notice licenses, e.g. ISC, MIT, etc. Your compiled programs must embed
our copyright notices. This toolchain is configured to do so default.
Cosmopolitan comes with absolutely NO WARRANTY of any kind.
For more information, see the Cosmopolitan LICENSE files.
Copyright (C) 2019 Free Software Foundation, Inc.
This launches GNU GCC/Binutils subprocesses, which is free software; see
Cosmopolitan's third_party/gcc/ for source code and copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.
EOF
exit
fi
if [ x"$1" = x"--help" ]; then
cat <<EOF
Usage: $PROG INPUT OUTPUT
Converts an executable linked by cosmocc to the Actually Portable
Executable file format. See $PROG --version for other details.
EOF
exit
fi
fatal_error() {
echo "$PROG: fatal error: $1" >&2
echo "operation terminated." >&2
exit 1
}
CROSS=1
ARCH=${PROG%%-*} # split(prog, '-')[0]
if [ x"$ARCH" = x"$PROG" ]; then
ARCH=x86_64
CROSS=0
fi
if [ x"$ARCH" = x"x86_64" ]; then
MODE=${MODE:-$m}
else
MODE=${MODE:-${m:-$ARCH}}
fi
if [ $CROSS -eq 0 ]; then
COSMOS=${COSMOS:-/opt/cosmos}
else
COSMOS=${COSMOS:-/opt/cosmos/$ARCH}
fi
if [ x"$ARCH" = x"x86_64" ]; then
OBJCOPYFLAGS="-S -O binary"
elif [ x"$ARCH" = x"aarch64" ]; then
OBJCOPYFLAGS="-S"
else
fatal_error "$ARCH: unsupported architecture"
fi
if [ ! -d "$COSMO" ]; then
fatal_error "you need to clone cosmopolitan to your $COSMO directory"
fi
if [ -x "$COSMO/o/third_party/gcc/bin/x86_64-linux-musl-gcc" ]; then
BRAND=musl
else
BRAND=cosmo
fi
if [ ! -f "$COSMO/o//tool/build/zipcopy.com" ] ||
[ ! -f "$COSMO/o/third_party/gcc/bin/$ARCH-linux-$BRAND-objcopy" ]; then
if [ $CROSS -eq 0 ]; then
fatal_error "you need to run: cosmocc --update"
else
fatal_error "you need to run: $ARCH-unknown-cosmo-cosmocc --update"
fi
fi
INPUT=$1
OUTPUT=$2
if [ ! -f "$INPUT" ]; then
fatal_error "$INPUT: missing input and output arguments"
elif [ ! -f "$INPUT" ]; then
fatal_error "$INPUT: no such file"
elif [ ! -f "$INPUT" ]; then
fatal_error "$INPUT: missing output argument"
elif [ x"$INPUT" = x"$OUTPUT" ]; then
fatal_error "$INPUT: input and output file can't be the same"
fi
"$COSMO/o/third_party/gcc/bin/$ARCH-linux-$BRAND-objcopy" \
$OBJCOPYFLAGS \
"$INPUT" \
"$OUTPUT" || exit
"$COSMO/o//tool/build/zipcopy.com" \
"$INPUT" \
"$OUTPUT" || exit
-3
View File
@@ -1,3 +0,0 @@
#!/bin/sh
COSMO=${COSMO:-/opt/cosmo}
exec "$COSMO/o//tool/build/apelink.com" "$@"
-31
View File
@@ -1,31 +0,0 @@
#!/bin/sh
COSMO=${COSMO:-/opt/cosmo}
set -- -apifCe "$@"
if [ -n "$ADDR2LINE" ]; then
exec "$ADDR2LINE" "$@"
fi
find_addr2line() {
if [ -x o/third_party/gcc/bin/$1-linux-cosmo-addr2line ]; then
ADDR2LINE=o/third_party/gcc/bin/$1-linux-cosmo-addr2line
elif [ -x o/third_party/gcc/bin/$1-linux-musl-addr2line ]; then
ADDR2LINE=o/third_party/gcc/bin/$1-linux-musl-addr2line
elif [ -x "$COSMO/o/third_party/gcc/bin/$1-linux-cosmo-addr2line" ]; then
ADDR2LINE="$COSMO/o/third_party/gcc/bin/$1-linux-cosmo-addr2line"
elif [ -x "$COSMO/o/third_party/gcc/bin/$1-linux-musl-addr2line" ]; then
ADDR2LINE="$COSMO/o/third_party/gcc/bin/$1-linux-musl-addr2line"
else
echo "error: toolchain not found (try running 'cosmocc --update' or 'make' in the cosmo monorepo)" >&2
exit 1
fi
}
for ARCH in x86_64 aarch64; do
find_addr2line $ARCH
"$ADDR2LINE" "$@" 2>/dev/null && exit
done
echo "error: addr2line failed" >&2
exit 1
-381
View File
@@ -1,381 +0,0 @@
#!/bin/sh
#
# cosmopolitan c/c++ compiler
#
# getting started
#
# sudo chmod 1777 /opt # sticky bit isn't required
# git clone https://github.com/jart/cosmopolitan /opt/cosmo
# export PATH="/opt/cosmo/bin:/opt/cosmos/bin:$PATH"
# echo 'export PATH="/opt/cosmo/bin:/opt/cosmos/bin:$PATH"' >>~/.profile
# ape-install # optionally install a faster systemwide ape loader
# cosmocc --update # pull and rebuild toolchain artifacts
#
# getting started synopsis
#
# cosmocc -o hello.com hello.c
# ./foo.com
# ./foo.com --strace
# ./foo.com --ftrace
#
# how to build a project like lua 5.4.6
#
# make all test CC=cosmocc
# src/lua -e 'print("hi")'
# make install INSTALL_TOP=/opt/cosmos
# apecopy src/lua src/lua.com # convert to portable ape binary
#
# how to cross compile a project like lua 5.4.6
#
# aarch64-unknown-cosmo-cc --update
# make clean all test CC=aarch64-unknown-cosmo-cc AR='aarch64-unknown-cosmo-ar rc' -j8
# qemu-aarch64 src/lua -e 'print("hi")'
# make install INSTALL_TOP=/opt/cosmos/aarch64
#
# building in tiny mode
#
# export MODE=tiny
# cosmocc --update
# cosmocc -Os -o foo.com foo.c
#
# building in tiniest mode (linux only)
#
# export MODE=tinylinux
# cosmocc --update
# cosmocc -Os -o foo.com foo.c
#
# hardening programs with memory safety
#
# export MODE=asan
# cosmocc --update
# cosmocc -o foo.com foo.c
#
# detecting this environment
#
# - `__COSMOCC__` is defined when this compiler is in play
# - `__COSMOPOLITAN__` is always defined by Cosmopolitan Libc
#
# some notes on this compiler
#
# - the underlying compiler itself is gcc
# - we use cosmopoiltan libc rather than glibc
# - we use llvm's compiler-rt and libcxx runtimes
# - we patched gcc so switch case can have symbols
# - our scanf() implementation is somewhat troubled
#
# compiler flags that aren't supported
#
# - `-r` partial linking not implemented yet (todo)
# - `-S` you need to put your assembly behind #ifdefs instead
# - `-fexceptions` cosmopolitan doesn't support c++ exceptions yet
# - `-frtti` cosmopolitan doesn't support c++ runtime reflection yet
# - `-mred-zone` the system v red zone doesn't exist on windows and metal
# - `-fpic`, '-fPIC', `-shared`, `-pie`, etc. no shared object support yet
# - `-fsanitize=thread` cosmopolitan doesn't have thread sanitizer runtime yet
# - `-fomit-frame-pointer` is partially supported (apple forbids full removal)
#
# for further details, run `man gcc`
PROG=${0##*/}
COSMO=${COSMO:-/opt/cosmo}
GCC_VERSION=11.2.0
if [ "$1" = "--version" ]; then
cat <<EOF
$PROG (GCC) $GCC_VERSION
Copyright (c) 2023 Justine Alexandra Roberts Tunney
Cosmopolitan Libc and LLVM libcxx/compiler-rt are subject to non-GPL
notice licenses, e.g. ISC, MIT, etc. Your compiled programs must embed
our copyright notices. This toolchain is configured to do so default.
Cosmopolitan comes with absolutely NO WARRANTY of any kind.
For more information, see the Cosmopolitan LICENSE files.
Copyright (C) 2019 Free Software Foundation, Inc.
This launches GNU GCC/Binutils subprocesses, which is free software; see
Cosmopolitan's third_party/gcc/ for source code and copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.
EOF
exit
fi
fatal_error() {
echo "$PROG: fatal error: $1" >&2
echo "compilation terminated." >&2
exit 1
}
log_command() {
if [ -n "$BUILDLOG" ]; then
printf '# %s\n(cd %s; %s)\n' "$ORIGINAL" "$PWD" "$*" >>"$BUILDLOG"
fi
}
ORIGINAL="$0 $*"
PLATFORM="-D__COSMOPOLITAN__ -D__COSMOCC__"
PREDEF="-include libc/integral/normalize.inc"
CFLAGS="-fportcosmo -fno-dwarf2-cfi-asm -fno-unwind-tables -fno-asynchronous-unwind-tables -fno-semantic-interposition"
CPPFLAGS="-fno-pie -nostdinc -fno-math-errno -iquote $COSMO"
LDFLAGS="-static -no-pie -nostdlib -fuse-ld=bfd"
APEFLAGS="-Wl,--gc-sections"
PRECIOUS="-fno-omit-frame-pointer"
CROSS=1
ARCH=${PROG%%-*} # split(prog, '-')[0]
if [ x"$ARCH" = x"$PROG" ]; then
ARCH=x86_64
CROSS=0
fi
if [ x"$ARCH" = x"x86_64" ]; then
MODE=${MODE:-$m}
else
MODE=${MODE:-${m:-$ARCH}}
fi
if [ $CROSS -eq 0 ]; then
COSMOS=${COSMOS:-/opt/cosmos}
else
COSMOS=${COSMOS:-/opt/cosmos/$ARCH}
fi
if [ -x "$COSMO/o/third_party/gcc/bin/x86_64-linux-musl-gcc" ]; then
BRAND=musl
else
BRAND=cosmo
fi
CC="$COSMO/o/third_party/gcc/bin/$ARCH-linux-$BRAND-gcc"
CRT="$COSMO/o/$MODE/libc/crt/crt.o"
LDLIBS="$COSMO/o/$MODE/cosmopolitan.a"
CPPFLAGS="$CPPFLAGS -isystem $COSMOS/include -isystem $COSMO/libc/isystem"
LDFLAGS="$LDFLAGS -L$COSMOS/lib"
if [ x"$PROG" != x"${PROG%++}" ]; then
CC="$COSMO/o/third_party/gcc/bin/$ARCH-linux-$BRAND-g++"
CFLAGS="$CFLAGS -fno-rtti -fno-exceptions -fuse-cxa-atexit"
LDLIBS="$COSMO/o/$MODE/third_party/libcxx/libcxx.a $LDLIBS"
fi
PAGESZ=4096
if [ x"$ARCH" = x"x86_64" ]; then
OBJCOPYFLAGS="-S -O binary"
CRT="$COSMO/o/$MODE/ape/ape-no-modify-self.o $CRT"
CPPFLAGS="$CPPFLAGS -mno-red-zone"
CFLAGS="$CFLAGS -mno-tls-direct-seg-refs"
LDFLAGS="$LDFLAGS -Wl,-T,$COSMO/o/$MODE/ape/ape.lds"
if [ x"$MODE" = x"aarch64" ]; then
fatal_error '$MODE must not be aarch64 when using x86_64 cross compiler'
elif [ x"$MODE" != x"${MODE#* }" ]; then
fatal_error '$MODE must not contain hyphens when targeting x86_64'
fi
elif [ x"$ARCH" = x"aarch64" ]; then
OBJCOPYFLAGS="-S"
PAGESZ=16384
CFLAGS="$CFLAGS -ffixed-x18 -ffixed-x28 -mno-outline-atomics"
LDFLAGS="$LDFLAGS -Wl,-T,$COSMO/o/$MODE/ape/aarch64.lds"
if [ x"$MODE" != x"aarch64" ] && [ x"$MODE" = x"${MODE#aarch64-*}" ]; then
fatal_error '$MODE must be either "aarch64" or "aarch64-FOO" when using aarch64 cross compiler'
fi
else
fatal_error "$ARCH: unsupported architecture"
fi
LDFLAGS="$LDFLAGS -Wl,-z,common-page-size=$PAGESZ -Wl,-z,max-page-size=16384"
if [ ! -d "$COSMO" ]; then
fatal_error "you need to clone cosmopolitan to your $COSMO directory"
fi
if [ "$1" = "--update" ]; then
cd $COSMO || exit
echo "building cosmo host toolchain..." >&2
make --silent -j toolchain MODE= ARCH=x86_64 || exit
echo "building cosmo target (MODE=$MODE) toolchain..." >&2
make --silent -j toolchain MODE="$MODE" || exit
echo "setting up your cosmos..." >&2
mkdir -p "$COSMOS/lib" || exit
for lib in c dl gcc_s m pthread resolv rt dl z stdc++; do
if [ ! -f "$COSMOS/lib/lib${lib}.a" ]; then
printf '\041\074\141\162\143\150\076\012' >"$COSMOS/lib/lib${lib}.a" || exit
fi
done
echo "successfully updated your cosmo toolchain" >&2
exit
fi
if [ ! -f "$COSMOS/lib/libc.a" ] ||
[ ! -f "$COSMO/o/$MODE/cosmopolitan.a" ]; then
fatal_error "you need to run: $PROG --update"
fi
OPT=
FIRST=1
OUTPUT=
SFLAG=0
INTENT=ld
GOT_SOME=0
NEED_OUTPUT=
RELOCATABLE=0
for x; do
if [ $FIRST -eq 1 ]; then
set --
FIRST=0
fi
if [ -n "$NEED_OUTPUT" ]; then
NEED_OUTPUT=
OUTPUT=$x
set -- "$@" "$x"
continue
fi
if [ x"$x" = x"-" ] || # is an argument
[ x"$x" = x"${x#-*}" ]; then # !startswith(x, "-")
GOT_SOME=1
elif [ x"$x" = x"-static-libstdc++" ]; then
continue
elif [ x"$x" = x"-static-libgcc" ]; then
continue
elif [ x"$x" != x"${x#-O}" ]; then
OPT=$x
elif [ x"$x" = x"-c" ]; then
INTENT=cc
elif [ x"$x" = x"-s" ]; then
SFLAG=1
continue
elif [ x"$x" = x"-r" ]; then
RELOCATABLE=1
elif [ x"$x" = x"-E" ]; then
INTENT=cpp
elif [ x"$x" = x"-o" ]; then
NEED_OUTPUT=1
elif [ x"$x" = x"-mcosmo" ]; then
CPPFLAGS="$CPPFLAGS -D_COSMO_SOURCE"
continue
elif [ x"$x" != x"${x#-o}" ]; then
OUTPUT=${x#-o}
elif [ x"$x" = x"-fpic" ]; then
continue
elif [ x"$x" = x"-fPIC" ]; then
continue
elif [ x"$x" = x"-r" ] ||
[ x"$x" = x"-pie" ] ||
[ x"$x" = x"-frtti" ] ||
[ x"$x" = x"-shared" ] ||
[ x"$x" = x"-nostdlib" ] ||
[ x"$x" = x"-mred-zone" ] ||
[ x"$x" = x"-fexceptions" ] ||
[ x"$x" = x"-fsanitize=thread" ]; then
echo "$PROG: $x not supported" >&2
exit 1
elif [ x"$x" = x"-fsanitize=all" ] ||
[ x"$x" = x"-fsanitize=address" ] ||
[ x"$x" = x"-fsanitize=undefined" ]; then
echo "$PROG: use cosmo MODE=dbg or MODE=asan rather than passing $x" >&2
exit 1
elif [ x"$x" = x"-fomit-frame-pointer" ]; then
# Quoth Apple "The frame pointer register must always address a
# valid frame record. Some functions — such as leaf functions or
# tail calls — may opt not to create an entry in this list. As a
# result, stack traces are always meaningful, even without debug
# information."
set -- "$@" -momit-leaf-frame-pointer -foptimize-sibling-calls
continue
elif [ x"$x" != x"${x#-Werror}" ] || \
[ x"$x" = x"-pedantic-errors" ]; then
# this toolchain is intended for building other people's code
# elevating warnings into errors, should only be done by devs
continue
elif [ x"$x" = x"-march=native" ]; then
if [ $CROSS -eq 0 ]; then
set -- "$@" $("$COSMO/o//tool/build/march-native.com")
else
fatal_error "-march=native can't be used when cross compiling"
fi
continue
elif [ x"$x" = x"-dumpversion" ]; then
echo $GCC_VERSION
exit 0
fi
set -- "$@" "$x"
done
if [ "$GOT_SOME" -eq 0 ]; then
fatal_error "no input files"
fi
if [ $RELOCATABLE -eq 1 ]; then
LDFLAGS="$LDFLAGS -r"
fi
if [ x"$MODE" = x"nox87" ]; then
CPPFLAGS="$CPPFLAGS -mlong-double-64"
fi
# support --ftrace unless optimizing for size
if [ x"$OPT" != x"-Os" ] && # $OPT != -Os
[ x"${MODE%tiny}" = x"${MODE}" ]; then # $MODE not in (tiny, aarch64-tiny)
if [ x"$ARCH" = x"x86_64" ]; then
CFLAGS="$CFLAGS -fpatchable-function-entry=18,16"
elif [ x"$ARCH" = x"aarch64" ]; then
CFLAGS="$CFLAGS -fpatchable-function-entry=7,6"
fi
fi
# ask compiler to generate sanitization code in debug mode
if [ x"$MODE" != x"${MODE%*dbg}" ]; then # endswith($MODE, "dbg")
if [ x"$ARCH" = x"x86_64" ]; then
CPPFLAGS="$CPPFLAGS -fsanitize=address -fsanitize=undefined"
else
CPPFLAGS="$CPPFLAGS -fsanitize=undefined"
fi
fi
# ask compiler to generate memory safety code in asan mode
if [ x"$MODE" != x"${MODE%*asan}" ]; then # endswith($MODE, "asan")
if [ x"$ARCH" = x"x86_64" ]; then
CPPFLAGS="$CPPFLAGS -fsanitize=address"
else
fatal_error "address sanitizer not supported on non-x86 yet"
fi
fi
# maximize frame pointers unless optimizing for size
if [ x"$OPT" != x"-Os" ] && # $OPT != "-Os"
[ x"$MODE" != x"${MODE%tiny}" ]; then # endswith($MODE, "tiny")
CFLAGS="$CFLAGS -fno-optimize-sibling-calls -mno-omit-leaf-frame-pointer"
fi
if [ $INTENT = cpp ]; then
set -- "$CC" $PLATFORM $CPPFLAGS "$@"
elif [ $INTENT = cc ]; then
set -- "$CC" $PLATFORM $PREDEF $CFLAGS $CPPFLAGS "$@" $PRECIOUS
else
set -- "$CC" $PLATFORM $PREDEF $CFLAGS $CPPFLAGS $CRT "$@" $LDFLAGS $LDLIBS $PRECIOUS
fi
log_command "$@"
"$@" || exit
if [ -n "$OUTPUT" ] && [ -f "$OUTPUT" ]; then
if [ $INTENT = cc ] || [ $INTENT = ld ]; then
"$COSMO/o//tool/build/fixupobj.com" \
"$OUTPUT" || exit
fi
if [ $INTENT = ld ]; then
if [ x"$OUTPUT" != x"${OUTPUT%.com}" ] ||
[ x"$OUTPUT" != x"${OUTPUT%.exe}" ]; then
# cosmocc -o foo.com ...
# -> foo.com (ape)
# -> foo.com.dbg (elf)
mv -f "$OUTPUT" "$OUTPUT.dbg" || exit
"$COSMO/o/third_party/gcc/bin/$ARCH-linux-$BRAND-objcopy" \
$OBJCOPYFLAGS \
"$OUTPUT.dbg" \
"$OUTPUT" || exit
"$COSMO/o//tool/build/zipcopy.com" \
"$OUTPUT.dbg" \
"$OUTPUT" || exit
elif [ $SFLAG -eq 1 ]; then
"$COSMO/o/third_party/gcc/bin/$ARCH-linux-$BRAND-strip" \
"$OUTPUT" || exit
fi
fi
fi
-119
View File
@@ -1,119 +0,0 @@
#!/bin/sh
PROG=${0##*/}
MODE=${MODE:-$m}
COSMO=${COSMO:-/opt/cosmo}
fatal_error() {
echo "$PROG: $1" >&2
exit 1
}
if [ -x "$COSMO/o/third_party/gcc/bin/x86_64-linux-musl-gcc" ]; then
BRAND=musl
else
BRAND=cosmo
fi
if [ ! -x "$COSMO/o/third_party/gcc/bin/x86_64-linux-$BRAND-ar" ]; then
fatal_error "you need to run: fatcosmocc --update"
fi
AR_X86_64="$COSMO/o/third_party/gcc/bin/x86_64-linux-$BRAND-ar"
AR_AARCH64="$COSMO/o/third_party/gcc/bin/aarch64-linux-$BRAND-ar"
if [ "$1" = "--version" ]; then
# note: only the underlying gnu compiler binaries are gpl
# our shell script is released with the isc license
cat <<EOF
$PROG (GNU Binutils) 2.38
Copyright (C) 2022 Free Software Foundation, Inc.
This program is free software; you may redistribute it under the terms of
the GNU General Public License version 3 or (at your option) any later version.
This program has absolutely no warranty.
EOF
exit 0
elif [ "$1" = "--help" ]; then
"$AR_X86_64" --help |
sed "s!$AR_X86_64!$PROG!g" 2>/dev/null ||
"$AR_X86_64" --help
exit
fi
FIRST=1
STATE=0
OUTPUT_X86_64=
OUTPUT_AARCH64=
INPUTS=
for x; do
if [ $FIRST -eq 1 ]; then
set --
FIRST=0
fi
if [ $STATE -eq 0 ]; then
if [ x"$x" != x"${x#--*}" ]; then # startswith(x, "--")
set -- "$@" "$x" # this is a flag
else
set -- "$@" "$x" # command argument, e.g. rcu, rcsD
STATE=1
fi
elif [ $STATE -eq 1 ]; then
OUTPUT_X86_64=$x
STATE=2
elif [ x"$x" != x"${x#* }" ]; then
fatal_error "input arguments containing spaces unsupported"
elif [ x"$x" != x"${x#@}" ]; then
fatal_error "input argument @file not supported yet"
elif [ -z "$INPUTS" ]; then
INPUTS=$x
else
INPUTS="$INPUTS $x"
fi
done
if [ -z "$OUTPUT_X86_64" ]; then
fatal_error "missing output path"
fi
mangle_object_path() {
path=$1
arch=$2
outdir=${path%/*}
outbas=${path##*/}
if [ x"$outdir" = x"$path" ]; then
outdir=
elif [ -n "$outdir" ]; then
outdir="$outdir/"
fi
if [ ! -d "$outdir.$arch" ]; then
mkdir -p "$outdir.$arch" || Exit
fi
mangled_path="${outdir}.$arch/$outbas"
}
OBJECTS_X86_64=
OBJECTS_AARCH64=
for x in $INPUTS; do
if [ ! -f "$x" ]; then
fatal_error "$x: no such file"
fi
mangle_object_path "$x" aarch64
if [ ! -f "$mangled_path" ]; then
fatal_error "$x: missing concomitant $mangled_path file"
fi
OBJECTS_X86_64="${OBJECTS_X86_64} $x"
OBJECTS_AARCH64="${OBJECTS_AARCH64} $mangled_path"
done
mangle_object_path "$OUTPUT_X86_64" aarch64
OUTPUT_AARCH64="$mangled_path"
"$AR_X86_64" "$@" "$OUTPUT_X86_64" $OBJECTS_X86_64 &
pid1=$!
"$AR_AARCH64" "$@" "$OUTPUT_AARCH64" $OBJECTS_AARCH64 &
pid2=$!
if ! wait $pid1; then
kill $pid2 2>/dev/null
wait
exit 1
fi
wait $pid2 || exit
-1
View File
@@ -1 +0,0 @@
fatcosmocc
-650
View File
@@ -1,650 +0,0 @@
#!/bin/sh
#
# fat cosmopolitan c/c++ compiler
#
# - this command is a drop-in replacement for the cc or gcc command.
# the difference is that (1) your binaries will be linked with the
# cosmopolitan c library, rather than your system specific tooling
# and (2) they'll be fat ape executables that run on the platforms
#
# * amd64
# + linux
# + macos
# + windows
# + freebsd
# + openbsd
# + netbsd
# * arm64
# + linux
# + macos
# + windows (non-native)
#
# - you need to use linux to build your binaries currently, but you
# can scp and distribute the output files to the above platforms!
#
# installation
#
# sudo chmod 1777 /opt # sticky bit isn't required
# git clone https://github.com/jart/cosmopolitan /opt/cosmo
# export PATH="/opt/cosmo/bin:/opt/cosmos/bin:$PATH"
# echo 'export PATH="/opt/cosmo/bin:/opt/cosmos/bin:$PATH"' >>~/.profile
# ape-install # optionally install a faster systemwide ape loader
# fatcosmocc --update # pull and rebuild toolchain artifacts
#
# getting started
#
# fatcosmocc -o hello.com hello.c
# ./foo.com
# unzip -vl ./foo.com
# ./foo.com --strace
# ./foo.com --ftrace
#
# building in tiny mode
#
# export MODE=tiny
# fatcosmocc --update
# fatcosmocc -Os -o foo.com foo.c
#
# building in debug mode
#
# export MODE=dbg
# fatcosmocc --update
# fatcosmocc -g -o foo.com foo.c
#
# how to build a project like lua 5.4.6
#
# make all test CC=fatcosmocc AR='fatcosmoar rcu'
# make install INSTALL_TOP=/opt/cosmos INSTALL=fatcosmoinstall
#
# how to build a project like ncurses 6.4
#
# ./configure CC=fatcosmocc \
# CXX=fatcosmoc++ \
# AR=fatcosmoar \
# INSTALL="$(command -v fatcosmoinstall)" \
# --prefix=/opt/cosmos \
# --disable-shared
# make -j8
# make install
#
# detecting this environment
#
# - `__FATCOSMOCC__` is defined by fatcosmocc
# - `__COSMOCC__` is defined by cosmocc and fatcosmocc
# - `__COSMOPOLITAN__` is always defined by cosmopolitan
#
# some notes on this compiler
#
# - the underlying compiler itself is gcc
# - we use cosmopolitan libc rather than glibc
# - we use llvm's compiler-rt and libcxx runtimes
# - we patched gcc so switch case can have symbols
# - our scanf() implementation is somewhat troubled
# - you may need to recalibrate `make -jN` as `N/2`
#
# compiler flags that work differently
#
# - `-v` will log fatcosmocc subcommands to stderr
# you can also use `export BUILDLOG=/tmp/build.log`
# - `-s` will ask apelink to not embed symbol tables in zip
# - `-E` can't be fat and runs once with x86_64 macros undefined
# - `-save-temps` will prevent deleting your arch-specific executables
#
# compiler flags that aren't supported
#
# - `-fexceptions` cosmopolitan doesn't support c++ exceptions yet
# - `-frtti` cosmopolitan doesn't support c++ runtime reflection yet
# - `-mred-zone` the system v red zone doesn't exist on windows and metal
# - `-fpic`, '-fPIC', `-shared`, `-pie`, etc. no shared object support yet
# - `-fsanitize=thread` cosmopolitan doesn't have thread sanitizer runtime yet
# - `-fomit-frame-pointer` is partially supported (apple forbids full removal)
#
# for further details, run `man gcc`
PROG=${0##*/}
COSMO=${COSMO:-/opt/cosmo}
COSMOS=${COSMOS:-/opt/cosmos}
ORIGINAL="$0 $*"
TMPDIR=${TMPDIR:-/tmp}
GCC_VERSION=11.2.0
if [ "$1" = "--version" ]; then
cat <<EOF
$PROG (GCC) $GCC_VERSION
Copyright (c) 2023 Justine Alexandra Roberts Tunney
Cosmopolitan Libc and LLVM libcxx/compiler-rt are subject to non-GPL
notice licenses, e.g. ISC, MIT, etc. Your compiled programs must embed
our copyright notices. This toolchain is configured to do so default.
Cosmopolitan comes with absolutely NO WARRANTY of any kind.
For more information, see the Cosmopolitan LICENSE files.
Copyright (C) 2019 Free Software Foundation, Inc.
This launches GNU GCC/Binutils subprocesses, which is free software; see
Cosmopolitan's third_party/gcc/ for source code and copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.
EOF
exit
fi
if [ "$1" = "--help" ]; then
if [ -t 1 ]; then
exec less "$0"
else
exec cat "$0"
fi
fi
MODE=${MODE:-$m}
if [ x"$MODE" = x"" ]; then
MODE_AARCH64=aarch64
elif [ x"$MODE" = x"tiny" ]; then
MODE_AARCH64=aarch64-tiny
elif [ x"$MODE" = x"zero" ]; then
MODE_AARCH64=aarch64-zero
elif [ x"$MODE" = x"dbg" ]; then
MODE_AARCH64=aarch64-dbg
else
echo "$PROG: build MODE=$MODE not supported by fatcosmocc" >&2
exit 1
fi
if [ "$1" = "--update" ]; then
cd $COSMO || exit
echo "building cosmo host toolchain..." >&2
make --silent -j toolchain MODE= || exit
echo "building cosmo x86_64 target (MODE=$MODE) toolchain..." >&2
make --silent -j toolchain MODE="$MODE" || exit
echo "building cosmo aarch64 target (MODE=$MODE_AARCH64) toolchain..." >&2
make --silent -j toolchain MODE="$MODE_AARCH64" || exit
echo "setting up your cosmos..." >&2
for arch in "" .aarch64/; do
mkdir -p "$COSMOS/lib/$arch" || exit
for lib in c dl gcc_s m pthread resolv rt dl z stdc++; do
if [ ! -f "$COSMOS/lib/${arch}lib${lib}.a" ]; then
printf '\041\074\141\162\143\150\076\012' >"$COSMOS/lib/${arch}lib${lib}.a" || exit
fi
done
done
echo "successfully updated your cosmo toolchain" >&2
exit
fi
if [ ! -d "$COSMO" ]; then
echo "$PROG: you need to clone cosmopolitan to your $COSMO directory" >&2
exit 1
fi
if [ ! -f "$COSMOS/lib/libc.a" ] ||
[ ! -f "$COSMOS/lib/.aarch64/libc.a" ] ||
[ ! -f "$COSMO/o/$MODE/cosmopolitan.a" ] ||
[ ! -f "$COSMO/o/$MODE_AARCH64/cosmopolitan.a" ]; then
echo "$PROG: you need to run: $PROG --update" >&2
exit 1
fi
if [ -x "$COSMO/o/third_party/gcc/bin/x86_64-linux-musl-gcc" ]; then
BRAND=musl
else
BRAND=cosmo
fi
FIXUPOBJ="$COSMO/o//tool/build/fixupobj.com"
TEMP_FILES=
SAVE_TEMPS=0
Exit() {
rc=${1:-$?}
if [ $SAVE_TEMPS -eq 0 ]; then
rm -f $TEMP_FILES
fi
exit $rc
}
show_warning() {
echo "$PROG: warning: $1" >&2
}
fatal_error() {
echo "$PROG: fatal error: $1" >&2
echo "compilation terminated." >&2
Exit 1
}
log_original() {
if [ -n "$BUILDLOG" ]; then
printf '# %s\n' "$ORIGINAL" >>"$BUILDLOG"
fi
}
log_command() {
if [ -n "$BUILDLOG" ]; then
printf '(cd %s; %s)\n' "$PWD" "$*" >>"$BUILDLOG"
fi
}
if [ x"$TMPDIR" != x"${TMPDIR#* }" ]; then
fatal_error '$TMPDIR containing spaces not supported'
elif [ ! -d "$TMPDIR" ]; then
if ! mkdir -p "$TMPDIR" 2>/dev/null; then
fatal_error "$TMPDIR: not a directory"
fi
fi
OPT=
ARGS=
FLAGS=
OUTPUT=
MCOSMO=0
INTENT=ld
NEED_JOIN=
NEED_EQUAL=
NEED_OUTPUT=
APELINKFLAGS=
INPUT_FILE_COUNT=0
for x; do
if [ x"$x" != x"${x#* }" ]; then
fatal_error "arguments containing spaces unsupported: $x"
fi
if [ -n "$NEED_OUTPUT" ]; then
NEED_OUTPUT=
OUTPUT=$x
continue
elif [ -n "$NEED_JOIN" ]; then
x="${NEED_JOIN}${x}"
NEED_JOIN=
elif [ -n "$NEED_EQUAL" ]; then
x="${NEED_EQUAL}=${x}"
NEED_EQUAL=
elif [ x"$x" = x"-" ] || # is alias for stdin
[ x"$x" = x"${x#-*}" ]; then # !startswith(x, "-")
if [ x"$x" != x"${x%.s}" ] ||
[ x"$x" != x"${x%.S}" ]; then
fatal_error "$x: assembler input files not supported"
elif [ x"$x" != x"${x%.so}" ] ||
[ x"$x" != x"${x%.dll}" ] ||
[ x"$x" != x"${x%.dylib}" ]; then
fatal_error "$x: dynamic shared object input files not supported"
elif [ x"$x" != x"-" ] && [ ! -f "$x" ]; then
fatal_error "$x: no such file"
fi
INPUT_FILE_COUNT=$((INPUT_FILE_COUNT + 1))
ARGS="$ARGS $x" # don't add to $FLAGS array
continue
elif [ x"$x" = x"-o" ]; then
NEED_OUTPUT=1
continue
elif [ x"$x" != x"${x#-o}" ]; then # startswith(x, "-o")
OUTPUT=${x#-o}
continue
elif [ x"$x" != x"${x#-O}" ]; then # startswith(x, "-O")
OPT=$x
elif [ x"$x" = x"-c" ]; then
INTENT=cc
elif [ x"$x" = x"-E" ]; then
INTENT=cpp
elif [ x"$x" = x"-s" ]; then
APELINKFLAGS="$APELINKFLAGS -s"
continue
elif [ x"$x" = x"-v" ]; then
exec 3<&2 # dup2(2, 3) b/c stderr will be redirected later
BUILDLOG=/dev/fd/3
continue
elif [ x"$x" = x"-save-temps" ]; then
SAVE_TEMPS=1
elif [ x"$x" = x"-mcosmo" ]; then
MCOSMO=1
continue
elif [ x"$x" = x"-fomit-frame-pointer" ]; then
# Quoth Apple: "The frame pointer register must always address a
# valid frame record. Some functions — such as leaf functions or
# tail calls — may opt not to create an entry in this list. As a
# result, stack traces are always meaningful, even without debug
# information."
x="-momit-leaf-frame-pointer -foptimize-sibling-calls"
elif [ x"$x" = x"-r" ] ||
[ x"$x" = x"-S" ] ||
[ x"$x" = x"-pie" ] ||
[ x"$x" = x"-frtti" ] ||
[ x"$x" = x"-shared" ] ||
[ x"$x" = x"-nostdlib" ] ||
[ x"$x" = x"-mred-zone" ] ||
[ x"$x" = x"-fexceptions" ] ||
[ x"$x" = x"-fsanitize=thread" ]; then
fatal_error "$x flag not supported"
elif [ x"$x" = x"-fsanitize=all" ] ||
[ x"$x" = x"-fsanitize=address" ] ||
[ x"$x" = x"-fsanitize=undefined" ]; then
fatal_error "use cosmo MODE=dbg rather than passing $x"
elif [ x"$x" = x"-mno-red-zone" ]; then
# "Any memory below the stack beyond the red zone is considered
# volatile and may be modified by the operating system at any time."
# https://devblogs.microsoft.com/oldnewthing/20190111-00/?p=100685
continue
elif [ x"$x" = x"-fpic" ] || [ x"$x" = x"-fPIC" ]; then
# no support for building dynamic shared objects yet. reports
# indicate that ignoring these flags, helps let autoconf know
continue
elif [ x"$x" = x"-Werror" ] || \
[ x"$x" = x"-pedantic-errors" ]; then
# this toolchain is intended for building other people's code
# elevating warnings into errors, should only be done by devs
continue
elif [ x"$x" = x"-static-libgcc" ] || \
[ x"$x" = x"-shared-libgcc" ]; then
# cosmopolitan.a always has llvm compiler runtime static code
continue
elif [ x"$x" = x"-march=native" ]; then
fatal_error "-march=native can't be used when building fat binaries"
elif [ x"$x" = x"-dumpversion" ]; then
echo $GCC_VERSION
Exit 0
elif [ x"$x" = x"-e" ] ||
[ x"$x" = x"-z" ] ||
[ x"$x" = x"-T" ] ||
[ x"$x" = x"-L" ] ||
[ x"$x" = x"-I" ] ||
[ x"$x" = x"-D" ] ||
[ x"$x" = x"-U" ] ||
[ x"$x" = x"-iquote" ] ||
[ x"$x" = x"-isystem" ] ||
[ x"$x" = x"-include" ]; then
NEED_JOIN=$x
continue
elif [ x"$x" = x"--param" ]; then
NEED_EQUAL=$x
continue
fi
FLAGS="$FLAGS $x"
ARGS="$ARGS $x"
done
if [ $INPUT_FILE_COUNT -eq 0 ]; then
fatal_error "no input files"
elif [ -z "$INPUT" ] &&
[ $INTENT != ld ] &&
[ $INPUT_FILE_COUNT -gt 1 ]; then
fatal_error "cannot specify '-o' with '-c', or '-E' with multiple files"
fi
PLATFORM="-D__COSMOPOLITAN__ -D__COSMOCC__ -D__FATCOSMOCC__"
PREDEF="-include libc/integral/normalize.inc"
CPPFLAGS="-fno-pie -nostdinc -fno-math-errno -iquote $COSMO -isystem $COSMOS/include -isystem $COSMO/libc/isystem"
CFLAGS="-fportcosmo -fno-dwarf2-cfi-asm -fno-unwind-tables -fno-asynchronous-unwind-tables -fno-semantic-interposition"
LDFLAGS="-static -nostdlib -no-pie -fuse-ld=bfd -Wl,-z,norelro -Wl,--gc-sections"
PRECIOUS="-fno-omit-frame-pointer"
if [ x"$OPT" != x"-Os" ] && [ x"$MODE" != x"tiny" ]; then
CFLAGS="$CFLAGS -fno-optimize-sibling-calls -mno-omit-leaf-frame-pointer"
fi
CC_X86_64="$COSMO/o/third_party/gcc/bin/x86_64-linux-$BRAND-gcc"
CC_AARCH64="$COSMO/o/third_party/gcc/bin/aarch64-linux-$BRAND-gcc"
if [ x"$PROG" != x"${PROG%++}" ]; then
CC_X86_64="$COSMO/o/third_party/gcc/bin/x86_64-linux-$BRAND-g++"
CC_AARCH64="$COSMO/o/third_party/gcc/bin/aarch64-linux-$BRAND-g++"
CFLAGS="$CFLAGS -fno-rtti -fno-exceptions -fuse-cxa-atexit"
fi
CRT_X86_64="$COSMO/o/$MODE/ape/ape.o $COSMO/o/$MODE/libc/crt/crt.o"
CPPFLAGS_X86_64="$CPPFLAGS -mno-red-zone"
CFLAGS_X86_64="$CFLAGS -mno-tls-direct-seg-refs"
LDFLAGS_X86_64="$LDFLAGS -L$COSMOS/lib -Wl,-T,$COSMO/o/$MODE/ape/ape.lds -Wl,-z,common-page-size=4096 -Wl,-z,max-page-size=16384"
LDLIBS_X86_64="$COSMO/o/$MODE/cosmopolitan.a"
if [ $MCOSMO -eq 1 ]; then
CPPFLAGS_X86_64="${CPPFLAGS_X86_64} -D_COSMO_SOURCE"
fi
CRT_AARCH64="$COSMO/o/$MODE_AARCH64/libc/crt/crt.o"
CPPFLAGS_AARCH64="$CPPFLAGS"
CFLAGS_AARCH64="$CFLAGS -ffixed-x18 -ffixed-x28 -mno-outline-atomics"
LDFLAGS_AARCH64="$LDFLAGS -L$COSMOS/lib/.aarch64 -Wl,-T,$COSMO/o/${MODE_AARCH64}/ape/aarch64.lds -Wl,-z,common-page-size=16384 -Wl,-z,max-page-size=16384"
LDLIBS_AARCH64="$COSMO/o/${MODE_AARCH64}/cosmopolitan.a"
if [ x"$OPT" != x"-Os" ] && [ x"$MODE" != x"tiny" ]; then
CFLAGS_X86_64="${CFLAGS_X86_64} -fpatchable-function-entry=18,16"
CFLAGS_AARCH64="${CFLAGS_AARCH64} -fpatchable-function-entry=7,6"
fi
if [ x"$PROG" != x"${PROG%++}" ]; then
LDLIBS_X86_64="$COSMO/o/$MODE/third_party/libcxx/libcxx.a ${LDLIBS_X86_64}"
LDLIBS_AARCH64="$COSMO/o/${MODE_AARCH64}/third_party/libcxx/libcxx.a ${LDLIBS_AARCH64}"
fi
if [ x"$MODE" = x"dbg" ]; then
CPPFLAGS_X86_64="${CPPFLAGS_X86_64} -fsanitize=address -fsanitize=undefined"
CPPFLAGS_AARCH64="${CPPFLAGS_AARCH64} -fsanitize=undefined"
fi
log_original
if [ $INTENT = cpp ]; then
if [ -n "$OUTPUT" ]; then
ARGS="$ARGS -o$OUTPUT"
fi
set -- \
"$CC_X86_64" \
-U__k8 \
-U__k8__ \
-U__amd64 \
-U__amd64__ \
-U__x86_64 \
-U__x86_64__ \
-U__SSE__ \
-U__SSE2__ \
-U__SSE2_MATH__ \
-mno-red-zone \
$PLATFORM \
$CPPFLAGS \
$ARGS
log_command "$@"
MODE="$MODE" exec "$@"
fi
mangle_object_path() {
path=$1
arch=$2
outdir=${path%/*}
outbas=${path##*/}
if [ x"$outdir" = x"$path" ]; then
outdir=
elif [ -n "$outdir" ]; then
outdir="$outdir/"
fi
if [ ! -d "$outdir.$arch" ]; then
mkdir -p "$outdir.$arch" || Exit
fi
mangled_path="${outdir}.$arch/$outbas"
}
mktemper() {
"$COSMO/o//tool/build/mktemper.com" \
"$TMPDIR/fatcosmocc.XXXXXXXXXXXXX$1"
}
build_object() {
out2=$(mktemper .txt) || Exit
TEMP_FILES="${TEMP_FILES} $out2"
(
set -- \
"$CC_X86_64" \
-o"$OUTPUT_X86_64" \
$PLATFORM \
$PREDEF \
$CFLAGS_X86_64 \
$CPPFLAGS_X86_64 \
"$@" \
$PRECIOUS
log_command "$@"
"$@" || exit
set -- \
"$COSMO/o//tool/build/fixupobj.com" \
"$OUTPUT_X86_64"
log_command "$@"
exec "$@"
) &
pid1=$!
(
set -- \
"$CC_AARCH64" \
-o"$OUTPUT_AARCH64" \
$PLATFORM \
$PREDEF \
$CFLAGS_AARCH64 \
$CPPFLAGS_AARCH64 \
"$@" \
$PRECIOUS &&
log_command "$@"
"$@" || exit
set -- \
"$COSMO/o//tool/build/fixupobj.com" \
"$OUTPUT_AARCH64"
log_command "$@"
exec "$@"
) 2>"$out2" &
pid2=$!
if ! wait $pid1; then
kill $pid2 2>/dev/null
wait
Exit 1
fi
if ! wait $pid2; then
echo "$PROG: x86_64 succeeded but aarch64 failed to build object" >&2
cat "$out2" >&2
Exit 1
fi
}
# turn source files into objects
LDARGS_X86_64=
LDARGS_AARCH64=
for x in $ARGS; do
if [ x"$x" != x"-" ] && # is alias for stdin
[ x"$x" != x"${x#-*}" ]; then # startswith(x, "-")
# this argument is a flag
LDARGS_X86_64="${LDARGS_X86_64} $x"
if [ x"$x" != x"${x#-L}" ]; then # startswith(x, "-L")
x="$x/.aarch64"
fi
LDARGS_AARCH64="${LDARGS_AARCH64} $x"
else
# this argument is an input file
if [ x"$x" != x"${x%.o}" ] ||
[ x"$x" != x"${x%.a}" ]; then
if [ $INTENT = cc ]; then
show_warning "$x: linker input file unused because linking not done"
else
mangle_object_path "$x" aarch64
if [ ! -f "$mangled_path" ]; then
fatal_error "$x: linker input missing concomitant $mangled_path file"
fi
LDARGS_X86_64="${LDARGS_X86_64} $x"
LDARGS_AARCH64="${LDARGS_AARCH64} $mangled_path"
fi
elif [ $INTENT = cc ]; then
if [ -n "$OUTPUT" ]; then
# e.g. `cc -c -o bar.o foo.c` is specified by user
OUTPUT_X86_64=$OUTPUT
mangle_object_path "$OUTPUT" aarch64
OUTPUT_AARCH64="$mangled_path"
build_object $FLAGS -c "$x"
else
# e.g. `cc -c dir/foo.c` builds foo.o
o=${x##*/}
OUTPUT_X86_64="${o%.*}.o"
mangle_object_path "${o%.*}.o" aarch64
OUTPUT_AARCH64="$mangled_path"
build_object $FLAGS -c "$x"
fi
else
# e.g. `cc foo.c` should build a.out
if [ -z "$OUTPUT" ]; then
OUTPUT=a.out
fi
# e.g. `cc -o foo foo.c` should *not* build foo.o
OUTPUT_X86_64=$(mktemper .o) || Exit
OUTPUT_AARCH64=$(mktemper .o) || Exit
TEMP_FILES="${TEMP_FILES} ${OUTPUT_X86_64} ${OUTPUT_AARCH64}"
build_object $FLAGS -c "$x"
LDARGS_X86_64="${LDARGS_X86_64} ${OUTPUT_X86_64}"
LDARGS_AARCH64="${LDARGS_AARCH64} ${OUTPUT_AARCH64}"
fi
fi
done
if [ $INTENT != ld ]; then
Exit
fi
OUTPUT_X86_64=$(mktemper ".com.dbg") || Exit
OUTPUT_AARCH64=$(mktemper ".aarch64.elf") || Exit
out2=$(mktemper .txt) || Exit
TEMP_FILES="${TEMP_FILES} $out2"
(
set -- \
"$CC_X86_64" \
-o"$OUTPUT_X86_64"\
$CRT_X86_64 \
$LDFLAGS_X86_64 \
$LDARGS_X86_64 \
$LDLIBS_X86_64
log_command "$@"
"$@" || exit
set -- \
"$COSMO/o//tool/build/fixupobj.com" \
"$OUTPUT_X86_64"
log_command "$@"
exec "$@"
) &
pid1=$!
(
set -- \
"$CC_AARCH64" \
-o"$OUTPUT_AARCH64"\
$CRT_AARCH64 \
$LDFLAGS_AARCH64 \
$LDARGS_AARCH64 \
$LDLIBS_AARCH64
log_command "$@"
"$@" || exit
set -- \
"$COSMO/o//tool/build/fixupobj.com" \
"$OUTPUT_AARCH64"
log_command "$@"
exec "$@"
) 2>"$out2" &
pid2=$!
if ! wait $pid1; then
kill $pid2 2>/dev/null
wait
Exit 1
fi
if ! wait $pid2; then
echo "$PROG: x86_64 succeeded but aarch64 failed to link executable" >&2
cat "$out2" >&2
Exit 1
fi
set -- \
"$COSMO/o//tool/build/apelink.com" \
-l "$COSMO/o/$MODE/ape/ape.elf" \
-l "$COSMO/o/$MODE_AARCH64/ape/ape.elf" \
-M "$COSMO/ape/ape-m1.c" \
-o "$OUTPUT" \
$APELINKFLAGS \
"$OUTPUT_X86_64" \
"$OUTPUT_AARCH64"
log_command "$@"
"$@" || Exit
set -- \
"$COSMO/o//tool/build/pecheck.com" "$OUTPUT"
log_command "$@"
"$@" || Exit
if [ $INTENT = ld ] && [ $SAVE_TEMPS -eq 0 ]; then
mv -f "$OUTPUT_X86_64" "${OUTPUT%.com}.com.dbg" || Exit
mv -f "$OUTPUT_AARCH64" "${OUTPUT%.com}.aarch64.elf" || Exit
fi
Exit
-1
View File
@@ -1 +0,0 @@
fatcosmoar
-1
View File
@@ -1 +0,0 @@
fatcosmocc
-1
View File
@@ -1 +0,0 @@
fatcosmocc
-1
View File
@@ -1 +0,0 @@
fatcosmoinstall
-19
View File
@@ -1,19 +0,0 @@
#!/bin/sh
MODE=${MODE:-$m}
COSMO=${COSMO:-/opt/cosmo}
COSMOS=${COSMOS:-/opt/cosmos}
if [ -x "$COSMO/o/third_party/gcc/bin/x86_64-linux-musl-gcc" ]; then
BRAND=musl
else
BRAND=cosmo
fi
TOOL="$COSMO/o/third_party/gcc/bin/x86_64-linux-$BRAND-addr2line"
if [ ! -x "$TOOL" ]; then
echo "$0: you need to run: x86_64-unknown-cosmo-cc --update" >&2
exit 1
fi
exec "$TOOL" "$@"
-1
View File
@@ -1 +0,0 @@
apecopy
-22
View File
@@ -1,22 +0,0 @@
#!/bin/sh
PROG=${0##*/}
MODE=${MODE:-$m}
COSMO=${COSMO:-/opt/cosmo}
fatal_error() {
echo "$PROG: $1" >&2
exit 1
}
if [ -x "$COSMO/o/third_party/gcc/bin/x86_64-linux-musl-gcc" ]; then
BRAND=musl
else
BRAND=cosmo
fi
if [ ! -x "$COSMO/o/third_party/gcc/bin/x86_64-linux-$BRAND-ar" ]; then
fatal_error "you need to run: x86_64-unknown-cosmo-cc --update"
fi
exec "$COSMO/o/third_party/gcc/bin/x86_64-linux-$BRAND-ar" "$@"
-19
View File
@@ -1,19 +0,0 @@
#!/bin/sh
MODE=${MODE:-$m}
COSMO=${COSMO:-/opt/cosmo}
COSMOS=${COSMOS:-/opt/cosmos}
if [ -x "$COSMO/o/third_party/gcc/bin/x86_64-linux-musl-gcc" ]; then
BRAND=musl
else
BRAND=cosmo
fi
TOOL="$COSMO/o/third_party/gcc/bin/x86_64-linux-$BRAND-as"
if [ ! -x "$TOOL" ]; then
echo "$0: you need to run: x86_64-unknown-cosmo-cc --update" >&2
exit 1
fi
exec "$TOOL" "$@"
-1
View File
@@ -1 +0,0 @@
cosmocc
-1
View File
@@ -1 +0,0 @@
cosmocc
-19
View File
@@ -1,19 +0,0 @@
#!/bin/sh
MODE=${MODE:-$m}
COSMO=${COSMO:-/opt/cosmo}
COSMOS=${COSMOS:-/opt/cosmos}
if [ -x "$COSMO/o/third_party/gcc/bin/x86_64-linux-musl-gcc" ]; then
BRAND=musl
else
BRAND=cosmo
fi
TOOL="$COSMO/o/third_party/gcc/bin/x86_64-linux-$BRAND-nm"
if [ ! -x "$TOOL" ]; then
echo "$0: you need to run: x86_64-unknown-cosmo-cc --update" >&2
exit 1
fi
exec "$TOOL" "$@"
-19
View File
@@ -1,19 +0,0 @@
#!/bin/sh
MODE=${MODE:-$m}
COSMO=${COSMO:-/opt/cosmo}
COSMOS=${COSMOS:-/opt/cosmos}
if [ -x "$COSMO/o/third_party/gcc/bin/x86_64-linux-musl-gcc" ]; then
BRAND=musl
else
BRAND=cosmo
fi
TOOL="$COSMO/o/third_party/gcc/bin/x86_64-linux-$BRAND-objcopy"
if [ ! -x "$TOOL" ]; then
echo "$0: you need to run: x86_64-unknown-cosmo-cc --update" >&2
exit 1
fi
exec "$TOOL" "$@"
-19
View File
@@ -1,19 +0,0 @@
#!/bin/sh
MODE=${MODE:-$m}
COSMO=${COSMO:-/opt/cosmo}
COSMOS=${COSMOS:-/opt/cosmos}
if [ -x "$COSMO/o/third_party/gcc/bin/x86_64-linux-musl-gcc" ]; then
BRAND=musl
else
BRAND=cosmo
fi
TOOL="$COSMO/o/third_party/gcc/bin/x86_64-linux-$BRAND-objdump"
if [ ! -x "$TOOL" ]; then
echo "$0: you need to run: x86_64-unknown-cosmo-cc --update" >&2
exit 1
fi
exec "$TOOL" "$@"
-19
View File
@@ -1,19 +0,0 @@
#!/bin/sh
MODE=${MODE:-$m}
COSMO=${COSMO:-/opt/cosmo}
COSMOS=${COSMOS:-/opt/cosmos}
if [ -x "$COSMO/o/third_party/gcc/bin/x86_64-linux-musl-gcc" ]; then
BRAND=musl
else
BRAND=cosmo
fi
TOOL="$COSMO/o/third_party/gcc/bin/x86_64-linux-$BRAND-strip"
if [ ! -x "$TOOL" ]; then
echo "$0: you need to run: x86_64-unknown-cosmo-cc --update" >&2
exit 1
fi
exec "$TOOL" "$@"
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
BIN
View File
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
+58
View File
@@ -0,0 +1,58 @@
--nocompress-debug-sections
--noexecstack
-Wa,--nocompress-debug-sections
-Wa,--noexecstack
-Wa,-msse2avx
-Werror=maybe-uninitialized
-Wno-literal-suffix
-Wno-unused-but-set-variable
-Wunsafe-loop-optimizations
-fbranch-target-load-optimize
-fcx-limited-range
-fdelete-dead-exceptions
-femit-struct-debug-baseonly
-ffp-int-builtin-inexact
-finline-functions-called-once
-fipa-pta
-fivopts
-flimit-function-alignment
-fmerge-constants
-fmodulo-sched
-fmodulo-sched-allow-regmoves
-fno-align-jumps
-fno-align-labels
-fno-align-loops
-fno-code-hoisting
-fno-cx-limited-range
-fno-fp-int-builtin-inexact
-fno-gnu-unique
-fno-inline-functions-called-once
-fno-instrument-functions
-fno-schedule-insns2
-fno-whole-program
-fopt-info-vec
-fopt-info-vec-missed
-freg-struct-return
-freschedule-modulo-scheduled-loops
-frounding-math
-fsched2-use-superblocks
-fschedule-insns
-fschedule-insns2
-fshrink-wrap
-fshrink-wrap-separate
-fsignaling-nans
-fstack-clash-protection
-ftracer
-ftrapv
-ftree-loop-im
-ftree-loop-vectorize
-funsafe-loop-optimizations
-fversion-loops-for-strides
-fwhole-program
-gdescribe-dies
-gstabs
-mcall-ms2sysv-xlogues
-mdispatch-scheduler
-mfpmath=sse+387
-mmitigate-rop
-mno-fentry
Binary file not shown.
BIN
View File
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Loaded 100 of 13645 files, more files were not shown because too many files have changed in this diff. Show more