Compare commits

...
467 Commits
Author SHA1 Message Date
Justine Tunney 4ae5223eae Release Cosmopolitan v3.0 2023-10-15 23:53:36 -07:00
Justine Tunney 8f82a5f968 Fix the build 2023-10-15 20:36:12 -07:00
Justine Tunney d1300623d2 Add tree command to third_party 2023-10-15 19:46:40 -07:00
Justine Tunney 7b8024d088 Supplement _GNU_SOURCE with _COSMO_SOURCE 2023-10-15 16:59:27 -07:00
Gautham e6b59bced2 Add tsearch from Musl Libc (#912)
https://git.musl-libc.org/cgit/musl/snapshot/musl-1.2.4.tar.gz
2023-10-15 16:50:54 -07:00
Justine Tunney c9fecf3a55 Make improvements
- You can now run `make -j8 toolchain` on Windows
- You can now run `make -j` on MacOS ARM64 and BSD OSes
- You can now use our Emacs dev environment on MacOS/Windows
- Fix bug where the x16 register was being corrupted by --ftrace
- The programs under build/bootstrap/ are updated as fat binaries
- The Makefile now explains how to download cosmocc-0.0.12 toolchain
- The build scripts under bin/ now support "cosmo" branded toolchains
- stat() now goes faster on Windows (shaves 100ms off `make` latency)
- Code cleanup and added review on the Windows signal checking code
- posix_spawnattr_setrlimit() now works around MacOS ARM64 bugs
- Landlock Make now favors posix_spawn() on non-Linux/OpenBSD
- posix_spawn() now has better --strace logging on Windows
- fstatat() can now avoid EACCES in more cases on Windows
- fchmod() can now change the readonly bit on Windows
2023-10-15 16:45:00 -07:00
Justine Tunney 06c6baaf50 Fix copy/paste issue in Windows console 2023-10-14 16:14:50 -07:00
Justine Tunney bd48e6c666 Fix the Landlock Make build 2023-10-14 10:40:05 -07:00
Justine Tunney cdf556e7d2 Implement signal handler tail recursion
GNU Make on Windows now appears to be working reliably. This change also
fixes a bug where, after fork the Windows thread handle wasn't reset and
that caused undefined behavior using SetThreadContext() with our signals
2023-10-14 10:38:15 -07:00
Justine Tunney a657f3e878 Delete old wcwidth() implementation
This shaves away 144kb of bss memory from every binary linking printf at
the expense of slightly increased binary footprint. Kudos for Byron Lai.
2023-10-14 03:15:05 -07:00
Justine Tunney 2db2f40a98 Rewrite special file handling on Windows
This change gets GNU grep working. What caused it to not work, is it
wouldn't write to an output file descriptor when its dev/ino equaled
/dev/null's. So now we invent special dev/ino values for these files
2023-10-14 02:53:34 -07:00
Justine Tunney aca2261cda Don't preempt WIN32 libraries
This change refactors our POSIX signals emulation for Windows so that it
performs some additional safety checks before calling SetThreadContext()
which needs to be locked and must never ever interrupt Microsoft's code.
Kudos to the the Go developers for figuring out how to do this properly.
2023-10-13 13:59:39 -07:00
Justine Tunney d1a283a588 Fix redbean LaunchBrowser() 2023-10-13 11:39:00 -07:00
Justine Tunney 3851025b77 Fix SQLite regressions caused by 3b086af91 2023-10-13 11:00:39 -07:00
Justine Tunney bdc453b22d Fix redbean "-1 workers remain" blemish 2023-10-13 08:33:09 -07:00
Justine Tunney 0e0398cd5d Fix MODE=tinylinux build 2023-10-13 08:30:18 -07:00
Justine Tunney 4bcb107cb0 Fix ctrl-c in redbean on Windows 2023-10-13 08:10:03 -07:00
Justine Tunney d458642790 Write more tests and improve kill() on Windows 2023-10-13 04:38:45 -07:00
Justine Tunney b81a1bd9a8 Fix bug with __sig_tramp() + sigaltstack() 2023-10-13 01:24:10 -07:00
Justine Tunney 49b0eaa69f Improve threading and i/o routines
- On Windows connect() can now be interrupted by a signal; connect() w/
  O_NONBLOCK will now raise EINPROGRESS; and connect() with SO_SNDTIMEO
  will raise ETIMEDOUT after the interval has elapsed.

- We now get the AcceptEx(), ConnectEx(), and TransmitFile() functions
  from the WIN32 API the officially blessed way, using WSAIoctl().

- Do nothing on Windows when fsync() is called on a directory handle.
  This was raising EACCES earlier becaues GENERIC_WRITE is required on
  the handle. It's possible to FlushFileBuffers() a directory handle if
  it's opened with write access but MSDN doesn't document what it does.
  If you have any idea, please let us know!

- Prefer manual reset event objects for read() and write() on Windows.

- Do some code cleanup on our dlmalloc customizations.

- Fix errno type error in Windows blocking routines.

- Make the futex polyfill simpler and faster.
2023-10-12 23:13:04 -07:00
Justine Tunney f7343319cc Cull the examples folder 2023-10-11 21:45:32 -07:00
Justine Tunney 3a1f887928 Introduce posix_spawn_file_actions_addchdir_np() 2023-10-11 21:45:32 -07:00
Paul Kulchenko f92ad74e6b Update pidfile processing to truncate when writing pid (#910) 2023-10-11 21:45:27 -07:00
Paul Kulchenko 6e4b9b6515 Add EncodeBase32() to Redbean (#856) 2023-10-11 20:06:20 -07:00
Justine Tunney 3b086af91b Fix issues for latest GCC toolchain 2023-10-11 14:54:42 -07:00
Justine Tunney 5cb9b2658c Remove last remnants of redbean repl thread 2023-10-11 12:27:06 -07:00
Justine Tunney 285c565051 Clean up some code 2023-10-11 11:45:31 -07:00
Justine Tunney ec3275179f Fix MODE=tiny build 2023-10-10 00:58:47 -07:00
Justine Tunney 9cc4f33c76 Fix some todos 2023-10-09 23:12:32 -07:00
Justine Tunney 9d372f48dd Fix some issues 2023-10-09 20:19:09 -07:00
Gautham 211d5d902e Changes made for cosmocc builds (#908) 2023-10-09 14:39:02 -07:00
Justine Tunney 3b4dbc9fdd Make some more fixes
This change deletes mkfifo() so that GNU Make on Windows will work in
parallel mode using its pipe-based implementation. There's an example
called greenbean2 now, which shows how to build a scalable web server
for Windows with 10k+ threads. The accuracy of clock_nanosleep is now
significantly improved on Linux.
2023-10-09 12:22:00 -07:00
Justine Tunney 820c3599ed Make some quick fixes 2023-10-08 17:56:59 -07:00
Justine Tunney 94dc7a684e Fix MODE=tiny build 2023-10-08 09:34:15 -07:00
Justine Tunney 791f79fcb3 Make improvements
- We now serialize the file descriptor table when spawning / executing
  processes on Windows. This means you can now inherit more stuff than
  just standard i/o. It's needed by bash, which duplicates the console
  to file descriptor #255. We also now do a better job serializing the
  environment variables, so you're less likely to encounter E2BIG when
  using your bash shell. We also no longer coerce environ to uppercase

- execve() on Windows now remotely controls its parent process to make
  them spawn a replacement for itself. Then it'll be able to terminate
  immediately once the spawn succeeds, without having to linger around
  for the lifetime as a shell process for proxying the exit code. When
  process worker thread running in the parent sees the child die, it's
  given a handle to the new child, to replace it in the process table.

- execve() and posix_spawn() on Windows will now provide CreateProcess
  an explicit handle list. This allows us to remove handle locks which
  enables better fork/spawn concurrency, with seriously correct thread
  safety. Other codebases like Go use the same technique. On the other
  hand fork() still favors the conventional WIN32 inheritence approach
  which can be a little bit messy, but is *controlled* by guaranteeing
  perfectly clean slates at both the spawning and execution boundaries

- sigset_t is now 64 bits. Having it be 128 bits was a mistake because
  there's no reason to use that and it's only supported by FreeBSD. By
  using the system word size, signal mask manipulation on Windows goes
  very fast. Furthermore @asyncsignalsafe funcs have been rewritten on
  Windows to take advantage of signal masking, now that it's much more
  pleasant to use.

- All the overlapped i/o code on Windows has been rewritten for pretty
  good signal and cancelation safety. We're now able to ensure overlap
  data structures are cleaned up so long as you don't longjmp() out of
  out of a signal handler that interrupted an i/o operation. Latencies
  are also improved thanks to the removal of lots of "busy wait" code.
  Waits should be optimal for everything except poll(), which shall be
  the last and final demon we slay in the win32 i/o horror show.

- getrusage() on Windows is now able to report RUSAGE_CHILDREN as well
  as RUSAGE_SELF, thanks to aggregation in the process manager thread.
2023-10-08 08:59:53 -07:00
Justine Tunney af7cb3c82f Improve Windows keyboard translation to Linux 2023-10-04 10:33:03 -07:00
Justine Tunney 982dc4db87 Fix some issues with select() 2023-10-04 09:10:58 -07:00
Justine Tunney 6918c3ffc2 Delete non-standard broken strtonum() function 2023-10-04 08:20:06 -07:00
Justine Tunney af8236264e Fix silly execve() regression 2023-10-04 08:07:07 -07:00
Justine Tunney 4631d34d0d Improve stack overflow recovery
It's now possible to use sigaltstack() to recover from stack overflows
on Windows. Several bugs in sigaltstack() have been fixed, for all our
supported platforms. There's a newer better example showing how to use
this, along with three independent unit tests just to further showcase
the various techniques.
2023-10-04 07:35:17 -07:00
Justine Tunney 1694edf85c Fix some additional Windows TTY issues 2023-10-04 02:17:25 -07:00
Justine Tunney f26a280cda Implement basic canonical mode for Windows
The `cat` command now works properly, when run by itself on the bash
command prompt. It's working beautifully so far, and is only missing
a few keystrokes for clearing words and lines. Definitely works more
well than the one that ships with WIN32 :-)
2023-10-03 22:36:22 -07:00
Justine Tunney 4825737509 Make pledge() less strict about the API
We were previously returning EINVAL but OpenBSD allows it.
2023-10-03 17:57:26 -07:00
Justine Tunney 695f74035d Use CLK_TCK for clock_nanosleep() spin threshold
This more accurately reflects how the kernels actually implement this
function and it most importantly avoids incurring startup latency.
2023-10-03 17:26:29 -07:00
Justine Tunney 11c18fa644 Make ulock stracing less noisy 2023-10-03 15:19:20 -07:00
Justine Tunney 85f64f3851 Make futexes 100x better on x86 MacOS
Thanks to @autumnjolitz (in #876) the Cosmopolitan codebase is now
acquainted with Apple's outstanding ulock system calls which offer
something much closer to futexes than Grand Central Dispatch which
wasn't quite as good, since its wait function can't be interrupted
by signals (therefore necessitating a busy loop) and it also needs
semaphore objects to be created and freed. Even though ulock is an
internal Apple API, strictly speaking, the benefits of futexes are
so great that it's worth the risk for now especially since we have
the GCD implementation still as a quick escape hatch if it changes

Here's why this change is important for x86 XNU users. Cosmo has a
suboptimal polyfill when the operating system doesn't offer an API
that let's us implement futexes properly. Sadly we had to use that
on X86 XNU until now. The polyfill works using clock_nanosleep, to
poll the futex in a busy loop with exponential backoff. On XNU x86
clock_nanosleep suffers from us not being able to use a fast clock
gettime implementation, which had a compounding effect that's made
the polyfill function even more poorly. On X86 XNU we also need to
polyfill sched_yield() using select(), which made things even more
troublesome. Now that we have futexes we don't have any busy loops
anymore for both condition variables and thread joining so optimal
performance is attained. To demonstrate, consider these benchmarks

Before:

    $ ./lockscale_test.com -b
    consumed 38.8377   seconds real time and
              0.087131 seconds cpu time

After:

    $ ./lockscale_test.com -b
    consumed 0.007955 seconds real time and
             0.011515 seconds cpu time

Fixes #876
2023-10-03 15:15:43 -07:00
Justine Tunney ff250a0c10 Revert "Rewrite ZipOS"
This reverts commit b01282e23e. Some tests
are broken. It's not clear how it'll impact metal yet. Let's revisit the
memory optimization benefits of this change again sometime soon.
2023-10-03 14:40:03 -07:00
Justine Tunney ee8a861635 Fix semaphore deadlock on Apple Silicon 2023-10-03 09:26:46 -07:00
Paul Kulchenko e8ecf31ad5 Fix handling of redbean assets without extension
Fixes #902
2023-10-03 08:25:47 -07:00
Justine Tunney a201e694ee Include missing symbols in <cstdint> for C++
Fixes #899
2023-10-03 08:19:09 -07:00
Justine Tunney 5212f78c33 Fix redbean history
Fixes #903
2023-10-03 08:14:01 -07:00
tkchiaandtkchia fcdda40f19 [metal] Fix regression causing early crash in __new_page( ) (#900)
There was a glitch in the refactoring of __map_phdrs( ) in
commit ec480f5aa0, which caused it to try to map the PT_NOTE
program segment into virtual memory, which in turn caused the
memory page at BANE to be wrongly remapped to the start of
the program image (0x100000) rather than physical address 0.
This affected subsequent page allocation operations because
the `struct mman` was located at BANE + 0x0500.

Co-authored-by: tkchia <[email protected]>
2023-10-03 07:48:55 -07:00
Justine Tunney b01282e23e Rewrite ZipOS
This reduces the virtual memory usage of Emacs for me by 30%. We now
have a simpler implementation that uses read(), rather mmap()ing the
whole executable.
2023-10-03 07:27:25 -07:00
Justine Tunney ff77f2a6af Make improvements
- This change fixes a bug that allowed unbuffered printf() output (to
  streams like stderr) to be truncated. This regression was introduced
  some time between now and the last release.

- POSIX specifies all functions as thread safe by default. This change
  works towards cleaning up our use of the @threadsafe / @threadunsafe
  documentation annotations to reflect that. The goal is (1) to use
  @threadunsafe to document functions which POSIX say needn't be thread
  safe, and (2) use @threadsafe to document functions that we chose to
  implement as thread safe even though POSIX didn't mandate it.

- Tidy up the clock_gettime() implementation. We're now trying out a
  cleaner approach to system call support that aims to maintain the
  Linux errno convention as long as possible. This also fixes bugs that
  existed previously, where the vDSO errno wasn't being translated
  properly. The gettimeofday() system call is now a wrapper for
  clock_gettime(), which reduces bloat in apps that use both.

- The recently-introduced improvements to the execute bit on Windows has
  had bugs fixed. access(X_OK) on a directory on Windows now succeeds.
  fstat() will now perform the MZ/#! ReadFile() operation correctly.

- Windows.h is no longer included in libc/isystem/, because it confused
  PCRE's build system into thinking Cosmopolitan is a WIN32 platform.
  Cosmo's Windows.h polyfill was never even really that good, since it
  only defines a subset of the subset of WIN32 APIs that Cosmo defines.

- The setlongerjmp() / longerjmp() APIs are removed. While they're nice
  APIs that are superior to the standardized setjmp / longjmp functions,
  they weren't superior enough to not be dead code in the monorepo. If
  you use these APIs, please file an issue and they'll be restored.

- The .com appending magic has now been removed from APE Loader.
2023-10-03 06:17:16 -07:00
Justine Tunney b99512ac58 Temporarily disable test_suite_x509parse.com
This test broke itself due to relying on the current time. Mocking out
gettimeofday() confirms this. However, two ipv4 subject alt name tests
still surprisingly fail even with a fake current time. We'll want this
investigated further soon.
2023-10-02 13:49:16 -07:00
Gavin Hayes c315315896 fleaks.c: initialize command buffer before calling system (#904) 2023-10-02 13:31:15 -07:00
Gavin Hayes cc8c56e814 Fix strtod NaN handling / fix SIGSEGV in testlib/showerror (#901) 2023-09-27 00:16:36 -07:00
Justine Tunney 3ffc17c50e Add Cosmopolitan to uname() 2023-09-21 23:51:55 -07:00
Justine Tunney ed316491ca Fix issue with pasting text into console 2023-09-21 13:28:52 -07:00
Justine Tunney 4f5d5a6813 Fix some more issues
- ARM Neon headers are now exported in libc/isystem/

- stat() and access() now do a better job reporting which files are
  executable which ones aren't. They do this by reading the first two
  bytes in a file to see if it's `MZ` or `#!`.
2023-09-21 11:41:42 -07:00
Justine Tunney 22cf6e11eb Add siglongjmp() for aarch64 2023-09-21 10:10:20 -07:00
Justine Tunney c88f95a892 Remove Windows executable path guessing logic
Unlike CMD.EXE, CreateProcess() doesn't care if an executable name ends
with .COM or .EXE. We now have the unbourne shell and bash working well
on Windows, so we don't need DOS anymore. Making this change will grant
us better performance, particularly for builds, because commandv() will
need to make fewer system calls. Path mangling magic still happens with
WinMain() and ntspawn() in order to do things like turn \ into / so the
interop works well at the borders. But all the code in libraries, which
did that, has been removed. It's not possible for libraries to abstract
the differences between paths.
2023-09-21 08:13:50 -07:00
Justine Tunney 0c5dd7b342 Make improvements
- Improved async signal safety of read() particularly for longjmp()
- Started adding cancel cleanup handlers for locks / etc on Windows
- Make /dev/tty work better particularly for uses like `foo | less`
- Eagerly read console input into a linked list, so poll can signal
- Fix some libc definitional bugs, which configure scripts detected
2023-09-21 07:30:39 -07:00
Justine Tunney d6c2830850 Rewrite Windows console input handling
This change removes our use of ENABLE_VIRTUAL_TERMINAL_INPUT (which
isn't very good) in favor of having read() translate Windows Console
input events to ANSI/XTERM sequences by hand. This makes it possible to
capture important keystrokes (e.g. ctrl-space) that weren't possible
before. Most importantly this change also removes the stdin/sigwinch
worker threads, which never really worked that well. Interactive TTY
sessions will now work reliably when a Cosmo process spawns or forks
another Cosmo process, e.g. unbourne.com launching emacs.com.
2023-09-19 11:53:27 -07:00
Justine Tunney ececec4c94 Fix some zipos directory related bugs 2023-09-19 02:30:42 -07:00
Justine Tunney ec480f5aa0 Make improvements
- Every unit test now passes on Apple Silicon. The final piece of this
  puzzle was porting our POSIX threads cancelation support, since that
  works differently on ARM64 XNU vs. AMD64. Our semaphore support on
  Apple Silicon is also superior now compared to AMD64, thanks to the
  grand central dispatch library which lets *NSYNC locks go faster.

- The Cosmopolitan runtime is now more stable, particularly on Windows.
  To do this, thread local storage is mandatory at all runtime levels,
  and the innermost packages of the C library is no longer being built
  using ASAN. TLS is being bootstrapped with a 128-byte TIB during the
  process startup phase, and then later on the runtime re-allocates it
  either statically or dynamically to support code using _Thread_local.
  fork() and execve() now do a better job cooperating with threads. We
  can now check how much stack memory is left in the process or thread
  when functions like kprintf() / execve() etc. call alloca(), so that
  ENOMEM can be raised, reduce a buffer size, or just print a warning.

- POSIX signal emulation is now implemented the same way kernels do it
  with pthread_kill() and raise(). Any thread can interrupt any other
  thread, regardless of what it's doing. If it's blocked on read/write
  then the killer thread will cancel its i/o operation so that EINTR can
  be returned in the mark thread immediately. If it's doing a tight CPU
  bound operation, then that's also interrupted by the signal delivery.
  Signal delivery works now by suspending a thread and pushing context
  data structures onto its stack, and redirecting its execution to a
  trampoline function, which calls SetThreadContext(GetCurrentThread())
  when it's done.

- We're now doing a better job managing locks and handles. On NetBSD we
  now close semaphore file descriptors in forked children. Semaphores on
  Windows can now be canceled immediately, which means mutexes/condition
  variables will now go faster. Apple Silicon semaphores can be canceled
  too. We're now using Apple's pthread_yield() funciton. Apple _nocancel
  syscalls are now used on XNU when appropriate to ensure pthread_cancel
  requests aren't lost. The MbedTLS library has been updated to support
  POSIX thread cancelations. See tool/build/runitd.c for an example of
  how it can be used for production multi-threaded tls servers. Handles
  on Windows now leak less often across processes. All i/o operations on
  Windows are now overlapped, which means file pointers can no longer be
  inherited across dup() and fork() for the time being.

- We now spawn a thread on Windows to deliver SIGCHLD and wakeup wait4()
  which means, for example, that posix_spawn() now goes 3x faster. POSIX
  spawn is also now more correct. Like Musl, it's now able to report the
  failure code of execve() via a pipe although our approach favors using
  shared memory to do that on systems that have a true vfork() function.

- We now spawn a thread to deliver SIGALRM to threads when setitimer()
  is used. This enables the most precise wakeups the OS makes possible.

- The Cosmopolitan runtime now uses less memory. On NetBSD for example,
  it turned out the kernel would actually commit the PT_GNU_STACK size
  which caused RSS to be 6mb for every process. Now it's down to ~4kb.
  On Apple Silicon, we reduce the mandatory upstream thread size to the
  smallest possible size to reduce the memory overhead of Cosmo threads.
  The examples directory has a program called greenbean which can spawn
  a web server on Linux with 10,000 worker threads and have the memory
  usage of the process be ~77mb. The 1024 byte overhead of POSIX-style
  thread-local storage is now optional; it won't be allocated until the
  pthread_setspecific/getspecific functions are called. On Windows, the
  threads that get spawned which are internal to the libc implementation
  use reserve rather than commit memory, which shaves a few hundred kb.

- sigaltstack() is now supported on Windows, however it's currently not
  able to be used to handle stack overflows, since crash signals are
  still generated by WIN32. However the crash handler will still switch
  to the alt stack, which is helpful in environments with tiny threads.

- Test binaries are now smaller. Many of the mandatory dependencies of
  the test runner have been removed. This ensures many programs can do a
  better job only linking the the thing they're testing. This caused the
  test binaries for LIBC_FMT for example, to decrease from 200kb to 50kb

- long double is no longer used in the implementation details of libc,
  except in the APIs that define it. The old code that used long double
  for time (instead of struct timespec) has now been thoroughly removed.

- ShowCrashReports() is now much tinier in MODE=tiny. Instead of doing
  backtraces itself, it'll just print a command you can run on the shell
  using our new `cosmoaddr2line` program to view the backtrace.

- Crash report signal handling now works in a much better way. Instead
  of terminating the process, it now relies on SA_RESETHAND so that the
  default SIG_IGN behavior can terminate the process if necessary.

- Our pledge() functionality has now been fully ported to AARCH64 Linux.
2023-09-18 21:04:47 -07:00
Gavin Hayes c4eb838516 README: note version zsh and fish were patched to support ape (#896) 2023-09-15 08:10:11 -07:00
Justine Tunney 81f391dd22 Rewrite Windows signal delivery system 2023-09-12 11:38:34 -07:00
Justine Tunney 00084577a3 Improve posix_spawn() some more 2023-09-12 08:58:57 -07:00
Justine Tunney 6430e474b4 Remove VM variable
You need to use qemu-user in binfmt_misc. For non-Linux we'll update
execve() to spawn under the appropriate blink or qemu when needed.
2023-09-12 01:27:30 -07:00
Justine Tunney 20c77338e6 Remove IMAGE_BASE_VIRTUAL 2023-09-12 01:21:36 -07:00
Justine Tunney 8a0008d985 Avoid leaking handles across processes 2023-09-12 01:07:51 -07:00
Justine Tunney a359de7893 Get rid of kmalloc()
This changes *NSYNC to allocate waiters on the stack so our locks don't
need to depend on dynamic memory. This make our runtiem simpler, and it
also fixes bugs with thread cancellation support.
2023-09-11 21:56:00 -07:00
Justine Tunney 77a7873057 Improve AARCH64 execution
This change fixes bugs in the APE loader. The execve() unit tests are
now enabled for MODE=aarch64. See the README for how you need to have
binfmt_misc configured with Qemu to run them. Apple Silicon bugs have
been fixed too, e.g. tkill() now works.
2023-09-11 14:46:46 -07:00
Justine Tunney 1965d7488e Improve performance and remove fd leaks 2023-09-10 11:52:03 -07:00
Justine Tunney 26e254fb4d Overhaul process spawning 2023-09-10 08:17:44 -07:00
Justine Tunney 99dc1281f5 Overhaul Windows signal handling
The new asynchronous signal delivery technique is now also being used
for tkill(), raise(), etc. Many subtle issues have been addresesd. We
now signal handling on Windows that's remarkably similar to the POSIX
behaviors. However that's just across threads. We're lacking a way to
have the signal semantics work well, across multiple WIN32 processes.
2023-09-08 01:49:41 -07:00
Justine Tunney 8bdaddd81d Make the Windows Console work better
The stdio reader thread now appears to be working recursively along
cosmopolitan subprocesses. For example, it's now possible to launch
vim.com from the unbourne.com bestline repl, thanks to hacks plus a
bug fix to select() timeouts.
2023-09-07 18:27:22 -07:00
Justine Tunney 032b1f3449 Implement thread cancellation for aarch64 2023-09-07 08:48:38 -07:00
Justine Tunney dcda6f7d8d Simplify the rusage.com command 2023-09-07 05:41:18 -07:00
Justine Tunney b592716d1c Reduce mandatory stack rss by 256kb 2023-09-07 04:33:01 -07:00
Justine Tunney 0e087143fd Make greenbean web server better
- Remove misguided __assert_disabled variable
- Change EPROCLIM to be EAGAIN on BSD distros
- Improve quality of greenbean with cancellations
- Fix thread race condition crash with file descriptors
2023-09-07 03:44:50 -07:00
Justine Tunney 6cff3137c5 Fix GitHub Actions 2023-09-06 23:39:47 -07:00
Justine Tunney 425c055116 Make improvements
- Polyfill readlink("foo/") dir check on Windows
- Support asynchronous signal delivery on Windows
- Restore Windows Console from execve() daisy chain
- Work around bug in AARCH64 Optimized Routines memcmp()
- Disable unbourne.com shell completion on Windows for now
- Don't always set virtual terminal input state on console
- Remove Musl Libc's unusual preservation of realpath("//")
- Make realpath() strongly link malloc() to pass configure test
- Delete cosh.com shell, now that unbourne.com works on Windows!
2023-09-06 22:48:05 -07:00
Justine Tunney f9c9a323fe Fix Landlock Make crash when TMPDIR isn't defined 2023-09-06 12:49:52 -07:00
Justine Tunney f531acc8f9 Make improvements
- Invent openatemp() API
- Invent O_UNLINK open flag
- Introduce getenv_secure() API
- Remove `git pull` from cosmocc
- Fix utimes() when path is NULL
- Fix mktemp() to never return NULL
- Fix utimensat() UTIME_OMIT on XNU
- Improve utimensat() code for RHEL5
- Turn `argv[0]` C:/ to /C/ on Windows
- Introduce tmpnam() and tmpnam_r() APIs
- Fix more const issues with internal APIs
- Permit utimes() on WIN32 in O_RDONLY mode
- Fix fdopendir() to check fd is a directory
- Fix recent crash regression in landlock make
- Fix futimens(AT_FDCWD, NULL) to return EBADF
- Use workaround so `make -j` doesn't fork bomb
- Rename dontdiscard to __wur (just like glibc)
- Fix st_size for WIN32 symlinks containing UTF-8
- Introduce stdio ext APIs needed by GNU coreutils
- Fix lstat() on WIN32 for symlinks to directories
- Move some constants from normalize.inc to limits.h
- Fix segv with memchr() and memcmp() overlapping page
- Implement POSIX fflush() behavior for reader streams
- Implement AT_SYMLINK_NOFOLLOW for utimensat() on WIN32
- Don't change read-only status of existing files on WIN32
- Correctly handle `0x[^[:xdigit:]]` case in strtol() functions
2023-09-06 12:34:59 -07:00
michalbiesek 8596e83cce Linux add definition for madvise flag (#883) 2023-09-06 12:34:54 -07:00
tkchiaandtkchia b21842ed7a [metal] Fix video mode filtering & frame buffer ref-counting (#889)
Co-authored-by: tkchia <[email protected]>
2023-09-06 03:41:07 -07:00
Justine Tunney 0d748ad58e Fix warnings
This change fixes Cosmopolitan so it has fewer opinions about compiler
warnings. The whole repository had to be cleaned up to be buildable in
-Werror -Wall mode. This lets us benefit from things like strict const
checking. Some actual bugs might have been caught too.
2023-09-01 20:50:18 -07:00
Justine Tunney e2b3c3618e Move zlib down 2023-08-31 15:17:57 -07:00
stellartux 4021cd0c1e Update redbean definitions (#887)
- Add OnServerReload signature
- Fix Pledge, Unveil, Slurp, ProgramLogPath signatures
- Add pledge flags
2023-08-26 18:40:07 -07:00
Justine Tunney 4c74f09393 Fix socket() EPROTONOSUPPORT fallback on MacOS 2023-08-22 14:34:37 -07:00
Justine Tunney 58ef4e6df8 Add i/o statistics to wait4() about child process 2023-08-21 21:25:33 -07:00
Justine Tunney b9eb656e41 Fix Windows stdin regression
The non-blocking standard input feature was broken by ec957491e.
2023-08-21 21:04:05 -07:00
Justine Tunney 7e08a97cea Further improve scanf 2023-08-21 16:55:29 -07:00
Justine Tunney 6ef2a471e4 Get GNU MPFR and MPC tests to pass
This change fixes more issues with our scanf() function.
2023-08-21 15:05:10 -07:00
Justine Tunney 63a1636e1f Get GNU GMP test suite fully passing
- Fix stdio fmemopen() buffer behaviors
- Fix scanf() to return EOF when appropriate
- Prefer fseek/ftell names over fseeko/ftello
- Ensure locale field is always set in the TIB
- Fix recent regression in vfprintf() return count
- Make %n directive in scanf() have standard behavior
2023-08-21 10:16:42 -07:00
Justine Tunney 755ae64e73 Remove long path tests for Windows 2023-08-21 04:43:53 -07:00
Justine Tunney fffcd98b0e Fix msync() flags on FreeBSD 2023-08-21 04:15:05 -07:00
Justine Tunney ebf784d4f5 Make improvements
- Introduce ualarm() function
- Make rename() report EISEMPTY on Windows
- Always raise EINVAL upon open(O_RDONLY|O_TRUNC)
- Add macro so ./configure will detect SOCK_CLOEXEC
- Fix O_TRUNC without O_CREAT not working on Windows
- Let fcntl(F_SETFL) change O_APPEND status on Windows
- Make sure pwrite() / pread() report ESPIPE on sockets
- Raise ESPIPE on Windows when pwrite() is used on pipe
- Properly compute O_APPEND CreateFile() flags on Windows
- Don't require O_DIRECTORY to open directories on Windows
- Fix more instances of Windows reporting EISDIR and ENOTDIR
- Normalize EFTYPE and EMLINK to ELOOP on NetBSD and FreeBSD
- Make unlink() / rmdir() work on read-only files on Windows
- Validate UTF-8 on Windows paths to fix bug with overlong NUL
- Always print signal name to stderr when crashing due to SIG_DFL
- Fix Windows bug where denormalized paths >260 chars didn't work
- Block signals on BSDs when thread exits before trashing its own stack
2023-08-21 02:34:17 -07:00
Justine Tunney ec957491ea Make posix_spawn faster on Windows 2023-08-20 02:50:07 -07:00
michalbiesek d6f72aa4a6 Extend pr.h header with recent options (#881) 2023-08-20 02:49:58 -07:00
michalbiesek 4f517503e4 Add PTRACE_SECCOMP_GET_METADATA (#882)
see
  linux commit: 26500475ac1b499d8636ff281311d633909f5d20
  ptrace, seccomp: add support for retrieving seccomp metadata
2023-08-20 02:49:22 -07:00
Justine Tunney 610e0d95cb Fix the MODE=tinylinux build 2023-08-19 06:56:05 -07:00
Justine Tunney 965516e313 Make improvements for Actually Portable Emacs
- Get SIGWINCH working again on the New Technology
- Correctly handle O_NOFOLLOW in open() on Windows
- Implement synthetic umask() functionality on Windows
- Do a better job managing file execute access on Windows
- Fill in `st_uid` and `st_gid` with username hash on Windows
- Munge UNICODE control pictures into control codes on Windows
- Do a better job ensuring Windows console settings are restored
- Introduce KPRINTF_LOG environment variable to log kprintf to a file
2023-08-19 06:44:58 -07:00
Justine Tunney 9c7b81ee0f Give Emacs another performance boost 2023-08-18 09:34:14 -07:00
Justine Tunney 5b42c810a5 Add filesystem index to ZipOS
This change brings the /zip/... read-only filesystem into performance
parity with the native Linux filesystem which doesn't use compression
therefore, imagine how much faster this could be with bloom filtering
rather than simple binary search, and if we used zstd instead of zlib
2023-08-18 07:11:48 -07:00
Justine Tunney 7100b1cf91 Get Fat Emacs working in Windows Console 2023-08-18 05:00:30 -07:00
Justine Tunney bf835de612 Get Fat Emacs working on Apple Silicon 2023-08-17 22:01:42 -07:00
Justine Tunney 3f9b39883f Make fat ape binaries smaller again 2023-08-17 11:15:58 -07:00
Justine Tunney 1d8937d528 Mint APE Loader v1.7
This change reduces the memory requirements of your APE Loader by 10x,
in terms of virtual memory size, thanks to the help of alloca(). We're
also now creating argument blocks with the same layout across systems.
2023-08-17 09:04:50 -07:00
Paul Kulchenko d967a94c9a Update ProgramDirectory to prepend Lua path instead of appending (#862). (#864)
* Update ProgramDirectory to prepend Lua path instead of appending (#862).

* Update redbean ProgramDirectory to raise Lua error for easier handling

* Update redbean default Lua path handling to simplify
2023-08-17 00:43:11 -07:00
Paul Kulchenko d0d027810a Add OnServerReload to replace /.reload.lua with a bit more flexibility (#851) 2023-08-17 00:42:23 -07:00
Justine Tunney 3a9cac4892 Fix small matters and improve sysconf()
- Fix mkdeps.com out of memory error
- Remove static memory from __get_cpu_count()
- Add support for passing hyphen to cat in cocmd
- Change more ZipOS errors from ENOTSUP to EROFS
- Specify mem_unit in sysinfo() output on BSD OSes
2023-08-17 00:32:11 -07:00
Gautham eebc24b9cd get rsync to build (#886) 2023-08-16 21:41:36 -07:00
Justine Tunney 04bd488da7 Introduce COSMOPOLITAN_DISABLE_ZIPOS environ var 2023-08-16 21:04:16 -07:00
Justine Tunney 8d1c81ac9f Emulate ENOTDIR better 2023-08-16 20:11:23 -07:00
Justine Tunney b76b2be2d0 Improve zip read-only filesystem
readdir() will now always yield an inode that's consistent with stat()
on ZipOS and Windows in general. More APIs have been updated to return
the appropriate error code when inappropriately trying to do ops, like
sockets, with a zip file descriptor. The path normalization algorithms
are now fully fleshed out. Some socket APIs have been fixed so they'll
raise EBADF vs. ENOTSOCK appropriately. Lastly seekdir() will now work
properly on NetBSD and FreeBSD (not sure why anyone would even use it)
2023-08-16 17:52:12 -07:00
Justine Tunney dc6c67256f Remove old stack code and improve dirstream 2023-08-16 07:54:40 -07:00
Justine Tunney 74caabb823 Fix getopt again 2023-08-15 20:06:19 -07:00
Justine Tunney 507d7a0b0b Fix stack memory, undefined behavior, etc. 2023-08-15 19:10:08 -07:00
Justine Tunney 110559ce6a Make ZipOS and Qemu work better
This change improves the dirstream library in a lot of respects,
especially for /zip/... files. Also turn off MAP_STACK on Aarch64
because Qemu seems to implement it differently than Linux and it's
probably responsible for a lot of mysterious crashes.
2023-08-15 18:32:50 -07:00
Justine Tunney 4658ae539f Remove old Windows thread hack from Redbean 2023-08-15 06:06:43 -07:00
Justine Tunney de3f3a9e5a Allocate explicit stack on aarch64 2023-08-15 04:40:19 -07:00
Justine Tunney 0e586c834a Refactor fatcosmocc into a single file 2023-08-14 22:26:17 -07:00
Justine Tunney b494d56549 Fix the cosmocc ar commands 2023-08-14 20:12:51 -07:00
Gautham f5b39e9f31 Fix typo in cosmocc (#884) 2023-08-13 23:31:39 -07:00
Justine Tunney 2b67e15b3d Fix MODE=tinylinux build 2023-08-13 22:59:56 -07:00
Justine Tunney 9c0821def7 Make stdin pollable on Windows
You can now play Super Mario Bros in CMD.EXE using Cosmopolitan! This is
thanks to a new worker thread that's spawned on Windows whenever any one
of poll(), select(), or ioctl(FIONREAD) is linked.
2023-08-13 22:42:25 -07:00
Justine Tunney ef6387ee5e Polyfill the 32-bit _bsr() function 2023-08-13 21:13:32 -07:00
Justine Tunney c776a32f75 Replace COSMO define with _COSMO_SOURCE
This change might cause ABI breakages for /opt/cosmos. It's needed to
help us better conform to header declaration practices.
2023-08-13 20:55:04 -07:00
Justine Tunney a033b65a33 Fix apelink shell script corruption bug
We were using a shell heredoc value '@' to terminate the dos stub, but
that's not sufficiently safe. We found out sh doesn't consider control
characters as contributing to the start of a line, and had the unlucky
chance of the linker choosing the number 2624 for e_lfanew, and that's
"@\n" in ASCII, which compromised the APE shell script.

We now use the heredoc 'justineXXXXXX' with 31 bits of entropy, that's
determistically generated by hashing apelink inputs w/ crc32 / blake2b
2023-08-13 17:42:11 -07:00
Justine Tunney 2366848db5 Perform some more touchups to toolchain 2023-08-13 15:03:09 -07:00
Justine Tunney d1b937bf1d Make cosmocc capable of cross compilation 2023-08-13 14:40:22 -07:00
Justine Tunney 504a4bbf84 Fix --ftrace with cosmocc 2023-08-13 08:25:36 -07:00
Justine Tunney 5d5b4ce691 Add ANSI version of bsr/bsf 2023-08-13 08:19:00 -07:00
Justine Tunney b40d41085d Fix the build 2023-08-13 07:17:40 -07:00
Justine Tunney 6942d7b820 Fix bug with temporary files on Windows 2023-08-13 07:10:33 -07:00
Justine Tunney ab9a284640 Further improve fatcosmocc 2023-08-13 01:51:39 -07:00
Justine Tunney 3f2f0e3a74 Make fatcosmocc good enough to build ncurses 6.4 2023-08-12 22:30:05 -07:00
Justine Tunney 399d14aadf Make fatcosmocc good enough to build Lua 5.4.6
make all test CC=fatcosmocc AR='fatcosmoar rcu'

This change introduces a program named mktemper.com which provides more
reliable and secure temporary file name generation for scripts. It also
makes our ar.com program more permissive in what commands it'll accept.
The cosmocc command is improved by this change too.
2023-08-12 16:44:04 -07:00
Justine Tunney 566cb5963f Make assimilate.com better
It's now safer to run. It'll now remove FreeBSD from the ELF os/abi so
that GDB is happier.
2023-08-12 07:46:24 -07:00
Justine Tunney f491276b62 Add support for C++ thread safe statics 2023-08-12 07:45:32 -07:00
Justine Tunney 60f2ea36ca Make default localtime() zone /etc/localtime 2023-08-12 07:44:15 -07:00
Justine Tunney e11fa30791 Move zipos into runtime package
This way complex runtime features (e.g. ftrace, symbol tables) can
always yoink zipos support. This is important now that apelink.com
automates embedding symbol tables for multiple cpus.
2023-08-11 23:14:02 -07:00
Justine Tunney d53c335a45 Introduce new fatcosmocc command
This new script is an alternative to the `cosmocc` command. It's still a
work in progress. It abstracts all the gory details of building separate
copies of your executable and then running the apelink.com program.
2023-08-11 22:52:11 -07:00
Justine Tunney 8fc778162e Make stderr go faster
This change makes _IONBF (unbuffered) stdio handles go 20x faster for
certain kinds of formatting directives by being smarter about buffers
2023-08-11 11:56:35 -07:00
Justine Tunney 2cbd09b4d4 Fix the build 2023-08-11 09:49:39 -07:00
Justine Tunney 0105e3e2b6 Introduce new linker for fat ape binaries 2023-08-11 04:39:19 -07:00
Michael Lenaghan e3c456d23a Don’t lowercase the test filename (#871)
This test fails if Cosmo builds from a path that contains an uppercase character.

Paths with uppercase characters aren’t so common in server Linux. But they are in *desktop* Linux. Guess how I…?

But I digress.

The real problem is that the path is lowercased on one line, but not the next:

```
        self.file_name = support.TESTFN.lower()
        self.file_path = FakePath(support.TESTFN)
```

Given that no other test in the suite lowercases `support.TESTFN`, I opted to remove it from the first line rather than adding it to the second.
2023-08-09 21:13:03 -07:00
Justine Tunney dd53f31147 Introduce post-linker that converts ELF to PE
If you build a static ELF executable in `ld -q` mode (which leaves rela
sections inside the binary) then you can run it through the elf2pe.com
program afterwards, which will turn it into a PE executable. We have a
new trick for defining WIN32 DLL imports in C without any assembly code.
This also achieves the optimally tiny and perfect PE binary structure.

We need this because it isn't possible to have a GNU ld linker script
generate a PE file where the virtual pointer and the file pointer can
drift apart. This post-linker can do that. One cool benefit is we can
now use a smaller 512-byte alignment in the file, and an even bigger
64kb alignment for the segment virtual addresses, and the executable
ends up being smaller.

Another program introduced by this change is pecheck.com which can do
extensive linting of PE static executables to help explain why Windows
won't load it.
2023-08-09 18:46:06 -07:00
Justine Tunney 2f35bbf046 Fix Windows stack size 2023-08-09 00:51:30 -07:00
Justine Tunney 9dd50f7dfc Better document PE headers 2023-08-09 00:29:01 -07:00
Justine Tunney 50394064d7 Invent systemvpe() function
It goes 5x faster than system() and it's safer too.
2023-08-09 00:27:55 -07:00
Justine Tunney 1a5ef5ba13 Mint APE Loader 1.6
This change fixes a bug with loading pure bss program headers.
2023-08-09 00:27:26 -07:00
Justine Tunney c48ee8e4fe Fix POLLIN|POLLOUT when together on Windows 2023-08-08 10:33:52 -07:00
Justine Tunney 33d280c8ba Improve Windows Console I/O
- Blocking read operations on the Windows Console can now EINTR
- Blocking read operations on Windows pipes now EINTR more reliably
- setitimer() will no longer be inherited across fork() on Windows
- It's now possible to use ECHO when the console is in raw mode
- The ECHOCTL flag now works correctly on the Windows Console
- The ICRNL flag now works correctly on the Windows Console
- pread() and pwrite() will now raise ESPIPE on Windows
- Opening /dev/tty on Windows is improved (untested)
- Overlapped I/O is now implemented in a better way
2023-08-08 05:44:40 -07:00
Justine Tunney decf216655 Perform inconsequential code cleanup 2023-08-07 20:24:50 -07:00
Justine Tunney 929478c524 Clean up more WIN32 APIs 2023-08-01 14:17:03 -07:00
Gautham 44b88d659d Fix x-axis blurring in nesemu1 tty conversion (#878) 2023-08-01 08:39:27 -07:00
Joel Yliluoma f729810f85 Losslessly recompress PNG files (#877)
Original sizes:

 639082 libc/dns/dns.png
  21281 usr/share/img/honeybadger.png
   4710 third_party/python/Lib/idlelib/Icons/idle_48.png
   2542 third_party/python/Lib/idlelib/Icons/idle_32.png
   1264 third_party/python/Lib/idlelib/Icons/idle_16.png
   1020 third_party/python/Lib/test/imghdrdata/python.png

Updated sizes:

 484841 libc/dns/dns.png
  17737 usr/share/img/honeybadger.png
   3041 third_party/python/Lib/idlelib/Icons/idle_48.png
   1606 third_party/python/Lib/idlelib/Icons/idle_32.png
    701 third_party/python/Lib/idlelib/Icons/idle_16.png
    602 third_party/python/Lib/test/imghdrdata/python.png
2023-07-31 17:59:16 -07:00
Justine Tunney 712b1aab3c Make quick fix for aarch64 build 2023-07-30 15:06:28 -07:00
Justine Tunney c8aa33e0e2 Improve wait statuses
This change has the insight that dwExitCode isn't an exit code but
rather should be used to pass the wait status. This lets us report
killing as a termination status, similar to UNIX. This change also
fixes the fact that exit(259) on Windows will break the parent due
way WIN32 is designed. We now work around that.

It turns out that NetBSD and OpenBSD, will let you have exit codes
beyond 255. This change will let you use them when it's possible.
2023-07-30 14:51:37 -07:00
Justine Tunney d9d5f45e2d Restore missing cosmopolitan documentation on website 2023-07-30 11:07:53 -07:00
Justine Tunney 58352df0a4 Make forking off threads reliable on Windows
This change makes posix_spawn_test no longer flaky on Windows, by (1)
fixing a race condition in wait(), and (2) removing a misguided vfork
implementation which was letting Windows bypass pthread_atfork().
2023-07-30 09:32:41 -07:00
Justine Tunney 2ebc5781a1 Fix flakes in runitd and popen_test 2023-07-30 04:26:34 -07:00
Justine Tunney 801224df67 Support symbol tables with arch specific name 2023-07-29 23:50:15 -07:00
Justine Tunney bd49ea1c3a Fix tests on aarch64 2023-07-29 19:02:25 -07:00
Justine Tunney 18bb5888e1 Make more fixes and improvements
- Remove PAGESIZE constant
- Fix realloc() documentation
- Fix ttyname_r() error reporting
- Make forking more reliable on Windows
- Make execvp() a few microseconds faster
- Make system() a few microseconds faster
- Tighten up the socket-related magic numbers
- Loosen restrictions on mmap() offset alignment
- Improve GetProgramExecutableName() with getenv("_")
- Use mkstemp() as basis for mktemp(), tmpfile(), tmpfd()
- Fix flakes in pthread_cancel_test, unix_test, fork_test
- Fix recently introduced futex stack overflow regression
- Let sockets be passed as stdio to subprocesses on Windows
- Improve security of bind() on Windows w/ SO_EXCLUSIVEADDRUSE
2023-07-29 18:44:15 -07:00
Justine Tunney 140a8a52e5 Remove 5000 WIN32 APIs 2023-07-28 14:32:34 -07:00
Justine Tunney 6c7b4fcbd3 Delete reference to old echo program 2023-07-28 14:10:22 -07:00
Justine Tunney 06082c7d37 Polyfill fchmodat() 2023-07-28 07:41:43 -07:00
Justine Tunney 7926aa8bfa Remove ELF binaries from tests 2023-07-28 07:20:57 -07:00
Justine Tunney 5018171fa5 Fix a bunch of Windows bugs reported on Discord
This change addresses everything from stack smashing to %SYSTEMROOT%
breaking socket(). Issues relating to compile.com not reporting text
printed to stderr has been resolved for Windows builds.
2023-07-28 06:17:34 -07:00
Justine Tunney b0e3258709 Fix close(1) bug on Windows 2023-07-28 03:56:56 -07:00
Justine Tunney 83341a4269 Remove hints from Windows imports 2023-07-27 14:09:07 -07:00
Justine Tunney 7e0a09feec Mint APE Loader v1.5
This change ports APE Loader to Linux AARCH64, so that Raspberry Pi
users can run programs like redbean, without the executable needing
to modify itself. Progress has also slipped into this change on the
issue of making progress better conforming to user expectations and
industry standards regarding which symbols we're allowed to declare
2023-07-26 13:54:49 -07:00
Justine Tunney 6843150e0c Mint APE Loader v1.4
This change also incorporates more bug fixes and improvements to a wide
variety of small things. For example this fixes #860 so Windows console
doesn't get corrupted after exit. An system stack memory map issue with
aarch64 has been fixed. We no longer use O_NONBLOCK on AF_UNIX sockets.
Crash reports on Arm64 will now demangle C++ symbols, even when c++filt
isn't available. Most importantly the Apple M1 version of APE Loader is
brought up to date by this change. A prebuilt unsigned binary for it is
being included in build/bootstrap/. One more thing: retrieving the term
dimensions under --strace was causing the stack to become corrupted and
now that's been solved too. PSS: We're now including an ELF PT_NOTE for
APE in the binaries we build, that has the APE Loader version.
2023-07-25 05:48:08 -07:00
Michael Lenaghan 53d3f9d9c5 Change the default TZ value from GST to GMT #861 (#867)
With this change, `unix.localtime()` will match `unix.gmtime()`
when `TZ` hasn't been defined.
2023-07-24 20:40:15 -07:00
Andy George 9c79c5682c Add sh code tagging (#855) 2023-07-24 20:15:18 -07:00
Justine Tunney e0c2b91b3e Remove _Hide keyword
It never did anything and isn't worthwhile as documentation.
2023-07-24 08:34:58 -07:00
Justine Tunney 4fb6cbc1fe Revert "Polyfill ENOTSOCK in getsockname() and getpeername()"
This reverts commit 925219bdf3.
2023-07-24 03:09:43 -07:00
Justine Tunney 925219bdf3 Polyfill ENOTSOCK in getsockname() and getpeername() 2023-07-24 02:11:25 -07:00
Justine Tunney eb84a25994 Check for interrupts more eagerly on Windows 2023-07-24 02:03:29 -07:00
Justine Tunney 94ea34367a Tune the page sizes 2023-07-24 00:49:06 -07:00
Justine Tunney ff198e7577 Introduce GetElfSegmentAddress() function 2023-07-23 17:36:18 -07:00
Justine Tunney 3d172c99fe Mint APE Loader v1.3
This version has better error messages and safety checks. It supports
loading static position-independent executables. It correctly handles
more kinds of weird ELF program header layouts. A force flag has been
added to avoid system execve(). Finally the longstanding misalignment
with our ELF PT_NOTE section has been addressed.
2023-07-23 17:08:14 -07:00
Justine Tunney 82b1e61443 Fix Landlock Make build config issue 2023-07-23 16:43:22 -07:00
Justine Tunney 0ba3199915 Fix some more socket bugs
- The functions that return a sockaddr now do so the same way the Linux
  Kernel does across platforms, e.g. getpeername(), accept4()

- Socket system calls on Windows will now only check for interrupts when
  a blocking operation needs to be performed.

- Write tests for recvfrom() system call
2023-07-23 16:31:10 -07:00
Danny Robinson ac92f25296 Update definitions.lua with latest progress (#852)
* Update definitions.lua with latest progress

* Remove duplicate definition
2023-07-23 11:14:55 -07:00
tkchiaandtkchia 0ffc0dd461 Let lz4toasm accept LZ4 files that lack extracted-size field (#858)
lz4toasm should now more easily accept LZ4 files output by
compressor programs that do not support the extracted-size
field, such as Stephan Brumme's smallz4.

This patch also proposes to add a new lz4len() function to
the libc: it parses an LZ4 compressed block to compute the
unpacked content size, without really unpacking the block.

Co-authored-by: tkchia <[email protected]>
2023-07-23 11:12:22 -07:00
Gautham 3eec69f9c3 Simplify cosmocc builds (#863)
- tcgetpgrp(STDIN_FILENO) should be equal to getpgrp() on Windows also,
  found while reading wget source code which uses this check to decide
  whether to print to stderr or to a file
- IN6_ADDR_ARE_EQUAL is a comparison macro used when IPV6 is allowed,
  found while reading CPython3.11 source code
- the changes in signal.h and addition of ucontext.h are because
  CPython3.11 source code expect sigaltstack to be available
- the sqlite3.mk change is because CPython3.11 requires sqlite3 to be
  built with -DOMIT_SHARED_CACHE
- unistd.h has getopt.h now, because some libraries like it there
2023-07-23 11:11:08 -07:00
Justine Tunney f83eb440f7 Incorporate more small improvements 2023-07-23 10:57:18 -07:00
Justine Tunney 1d4eb08fa1 Support non-blocking i/o across platforms
This change introduces new tests for `O_NONBLOCK` and `SOCK_NONBLOCK` to
confirm that non-blocking i/o is now working on all supported platforms,
including Windows. For example, you can now say on Windows, MacOS, etc.:

    socket(AF_INET, SOCK_STREAM | SOCK_NONBLOCK, IPPROTO_TCP);

To create a non-blocking IPv4 TCP socket. Or you can enable non-blocking
i/o on an existing socket / pipe / etc. file descriptor by calling fcntl

    fcntl(fd, F_SETFL, fcntl(fd, F_GETFL) | O_NONBLOCK);

This functionality is polyfilled on older Linux kernels too, e.g. RHEL5.
Now that fcntl() support is much better the FIOCLEX / FIONCLEX polyfills
for ioctl() have been removed since they're ugly non-POSIX diameond APIs

This change fixes a weakness in kprintf() that was causing Windows trace
tools to frequently crash.
2023-07-23 02:56:47 -07:00
Justine Tunney 5c9e03e3e0 Temporarily turn down redbean StoreAsset() feature 2023-07-22 15:51:44 -07:00
Justine Tunney 18536950b3 Fix the build 2023-07-11 05:48:39 -07:00
Justine Tunney 1ee2e89326 Make improvements
- This commit mints a new release of APE Loader v1.2 which supports
  loading ELF programs with a non-contiguous virtual address layout
  even though we've never been able to take advantage of it, due to
  how `objcopy -SO binary` fills any holes left by PT_LOAD. This'll
  change soon, since we'll have a new way of creating APE binaries.

- The undiamonding trick with our ioctl() implementation is removed
  since POSIX has been killing ioctl() for years and they've done a
  much better job. One problem it resolves, is that ioctl(FIONREAD)
  wasn't working earlier and that caused issues when building Emacs
2023-07-11 04:41:50 -07:00
Justine Tunney a1b1fdd1a4 Reconfigure GitHub Actions 2023-07-10 12:17:18 -07:00
Justine Tunney a2d269dc38 Brush up some more code 2023-07-10 10:17:26 -07:00
Justine Tunney ee6566a152 Further improve usability of cosmocc
- Support -s flag in cosmocc
- Support posix_spawn() setsid() feature
- Disable monorepo debug path prefix stripping
2023-07-10 05:55:00 -07:00
Justine Tunney f7ae50462a Make improvements
- Fix unused local variable errors
- Remove yoinks from sigaction() header
- Add nox87 and aarch64 to github actions
- Fix cosmocc -fportcosmo in linking mode
- It's now possible to build `make m=llvm o/llvm/libc`
2023-07-10 04:35:14 -07:00
Justine Tunney 3dc86ce154 Fix breakages in Linux-only build modes
- compile.com now polyfills -march=native which gcc/clang removed
- Guarantee zero Windows code is linked into non-Windows binaries
- MODE=tinylinux binaries are now back to being as tiny as ~4kb
- Improve the runtime's stack allocation / alignment hack
- GitHub Actions now tests Linux modes for assurance
2023-07-09 19:51:44 -07:00
Justine Tunney 0e4c828a8e Fix alignment of ftrace hook 2023-07-09 16:52:08 -07:00
Justine Tunney e1316d6565 Fix chibicc build breakage 2023-07-09 15:00:33 -07:00
Justine Tunney d3cf9d4ef1 Use long as implicit type in chibicc
This makes it possible to write 64-bit C without headers and prototypes.
2023-07-09 10:11:06 -07:00
Justine Tunney 42ba9901e4 Fix some behavioral issues on Windows 2023-07-09 09:59:22 -07:00
Justine Tunney 41396ff48a Make fixes and improvements
- Fix handling of precision in hex float formatting
- Enhance the cocmd interpreter for system() and popen()
- Manually ran the Lua unit tests, which are now passing
- Let stdio i/o operations happen when file is in error state
- We're now saving and restoring xmm in ftrace out of paranoia
2023-07-09 05:21:11 -07:00
Justine Tunney 95fbdb4f76 Have function tracer save all registers 2023-07-08 12:04:44 -07:00
Justine Tunney 1cc8ff6a55 Fix some aarch64 build issues 2023-07-08 10:47:45 -07:00
Justine Tunney 9ae230afad Fix more build configuration errors 2023-07-08 09:08:13 -07:00
Justine Tunney a75175fe94 Make build hermetically sealed again
It turned out that Landlock Make hasn't been applying sandboxing for a
while, due to a mistyped if statement for `$(USE_SYSTEM_TOOLCHAIN)` it
should have had the opposite meaning. Regressions in the build configs
have been fixed. The rmrf() function works better now. The rm.com tool
works according to POSIX with the exception of supporting prompts.
2023-07-08 07:06:25 -07:00
Justine Tunney 0c43c98de1 Add libcxx filesystem modules 2023-07-08 02:40:17 -07:00
Justine Tunney a092fda388 Make some fixes for libcxx
This change figures out some of the build configuration issues we've
been having with libcxx. The c++ span header is added. Per a Discord
discussion we're now turning off `-g` for the default build mode, so
consider using `make MODE=dbg` or `make MODE=zero` for GDB debugging
which works much better than `MODE=` ever has. Note that the default
build mode has always had very good function call / system call logs
plus you can still use ShowCrashReports() for backtrace. Making this
change ensures cosmocc will better conform to FOSS norms. Lastly the
LoadZipArgs() API has been added to cosmopolitan.a and <cosmo.h>.
2023-07-07 19:35:58 -07:00
Justine Tunney 3a8579252d Move demangling utility to stdio
Closes #848
2023-07-07 10:47:50 -07:00
Farid Zakaria a1dd777f22 add support for __cxa_demangle
This adds support for __cxa_demangle through the cxxabi.h file.

At the moment it is the only symbol included.
The source was taken from FreeBSD
contrib/libcxxrt/libelftc_dem_gnu3.c
2176c9ab71c85efd90a6c7af4a9e04fe8e3d49ca

FreeBSD does say this is also almost verbatim from ELFToolkit
2023-07-07 10:29:03 -07:00
Justine Tunney 0d3c1c8b1a Do work on curl/mbedtls/zstd
This change fixes stderr to be unbuffered. Added hardware AES on ARM64
to help safeguard against timing attacks. The curl.com command will be
somewhat more pleasant to use.
2023-07-07 10:13:35 -07:00
Justine Tunney a186143f62 Add EncodeHex() and DecodeHex() to Redbean 2023-07-06 15:38:08 -07:00
Justine Tunney 00acd81b2f Delete more dead code 2023-07-06 09:12:28 -07:00
Justine Tunney 0a24b4fc3c Clean up more code
The *NSYNC linked list API is good enough that it deserves to be part of
the C libray, so this change writes an improved version of it which uses
that offsetof() trick from the Linux Kernel. We vendor all of the *NSYNC
tests in third_party which helped confirm the needed refactoring is safe

This change also deletes more old code that didn't pan out. My goal here
is to work towards a vision where the Cosmopolitan core libraries become
less experimental and more focused on curation. This better reflects the
current level of quality we've managed to achieve.
2023-07-06 08:03:24 -07:00
Justine Tunney 88612a2cd7 Restore original redbean build linking 2023-07-05 20:04:43 -07:00
Justine Tunney 97b7116953 Hunt down more bugs
After going through the MODE=dbg and MODE=zero build modes, a bunch of
little issues were identified, which have been addressed. Fixing those
issues created even more troubles for the project, because it improved
our ability to detect latent problems which are getting fixed so fast.
2023-07-03 18:43:29 -07:00
Justine Tunney 73c0faa1b5 Remove some dead code 2023-07-03 02:48:29 -07:00
Justine Tunney 168d1c157e Fix build breakage under special strace config 2023-07-03 00:05:51 -07:00
Justine Tunney fe044e22cc Switch public headers to getopt_long() entirely
Cosmopolitan's getopt() is now redefined as __getopt().
2023-07-02 19:57:43 -07:00
Justine Tunney 0c630d95b5 Rewrite Cosmopolitan Ar
The build/bootstrap/ar.com program is now tinier. This change reduces
its size from 140kb to 53kb. Nothing was traded away. Cosmopolitan Ar
performance is now 2x better than llvm-ar largely thanks to using the
copy_file_range() system call. This change homebrews a new allocation
API that addresses the shortcomings of the C standard library design.
Using these new balloc() and reballoc() functions I managed to reduce
memory consumption so much that Cosmpolitan Ar should now use roughly
100x fewer bytes of peak resident memory compared to llvm-ar. Correct
behavior with better compatibility has been assured. Binary output is
now pretty much bit-identical to llvm-ar, as of this change. This can
and should be the living proof we need to show that a better world is
possible for software.
2023-07-02 10:19:16 -07:00
Justine Tunney 197aa0d465 Implement swapcontext() and makecontext()
This change introduces support for Linux-style uc_context manipulation
that's fast and works well on all supported OSes and architectures. It
also integrates with the Cosmpolitan runtime which can show backtraces
comprised of multiple stacks and fibers. See the test and example code
for further details. This will be used by Mold once it's been vendored
2023-07-02 09:01:44 -07:00
Justine Tunney 7ec84655b4 Get setcontext() and getcontext() working on Aarch64
This change also adds the missing code for getting and restoring the
thread's signal mask, since that's explicitly listed by the man page
2023-07-01 22:53:23 -07:00
Farid Zakaria 8b62bff364 Introduce xxhash into third_party (#837) 2023-07-01 20:56:53 -07:00
Farid Zakaria 5f1e129222 Introduce zstd into cosmopolitan (#843) 2023-07-01 20:30:38 -07:00
Paul Kulchenko c0348330a5 Add path information in logged error messages (#832) 2023-07-01 19:47:59 -07:00
Paul Kulchenko dc8fbf38ad Update Windows IP_MULTICAST constants to use Ws2tcpip.h values (#839) (#842) 2023-07-01 19:44:27 -07:00
Justine Tunney ce0a3065f3 Update build/bootstrap/make.com
This is important, because the execve() workflow has been upgraded for
launching APE binaries. Programs that call execve() need to be remade,
otherwise they might attempt to use ~/.ape rather than ~/ape-1.1.
2023-07-01 18:37:19 -07:00
Justine Tunney 40eb3b9d5d Fully support OpenBSD 7.3
This change (1) upgrades to OpenBSD's newer kernel ABIs, and (2)
modifies APE to have a read-only data segment. Doing this required
creating APE Loader v1.1, which is backwards and forwards compatible
with the previous version.

If you've run the following commands in the past to install your APE
Loader systemwide, then you need to run them again. Ad-hoc installations
shouldn't be impacted. It's also recommended that APE binaries be remade
after upgrading, since they embed old versions of the APE Loader.

    ape/apeuninstall.sh
    ape/apeinstall.sh

This change does more than just fix OpenBSD. The new loader is smarter
and more reliable. We're now able create much tinier ELF and Mach-O data
structures than we could before. Both APE Loader and execvpe() will now
normalize ambiguous argv[0] resolution the same way as the UNIX shell.
Badness with TLS linkage has been solved.

Fixes #826
2023-07-01 18:14:27 -07:00
Justine Tunney 963e10b9bf Introduce new tinyprint() api 2023-07-01 05:09:32 -07:00
Justine Tunney 053681cb97 Fix BSD regressions
The recent change to crt.S that aggressively aligns the system-provided
stack has been rolled back on non-Linux until we can find a better way,
since it can cause a segfault early in execution on several platforms.

This change fixes a regression in tcgetattr() and tcsetattr() on OpenBSD
and NetBSD caused by 4778cd4d27.

This change has been tested across the runitd test fleet which is green.
2023-07-01 00:17:33 -07:00
Paul Kulchenko 5a77db2cd5 Fix recvfrom on Windows (closes #841) (#844) 2023-06-30 23:39:55 -07:00
Justine Tunney 48b2afb192 Address weakness with new pledge("anet") promise
The intent with pledge("anet") has been to prevent outbound connections.
However we were only doing that for TCP sockets, and outbound UDP could
still get through, by using socket() plus sendto(). This change fixed
that by preventing UDP sockets from being created.

Credit goes to chc4 on Hacker News for finding this.
2023-06-18 18:06:47 -07:00
Justine Tunney fb2bd313ae Fix redbean unix.poll() with -1 timeout 2023-06-18 18:05:26 -07:00
Justine Tunney f10845ab9f Add blackhole support to Python tokenbucket module 2023-06-18 07:50:11 -07:00
Justine Tunney 461c835784 Remove removed GCC symlinks 2023-06-18 05:41:55 -07:00
Justine Tunney 226375933a Implement more toolchain fixes 2023-06-18 05:39:31 -07:00
Justine Tunney 0409096658 Get us closer to building busybox
This change undefines __linux__ and adds APIs like clock_settime(). The
gosh darned getopt_long() API has been reintroduced, thanks to OpenBSD.
2023-06-18 04:13:45 -07:00
Justine Tunney 5ac522f0de Add make clean build rule 2023-06-18 01:23:54 -07:00
Justine Tunney 8514933b60 Delete itoa() prototype 2023-06-18 01:17:15 -07:00
Justine Tunney d7c79f43ef Clean up more code
- Found some bugs in LLVM compiler-rt library
- The useless LIBC_STUBS package is now deleted
- Improve the overflow checking story even further
- Get chibicc tests working in MODE=dbg mode again
- The libc/isystem/ headers now have correctly named guards
2023-06-18 01:00:05 -07:00
Justine Tunney afc58a8b41 Upgrade cosmopolitan gcc and binutils to 2.35.2
This change upgrades to the latest portcosmo gcc patch
6728fe1a25185560603ca312a8d4352af2a4e515 which lets us avoid needing to
define __tmpcosmo_FOO constants. We're now using an appropriate binutils
version for GCC 11. The older binutils sometimes wasn't able to print
backtraces, due to not being able to find a .debug_ranges section.

This is breaking change for /opt/cosmos libraries :'( due to this weird
"error: need linked-to section for --gc-sections" that pops up.

Please run `make clean` in the monorepo before rebuilding.
2023-06-18 00:54:14 -07:00
Justine Tunney 2c0f0bab0b Fix mkfifo() on Linux 2023-06-17 16:04:08 -07:00
Justine Tunney 12452976bd Remove getopt_long() from headers
We removed this API a long time ago and it was breaking the GNU make
open source build.
2023-06-17 16:03:16 -07:00
Justine Tunney b881c0ec9e Remove printf() linking hack 2023-06-17 10:13:50 -07:00
Justine Tunney ba03cd95c5 Modernize the README file 2023-06-17 07:55:48 -07:00
Justine Tunney 562a1384cd Make blink support conditionally linkable into APE 2023-06-17 07:55:35 -07:00
Justine Tunney 52d28966f7 Remove old zip base skew hack
Since 8ff48201ca we no longer need the
hack where, when running .com.dbg files, we scanned for the embedded
.com file offset, and then computed zip offsets realtive to that. It
wasn't very reliable in the first place, and was causing issues with
running our new .com.dbg executables, which are true zip files.
2023-06-17 04:20:16 -07:00
Justine Tunney 207e18a060 Unbreak the x86-64 build 2023-06-16 20:05:24 -07:00
Justine Tunney 7bc20bff3e Resolve argv[0] based on path search in ape-m1 2023-06-16 16:58:30 -07:00
Justine Tunney 1353db7d3f Fix rwx .rodata in python.com on aarch64 2023-06-16 16:25:04 -07:00
Justine Tunney 4eebd6b9dc Improve new C23 checked arithmetic feature 2023-06-16 15:32:18 -07:00
Justine Tunney 2a1c588826 Add tokenbucket module to python.com 2023-06-16 15:32:09 -07:00
Justine Tunney e6b7c16a53 Make changes needed for new demo 2023-06-15 23:22:49 -07:00
Justine Tunney c3440d040c Make improvements
- More timspec_*() and timeval_*() APIs have been introduced.
- The copyfd() function is now simplified thanks to POSIX rules.
- More Cosmo-specific APIs have been moved behind the COSMO define.
- The setitimer() polyfill for Windows NT is now much higher quality.
- Fixed build error for MODE=aarch64 due to -mstringop-strategy=loop.
- This change introduces `make MODE=nox87 toolchain` which makes it
  possible to build programs using your cosmocc toolchain that don't
  have legacy fpu instructions. This is useful, for example, if you
  want to have a ~22kb tinier blink virtual machine.
2023-06-15 14:50:53 -07:00
Justine Tunney 8dc11afcf6 Upgrade portcosmo patch and GCC third party docs 2023-06-14 19:53:06 -07:00
Justine Tunney 4778cd4d27 Fix bugs in termios library and cleanup code
This change fixes an issue with the tcflow() magic numbers that was
causing bash to freeze up on Linux. While auditing termios polyfills,
several other issues were identified with XNU/BSD compatibility.

Out of an abundance of caution this change undefines as much surface
area from libc/calls/struct/termios.h as possible, so that autoconf
scripts are less likely to detect non-POSIX teletypewriter APIs that
haven't been polyfilled by Cosmopolitan.

This is a *breaking change* for your static archives in /opt/cosmos if
you use the cosmocc toolchain. That's because this change disables the
ioctl() undiamonding trick for code outside the monorepo, specifically
because it'll lead to brittle ABI breakages like this. If you're using
the cosmocc toolchain, you'll need to rebuild libraries like ncurses,
readline, etc. Yes diamonds cause bloat. To work around that, consider
using tcgetwinsize() instead of ioctl(TIOCGWINSZ) since it'll help you
avoid pulling every single ioctl-related polyfill into the linkage.

The cosmocc script was specifying -DNDEBUG for some reason. It's fixed.
2023-06-14 19:30:52 -07:00
Justine Tunney 06b749ae03 Remove blinkenlights
New home: https://github.com/jart/blink
2023-06-14 19:30:52 -07:00
Gabriel Ravier 50d8d953ce Fix syscall(2) returning -errno instead of using POSIX errno scheme (#830)
As of now, the syscall function is implemented as alike to how the
linux kernel sycall ABI works, returning -errno upon errors without
setting the value of errno. However, this does not conform to the
expectations of most software, which expect it to return -1 and set
errno, which is how it works on other libcs, which document it
accordingly:

> The return value is defined by the system call being invoked.  In
> general, a 0 return value indicates success.  A -1 return value
> indicates an error, and an error number is stored in errno.
- Linux man-pages, syscall(2)

> The return value is the return value from the system call, unless
> the system call failed.  In that case, ‘syscall’ returns ‘-1’ and
> sets ‘errno’ to an error code that the system call returned.
- glibc manual, (libc)System Calls

> When the C-bit is set, syscall returns -1 and sets the external
> variable errno (see intro(2)).
- 4BSD manual, syscall(2)

> A -1 return value indicates an error, and an error code is stored in
> errno.
- 4.4BSD, FreeBSD, OpenBSD and NetBSD manuals (same quote is found in
  all of them), syscall(2)

This patch corrects the syscall function to work in the same way as in
other libcs.
2023-06-11 10:33:28 -07:00
tkchiaandtkchia e47c0cc929 [metal] Clean up code for getting pointer to struct mman (#828)
Co-authored-by: tkchia <[email protected]>
2023-06-11 10:32:39 -07:00
Justine Tunney 5a6a4819d4 Fix -iquote in cosmocc toolchain 2023-06-10 16:35:53 -07:00
Justine Tunney 2676ec55de Make improvements to cosmocc toolchain 2023-06-10 15:50:01 -07:00
Justine Tunney 8ff48201ca Rewrite .zip.o file linker
This change takes an entirely new approach to the incremental linking of
pkzip executables. The assets created by zipobj.com are now treated like
debug data. After a .com.dbg is compiled, fixupobj.com should be run, so
it can apply fixups to the offsets and move the zip directory to the end
of the file. Since debug data doesn't get objcopy'd, a new tool has been
introduced called zipcopy.com which should be run after objcopy whenever
a .com file is created. This is all automated by the `cosmocc` toolchain
which is rapidly becoming the new recommended approach.

This change also introduces the new C23 checked arithmetic macros.
2023-06-10 09:29:44 -07:00
Justine Tunney f6407d5f7c Fix missing includes 2023-06-09 18:11:53 -07:00
Justine Tunney 4a59210008 Introduce #include <cosmo.h> to toolchain users
This change improves the way internal APIs are being hidden behind the
`COSMO` define. The cosmo.h header will take care of defining that, so
that a separate define statement isn't needed. This change also does a
lot more to define which APIs are standard, and which belong to Cosmo.
2023-06-09 18:03:05 -07:00
Justine Tunney 9b55dbe417 Get GCC to mostly build with Cosmo 2023-06-09 06:41:34 -07:00
Justine Tunney 2eb7148474 Create a fake syscall() function due to Python 2023-06-09 05:18:38 -07:00
Justine Tunney 4b2023ffab Disable linker map generation and improve tinyness 2023-06-09 03:29:26 -07:00
Justine Tunney 23e235b7a5 Fix bugs in cosmocc toolchain
This change integrates e58abc1110b335a3341e8ad5821ad8e3880d9bb2 from
https://github.com/ahgamut/musl-cross-make/ which fixes the issues we
were having with our C language extension for symbolic constants. This
change also performs some code cleanup and bug fixes to getaddrinfo().
It's now possible to compile projects like ncurses, readline and python
without needing to patch anything upstream, except maybe a line or two.
Pretty soon it should be possible to build a Linux distro on Cosmo.
2023-06-08 23:44:03 -07:00
Justine Tunney 22f81a8d50 Improve cosmocc / cosmoc++ toolchain scripts
- Get out of the red zone
- Generate --ftrace nops unless -Os is passed
- Intercept -o path to generate .com / .com.dbg appropriately
2023-06-08 14:29:22 -07:00
Justine Tunney 25678db2a0 Fix remaining regressions in ASAN support
See #829
2023-06-08 08:00:50 -07:00
Justine Tunney 4d629fd424 Fix stack abuse in llama.cc
This change also incorporates improvements for MODE=asan. It's been
confirmed that o/asan/third_party/ggml/llama.com will work.

Fixes #829
2023-06-08 07:12:26 -07:00
Justine Tunney 32682f0ce7 Remove some problematic APIs
In order to improve our chances of success building other open source
projects we shouldn't define APIs that'll lead any ./configure script
astray. For example:

- brk() and sbrk() can break mac/windows support
- syscall() is a superb way to break portability
- arch_prctl() is the greatest of all horror shows
2023-06-08 06:12:26 -07:00
Justine Tunney 7512318a2a Fix MODE=aarch64 build 2023-06-08 05:17:37 -07:00
Justine Tunney 8767e9ad6a Fix --ftrace regression 2023-06-08 04:46:11 -07:00
Justine Tunney daf4454a06 Validate privileged code relationships
- Work towards improving non-optimized build support
- Introduce MODE=zero which is -O0 without ASAN/UBSAN
- Use system GCC when ~/.cosmo.mk has USE_SYSTEM_TOOLCHAIN=1
- Have package.com check .privileged code doesn't call non-privileged
2023-06-08 04:38:06 -07:00
Justine Tunney 01fd655097 Get garbage collector working on aarch64
Garbage collection will now happen on arm64 when a function returns,
rather than kicking the can down the road to when the process exits.
This change also does some code cleanup and incorporates suggestions
2023-06-07 03:34:45 -07:00
Justine Tunney 9793d3524f Remove superfluous RET instructions 2023-06-06 19:46:56 -07:00
Justine Tunney b6182db813 Simplify ftrace_hook()
We now have a test to prove that its transitive closure doesn't perform
floating point computations.
2023-06-06 11:10:38 -07:00
Justine Tunney 61b9677c05 Make improvements
- Get mprotect_test working on aarch64
- Get completion working on python.com repl again
- Improve quality of printvideo.com and printimage.com
- Fix bug in openpty() so examples/script.c works again
2023-06-06 09:12:30 -07:00
Justine Tunney b94b29d79c Prevent ftrace from misaligning functions 2023-06-06 06:00:31 -07:00
Justine Tunney b8a6a989c0 Create ELF aliases for identical symbols
This change greatly reduces the number of modules that need to be
compiled. The only issue right now is that sometimes when viewing
symbol table entries, the aliased symbol is chosen.
2023-06-06 03:33:49 -07:00
Justine Tunney e1b83399bd Fix memmove() alias on aarch64 with --ftrace 2023-06-06 00:39:55 -07:00
Justine Tunney e9e8bbe6da Avoid /etc/services in whois command
Apparently IANA has abolished the WHOIS protocol and no longer lists it
as a service. Therefore distros which naively create /etc/services from
IANA's braindead recommendation will inadvertently break any tools that
rely on /etc/services to determine this well-known Internet port.
2023-06-06 00:11:41 -07:00
Justine Tunney eb40cb371d Get --ftrace working on aarch64
This change implements a new approach to function call logging, that's
based on the GCC flag: -fpatchable-function-entry. Read the commentary
in build/config.mk to learn how it works.
2023-06-05 23:35:31 -07:00
Justine Tunney 5b908bc756 Fix some build errors 2023-06-05 15:53:44 -07:00
Justine Tunney 7558549d44 Test m=aarch64 on GitHub Actions 2023-06-05 15:23:00 -07:00
Justine Tunney 42b9b75749 Fix aarch64 build 2023-06-05 05:57:44 -07:00
Justine Tunney f554dd800e Make it possible to log kprintf() to file
It's now possible to compile Emacs using cosmocc. However we need to
troubleshoot why it's event loop isn't working correctly at runtime.
2023-06-05 04:16:15 -07:00
Justine Tunney 9cc3e37263 Upgrade to Cosmopolitan GCC 11.2.0 for aarch64 2023-06-05 02:07:28 -07:00
Justine Tunney 39f20dbb13 Upgrade to Cosmopolitan GCC 11.2.0 for x86_64 2023-06-05 02:06:18 -07:00
Justine Tunney 682b74ed88 Incorporate Blink 1.0.0 release 2023-06-04 11:07:31 -07:00
Justine Tunney 35dcaca53c Fix test fleet errors 2023-06-04 10:57:11 -07:00
Justine Tunney a91a945b88 Fix build breakage 2023-06-04 09:33:48 -07:00
Justine Tunney 4aa1d09b9e Improve aarch64 native support some more
This change introduces partial support for automating remote testing of
aarch64 binaries on Raspberry Pi and Apple Silicon.
2023-06-04 08:58:47 -07:00
Justine Tunney fc34ba2596 Fix Linenoise REPL on AARCH64 2023-06-04 02:57:17 -07:00
Justine Tunney bcf9af94bf Get threads working well on MacOS Arm64
- Now using 10x better GCD semaphores
- We now generate Linux-like thread ids
- We now use fast system clock / sleep libraries
- The APE M1 loader now generates Linux-like stacks
2023-06-04 01:57:10 -07:00
Justine Tunney b5eab2b0b7 Get POSIX threads working on Apple Silicon
It's now possible to run a working

    ape-m1 o/aarch64/third_party/ggml/llama.com

on Apple M1 hardware running XNU!
2023-06-03 18:33:01 -07:00
Justine Tunney 8fdb31681a Introduce support for GGJT v3 file format
llama.com can now load weights that use the new file format which was
introduced a few weeks ago. Note that, unlike llama.cpp, we will keep
support for old file formats in our tool so you don't need to convert
your weights when the upstream project makes breaking changes. Please
note that using ggjt v3 does make avx2 inference go 5% faster for me.
2023-06-03 15:46:21 -07:00
Justine Tunney 6ae18a10ba Fix MODE=tiny build 2023-06-03 10:30:48 -07:00
Justine Tunney 1904a3cae8 Sync llama.cpp to 6986c7835adc13ba3f9d933b95671bb1f3984dc6 2023-06-03 10:29:12 -07:00
Justine Tunney 8f522cb702 Make improvements
This change progresses our AARCH64 support:

- The AARCH64 build and tests are now passing
- Add 128-bit floating-point support to printf()
- Fix clone() so it initializes cosmo's x28 TLS register
- Fix TLS memory layout issue with aarch64 _Alignas vars
- Revamp microbenchmarking tools so they work on aarch64
- Make some subtle improvements to aarch64 crash reporting
- Make kisdangerous() memory checks more accurate on aarch64
- Remove sys_open() since it's not available on Linux AARCH64

This change makes general improvements to Cosmo and Redbean:

- Introduce GetHostIsa() function in Redbean
- You can now feature check using pledge(0, 0)
- You can now feature check using unveil("",0)
- Refactor some more x86-specific asm comments
- Refactor and write docs for some libm functions
- Make the mmap() API behave more similar to Linux
- Fix WIFSIGNALED() which wrongly returned true for zero
- Rename some obscure cosmo keywords from noFOO to dontFOO
2023-06-03 08:12:22 -07:00
Paul Kulchenko 5655c9a4e7 Extend Pledge with anet (same as inet, but with no connect) (#827)
* Add `anet` pledge for `inet` without connect

This is useful for configurations where it's desirable to start redbean
under these restrictions, but not to allow `connect` socket calls.

* Update message on protected/unpledged syscalls for clarity

* Update redbean to add reporting for unpledged sigaction

Previously it would abort without indicating what signal it failed to
install when sigaction is not pledged (although it fails all of them).

* Move GetHostIps before processing command line options

This allows using unix.pledge as part of the options without affecting
retrieving host IP addresses (which requires `connect`). It may still
fail under external `pledge` command as expected; in this case IPs
would need to be passed manually.

* Update tests for pledge anet promise
2023-06-03 07:50:29 -07:00
Justine Tunney 72f8bd10b7 Perform some code cleanup 2023-05-28 19:42:00 -07:00
Connor 992a4638ae Fix rgbtoxterm tool to output the proper colors (#819) 2023-05-26 18:30:24 -07:00
tkchiaandtkchia 90645f4dac [metal] Fix some issues that were breaking UEFI boot (#822)
Co-authored-by: tkchia <[email protected]>
2023-05-26 18:29:25 -07:00
Paul Kulchenko 5c40800d69 Update redbean ProgramDirectory to update package.path (#823) 2023-05-26 18:29:00 -07:00
Justine Tunney 1422e96b4e Introduce native support for MacOS ARM64
There's a new program named ape/ape-m1.c which will be used to build an
embeddable binary that can load ape and elf executables. The support is
mostly working so far, but still chasing down ABI issues.
2023-05-20 04:17:03 -07:00
Justine Tunney b852650c08 Fiddle around with Mach-O 2023-05-20 04:13:49 -07:00
Paul Kulchenko 6881a2ecea Extend redbean Fetch to add option to keeping connection open (#818) 2023-05-17 20:49:26 -07:00
Paul Kulchenko 021c71b651 Fix redbean to avoid compressing the first yielded chunk. (#813)
This may happen if no Status or Content-Type is set and if the written
content is longer than 100B, the compression is going to be applied
after coroutine.yield, even though it shouldn't be.
2023-05-17 20:46:27 -07:00
Justine Tunney e7eb0b3070 Make more ML improvements
- Fix UX issues with llama.com
- Do housekeeping on libm code
- Add more vectorization to GGML
- Get GGJT quantizer programs working well
- Have the quantizer keep the output layer as f16c
- Prefetching improves performance 15% if you use fewer threads
2023-05-16 08:07:23 -07:00
Justine Tunney 80db9de173 Make the intrinsics more readable 2023-05-15 23:12:11 -07:00
Justine Tunney 210187cf77 Perform some code cleanup 2023-05-15 16:32:10 -07:00
Justine Tunney cc1732bc42 Make AARCH64 harder, better, faster, stronger
- Perform some housekeeping on scalar math function code
- Import ARM's Optimized Routines for SIMD string processing
- Upgrade to latest Chromium zlib and enable more SIMD optimizations
2023-05-15 02:15:34 -07:00
Justine Tunney 550b52abf6 Port a lot more code to AARCH64
- Introduce epoll_pwait()
- Rewrite -ftrapv and ffs() libraries in C code
- Use more FreeBSD code in math function library
- Get significantly more tests passing on qemu-aarch64
- Fix many Musl long double functions that were broken on AARCH64
2023-05-14 09:37:26 -07:00
Ariel Núñez 91791e9f38 Started removing features from RedPajama to make it easier to understand for beginners (#817) 2023-05-14 09:16:22 -07:00
Justine Tunney 89d1fad7ee Enable crash reports for radpajama executables 2023-05-13 21:16:03 -07:00
Justine Tunney 296ee3ec58 Make some other fixes to radpajama build config 2023-05-13 21:09:28 -07:00
Justine Tunney 282dd8e7b7 Get radpajama to build
make -j8 o//third_party/radpajama/radpajama.com
    make -j8 o//third_party/radpajama/radpajama-chat.com

This change gets the radpajama.mk config working. This package depends
on THIRD_PARTY_GGML but it's configured to call ggjt_v1(), so that the
library will provide the old quantizers. The ggml_quantize_chunk() API
will now dispatch to older quantizers based on the configured version.
2023-05-13 20:44:36 -07:00
Justine Tunney 410c8785c9 Fix the AARCH64 build 2023-05-13 08:19:44 -07:00
Justine Tunney 5a4cf9560f Add support for new GGJT v2 quantizers
This change makes quantized models (e.g. q4_0) go 10% faster on Macs
however doesn't offer much improvement for Intel PC hardware.

This change syncs llama.cpp 699b1ad7fe6f7b9e41d3cb41e61a8cc3ea5fc6b5
which recently made a breaking change to nearly all its file formats
without any migration. Since that'll break hundreds upon hundreds of
models on websites like HuggingFace llama.com will support both file
formats because llama.com will never ever break the GGJT file format
2023-05-13 08:08:32 -07:00
Justine Tunney ba49e86e20 Get TEST_LIBC_CALLS passing on AARCH64 2023-05-13 02:41:41 -07:00
Justine Tunney 802e7eb4ef Mop up more test regressions 2023-05-13 01:09:44 -07:00
Justine Tunney 4a8a81eb9f Fix llama.com interactive mode regressions 2023-05-13 00:09:38 -07:00
Justine Tunney fd34ef732d Make considerably more progress on AARCH64
- Utilities like pledge.com now build
- kprintf() will no longer balk at 48-bit addresses
- There's a new aarch64-dbg build mode that should work
- gc() and defer() are mostly pacified; avoid using them on aarch64
- THIRD_PART_STB now has Arm Neon intrinsics for fast image handling
2023-05-12 22:42:57 -07:00
Justine Tunney 1bfb3aab1b Make Arm Neon intrinsics work with make tags 2023-05-12 18:32:53 -07:00
Justine Tunney 414667b1c9 Get TEST_LIBC_STR passing on AARCH64
It's now possible to run commands like:

    make -j8 m=aarch64 o/aarch64/test/libc/str

Which will cross-compile and run the test suites in a qemu-aarch64
binary that's vendored in the third_party/qemu/ folder within your
x86_64 build environment.
2023-05-12 18:09:23 -07:00
Justine Tunney 45186c74ac Introduce -q (quiet flag) and improve ctrl-c ux 2023-05-12 09:46:07 -07:00
Justine Tunney e8de1e4766 Fix subtoken antiprompt scanning 2023-05-12 08:55:40 -07:00
Justine Tunney 80c174d494 Clean up llama.com anti/stop/reverse-prompt code
Example use case for JSON completion:

    $ m=opt
    $ make -j16 m=$m o/$m/third_party/ggml/llama.com
    $ o/$m/third_party/ggml/llama.com -m llama.bin -p '{"key": "life", "val": ' -r '}'
    42}

This provides better control. More sophisticated facilities for
controlling text generation will be provided soon enough.
2023-05-12 08:20:58 -07:00
Justine Tunney bbfe4fbd11 Make llama.com n_predict be -1 by default 2023-05-12 08:20:34 -07:00
Justine Tunney ca19ecf49c Fine tune crash reports for llama.com 2023-05-12 06:24:26 -07:00
Justine Tunney 1f2a5a8fc1 Implement crash reporting for AARCH64
The ShowCrashReports() feature for aarch64 should work even better than
the x86 crash reports. Thanks to the benefit of hindsight these reports
should be rock solid reliable and beautiful to read.

This change also improves the syscall polyfills for aarch64. Some of the
sys_foo() functions have been removed, usually because they're legacy or
downright footguns not worth building.
2023-05-12 05:47:54 -07:00
Justine Tunney 285e8a2348 Get backtraces working on AARCH64 2023-05-11 22:12:41 -07:00
Justine Tunney 4edbc98811 Get MbedTLS and its unit tests passing AARCH64 2023-05-11 21:53:15 -07:00
Justine Tunney 5e2f7f7ced Get LIBC_TESTLIB building on AARCH64 2023-05-11 19:57:09 -07:00
Justine Tunney 95fab334e4 Use yield on aarch in spin locks 2023-05-11 19:57:09 -07:00
Ariel Núñez b3e3359d22 Import radpajama (a redpajama.cpp fork) (#814)
This is the relevant commit: https://github.com/togethercomputer/redpajama.cpp/commit/bfa6466199b8ef92185ecb72e2a550e12baf6602

Model download links:
https://huggingface.co/ceonlabs/radpajama/tree/main
2023-05-11 07:12:08 -07:00
Justine Tunney cba15bc77a Fix regression in DNS client
Calling FreeResolvConf() would crash, due to a recent change from
yesterday where free_s() was removed.
2023-05-10 23:27:10 -07:00
Justine Tunney 8d24a18c62 Tune magic division for size
This helps it avoid generating a branch instruction.
2023-05-10 21:32:04 -07:00
Justine Tunney 1f6f9e6701 Remove division from matrix multiplication
This change reduces llama.com CPU cycles systemically by 2.5% according
to the Linux Kernel `perf stat -Bddd` utility.
2023-05-10 21:19:54 -07:00
Justine Tunney a88290e595 Make sure llama.com terminal cleanup happens 2023-05-10 15:56:01 -07:00
Justine Tunney 5250feb7ad There must only be one strerror() 2023-05-10 15:34:13 -07:00
Justine Tunney c91de6a356 Avoid running clang-format on ggml.c 2023-05-10 14:59:33 -07:00
Justine Tunney ea607781f8 Fix the build 2023-05-10 10:14:15 -07:00
Justine Tunney bb3ebedfce Fix load time measurement 2023-05-10 07:54:21 -07:00
Justine Tunney 290a49952e Fix some more issues with aarch64 and llama.cpp 2023-05-10 07:34:26 -07:00
Justine Tunney 64aca4dc4f Fix weird behavior issue w/ kprintf() on aarch64 2023-05-10 06:17:33 -07:00
Justine Tunney 4e81d3277c Do some basic housekeeping in LIBC_STR 2023-05-10 06:17:20 -07:00
Justine Tunney 12a33858c9 There must be only one clock() 2023-05-10 06:16:01 -07:00
Justine Tunney 6cb9553706 Fix alignment bug in llama.com 2023-05-10 06:15:32 -07:00
Justine Tunney ca990ef091 Make llama.com -h print to stdout 2023-05-10 04:55:59 -07:00
Justine Tunney f312f706f4 Bring MODE=tiny binary sizes down to 20kb minimum
aarch64 binaries start at 4kb.
2023-05-10 04:20:48 -07:00
Justine Tunney 5f57fc1f59 Upgrade llama.cpp to e6a46b0ed1884c77267dc70693183e3b7164e0e0 2023-05-10 04:20:48 -07:00
Justine Tunney 5a455eaa0b Work on magic numbers for aarch64 2023-05-10 04:20:48 -07:00
Justine Tunney 59766efd3e Do some more aarch64 fixups 2023-05-10 04:20:47 -07:00
Justine Tunney 86d9323a43 Remove sys_getrandom() on NetBSD
This fixes an apparent regression caused by
3f0bcdc3ef where getrandom() on NetBSD 9.2
doesn't appear to work; ktrace oddly reports:

    1446      1 .ape     CALL  #91 (unimplemented getdopt)
    1446      1 .ape     RET   #91 (unimplemented getdopt) -1 errno 78
    Function not implemented
    1446      1 .ape     PSIG  SIGSYS SIG_DFL: code=SI_NOINFO
2023-05-10 04:20:47 -07:00
Justine Tunney a0237a017c Get llama.com working on aarch64 2023-05-10 04:20:47 -07:00
Justine Tunney 4c093155a3 Get llama.com building as an aarch64 native binary 2023-05-10 04:20:47 -07:00
Justine Tunney d04430f4ef Get LIBC_MEM and LIBC_STDIO building with aarch64 2023-05-10 04:20:47 -07:00
Justine Tunney ae0ee59614 Get aarch64 hello world working
$ m=aarch64-tiny
    $ make -j8 m=$m o/$m/tool/hello/hello.com o/third_party/qemu/qemu-aarch64
    $ o/third_party/qemu/qemu-aarch64 o/$m/tool/hello/hello.com
    hello world
    $ ls -hal o/$m/tool/hello/hello.com
    -rwxr-xr-x 1 jart jart 4.0K May  9 05:04 o/aarch64-tiny/tool/hello/hello.com
2023-05-10 04:20:47 -07:00
Justine Tunney e5e3cdf447 Get LIBC_RUNTIME and LIBC_CALLS building on aarch64 2023-05-10 04:20:47 -07:00
Justine Tunney 7e46645193 Get LIBC_FMT to build for aarch64 2023-05-10 04:20:47 -07:00
Justine Tunney 036b9a0002 Make further progress on non-x86 support 2023-05-10 04:20:47 -07:00
Justine Tunney aef9a69a60 Make more progress on aarch64 2023-05-10 04:20:47 -07:00
Justine Tunney 135080fd3e Get libc/tinymath/ compiling on aarch64 2023-05-10 04:20:46 -07:00
Justine Tunney 2b73e72d59 Make more code aarch64 friendly 2023-05-10 04:20:46 -07:00
Justine Tunney ca2860947f Make progress towards aarch64 build 2023-05-10 04:20:46 -07:00
Justine Tunney 08ff26c817 Add qemu-aarch64 2023-05-10 04:20:46 -07:00
Justine Tunney 8303e23b3a Do some basic build tuning 2023-05-10 04:20:46 -07:00
Justine Tunney 57cc257f58 Vendor musl-cross-make gcc 9.2.0 aarch64 2023-05-10 04:20:46 -07:00
Paul Kulchenko a493b64233 Merge pull request #807 from jart/evalarg
Fix the behavior of `redbean -i -e CODE`
2023-05-02 07:48:43 -07:00
Justine Tunney b008950d90 Add libcxx to make toolchain 2023-05-01 14:12:46 -07:00
Justine Tunney 12438cce16 Fix regression with Python linker eaxmples
We can once again create 2mb statically-linked Python binaries:

    $ make -j8 m=tiny o/tiny/examples/pyapp/pyapp.com
    $ ls -hal o/tiny/examples/pyapp/pyapp.com
    -rwxr-xr-x 1 jart jart 2.1M May  1 14:04 o/tiny/examples/pyapp/pyapp.com
    $ o/tiny/examples/pyapp/pyapp.com
    cosmopolitan is cool!

The regression was caused by Python thread support in b15f9eb58
2023-05-01 14:12:15 -07:00
Justine Tunney fc82f77a46 Fix bug in cosmocc / cosmoc++ scripts
The necessary flags weren't being passed when compiling programs as a
single step (i.e. not using `-c` to make intermediate objects).
2023-05-01 13:46:15 -07:00
Justine Tunney 3dac9f8999 Use Companion AI in llama.com by default 2023-04-30 23:08:15 -07:00
Justine Tunney d9e27203d4 Incorporate some fixes and updates for GGML 2023-04-28 20:24:55 -07:00
Justine Tunney b31ba86ace Introduce prompt caching so prompts load instantly
This change also introduces an ephemeral status line in non-verbose mode
to display a load percentage status when slow operations are happening.
2023-04-28 16:15:26 -07:00
Justine Tunney bf6459e324 Fix bugs with cli flags in gzip.com 2023-04-28 05:15:58 -07:00
Justine Tunney 1c2da3a55a Make shell usability improvements to llama.cpp
- Introduce -v and --verbose flags
- Don't print stats / diagnostics unless -v is passed
- Reduce --top_p default from 0.95 to 0.70
- Change --reverse-prompt to no longer imply --interactive
- Permit --reverse-prompt specifying custom EOS if non-interactive
2023-04-28 02:54:11 -07:00
Justine Tunney 420f889ac3 Further optimize the math library
The sincosf() function is now twice as fast, thanks to ARM Limited. The
same might also be true of logf() and expm1f() which have been updated.
2023-04-28 01:20:47 -07:00
Justine Tunney e8b43903b2 Import llama.cpp
https://github.com/ggerganov/llama.cpp
0b2da20538d01926b77ea237dd1c930c4d20b686
See third_party/ggml/README.cosmo for changes
2023-04-27 14:37:14 -07:00
Justine Tunney f42089d5c6 Be more lenient about stack memory
The new stack size is 256kb in order to compromise with llama.cpp's
aggressive use of stack memory, which can't be easily patched. This
change disables the dynamic alloca() and VLA warnings for now, plus
frame sizes for individual functions may be <=50% of the stack size

This only applies to code in the cosmo monorepo. Open source builds
should already be using an 8mb stack by default, like everyone else
2023-04-27 10:45:47 -07:00
Justine Tunney fec139f423 Add mlock() and munlock() 2023-04-27 10:42:52 -07:00
Justine Tunney 7c9ef924bf Add more sorting algorithms 2023-04-27 05:44:44 -07:00
Justine Tunney b7bf052a4b Add Intel intrinsics headers 2023-04-27 05:44:38 -07:00
Justine Tunney 369f9740de Run clang-format on most sources 2023-04-27 05:44:32 -07:00
Justine Tunney 497d4fa6d1 Fix the behavior of redbean -i -e CODE
When redbean is functioning as a Lua interpreter, the `-e` flag should
behave the same way as other open source language interpreters. Namely
it should exit after evaluating the code rather than showing the REPL.
2023-04-25 01:52:13 -07:00
Justine Tunney 614229e3f4 Make some sorting functions faster 2023-04-24 23:11:18 -07:00
Gautham 1a839ba41d add os.geteuid to python (#806) 2023-04-24 20:15:24 -07:00
Justine Tunney 4a6be2154d Fix hermeticity issue with new test 2023-04-18 06:37:01 -07:00
Justine Tunney 3e702e8f65 Push new rolling release 2023-04-18 05:11:15 -07:00
Justine Tunney a8ff220654 Fix some minor issues 2023-04-18 05:02:45 -07:00
Gavin Hayes d484e1dbd4 Improve system() / popen() builtin shell (#753)
* cocmd: stop splitting redirect expressions
* cocmd: allow passing unpaired } for passing Perl test
* cocmd: propagate exit status from semicolon
2023-04-17 16:18:16 -07:00
Gabriel Ravier d5b8b644c2 unveil: Added truncate support on Linux 6.2+ (#803)
Right now, cosmopolitan uses Linux Landlock ABI version 2 on Linux,
meaning that the polyfill for unveil() cannot restrict operations such
as truncate() (a limitation of Landlock's ABI from then). This means
that to restrict truncation operations Cosmopolitan instead has to ban
the syscall through a SECCOMP BPF filter, meaning that completely
legitimate truncate() calls are blocked

However, the newest version of the Landlock ABI (version 3) introduced
in Linux 6.2, released in February 2023, implements support for controlling truncation
operations. As such, the previous SECCOMP BPF truncate() filtering is
no longer needed when the new ABI is available

This patch implements unveil truncate support for Linux Landlock ABI
version 3
2023-04-17 16:17:02 -07:00
Gabriel Ravier f7bccf5513 Fix pledge rpath support for old getdents syscall (#804)
The rpath pledge as currently implemented in cosmopolitan does not
allow for usage of the old getdents syscall (0x4e), which is different
from the newer getdents syscall (0xd9) solely in that it does not
support 64-bit filesystems.

This means that, for example, old statically linked binaries cannot
use `readdir` and other such functions which use this syscall instead
of the more modern one, even though there is no threat in allowing
that syscall alongside the more modern one (except that the binary may
have issues with 64-bit filesystems, but that's a separate problem).

This patch fixes this.
2023-04-17 16:15:01 -07:00
Gavin Hayes 6758c3f400 commandv: fix dir tests on non-NT (#764) 2023-04-15 06:26:13 -07:00
Gabriel Ravier 9a5d69c842 Fix scanf x specifier with string of 0 (#793)
The C standard states that, in the context of an x conversion
specifier given to scanf:
> Matches an optionally signed hexadecimal integer, whose format is
> the same as expected for the subject sequence of the strtoul
> function with the value 16 for the base argument.
- C standard, 7.23.6.2.11. The fscanf function

Cosmopolitan fails to do this, as 0 should be parsed as a 0 by such an
invocation of strtoul. Instead, cosmopolitan errors out as though such
input is invalid, which is wrong.

This means that a program such as this:

 #include <stdio.h>
 #undef NDEBUG
 #include <assert.h>

int main()
{
    int v = 0;
    assert(sscanf("0", "%x", &v) == 1);
}

will not run correctly on cosmpolitan, instead failing the assertion.

This patch fixes this, along with the associated GitHub issue,
https://github.com/jart/cosmopolitan/issues/778
2023-04-15 06:25:35 -07:00
Gabriel Ravier 12e07798df Fix printf precision/field width being limited by internal buffer size (#799)
The C standard, when defining field width and precision, never gives
any limit on the values used for them (except, I believe, that they
fit within an int). In other words, if the user gives a field width of
32145 and a precision of 9218, the implementation has to handle these
values correctly. However, when such kinds of high numbers are used
with integer conversions, cosmopolitan is limited by an internal
buffer size of 144, which means precisions and field widths have to
fit within this, which violates the standard.

This means that for example, the following program:

 #include <stdio.h>
 #include <string.h>

int main()
{
    char buf2[512] = {};

    int i = snprintf(buf2, sizeof(buf2), "%.9999u", 10);
    printf("%d %zu\n", i, strlen(buf2));
}

would, instead of printing "9999 511" (the correct output), instead
print "144 144" under cosmopolitan.

This patch fixes this.
2023-04-04 14:16:34 -04:00
Gabriel Ravier 9c6d78c26d Fix vfprintf and derived functions not handling write errors (#798)
The C standard states:
> The fprintf function returns the number of characters transmitted,
> or a negative value if an output or encoding error occurred or if
> the implementation does not support a specified width length
> modifier.
- C Standard, 7.23.6.1.15. The fprintf function

However, cosmopolitan fails to return a negative value in the case of
an output error, meaning that a program such as:

 #include <stdio.h>

int main()
{
    FILE *fp = fopen("/dev/full", "w");

    setbuf(fp, NULL);
    printf("fprintf: %d\n", fprintf(fp, "test\n"));
    printf("fflush: %d\n", fflush(fp));
}

will, under cosmopolitan, print that no error occured in either of the
calls to fprintf and fflush.

This patch fixes this, along with the associated GitHub issue,
https://github.com/jart/cosmopolitan/issues/784
2023-03-31 09:57:29 -04:00
Justine Tunney 221adfa606 Fix quick typo in fsync() --ftrace 2023-03-30 16:17:40 -07:00
Justine Tunney 4ad4c2625d Add Discord invite link to README 2023-03-30 15:42:58 -07:00
Justine Tunney e3eb55cc8f Add note to README about Discord chatroom 2023-03-30 10:49:19 -07:00
Gabriel Ravier 36f52ea687 Fix PFLINK mechanism for uppercase float conversion specifiers (#796)
_PFLINK is supposed to automatically pull in required functions for
specific conversion specifiers. However, it fails to do so for the F,
G and E conversion specifiers.

This means that, for example, the following program:

 #include <stdio.h>

int main()
{
    printf("%F %G %E\n", .0, .0, .0);
}

fails to run correctly, printing "? ? ?" instead of
"0.000000 0 0.000000E+00".

This patch fixes this.
2023-03-29 22:18:59 -04:00
Gavin Hayes 669b4c5f19 Add memfd fexecve zipos support (#752) 2023-03-29 18:16:46 -07:00
Gabriel Ravier ba42248575 Fix larger than WCHAR_MAX differences in wcs{,n}cmp (#795)
The C standard states:

> Unless explicitly stated otherwise, the functions described in this
> subclause order two wide characters the same way as two integers of
> the underlying integer type designated by wchar_t.
>
> [...]
>
> The wcscmp function returns an integer greater than, equal to, or
> less than zero, accordingly as the wide string pointed to by s1 is
> greater than, equal to, or less than the wide string pointed to by
> s2.
>
> [...]
>
> The wcsncmp function returns an integer greater than, equal to, or
> less than zero, accordingly as the possibly null-terminated array
> pointed to by s1 is greater than, equal to, or less than the
> possibly null-terminated array pointed to by s2.
- C Standard, 7.31.4.4. Wide string comparison functions

Cosmopolitan fails to obey this in cases where the difference between
two wide characters is larger than WCHAR_MAX.

This means that, for example, the following program:

 #include <stdio.h>
 #include <wchar.h>
 #include <limits.h>

int main()
{
    wchar_t str1[] = { WCHAR_MIN, L'\0' };
    wchar_t str2[] = { WCHAR_MAX, L'\0' };
    printf("%d\n", wcscmp(str1, str2));
    printf("%d\n", wcsncmp(str1, str2, 2));
}

will print `1` twice, instead of the negative numbers mandated by the
standard (as WCHAR_MIN is less than WCHAR_MAX)

This patch fixes this, along with the associated Github issue,
https://github.com/jart/cosmopolitan/issues/783
2023-03-29 20:28:18 -04:00
Justine Tunney 999481ace0 Fix memcpy(size=0) ubsan warning in vsnprintf()
Fixes #785
2023-03-29 01:28:10 -07:00
Gabriel Ravier 7f925e6be9 Fix issues 774, 782 and 789 (printf precision bugs) (#790)
The C standard states that, within the context of a printf-family
function, when specifying the precision of a conversion specification:

> A negative precision argument is taken as if the precision were
> omitted.
- Quoth the C Standard, 7.23.6.1. The fprintf function

Cosmopolitan instead treated negative precision arguments as
though they had a value of 0, which was non-conforming. This
change fixes that. Another issue we found relates to:

> For o conversion, it increases the precision, if and only if
> necessary, to force the first digit of the result to be a zero (if
> the value and precision are both 0, a single 0 is printed).
- Quoth the C standard, 7.23.6.1.6. The fprintf function

When printing numbers in their alternative form, with a precision and
with a conversion specifier of o (octal), Cosmopolitan wasn't following
the standard in two ways:

1. When printing a value with a precision that results in 0-padding,
   cosmopolitan would still add an extra 0 even though this should be
   done "if and only if necessary"
2. When printing a value of 0 with a precision of 0, nothing is
   printed, even though the standard specifically states that a single
   0 is printed in this case

This change fixes those issues too. Furthermore, regression tests have
been introduced to ensure Cosmopolitan continues to be conformant
going forward.

Fixes #774 
Fixes #782 
Fixes #789
2023-03-29 01:11:48 -07:00
Gabriel Ravier 2f4335e081 Fix inttypes.h FAST16 macros to have a correct definition (#791)
Cosmopolitan now conforms to the C Standard 7.8.1 specification
of the PRI and SCN macros, because this change fixes a bug where
the FAST16 ones were incorrectly using the %hd specifier.
2023-03-29 00:19:40 -07:00
Gabriel Ravier 0adefbf152 Fix the X conversion specifier's alternative form (#788)
The standard states that, when the # flag is used:
> The result is converted to an "alternative form". [...] For x (or X)
conversion, a nonzero result has 0x (or 0X) prefixed to it.
- C standard, 7.23.6.1. The fprintf function

cosmopolitan fails to use the correct alternative form (0X) when the X
conversion specifier is used, instead using 0x, which is not
capitalized.

This patch fixes this, along with the several tests that test for the
wrong behavior.
2023-03-29 00:10:53 -07:00
Gavin Hayes a412ca7a77 clang-format: merge include blocks (#792) 2023-03-29 00:05:58 -07:00
Justine Tunney 426d441994 Fix quick regression due to header line 2023-03-28 23:35:09 -07:00
Justine Tunney 4f39139fe0 Support kill(pid, 0) behavior on Windows
It's specified by POSIX as a way to check if a pid can be signalled.
2023-03-28 22:28:09 -07:00
Justine Tunney 0d084d01b9 Discern MAP_ANONYMOUS the proper way on NT
We were checking for anonymous mappings earlier on Windows by seeing if
the file descriptor argument to mmap() was supplied as -1. This was not
correct. The proper thing to do is check `flags & MAP_ANONYMOUS`.
2023-03-28 22:16:26 -07:00
Justine Tunney 390aee960a Use enum and typedef in stdatomic.h 2023-03-28 22:15:58 -07:00
Gabriel Ravier 792b1c84c0 Fix padding+minus flag on numbers for printf-family functions (#787)
The C standard states, for conversions using the d, i, b, B, o, u, x or X conversion specifiers:
> The precision specifies the minimum number of digits to appear; if
> the value being converted can be represented in fewer digits, it is
> expanded with leading zeros.
- C standard, 7.23.6.1. The fprintf function

However, cosmopolitan currently suppresses the addition of leading
zeros when the minus flag is set. This is not reflected by anything
within the C standard, meaning that behavior is incorrect.

This patch fixes this.
2023-03-25 14:39:25 -04:00
Gabriel RavierandGavin Hayes 2d6ea2fbc9 Fix issue #771 by implementing S conversion specifier for printf-related functions (#786)
* Implement S conversion specifier for printf-related functions

POSIX specifies that a conversion specifier of S must be interpreted
the same way as %ls. This patch implements this.

* clang-format

---------

Co-authored-by: Gavin Hayes <[email protected]>
2023-03-25 14:38:21 -04:00
Paul Kulchenko 58b2f9bd6b Fix missing detail in the error message in redbean (#768)
This only happens on calls that need to happen during active connection.
2023-03-20 11:10:32 -04:00
Ariel Núñez 893703a07b Fix import asyncio (#763) 2023-03-11 23:29:03 -08:00
Alexandre Gomes Gaigalas 775f456d4c Avoid matching directories when searching PATH (#717)
When searching for an executable, performs an additional check to
determine if the path is a file.
2023-03-06 11:15:32 -05:00
Paul Kulchenko e2f429b742 Redbean documentation typo fixes. (#756)
* Redbean documentation typo fixes.

* Add brief description to some unix.* calls missing it
2023-03-06 00:15:09 -08:00
Justine Tunney ba180e754d Improve cosmocc toolchain
- Decouple zlib from libc
- Add some underscores to mostly internal names
2023-03-05 23:52:49 -08:00
Justine Tunney 713d4424c6 Don't add loopback address to generated SSL cert 2023-03-05 22:57:22 -08:00
Justine Tunney e16b969829 Make intrin_test go faster 2023-03-05 22:57:07 -08:00
Justine Tunney 2112fb1736 Fix SHUFPD and SHUFPS and add fuzz tests
See: https://github.com/jart/blink/issues/72
See: https://github.com/jart/blink/issues/68
2023-03-05 16:12:28 -08:00
Justine Tunney 2aa044cb0c Interalize msr.h 2023-03-05 16:12:11 -08:00
Justine Tunney c8a9d3458e Update commentary in ape.S 2023-03-05 16:11:03 -08:00
Gavin Hayes c5de653b98 GetZipCdir: prevent integer underflow. posix_spawn_test.c: stop attempting to load zipos from /usr/bin/ape. (#758) 2023-03-05 14:29:38 -08:00
Paul Kulchenko 22fcab131c Fix redbean SQLite to report results on failure to open db (#760) 2023-03-05 14:26:56 -08:00
imawaki da7c8c7b54 Serve .mjs files as text/javascript (#761) 2023-03-05 14:26:18 -08:00
Gavin Hayes f5520209f5 execve_test.com: fix run with landlock make. Cleanup execve/fexecve tests (#755) 2023-03-01 22:32:15 -08:00
tkchia ae365928c5 [metal] Minor corrections to constants for paging structures (#751) 2023-02-24 11:49:08 -08:00
Gavin Hayes 5923d483a4 Add execve / fexecve support to ZIpOS (#727) 2023-02-24 11:48:24 -08:00
Paul Kulchenko 0312898979 Make redbean cache directive configurable (#750)
Thank you @johnmuhl for the proposal. Fixes #739
2023-02-23 22:24:40 -08:00
Paul Kulchenko db16f0129a Redbean sqlite fix multi close (#749)
* Update redbean SQLite config to handle more options

This requires moving sqlite3_initialize call to open, as configuration
should be allowed before initialization is done. This call is effective
only for the first time and then no-op after that.

* Fix redbean SQLite for closing db with already finalized statements

There is a loop in cleanupdb that finalizes all vms that are associated
with that db when it's being closed. Under some circumstances (detailed
below) that loop may contain references pointing to already collected
objects, thus leading to SIGSEGV when those references are used.

This may happen with the following sequence of events ("VM" is the name
used in lsqlite and describes the same thing as "statement"):
1. A finalized statement is created (for example, by preparing an empty
string or a string with no statement that is still grammatically valid).
2. This statement goes out of scope before the DB object it's associated
with does and is garbage collected.
3. When it's garbage collected, dbvm_gc method is called, which checks
for svm->vm being not NULL.
4. Since the VM is already finalized, cleanupvm method is not called,
so the VM reference is not removed from the table of VMs tracked for
that DB.
5. When the DB is finally closed or garbage collected, all the VMs
associated with it are accessed to be finalized, including the ones that
have been garbage collected and have invalid references (thus leading
to a memory access error).

Here is an example of a stacktrace from the resulting SIGSEGV:

70000003de20 5df71a getgeneric+26
70000003fac0 5dfc7f luaH_get+111
70000003faf0 5e06c8 luaH_set+40
70000003fb20 5c5bd7 aux_rawset+55
70000003fb50 5c70cb lua_rawset+27
70000003fb60 4fa8e7 cleanupvm+71
70000003fb80 4fa988 cleanupdb+88
70000003fbc0 4fe899 db_gc+41

One way to fix this is to use userdata references (which anchor their
targets) instead of lightuserdata references (which do not), but this
would prevent the targets (VMs) from being garbage collected until the
DB itself is garbage collected, so this needs to be combined with
weakening the keys in the DB table. The code in cleanupdb to remove the
VM references is no longer needed, as this is handled by having weak keys.

The patch also switches to using close_v2, as it is intended for use
with garbage collected languages where the order in which destructors
are called is arbitrary, as is the case here.

* Remove GC collection from redbean SQLite session

The behavior of sqlite3session_delete is undefined after the DB
connection is closed, so we need to avoid calling it from gc handler.
2023-02-23 20:10:03 -08:00
Justine Tunney c395d16230 Upgrade embedded blink to 0.9.2 2023-02-23 12:25:01 -08:00
Justine Tunney 1011db793f Bump embedded blink binary to 0.9.0
This is an official tagged version. The 1.1 version used earlier was
informal. Formal versions will always have major.minor.patch going
forward. See https://github.com/jart/blink/tags 1.0.0 coming soon.
2023-02-23 09:57:08 -08:00
Michael Ford ca88fc34c3 doc: remove "everything everything" from ParseUrl doc (#704)
Closes: #702.
2023-02-23 09:34:48 -08:00
Justine Tunney ad97d7b9db Disable Python test_sys 2023-02-23 08:56:31 -08:00
Justine Tunney 18a24935cc Fix Python stack overflow checking in MODE=dbg 2023-02-23 08:13:33 -08:00
Justine Tunney 8b69acc1cc Make mkdeps more helpful when files are missing 2023-02-23 08:11:08 -08:00
Justine Tunney 0468ca02fe Support building APE binaries greater than 30mb 2023-02-23 08:10:43 -08:00
Justine Tunney 7355499a6a Remove a misplaced include 2023-02-23 08:10:38 -08:00
Justine Tunney 1208ac9f10 Add some POSIX feature definitions
This helps Python configure itself. We should focus more on vetting
these to make 100% sure they're accurate to define.
2023-02-23 08:09:34 -08:00
Justine Tunney 298ba74a45 Make POSIX semaphores always process shared
Python triggered the undefined behavior previously since it appears to
be posting to a semaphore owned by a different process that wasn't set
to process shared mode. The performance loss to process shared futexes
is so low and semaphores are generally used for this purpose, so it'll
be much simpler to simply not impose undefined behavior here.
2023-02-23 08:07:54 -08:00
Justine Tunney b15f9eb58f Improve Python's threading story
Python threads are now generally working, however some parts of Python's
regression tests for threads are flaky. This is possibly due to needing
more locking primitives in Cosmo's IO system call wrappers, e.g. close.

    make o//third_party/python/Lib/test/test_threading.py.runs

See #747
2023-02-23 06:55:54 -08:00
ahgamut 60eb34509b quick addition of cosmo pthreads to python.com
- enable WITH_THREAD and _POSIX_THREADS
- add headers everywhere
- breaks only two tests (faulthandler and signal)
- disabled terminal completion because it causes segfaults for some
  reason (probably could not get the current thread)
2023-02-23 06:55:17 -08:00
Justine Tunney a808b3e738 Serialize ZipOS handle IO across threads 2023-02-23 06:55:10 -08:00
Justine Tunney 0f1ead8943 Fix select() on Linux
This fixes a bug where the caller's timeval will be clobbered on Linux.
The Kernel ABI *always* modifies the timeout argument but POSIX says it
should be a const parameter. The wrapper now handles the difference and
sys_select() may be used if obtaining the remainder on Linux is needed.
2023-02-23 06:55:05 -08:00
Justine Tunney dfabcd84c1 Fix sysinfo()
The system call wrapper was wrongfully reinterpreting kernel data. The
examples/sysinfo.c program is now updated to show how to correctly use
what's returned.
2023-02-23 06:54:58 -08:00
Justine Tunney 71d955be1d Add blog link for access() impl inspiration 2023-02-23 06:54:53 -08:00
James Hulce e4ee1da652 Fix filenames in help.txt (#722)
Many functions incorrectly direct users to see a .c file, when they should be .S
2023-02-22 19:00:49 -08:00
Gavin Hayes ff9c15f48a Add APE fexecve() support (#733) 2023-02-22 18:58:23 -08:00
Gavin Hayes b275e664ec Make ZipOS mmap safer (#735)
- It now runs entirely under __mmi_lock
- Hide implementation strace
2023-02-22 18:57:36 -08:00
Paul Kulchenko e323527ffa Remove slowloris handling from Redbean (#737)
It's been superseded by token bucket processing, does not take time
into considerations (only the number of fragments), and affects file
uploads that may require a large number of reads.
2023-02-22 18:56:02 -08:00
Joshua Wierenga 08fef9b277 Fix nt path format check (#744) 2023-02-22 18:42:13 -08:00
Gavin Hayes 94dcf81954 Add glob to cocmd (#740) 2023-02-21 10:31:04 -08:00
Justine Tunney 2b6261a52d Make some system call fixes
- Fix minor ABI issue with SIOCGIFCONF
- Fix ABI translation issues with statfs() on BSDs
- Fix SQLite angled header line
2023-02-12 22:16:34 -08:00
Justine Tunney 0eb621f75e Remove non-JIT'd blink builds for now 2023-02-12 22:14:16 -08:00
Justine Tunney 9634227181 Polyfill Linux unlink() EISDIR on POSIX platforms 2023-02-05 16:50:11 -08:00
Justine Tunney 9fce2751c8 Remove problematic test 2023-02-05 16:50:04 -08:00
Justine Tunney 49c1dea03b Add msync_test 2023-02-05 16:49:52 -08:00
16338 changed files with 451600 additions and 162948 deletions

No files matched your search

+1
View File
@@ -8,6 +8,7 @@ AlwaysBreakBeforeMultilineStrings: false
AllowShortFunctionsOnASingleLine: false
KeepEmptyLinesAtTheStartOfBlocks: true
ConstructorInitializerAllOnOneLineOrOnePerLine: true
IncludeBlocks: Merge
---
Language: Cpp
AllowShortFunctionsOnASingleLine: false
+1
View File
@@ -1,3 +1,4 @@
# -*- conf -*-
*.gz binary
/build/bootstrap/*.com binary
/usr/share/zoneinfo/* binary
+1 -1
View File
@@ -17,7 +17,7 @@ jobs:
runs-on: ubuntu-latest
strategy:
matrix:
mode: ["", tiny, opt, rel]
mode: ["", tiny, rel, tinylinux, optlinux]
steps:
- uses: actions/checkout@v3
+1
View File
@@ -1,6 +1,7 @@
# -*- conf -*-
/o
/.prompt.jtlp
# TODO: Find some way to have Python write to o/
__pycache__
+5 -8
View File
@@ -15,7 +15,7 @@
"alignas(x)",
"alignof(x)",
"artificial=",
"dontdiscard=",
"__wur=",
"mayalias=",
"forceinline=",
"forcealign(x)=",
@@ -35,14 +35,13 @@
"returnstwice=",
"textwindows=",
"privileged=",
"compatfn=",
"noinstrument=",
"dontinstrument=",
"nodebuginfo=",
"interruptfn=",
"optimizespeed=",
"forcealignargpointer=",
"noasan=",
"noubsan=",
"dontasan=",
"dontubsan=",
"donothing=",
"nosideeffect=",
"unreachable=",,
@@ -78,7 +77,6 @@
"-Wa,-W",
"-Wa,-I.",
"-Wa,--noexecstack",
"-Wa,--nocompress-debug-sections",
"-Og",
"-g",
"-gdescribe-dies",
@@ -103,8 +101,7 @@
],
"defines": [
"COSMO",
"MODE=",
"IMAGE_BASE_VIRTUAL=0x400000"
"MODE="
]
}
],
+150 -56
View File
@@ -60,13 +60,12 @@
# build/config.mk
SHELL = build/bootstrap/cocmd.com
HOSTS ?= freebsd openbsd netbsd rhel7 rhel5 xnu win10
MAKEFLAGS += --no-builtin-rules
.SUFFIXES:
.DELETE_ON_ERROR:
.FEATURES: output-sync
.PHONY: all o bins check test depend tags
.PHONY: all o bins check test depend tags aarch64 clean
ifneq ($(m),)
ifeq ($(MODE),)
@@ -74,9 +73,38 @@ MODE := $(m)
endif
endif
UNAME_M = $(shell uname -m)
UNAME_S = $(shell uname -s)
# apple still distributes a 17 year old version of gnu make
ifeq ($(MAKE_VERSION), 3.81)
$(error please use build/bootstrap/make.com)
endif
# provide instructions to non-linux users on unbundling gcc
ifeq ($(TOOLCHAIN),) # if TOOLCHAIN isn't defined
ifeq ("$(wildcard o/third_party/gcc/bin/x86_64-linux-cosmo-*)","") # if our gcc isn't unbundled
ifneq ($(UNAME_M)-$(UNAME_S), x86_64-Linux) # if this is not amd64 linux
$(error you need to download https://justine.lol/cosmocc-0.0.12.zip and unzip it inside the cosmo directory)
endif
endif
endif
# the default build modes is empty string
# on x86_64 hosts, MODE= is the same as MODE=x86_64
# on aarch64 hosts, MODE= is changed to MODE=aarch64
ifeq ($(MODE),)
ifeq ($(UNAME_M),arm64)
MODE := aarch64
endif
ifeq ($(UNAME_M),aarch64)
MODE := aarch64
endif
endif
# primary build rules
all: o
o: o/$(MODE)
o/$(MODE): \
o/$(MODE)/ape \
o/$(MODE)/dsp \
@@ -87,20 +115,39 @@ o/$(MODE): \
o/$(MODE)/examples \
o/$(MODE)/third_party
# check if we're using o//third_party/make/make.com
# we added sandboxing to guarantee cosmo's makefile is hermetic
# it also shaves away 200ms of startup latency with native $(uniq)
ifneq ($(LANDLOCKMAKE_VERSION),)
ifeq ($(UNAME_S),Linux)
ifeq ($(wildcard /usr/bin/ape),)
$(warning please run ape/apeinstall.sh if you intend to use landlock make)
$(shell sleep .5)
endif
endif
ifeq ($(USE_SYSTEM_TOOLCHAIN),)
.STRICT = 1
.PLEDGE = stdio rpath wpath cpath fattr proc
.UNVEIL = \
libc/integral \
libc/disclaimer.inc \
rwc:/dev/shm \
rx:build/bootstrap \
rx:o/third_party/gcc \
/proc/stat \
rw:/dev/null \
w:o/stack.log \
/etc/hosts \
~/.runit.psk \
/proc/self/status \
endif
endif
.PLEDGE += stdio rpath wpath cpath fattr proc
.UNVEIL += \
libc/integral \
libc/stdbool.h \
libc/disclaimer.inc \
rwc:/dev/shm \
rx:build/bootstrap \
rx:o/third_party/gcc \
r:build/portcosmo.h \
/proc/stat \
rw:/dev/null \
rw:/dev/full \
w:o/stack.log \
/etc/hosts \
~/.runit.psk \
/proc/self/status \
rx:/usr/bin/qemu-aarch64 \
rx:o/third_party/qemu/qemu-aarch64 \
/sys/devices/system/cpu/cpu0/cpufreq/scaling_governor
PKGS =
@@ -110,54 +157,59 @@ include build/functions.mk #─┐
include build/definitions.mk # ├──META
include build/config.mk # │ You can build
include build/rules.mk # │ You can topologically order
include build/online.mk # │
include libc/stubs/stubs.mk #─┘
include build/online.mk #─┘
include libc/nexgen32e/nexgen32e.mk #─┐
include libc/sysv/sysv.mk # ├──SYSTEM SUPPORT
include libc/nt/nt.mk # │ You can do math
include libc/intrin/intrin.mk # │ You can use the stack
include libc/linux/linux.mk # │ You can manipulate arrays
include third_party/compiler_rt/compiler_rt.mk # │ You can manipulate arrays
include libc/tinymath/tinymath.mk # │ You can issue raw system calls
include third_party/compiler_rt/compiler_rt.mk # │
include libc/str/str.mk # │
include third_party/xed/xed.mk # │
include third_party/zlib/zlib.mk # │
include third_party/double-conversion/dc.mk # │
include third_party/puff/puff.mk # │
include libc/elf/elf.mk # │
include ape/ape.mk # │
include libc/fmt/fmt.mk # │
include libc/vga/vga.mk #─┘
include libc/calls/calls.mk #─┐
include third_party/getopt/getopt.mk # │
include libc/runtime/runtime.mk # ├──SYSTEMS RUNTIME
include libc/crt/crt.mk # │ You can issue system calls
include third_party/nsync/nsync.mk # │
include libc/runtime/runtime.mk # ├──SYSTEMS RUNTIME
include third_party/double-conversion/dc.mk # │ You can issue system calls
include libc/crt/crt.mk # │
include third_party/dlmalloc/dlmalloc.mk #─┘
include libc/mem/mem.mk #─┐
include libc/zipos/zipos.mk # ├──DYNAMIC RUNTIME
include third_party/gdtoa/gdtoa.mk # │ You can now use stdio
include libc/time/time.mk # │ You can finally call malloc()
include third_party/nsync/mem/mem.mk # │
include libc/thread/thread.mk # │
include third_party/gdtoa/gdtoa.mk # ├──DYNAMIC RUNTIME
include third_party/nsync/mem/mem.mk # │ You can now use stdio
include libc/proc/proc.mk # │ You can now use threads
include libc/thread/thread.mk # │ You can now use processes
include tool/hello/hello.mk # │ You can finally call malloc()
include third_party/zlib/zlib.mk # │
include libc/stdio/stdio.mk # │
include third_party/libcxx/libcxx.mk # │
include libc/time/time.mk # │
include net/net.mk # │
include third_party/vqsort/vqsort.mk # │
include libc/log/log.mk # │
include third_party/getopt/getopt.mk # │
include third_party/bzip2/bzip2.mk # │
include dsp/core/core.mk # │
include third_party/musl/musl.mk # │
include libc/x/x.mk # │
include third_party/stb/stb.mk # │
include dsp/scale/scale.mk # │
include dsp/mpeg/mpeg.mk # │
include dsp/dsp.mk # │
include third_party/zlib/gz/gz.mk # │
include third_party/musl/musl.mk # │
include third_party/intel/intel.mk # │
include third_party/aarch64/aarch64.mk # │
include libc/libc.mk #─┘
include libc/sock/sock.mk #─┐
include dsp/tty/tty.mk # ├──ONLINE RUNTIME
include libc/dns/dns.mk # │ You can communicate with the network
include net/http/http.mk # │
include third_party/mbedtls/mbedtls.mk # │
include third_party/libcxx/libcxx.mk # │
include third_party/ggml/ggml.mk # │
include third_party/radpajama/radpajama.mk # │
include net/https/https.mk # │
include third_party/regex/regex.mk #─┘
include third_party/tidy/tidy.mk
@@ -172,6 +224,8 @@ include third_party/maxmind/maxmind.mk
include net/finger/finger.mk
include third_party/double-conversion/test/test.mk
include third_party/lua/lua.mk
include third_party/tree/tree.mk
include third_party/zstd/zstd.mk
include third_party/tr/tr.mk
include third_party/sed/sed.mk
include third_party/awk/awk.mk
@@ -186,15 +240,15 @@ include third_party/mbedtls/test/test.mk
include third_party/quickjs/quickjs.mk
include third_party/lz4cli/lz4cli.mk
include third_party/zip/zip.mk
include third_party/xxhash/xxhash.mk
include third_party/unzip/unzip.mk
include tool/build/lib/buildlib.mk
include third_party/chibicc/chibicc.mk
include third_party/chibicc/test/test.mk
include third_party/python/python.mk
include tool/build/emucrt/emucrt.mk
include tool/build/emubin/emubin.mk
include tool/build/build.mk
include tool/curl/curl.mk
include third_party/qemu/qemu.mk
include examples/examples.mk
include examples/pyapp/pyapp.mk
include examples/pylife/pylife.mk
@@ -205,7 +259,6 @@ include tool/lambda/lambda.mk
include tool/plinko/lib/lib.mk
include tool/plinko/plinko.mk
include test/tool/plinko/test.mk
include tool/hash/hash.mk
include tool/net/net.mk
include tool/viz/viz.mk
include tool/tool.mk
@@ -226,8 +279,8 @@ include test/libc/xed/test.mk
include test/libc/fmt/test.mk
include test/libc/dns/test.mk
include test/libc/time/test.mk
include test/libc/proc/test.mk
include test/libc/stdio/test.mk
include test/libc/zipos/test.mk
include test/libc/release/test.mk
include test/libc/test.mk
include test/net/http/test.mk
@@ -266,14 +319,14 @@ tags: TAGS HTAGS
o/$(MODE)/.x:
@$(COMPILE) -AMKDIR -tT$@ $(MKDIR) $(@D)
o/$(MODE)/srcs.txt: o/$(MODE)/.x $(MAKEFILES) $(call uniq,$(foreach x,$(SRCS),$(dir $(x))))
o/$(MODE)/srcs.txt: o/$(MODE)/.x $(MAKEFILES) $(call uniq,$(foreach x,$(SRCS),$(dir $(x)))) $(SRCS)
$(file >$@,$(SRCS))
o/$(MODE)/hdrs.txt: o/$(MODE)/.x $(MAKEFILES) $(call uniq,$(foreach x,$(HDRS) $(INCS),$(dir $(x))))
o/$(MODE)/hdrs.txt: o/$(MODE)/.x $(MAKEFILES) $(call uniq,$(foreach x,$(HDRS) $(INCS),$(dir $(x)))) $(HDRS) $(INCS)
$(file >$@,$(HDRS) $(INCS))
o/$(MODE)/incs.txt: o/$(MODE)/.x $(MAKEFILES) $(call uniq,$(foreach x,$(INCS) $(INCS),$(dir $(x))))
o/$(MODE)/incs.txt: o/$(MODE)/.x $(MAKEFILES) $(call uniq,$(foreach x,$(INCS) $(INCS),$(dir $(x)))) $(INCS) $(INCS)
$(file >$@,$(INCS))
o/$(MODE)/depend: o/$(MODE)/.x o/$(MODE)/srcs.txt o/$(MODE)/hdrs.txt o/$(MODE)/incs.txt $(SRCS) $(HDRS) $(INCS)
@$(COMPILE) -AMKDEPS -L320 $(MKDEPS) -o $@ -r o/$(MODE)/ @o/$(MODE)/srcs.txt @o/$(MODE)/hdrs.txt @o/$(MODE)/incs.txt
$(COMPILE) -AMKDEPS -L320 $(MKDEPS) -o $@ -r o/$(MODE)/ @o/$(MODE)/srcs.txt @o/$(MODE)/hdrs.txt @o/$(MODE)/incs.txt
o/$(MODE)/srcs-old.txt: o/$(MODE)/.x $(MAKEFILES) $(call uniq,$(foreach x,$(SRCS),$(dir $(x))))
$(file >$@) $(foreach x,$(SRCS),$(file >>$@,$(x)))
@@ -281,12 +334,12 @@ o/$(MODE)/hdrs-old.txt: o/$(MODE)/.x $(MAKEFILES) $(call uniq,$(foreach x,$(HDRS
$(file >$@) $(foreach x,$(HDRS) $(INCS),$(file >>$@,$(x)))
TAGS: private .UNSANDBOXED = 1
TAGS: o/$(MODE)/srcs-old.txt $(SRCS) o/$(MODE)/third_party/ctags/ctags.com
TAGS: o/$(MODE)/srcs-old.txt $(SRCS) #o/$(MODE)/third_party/ctags/ctags.com
@$(RM) $@
@o/$(MODE)/third_party/ctags/ctags.com $(TAGSFLAGS) -L $< -o $@
HTAGS: private .UNSANDBOXED = 1
HTAGS: o/$(MODE)/hdrs-old.txt $(HDRS) o/$(MODE)/third_party/ctags/ctags.com
HTAGS: o/$(MODE)/hdrs-old.txt $(filter-out third_party/libcxx/%,$(HDRS)) #o/$(MODE)/third_party/ctags/ctags.com
@$(RM) $@
@build/htags o/$(MODE)/third_party/ctags/ctags.com -L $< -o $@
@@ -298,23 +351,23 @@ loc: o/$(MODE)/tool/build/summy.com
# PLEASE: MAINTAIN TOPOLOGICAL ORDER
# FROM HIGHEST LEVEL TO LOWEST LEVEL
COSMOPOLITAN_OBJECTS = \
TOOL_ARGS \
NET_HTTP \
LIBC_DNS \
LIBC_SOCK \
LIBC_NT_WS2_32 \
LIBC_NT_IPHLPAPI \
LIBC_NT_MSWSOCK \
LIBC_X \
THIRD_PARTY_GETOPT \
LIBC_LOG \
LIBC_TIME \
LIBC_ZIPOS \
THIRD_PARTY_ZLIB \
THIRD_PARTY_MUSL \
THIRD_PARTY_ZLIB_GZ \
LIBC_STDIO \
THIRD_PARTY_GDTOA \
THIRD_PARTY_REGEX \
LIBC_THREAD \
LIBC_PROC \
THIRD_PARTY_NSYNC_MEM \
LIBC_MEM \
THIRD_PARTY_DLMALLOC \
@@ -329,12 +382,13 @@ COSMOPOLITAN_OBJECTS = \
LIBC_NT_PDH \
LIBC_NT_GDI32 \
LIBC_NT_COMDLG32 \
LIBC_NT_URL \
LIBC_NT_USER32 \
LIBC_NT_NTDLL \
LIBC_NT_ADVAPI32 \
LIBC_NT_SYNCHRONIZATION \
LIBC_FMT \
THIRD_PARTY_ZLIB \
THIRD_PARTY_PUFF \
THIRD_PARTY_COMPILER_RT \
LIBC_TINYMATH \
THIRD_PARTY_XED \
@@ -359,6 +413,7 @@ COSMOPOLITAN_HEADERS = \
LIBC_RUNTIME \
LIBC_SOCK \
LIBC_STDIO \
LIBC_PROC \
THIRD_PARTY_NSYNC \
THIRD_PARTY_XED \
LIBC_STR \
@@ -367,9 +422,9 @@ COSMOPOLITAN_HEADERS = \
LIBC_TIME \
LIBC_TINYMATH \
LIBC_X \
LIBC_ZIPOS \
LIBC_VGA \
NET_HTTP \
TOOL_ARGS \
THIRD_PARTY_DLMALLOC \
THIRD_PARTY_GDTOA \
THIRD_PARTY_GETOPT \
@@ -385,7 +440,10 @@ o/cosmopolitan.h: \
$(foreach x,$(COSMOPOLITAN_HEADERS),$($(x)_HDRS)) \
$(foreach x,$(COSMOPOLITAN_HEADERS),$($(x)_INCS))
$(file >$(TMPDIR)/$(subst /,_,$@),libc/integral/normalize.inc $(foreach x,$(COSMOPOLITAN_HEADERS),$($(x)_HDRS)))
@$(COMPILE) -AROLLUP -T$@ o/$(MODE)/tool/build/rollup.com @$(TMPDIR)/$(subst /,_,$@) >$@
@$(ECHO) '#ifndef __STRICT_ANSI__' >$@
@$(ECHO) '#define _COSMO_SOURCE' >>$@
@$(ECHO) '#endif' >>$@
@$(COMPILE) -AROLLUP -T$@ o/$(MODE)/tool/build/rollup.com @$(TMPDIR)/$(subst /,_,$@) >>$@
o/cosmopolitan.html: private .UNSANDBOXED = 1
o/cosmopolitan.html: \
@@ -396,7 +454,7 @@ o/cosmopolitan.html: \
$(file >$(TMPDIR)/$(subst /,_,$@),$(filter-out %.s,$(foreach x,$(COSMOPOLITAN_OBJECTS),$($(x)_SRCS))))
o/$(MODE)/third_party/chibicc/chibicc.com.dbg -J \
-fno-common -include libc/integral/normalize.inc -o $@ \
@$(TMPDIR)/$(subst /,_,$@)
-DCOSMO @$(TMPDIR)/$(subst /,_,$@)
$(SRCS): \
libc/integral/normalize.inc \
@@ -406,14 +464,50 @@ $(SRCS): \
libc/integral/lp64arg.inc \
libc/integral/lp64.inc
ifeq ($(ARCH), x86_64)
TOOLCHAIN_ARTIFACTS = \
o/cosmopolitan.h \
o/$(MODE)/ape/ape.lds \
o/$(MODE)/libc/crt/crt.o \
o/$(MODE)/ape/ape.elf \
o/$(MODE)/ape/ape.o \
o/$(MODE)/ape/ape-copy-self.o \
o/$(MODE)/ape/ape-no-modify-self.o \
o/$(MODE)/cosmopolitan.a \
o/$(MODE)/third_party/libcxx/libcxx.a \
o/$(MODE)/tool/build/march-native.com \
o/$(MODE)/tool/build/ar.com \
o/$(MODE)/tool/build/mktemper.com \
o/$(MODE)/tool/build/fixupobj.com \
o/$(MODE)/tool/build/zipcopy.com \
o/$(MODE)/tool/build/apelink.com \
o/$(MODE)/tool/build/pecheck.com
else
TOOLCHAIN_ARTIFACTS = \
o/$(MODE)/ape/ape.elf \
o/$(MODE)/ape/aarch64.lds \
o/$(MODE)/libc/crt/crt.o \
o/$(MODE)/cosmopolitan.a \
o/$(MODE)/third_party/libcxx/libcxx.a \
o/$(MODE)/tool/build/march-native.com \
o/$(MODE)/tool/build/fixupobj.com \
o/$(MODE)/tool/build/zipcopy.com
endif
.PHONY: toolchain
toolchain: o/cosmopolitan.h \
o/$(MODE)/ape/public/ape.lds \
o/$(MODE)/libc/crt/crt.o \
o/$(MODE)/ape/ape.o \
o/$(MODE)/ape/ape-copy-self.o \
o/$(MODE)/ape/ape-no-modify-self.o \
o/$(MODE)/cosmopolitan.a
toolchain: $(TOOLCHAIN_ARTIFACTS)
.PHONY: clean_toolchain
clean_toolchain:
$(RM) $(TOOLCHAIN_ARTIFACTS)
aarch64: private .INTERNET = true
aarch64: private .UNSANDBOXED = true
aarch64:
$(MAKE) m=aarch64
clean:
$(RM) -r o
# UNSPECIFIED PREREQUISITES TUTORIAL
#
+428 -89
View File
@@ -19,133 +19,334 @@ libc](https://justine.lol/cosmopolitan/index.html) website. We also have
## Getting Started
If you're doing your development work on Linux or BSD then you need just
five files to get started. Here's what you do on Linux:
It's recommended that Cosmopolitan be installed to `/opt/cosmo` and
`/opt/cosmos` on your computer. The first has the monorepo. The second
contains your non-monorepo artifacts.
```sh
wget https://justine.lol/cosmopolitan/cosmopolitan-amalgamation-2.2.zip
unzip cosmopolitan-amalgamation-2.2.zip
printf 'main() { printf("hello world\\n"); }\n' >hello.c
gcc -g -Os -static -nostdlib -nostdinc -fno-pie -no-pie -mno-red-zone \
-fno-omit-frame-pointer -pg -mnop-mcount -mno-tls-direct-seg-refs -gdwarf-4 \
-o hello.com.dbg hello.c -fuse-ld=bfd -Wl,-T,ape.lds -Wl,--gc-sections \
-include cosmopolitan.h crt.o ape-no-modify-self.o cosmopolitan.a
objcopy -S -O binary hello.com.dbg hello.com
sudo mkdir -p /opt
sudo chmod 1777 /opt
git clone https://github.com/jart/cosmopolitan /opt/cosmo
export PATH="/opt/cosmo/bin:/opt/cosmos/bin:$PATH"
echo 'PATH="/opt/cosmo/bin:/opt/cosmos/bin:$PATH"' >>~/.profile
ape-install # optionally install a faster systemwide ape loader
cosmocc --update # pull cosmo and rebuild toolchain
```
You now have a portable program.
You've now successfully installed your very own cosmos. Now let's build
an example program:
```c
// hello.c
#include <stdio.h>
int main() {
printf("hello world\n");
}
```
To compile the program, you can run the `cosmocc` command. It's
important to give it an output path that ends with `.com` so the output
format will be Actually Portable Executable. When this happens, a
concomitant debug binary is created automatically too.
```sh
cosmocc -o hello.com hello.c
./hello.com
bash -c './hello.com' # zsh/fish workaround (we patched them in 2021)
./hello.com.dbg
```
If `./hello.com` executed on Linux throws an error about not finding an
interpreter, it should be fixed by running the following command (although
note that it may not survive a system restart):
You can use the `cosmocc` toolchain to build conventional open source
projects which use autotools. This strategy normally works:
```sh
sudo sh -c "echo ':APE:M::MZqFpD::/bin/sh:' >/proc/sys/fs/binfmt_misc/register"
export CC=cosmocc
export CXX=cosmoc++
./configure --prefix=/opt/cosmos
make -j
make install
```
If the same command produces puzzling errors on WSL or WINE when using
Redbean 2.x, they may be fixed by disabling binfmt_misc:
The Cosmopolitan Libc runtime links some heavyweight troubleshooting
features by default, which are very useful for developers and admins.
Here's how you can log system calls:
```sh
sudo sh -c 'echo -1 >/proc/sys/fs/binfmt_misc/status'
./hello.com --strace
```
Since we used the `ape-no-modify-self.o` bootloader (rather than
`ape.o`) your executable will not modify itself when it's run. What
it'll instead do, is extract a 4kb program (the [APE loader](https://justine.lol/apeloader/))
to `${TMPDIR:-${HOME:-.}}` that maps your program into memory without
needing to copy it. The APE loader must be in an executable location
(e.g. not stored on a `noexec` mount) for it to run. See below for
alternatives:
It's possible to install the APE loader systemwide as follows.
Here's how you can get a much more verbose log of function calls:
```sh
# System-Wide APE Install
# for Linux, Darwin, and BSDs
# 1. Copies APE Loader to /usr/bin/ape
# 2. Registers w/ binfmt_misc too if Linux
ape/apeinstall.sh
# System-Wide APE Uninstall
# for Linux, Darwin, and BSDs
ape/apeuninstall.sh
./hello.com --ftrace
```
It's also possible to convert APE binaries into the system-local format
by using the `--assimilate` flag. Please note that if binfmt_misc is in
play, you'll need to unregister it temporarily before doing this, since
the assimilate feature is part of the shell script header.
If you don't want rich runtime features like the above included, and you
just want libc, and you want smaller simpler programs. In that case, you
can consider using `MODE=tiny`, which is preconfigured by the repo in
[build/config.mk](build/config.mk). Using this mode is much more
effective at reducing binary footprint than the `-Os` flag alone. You
can change your build mode by doing the following:
```sh
$ file hello.com
hello.com: DOS/MBR boot sector
./hello.com --assimilate
$ file hello.com
hello.com: ELF 64-bit LSB executable
export MODE=tiny
cosmocc --update
```
Now that you're up and running with Cosmopolitan Libc and APE, here's
some of the most important troubleshooting tools APE offers that you
should know, in case you encounter any issues:
We can also make our program slightly smaller by using the system call
interface directly, which is fine, since Cosmopolitan polyfills these
interfaces across platforms, including Windows. For example:
```c
// hello2.c
#include <unistd.h>
int main() {
write(1, "hello world\n", 12);
}
```
Once compiled, your APE binary should be ~36kb in size.
```sh
./hello.com --strace # log system calls to stderr
./hello.com --ftrace # log function calls to stderr
export MODE=tiny
cosmocc -Os -o hello2.com hello2.c
./hello2.com
```
Do you love tiny binaries? If so, you may not be happy with Cosmo adding
heavyweight features like tracing to your binaries by default. In that
case, you may want to consider using our build system:
But let's say you only care about your binaries running on Linux and you
don't want to use up all this additional space for platforms like WIN32.
In that case, you can try `MODE=tinylinux` for example which will create
executables more on the order of 8kb (similar to Musl Libc).
```sh
make m=tiny
export MODE=tinylinux
cosmocc --update
cosmocc -Os -o hello2.com hello2.c
./hello2.com # <-- actually an ELF executable
```
Which will cause programs such as `hello.com` and `life.com` to shrink
from 60kb in size to about 16kb. There's also a prebuilt amalgamation
online <https://justine.lol/cosmopolitan/cosmopolitan-tiny.zip> hosted
on our download page <https://justine.lol/cosmopolitan/download.html>.
## ARM
### MacOS
If you're developing on MacOS you can install the GNU compiler
collection for x86_64-elf via homebrew:
Cosmo supports cross-compiling binaries for machines with ARM
microprocessors. For example:
```sh
brew install x86_64-elf-gcc
make -j8 m=aarch64 o/aarch64/third_party/ggml/llama.com
make -j8 m=aarch64-tiny o/aarch64-tiny/third_party/ggml/llama.com
```
Then in the above scripts just replace `gcc` and `objcopy` with
`x86_64-elf-gcc` and `x86_64-elf-objcopy` to compile your APE binary.
That'll produce ELF executables that run natively on two operating
systems: Linux Arm64 (e.g. Raspberry Pi) and MacOS Arm64 (i.e. Apple
Silicon), thus giving you full performance. The catch is you have to
compile these executables on an x86_64-linux machine. The second catch
is that MacOS needs a little bit of help understanding the ELF format.
To solve that, we provide a tiny APE loader you can use on M1 machines.
### Windows
If you're developing on Windows then you need to download an
x86_64-pc-linux-gnu toolchain beforehand. See the [Compiling on
Windows](https://justine.lol/cosmopolitan/windows-compiling.html)
tutorial. It's needed because the ELF object format is what makes
universal binaries possible.
Cosmopolitan officially only builds on Linux. However, one highly
experimental (and currently broken) thing you could try, is building the
entire cosmo repository from source using the cross9 toolchain.
```
mkdir -p o/third_party
rm -rf o/third_party/gcc
wget https://justine.lol/linux-compiler-on-windows/cross9.zip
unzip cross9.zip
mv cross9 o/third_party/gcc
build/bootstrap/make.com
```sh
scp ape/ape-m1.c macintosh:
scp o/aarch64/third_party/ggml/llama.com macintosh:
ssh macintosh
xcode-install
cc -o ape ape-m1.c
sudo cp ape /usr/local/bin/ape
```
## Source Builds
You can run your ELF AARCH64 executable on Apple Silicon as follows:
```sh
ape ./llama.com
```
If you want to run the `MODE=aarch64` unit tests, you need to have
qemu-aarch64 installed as a binfmt_misc interpreter. It needs to be a
static binary if you want it to work with Landlock Make's security. You
can use the build included in our `third_party/qemu/` folder.
```
doas cp o/third_party/qemu/qemu-aarch64 /usr/bin/qemu-aarch64
doas sh -c "echo ':qemu-aarch64:M::\x7fELF\x02\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\xb7\x00:\xff\xff\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff\xff:/usr/bin/qemu-aarch64:CF' > /proc/sys/fs/binfmt_misc/register"
make -j8 m=aarch64
```
Please note that the qemu-aarch64 binfmt_misc interpreter installation
process is *essential* for being able to use the `aarch64-unknown-cosmo`
toolchain to build fat APE binaries on your x86-64 machine.
## AMD64 + ARM64 fat APE binaries
If you've setup the qemu binfmt_misc interpreter, then you can can use
cosmo's toolchains to build fat ape binaries. It works by compiling your
program twice, so you can have a native build for both architectures in
the same file. The two programs are merged together by apelink.com which
also embeds multiple copies of APE loader and multiple symbols tables.
The easiest way to build fat APE is using `fatcosmocc`. This compiler
works by creating a concomitant `.aarch64/foo.o` for every `foo.o` you
compile. The only exception is the C preprocessor mode, which actually
runs x86-64 GCC except with macros like `__x86_64__` undefined.
This toolchain works great for C projects that are written in a portable
way and don't produce architecture-specific artifacts. One example of a
large project that can be easily built is GNU coreutils.
```sh
cd coreutils
fatcosmocc --update ||exit
./configure CC=fatcosmocc \
AR=fatcosmoar \
INSTALL=$(command -v fatcosmoinstall) \
--prefix=/opt/cosmos \
--disable-nls \
--disable-dependency-tracking \
--disable-silent-rules
make -j8
```
You'll then have a bunch of files like `src/ls` which are fat ape
binaries. If you want to run them on Windows, then you simply need to
rename the file so that it has the `.com` suffix. Better yet, consider
making that a symlink (a.k.a. reparse point). The biggest gotcha with
`fatcosmocc` though is ensuring builds don't strip binaries. For
example, Linux's `install -s` command actually understands Windows'
Portable Executable format well enough to remove the MS-DOS stub, which
is where the APE shell script is stored. You need to ensure that
`fatcosmoinstall` is used instead. Especially if your project needs to
install the libraries built by `fatacosmoar` into `/opt/cosmos`.
## Advanced Fat APE Builds
Once you get seriously involved in creating fat APE builds of software
you're going to eventually outgrow `fatcosmocc`. One example is Emacs
which is trickier to build, because it produces architecture-specific
files, and it also depends on shared files, e.g. zoneinfo. Since we like
having everything in a neat little single-file executable container that
doesn't need an "installation wizard", this tutorial will explain how we
manage to accomplish that.
What you're going to do is, instead of using `fatcosmocc`, you're going
to use both the `x86_64-unknown-cosmo-cc` and `aarch64-unknown-cosmo-cc`
toolchains independently, and then run `apelink` and `zip` to manually
build the final files. But there's a few tricks to learn first.
The first trick is to create a symlink on your system called `/zip`.
Cosmopolitan Libc normally uses that as a synthetic folder that lets you
access the assets in your zip executable. But since that's a read-only
file system, your build system should use the normal one.
```sh
doas ln -sf /opt/cosmos /zip
```
Now create a file named `rebuild-fat.sh` which runs the build twice:
```sh
#!/bin/sh
set -ex
export MODE=aarch64
export COSMOS=/opt/cosmos/aarch64
rebuild-cosmos.sh aarch64
export MODE=
export COSMOS=/opt/cosmos/x86_64
rebuild-cosmos.sh x86_64
wall.com 'finished building'
```
Then create a second file `rebuild-cosmos.sh` which runs your build:
```sh
#!/bin/bash
set -ex
ARCH=${1:-x86_64}
export COSMO=${COSMO:-/opt/cosmo}
export COSMOS=${COSMOS:-/opt/cosmos/$ARCH}
export AS=$(command -v $ARCH-unknown-cosmo-as) || exit
export CC=$(command -v $ARCH-unknown-cosmo-cc) || exit
export CXX=$(command -v $ARCH-unknown-cosmo-c++) || exit
export AR=$(command -v $ARCH-unknown-cosmo-ar) || exit
export STRIP=$(command -v $ARCH-unknown-cosmo-strip) || exit
export INSTALL=$(command -v $ARCH-unknown-cosmo-install) || exit
export OBJCOPY=$(command -v $ARCH-unknown-cosmo-objcopy) || exit
export OBJDUMP=$(command -v $ARCH-unknown-cosmo-objdump) || exit
export ADDR2LINE=$(command -v $ARCH-unknown-cosmo-addr2line) || exit
$CC --update
export COSMOPOLITAN_DISABLE_ZIPOS=1
cd ~/vendor/zlib
./configure --prefix=$COSMOS --static
make clean
make -j
make install
cd ~/vendor/ncurses-6.4
./configure --prefix=$COSMOS --sysconfdir=/zip --datarootdir=/zip/share --exec-prefix=/zip/$ARCH --disable-shared
make clean
make -j
make install
cd ~/vendor/readline-8.2
./configure --prefix=$COSMOS --sysconfdir=/zip --datarootdir=/zip/share --exec-prefix=/zip/$ARCH --disable-shared
make uninstall || true
make clean
make -j
make install
# NOTES:
# 1. You'll need to patch enum { FOO = x } that fails to build into a #define FOO
# 2. You'll need to patch configure.ac so it DOES NOT define USABLE_FIONREAD to 1
# 2. You'll need to patch configure.ac so it DOES NOT define INTERRUPT_INPUT to 1
cd ~/vendor/emacs-28.2
./configure --prefix=$COSMOS --sysconfdir=/zip --datarootdir=/zip/share --exec-prefix=/zip/$ARCH \
--without-x --with-threads --without-gnutls --disable-silent-rules --with-file-notification=no
make uninstall || true
make clean
make -j
make install
```
Once you've completed this build process, you'll have the ELF files
`/opt/cosmos/x86_64/bin/emacs` and `/opt/cosmos/aarch64/bin/emacs`. Your
next move is to combine them into a single pristine `emacs.com` file.
```sh
cd /zip
COSMO=${COSMO:-/opt/cosmo}
mkdir -p /opt/cosmos/bin
apelink \
-o /opt/cosmos/bin/emacs.com \
-l "$COSMO/o//ape/ape.elf" \
-l "$COSMO/o/aarch64/ape/ape.elf" \
-M "$COSMO/ape/ape-m1.c" \
/opt/cosmos/x86_64/bin/emacs \
/opt/cosmos/aarch64/bin/emacs
cd /zip
zip -r /opt/cosmos/bin/emacs.com \
aarch64/libexec \
x86_64/libexec \
share/terminfo \
$(find share/emacs -type f |
grep -v '\.el.gz$' |
grep -v refcards |
grep -v images)
```
You can now scp your `emacs.com` build to seven operating systems for
two distinct kinds of microprocessors without any dependencies. All the
LISP, zoneinfo, and termcap files it needs are stored inside the ZIP
structure of the binary, which has performance that's equivalent to the
Linux filesystem (even though it decompresses artifacts on the fly!) For
this reason, you might actually find that fat APE Emacs goes faster if
you're using an operating system like Windows where files are go slow.
If you like to use Vim instead of Emacs, then you can build that too.
However Vim's build system makes it a bit harder, since it's configured
to always strip binaries. The `apelink` program needs the symbol tables
to still be there when it creates the fat version. Otherwise tools like
`--ftrace` won't work.
## Monolithic Source Builds
Cosmopolitan can be compiled from source on any Linux distro. First, you
need to download or clone the repository.
@@ -169,7 +370,7 @@ WINE installed to `binfmt_misc`. You can fix that by installing the the
APE loader as an interpreter. It'll improve build performance too!
```sh
ape/apeinstall.sh
bin/ape-install
```
Since the Cosmopolitan repository is very large, you might only want to
@@ -212,6 +413,104 @@ build/bootstrap/make.com m=tinylinux # tiniest linux-only 4kb binaries
For further details, see [//build/config.mk](build/config.mk).
## Cosmopolitan Amalgamation
Another way to use Cosmopolitan is via our amalgamated release, where
we've combined everything into a single static archive and a single
header file. If you're doing your development work on Linux or BSD then
you need just five files to get started. Here's what you do on Linux:
```sh
wget https://justine.lol/cosmopolitan/cosmopolitan-amalgamation-3.0.zip
unzip cosmopolitan-amalgamation-3.0.zip
printf 'main() { printf("hello world\\n"); }\n' >hello.c
gcc -g -Os -static -nostdlib -nostdinc -fno-pie -no-pie -mno-red-zone \
-fno-omit-frame-pointer -pg -mnop-mcount -mno-tls-direct-seg-refs -gdwarf-4 \
-o hello.com.dbg hello.c -fuse-ld=bfd -Wl,-T,ape.lds -Wl,--gc-sections \
-Wl,-z,common-page-size=0x1000 -Wl,-z,max-page-size=0x1000 \
-include cosmopolitan.h crt.o ape-no-modify-self.o cosmopolitan.a
objcopy -S -O binary hello.com.dbg hello.com
```
You now have a portable program.
```sh
./hello.com
bash -c './hello.com' # older zsh/fish workaround (patched in zsh 5.9 and fish 3.3.0)
```
If `./hello.com` executed on Linux throws an error about not finding an
interpreter, it should be fixed by running the following command (although
note that it may not survive a system restart):
```sh
sudo sh -c "echo ':APE:M::MZqFpD::/bin/sh:' >/proc/sys/fs/binfmt_misc/register"
```
If the same command produces puzzling errors on WSL or WINE when using
Redbean 2.x, they may be fixed by disabling binfmt_misc:
```sh
sudo sh -c 'echo -1 >/proc/sys/fs/binfmt_misc/status'
```
Since we used the `ape-no-modify-self.o` bootloader (rather than
`ape.o`) your executable will not modify itself when it's run. What
it'll instead do, is extract a 4kb program (the [APE loader](https://justine.lol/apeloader/))
to `${TMPDIR:-${HOME:-.}}` that maps your program into memory without
needing to copy it. The APE loader must be in an executable location
(e.g. not stored on a `noexec` mount) for it to run. See below for
alternatives:
It's possible to install the APE loader systemwide as follows.
```sh
# System-Wide APE Install
# for Linux, Darwin, and BSDs
# 1. Copies APE Loader to /usr/bin/ape
# 2. Registers w/ binfmt_misc too if Linux
bin/ape-install
# System-Wide APE Uninstall
# for Linux, Darwin, and BSDs
bin/ape-uninstall
```
It's also possible to convert APE binaries into the system-local format
by using the `--assimilate` flag. Please note that if binfmt_misc is in
play, you'll need to unregister it temporarily before doing this, since
the assimilate feature is part of the shell script header.
```sh
$ file hello.com
hello.com: DOS/MBR boot sector
./hello.com --assimilate
$ file hello.com
hello.com: ELF 64-bit LSB executable
```
Now that you're up and running with Cosmopolitan Libc and APE, here's
some of the most important troubleshooting tools APE offers that you
should know, in case you encounter any issues:
```sh
./hello.com --strace # log system calls to stderr
./hello.com --ftrace # log function calls to stderr
```
Do you love tiny binaries? If so, you may not be happy with Cosmo adding
heavyweight features like tracing to your binaries by default. In that
case, you may want to consider using our build system:
```sh
make m=tiny
```
Which will cause programs such as `hello.com` and `life.com` to shrink
from 60kb in size to about 16kb. There's also a prebuilt amalgamation
online <https://justine.lol/cosmopolitan/cosmopolitan-tiny.zip> hosted
on our download page <https://justine.lol/cosmopolitan/download.html>.
## GDB
Here's the recommended `~/.gdbinit` config:
@@ -239,10 +538,50 @@ src
You normally run the `.com.dbg` file under gdb. If you need to debug the
`.com` file itself, then you can load the debug symbols independently as
```
```sh
gdb foo.com -ex 'add-symbol-file foo.com.dbg 0x401000'
```
## Alternative Development Environments
### MacOS
If you're developing on MacOS you can install the GNU compiler
collection for x86_64-elf via homebrew:
```sh
brew install x86_64-elf-gcc
```
Then in the above scripts just replace `gcc` and `objcopy` with
`x86_64-elf-gcc` and `x86_64-elf-objcopy` to compile your APE binary.
### Windows
If you're developing on Windows then you need to download an
x86_64-pc-linux-gnu toolchain beforehand. See the [Compiling on
Windows](https://justine.lol/cosmopolitan/windows-compiling.html)
tutorial. It's needed because the ELF object format is what makes
universal binaries possible.
Cosmopolitan officially only builds on Linux. However, one highly
experimental (and currently broken) thing you could try, is building the
entire cosmo repository from source using the cross9 toolchain.
```sh
mkdir -p o/third_party
rm -rf o/third_party/gcc
wget https://justine.lol/linux-compiler-on-windows/cross9.zip
unzip cross9.zip
mv cross9 o/third_party/gcc
build/bootstrap/make.com
```
## Discord Chatroom
The Cosmopolitan development team collaborates on the Redbean Discord
server. You're welcome to join us! <https://discord.gg/FwAVVu7eJ4>
## Support Vector
| Platform | Min Version | Circa |
@@ -252,7 +591,7 @@ gdb foo.com -ex 'add-symbol-file foo.com.dbg 0x401000'
| Linux | 2.6.18 | 2007 |
| Windows | 8 [1] | 2012 |
| Mac OS X | 15.6 | 2018 |
| OpenBSD | 6.4 | 2018 |
| OpenBSD | 7 | 2021 |
| FreeBSD | 13 | 2020 |
| NetBSD | 9.2 | 2021 |
+294
View File
@@ -0,0 +1,294 @@
/*-*- mode: ld-script; indent-tabs-mode: nil; tab-width: 2; coding: utf-8 -*-│
│vi: set et sts=2 tw=2 fenc=utf-8 :vi│
╚─────────────────────────────────────────────────────────────────────────────*/
#include "libc/thread/tls.h"
ENTRY(_start)
OUTPUT_ARCH(aarch64)
OUTPUT_FORMAT("elf64-littleaarch64",
"elf64-bigaarch64",
"elf64-littleaarch64")
SECTIONS {
. = SEGMENT_START("text-segment", 0x010000000000);
__executable_start = .;
. += SIZEOF_HEADERS;
.interp : { *(.interp) }
.note.gnu.build-id : { *(.note.gnu.build-id) }
.hash : { *(.hash) }
.gnu.hash : { *(.gnu.hash) }
.dynsym : { *(.dynsym) }
.dynstr : { *(.dynstr) }
.gnu.version : { *(.gnu.version) }
.gnu.version_d : { *(.gnu.version_d) }
.gnu.version_r : { *(.gnu.version_r) }
.rela.init : { *(.rela.init) }
.rela.text : { *(.rela.text .rela.text.* .rela.gnu.linkonce.t.*) }
.rela.fini : { *(.rela.fini) }
.rela.rodata : { *(.rela.rodata .rela.rodata.* .rela.gnu.linkonce.r.*) }
.rela.data.rel.ro : { *(.rela.data.rel.ro .rela.data.rel.ro.* .rela.gnu.linkonce.d.rel.ro.*) }
.rela.data : { *(.rela.data .rela.data.* .rela.gnu.linkonce.d.*) }
.rela.tdata : { *(.rela.tdata .rela.tdata.* .rela.gnu.linkonce.td.*) }
.rela.tbss : { *(.rela.tbss .rela.tbss.* .rela.gnu.linkonce.tb.*) }
.rela.ctors : { *(.rela.ctors) }
.rela.dtors : { *(.rela.dtors) }
.rela.got : { *(.rela.got) }
.rela.bss : { *(.rela.bss .rela.bss.* .rela.gnu.linkonce.b.*) }
.rela.ifunc : { *(.rela.ifunc) }
.rela.plt : {
*(.rela.plt)
__rela_iplt_start = .;
*(.rela.iplt)
__rela_iplt_end = .;
}
.init : {
*(.start)
KEEP(*(.initprologue))
KEEP(*(SORT_NONE(.init)))
KEEP(*(.initepilogue))
} =0x1f2003d5
.plt : ALIGN(16) {
*(.plt)
*(.iplt)
}
.text : {
*(.text.unlikely .text.*_unlikely .text.unlikely.*)
*(.text.antiquity .text.antiquity.*)
*(.text.exit .text.exit.*)
*(.text.startup .text.startup.*)
*(.text.hot .text.hot.*)
*(.text.modernity .text.modernity.*)
*(.text .stub .text.* .gnu.linkonce.t.*)
*(.gnu.warning)
} =0x1f2003d5
.fini : {
KEEP(*(SORT_NONE(.fini)))
} =0x1f2003d5
. += CONSTANT(MAXPAGESIZE);
.privileged : {
__privileged_start = ABSOLUTE(.) & -CONSTANT(MAXPAGESIZE);
*(.privileged*)
} =0x1f2003d6
.rodata : {
KEEP(*(.rodata.pytab.0));
KEEP(*(.rodata.pytab.1));
KEEP(*(.rodata.pytab.2));
KEEP(*(SORT_BY_NAME(.sort.rodata.*)))
*(.rodata .rodata.* .gnu.linkonce.r.*)
*(.ubsan.types)
*(.ubsan.data)
}
.comment : {
__comment_start = .;
KEEP(*(.comment))
BYTE(0);
}
.eh_frame_hdr : {
*(.eh_frame_hdr)
*(.eh_frame_entry .eh_frame_entry.*)
}
.eh_frame : ONLY_IF_RO {
KEEP(*(.eh_frame))
*(.eh_frame.*)
}
.gcc_except_table : ONLY_IF_RO {
*(.gcc_except_table .gcc_except_table.*)
}
.gnu_extab : ONLY_IF_RO {
*(.gnu_extab*)
}
.exception_ranges : ONLY_IF_RO {
*(.exception_ranges*)
}
__etext = .;
_etext = .;
PROVIDE(etext = .);
. += CONSTANT(MAXPAGESIZE);
. = DATA_SEGMENT_ALIGN(CONSTANT(MAXPAGESIZE), CONSTANT(COMMONPAGESIZE));
.eh_frame : ONLY_IF_RW {
KEEP(*(.eh_frame))
*(.eh_frame.*)
}
.gnu_extab : ONLY_IF_RW {
*(.gnu_extab)
}
.gcc_except_table : ONLY_IF_RW {
*(.gcc_except_table .gcc_except_table.*)
}
.exception_ranges : ONLY_IF_RW {
*(.exception_ranges*)
}
.tdata : {
_tdata_start = .;
__tdata_start = .;
*(.tdata .tdata.* .gnu.linkonce.td.*)
_tdata_end = .;
}
.tbss : {
_tbss_start = .;
*(.tbss .tbss.* .gnu.linkonce.tb.*)
*(.tcommon)
_tbss_end = .;
}
.init_array : {
__init_array_start = .;
KEEP(*(SORT_BY_INIT_PRIORITY(.init_array.*) SORT_BY_INIT_PRIORITY(.ctors.*)))
KEEP(*(.init_array EXCLUDE_FILE(*crtbegin.o *crtbegin?.o *crtend.o *crtend?.o ) .ctors))
__init_array_end = .;
}
.fini_array : {
__fini_array_start = .;
KEEP(*(SORT_BY_INIT_PRIORITY(.fini_array.*) SORT_BY_INIT_PRIORITY(.dtors.*)))
KEEP(*(.fini_array EXCLUDE_FILE(*crtbegin.o *crtbegin?.o *crtend.o *crtend?.o ) .dtors))
__fini_array_end = .;
}
.data.rel.ro : {
KEEP(*(SORT_BY_NAME(.piro.relo.sort.*)))
*(.data.rel.ro.local* .gnu.linkonce.d.rel.ro.local.*)
*(.data.rel.ro .data.rel.ro.* .gnu.linkonce.d.rel.ro.*)
}
.dynamic : {
*(.dynamic)
}
.got : {
*(.got)
*(.igot)
}
. = DATA_SEGMENT_RELRO_END(24, .);
.got.plt : {
*(.got.plt)
*(.igot.plt)
}
.data : {
__data_start = .;
KEEP(*(SORT_BY_NAME(.piro.data.sort.*)))
*(.data .data.* .gnu.linkonce.d.*)
KEEP(*(SORT_BY_NAME(.sort.data.*)))
SORT(CONSTRUCTORS)
}
_edata = .;
PROVIDE(edata = .);
. = .;
__bss_start = .;
__bss_start__ = .;
.bss : {
*(.dynbss)
*(.bss .bss.* .gnu.linkonce.b.*)
KEEP(*(SORT_BY_NAME(.piro.bss.sort.*)))
*(COMMON)
. = ALIGN(CONSTANT(COMMONPAGESIZE));
}
_bss_end__ = .;
__bss_end__ = .;
. = ALIGN(64 / 8);
. = SEGMENT_START("ldata-segment", .);
. = ALIGN(64 / 8);
__end__ = .;
_end = .;
PROVIDE(end = .);
. = DATA_SEGMENT_END(.);
.stab 0 : { *(.stab) }
.stabstr 0 : { *(.stabstr) }
.stab.excl 0 : { *(.stab.excl) }
.stab.exclstr 0 : { *(.stab.exclstr) }
.stab.index 0 : { *(.stab.index) }
.stab.indexstr 0 : { *(.stab.indexstr) }
.gnu.build.attributes : { *(.gnu.build.attributes .gnu.build.attributes.*) }
.debug 0 : { *(.debug) }
.line 0 : { *(.line) }
.debug_srcinfo 0 : { *(.debug_srcinfo) }
.debug_sfnames 0 : { *(.debug_sfnames) }
.debug_aranges 0 : { *(.debug_aranges) }
.debug_pubnames 0 : { *(.debug_pubnames) }
.debug_info 0 : { *(.debug_info .gnu.linkonce.wi.*) }
.debug_abbrev 0 : { *(.debug_abbrev) }
.debug_line 0 : { *(.debug_line .debug_line.* .debug_line_end) }
.debug_frame 0 : { *(.debug_frame) }
.debug_str 0 : { *(.debug_str) }
.debug_loc 0 : { *(.debug_loc) }
.debug_macinfo 0 : { *(.debug_macinfo) }
.debug_weaknames 0 : { *(.debug_weaknames) }
.debug_funcnames 0 : { *(.debug_funcnames) }
.debug_typenames 0 : { *(.debug_typenames) }
.debug_varnames 0 : { *(.debug_varnames) }
.debug_pubtypes 0 : { *(.debug_pubtypes) }
.debug_ranges 0 : { *(.debug_ranges) }
.debug_macro 0 : { *(.debug_macro) }
.debug_addr 0 : { *(.debug_addr) }
.ARM.attributes 0 : { KEEP(*(.ARM.attributes)) KEEP(*(.gnu.attributes)) }
.note.gnu.arm.ident 0 : { KEEP(*(.note.gnu.arm.ident)) }
.zip 0 : {
KEEP(*(.zip.file))
__zip_cdir_start = .;
KEEP(*(.zip.cdir))
__zip_cdir_size = . - __zip_cdir_start;
KEEP(*(.zip.eocd))
}
/DISCARD/ : {
*(__patchable_function_entries)
*(.GCC.command.line)
*(.note.GNU-stack)
*(.gnu_debuglink)
*(.text.windows)
*(.gnu.lto_*)
*(.eh_frame)
*(.idata.*)
*(.yoink)
*(.head)
}
}
ape_stack_vaddr = DEFINED(ape_stack_vaddr) ? ape_stack_vaddr : 0x700000000000;
ape_stack_memsz = DEFINED(ape_stack_memsz) ? ape_stack_memsz : 8 * 1024 * 1024;
ape_stack_align = DEFINED(ape_stack_align) ? ape_stack_align : 16;
ape_stack_round = -ape_stack_align;
_tls_size = _tbss_end - _tdata_start;
_tdata_size = _tdata_end - _tdata_start;
_tbss_size = _tbss_end - _tbss_start;
_tbss_offset = _tbss_start - _tdata_start;
_tls_content = (_tdata_end - _tdata_start) + (_tbss_end - _tbss_start);
_tls_align = MAX(ALIGNOF(.tdata), ALIGNOF(.tbss));
ASSERT(ALIGNOF(.tdata) <= TLS_ALIGNMENT && ALIGNOF(.tbss) <= TLS_ALIGNMENT,
"_Thread_local _Alignof can't exceed TLS_ALIGNMENT");
+1063
View File
File diff suppressed because it is too large. Load diff
+509 -863
View File
File diff suppressed because it is too large. Load diff
+13
View File
@@ -0,0 +1,13 @@
#ifndef COSMOPOLITAN_APE_APE_H_
#define COSMOPOLITAN_APE_APE_H_
#define APE_VERSION_MAJOR 1
#define APE_VERSION_MINOR 8
#define APE_VERSION_STR APE_VERSION_STR_(APE_VERSION_MAJOR, APE_VERSION_MINOR)
#define APE_VERSION_NOTE APE_VERSION_NOTE_(APE_VERSION_MAJOR, APE_VERSION_MINOR)
#define APE_VERSION_STR__(x, y) #x "." #y
#define APE_VERSION_STR_(x, y) APE_VERSION_STR__(x, y)
#define APE_VERSION_NOTE_(x, y) (100000000 * (x) + 1000000 * (y))
#endif /* COSMOPOLITAN_APE_APE_H_ */
+9
View File
@@ -0,0 +1,9 @@
#ifndef COSMOPOLITAN_APE_APE_INTERNAL_H_
#define COSMOPOLITAN_APE_APE_INTERNAL_H_
#include "libc/dce.h"
#if SupportsWindows() || SupportsMetal() || SupportsXnu()
#define APE_IS_SHELL_SCRIPT
#endif
#endif /* COSMOPOLITAN_APE_APE_INTERNAL_H_ */
+234 -382
View File
@@ -181,16 +181,39 @@
#include "libc/elf/pf2prot.internal.h"
#include "libc/nt/pedef.internal.h"
#include "libc/thread/tls.h"
#include "libc/zip.h"
#include "ape/ape.internal.h"
/* uncomment if .com.dbg won't execute on your kernel (will break .com file) */
/* #define APE_FIX_COM_DBG */
#ifdef __x86__
#define CODE_GRANULE 1
#else
#define CODE_GRANULE 4
#endif
#ifdef APE_FIX_COM_DBG
#define SKEW SIZEOF_HEADERS
#else
#define SKEW 0
#endif
#ifndef IMAGE_BASE_VIRTUAL
#define IMAGE_BASE_VIRTUAL 0x400000
#endif
#if IMAGE_BASE_VIRTUAL > 0xffffffff
#error "please use 32-bit addresses for image data"
#endif
ENTRY(_start)
PHDRS {
Head PT_LOAD FLAGS(PF_X|PF_R);
Rom PT_LOAD FLAGS(PF_X|PF_R);
Ram PT_LOAD FLAGS(PF_W|PF_R);
Cod PT_LOAD FLAGS(PF_X|PF_R);
Rom PT_LOAD FLAGS(PF_R);
Tls PT_TLS FLAGS(PF_W|PF_R);
Bss PT_LOAD FLAGS(PF_W|PF_R);
Ram PT_LOAD FLAGS(PF_W|PF_R);
stack PT_GNU_STACK FLAGS(PF_W|PF_R);
}
@@ -201,44 +224,49 @@ SECTIONS {
/*BEGIN: linux addressability guarantee */
/*BEGIN: bsd addressability guarantee */
.head SEGMENT_START("text-segment", IMAGE_BASE_VIRTUAL) : AT(IMAGE_BASE_REAL) {
HIDDEN(_base = .);
.head SEGMENT_START("text-segment", IMAGE_BASE_VIRTUAL) + SKEW : AT(IMAGE_BASE_REAL) {
__executable_start = .;
/* Real Mode */
KEEP(*(.head))
KEEP(*(.apesh))
KEEP(*(.head2))
KEEP(*(.text.head))
/* Executable & Linkable Format */
. = ALIGN(__SIZEOF_POINTER__);
HIDDEN(ape_phdrs = .);
. = ALIGN(. != 0 ? __SIZEOF_POINTER__ : 0);
ape_phdrs = .;
KEEP(*(.elf.phdrs))
HIDDEN(ape_phdrs_end = .);
ape_phdrs_end = .;
KEEP(*(.emushprologue))
KEEP(*(.emush))
KEEP(*(.emushepilogue))
/* OpenBSD */
. = ALIGN(__SIZEOF_POINTER__);
HIDDEN(ape_note = .);
. = ALIGN(. != 0 ? __SIZEOF_POINTER__ : 0);
ape_note = .;
KEEP(*(.note.openbsd.ident))
KEEP(*(.note.netbsd.ident))
HIDDEN(ape_note_end = .);
ape_note_end = .;
/* Portable Executable */
KEEP(*(.pe.header))
HIDDEN(ape_pe_sections = .);
ape_pe_sections = .;
KEEP(*(.pe.sections))
HIDDEN(ape_pe_sections_end = .);
ape_pe_sections_end = .;
/* Mach-O */
KEEP(*(.macho))
. = ALIGN(__SIZEOF_POINTER__);
HIDDEN(ape_macho_end = .);
. = ALIGN(. != 0 ? __SIZEOF_POINTER__ : 0);
ape_macho_end = .;
/* APE loader */
KEEP(*(.ape.loader))
. = ALIGN(. != 0 ? CODE_GRANULE : 0);
KEEP(*(.ape.pad.head))
. = ALIGN(SupportsWindows() || SupportsMetal() ? PAGESIZE : 16);
HIDDEN(_ehead = .);
. = ALIGN(. != 0 ? (SupportsWindows() || SupportsMetal() ? CONSTANT(COMMONPAGESIZE) : 16) : 0);
_ehead = .;
} :Head
/*BEGIN: nt addressability guarantee */
@@ -248,12 +276,10 @@ SECTIONS {
/* Code that needs to be addressable in Real Mode */
*(.text.real)
KEEP(*(SORT_BY_NAME(.sort.text.real.*)))
/* Code we want earlier in the binary w/o modifications */
KEEP(*(.ape.loader))
HIDDEN(_ereal = .);
_ereal = .;
/*END: realmode addressability guarantee */
/*BEGIN: morphable code */
. += 1;
. += CODE_GRANULE;
/* Normal Code */
*(.start)
@@ -283,7 +309,6 @@ SECTIONS {
KEEP(*(.keep.text))
*(.text .stub .text.*)
KEEP(*(SORT_BY_NAME(.sort.text.*)))
*(.subrs)
KEEP(*(.ape.pad.test));
*(.test.unlikely)
@@ -291,17 +316,20 @@ SECTIONS {
/* Privileged code invulnerable to magic */
KEEP(*(.ape.pad.privileged));
. += . > 0 ? 1 : 0;
. = ALIGN(__privileged_end > __privileged_start ? CONSTANT(COMMONPAGESIZE) : 0);
/*END: morphable code */
HIDDEN(__privileged_start = .);
. += . > 0 ? 1 : 0;
__privileged_start = .;
*(.privileged)
HIDDEN(__privileged_end = .);
. += . > 0 ? 1 : 0;
__privileged_end = .;
KEEP(*(.ape.pad.text))
. = ALIGN(. != 0 ? CONSTANT(COMMONPAGESIZE) : 0);
/*END: Read Only Data (only needed for initialization) */
} :Cod
/*BEGIN: Read Only Data */
KEEP(*(.ape.pad.rodata));
.rodata . : {
KEEP(*(.rodata.pytab.0));
KEEP(*(.rodata.pytab.1));
KEEP(*(.rodata.pytab.2));
@@ -310,10 +338,6 @@ SECTIONS {
*(.ubsan.types)
*(.ubsan.data)
/* Unit Test & Fixture Registry */
/*BEGIN: Read only data that needn't be mapped after initialization */
/* Legal Notices */
#if !defined(IM_FEELING_NAUGHTY) || defined(EMBED_NOTICES)
KEEP(*(.commentprologue))
@@ -321,37 +345,23 @@ SECTIONS {
KEEP(*(.commentepilogue))
#endif
/*BEGIN: read-only data that's only needed for initialization */
#if SupportsWindows()
/* Windows DLL Import Directory */
KEEP(*(.idata.ro));
KEEP(*(SORT_BY_NAME(.idata.ro.*)))
. = ALIGN(__SIZEOF_POINTER__);
PROVIDE_HIDDEN(__init_array_start = .);
KEEP(*(SORT_BY_INIT_PRIORITY(.init_array.*)
SORT_BY_INIT_PRIORITY(.ctors.*)))
KEEP(*(.ctors))
KEEP(*(.init_array))
KEEP(*(.preinit_array))
PROVIDE_HIDDEN(__init_array_end = .);
. = ALIGN(__SIZEOF_POINTER__);
PROVIDE_HIDDEN(__fini_array_start = .);
KEEP(*(SORT_BY_INIT_PRIORITY(.fini_array.*)
SORT_BY_INIT_PRIORITY(.dtors.*)))
KEEP(*(.dtors))
PROVIDE_HIDDEN(__fini_array_end = .);
#endif
/* Encoded Data Structures w/ Linear Initialization Order */
KEEP(*(.initroprologue))
KEEP(*(SORT_BY_NAME(.initro.*)))
KEEP(*(.initroepilogue))
KEEP(*(SORT_BY_NAME(.sort.rodata.*)))
. = ALIGN(. != 0 ? CONSTANT(COMMONPAGESIZE) : 0); /* don't delete this line :o */
/*END: read-only data that's only needed for initialization */
KEEP(*(.ape.pad.text))
. = ALIGN(PAGESIZE);
HIDDEN(_etext = .);
PROVIDE_HIDDEN(etext = .);
/*END: Read Only Data (only needed for initialization) */
} :Rom
/* initialization image for thread-local storage, this is copied */
@@ -361,29 +371,38 @@ SECTIONS {
*(SORT_BY_ALIGNMENT(.tdata))
*(SORT_BY_ALIGNMENT(.tdata.*))
_tdata_end = .;
. = ALIGN(PAGESIZE);
KEEP(*(.ape.pad.rodata))
. = ALIGN(. != 0 ? CONSTANT(COMMONPAGESIZE) : 0);
_etext = .;
PROVIDE(etext = .);
} :Tls :Rom
/*END: Read Only Data */
. = ALIGN(CONSTANT(COMMONPAGESIZE));
/* this only tells the linker about the layout of uninitialized */
/* TLS data, and does not advance the linker's location counter */
.tbss . : {
.tbss : {
_tbss_start = .;
*(SORT_BY_ALIGNMENT(.tbss))
*(SORT_BY_ALIGNMENT(.tbss.*))
. = ALIGN(TLS_ALIGNMENT);
KEEP(*(.fstls))
/* the %fs register is based on this location */
_tbss_end = .;
} :Tls
.data ALIGN(PAGESIZE) : {
.data . : {
/*BEGIN: Read/Write Data */
#if SupportsWindows()
KEEP(*(SORT_BY_NAME(.piro.data.sort.iat.*)))
#endif
/*BEGIN: NT FORK COPYING */
KEEP(*(.dataprologue))
*(.data .data.*)
*(.PyRuntime) /* for python */
*(.subrs) /* for emacs */
KEEP(*(SORT_BY_NAME(.sort.data.*)))
. += . > 0 ? 1 : 0;
. += . > 0 ? CODE_GRANULE : 0;
KEEP(*(.gotprologue))
*(.got)
@@ -393,34 +412,51 @@ SECTIONS {
*(.got.plt)
KEEP(*(.gotpltepilogue))
. = ALIGN(. != 0 ? __SIZEOF_POINTER__ : 0);
__init_array_start = .;
KEEP(*(SORT_BY_INIT_PRIORITY(.init_array.*)
SORT_BY_INIT_PRIORITY(.ctors.*)))
KEEP(*(.ctors))
KEEP(*(.init_array))
KEEP(*(.preinit_array))
__init_array_end = .;
. = ALIGN(. != 0 ? __SIZEOF_POINTER__ : 0);
__fini_array_start = .;
KEEP(*(SORT_BY_INIT_PRIORITY(.fini_array.*)
SORT_BY_INIT_PRIORITY(.dtors.*)))
KEEP(*(.fini_array))
KEEP(*(.dtors))
__fini_array_end = .;
/*BEGIN: Post-Initialization Read-Only */
. = ALIGN(__SIZEOF_POINTER__);
. = ALIGN(. != 0 ? __SIZEOF_POINTER__ : 0);
KEEP(*(SORT_BY_NAME(.piro.relo.sort.*)))
PROVIDE_HIDDEN(__relo_end = .);
. = ALIGN(__SIZEOF_POINTER__);
. = ALIGN(. != 0 ? __SIZEOF_POINTER__ : 0);
KEEP(*(SORT_BY_NAME(.piro.data.sort.*)))
KEEP(*(.piro.pad.data))
KEEP(*(.dataepilogue))
. = ALIGN(. != 0 ? CONSTANT(COMMONPAGESIZE) : 0);
/*END: NT FORK COPYING */
HIDDEN(_edata = .);
PROVIDE_HIDDEN(edata = .);
KEEP(*(SORT_BY_NAME(.zip.*)))
HIDDEN(_ezip = .);
. = ALIGN(PAGESIZE);
_edata = .;
PROVIDE(edata = .);
_ezip = .; /* <-- very deprecated */
} :Ram
. = ALIGN(CONSTANT(COMMONPAGESIZE));
/*END: file content that's loaded by o/s */
/*END: file content */
/*BEGIN: bss memory that's addressable */
.bss ALIGN(64) : {
.bss : {
/*BEGIN: NT FORK COPYING */
KEEP(*(.bssprologue))
KEEP(*(SORT_BY_NAME(.piro.bss.init.*)))
*(.piro.bss)
KEEP(*(SORT_BY_NAME(.piro.bss.sort.*)))
HIDDEN(__piro_end = .);
. += . > 0 ? 1 : 0;
__piro_end = .;
. += . > 0 ? CODE_GRANULE : 0;
/*END: Post-Initialization Read-Only */
/* Statically Allocated Empty Space */
@@ -431,11 +467,15 @@ SECTIONS {
KEEP(*(SORT_BY_NAME(.sort.bss.*)))
KEEP(*(.bssepilogue))
. = ALIGN(. != 0 ? CONSTANT(COMMONPAGESIZE) : 0);
/*END: NT FORK COPYING */
. = ALIGN(PAGESIZE);
HIDDEN(_end = .);
PROVIDE_HIDDEN(end = .);
} :Bss
} :Ram
. = ALIGN(CONSTANT(COMMONPAGESIZE));
_end = .;
PROVIDE(end = .);
/*END: nt addressability guarantee */
/*END: bsd addressability guarantee */
@@ -478,14 +518,32 @@ SECTIONS {
.gnu.attributes 0 : { KEEP(*(.gnu.attributes)) }
.GCC.command.line 0 : { *(.GCC.command.line) }
.zip 0 : {
KEEP(*(.zip.file))
__zip_cdir_start = .;
KEEP(*(.zip.cdir))
__zip_cdir_size = . - __zip_cdir_start;
KEEP(*(.zip.eocd))
}
/DISCARD/ : {
#if !SupportsWindows()
*(.idata.ro);
*(.idata.ro.*)
*(.piro.data.sort.iat.*)
#endif
*(__patchable_function_entries)
*(__mcount_loc)
*(.discard)
*(.yoink)
*(.*)
}
}
ape_pe_filealignment = 512;
ape_pe_sectionalignment = 4096;
ape_pe_sizeofheaders = SIZEOF(.head);
ape_pe_sizeofimage = ROUNDUP(_end - __executable_start, ape_pe_sectionalignment);
PFSTUB8(ape_elf_entry, _start);
PFSTUB8(ape_elf_phoff, RVA(ape_phdrs));
PFSTUB8(ape_elf_shoff, 0);
@@ -493,70 +551,58 @@ PFSTUB4(ape_elf_phnum, (ape_phdrs_end - ape_phdrs) / 56);
PFSTUB4(ape_elf_shnum, 0);
PFSTUB4(ape_elf_shstrndx, 0);
HIDDEN(_tls_size = _tbss_end - _tdata_start);
HIDDEN(_tdata_size = _tdata_end - _tdata_start);
HIDDEN(_tbss_size = _tbss_end - _tbss_start);
HIDDEN(_tbss_offset = _tbss_start - _tdata_start);
HIDDEN(_tls_content = (_tdata_end - _tdata_start) + (_tbss_end - _tbss_start));
_tls_size = _tbss_end - _tdata_start;
_tdata_size = _tdata_end - _tdata_start;
_tbss_size = _tbss_end - _tbss_start;
_tbss_offset = _tbss_start - _tdata_start;
_tls_content = (_tdata_end - _tdata_start) + (_tbss_end - _tbss_start);
_tls_align = 1;
HIDDEN(__privileged_addr = ROUNDDOWN(__privileged_start, PAGESIZE));
HIDDEN(__privileged_size = (ROUNDUP(__privileged_end, PAGESIZE) -
ROUNDDOWN(__privileged_start, PAGESIZE)));
ape_cod_offset = 0;
ape_cod_vaddr = ADDR(.head);
ape_cod_paddr = LOADADDR(.head);
ape_cod_filesz = ADDR(.rodata) - ADDR(.head);
ape_cod_memsz = ape_cod_filesz;
ape_cod_align = CONSTANT(COMMONPAGESIZE);
ape_cod_rva = RVA(ape_cod_vaddr);
HIDDEN(ape_rom_offset = 0);
HIDDEN(ape_rom_vaddr = ADDR(.head));
HIDDEN(ape_rom_paddr = LOADADDR(.head));
HIDDEN(ape_rom_filesz = LOADADDR(.data) - ape_rom_paddr);
HIDDEN(ape_rom_memsz = ADDR(.data) - ADDR(.head));
HIDDEN(ape_rom_align = PAGESIZE);
HIDDEN(ape_rom_rva = RVA(ape_rom_vaddr));
ape_rom_vaddr = ADDR(.rodata);
ape_rom_offset = ape_rom_vaddr - __executable_start;
ape_rom_paddr = LOADADDR(.rodata);
ape_rom_filesz = ADDR(.tbss) - ADDR(.rodata);
ape_rom_memsz = ape_rom_filesz;
ape_rom_align = CONSTANT(COMMONPAGESIZE);
ape_rom_rva = RVA(ape_rom_vaddr);
HIDDEN(ape_ram_offset = ape_rom_offset + ape_rom_filesz);
HIDDEN(ape_ram_vaddr = ADDR(.data));
HIDDEN(ape_ram_paddr = LOADADDR(.data));
HIDDEN(ape_ram_filesz = SIZEOF(.data));
HIDDEN(ape_ram_memsz = ADDR(.bss) + SIZEOF(.bss) - ape_ram_vaddr);
HIDDEN(ape_ram_align = PAGESIZE);
HIDDEN(ape_ram_rva = RVA(ape_ram_vaddr));
ape_ram_vaddr = ADDR(.data);
ape_ram_offset = ape_ram_vaddr - __executable_start;
ape_ram_paddr = LOADADDR(.data);
ape_ram_filesz = ADDR(.bss) - ADDR(.data);
ape_ram_memsz = _end - ADDR(.data);
ape_ram_align = CONSTANT(COMMONPAGESIZE);
ape_ram_rva = RVA(ape_ram_vaddr);
HIDDEN(ape_stack_pf = DEFINED(ape_stack_pf) ? ape_stack_pf : PF_R | PF_W);
HIDDEN(ape_stack_prot = _PF2PROT(ape_stack_pf));
HIDDEN(ape_stack_offset = ape_ram_offset + ape_ram_filesz);
HIDDEN(ape_stack_vaddr = DEFINED(ape_stack_vaddr) ? ape_stack_vaddr : 0x700000000000);
HIDDEN(ape_stack_paddr = ape_ram_paddr + ape_ram_filesz);
HIDDEN(ape_stack_filesz = 0);
HIDDEN(ape_stack_memsz = DEFINED(ape_stack_memsz) ? ape_stack_memsz : STACKSIZE);
HIDDEN(ape_stack_align = 16);
ape_stack_pf = DEFINED(ape_stack_pf) ? ape_stack_pf : PF_R | PF_W;
ape_stack_prot = _PF2PROT(ape_stack_pf);
ape_stack_offset = 0;
ape_stack_vaddr = DEFINED(ape_stack_vaddr) ? ape_stack_vaddr : 0x700000000000;
ape_stack_paddr = ape_ram_paddr + ape_ram_filesz;
ape_stack_filesz = 0;
ape_stack_memsz = DEFINED(ape_stack_memsz) ? ape_stack_memsz : 8 * 1024 * 1024;
ape_stack_align = DEFINED(ape_stack_align) ? ape_stack_align : 16;
ape_stack_round = -ape_stack_align;
HIDDEN(ape_note_offset = ape_rom_offset + (ape_note - ape_rom_vaddr));
HIDDEN(ape_note_vaddr = ape_note);
HIDDEN(ape_note_paddr = ape_rom_paddr + ape_note_offset);
HIDDEN(ape_note_filesz = ape_note_end - ape_note);
HIDDEN(ape_note_memsz = ape_note_filesz);
HIDDEN(ape_note_align = __SIZEOF_POINTER__);
ape_note_offset = ape_cod_offset + (ape_note - ape_cod_vaddr);
ape_note_filesz = ape_note_end - ape_note;
ape_note_memsz = ape_note_filesz;
HIDDEN(ape_text_offset = ape_rom_offset + LOADADDR(.text) - ape_rom_paddr);
HIDDEN(ape_text_paddr = LOADADDR(.text));
HIDDEN(ape_text_vaddr = ADDR(.text));
HIDDEN(ape_text_filesz = SIZEOF(.text) + SIZEOF(.tdata));
HIDDEN(ape_text_memsz = SIZEOF(.text) + SIZEOF(.tdata));
HIDDEN(ape_text_align = PAGESIZE);
HIDDEN(ape_text_rva = RVA(ape_text_vaddr));
HIDDEN(ape_data_offset = ape_ram_offset + LOADADDR(.data) - ape_ram_paddr);
HIDDEN(ape_data_paddr = LOADADDR(.data));
HIDDEN(ape_data_vaddr = ADDR(.data));
HIDDEN(ape_data_filesz = SIZEOF(.data));
HIDDEN(ape_data_memsz = SIZEOF(.data));
HIDDEN(ape_data_align = PAGESIZE);
HIDDEN(ape_data_rva = RVA(ape_data_vaddr));
HIDDEN(ape_bss_offset = ape_ram_offset + LOADADDR(.bss) - ape_ram_paddr);
HIDDEN(ape_bss_paddr = LOADADDR(.bss));
HIDDEN(ape_bss_vaddr = ADDR(.bss));
HIDDEN(ape_bss_filesz = 0);
HIDDEN(ape_bss_memsz = SIZEOF(.bss));
HIDDEN(ape_bss_align = PAGESIZE);
ape_text_vaddr = ADDR(.text);
ape_text_offset = ape_text_vaddr - __executable_start;
ape_text_paddr = LOADADDR(.text);
ape_text_filesz = ADDR(.rodata) - ADDR(.text);
ape_text_memsz = ape_text_filesz;
ape_text_align = CONSTANT(COMMONPAGESIZE);
ape_text_rva = RVA(ape_text_vaddr);
/* we roundup here because xnu wants the file load segments page-aligned */
/* but we don't want to add the nop padding to the ape program, so we'll */
@@ -564,226 +610,42 @@ HIDDEN(ape_bss_align = PAGESIZE);
SHSTUB2(ape_loader_dd_skip, DEFINED(ape_loader) ? RVA(ape_loader) / 64 : 0);
SHSTUB2(ape_loader_dd_count,
DEFINED(ape_loader_end)
? ROUNDUP(ape_loader_end - ape_loader, PAGESIZE) / 64
? ROUNDUP(ape_loader_end - ape_loader, CONSTANT(COMMONPAGESIZE)) / 64
: 0);
HIDDEN(blink_aarch64_b0 = RVA(blink_aarch64) % 10 + 48);
HIDDEN(blink_aarch64_b1 = RVA(blink_aarch64) < 10 ? 32 : RVA(blink_aarch64) / 10 % 10 + 48);
HIDDEN(blink_aarch64_b2 = RVA(blink_aarch64) < 100 ? 32 : RVA(blink_aarch64) / 100 % 10 + 48);
HIDDEN(blink_aarch64_b3 = RVA(blink_aarch64) < 1000 ? 32 : RVA(blink_aarch64) / 1000 % 10 + 48);
HIDDEN(blink_aarch64_b4 = RVA(blink_aarch64) < 10000 ? 32 : RVA(blink_aarch64) / 10000 % 10 + 48);
HIDDEN(blink_aarch64_b5 = RVA(blink_aarch64) < 100000 ? 32 : RVA(blink_aarch64) / 100000 % 10 + 48);
HIDDEN(blink_aarch64_b6 = RVA(blink_aarch64) < 1000000 ? 32 : RVA(blink_aarch64) / 1000000 % 10 + 48);
HIDDEN(blink_aarch64_b7 = RVA(blink_aarch64) < 10000000 ? 32 : RVA(blink_aarch64) / 10000000 % 10 + 48);
HIDDEN(blink_aarch64_b8 = RVA(blink_aarch64) < 100000000 ? 32 : RVA(blink_aarch64) / 100000000 % 10 + 48);
HIDDEN(blink_aarch64_b9 = RVA(blink_aarch64) < 1000000000 ? 32 : RVA(blink_aarch64) / 1000000000 % 10 + 48);
#if defined(APE_IS_SHELL_SCRIPT) && !IsTiny()
HIDDEN(blink_aarch64_size_b0 = blink_aarch64_size % 10 + 48);
HIDDEN(blink_aarch64_size_b1 = blink_aarch64_size < 10 ? 32 : blink_aarch64_size / 10 % 10 + 48);
HIDDEN(blink_aarch64_size_b2 = blink_aarch64_size < 100 ? 32 : blink_aarch64_size / 100 % 10 + 48);
HIDDEN(blink_aarch64_size_b3 = blink_aarch64_size < 1000 ? 32 : blink_aarch64_size / 1000 % 10 + 48);
HIDDEN(blink_aarch64_size_b4 = blink_aarch64_size < 10000 ? 32 : blink_aarch64_size / 10000 % 10 + 48);
HIDDEN(blink_aarch64_size_b5 = blink_aarch64_size < 100000 ? 32 : blink_aarch64_size / 100000 % 10 + 48);
HIDDEN(blink_aarch64_size_b6 = blink_aarch64_size < 1000000 ? 32 : blink_aarch64_size / 1000000 % 10 + 48);
HIDDEN(blink_aarch64_size_b7 = blink_aarch64_size < 10000000 ? 32 : blink_aarch64_size / 10000000 % 10 + 48);
HIDDEN(blink_aarch64_size_b8 = blink_aarch64_size < 100000000 ? 32 : blink_aarch64_size / 100000000 % 10 + 48);
HIDDEN(blink_aarch64_size_b9 = blink_aarch64_size < 1000000000 ? 32 : blink_aarch64_size / 1000000000 % 10 + 48);
#define IDENTITY(X) X
HIDDEN(blink_darwin_arm64_b0 = RVA(blink_darwin_arm64) % 10 + 48);
HIDDEN(blink_darwin_arm64_b1 = RVA(blink_darwin_arm64) < 10 ? 32 : RVA(blink_darwin_arm64) / 10 % 10 + 48);
HIDDEN(blink_darwin_arm64_b2 = RVA(blink_darwin_arm64) < 100 ? 32 : RVA(blink_darwin_arm64) / 100 % 10 + 48);
HIDDEN(blink_darwin_arm64_b3 = RVA(blink_darwin_arm64) < 1000 ? 32 : RVA(blink_darwin_arm64) / 1000 % 10 + 48);
HIDDEN(blink_darwin_arm64_b4 = RVA(blink_darwin_arm64) < 10000 ? 32 : RVA(blink_darwin_arm64) / 10000 % 10 + 48);
HIDDEN(blink_darwin_arm64_b5 = RVA(blink_darwin_arm64) < 100000 ? 32 : RVA(blink_darwin_arm64) / 100000 % 10 + 48);
HIDDEN(blink_darwin_arm64_b6 = RVA(blink_darwin_arm64) < 1000000 ? 32 : RVA(blink_darwin_arm64) / 1000000 % 10 + 48);
HIDDEN(blink_darwin_arm64_b7 = RVA(blink_darwin_arm64) < 10000000 ? 32 : RVA(blink_darwin_arm64) / 10000000 % 10 + 48);
HIDDEN(blink_darwin_arm64_b8 = RVA(blink_darwin_arm64) < 100000000 ? 32 : RVA(blink_darwin_arm64) / 100000000 % 10 + 48);
HIDDEN(blink_darwin_arm64_b9 = RVA(blink_darwin_arm64) < 1000000000 ? 32 : RVA(blink_darwin_arm64) / 1000000000 % 10 + 48);
#define APE_DECLARE_FIXED_DECIMAL(F, X) \
X##_quad = DEFINED(X) ? ((F(X) < 1000000000 ? 32 : F(X) / 1000000000 % 10 + 48) << 000 | \
(F(X) < 100000000 ? 32 : F(X) / 100000000 % 10 + 48) << 010 | \
(F(X) < 10000000 ? 32 : F(X) / 10000000 % 10 + 48) << 020 | \
(F(X) < 1000000 ? 32 : F(X) / 1000000 % 10 + 48) << 030 | \
(F(X) < 100000 ? 32 : F(X) / 100000 % 10 + 48) << 040 | \
(F(X) < 10000 ? 32 : F(X) / 10000 % 10 + 48) << 050 | \
(F(X) < 1000 ? 32 : F(X) / 1000 % 10 + 48) << 060 | \
(F(X) < 100 ? 32 : F(X) / 100 % 10 + 48) << 070) : 0; \
X##_short = DEFINED(X) ? ((F(X) < 10 ? 32 : F(X) / 10 % 10 + 48) << 000 | \
(F(X) % 10 + 48) << 010) : 0
HIDDEN(blink_darwin_arm64_size_b0 = blink_darwin_arm64_size % 10 + 48);
HIDDEN(blink_darwin_arm64_size_b1 = blink_darwin_arm64_size < 10 ? 32 : blink_darwin_arm64_size / 10 % 10 + 48);
HIDDEN(blink_darwin_arm64_size_b2 = blink_darwin_arm64_size < 100 ? 32 : blink_darwin_arm64_size / 100 % 10 + 48);
HIDDEN(blink_darwin_arm64_size_b3 = blink_darwin_arm64_size < 1000 ? 32 : blink_darwin_arm64_size / 1000 % 10 + 48);
HIDDEN(blink_darwin_arm64_size_b4 = blink_darwin_arm64_size < 10000 ? 32 : blink_darwin_arm64_size / 10000 % 10 + 48);
HIDDEN(blink_darwin_arm64_size_b5 = blink_darwin_arm64_size < 100000 ? 32 : blink_darwin_arm64_size / 100000 % 10 + 48);
HIDDEN(blink_darwin_arm64_size_b6 = blink_darwin_arm64_size < 1000000 ? 32 : blink_darwin_arm64_size / 1000000 % 10 + 48);
HIDDEN(blink_darwin_arm64_size_b7 = blink_darwin_arm64_size < 10000000 ? 32 : blink_darwin_arm64_size / 10000000 % 10 + 48);
HIDDEN(blink_darwin_arm64_size_b8 = blink_darwin_arm64_size < 100000000 ? 32 : blink_darwin_arm64_size / 100000000 % 10 + 48);
HIDDEN(blink_darwin_arm64_size_b9 = blink_darwin_arm64_size < 1000000000 ? 32 : blink_darwin_arm64_size / 1000000000 % 10 + 48);
APE_DECLARE_FIXED_DECIMAL(RVA, blink_linux_aarch64);
APE_DECLARE_FIXED_DECIMAL(IDENTITY, blink_linux_aarch64_size);
APE_DECLARE_FIXED_DECIMAL(RVA, blink_xnu_aarch64);
APE_DECLARE_FIXED_DECIMAL(IDENTITY, blink_xnu_aarch64_size);
HIDDEN(blink_arm_b0 = RVA(blink_arm) % 10 + 48);
HIDDEN(blink_arm_b1 = RVA(blink_arm) < 10 ? 32 : RVA(blink_arm) / 10 % 10 + 48);
HIDDEN(blink_arm_b2 = RVA(blink_arm) < 100 ? 32 : RVA(blink_arm) / 100 % 10 + 48);
HIDDEN(blink_arm_b3 = RVA(blink_arm) < 1000 ? 32 : RVA(blink_arm) / 1000 % 10 + 48);
HIDDEN(blink_arm_b4 = RVA(blink_arm) < 10000 ? 32 : RVA(blink_arm) / 10000 % 10 + 48);
HIDDEN(blink_arm_b5 = RVA(blink_arm) < 100000 ? 32 : RVA(blink_arm) / 100000 % 10 + 48);
HIDDEN(blink_arm_b6 = RVA(blink_arm) < 1000000 ? 32 : RVA(blink_arm) / 1000000 % 10 + 48);
HIDDEN(blink_arm_b7 = RVA(blink_arm) < 10000000 ? 32 : RVA(blink_arm) / 10000000 % 10 + 48);
HIDDEN(blink_arm_b8 = RVA(blink_arm) < 100000000 ? 32 : RVA(blink_arm) / 100000000 % 10 + 48);
HIDDEN(blink_arm_b9 = RVA(blink_arm) < 1000000000 ? 32 : RVA(blink_arm) / 1000000000 % 10 + 48);
HIDDEN(blink_arm_size_b0 = blink_arm_size % 10 + 48);
HIDDEN(blink_arm_size_b1 = blink_arm_size < 10 ? 32 : blink_arm_size / 10 % 10 + 48);
HIDDEN(blink_arm_size_b2 = blink_arm_size < 100 ? 32 : blink_arm_size / 100 % 10 + 48);
HIDDEN(blink_arm_size_b3 = blink_arm_size < 1000 ? 32 : blink_arm_size / 1000 % 10 + 48);
HIDDEN(blink_arm_size_b4 = blink_arm_size < 10000 ? 32 : blink_arm_size / 10000 % 10 + 48);
HIDDEN(blink_arm_size_b5 = blink_arm_size < 100000 ? 32 : blink_arm_size / 100000 % 10 + 48);
HIDDEN(blink_arm_size_b6 = blink_arm_size < 1000000 ? 32 : blink_arm_size / 1000000 % 10 + 48);
HIDDEN(blink_arm_size_b7 = blink_arm_size < 10000000 ? 32 : blink_arm_size / 10000000 % 10 + 48);
HIDDEN(blink_arm_size_b8 = blink_arm_size < 100000000 ? 32 : blink_arm_size / 100000000 % 10 + 48);
HIDDEN(blink_arm_size_b9 = blink_arm_size < 1000000000 ? 32 : blink_arm_size / 1000000000 % 10 + 48);
HIDDEN(blink_i486_b0 = RVA(blink_i486) % 10 + 48);
HIDDEN(blink_i486_b1 = RVA(blink_i486) < 10 ? 32 : RVA(blink_i486) / 10 % 10 + 48);
HIDDEN(blink_i486_b2 = RVA(blink_i486) < 100 ? 32 : RVA(blink_i486) / 100 % 10 + 48);
HIDDEN(blink_i486_b3 = RVA(blink_i486) < 1000 ? 32 : RVA(blink_i486) / 1000 % 10 + 48);
HIDDEN(blink_i486_b4 = RVA(blink_i486) < 10000 ? 32 : RVA(blink_i486) / 10000 % 10 + 48);
HIDDEN(blink_i486_b5 = RVA(blink_i486) < 100000 ? 32 : RVA(blink_i486) / 100000 % 10 + 48);
HIDDEN(blink_i486_b6 = RVA(blink_i486) < 1000000 ? 32 : RVA(blink_i486) / 1000000 % 10 + 48);
HIDDEN(blink_i486_b7 = RVA(blink_i486) < 10000000 ? 32 : RVA(blink_i486) / 10000000 % 10 + 48);
HIDDEN(blink_i486_b8 = RVA(blink_i486) < 100000000 ? 32 : RVA(blink_i486) / 100000000 % 10 + 48);
HIDDEN(blink_i486_b9 = RVA(blink_i486) < 1000000000 ? 32 : RVA(blink_i486) / 1000000000 % 10 + 48);
HIDDEN(blink_i486_size_b0 = blink_i486_size % 10 + 48);
HIDDEN(blink_i486_size_b1 = blink_i486_size < 10 ? 32 : blink_i486_size / 10 % 10 + 48);
HIDDEN(blink_i486_size_b2 = blink_i486_size < 100 ? 32 : blink_i486_size / 100 % 10 + 48);
HIDDEN(blink_i486_size_b3 = blink_i486_size < 1000 ? 32 : blink_i486_size / 1000 % 10 + 48);
HIDDEN(blink_i486_size_b4 = blink_i486_size < 10000 ? 32 : blink_i486_size / 10000 % 10 + 48);
HIDDEN(blink_i486_size_b5 = blink_i486_size < 100000 ? 32 : blink_i486_size / 100000 % 10 + 48);
HIDDEN(blink_i486_size_b6 = blink_i486_size < 1000000 ? 32 : blink_i486_size / 1000000 % 10 + 48);
HIDDEN(blink_i486_size_b7 = blink_i486_size < 10000000 ? 32 : blink_i486_size / 10000000 % 10 + 48);
HIDDEN(blink_i486_size_b8 = blink_i486_size < 100000000 ? 32 : blink_i486_size / 100000000 % 10 + 48);
HIDDEN(blink_i486_size_b9 = blink_i486_size < 1000000000 ? 32 : blink_i486_size / 1000000000 % 10 + 48);
HIDDEN(blink_riscv64_b0 = RVA(blink_riscv64) % 10 + 48);
HIDDEN(blink_riscv64_b1 = RVA(blink_riscv64) < 10 ? 32 : RVA(blink_riscv64) / 10 % 10 + 48);
HIDDEN(blink_riscv64_b2 = RVA(blink_riscv64) < 100 ? 32 : RVA(blink_riscv64) / 100 % 10 + 48);
HIDDEN(blink_riscv64_b3 = RVA(blink_riscv64) < 1000 ? 32 : RVA(blink_riscv64) / 1000 % 10 + 48);
HIDDEN(blink_riscv64_b4 = RVA(blink_riscv64) < 10000 ? 32 : RVA(blink_riscv64) / 10000 % 10 + 48);
HIDDEN(blink_riscv64_b5 = RVA(blink_riscv64) < 100000 ? 32 : RVA(blink_riscv64) / 100000 % 10 + 48);
HIDDEN(blink_riscv64_b6 = RVA(blink_riscv64) < 1000000 ? 32 : RVA(blink_riscv64) / 1000000 % 10 + 48);
HIDDEN(blink_riscv64_b7 = RVA(blink_riscv64) < 10000000 ? 32 : RVA(blink_riscv64) / 10000000 % 10 + 48);
HIDDEN(blink_riscv64_b8 = RVA(blink_riscv64) < 100000000 ? 32 : RVA(blink_riscv64) / 100000000 % 10 + 48);
HIDDEN(blink_riscv64_b9 = RVA(blink_riscv64) < 1000000000 ? 32 : RVA(blink_riscv64) / 1000000000 % 10 + 48);
HIDDEN(blink_riscv64_size_b0 = blink_riscv64_size % 10 + 48);
HIDDEN(blink_riscv64_size_b1 = blink_riscv64_size < 10 ? 32 : blink_riscv64_size / 10 % 10 + 48);
HIDDEN(blink_riscv64_size_b2 = blink_riscv64_size < 100 ? 32 : blink_riscv64_size / 100 % 10 + 48);
HIDDEN(blink_riscv64_size_b3 = blink_riscv64_size < 1000 ? 32 : blink_riscv64_size / 1000 % 10 + 48);
HIDDEN(blink_riscv64_size_b4 = blink_riscv64_size < 10000 ? 32 : blink_riscv64_size / 10000 % 10 + 48);
HIDDEN(blink_riscv64_size_b5 = blink_riscv64_size < 100000 ? 32 : blink_riscv64_size / 100000 % 10 + 48);
HIDDEN(blink_riscv64_size_b6 = blink_riscv64_size < 1000000 ? 32 : blink_riscv64_size / 1000000 % 10 + 48);
HIDDEN(blink_riscv64_size_b7 = blink_riscv64_size < 10000000 ? 32 : blink_riscv64_size / 10000000 % 10 + 48);
HIDDEN(blink_riscv64_size_b8 = blink_riscv64_size < 100000000 ? 32 : blink_riscv64_size / 100000000 % 10 + 48);
HIDDEN(blink_riscv64_size_b9 = blink_riscv64_size < 1000000000 ? 32 : blink_riscv64_size / 1000000000 % 10 + 48);
HIDDEN(blink_mipsel_b0 = RVA(blink_mipsel) % 10 + 48);
HIDDEN(blink_mipsel_b1 = RVA(blink_mipsel) < 10 ? 32 : RVA(blink_mipsel) / 10 % 10 + 48);
HIDDEN(blink_mipsel_b2 = RVA(blink_mipsel) < 100 ? 32 : RVA(blink_mipsel) / 100 % 10 + 48);
HIDDEN(blink_mipsel_b3 = RVA(blink_mipsel) < 1000 ? 32 : RVA(blink_mipsel) / 1000 % 10 + 48);
HIDDEN(blink_mipsel_b4 = RVA(blink_mipsel) < 10000 ? 32 : RVA(blink_mipsel) / 10000 % 10 + 48);
HIDDEN(blink_mipsel_b5 = RVA(blink_mipsel) < 100000 ? 32 : RVA(blink_mipsel) / 100000 % 10 + 48);
HIDDEN(blink_mipsel_b6 = RVA(blink_mipsel) < 1000000 ? 32 : RVA(blink_mipsel) / 1000000 % 10 + 48);
HIDDEN(blink_mipsel_b7 = RVA(blink_mipsel) < 10000000 ? 32 : RVA(blink_mipsel) / 10000000 % 10 + 48);
HIDDEN(blink_mipsel_b8 = RVA(blink_mipsel) < 100000000 ? 32 : RVA(blink_mipsel) / 100000000 % 10 + 48);
HIDDEN(blink_mipsel_b9 = RVA(blink_mipsel) < 1000000000 ? 32 : RVA(blink_mipsel) / 1000000000 % 10 + 48);
HIDDEN(blink_mipsel_size_b0 = blink_mipsel_size % 10 + 48);
HIDDEN(blink_mipsel_size_b1 = blink_mipsel_size < 10 ? 32 : blink_mipsel_size / 10 % 10 + 48);
HIDDEN(blink_mipsel_size_b2 = blink_mipsel_size < 100 ? 32 : blink_mipsel_size / 100 % 10 + 48);
HIDDEN(blink_mipsel_size_b3 = blink_mipsel_size < 1000 ? 32 : blink_mipsel_size / 1000 % 10 + 48);
HIDDEN(blink_mipsel_size_b4 = blink_mipsel_size < 10000 ? 32 : blink_mipsel_size / 10000 % 10 + 48);
HIDDEN(blink_mipsel_size_b5 = blink_mipsel_size < 100000 ? 32 : blink_mipsel_size / 100000 % 10 + 48);
HIDDEN(blink_mipsel_size_b6 = blink_mipsel_size < 1000000 ? 32 : blink_mipsel_size / 1000000 % 10 + 48);
HIDDEN(blink_mipsel_size_b7 = blink_mipsel_size < 10000000 ? 32 : blink_mipsel_size / 10000000 % 10 + 48);
HIDDEN(blink_mipsel_size_b8 = blink_mipsel_size < 100000000 ? 32 : blink_mipsel_size / 100000000 % 10 + 48);
HIDDEN(blink_mipsel_size_b9 = blink_mipsel_size < 1000000000 ? 32 : blink_mipsel_size / 1000000000 % 10 + 48);
HIDDEN(blink_powerpc64le_b0 = RVA(blink_powerpc64le) % 10 + 48);
HIDDEN(blink_powerpc64le_b1 = RVA(blink_powerpc64le) < 10 ? 32 : RVA(blink_powerpc64le) / 10 % 10 + 48);
HIDDEN(blink_powerpc64le_b2 = RVA(blink_powerpc64le) < 100 ? 32 : RVA(blink_powerpc64le) / 100 % 10 + 48);
HIDDEN(blink_powerpc64le_b3 = RVA(blink_powerpc64le) < 1000 ? 32 : RVA(blink_powerpc64le) / 1000 % 10 + 48);
HIDDEN(blink_powerpc64le_b4 = RVA(blink_powerpc64le) < 10000 ? 32 : RVA(blink_powerpc64le) / 10000 % 10 + 48);
HIDDEN(blink_powerpc64le_b5 = RVA(blink_powerpc64le) < 100000 ? 32 : RVA(blink_powerpc64le) / 100000 % 10 + 48);
HIDDEN(blink_powerpc64le_b6 = RVA(blink_powerpc64le) < 1000000 ? 32 : RVA(blink_powerpc64le) / 1000000 % 10 + 48);
HIDDEN(blink_powerpc64le_b7 = RVA(blink_powerpc64le) < 10000000 ? 32 : RVA(blink_powerpc64le) / 10000000 % 10 + 48);
HIDDEN(blink_powerpc64le_b8 = RVA(blink_powerpc64le) < 100000000 ? 32 : RVA(blink_powerpc64le) / 100000000 % 10 + 48);
HIDDEN(blink_powerpc64le_b9 = RVA(blink_powerpc64le) < 1000000000 ? 32 : RVA(blink_powerpc64le) / 1000000000 % 10 + 48);
HIDDEN(blink_powerpc64le_size_b0 = blink_powerpc64le_size % 10 + 48);
HIDDEN(blink_powerpc64le_size_b1 = blink_powerpc64le_size < 10 ? 32 : blink_powerpc64le_size / 10 % 10 + 48);
HIDDEN(blink_powerpc64le_size_b2 = blink_powerpc64le_size < 100 ? 32 : blink_powerpc64le_size / 100 % 10 + 48);
HIDDEN(blink_powerpc64le_size_b3 = blink_powerpc64le_size < 1000 ? 32 : blink_powerpc64le_size / 1000 % 10 + 48);
HIDDEN(blink_powerpc64le_size_b4 = blink_powerpc64le_size < 10000 ? 32 : blink_powerpc64le_size / 10000 % 10 + 48);
HIDDEN(blink_powerpc64le_size_b5 = blink_powerpc64le_size < 100000 ? 32 : blink_powerpc64le_size / 100000 % 10 + 48);
HIDDEN(blink_powerpc64le_size_b6 = blink_powerpc64le_size < 1000000 ? 32 : blink_powerpc64le_size / 1000000 % 10 + 48);
HIDDEN(blink_powerpc64le_size_b7 = blink_powerpc64le_size < 10000000 ? 32 : blink_powerpc64le_size / 10000000 % 10 + 48);
HIDDEN(blink_powerpc64le_size_b8 = blink_powerpc64le_size < 100000000 ? 32 : blink_powerpc64le_size / 100000000 % 10 + 48);
HIDDEN(blink_powerpc64le_size_b9 = blink_powerpc64le_size < 1000000000 ? 32 : blink_powerpc64le_size / 1000000000 % 10 + 48);
HIDDEN(blink_powerpc_b0 = RVA(blink_powerpc) % 10 + 48);
HIDDEN(blink_powerpc_b1 = RVA(blink_powerpc) < 10 ? 32 : RVA(blink_powerpc) / 10 % 10 + 48);
HIDDEN(blink_powerpc_b2 = RVA(blink_powerpc) < 100 ? 32 : RVA(blink_powerpc) / 100 % 10 + 48);
HIDDEN(blink_powerpc_b3 = RVA(blink_powerpc) < 1000 ? 32 : RVA(blink_powerpc) / 1000 % 10 + 48);
HIDDEN(blink_powerpc_b4 = RVA(blink_powerpc) < 10000 ? 32 : RVA(blink_powerpc) / 10000 % 10 + 48);
HIDDEN(blink_powerpc_b5 = RVA(blink_powerpc) < 100000 ? 32 : RVA(blink_powerpc) / 100000 % 10 + 48);
HIDDEN(blink_powerpc_b6 = RVA(blink_powerpc) < 1000000 ? 32 : RVA(blink_powerpc) / 1000000 % 10 + 48);
HIDDEN(blink_powerpc_b7 = RVA(blink_powerpc) < 10000000 ? 32 : RVA(blink_powerpc) / 10000000 % 10 + 48);
HIDDEN(blink_powerpc_b8 = RVA(blink_powerpc) < 100000000 ? 32 : RVA(blink_powerpc) / 100000000 % 10 + 48);
HIDDEN(blink_powerpc_b9 = RVA(blink_powerpc) < 1000000000 ? 32 : RVA(blink_powerpc) / 1000000000 % 10 + 48);
HIDDEN(blink_powerpc_size_b0 = blink_powerpc_size % 10 + 48);
HIDDEN(blink_powerpc_size_b1 = blink_powerpc_size < 10 ? 32 : blink_powerpc_size / 10 % 10 + 48);
HIDDEN(blink_powerpc_size_b2 = blink_powerpc_size < 100 ? 32 : blink_powerpc_size / 100 % 10 + 48);
HIDDEN(blink_powerpc_size_b3 = blink_powerpc_size < 1000 ? 32 : blink_powerpc_size / 1000 % 10 + 48);
HIDDEN(blink_powerpc_size_b4 = blink_powerpc_size < 10000 ? 32 : blink_powerpc_size / 10000 % 10 + 48);
HIDDEN(blink_powerpc_size_b5 = blink_powerpc_size < 100000 ? 32 : blink_powerpc_size / 100000 % 10 + 48);
HIDDEN(blink_powerpc_size_b6 = blink_powerpc_size < 1000000 ? 32 : blink_powerpc_size / 1000000 % 10 + 48);
HIDDEN(blink_powerpc_size_b7 = blink_powerpc_size < 10000000 ? 32 : blink_powerpc_size / 10000000 % 10 + 48);
HIDDEN(blink_powerpc_size_b8 = blink_powerpc_size < 100000000 ? 32 : blink_powerpc_size / 100000000 % 10 + 48);
HIDDEN(blink_powerpc_size_b9 = blink_powerpc_size < 1000000000 ? 32 : blink_powerpc_size / 1000000000 % 10 + 48);
HIDDEN(blink_s390x_b0 = RVA(blink_s390x) % 10 + 48);
HIDDEN(blink_s390x_b1 = RVA(blink_s390x) < 10 ? 32 : RVA(blink_s390x) / 10 % 10 + 48);
HIDDEN(blink_s390x_b2 = RVA(blink_s390x) < 100 ? 32 : RVA(blink_s390x) / 100 % 10 + 48);
HIDDEN(blink_s390x_b3 = RVA(blink_s390x) < 1000 ? 32 : RVA(blink_s390x) / 1000 % 10 + 48);
HIDDEN(blink_s390x_b4 = RVA(blink_s390x) < 10000 ? 32 : RVA(blink_s390x) / 10000 % 10 + 48);
HIDDEN(blink_s390x_b5 = RVA(blink_s390x) < 100000 ? 32 : RVA(blink_s390x) / 100000 % 10 + 48);
HIDDEN(blink_s390x_b6 = RVA(blink_s390x) < 1000000 ? 32 : RVA(blink_s390x) / 1000000 % 10 + 48);
HIDDEN(blink_s390x_b7 = RVA(blink_s390x) < 10000000 ? 32 : RVA(blink_s390x) / 10000000 % 10 + 48);
HIDDEN(blink_s390x_b8 = RVA(blink_s390x) < 100000000 ? 32 : RVA(blink_s390x) / 100000000 % 10 + 48);
HIDDEN(blink_s390x_b9 = RVA(blink_s390x) < 1000000000 ? 32 : RVA(blink_s390x) / 1000000000 % 10 + 48);
HIDDEN(blink_s390x_size_b0 = blink_s390x_size % 10 + 48);
HIDDEN(blink_s390x_size_b1 = blink_s390x_size < 10 ? 32 : blink_s390x_size / 10 % 10 + 48);
HIDDEN(blink_s390x_size_b2 = blink_s390x_size < 100 ? 32 : blink_s390x_size / 100 % 10 + 48);
HIDDEN(blink_s390x_size_b3 = blink_s390x_size < 1000 ? 32 : blink_s390x_size / 1000 % 10 + 48);
HIDDEN(blink_s390x_size_b4 = blink_s390x_size < 10000 ? 32 : blink_s390x_size / 10000 % 10 + 48);
HIDDEN(blink_s390x_size_b5 = blink_s390x_size < 100000 ? 32 : blink_s390x_size / 100000 % 10 + 48);
HIDDEN(blink_s390x_size_b6 = blink_s390x_size < 1000000 ? 32 : blink_s390x_size / 1000000 % 10 + 48);
HIDDEN(blink_s390x_size_b7 = blink_s390x_size < 10000000 ? 32 : blink_s390x_size / 10000000 % 10 + 48);
HIDDEN(blink_s390x_size_b8 = blink_s390x_size < 100000000 ? 32 : blink_s390x_size / 100000000 % 10 + 48);
HIDDEN(blink_s390x_size_b9 = blink_s390x_size < 1000000000 ? 32 : blink_s390x_size / 1000000000 % 10 + 48);
#endif /* APE_IS_SHELL_SCRIPT */
#if SupportsMetal()
HIDDEN(v_ape_realsectors =
MIN(0x70000 - IMAGE_BASE_REAL, ROUNDUP(RVA(_ezip), 512)) / 512);
HIDDEN(v_ape_realbytes = v_ape_realsectors * 512);
HIDDEN(v_ape_realdwords = v_ape_realsectors * (512 / 4));
HIDDEN(v_ape_allsectors = ROUNDUP(RVA(_ezip), 512) / 512);
HIDDEN(v_ape_allbytes = v_ape_allsectors * 512);
HIDDEN(v_ape_highsectors = v_ape_allsectors - v_ape_realsectors);
v_ape_realsectors = MIN(0x70000 - IMAGE_BASE_REAL, ROUNDUP(RVA(_ezip), 512)) / 512;
v_ape_realbytes = v_ape_realsectors * 512;
v_ape_realdwords = v_ape_realsectors * (512 / 4);
v_ape_allsectors = ROUNDUP(RVA(_ezip), 512) / 512;
v_ape_allbytes = v_ape_allsectors * 512;
v_ape_highsectors = MIN(0xffff, v_ape_allsectors - v_ape_realsectors);
TSSDESCSTUB2(_tss, _tss, _tss_end ? _tss_end - _tss - 1 : 0);
#endif
/* ZIP End of Central Directory header */
#define ZIPCONST(NAME, VAL) HIDDEN(NAME = DEFINED(__zip_start) ? VAL : 0);
ZIPCONST(v_zip_cdoffset, __zip_start - IMAGE_BASE_VIRTUAL);
ZIPCONST(v_zip_cdirsize, __zip_end - __zip_start);
ASSERT(v_zip_cdirsize % kZipCdirHdrLinkableSize == 0, "bad zip cdir");
ZIPCONST(v_zip_records, v_zip_cdirsize / kZipCdirHdrLinkableSize);
ZIPCONST(v_zip_commentsize, _ezip - __zip_end - kZipCdirHdrMinSize);
#if SupportsXnu()
/* Generates deterministic ID. */
#define PHI 0x9e3779b9925d4c17
@@ -802,19 +664,8 @@ ZIPCONST(v_zip_commentsize, _ezip - __zip_end - kZipCdirHdrMinSize);
KMH(ape_uuid1, X); \
XORSHIFT(ape_uuid2, X); \
KMH(ape_uuid2, X)
HIDDEN(ape_uuid1 = 88172645463325252);
HIDDEN(ape_uuid2 = 88172645463325252);
CHURN(ape_bss_align);
CHURN(ape_bss_filesz);
CHURN(ape_bss_memsz);
CHURN(ape_bss_offset);
CHURN(ape_bss_paddr);
CHURN(ape_data_filesz);
CHURN(ape_data_memsz);
CHURN(ape_data_offset);
CHURN(ape_data_paddr);
CHURN(ape_data_rva);
CHURN(ape_data_vaddr);
ape_uuid1 = 88172645463325252;
ape_uuid2 = 88172645463325252;
CHURN(ape_elf_entry);
CHURN(ape_elf_phnum);
CHURN(ape_elf_phoff);
@@ -823,13 +674,10 @@ CHURN(ape_elf_shoff);
CHURN(ape_elf_shstrndx);
CHURN(ape_macho_end);
CHURN(ape_note);
CHURN(ape_note_align);
CHURN(ape_note_end);
CHURN(ape_note_filesz);
CHURN(ape_note_memsz);
CHURN(ape_note_offset);
CHURN(ape_note_paddr);
CHURN(ape_note_vaddr);
CHURN(ape_ram_align);
CHURN(ape_ram_filesz);
CHURN(ape_ram_memsz);
@@ -843,7 +691,14 @@ CHURN(ape_rom_memsz);
CHURN(ape_rom_offset);
CHURN(ape_rom_paddr);
CHURN(ape_rom_rva);
CHURN(ape_rom_vaddr);
CHURN(ape_cod_vaddr);
CHURN(ape_cod_align);
CHURN(ape_cod_filesz);
CHURN(ape_cod_memsz);
CHURN(ape_cod_offset);
CHURN(ape_cod_paddr);
CHURN(ape_cod_rva);
CHURN(ape_cod_vaddr);
CHURN(ape_text_align);
CHURN(ape_text_filesz);
CHURN(ape_text_memsz);
@@ -876,22 +731,23 @@ CHURN(WinMain);
#if SupportsWindows() || SupportsMetal()
#define LINK_WINDOWS (SupportsWindows() && !DEFINED(EfiMain))
PFSTUB4(ape_pe_offset, ape_pe - ape_mz);
HIDDEN(ape_pe_optsz = ape_pe_sections - (ape_pe + 24));
HIDDEN(ape_pe_shnum = (ape_pe_sections_end - ape_pe_sections) / 40);
HIDDEN(ape_pe_base = IMAGE_BASE_VIRTUAL);
HIDDEN(ape_idata = LINK_WINDOWS ? RVA(ape_idata_iat) : 0);
HIDDEN(ape_idata_iatsize = LINK_WINDOWS ? ape_idata_iatend - ape_idata_iat : 0);
HIDDEN(ape_idata = LINK_WINDOWS ? RVA(ape_idata_idt) : 0);
HIDDEN(ape_idata_idtsize = LINK_WINDOWS ? ape_idata_idtend - ape_idata_idt : 0);
HIDDEN(v_ntversion = LINK_WINDOWS ? 6 : 1);
HIDDEN(v_ntdllchar = LINK_WINDOWS ? 288 : 0);
HIDDEN(v_ntsubversion = LINK_WINDOWS ? 6 : 5);
HIDDEN(v_ntsubsystem = (LINK_WINDOWS
? (DEFINED(GetMessage)
? kNtImageSubsystemWindowsGui
: kNtImageSubsystemWindowsCui)
: kNtImageSubsystemEfiApplication));
HIDDEN(ape_pe_entry = LINK_WINDOWS ? WinMain : EfiMain);
ape_pe_optsz = ape_pe_sections - (ape_pe + 24);
ASSERT(ape_pe_optsz % 8 == 0, "SizeOfOptionalHeader must be multiple of 8");
ape_pe_shnum = (ape_pe_sections_end - ape_pe_sections) / 40;
ape_pe_base = IMAGE_BASE_VIRTUAL;
ape_idataz = LINK_WINDOWS ? RVA(ape_idata_iat) : 0;
ape_idata_iatsize = LINK_WINDOWS ? ape_idata_iatend - ape_idata_iat : 0;
ape_idata = LINK_WINDOWS ? RVA(ape_idata_idt) : 0;
ape_idata_idtsize = LINK_WINDOWS ? ape_idata_idtend - ape_idata_idt : 0;
v_ntversion = LINK_WINDOWS ? 6 : 1;
v_ntdllchar = LINK_WINDOWS ? 288 : 0;
v_ntsubversion = LINK_WINDOWS ? 6 : 5;
v_ntsubsystem = (LINK_WINDOWS
? (DEFINED(GetMessage)
? kNtImageSubsystemWindowsGui
: kNtImageSubsystemWindowsCui)
: kNtImageSubsystemEfiApplication);
ape_pe_entry = LINK_WINDOWS ? WinMain : EfiMain;
#endif
#if SupportsXnu()
@@ -909,12 +765,6 @@ ASSERT(((DEFINED(__init_rodata_end) ? __init_rodata_end : 0) %
ASSERT((!DEFINED(ape_grub) ? 1 : RVA(ape_grub) < 8192),
"grub stub needs to be in first 8kb of image");
ASSERT(DEFINED(_start) || DEFINED(_start16),
"please link a _start() or _start16() entrypoint");
ASSERT(!DEFINED(_start16) || REAL(_end) < 65536,
"ape won't support non-tiny real mode programs");
ASSERT(IS2POW(ape_stack_memsz),
"ape_stack_memsz must be a two power");
@@ -924,6 +774,8 @@ ASSERT(ape_stack_vaddr % ape_stack_memsz == 0,
ASSERT(ALIGNOF(.tdata) <= TLS_ALIGNMENT && ALIGNOF(.tbss) <= TLS_ALIGNMENT,
"_Thread_local _Alignof can't exceed TLS_ALIGNMENT");
ASSERT(DEFINED(main), "main() function not defined");
/* Let's not be like Knight Capital. */
/* NOCROSSREFS_TO(.test .text) */
+167 -97
View File
@@ -15,23 +15,51 @@
PKGS += APE
APE = o/$(MODE)/ape/ape.o \
o/$(MODE)/ape/ape.lds
APE_FILES := $(wildcard ape/*.*)
APE_HDRS = $(filter %.h,$(APE_FILES))
APE_INCS = $(filter %.inc,$(APE_FILES))
APE_NO_MODIFY_SELF = \
o/$(MODE)/ape/ape.lds \
o/$(MODE)/ape/ape-no-modify-self.o
ifeq ($(ARCH), aarch64)
APE_COPY_SELF = \
o/$(MODE)/ape/ape.lds \
o/$(MODE)/ape/ape-copy-self.o
APE = o/$(MODE)/ape/aarch64.lds
APELINK = \
$(COMPILE) \
-ALINK.ape \
$(LINK) \
$(LINKARGS) \
$(OUTPUT_OPTION)
$(OUTPUT_OPTION) && \
$(COMPILE) \
-AFIXUP.ape \
-wT$@ \
$(FIXUPOBJ) \
$@
APE_SRCS = ape/ape.S ape/start.S ape/launch.S ape/systemcall.S
APE_OBJS = o/$(MODE)/ape/ape.o
APE_NO_MODIFY_SELF = $(APE)
APE_COPY_SELF = $(APE)
.PHONY: o/$(MODE)/ape
o/$(MODE)/ape: $(APE)
o/$(MODE)/ape/aarch64.lds: \
ape/aarch64.lds \
libc/zip.internal.h \
libc/intrin/bits.h \
libc/thread/tls.h \
libc/calls/struct/timespec.h \
libc/macros.internal.h \
libc/str/str.h
APE_LOADER_LDFLAGS = \
-pie \
-static \
-nostdlib \
--no-dynamic-linker \
-z norelro \
-z common-page-size=0x4000 \
-z max-page-size=0x4000
APE_LOADER_FLAGS = \
-DNDEBUG \
@@ -42,78 +70,99 @@ APE_LOADER_FLAGS = \
-Os \
-ffreestanding \
-mgeneral-regs-only \
-mno-red-zone \
-fno-asynchronous-unwind-tables \
-fno-stack-protector \
-fno-ident \
-fno-gnu-unique \
-c \
$(OUTPUT_OPTION) \
$<
o/$(MODE)/ape/ape.elf: o/$(MODE)/ape/ape.elf.dbg
$(COMPILE) -AOBJCOPY -T$@ $(OBJCOPY) -g $< $@
o/$(MODE)/ape/ape.elf.dbg: \
o/$(MODE)/ape/start.o \
o/$(MODE)/ape/loader.o \
o/$(MODE)/ape/launch.o \
o/$(MODE)/ape/systemcall.o
@$(COMPILE) -ALINK.elf $(LD) $(APE_LOADER_LDFLAGS) -o $@ $(patsubst %.lds,-T %.lds,$^)
o/$(MODE)/ape/loader.o: ape/loader.c ape/ape.h
@$(COMPILE) -AOBJECTIFY.c $(CC) -DSUPPORT_VECTOR=1 -g $(APE_LOADER_FLAGS)
o/$(MODE)/ape/start.o: ape/start.S
@$(COMPILE) -AOBJECTIFY.S $(OBJECTIFY.S) $(OUTPUT_OPTION) -c $<
o/$(MODE)/ape/launch.o: ape/launch.S
@$(COMPILE) -AOBJECTIFY.S $(OBJECTIFY.S) $(OUTPUT_OPTION) -c $<
o/$(MODE)/ape/systemcall.o: ape/systemcall.S
@$(COMPILE) -AOBJECTIFY.S $(OBJECTIFY.S) $(OUTPUT_OPTION) -c $<
.PHONY: o/$(MODE)/ape
o/$(MODE)/ape: o/$(MODE)/ape/ape.elf
else
APE = o/$(MODE)/ape/ape.o \
o/$(MODE)/ape/ape.lds
APELINK = \
$(COMPILE) \
-ALINK.ape \
$(LINK) \
$(LINKARGS) \
$(OUTPUT_OPTION) && \
$(COMPILE) \
-AFIXUP.ape \
-wT$@ \
$(FIXUPOBJ) \
$@
APE_NO_MODIFY_SELF = \
o/$(MODE)/ape/ape.lds \
o/$(MODE)/ape/ape-no-modify-self.o
APE_COPY_SELF = \
o/$(MODE)/ape/ape.lds \
o/$(MODE)/ape/ape-copy-self.o
APE_LOADER_FLAGS = \
-DNDEBUG \
-iquote. \
-Wall \
-Wextra \
-Werror \
-pedantic-errors \
-fpie \
-Os \
-ffreestanding \
-mgeneral-regs-only \
-fno-stack-protector \
-fno-ident \
-fno-gnu-unique \
-c \
$(OUTPUT_OPTION) \
$<
APE_FILES := $(wildcard ape/*.*)
APE_HDRS = $(filter %.h,$(APE_FILES))
APE_INCS = $(filter %.inc,$(APE_FILES))
APE_SRCS_C = ape/loader.c
APE_SRCS_S = $(filter %.S,$(APE_FILES))
APE_SRCS = $(APE_SRCS_C) $(APE_SRCS_S)
APE_OBJS = $(APE_SRCS_S:%.S=o/$(MODE)/%.o)
APE_CHECKS = $(APE_HDRS:%=o/%.ok)
o/$(MODE)/ape/ape.lds: \
ape/ape.lds \
ape/macros.internal.h \
ape/relocations.h \
libc/intrin/bits.h \
libc/thread/tls.h \
libc/calls/struct/timespec.h \
libc/thread/thread.h \
libc/dce.h \
libc/elf/def.h \
libc/elf/pf2prot.internal.h \
libc/macros.internal.h \
libc/nt/pedef.internal.h \
libc/str/str.h \
libc/zip.h
o/$(MODE)/ape/public/ape.lds: OVERRIDE_CPPFLAGS += -UCOSMO
o/$(MODE)/ape/public/ape.lds: \
ape/public/ape.lds \
ape/ape.lds \
ape/macros.internal.h \
ape/relocations.h \
libc/intrin/bits.h \
libc/thread/tls.h \
libc/calls/struct/timespec.h \
libc/thread/thread.h \
libc/dce.h \
libc/elf/def.h \
libc/elf/pf2prot.internal.h \
libc/macros.internal.h \
libc/nt/pedef.internal.h \
libc/str/str.h \
libc/zip.h
o/ape/idata.inc: \
ape/idata.internal.h \
ape/relocations.h
o/$(MODE)/ape/ape.o: \
ape/blink-aarch64.gz \
ape/blink-arm.gz \
ape/blink-darwin-arm64.gz \
ape/blink-i486.gz \
ape/blink-mipsel.gz \
ape/blink-powerpc.gz \
ape/blink-powerpc64le.gz \
ape/blink-riscv64.gz \
ape/blink-s390x.gz
o/$(MODE)/ape/ape-no-modify-self.o: \
ape/ape.S \
ape/ape.h \
ape/macros.internal.h \
ape/notice.inc \
ape/relocations.h \
ape/ape.internal.h \
libc/dce.h \
libc/elf/def.h \
libc/thread/tls.h \
libc/intrin/asancodes.h \
libc/macho.internal.h \
libc/macros.internal.h \
@@ -124,17 +173,9 @@ o/$(MODE)/ape/ape-no-modify-self.o: \
libc/runtime/e820.internal.h \
libc/runtime/mman.internal.h \
libc/runtime/pc.internal.h \
libc/runtime/symbolic.h \
libc/sysv/consts/prot.h \
ape/blink-aarch64.gz \
ape/blink-arm.gz \
ape/blink-darwin-arm64.gz \
ape/blink-i486.gz \
ape/blink-mipsel.gz \
ape/blink-powerpc.gz \
ape/blink-powerpc64le.gz \
ape/blink-riscv64.gz \
ape/blink-s390x.gz \
ape/blink-linux-aarch64.gz \
ape/blink-xnu-aarch64.gz \
o/$(MODE)/ape/ape.elf
@$(COMPILE) \
-AOBJECTIFY.S \
@@ -145,11 +186,14 @@ o/$(MODE)/ape/ape-no-modify-self.o: \
o/$(MODE)/ape/ape-copy-self.o: \
ape/ape.S \
ape/ape.h \
ape/macros.internal.h \
ape/notice.inc \
ape/relocations.h \
ape/ape.internal.h \
libc/dce.h \
libc/elf/def.h \
libc/thread/tls.h \
libc/intrin/asancodes.h \
libc/macho.internal.h \
libc/macros.internal.h \
@@ -160,58 +204,84 @@ o/$(MODE)/ape/ape-copy-self.o: \
libc/runtime/e820.internal.h \
libc/runtime/mman.internal.h \
libc/runtime/pc.internal.h \
libc/runtime/symbolic.h \
libc/sysv/consts/prot.h \
ape/blink-aarch64.gz \
ape/blink-arm.gz \
ape/blink-darwin-arm64.gz \
ape/blink-i486.gz \
ape/blink-mipsel.gz \
ape/blink-powerpc.gz \
ape/blink-powerpc64le.gz \
ape/blink-riscv64.gz \
ape/blink-s390x.gz
ape/blink-linux-aarch64.gz \
ape/blink-xnu-aarch64.gz
@$(COMPILE) \
-AOBJECTIFY.S \
$(OBJECTIFY.S) \
$(OUTPUT_OPTION) \
-DAPE_NO_MODIFY_SELF $<
o/$(MODE)/ape/loader.o: ape/loader.c ape/loader.h
@$(COMPILE) -AOBJECTIFY.c $(CC) -DSUPPORT_VECTOR=0b01111001 -g $(APE_LOADER_FLAGS)
o/$(MODE)/ape/loader.o: ape/loader.c
@$(COMPILE) -AOBJECTIFY.c $(CC) -DSUPPORT_VECTOR=121 -g $(APE_LOADER_FLAGS)
o/$(MODE)/ape/loader-gcc.asm: ape/loader.c
@$(COMPILE) -AOBJECTIFY.c $(CC) -DSUPPORT_VECTOR=0b01111001 -S -g0 $(APE_LOADER_FLAGS)
@$(COMPILE) -AOBJECTIFY.c $(CC) -DSUPPORT_VECTOR=121 -S -g0 $(APE_LOADER_FLAGS)
o/$(MODE)/ape/loader-clang.asm: ape/loader.c
@$(COMPILE) -AOBJECTIFY.c $(CLANG) -DSUPPORT_VECTOR=0b01111001 -S -g0 $(APE_LOADER_FLAGS)
@$(COMPILE) -AOBJECTIFY.c $(CLANG) -DSUPPORT_VECTOR=121 -S -g0 $(APE_LOADER_FLAGS)
o/$(MODE)/ape/loader-xnu.o: ape/loader.c ape/loader.h
@$(COMPILE) -AOBJECTIFY.c $(CC) -DSUPPORT_VECTOR=0b00001000 -g $(APE_LOADER_FLAGS)
o/$(MODE)/ape/loader-xnu.o: ape/loader.c
@$(COMPILE) -AOBJECTIFY.c $(CC) -DSUPPORT_VECTOR=8 -g $(APE_LOADER_FLAGS)
o/$(MODE)/ape/loader-xnu-gcc.asm: ape/loader.c
@$(COMPILE) -AOBJECTIFY.c $(CC) -DSUPPORT_VECTOR=0b00001000 -S -g0 $(APE_LOADER_FLAGS)
@$(COMPILE) -AOBJECTIFY.c $(CC) -DSUPPORT_VECTOR=8 -S -g0 $(APE_LOADER_FLAGS)
o/$(MODE)/ape/loader-xnu-clang.asm: ape/loader.c
@$(COMPILE) -AOBJECTIFY.c $(CLANG) -DSUPPORT_VECTOR=0b00001000 -S -g0 $(APE_LOADER_FLAGS)
@$(COMPILE) -AOBJECTIFY.c $(CLANG) -DSUPPORT_VECTOR=8 -S -g0 $(APE_LOADER_FLAGS)
o/$(MODE)/ape/ape.elf: o/$(MODE)/ape/ape.elf.dbg
o/$(MODE)/ape/ape.macho: o/$(MODE)/ape/ape.macho.dbg
@$(COMPILE) -AOBJBINCOPY -w build/bootstrap/objbincopy.com -f -o $@ $<
o/$(MODE)/ape/ape.macho: o/$(MODE)/ape/ape.elf.dbg
@$(COMPILE) -AOBJBINCOPY -w build/bootstrap/objbincopy.com -fm -o $@ $<
APE_LOADER_LDFLAGS = \
-static \
-nostdlib \
--no-dynamic-linker \
-z separate-code \
-z common-page-size=0x1000 \
-z max-page-size=0x10000
o/$(MODE)/ape/ape.elf.dbg: \
o/$(MODE)/ape/loader.o \
o/$(MODE)/ape/loader-elf.o \
ape/loader.lds
@$(ELFLINK) -z max-page-size=0x10
o/$(MODE)/ape/ape.macho.dbg: \
o/$(MODE)/ape/loader-xnu.o \
o/$(MODE)/ape/loader-macho.o \
ape/loader-macho.lds
@$(ELFLINK) -z max-page-size=0x10
o/$(MODE)/ape/start.o \
o/$(MODE)/ape/loader.o \
o/$(MODE)/ape/launch.o \
o/$(MODE)/ape/systemcall.o \
ape/loader.lds
@$(COMPILE) -ALINK.elf $(LD) $(APE_LOADER_LDFLAGS) -o $@ $(patsubst %.lds,-T %.lds,$^)
.PHONY: o/$(MODE)/ape
o/$(MODE)/ape: $(APE_CHECKS) \
o/$(MODE)/ape/ape.o \
o/$(MODE)/ape/ape.lds \
o/$(MODE)/ape/public/ape.lds \
o/$(MODE)/ape/ape.elf \
o/$(MODE)/ape/ape.macho \
o/$(MODE)/ape/ape-copy-self.o \
o/$(MODE)/ape/ape-no-modify-self.o
endif
# these assembly files are safe to build on aarch64
o/$(MODE)/ape/ape.o: ape/ape.S
@$(COMPILE) -AOBJECTIFY.S $(OBJECTIFY.S) $(OUTPUT_OPTION) -c $<
o/$(MODE)/ape/ape.o: \
ape/blink-linux-aarch64.gz \
ape/blink-xnu-aarch64.gz
o/$(MODE)/ape/ape.lds: \
ape/ape.lds \
ape/macros.internal.h \
ape/relocations.h \
ape/ape.internal.h \
libc/intrin/bits.h \
libc/thread/tls.h \
libc/calls/struct/timespec.h \
libc/thread/thread.h \
libc/dce.h \
libc/elf/def.h \
libc/elf/pf2prot.internal.h \
libc/macros.internal.h \
libc/nt/pedef.internal.h \
libc/str/str.h \
libc/zip.internal.h
-128
View File
@@ -1,128 +0,0 @@
#!/bin/sh
if [ "$(id -u)" -eq 0 ]; then
SUDO=
else
SUDO=sudo
fi
echo "Actually Portable Executable (APE) Installer" >&2
echo "Author: Justine Tunney <[email protected]>" >&2
################################################################################
# INSTALL APE LOADER SYSTEMWIDE
if [ -f o/depend ]; then
# mkdeps.com build was successfully run so assume we can build
echo >&2
echo "recompiling ape loader" >&2
echo "running: make -j8 o//ape" >&2
make -j8 o//ape || exit
echo "done" >&2
elif [ -d build/bootstrap ]; then
# if make isn't being used then it's unlikely the user changed the sources
# in that case the prebuilt binaries should be completely up-to-date
echo "using prebuilt ape loader from cosmo repo" >&2
mkdir -p o//ape || exit
cp -af build/bootstrap/ape.elf o//ape/ape.elf || exit
cp -af build/bootstrap/ape.macho o//ape/ape.macho || exit
else
echo "no cosmopolitan libc repository here" >&2
echo "fetching ape loader from justine.lol" >&2
mkdir -p o//ape || exit
if command -v wget >/dev/null 2>&1; then
wget -qO o//ape/ape.elf https://justine.lol/ape.elf || exit
wget -qO o//ape/ape.macho https://justine.lol/ape.macho || exit
else
curl -Rso o//ape/ape.elf https://justine.lol/ape.elf || exit
curl -Rso o//ape/ape.macho https://justine.lol/ape.macho || exit
fi
chmod +x o//ape/ape.elf || exit
chmod +x o//ape/ape.macho || exit
fi
if [ "$(uname -s)" = "Darwin" ]; then
if ! [ /usr/bin/ape -nt o//ape/ape.macho ]; then
echo >&2
echo "installing o//ape/ape.elf to /usr/bin/ape" >&2
echo "$SUDO mv -f o//ape/ape.elf /usr/bin/ape" >&2
$SUDO cp -f o//ape/ape.macho /usr/bin/ape || exit
echo "done" >&2
fi
else
if ! [ /usr/bin/ape -nt o//ape/ape.elf ]; then
echo >&2
echo "installing o//ape/ape.elf to /usr/bin/ape" >&2
echo "$SUDO mv -f o//ape/ape.elf /usr/bin/ape" >&2
$SUDO cp -f o//ape/ape.elf /usr/bin/ape || exit
echo "done" >&2
fi
fi
################################################################################
# REGISTER APE LOADER WITH BINFMT_MISC TOO (LINUX-ONLY)
if [ x"$(uname -s)" = xLinux ]; then
if [ -e /proc/sys/fs/binfmt_misc/APE ]; then
echo >&2
echo it looks like APE is already registered with binfmt_misc >&2
echo To reinstall please run ape/apeuninstall.sh first >&2
echo please check that it is mapped to ape not /bin/sh >&2
echo cat /proc/sys/fs/binfmt_misc/APE >&2
cat /proc/sys/fs/binfmt_misc/APE >&2
exit
fi
if ! [ -e /proc/sys/fs/binfmt_misc ]; then
echo >&2
echo loading binfmt_misc into your kernel >&2
echo you may need to edit configs to persist across reboot >&2
echo $SUDO modprobe binfmt_misc >&2
$SUDO modprobe binfmt_misc || exit
echo done >&2
fi
if ! [ -e /proc/sys/fs/binfmt_misc/register ]; then
echo >&2
echo mounting binfmt_misc into your kernel >&2
echo you may need to edit configs to persist across reboot >&2
echo $SUDO mount -t binfmt_misc none /proc/sys/fs/binfmt_misc >&2
$SUDO mount -t binfmt_misc none /proc/sys/fs/binfmt_misc || exit
echo done >&2
fi
echo >&2
echo registering APE with binfmt_misc >&2
echo you may need to edit configs to persist across reboot >&2
echo '$SUDO sh -c "echo '"'"':APE:M::MZqFpD::/usr/bin/ape:'"'"' >/proc/sys/fs/binfmt_misc/register"' >&2
$SUDO sh -c "echo ':APE:M::MZqFpD::/usr/bin/ape:' >/proc/sys/fs/binfmt_misc/register" || exit
echo '$SUDO sh -c "echo '"'"':APE-jart:M::jartsr::/usr/bin/ape:'"'"' >/proc/sys/fs/binfmt_misc/register"' >&2
$SUDO sh -c "echo ':APE-jart:M::jartsr::/usr/bin/ape:' >/proc/sys/fs/binfmt_misc/register" || exit
echo done >&2
if [ x"$(cat /proc/sys/fs/binfmt_misc/status)" = xdisabled ]; then
echo >&2
echo enabling binfmt_misc >&2
echo you may need to edit configs to persist across reboot >&2
echo $SUDO sh -c 'echo 1 >/proc/sys/fs/binfmt_misc/status' >&2
$SUDO sh -c 'echo 1 >/proc/sys/fs/binfmt_misc/status' || exit
echo done >&2
fi
fi
################################################################################
{
echo
echo "------------------------------------------------------------------"
echo
echo "APE INSTALL COMPLETE"
echo
echo "If you decide to uninstall APE later on"
echo "you may do so using ape/apeuninstall.sh"
echo
echo "Enjoy your APE loader (>'.')>"
echo
} >&2
+1
View File
@@ -0,0 +1 @@
../bin/ape-install
-26
View File
@@ -1,26 +0,0 @@
#!/bin/sh
if [ "$UID" = "0" ]; then
SUDO=
else
SUDO=sudo
fi
{
echo
echo "APE Uninstaller intends to run (in pseudo-shell)"
echo
echo " sudo echo -1 into /proc/sys/fs/binfmt_misc/APE*"
echo " sudo rm -f /usr/bin/ape ~/.ape o/tmp/.ape /tmp/.ape"
echo
echo "You may then use ape/apeinstall.sh to reinstall it"
echo
} >&2
set -ex
for f in /proc/sys/fs/binfmt_misc/APE*; do
if [ -f $f ]; then
$SUDO sh -c "echo -1 >$f" || exit
fi
done
$SUDO rm -f /usr/bin/ape ~/.ape o/tmp/.ape o/tmp/ape /tmp/.ape /tmp/ape || exit
+1
View File
@@ -0,0 +1 @@
../bin/ape-uninstall
Binary file not shown.
BIN
View File
Binary file not shown.
Binary file not shown.
BIN
View File
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
BIN
View File
Binary file not shown.
-10
View File
@@ -1,10 +0,0 @@
#ifndef COSMOPOLITAN_APE_CONFIG_H_
#define COSMOPOLITAN_APE_CONFIG_H_
#if !(__ASSEMBLER__ + __LINKER__ + 0)
#define STATIC_SYMBOL(NAME, VALUE) \
asm(".equ\t" NAME "," VALUE "\n\t" \
".globl\t" NAME)
#endif /* !(__ASSEMBLER__ + __LINKER__ + 0) */
#endif /* COSMOPOLITAN_APE_CONFIG_H_ */
+22 -18
View File
@@ -31,7 +31,8 @@
// @see libc/nt/master.sh
// @see ape/ape.lds
// @see winimp
.macro .imp dll:req fn:req actual:req hint
.macro .imp dll:req fn:req actual:req
#ifdef __x86_64__
.dll "\dll"
.section ".piro.data.sort.iat.2.\dll\().2.\actual","aw",@progbits
.type \fn,@object
@@ -42,25 +43,28 @@
.hidden \fn
.previous
.section ".idata.ro.ilt.\dll\().2.\actual","a",@progbits
".Lidata.ilt.\dll\().\actual":
"idata.ilt.\dll\().\actual":
.quad RVA("\dll\().\actual")
.type ".Lidata.ilt.\dll\().\actual",@object
.size ".Lidata.ilt.\dll\().\actual",.-".Lidata.ilt.\dll\().\actual"
.type "idata.ilt.\dll\().\actual",@object
.size "idata.ilt.\dll\().\actual",.-"idata.ilt.\dll\().\actual"
.previous
.section ".idata.ro.hnt.\dll\().2.\actual","a",@progbits
"\dll\().\actual":
.ifnb \hint # hint i.e. guess function ordinal
.short \hint
.else
.short 0
.endif
.short 0 // hint
.asciz "\actual"
.align 2 # documented requirement
.align 2 // documented requirement
.globl "\dll\().\actual"
.hidden "\dll\().\actual"
.type "\dll\().\actual",@object
.size "\dll\().\actual",.-"\dll\().\actual"
.previous
#else
.section ".data.nt.\actual","aw",@progbits
.globl "\fn"
.balign 8
.weak "\actual"
"\fn": .quad "\actual"
#endif
.endm
// Defines DLL import.
@@ -68,16 +72,16 @@
.macro .dll name:req
.section ".idata.ro.idt.2.\name","aG",@progbits,"\name",comdat
.equ ".Lidata.idt.\name",.
.long RVA("idata.ilt.\name") # ImportLookupTable
.long 0 # TimeDateStamp
.long 0 # ForwarderChain
.long RVA(".Lidata.str.\name") # DllNameRva
.long RVA("idata.iat.\name") # ImportAddressTable
.long RVA("idata.ilt.\name") // ImportLookupTable
.long 0 // TimeDateStamp
.long 0 // ForwarderChain
.long RVA(".Lidata.str.\name") // DllNameRva
.long RVA("idata.iat.\name") // ImportAddressTable
.type ".Lidata.idt.\name",@object
.size ".Lidata.idt.\name",.-".Lidata.idt.\name"
.previous
.section ".idata.ro.ilt.\name\().1","aG",@progbits,"\name",comdat
.align __SIZEOF_POINTER__
.balign __SIZEOF_POINTER__
.type "idata.ilt.\name",@object
"idata.ilt.\name":
.previous/*
@@ -88,12 +92,12 @@
.quad 0
.previous
.section ".idata.ro.hnt.\name\().1","aG",@progbits,"\name",comdat
.align __SIZEOF_POINTER__
.balign __SIZEOF_POINTER__
.type "idata.hnt.\name",@object
.equ "idata.hnt.\name",.
.previous
.section ".piro.data.sort.iat.2.\name\().1","awG",@progbits,"\name",comdat
.align __SIZEOF_POINTER__
.balign __SIZEOF_POINTER__
.type "idata.iat.\name",@object
"idata.iat.\name":
.previous/*
+92
View File
@@ -0,0 +1,92 @@
/*-*- mode:unix-assembly; indent-tabs-mode:t; tab-width:8; coding:utf-8 -*-│
│vi: set et ft=asm ts=8 tw=8 fenc=utf-8 :vi│
╞══════════════════════════════════════════════════════════════════════════════╡
│ Copyright 2023 Justine Alexandra Roberts Tunney │
│ │
│ Permission to use, copy, modify, and/or distribute this software for │
│ any purpose with or without fee is hereby granted, provided that the │
│ above copyright notice and this permission notice appear in all copies. │
│ │
│ THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL │
│ WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED │
│ WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE │
│ AUTHOR BE LIABLE FOR ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL │
│ DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR │
│ PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER │
│ TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR │
│ PERFORMANCE OF THIS SOFTWARE. │
╚─────────────────────────────────────────────────────────────────────────────*/
#include "libc/macros.internal.h"
// Calls _start() function of loaded program.
//
// When the program entrypoint is called, all registers shall be
// cleared, with the exception of (1) %rdi will be equal to %rsp
// on FreeBSD and (2) %cl will contain the detected host OS code
//
// We clear all the general registers we can to have some wiggle
// room, to extend the behavior of this loader in the future. We
// don't need to clear the XMM registers because your APE loader
// should be compiled using gcc/clang's -mgeneral-regs-only flag
//
// @param rdi is passed through as-is
// @param rsi is address of entrypoint (becomes zero)
// @param rdx is stack pointer (becomes zero)
// @param rcx is passed through as-is
// @noreturn
Launch:
#ifdef __aarch64__
mov x16,x1
mov sp,x2
mov x1,0
mov x2,0
mov x3,0
mov x4,0
mov x5,0
mov x6,0
mov x7,0
mov x8,0
mov x9,0
mov x10,0
mov x11,0
mov x12,0
mov x13,0
mov x14,0
mov x15,0
mov x17,0
mov x19,0
mov x20,0
mov x21,0
mov x22,0
mov x23,0
mov x24,0
mov x25,0
mov x26,0
mov x27,0
mov x28,0
mov x29,0
mov x30,0
br x16
#else
xor %r8d,%r8d
xor %r9d,%r9d
xor %r10d,%r10d
xor %r11d,%r11d
xor %r12d,%r12d
xor %r13d,%r13d
xor %r14d,%r14d
xor %r15d,%r15d
mov %rdx,%rsp
xor %edx,%edx
push %rsi
xor %esi,%esi
xor %ebp,%ebp
xor %ebx,%ebx
xor %eax,%eax
ret
#endif
.endfn Launch,globl
-249
View File
@@ -1,249 +0,0 @@
/*-*- mode:unix-assembly; indent-tabs-mode:t; tab-width:8; coding:utf-8 -*-│
│vi: set et ft=asm ts=8 tw=8 fenc=utf-8 :vi│
╞══════════════════════════════════════════════════════════════════════════════╡
│ Copyright 2021 Justine Alexandra Roberts Tunney │
│ │
│ Permission to use, copy, modify, and/or distribute this software for │
│ any purpose with or without fee is hereby granted, provided that the │
│ above copyright notice and this permission notice appear in all copies. │
│ │
│ THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL │
│ WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED │
│ WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE │
│ AUTHOR BE LIABLE FOR ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL │
│ DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR │
│ PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER │
│ TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR │
│ PERFORMANCE OF THIS SOFTWARE. │
╚─────────────────────────────────────────────────────────────────────────────*/
#include "libc/elf/def.h"
#include "libc/sysv/consts/prot.h"
#include "libc/macho.internal.h"
#include "libc/dce.h"
#include "libc/macros.internal.h"
// APE Loader Executable Structure
// Linux, FreeBSD, NetBSD, OpenBSD
.align 8
ehdr: .ascii "\177ELF"
.byte ELFCLASS64
.byte ELFDATA2LSB
.byte 1
.byte ELFOSABI_FREEBSD
.quad 0
.word ET_EXEC # e_type
.word EM_NEXGEN32E # e_machine
.long 1 # e_version
.quad _start # e_entry
.quad phdrs - ehdr # e_phoff
.quad 0 # e_shoff
.long 0 # e_flags
.word 64 # e_ehsize
.word 56 # e_phentsize
.word 4 # e_phnum
.word 0 # e_shentsize
.word 0 # e_shnum
.word 0 # e_shstrndx
.endobj ehdr,globl
.align 8
phdrs: .long PT_LOAD # p_type
.long PF_R|PF_X # p_flags
.quad 0 # p_offset
.quad ehdr # p_vaddr
.quad ehdr # p_paddr
.quad filesz # p_filesz
.quad filesz # p_memsz
.quad PAGESIZE # p_align
.long PT_LOAD # p_type
.long PF_R|PF_W # p_flags
.quad 0 # p_offset
.quad bss # p_vaddr
.quad bss # p_paddr
.quad 0 # p_filesz
.quad bsssize # p_memsz
.quad PAGESIZE # p_align
.long PT_GNU_STACK # p_type
.long PF_R|PF_W # p_flags
.quad 0 # p_offset
.quad 0 # p_vaddr
.quad 0 # p_paddr
.quad 0 # p_filesz
.quad 0 # p_memsz
.quad 16 # p_align
.long PT_NOTE # p_type
.long PF_R # p_flags
.quad note - ehdr # p_offset
.quad note # p_vaddr
.quad note # p_paddr
.quad notesize # p_filesz
.quad notesize # p_memsz
.quad 8 # p_align
.endobj phdrs
note: .long 2f-1f
.long 4f-3f
.long 1
1: .asciz "OpenBSD"
2: .align 4
3: .long 0
4: .long 2f-1f
.long 4f-3f
.long 1
1: .asciz "NetBSD"
2: .align 4
3: .long 901000000
4: .endobj note
notesize = . - note
.align 64,0 # for ape.S dd
.org 0x180 # for ape.S dd
// APE Loader XNU Header
//
// This header is dd'd backwards by the APE shell script when
// running on Mac OS X.
//
// @see ape/ape.S
macho: .long 0xFEEDFACE+1
.long MAC_CPU_NEXGEN32E
.long MAC_CPU_NEXGEN32E_ALL
.long MAC_EXECUTE
.long 5 # number of load commands
.long 60f-10f # size of all load commands
.long MAC_NOUNDEFS # flags
.long 0 # reserved
10: .long MAC_LC_SEGMENT_64
.long 20f-10b # unmaps first page dir
.ascin "__PAGEZERO",16 # consistent with linux
.quad 0,0x200000,0,0 # which forbids mem <2m
.long 0,0,0,0
20: .long MAC_LC_SEGMENT_64
.long 30f-20b
.ascin "__TEXT",16
.quad ehdr # vaddr
.quad 4096 # memsz
.quad 0 # file offset
.quad filesz # file size
.long PROT_EXEC|PROT_READ|PROT_WRITE # maxprot
.long PROT_EXEC|PROT_READ # initprot
.long 1 # segment section count
.long 0 # flags
210: .ascin "__text",16 # section name (.text)
.ascin "__TEXT",16
.quad _start # vaddr
.quad textsz # memsz
.long textoff # offset
.long 6 # align 2**6 = 64
.long 0 # reloc table offset
.long 0 # relocation count
.long MAC_S_ATTR_SOME_INSTRUCTIONS # section type & attributes
.long 0,0,0 # reserved
30: .long MAC_LC_SEGMENT_64
.long 40f-30b
.ascin "__DATA",16
.quad bss # vaddr
.quad bsssize # memsz
.quad 0 # offset
.quad 0 # file size
.long PROT_EXEC|PROT_READ|PROT_WRITE # maxprot
.long PROT_READ|PROT_WRITE # initprot
.long 1 # segment section count
.long 0 # flags
310: .ascin "__bss",16 # section name (.bss)
.ascin "__DATA",16
.quad bss # vaddr
.quad bsssize # memsz
.long 0 # offset
.long 12 # align 2**12 = 4096
.long 0 # reloc table offset
.long 0 # relocation count
.long MAC_S_ZEROFILL # section type & attributes
.long 0,0,0 # reserved
40: .long MAC_LC_UUID
.long 50f-40b
.quad 0x3fb29ee4ac6c87aa # uuid1
.quad 0xdd2c9bb866d9eef8 # uuid2
50: .long MAC_LC_UNIXTHREAD
.long 60f-50b # cmdsize
.long MAC_THREAD_NEXGEN32E # flavaflav
.long (520f-510f)/4 # count
510: .quad 0 # rax
.quad 0 # rbx
.quad 0 # rcx
.quad _HOSTXNU # rdx
.quad 0 # rdi
.quad 0 # rsi
.quad 0 # rbp
.quad 0 # rsp
.quad 0 # r8
.quad 0 # r9
.quad 0 # r10
.quad 0 # r11
.quad 0 # r12
.quad 0 # r13
.quad 0 # r14
.quad 0 # r15
.quad _start # rip
.quad 0 # rflags
.quad 0 # cs
.quad 0 # fs
.quad 0 # gs
520:
60:
.endobj macho
.align 64,0 # for ape.S dd
.org 0x400 # for ape.S dd
// Ape Loader Entrpoint
//
// This is normally called by the operating system. However it may
// be called by the Actually Portable Executables themselves, when
// re-executing a program. Just do this:
//
// memcpy(0x200000, loader)
// lea handoff(%rip),%rcx
// lea argblock(%rip),%rsp
// jmp 0x200400
//
// @see APE_LOADER_ENTRY
// @see ape/loader.h
_start:
// Hack for detecting M1 Rosetta environment.
// https://github.com/jart/cosmopolitan/issues/429#issuecomment-1166704377
cmp $-1,%ebx
jne 0f
cmp $+1,%edx
jne 0f
mov $_HOSTXNU,%dl
xor %ecx,%ecx
0: mov %rsp,%rsi
jmp ApeLoader
.endfn _start,globl
// System Call Entrpoint
//
// This function is used by the APE loader to make system calls.
// We also pass a reference to this function to the APE binary's
// _start() function. It's needed because on OpenBSD, msyscall()
// restricts which pages can issue system calls, and it can only
// be called once. Therefore if we want to be load and re-load a
// binary multiple times without calling the system execve(), we
// need to be able to handover the SYSCALL function. We hardcode
// this to a fixed address, but that shouldn't be used, since we
// would ideally want to move it to a random page in the future.
__syscall_loader:
clc
syscall
jc 1f
ret
1: neg %rax
ret
.endfn __syscall_loader,globl
+56 -99
View File
@@ -21,118 +21,75 @@
#include "libc/dce.h"
#include "libc/macros.internal.h"
// APE Loader Executable Structure for XNU
// Apple Mach-O Executable Headers
// Fixups are applied by objbincopy.com
// There must exist a MAC_LC_SEGMENT_64 for every PT_LOAD
.align 4096
macho: .long 0xFEEDFACE+1
.section .macho,"a",@progbits
.balign 64
.long 0xFEEDFACE+1
.long MAC_CPU_NEXGEN32E
.long MAC_CPU_NEXGEN32E_ALL
.long MAC_EXECUTE
.long 5 # number of load commands
.long 60f-10f # size of all load commands
.long MAC_NOUNDEFS # flags
.long 0 # reserved
.long 5 // number of load commands
.long 60f-10f // size of all load commands
.long MAC_NOUNDEFS|MAC_SPLIT_SEGS // flags
.long 0 // reserved
10: .long MAC_LC_SEGMENT_64
.long 20f-10b # unmaps first page dir
.ascin "__PAGEZERO",16 # consistent with linux
.quad 0,0x200000,0,0 # which forbids mem <2m
.long 20f-10b // unmaps first page dir
.ascin "__PAGEZERO",16 // consistent with linux
.quad 0,0x200000,0,0 // which forbids mem <2m
.long 0,0,0,0
20: .long MAC_LC_SEGMENT_64
.long 30f-20b
.ascin "__TEXT",16
.quad macho # vaddr
.quad 4096 # memsz
.quad 0 # file offset
.quad filesz # file size
.long PROT_EXEC|PROT_READ|PROT_WRITE # maxprot
.long PROT_EXEC|PROT_READ # initprot
.long 1 # segment section count
.long 0 # flags
210: .ascin "__text",16 # section name (.text)
.ascin "__TEXT",16
.quad _start # vaddr
.quad textsz # memsz
.long textoff # offset
.long 6 # align 2**3 = 64
.long 0 # reloc table offset
.long 0 # relocation count
.long MAC_S_ATTR_SOME_INSTRUCTIONS # section type & attributes
.long 0,0,0 # reserved
.quad 0 // vaddr
.quad 0 // memsz
.quad 0 // file offset
.quad 0 // file size
.long 0 // maxprot
.long 0 // initprot
.long 0 // segment section count
.long 0 // flags
30: .long MAC_LC_SEGMENT_64
.long 40f-30b
.ascin "__DATA",16
.quad bss # vaddr
.quad bsssize # memsz
.quad 0 # offset
.quad 0 # file size
.long PROT_EXEC|PROT_READ|PROT_WRITE # maxprot
.long PROT_READ|PROT_WRITE # initprot
.long 1 # segment section count
.long 0 # flags
310: .ascin "__bss",16 # section name (.bss)
.ascin "__DATA",16
.quad bss # vaddr
.quad bsssize # memsz
.long 0 # offset
.long 12 # align 2**12 = 4096
.long 0 # reloc table offset
.long 0 # relocation count
.long MAC_S_ZEROFILL # section type & attributes
.long 0,0,0 # reserved
.ascin "__RODATA",16
.quad 0 // vaddr
.quad 0 // memsz
.quad 0 // file offset
.quad 0 // file size
.long 0 // maxprot
.long 0 // initprot
.long 0 // segment section count
.long 0 // flags
40: .long MAC_LC_UUID
.long 50f-40b
.quad 0x3fb29ee4ac6c87aa # uuid1
.quad 0xdd2c9bb866d9eef8 # uuid2
.quad 0x4527148ba7a513ef // uuid1
.quad 0x56fa865940665e8f // uuid2
50: .long MAC_LC_UNIXTHREAD
.long 60f-50b # cmdsize
.long MAC_THREAD_NEXGEN32E # flavaflav
.long (520f-510f)/4 # count
510: .quad 0 # rax
.quad 0 # rbx
.quad 0 # rcx
.quad _HOSTXNU # rdx
.quad 0 # rdi
.quad 0 # rsi
.quad 0 # rbp
.quad 0 # rsp
.quad 0 # r8
.quad 0 # r9
.quad 0 # r10
.quad 0 # r11
.quad 0 # r12
.quad 0 # r13
.quad 0 # r14
.quad 0 # r15
.quad _start # rip
.quad 0 # rflags
.quad 0 # cs
.quad 0 # fs
.quad 0 # gs
.long 60f-50b // cmdsize
.long MAC_THREAD_NEXGEN32E // flavaflav
.long (520f-510f)/4 // count
510: .quad 0 // rax
.quad 0 // rbx
.quad 0 // rcx
.quad 0 // rdx
.quad 0 // rdi
.quad 0 // rsi
.quad 0 // rbp
.quad 0 // rsp
.quad 0 // r8
.quad 0 // r9
.quad 0 // r10
.quad 0 // r11
.quad 0 // r12
.quad 0 // r13
.quad 0 // r14
.quad 0 // r15
.quad XnuEntrypoint // rip
.quad 0 // rflags
.quad 0 // cs
.quad 0 // fs
.quad 0 // gs
520:
60:
.endobj macho,globl
.align 64
_start:
// Hack for detecting M1 Rosetta environment.
// https://github.com/jart/cosmopolitan/issues/429#issuecomment-1166704377
cmp $-1,%ebx
jne 0f
cmp $+1,%edx
jne 0f
mov $_HOSTXNU,%dl
xor %ecx,%ecx
0: mov %rsp,%rsi
jmp ApeLoader
.endfn _start,globl
__syscall_loader:
clc
syscall
jc 1f
ret
1: neg %rax
ret
.endfn __syscall_loader,globl
-42
View File
@@ -1,42 +0,0 @@
/*-*- mode: ld-script; indent-tabs-mode: nil; tab-width: 2; coding: utf-8 -*-│
│vi: set et sts=2 tw=2 fenc=utf-8 :vi│
╞══════════════════════════════════════════════════════════════════════════════╡
│ Copyright 2021 Justine Alexandra Roberts Tunney │
│ │
│ Permission to use, copy, modify, and/or distribute this software for │
│ any purpose with or without fee is hereby granted, provided that the │
│ above copyright notice and this permission notice appear in all copies. │
│ │
│ THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL │
│ WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED │
│ WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE │
│ AUTHOR BE LIABLE FOR ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL │
│ DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR │
│ PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER │
│ TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR │
│ PERFORMANCE OF THIS SOFTWARE. │
╚─────────────────────────────────────────────────────────────────────────────*/
ENTRY(_start)
SECTIONS {
. = 0x200000;
.text : {
*(.text)
*(.rodata .rodata.*)
. = ALIGN(4096);
}
filesz = . - macho;
textsz = . - _start;
.bss ALIGN(4096) : {
bss = .;
*(.bss)
. = ALIGN(4096);
}
memsz = . - macho;
/DISCARD/ : {
*(.*)
}
}
bsssize = SIZEOF(.bss);
textoff = _start - macho;
+735 -443
View File
File diff suppressed because it is too large. Load diff
-20
View File
@@ -1,20 +0,0 @@
#ifndef COSMOPOLITAN_APE_LOADER_H_
#define COSMOPOLITAN_APE_LOADER_H_
#define APE_LOADER_BASE 0x200000
#define APE_LOADER_SIZE 0x200000
#define APE_LOADER_ENTRY 0x200400
#define APE_LOADER_BSS (PAGESIZE * 2)
#define APE_LOADER_STACK 0x7f0000000000
#define APE_BLOCK_BASE 0x7e0000000000
#define APE_BLOCK_SIZE 0x000200000000
struct ApeLoader {
int fd;
int os;
char *prog;
char *page;
void *systemcall;
};
#endif /* COSMOPOLITAN_APE_LOADER_H_ */
+70 -18
View File
@@ -1,7 +1,7 @@
/*-*- mode: ld-script; indent-tabs-mode: nil; tab-width: 2; coding: utf-8 -*-│
│vi: set et sts=2 tw=2 fenc=utf-8 :vi│
╞══════════════════════════════════════════════════════════════════════════════╡
│ Copyright 2021 Justine Alexandra Roberts Tunney │
│ Copyright 2023 Justine Alexandra Roberts Tunney │
│ │
│ Permission to use, copy, modify, and/or distribute this software for │
│ any purpose with or without fee is hereby granted, provided that the │
@@ -16,28 +16,80 @@
│ TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR │
│ PERFORMANCE OF THIS SOFTWARE. │
╚─────────────────────────────────────────────────────────────────────────────*/
ENTRY(_start)
ENTRY(ElfEntrypoint)
PHDRS {
text PT_LOAD FLAGS(1) FILEHDR PHDRS; /* PF_X */
rodata PT_LOAD FLAGS(4); /* PF_R */
stack PT_GNU_STACK FLAGS(6); /* PF_W|PF_R */
note PT_NOTE FLAGS(4); /* PF_R */
}
SECTIONS {
. = 0x200000;
. = SEGMENT_START("text-segment", 0x7f000000);
__executable_start = .;
. += SIZEOF_HEADERS;
.macho : {
KEEP(*(.macho))
} :text
.note : {
KEEP(*(.note))
} :text :note
.text : {
*(.text)
*(.rodata .rodata.*)
. = ALIGN(64);
}
filesz = . - ehdr;
textsz = . - _start;
.bss ALIGN(4096) : {
bss = .;
*(.bss)
. = ALIGN(4096);
}
*(.text .text.* .gnu.linkonce.t.*)
} :text
.rodata ALIGN(CONSTANT(COMMONPAGESIZE)) : {
*(.rodata .rodata.* .gnu.linkonce.r.*)
} :rodata
.stack : {
*(.stack)
} :stack
_end = .;
memsz = _end - ehdr;
.stab 0 : { *(.stab) }
.stabstr 0 : { *(.stabstr) }
.stab.excl 0 : { *(.stab.excl) }
.stab.exclstr 0 : { *(.stab.exclstr) }
.stab.index 0 : { *(.stab.index) }
.stab.indexstr 0 : { *(.stab.indexstr) }
.gnu.build.attributes : { *(.gnu.build.attributes .gnu.build.attributes.*) }
.debug 0 : { *(.debug) }
.line 0 : { *(.line) }
.debug_srcinfo 0 : { *(.debug_srcinfo) }
.debug_sfnames 0 : { *(.debug_sfnames) }
.debug_aranges 0 : { *(.debug_aranges) }
.debug_pubnames 0 : { *(.debug_pubnames) }
.debug_info 0 : { *(.debug_info .gnu.linkonce.wi.*) }
.debug_abbrev 0 : { *(.debug_abbrev) }
.debug_line 0 : { *(.debug_line .debug_line.* .debug_line_end) }
.debug_frame 0 : { *(.debug_frame) }
.debug_str 0 : { *(.debug_str) }
.debug_loc 0 : { *(.debug_loc) }
.debug_macinfo 0 : { *(.debug_macinfo) }
.debug_weaknames 0 : { *(.debug_weaknames) }
.debug_funcnames 0 : { *(.debug_funcnames) }
.debug_typenames 0 : { *(.debug_typenames) }
.debug_varnames 0 : { *(.debug_varnames) }
.debug_pubtypes 0 : { *(.debug_pubtypes) }
.debug_ranges 0 : { *(.debug_ranges) }
.debug_macro 0 : { *(.debug_macro) }
.debug_addr 0 : { *(.debug_addr) }
.debug_loclists 0 : { *(.debug_loclists) }
.debug_rnglists 0 : { *(.debug_rnglists) }
.debug_line_str 0 : { *(.debug_line_str) }
.debug_extra 0 : { *(.debug_line_str) }
.debug_pubtypes 0 : { *(.debug_pubtypes) }
.gnu.attributes 0 : { KEEP(*(.gnu.attributes)) }
/DISCARD/ : {
*(.*)
}
}
bsssize = SIZEOF(.bss);
textoff = _start - ehdr;
+20 -25
View File
@@ -3,31 +3,26 @@
#if !(__ASSEMBLER__ + __LINKER__ + 0)
COSMOPOLITAN_C_START_
extern unsigned char _base[];
extern unsigned char _ehead[];
extern unsigned char _etext[];
extern unsigned char _edata[];
extern unsigned char _ezip[];
extern unsigned char _end[];
extern unsigned char _ereal[];
extern unsigned char _tdata_start[];
extern unsigned char _tdata_end[];
extern unsigned char _tbss_start[];
extern unsigned char _tbss_end[];
extern unsigned char __privileged_start[];
extern unsigned char __privileged_addr[];
extern unsigned char __privileged_size[];
extern unsigned char __privileged_end[];
extern unsigned char __test_start[];
extern unsigned char __ro[];
extern unsigned char *__relo_start[];
extern unsigned char *__relo_end[];
extern uint8_t __zip_start[];
extern uint8_t __zip_end[];
extern uint8_t __data_start[];
extern uint8_t __data_end[];
extern uint8_t __bss_start[];
extern uint8_t __bss_end[];
extern const char __comment_start[] __attribute__((__weak__));
extern unsigned char __executable_start[] __attribute__((__weak__));
extern unsigned char __privileged_start[] __attribute__((__weak__));
extern unsigned char _ehead[] __attribute__((__weak__));
extern unsigned char _etext[] __attribute__((__weak__));
extern unsigned char _edata[] __attribute__((__weak__));
extern unsigned char _ezip[] __attribute__((__weak__));
extern unsigned char _end[] __attribute__((__weak__));
extern unsigned char _ereal[] __attribute__((__weak__));
extern unsigned char _tdata_start[] __attribute__((__weak__));
extern unsigned char _tdata_end[] __attribute__((__weak__));
extern unsigned char _tbss_start[] __attribute__((__weak__));
extern unsigned char _tbss_end[] __attribute__((__weak__));
extern unsigned char _tls_align[] __attribute__((__weak__));
extern unsigned char __test_start[] __attribute__((__weak__));
extern unsigned char __ro[] __attribute__((__weak__));
extern uint8_t __data_start[] __attribute__((__weak__));
extern uint8_t __data_end[] __attribute__((__weak__));
extern uint8_t __bss_start[] __attribute__((__weak__));
extern uint8_t __bss_end[] __attribute__((__weak__));
COSMOPOLITAN_C_END_
#endif /* !(__ASSEMBLER__ + __LINKER__ + 0) */
+72
View File
@@ -0,0 +1,72 @@
/*-*- mode:unix-assembly; indent-tabs-mode:t; tab-width:8; coding:utf-8 -*-│
│vi: set et ft=asm ts=8 tw=8 fenc=utf-8 :vi│
╞══════════════════════════════════════════════════════════════════════════════╡
│ Copyright 2023 Justine Alexandra Roberts Tunney │
│ │
│ Permission to use, copy, modify, and/or distribute this software for │
│ any purpose with or without fee is hereby granted, provided that the │
│ above copyright notice and this permission notice appear in all copies. │
│ │
│ THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL │
│ WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED │
│ WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE │
│ AUTHOR BE LIABLE FOR ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL │
│ DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR │
│ PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER │
│ TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR │
│ PERFORMANCE OF THIS SOFTWARE. │
╚─────────────────────────────────────────────────────────────────────────────*/
#include "libc/dce.h"
#include "ape/ape.h"
#include "libc/macros.internal.h"
#ifdef __aarch64__
_start: mov x1,sp
mov x29,0
bl ApeLoader
.endfn _start,globl
#else
XnuEntrypoint:
mov $_HOSTXNU,%dl // xnu's not unix!
ElfEntrypoint:
mov %rsp,%rsi // save real stack
call ApeLoader
.endfn ElfEntrypoint,globl
.endfn XnuEntrypoint,globl
.section .note,"a",@progbits
.balign 4
openbsd.ident:
.long 2f-1f
.long 4f-3f
.long 1
1: .asciz "OpenBSD"
2: .balign 4
3: .long 0
4: .size openbsd.ident,.-openbsd.ident
.type openbsd.ident,@object
.balign 4
netbsd.ident:
.long 2f-1f
.long 4f-3f
.long 1
1: .asciz "NetBSD"
2: .balign 4
3: .long 901000000
4: .size netbsd.ident,.-netbsd.ident
.type netbsd.ident,@object
.balign 4
ape.ident:
.long 2f-1f
.long 4f-3f
.long 1
1: .asciz "APE"
2: .balign 4
3: .long APE_VERSION_NOTE
4: .size ape.ident,.-ape.ident
.type ape.ident,@object
#endif
+49
View File
@@ -0,0 +1,49 @@
/*-*- mode:unix-assembly; indent-tabs-mode:t; tab-width:8; coding:utf-8 -*-│
│vi: set et ft=asm ts=8 tw=8 fenc=utf-8 :vi│
╞══════════════════════════════════════════════════════════════════════════════╡
│ Copyright 2023 Justine Alexandra Roberts Tunney │
│ │
│ Permission to use, copy, modify, and/or distribute this software for │
│ any purpose with or without fee is hereby granted, provided that the │
│ above copyright notice and this permission notice appear in all copies. │
│ │
│ THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL │
│ WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED │
│ WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE │
│ AUTHOR BE LIABLE FOR ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL │
│ DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR │
│ PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER │
│ TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR │
│ PERFORMANCE OF THIS SOFTWARE. │
╚─────────────────────────────────────────────────────────────────────────────*/
#include "libc/macros.internal.h"
// Invokes system call.
//
// This function has eight parameters. The first seven are for
// arguments passed along to the system call. The eight is for
// the magic number that indicates which system call is called
//
// The return value follows the Linux kernel convention, where
// errors are returned as `-errno`. BSD systems are normalized
// to follow this convention automatically.
SystemCall:
#ifdef __aarch64__
mov x8,x7
mov x9,0
adds x9,x9,0
svc 0
bcs 1f
ret
1: neg x0,x0
ret
#else
mov %rcx,%r10
mov 16(%rsp),%eax
clc
syscall
jnc 1f
neg %rax
1: ret
#endif
.endfn SystemCall,globl
+19
View File
@@ -0,0 +1,19 @@
#!/bin/sh
MODE=${MODE:-${m:-aarch64}}
COSMO=${COSMO:-/opt/cosmo}
COSMOS=${COSMOS:-/opt/cosmos/aarch64}
if [ -x "$COSMO/o/third_party/gcc/bin/aarch64-linux-musl-gcc" ]; then
BRAND=musl
else
BRAND=cosmo
fi
TOOL="$COSMO/o/third_party/gcc/bin/aarch64-linux-$BRAND-addr2line"
if [ ! -x "$TOOL" ]; then
echo "$0: you need to run: aarch64-unknown-cosmo-cc --update" >&2
exit 1
fi
exec "$TOOL" "$@"
+1
View File
@@ -0,0 +1 @@
apecopy
+21
View File
@@ -0,0 +1,21 @@
#!/bin/sh
PROG=${0##*/}
MODE=${MODE:-${m:-aarch64}}
COSMO=${COSMO:-/opt/cosmo}
fatal_error() {
echo "$PROG: $1" >&2
exit 1
}
if [ -x "$COSMO/o/third_party/gcc/bin/aarch64-linux-musl-gcc" ]; then
BRAND=musl
else
BRAND=cosmo
fi
if [ ! -x "$COSMO/o/third_party/gcc/bin/aarch64-linux-$BRAND-ar" ]; then
fatal_error "you need to run: aarch64-unknown-cosmo-cc --update"
fi
exec "$COSMO/o/third_party/gcc/bin/aarch64-linux-$BRAND-ar" "$@"
+19
View File
@@ -0,0 +1,19 @@
#!/bin/sh
MODE=${MODE:-${m:-aarch64}}
COSMO=${COSMO:-/opt/cosmo}
COSMOS=${COSMOS:-/opt/cosmos/aarch64}
if [ -x "$COSMO/o/third_party/gcc/bin/aarch64-linux-musl-gcc" ]; then
BRAND=musl
else
BRAND=cosmo
fi
TOOL="$COSMO/o/third_party/gcc/bin/aarch64-linux-$BRAND-as"
if [ ! -x "$TOOL" ]; then
echo "$0: you need to run: aarch64-unknown-cosmo-cc --update" >&2
exit 1
fi
exec "$TOOL" "$@"
+1
View File
@@ -0,0 +1 @@
cosmocc
+1
View File
@@ -0,0 +1 @@
cosmocc
+15
View File
@@ -0,0 +1,15 @@
#!/bin/sh
FIRST=1
for x; do
if [ $FIRST -eq 1 ]; then
set --
FIRST=0
fi
if [ x"$x" = x"-s" ]; then
continue
fi
set -- "$@" "$x"
done
exec install "$@"
+19
View File
@@ -0,0 +1,19 @@
#!/bin/sh
MODE=${MODE:-${m:-aarch64}}
COSMO=${COSMO:-/opt/cosmo}
COSMOS=${COSMOS:-/opt/cosmos/aarch64}
if [ -x "$COSMO/o/third_party/gcc/bin/aarch64-linux-musl-gcc" ]; then
BRAND=musl
else
BRAND=cosmo
fi
TOOL="$COSMO/o/third_party/gcc/bin/aarch64-linux-$BRAND-nm"
if [ ! -x "$TOOL" ]; then
echo "$0: you need to run: aarch64-unknown-cosmo-cc --update" >&2
exit 1
fi
exec "$TOOL" "$@"
+19
View File
@@ -0,0 +1,19 @@
#!/bin/sh
MODE=${MODE:-${m:-aarch64}}
COSMO=${COSMO:-/opt/cosmo}
COSMOS=${COSMOS:-/opt/cosmos/aarch64}
if [ -x "$COSMO/o/third_party/gcc/bin/aarch64-linux-musl-gcc" ]; then
BRAND=musl
else
BRAND=cosmo
fi
TOOL="$COSMO/o/third_party/gcc/bin/aarch64-linux-$BRAND-objcopy"
if [ ! -x "$TOOL" ]; then
echo "$0: you need to run: aarch64-unknown-cosmo-cc --update" >&2
exit 1
fi
exec "$TOOL" "$@"
+19
View File
@@ -0,0 +1,19 @@
#!/bin/sh
MODE=${MODE:-${m:-aarch64}}
COSMO=${COSMO:-/opt/cosmo}
COSMOS=${COSMOS:-/opt/cosmos/aarch64}
if [ -x "$COSMO/o/third_party/gcc/bin/aarch64-linux-musl-gcc" ]; then
BRAND=musl
else
BRAND=cosmo
fi
TOOL="$COSMO/o/third_party/gcc/bin/aarch64-linux-$BRAND-objdump"
if [ ! -x "$TOOL" ]; then
echo "$0: you need to run: aarch64-unknown-cosmo-cc --update" >&2
exit 1
fi
exec "$TOOL" "$@"
+19
View File
@@ -0,0 +1,19 @@
#!/bin/sh
MODE=${MODE:-${m:-aarch64}}
COSMO=${COSMO:-/opt/cosmo}
COSMOS=${COSMOS:-/opt/cosmos/aarch64}
if [ -x "$COSMO/o/third_party/gcc/bin/aarch64-linux-musl-gcc" ]; then
BRAND=musl
else
BRAND=cosmo
fi
TOOL="$COSMO/o/third_party/gcc/bin/aarch64-linux-$BRAND-strip"
if [ ! -x "$TOOL" ]; then
echo "$0: you need to run: aarch64-unknown-cosmo-cc --update" >&2
exit 1
fi
exec "$TOOL" "$@"
+134
View File
@@ -0,0 +1,134 @@
#!/bin/sh
PROG=${0##*/}
MODE=${MODE:-$m}
COSMO=${COSMO:-/opt/cosmo}
COSMOS=${COSMOS:-/opt/cosmos}
if [ ! -f ape/loader.c ]; then
cd "$COSMO" || exit
fi
if [ "$(id -u)" -eq 0 ]; then
SUDO=
else
SUDO=sudo
fi
echo "Actually Portable Executable (APE) Installer" >&2
echo "Author: Justine Tunney <[email protected]>" >&2
################################################################################
# INSTALL APE LOADER SYSTEMWIDE
if [ -f o/depend ] && make -j8 o//ape; then
echo "successfully recompiled ape loader" >&2
elif [ -x o//ape/ape.elf ] && [ -x o//ape/ape.macho ]; then
echo "using ape loader you compiled earlier" >&2
elif [ -d build/bootstrap ]; then
# if make isn't being used then it's unlikely the user changed the sources
# in that case the prebuilt binaries should be completely up-to-date
echo "using prebuilt ape loader from cosmo repo" >&2
mkdir -p o//ape || exit
cp -af build/bootstrap/ape.elf o//ape/ape.elf || exit
cp -af build/bootstrap/ape.macho o//ape/ape.macho || exit
else
echo "no cosmopolitan libc repository here" >&2
echo "fetching ape loader from justine.lol" >&2
mkdir -p o//ape || exit
if command -v wget >/dev/null 2>&1; then
wget -qO o//ape/ape.elf https://justine.lol/ape.elf || exit
wget -qO o//ape/ape.macho https://justine.lol/ape.macho || exit
else
curl -Rso o//ape/ape.elf https://justine.lol/ape.elf || exit
curl -Rso o//ape/ape.macho https://justine.lol/ape.macho || exit
fi
chmod +x o//ape/ape.elf || exit
chmod +x o//ape/ape.macho || exit
fi
if [ "$(uname -s)" = "Darwin" ]; then
if ! [ /usr/bin/ape -nt o//ape/ape.macho ]; then
echo >&2
echo "installing o//ape/ape.elf to /usr/bin/ape" >&2
echo "$SUDO mv -f o//ape/ape.elf /usr/bin/ape" >&2
$SUDO cp -f o//ape/ape.macho /usr/bin/ape || exit
echo "done" >&2
fi
else
if ! [ /usr/bin/ape -nt o//ape/ape.elf ]; then
echo >&2
echo "installing o//ape/ape.elf to /usr/bin/ape" >&2
echo "$SUDO mv -f o//ape/ape.elf /usr/bin/ape" >&2
$SUDO cp -f o//ape/ape.elf /usr/bin/ape || exit
echo "done" >&2
fi
fi
################################################################################
# REGISTER APE LOADER WITH BINFMT_MISC TOO (LINUX-ONLY)
if [ x"$(uname -s)" = xLinux ]; then
if [ -e /proc/sys/fs/binfmt_misc/APE ]; then
echo >&2
echo it looks like APE is already registered with binfmt_misc >&2
echo To reinstall please run ape/apeuninstall.sh first >&2
echo please check that it is mapped to ape not /bin/sh >&2
echo cat /proc/sys/fs/binfmt_misc/APE >&2
cat /proc/sys/fs/binfmt_misc/APE >&2
exit
fi
if ! [ -e /proc/sys/fs/binfmt_misc ]; then
echo >&2
echo loading binfmt_misc into your kernel >&2
echo you may need to edit configs to persist across reboot >&2
echo $SUDO modprobe binfmt_misc >&2
$SUDO modprobe binfmt_misc || exit
echo done >&2
fi
if ! [ -e /proc/sys/fs/binfmt_misc/register ]; then
echo >&2
echo mounting binfmt_misc into your kernel >&2
echo you may need to edit configs to persist across reboot >&2
echo $SUDO mount -t binfmt_misc none /proc/sys/fs/binfmt_misc >&2
$SUDO mount -t binfmt_misc none /proc/sys/fs/binfmt_misc || exit
echo done >&2
fi
echo >&2
echo registering APE with binfmt_misc >&2
echo you may need to edit configs to persist across reboot >&2
echo '$SUDO sh -c "echo '"'"':APE:M::MZqFpD::/usr/bin/ape:'"'"' >/proc/sys/fs/binfmt_misc/register"' >&2
$SUDO sh -c "echo ':APE:M::MZqFpD::/usr/bin/ape:' >/proc/sys/fs/binfmt_misc/register" || exit
echo '$SUDO sh -c "echo '"'"':APE-jart:M::jartsr::/usr/bin/ape:'"'"' >/proc/sys/fs/binfmt_misc/register"' >&2
$SUDO sh -c "echo ':APE-jart:M::jartsr::/usr/bin/ape:' >/proc/sys/fs/binfmt_misc/register" || exit
echo done >&2
if [ x"$(cat /proc/sys/fs/binfmt_misc/status)" = xdisabled ]; then
echo >&2
echo enabling binfmt_misc >&2
echo you may need to edit configs to persist across reboot >&2
echo $SUDO sh -c 'echo 1 >/proc/sys/fs/binfmt_misc/status' >&2
$SUDO sh -c 'echo 1 >/proc/sys/fs/binfmt_misc/status' || exit
echo done >&2
fi
fi
################################################################################
{
echo
echo "------------------------------------------------------------------"
echo
echo "APE INSTALL COMPLETE"
echo
echo "If you decide to uninstall APE later on"
echo "you may do so using ape/apeuninstall.sh"
echo
echo "Enjoy your APE loader (>'.')>"
echo
} >&2
+60
View File
@@ -0,0 +1,60 @@
#!/bin/sh
PROG=${0##*/}
MODE=${MODE:-$m}
COSMO=${COSMO:-/opt/cosmo}
COSMOS=${COSMOS:-/opt/cosmos}
if [ ! -f ape/loader.c ]; then
cd "$COSMO" || exit
fi
if [ "$UID" = "0" ]; then
SUDO=
else
SUDO=sudo
fi
{
echo
echo "APE Uninstaller intends to run (in pseudo-shell)"
echo
echo " sudo echo -1 into /proc/sys/fs/binfmt_misc/APE*"
echo " sudo rm -f /usr/bin/ape ~/.ape /tmp/.ape # etc."
echo
echo "You may then use ape/apeinstall.sh to reinstall it"
echo
} >&2
set -ex
for f in /proc/sys/fs/binfmt_misc/APE*; do
if [ -f $f ]; then
$SUDO sh -c "echo -1 >$f" || exit
fi
done
# system installation
if [ -f /usr/bin/ape ]; then
$SUDO rm -f /usr/bin/ape
fi
# legacy installations
rm -f o/tmp/ape /tmp/ape "${TMPDIR:-/tmp}/ape"
# ad-hoc installations
for x in .ape \
.ape-1.1 \
.ape-1.3 \
.ape-1.4 \
.ape-1.5 \
.ape-1.6 \
.ape-1.7 \
.ape-1.8 \
.ape-blink-0.9.2 \
.ape-blink-1.0.0; do
rm -f \
~/$x \
/tmp/$x \
o/tmp/$x \
"${TMPDIR:-/tmp}/$x"
done
Executable
+102
View File
@@ -0,0 +1,102 @@
#!/bin/sh
PROG=${0##*/}
COSMO=${COSMO:-/opt/cosmo}
if [ x"$1" = x"--version" ]; then
cat <<EOF
$PROG 0.1
Copyright (c) 2023 Justine Alexandra Roberts Tunney
Cosmopolitan Libc and LLVM libcxx/compiler-rt are subject to non-GPL
notice licenses, e.g. ISC, MIT, etc. Your compiled programs must embed
our copyright notices. This toolchain is configured to do so default.
Cosmopolitan comes with absolutely NO WARRANTY of any kind.
For more information, see the Cosmopolitan LICENSE files.
Copyright (C) 2019 Free Software Foundation, Inc.
This launches GNU GCC/Binutils subprocesses, which is free software; see
Cosmopolitan's third_party/gcc/ for source code and copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.
EOF
exit
fi
if [ x"$1" = x"--help" ]; then
cat <<EOF
Usage: $PROG INPUT OUTPUT
Converts an executable linked by cosmocc to the Actually Portable
Executable file format. See $PROG --version for other details.
EOF
exit
fi
fatal_error() {
echo "$PROG: fatal error: $1" >&2
echo "operation terminated." >&2
exit 1
}
CROSS=1
ARCH=${PROG%%-*} # split(prog, '-')[0]
if [ x"$ARCH" = x"$PROG" ]; then
ARCH=x86_64
CROSS=0
fi
if [ x"$ARCH" = x"x86_64" ]; then
MODE=${MODE:-$m}
else
MODE=${MODE:-${m:-$ARCH}}
fi
if [ $CROSS -eq 0 ]; then
COSMOS=${COSMOS:-/opt/cosmos}
else
COSMOS=${COSMOS:-/opt/cosmos/$ARCH}
fi
if [ x"$ARCH" = x"x86_64" ]; then
OBJCOPYFLAGS="-S -O binary"
elif [ x"$ARCH" = x"aarch64" ]; then
OBJCOPYFLAGS="-S"
else
fatal_error "$ARCH: unsupported architecture"
fi
if [ ! -d "$COSMO" ]; then
fatal_error "you need to clone cosmopolitan to your $COSMO directory"
fi
if [ -x "$COSMO/o/third_party/gcc/bin/x86_64-linux-musl-gcc" ]; then
BRAND=musl
else
BRAND=cosmo
fi
if [ ! -f "$COSMO/o//tool/build/zipcopy.com" ] ||
[ ! -f "$COSMO/o/third_party/gcc/bin/$ARCH-linux-$BRAND-objcopy" ]; then
if [ $CROSS -eq 0 ]; then
fatal_error "you need to run: cosmocc --update"
else
fatal_error "you need to run: $ARCH-unknown-cosmo-cosmocc --update"
fi
fi
INPUT=$1
OUTPUT=$2
if [ ! -f "$INPUT" ]; then
fatal_error "$INPUT: missing input and output arguments"
elif [ ! -f "$INPUT" ]; then
fatal_error "$INPUT: no such file"
elif [ ! -f "$INPUT" ]; then
fatal_error "$INPUT: missing output argument"
elif [ x"$INPUT" = x"$OUTPUT" ]; then
fatal_error "$INPUT: input and output file can't be the same"
fi
"$COSMO/o/third_party/gcc/bin/$ARCH-linux-$BRAND-objcopy" \
$OBJCOPYFLAGS \
"$INPUT" \
"$OUTPUT" || exit
"$COSMO/o//tool/build/zipcopy.com" \
"$INPUT" \
"$OUTPUT" || exit
Executable
+3
View File
@@ -0,0 +1,3 @@
#!/bin/sh
COSMO=${COSMO:-/opt/cosmo}
exec "$COSMO/o//tool/build/apelink.com" "$@"
+31
View File
@@ -0,0 +1,31 @@
#!/bin/sh
COSMO=${COSMO:-/opt/cosmo}
set -- -apifCe "$@"
if [ -n "$ADDR2LINE" ]; then
exec "$ADDR2LINE" "$@"
fi
find_addr2line() {
if [ -x o/third_party/gcc/bin/$1-linux-cosmo-addr2line ]; then
ADDR2LINE=o/third_party/gcc/bin/$1-linux-cosmo-addr2line
elif [ -x o/third_party/gcc/bin/$1-linux-musl-addr2line ]; then
ADDR2LINE=o/third_party/gcc/bin/$1-linux-musl-addr2line
elif [ -x "$COSMO/o/third_party/gcc/bin/$1-linux-cosmo-addr2line" ]; then
ADDR2LINE="$COSMO/o/third_party/gcc/bin/$1-linux-cosmo-addr2line"
elif [ -x "$COSMO/o/third_party/gcc/bin/$1-linux-musl-addr2line" ]; then
ADDR2LINE="$COSMO/o/third_party/gcc/bin/$1-linux-musl-addr2line"
else
echo "error: toolchain not found (try running 'cosmocc --update' or 'make' in the cosmo monorepo)" >&2
exit 1
fi
}
for ARCH in x86_64 aarch64; do
find_addr2line $ARCH
"$ADDR2LINE" "$@" 2>/dev/null && exit
done
echo "error: addr2line failed" >&2
exit 1
+1
View File
@@ -0,0 +1 @@
cosmocc
Executable
+381
View File
@@ -0,0 +1,381 @@
#!/bin/sh
#
# cosmopolitan c/c++ compiler
#
# getting started
#
# sudo chmod 1777 /opt # sticky bit isn't required
# git clone https://github.com/jart/cosmopolitan /opt/cosmo
# export PATH="/opt/cosmo/bin:/opt/cosmos/bin:$PATH"
# echo 'export PATH="/opt/cosmo/bin:/opt/cosmos/bin:$PATH"' >>~/.profile
# ape-install # optionally install a faster systemwide ape loader
# cosmocc --update # pull and rebuild toolchain artifacts
#
# getting started synopsis
#
# cosmocc -o hello.com hello.c
# ./foo.com
# ./foo.com --strace
# ./foo.com --ftrace
#
# how to build a project like lua 5.4.6
#
# make all test CC=cosmocc
# src/lua -e 'print("hi")'
# make install INSTALL_TOP=/opt/cosmos
# apecopy src/lua src/lua.com # convert to portable ape binary
#
# how to cross compile a project like lua 5.4.6
#
# aarch64-unknown-cosmo-cc --update
# make clean all test CC=aarch64-unknown-cosmo-cc AR='aarch64-unknown-cosmo-ar rc' -j8
# qemu-aarch64 src/lua -e 'print("hi")'
# make install INSTALL_TOP=/opt/cosmos/aarch64
#
# building in tiny mode
#
# export MODE=tiny
# cosmocc --update
# cosmocc -Os -o foo.com foo.c
#
# building in tiniest mode (linux only)
#
# export MODE=tinylinux
# cosmocc --update
# cosmocc -Os -o foo.com foo.c
#
# hardening programs with memory safety
#
# export MODE=asan
# cosmocc --update
# cosmocc -o foo.com foo.c
#
# detecting this environment
#
# - `__COSMOCC__` is defined when this compiler is in play
# - `__COSMOPOLITAN__` is always defined by Cosmopolitan Libc
#
# some notes on this compiler
#
# - the underlying compiler itself is gcc
# - we use cosmopoiltan libc rather than glibc
# - we use llvm's compiler-rt and libcxx runtimes
# - we patched gcc so switch case can have symbols
# - our scanf() implementation is somewhat troubled
#
# compiler flags that aren't supported
#
# - `-r` partial linking not implemented yet (todo)
# - `-S` you need to put your assembly behind #ifdefs instead
# - `-fexceptions` cosmopolitan doesn't support c++ exceptions yet
# - `-frtti` cosmopolitan doesn't support c++ runtime reflection yet
# - `-mred-zone` the system v red zone doesn't exist on windows and metal
# - `-fpic`, '-fPIC', `-shared`, `-pie`, etc. no shared object support yet
# - `-fsanitize=thread` cosmopolitan doesn't have thread sanitizer runtime yet
# - `-fomit-frame-pointer` is partially supported (apple forbids full removal)
#
# for further details, run `man gcc`
PROG=${0##*/}
COSMO=${COSMO:-/opt/cosmo}
GCC_VERSION=11.2.0
if [ "$1" = "--version" ]; then
cat <<EOF
$PROG (GCC) $GCC_VERSION
Copyright (c) 2023 Justine Alexandra Roberts Tunney
Cosmopolitan Libc and LLVM libcxx/compiler-rt are subject to non-GPL
notice licenses, e.g. ISC, MIT, etc. Your compiled programs must embed
our copyright notices. This toolchain is configured to do so default.
Cosmopolitan comes with absolutely NO WARRANTY of any kind.
For more information, see the Cosmopolitan LICENSE files.
Copyright (C) 2019 Free Software Foundation, Inc.
This launches GNU GCC/Binutils subprocesses, which is free software; see
Cosmopolitan's third_party/gcc/ for source code and copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.
EOF
exit
fi
fatal_error() {
echo "$PROG: fatal error: $1" >&2
echo "compilation terminated." >&2
exit 1
}
log_command() {
if [ -n "$BUILDLOG" ]; then
printf '# %s\n(cd %s; %s)\n' "$ORIGINAL" "$PWD" "$*" >>"$BUILDLOG"
fi
}
ORIGINAL="$0 $*"
PLATFORM="-D__COSMOPOLITAN__ -D__COSMOCC__"
PREDEF="-include libc/integral/normalize.inc"
CFLAGS="-fportcosmo -fno-dwarf2-cfi-asm -fno-unwind-tables -fno-asynchronous-unwind-tables -fno-semantic-interposition"
CPPFLAGS="-fno-pie -nostdinc -fno-math-errno -iquote $COSMO"
LDFLAGS="-static -no-pie -nostdlib -fuse-ld=bfd"
APEFLAGS="-Wl,--gc-sections"
PRECIOUS="-fno-omit-frame-pointer"
CROSS=1
ARCH=${PROG%%-*} # split(prog, '-')[0]
if [ x"$ARCH" = x"$PROG" ]; then
ARCH=x86_64
CROSS=0
fi
if [ x"$ARCH" = x"x86_64" ]; then
MODE=${MODE:-$m}
else
MODE=${MODE:-${m:-$ARCH}}
fi
if [ $CROSS -eq 0 ]; then
COSMOS=${COSMOS:-/opt/cosmos}
else
COSMOS=${COSMOS:-/opt/cosmos/$ARCH}
fi
if [ -x "$COSMO/o/third_party/gcc/bin/x86_64-linux-musl-gcc" ]; then
BRAND=musl
else
BRAND=cosmo
fi
CC="$COSMO/o/third_party/gcc/bin/$ARCH-linux-$BRAND-gcc"
CRT="$COSMO/o/$MODE/libc/crt/crt.o"
LDLIBS="$COSMO/o/$MODE/cosmopolitan.a"
CPPFLAGS="$CPPFLAGS -isystem $COSMOS/include -isystem $COSMO/libc/isystem"
LDFLAGS="$LDFLAGS -L$COSMOS/lib"
if [ x"$PROG" != x"${PROG%++}" ]; then
CC="$COSMO/o/third_party/gcc/bin/$ARCH-linux-$BRAND-g++"
CFLAGS="$CFLAGS -fno-rtti -fno-exceptions -fuse-cxa-atexit"
LDLIBS="$COSMO/o/$MODE/third_party/libcxx/libcxx.a $LDLIBS"
fi
PAGESZ=4096
if [ x"$ARCH" = x"x86_64" ]; then
OBJCOPYFLAGS="-S -O binary"
CRT="$COSMO/o/$MODE/ape/ape-no-modify-self.o $CRT"
CPPFLAGS="$CPPFLAGS -mno-red-zone"
CFLAGS="$CFLAGS -mno-tls-direct-seg-refs"
LDFLAGS="$LDFLAGS -Wl,-T,$COSMO/o/$MODE/ape/ape.lds"
if [ x"$MODE" = x"aarch64" ]; then
fatal_error '$MODE must not be aarch64 when using x86_64 cross compiler'
elif [ x"$MODE" != x"${MODE#* }" ]; then
fatal_error '$MODE must not contain hyphens when targeting x86_64'
fi
elif [ x"$ARCH" = x"aarch64" ]; then
OBJCOPYFLAGS="-S"
PAGESZ=16384
CFLAGS="$CFLAGS -ffixed-x18 -ffixed-x28 -mno-outline-atomics"
LDFLAGS="$LDFLAGS -Wl,-T,$COSMO/o/$MODE/ape/aarch64.lds"
if [ x"$MODE" != x"aarch64" ] && [ x"$MODE" = x"${MODE#aarch64-*}" ]; then
fatal_error '$MODE must be either "aarch64" or "aarch64-FOO" when using aarch64 cross compiler'
fi
else
fatal_error "$ARCH: unsupported architecture"
fi
LDFLAGS="$LDFLAGS -Wl,-z,common-page-size=$PAGESZ -Wl,-z,max-page-size=16384"
if [ ! -d "$COSMO" ]; then
fatal_error "you need to clone cosmopolitan to your $COSMO directory"
fi
if [ "$1" = "--update" ]; then
cd $COSMO || exit
echo "building cosmo host toolchain..." >&2
make --silent -j toolchain MODE= ARCH=x86_64 || exit
echo "building cosmo target (MODE=$MODE) toolchain..." >&2
make --silent -j toolchain MODE="$MODE" || exit
echo "setting up your cosmos..." >&2
mkdir -p "$COSMOS/lib" || exit
for lib in c dl gcc_s m pthread resolv rt dl z stdc++; do
if [ ! -f "$COSMOS/lib/lib${lib}.a" ]; then
printf '\041\074\141\162\143\150\076\012' >"$COSMOS/lib/lib${lib}.a" || exit
fi
done
echo "successfully updated your cosmo toolchain" >&2
exit
fi
if [ ! -f "$COSMOS/lib/libc.a" ] ||
[ ! -f "$COSMO/o/$MODE/cosmopolitan.a" ]; then
fatal_error "you need to run: $PROG --update"
fi
OPT=
FIRST=1
OUTPUT=
SFLAG=0
INTENT=ld
GOT_SOME=0
NEED_OUTPUT=
RELOCATABLE=0
for x; do
if [ $FIRST -eq 1 ]; then
set --
FIRST=0
fi
if [ -n "$NEED_OUTPUT" ]; then
NEED_OUTPUT=
OUTPUT=$x
set -- "$@" "$x"
continue
fi
if [ x"$x" = x"-" ] || # is an argument
[ x"$x" = x"${x#-*}" ]; then # !startswith(x, "-")
GOT_SOME=1
elif [ x"$x" = x"-static-libstdc++" ]; then
continue
elif [ x"$x" = x"-static-libgcc" ]; then
continue
elif [ x"$x" != x"${x#-O}" ]; then
OPT=$x
elif [ x"$x" = x"-c" ]; then
INTENT=cc
elif [ x"$x" = x"-s" ]; then
SFLAG=1
continue
elif [ x"$x" = x"-r" ]; then
RELOCATABLE=1
elif [ x"$x" = x"-E" ]; then
INTENT=cpp
elif [ x"$x" = x"-o" ]; then
NEED_OUTPUT=1
elif [ x"$x" = x"-mcosmo" ]; then
CPPFLAGS="$CPPFLAGS -D_COSMO_SOURCE"
continue
elif [ x"$x" != x"${x#-o}" ]; then
OUTPUT=${x#-o}
elif [ x"$x" = x"-fpic" ]; then
continue
elif [ x"$x" = x"-fPIC" ]; then
continue
elif [ x"$x" = x"-r" ] ||
[ x"$x" = x"-pie" ] ||
[ x"$x" = x"-frtti" ] ||
[ x"$x" = x"-shared" ] ||
[ x"$x" = x"-nostdlib" ] ||
[ x"$x" = x"-mred-zone" ] ||
[ x"$x" = x"-fexceptions" ] ||
[ x"$x" = x"-fsanitize=thread" ]; then
echo "$PROG: $x not supported" >&2
exit 1
elif [ x"$x" = x"-fsanitize=all" ] ||
[ x"$x" = x"-fsanitize=address" ] ||
[ x"$x" = x"-fsanitize=undefined" ]; then
echo "$PROG: use cosmo MODE=dbg or MODE=asan rather than passing $x" >&2
exit 1
elif [ x"$x" = x"-fomit-frame-pointer" ]; then
# Quoth Apple "The frame pointer register must always address a
# valid frame record. Some functions — such as leaf functions or
# tail calls — may opt not to create an entry in this list. As a
# result, stack traces are always meaningful, even without debug
# information."
set -- "$@" -momit-leaf-frame-pointer -foptimize-sibling-calls
continue
elif [ x"$x" != x"${x#-Werror}" ] || \
[ x"$x" = x"-pedantic-errors" ]; then
# this toolchain is intended for building other people's code
# elevating warnings into errors, should only be done by devs
continue
elif [ x"$x" = x"-march=native" ]; then
if [ $CROSS -eq 0 ]; then
set -- "$@" $("$COSMO/o//tool/build/march-native.com")
else
fatal_error "-march=native can't be used when cross compiling"
fi
continue
elif [ x"$x" = x"-dumpversion" ]; then
echo $GCC_VERSION
exit 0
fi
set -- "$@" "$x"
done
if [ "$GOT_SOME" -eq 0 ]; then
fatal_error "no input files"
fi
if [ $RELOCATABLE -eq 1 ]; then
LDFLAGS="$LDFLAGS -r"
fi
if [ x"$MODE" = x"nox87" ]; then
CPPFLAGS="$CPPFLAGS -mlong-double-64"
fi
# support --ftrace unless optimizing for size
if [ x"$OPT" != x"-Os" ] && # $OPT != -Os
[ x"${MODE%tiny}" = x"${MODE}" ]; then # $MODE not in (tiny, aarch64-tiny)
if [ x"$ARCH" = x"x86_64" ]; then
CFLAGS="$CFLAGS -fpatchable-function-entry=18,16"
elif [ x"$ARCH" = x"aarch64" ]; then
CFLAGS="$CFLAGS -fpatchable-function-entry=7,6"
fi
fi
# ask compiler to generate sanitization code in debug mode
if [ x"$MODE" != x"${MODE%*dbg}" ]; then # endswith($MODE, "dbg")
if [ x"$ARCH" = x"x86_64" ]; then
CPPFLAGS="$CPPFLAGS -fsanitize=address -fsanitize=undefined"
else
CPPFLAGS="$CPPFLAGS -fsanitize=undefined"
fi
fi
# ask compiler to generate memory safety code in asan mode
if [ x"$MODE" != x"${MODE%*asan}" ]; then # endswith($MODE, "asan")
if [ x"$ARCH" = x"x86_64" ]; then
CPPFLAGS="$CPPFLAGS -fsanitize=address"
else
fatal_error "address sanitizer not supported on non-x86 yet"
fi
fi
# maximize frame pointers unless optimizing for size
if [ x"$OPT" != x"-Os" ] && # $OPT != "-Os"
[ x"$MODE" != x"${MODE%tiny}" ]; then # endswith($MODE, "tiny")
CFLAGS="$CFLAGS -fno-optimize-sibling-calls -mno-omit-leaf-frame-pointer"
fi
if [ $INTENT = cpp ]; then
set -- "$CC" $PLATFORM $CPPFLAGS "$@"
elif [ $INTENT = cc ]; then
set -- "$CC" $PLATFORM $PREDEF $CFLAGS $CPPFLAGS "$@" $PRECIOUS
else
set -- "$CC" $PLATFORM $PREDEF $CFLAGS $CPPFLAGS $CRT "$@" $LDFLAGS $LDLIBS $PRECIOUS
fi
log_command "$@"
"$@" || exit
if [ -n "$OUTPUT" ] && [ -f "$OUTPUT" ]; then
if [ $INTENT = cc ] || [ $INTENT = ld ]; then
"$COSMO/o//tool/build/fixupobj.com" \
"$OUTPUT" || exit
fi
if [ $INTENT = ld ]; then
if [ x"$OUTPUT" != x"${OUTPUT%.com}" ] ||
[ x"$OUTPUT" != x"${OUTPUT%.exe}" ]; then
# cosmocc -o foo.com ...
# -> foo.com (ape)
# -> foo.com.dbg (elf)
mv -f "$OUTPUT" "$OUTPUT.dbg" || exit
"$COSMO/o/third_party/gcc/bin/$ARCH-linux-$BRAND-objcopy" \
$OBJCOPYFLAGS \
"$OUTPUT.dbg" \
"$OUTPUT" || exit
"$COSMO/o//tool/build/zipcopy.com" \
"$OUTPUT.dbg" \
"$OUTPUT" || exit
elif [ $SFLAG -eq 1 ]; then
"$COSMO/o/third_party/gcc/bin/$ARCH-linux-$BRAND-strip" \
"$OUTPUT" || exit
fi
fi
fi
Executable
+119
View File
@@ -0,0 +1,119 @@
#!/bin/sh
PROG=${0##*/}
MODE=${MODE:-$m}
COSMO=${COSMO:-/opt/cosmo}
fatal_error() {
echo "$PROG: $1" >&2
exit 1
}
if [ -x "$COSMO/o/third_party/gcc/bin/x86_64-linux-musl-gcc" ]; then
BRAND=musl
else
BRAND=cosmo
fi
if [ ! -x "$COSMO/o/third_party/gcc/bin/x86_64-linux-$BRAND-ar" ]; then
fatal_error "you need to run: fatcosmocc --update"
fi
AR_X86_64="$COSMO/o/third_party/gcc/bin/x86_64-linux-$BRAND-ar"
AR_AARCH64="$COSMO/o/third_party/gcc/bin/aarch64-linux-$BRAND-ar"
if [ "$1" = "--version" ]; then
# note: only the underlying gnu compiler binaries are gpl
# our shell script is released with the isc license
cat <<EOF
$PROG (GNU Binutils) 2.38
Copyright (C) 2022 Free Software Foundation, Inc.
This program is free software; you may redistribute it under the terms of
the GNU General Public License version 3 or (at your option) any later version.
This program has absolutely no warranty.
EOF
exit 0
elif [ "$1" = "--help" ]; then
"$AR_X86_64" --help |
sed "s!$AR_X86_64!$PROG!g" 2>/dev/null ||
"$AR_X86_64" --help
exit
fi
FIRST=1
STATE=0
OUTPUT_X86_64=
OUTPUT_AARCH64=
INPUTS=
for x; do
if [ $FIRST -eq 1 ]; then
set --
FIRST=0
fi
if [ $STATE -eq 0 ]; then
if [ x"$x" != x"${x#--*}" ]; then # startswith(x, "--")
set -- "$@" "$x" # this is a flag
else
set -- "$@" "$x" # command argument, e.g. rcu, rcsD
STATE=1
fi
elif [ $STATE -eq 1 ]; then
OUTPUT_X86_64=$x
STATE=2
elif [ x"$x" != x"${x#* }" ]; then
fatal_error "input arguments containing spaces unsupported"
elif [ x"$x" != x"${x#@}" ]; then
fatal_error "input argument @file not supported yet"
elif [ -z "$INPUTS" ]; then
INPUTS=$x
else
INPUTS="$INPUTS $x"
fi
done
if [ -z "$OUTPUT_X86_64" ]; then
fatal_error "missing output path"
fi
mangle_object_path() {
path=$1
arch=$2
outdir=${path%/*}
outbas=${path##*/}
if [ x"$outdir" = x"$path" ]; then
outdir=
elif [ -n "$outdir" ]; then
outdir="$outdir/"
fi
if [ ! -d "$outdir.$arch" ]; then
mkdir -p "$outdir.$arch" || Exit
fi
mangled_path="${outdir}.$arch/$outbas"
}
OBJECTS_X86_64=
OBJECTS_AARCH64=
for x in $INPUTS; do
if [ ! -f "$x" ]; then
fatal_error "$x: no such file"
fi
mangle_object_path "$x" aarch64
if [ ! -f "$mangled_path" ]; then
fatal_error "$x: missing concomitant $mangled_path file"
fi
OBJECTS_X86_64="${OBJECTS_X86_64} $x"
OBJECTS_AARCH64="${OBJECTS_AARCH64} $mangled_path"
done
mangle_object_path "$OUTPUT_X86_64" aarch64
OUTPUT_AARCH64="$mangled_path"
"$AR_X86_64" "$@" "$OUTPUT_X86_64" $OBJECTS_X86_64 &
pid1=$!
"$AR_AARCH64" "$@" "$OUTPUT_AARCH64" $OBJECTS_AARCH64 &
pid2=$!
if ! wait $pid1; then
kill $pid2 2>/dev/null
wait
exit 1
fi
wait $pid2 || exit
+1
View File
@@ -0,0 +1 @@
fatcosmocc
Executable
+650
View File
@@ -0,0 +1,650 @@
#!/bin/sh
#
# fat cosmopolitan c/c++ compiler
#
# - this command is a drop-in replacement for the cc or gcc command.
# the difference is that (1) your binaries will be linked with the
# cosmopolitan c library, rather than your system specific tooling
# and (2) they'll be fat ape executables that run on the platforms
#
# * amd64
# + linux
# + macos
# + windows
# + freebsd
# + openbsd
# + netbsd
# * arm64
# + linux
# + macos
# + windows (non-native)
#
# - you need to use linux to build your binaries currently, but you
# can scp and distribute the output files to the above platforms!
#
# installation
#
# sudo chmod 1777 /opt # sticky bit isn't required
# git clone https://github.com/jart/cosmopolitan /opt/cosmo
# export PATH="/opt/cosmo/bin:/opt/cosmos/bin:$PATH"
# echo 'export PATH="/opt/cosmo/bin:/opt/cosmos/bin:$PATH"' >>~/.profile
# ape-install # optionally install a faster systemwide ape loader
# fatcosmocc --update # pull and rebuild toolchain artifacts
#
# getting started
#
# fatcosmocc -o hello.com hello.c
# ./foo.com
# unzip -vl ./foo.com
# ./foo.com --strace
# ./foo.com --ftrace
#
# building in tiny mode
#
# export MODE=tiny
# fatcosmocc --update
# fatcosmocc -Os -o foo.com foo.c
#
# building in debug mode
#
# export MODE=dbg
# fatcosmocc --update
# fatcosmocc -g -o foo.com foo.c
#
# how to build a project like lua 5.4.6
#
# make all test CC=fatcosmocc AR='fatcosmoar rcu'
# make install INSTALL_TOP=/opt/cosmos INSTALL=fatcosmoinstall
#
# how to build a project like ncurses 6.4
#
# ./configure CC=fatcosmocc \
# CXX=fatcosmoc++ \
# AR=fatcosmoar \
# INSTALL="$(command -v fatcosmoinstall)" \
# --prefix=/opt/cosmos \
# --disable-shared
# make -j8
# make install
#
# detecting this environment
#
# - `__FATCOSMOCC__` is defined by fatcosmocc
# - `__COSMOCC__` is defined by cosmocc and fatcosmocc
# - `__COSMOPOLITAN__` is always defined by cosmopolitan
#
# some notes on this compiler
#
# - the underlying compiler itself is gcc
# - we use cosmopolitan libc rather than glibc
# - we use llvm's compiler-rt and libcxx runtimes
# - we patched gcc so switch case can have symbols
# - our scanf() implementation is somewhat troubled
# - you may need to recalibrate `make -jN` as `N/2`
#
# compiler flags that work differently
#
# - `-v` will log fatcosmocc subcommands to stderr
# you can also use `export BUILDLOG=/tmp/build.log`
# - `-s` will ask apelink to not embed symbol tables in zip
# - `-E` can't be fat and runs once with x86_64 macros undefined
# - `-save-temps` will prevent deleting your arch-specific executables
#
# compiler flags that aren't supported
#
# - `-fexceptions` cosmopolitan doesn't support c++ exceptions yet
# - `-frtti` cosmopolitan doesn't support c++ runtime reflection yet
# - `-mred-zone` the system v red zone doesn't exist on windows and metal
# - `-fpic`, '-fPIC', `-shared`, `-pie`, etc. no shared object support yet
# - `-fsanitize=thread` cosmopolitan doesn't have thread sanitizer runtime yet
# - `-fomit-frame-pointer` is partially supported (apple forbids full removal)
#
# for further details, run `man gcc`
PROG=${0##*/}
COSMO=${COSMO:-/opt/cosmo}
COSMOS=${COSMOS:-/opt/cosmos}
ORIGINAL="$0 $*"
TMPDIR=${TMPDIR:-/tmp}
GCC_VERSION=11.2.0
if [ "$1" = "--version" ]; then
cat <<EOF
$PROG (GCC) $GCC_VERSION
Copyright (c) 2023 Justine Alexandra Roberts Tunney
Cosmopolitan Libc and LLVM libcxx/compiler-rt are subject to non-GPL
notice licenses, e.g. ISC, MIT, etc. Your compiled programs must embed
our copyright notices. This toolchain is configured to do so default.
Cosmopolitan comes with absolutely NO WARRANTY of any kind.
For more information, see the Cosmopolitan LICENSE files.
Copyright (C) 2019 Free Software Foundation, Inc.
This launches GNU GCC/Binutils subprocesses, which is free software; see
Cosmopolitan's third_party/gcc/ for source code and copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.
EOF
exit
fi
if [ "$1" = "--help" ]; then
if [ -t 1 ]; then
exec less "$0"
else
exec cat "$0"
fi
fi
MODE=${MODE:-$m}
if [ x"$MODE" = x"" ]; then
MODE_AARCH64=aarch64
elif [ x"$MODE" = x"tiny" ]; then
MODE_AARCH64=aarch64-tiny
elif [ x"$MODE" = x"zero" ]; then
MODE_AARCH64=aarch64-zero
elif [ x"$MODE" = x"dbg" ]; then
MODE_AARCH64=aarch64-dbg
else
echo "$PROG: build MODE=$MODE not supported by fatcosmocc" >&2
exit 1
fi
if [ "$1" = "--update" ]; then
cd $COSMO || exit
echo "building cosmo host toolchain..." >&2
make --silent -j toolchain MODE= || exit
echo "building cosmo x86_64 target (MODE=$MODE) toolchain..." >&2
make --silent -j toolchain MODE="$MODE" || exit
echo "building cosmo aarch64 target (MODE=$MODE_AARCH64) toolchain..." >&2
make --silent -j toolchain MODE="$MODE_AARCH64" || exit
echo "setting up your cosmos..." >&2
for arch in "" .aarch64/; do
mkdir -p "$COSMOS/lib/$arch" || exit
for lib in c dl gcc_s m pthread resolv rt dl z stdc++; do
if [ ! -f "$COSMOS/lib/${arch}lib${lib}.a" ]; then
printf '\041\074\141\162\143\150\076\012' >"$COSMOS/lib/${arch}lib${lib}.a" || exit
fi
done
done
echo "successfully updated your cosmo toolchain" >&2
exit
fi
if [ ! -d "$COSMO" ]; then
echo "$PROG: you need to clone cosmopolitan to your $COSMO directory" >&2
exit 1
fi
if [ ! -f "$COSMOS/lib/libc.a" ] ||
[ ! -f "$COSMOS/lib/.aarch64/libc.a" ] ||
[ ! -f "$COSMO/o/$MODE/cosmopolitan.a" ] ||
[ ! -f "$COSMO/o/$MODE_AARCH64/cosmopolitan.a" ]; then
echo "$PROG: you need to run: $PROG --update" >&2
exit 1
fi
if [ -x "$COSMO/o/third_party/gcc/bin/x86_64-linux-musl-gcc" ]; then
BRAND=musl
else
BRAND=cosmo
fi
FIXUPOBJ="$COSMO/o//tool/build/fixupobj.com"
TEMP_FILES=
SAVE_TEMPS=0
Exit() {
rc=${1:-$?}
if [ $SAVE_TEMPS -eq 0 ]; then
rm -f $TEMP_FILES
fi
exit $rc
}
show_warning() {
echo "$PROG: warning: $1" >&2
}
fatal_error() {
echo "$PROG: fatal error: $1" >&2
echo "compilation terminated." >&2
Exit 1
}
log_original() {
if [ -n "$BUILDLOG" ]; then
printf '# %s\n' "$ORIGINAL" >>"$BUILDLOG"
fi
}
log_command() {
if [ -n "$BUILDLOG" ]; then
printf '(cd %s; %s)\n' "$PWD" "$*" >>"$BUILDLOG"
fi
}
if [ x"$TMPDIR" != x"${TMPDIR#* }" ]; then
fatal_error '$TMPDIR containing spaces not supported'
elif [ ! -d "$TMPDIR" ]; then
if ! mkdir -p "$TMPDIR" 2>/dev/null; then
fatal_error "$TMPDIR: not a directory"
fi
fi
OPT=
ARGS=
FLAGS=
OUTPUT=
MCOSMO=0
INTENT=ld
NEED_JOIN=
NEED_EQUAL=
NEED_OUTPUT=
APELINKFLAGS=
INPUT_FILE_COUNT=0
for x; do
if [ x"$x" != x"${x#* }" ]; then
fatal_error "arguments containing spaces unsupported: $x"
fi
if [ -n "$NEED_OUTPUT" ]; then
NEED_OUTPUT=
OUTPUT=$x
continue
elif [ -n "$NEED_JOIN" ]; then
x="${NEED_JOIN}${x}"
NEED_JOIN=
elif [ -n "$NEED_EQUAL" ]; then
x="${NEED_EQUAL}=${x}"
NEED_EQUAL=
elif [ x"$x" = x"-" ] || # is alias for stdin
[ x"$x" = x"${x#-*}" ]; then # !startswith(x, "-")
if [ x"$x" != x"${x%.s}" ] ||
[ x"$x" != x"${x%.S}" ]; then
fatal_error "$x: assembler input files not supported"
elif [ x"$x" != x"${x%.so}" ] ||
[ x"$x" != x"${x%.dll}" ] ||
[ x"$x" != x"${x%.dylib}" ]; then
fatal_error "$x: dynamic shared object input files not supported"
elif [ x"$x" != x"-" ] && [ ! -f "$x" ]; then
fatal_error "$x: no such file"
fi
INPUT_FILE_COUNT=$((INPUT_FILE_COUNT + 1))
ARGS="$ARGS $x" # don't add to $FLAGS array
continue
elif [ x"$x" = x"-o" ]; then
NEED_OUTPUT=1
continue
elif [ x"$x" != x"${x#-o}" ]; then # startswith(x, "-o")
OUTPUT=${x#-o}
continue
elif [ x"$x" != x"${x#-O}" ]; then # startswith(x, "-O")
OPT=$x
elif [ x"$x" = x"-c" ]; then
INTENT=cc
elif [ x"$x" = x"-E" ]; then
INTENT=cpp
elif [ x"$x" = x"-s" ]; then
APELINKFLAGS="$APELINKFLAGS -s"
continue
elif [ x"$x" = x"-v" ]; then
exec 3<&2 # dup2(2, 3) b/c stderr will be redirected later
BUILDLOG=/dev/fd/3
continue
elif [ x"$x" = x"-save-temps" ]; then
SAVE_TEMPS=1
elif [ x"$x" = x"-mcosmo" ]; then
MCOSMO=1
continue
elif [ x"$x" = x"-fomit-frame-pointer" ]; then
# Quoth Apple: "The frame pointer register must always address a
# valid frame record. Some functions — such as leaf functions or
# tail calls — may opt not to create an entry in this list. As a
# result, stack traces are always meaningful, even without debug
# information."
x="-momit-leaf-frame-pointer -foptimize-sibling-calls"
elif [ x"$x" = x"-r" ] ||
[ x"$x" = x"-S" ] ||
[ x"$x" = x"-pie" ] ||
[ x"$x" = x"-frtti" ] ||
[ x"$x" = x"-shared" ] ||
[ x"$x" = x"-nostdlib" ] ||
[ x"$x" = x"-mred-zone" ] ||
[ x"$x" = x"-fexceptions" ] ||
[ x"$x" = x"-fsanitize=thread" ]; then
fatal_error "$x flag not supported"
elif [ x"$x" = x"-fsanitize=all" ] ||
[ x"$x" = x"-fsanitize=address" ] ||
[ x"$x" = x"-fsanitize=undefined" ]; then
fatal_error "use cosmo MODE=dbg rather than passing $x"
elif [ x"$x" = x"-mno-red-zone" ]; then
# "Any memory below the stack beyond the red zone is considered
# volatile and may be modified by the operating system at any time."
# https://devblogs.microsoft.com/oldnewthing/20190111-00/?p=100685
continue
elif [ x"$x" = x"-fpic" ] || [ x"$x" = x"-fPIC" ]; then
# no support for building dynamic shared objects yet. reports
# indicate that ignoring these flags, helps let autoconf know
continue
elif [ x"$x" = x"-Werror" ] || \
[ x"$x" = x"-pedantic-errors" ]; then
# this toolchain is intended for building other people's code
# elevating warnings into errors, should only be done by devs
continue
elif [ x"$x" = x"-static-libgcc" ] || \
[ x"$x" = x"-shared-libgcc" ]; then
# cosmopolitan.a always has llvm compiler runtime static code
continue
elif [ x"$x" = x"-march=native" ]; then
fatal_error "-march=native can't be used when building fat binaries"
elif [ x"$x" = x"-dumpversion" ]; then
echo $GCC_VERSION
Exit 0
elif [ x"$x" = x"-e" ] ||
[ x"$x" = x"-z" ] ||
[ x"$x" = x"-T" ] ||
[ x"$x" = x"-L" ] ||
[ x"$x" = x"-I" ] ||
[ x"$x" = x"-D" ] ||
[ x"$x" = x"-U" ] ||
[ x"$x" = x"-iquote" ] ||
[ x"$x" = x"-isystem" ] ||
[ x"$x" = x"-include" ]; then
NEED_JOIN=$x
continue
elif [ x"$x" = x"--param" ]; then
NEED_EQUAL=$x
continue
fi
FLAGS="$FLAGS $x"
ARGS="$ARGS $x"
done
if [ $INPUT_FILE_COUNT -eq 0 ]; then
fatal_error "no input files"
elif [ -z "$INPUT" ] &&
[ $INTENT != ld ] &&
[ $INPUT_FILE_COUNT -gt 1 ]; then
fatal_error "cannot specify '-o' with '-c', or '-E' with multiple files"
fi
PLATFORM="-D__COSMOPOLITAN__ -D__COSMOCC__ -D__FATCOSMOCC__"
PREDEF="-include libc/integral/normalize.inc"
CPPFLAGS="-fno-pie -nostdinc -fno-math-errno -iquote $COSMO -isystem $COSMOS/include -isystem $COSMO/libc/isystem"
CFLAGS="-fportcosmo -fno-dwarf2-cfi-asm -fno-unwind-tables -fno-asynchronous-unwind-tables -fno-semantic-interposition"
LDFLAGS="-static -nostdlib -no-pie -fuse-ld=bfd -Wl,-z,norelro -Wl,--gc-sections"
PRECIOUS="-fno-omit-frame-pointer"
if [ x"$OPT" != x"-Os" ] && [ x"$MODE" != x"tiny" ]; then
CFLAGS="$CFLAGS -fno-optimize-sibling-calls -mno-omit-leaf-frame-pointer"
fi
CC_X86_64="$COSMO/o/third_party/gcc/bin/x86_64-linux-$BRAND-gcc"
CC_AARCH64="$COSMO/o/third_party/gcc/bin/aarch64-linux-$BRAND-gcc"
if [ x"$PROG" != x"${PROG%++}" ]; then
CC_X86_64="$COSMO/o/third_party/gcc/bin/x86_64-linux-$BRAND-g++"
CC_AARCH64="$COSMO/o/third_party/gcc/bin/aarch64-linux-$BRAND-g++"
CFLAGS="$CFLAGS -fno-rtti -fno-exceptions -fuse-cxa-atexit"
fi
CRT_X86_64="$COSMO/o/$MODE/ape/ape.o $COSMO/o/$MODE/libc/crt/crt.o"
CPPFLAGS_X86_64="$CPPFLAGS -mno-red-zone"
CFLAGS_X86_64="$CFLAGS -mno-tls-direct-seg-refs"
LDFLAGS_X86_64="$LDFLAGS -L$COSMOS/lib -Wl,-T,$COSMO/o/$MODE/ape/ape.lds -Wl,-z,common-page-size=4096 -Wl,-z,max-page-size=16384"
LDLIBS_X86_64="$COSMO/o/$MODE/cosmopolitan.a"
if [ $MCOSMO -eq 1 ]; then
CPPFLAGS_X86_64="${CPPFLAGS_X86_64} -D_COSMO_SOURCE"
fi
CRT_AARCH64="$COSMO/o/$MODE_AARCH64/libc/crt/crt.o"
CPPFLAGS_AARCH64="$CPPFLAGS"
CFLAGS_AARCH64="$CFLAGS -ffixed-x18 -ffixed-x28 -mno-outline-atomics"
LDFLAGS_AARCH64="$LDFLAGS -L$COSMOS/lib/.aarch64 -Wl,-T,$COSMO/o/${MODE_AARCH64}/ape/aarch64.lds -Wl,-z,common-page-size=16384 -Wl,-z,max-page-size=16384"
LDLIBS_AARCH64="$COSMO/o/${MODE_AARCH64}/cosmopolitan.a"
if [ x"$OPT" != x"-Os" ] && [ x"$MODE" != x"tiny" ]; then
CFLAGS_X86_64="${CFLAGS_X86_64} -fpatchable-function-entry=18,16"
CFLAGS_AARCH64="${CFLAGS_AARCH64} -fpatchable-function-entry=7,6"
fi
if [ x"$PROG" != x"${PROG%++}" ]; then
LDLIBS_X86_64="$COSMO/o/$MODE/third_party/libcxx/libcxx.a ${LDLIBS_X86_64}"
LDLIBS_AARCH64="$COSMO/o/${MODE_AARCH64}/third_party/libcxx/libcxx.a ${LDLIBS_AARCH64}"
fi
if [ x"$MODE" = x"dbg" ]; then
CPPFLAGS_X86_64="${CPPFLAGS_X86_64} -fsanitize=address -fsanitize=undefined"
CPPFLAGS_AARCH64="${CPPFLAGS_AARCH64} -fsanitize=undefined"
fi
log_original
if [ $INTENT = cpp ]; then
if [ -n "$OUTPUT" ]; then
ARGS="$ARGS -o$OUTPUT"
fi
set -- \
"$CC_X86_64" \
-U__k8 \
-U__k8__ \
-U__amd64 \
-U__amd64__ \
-U__x86_64 \
-U__x86_64__ \
-U__SSE__ \
-U__SSE2__ \
-U__SSE2_MATH__ \
-mno-red-zone \
$PLATFORM \
$CPPFLAGS \
$ARGS
log_command "$@"
MODE="$MODE" exec "$@"
fi
mangle_object_path() {
path=$1
arch=$2
outdir=${path%/*}
outbas=${path##*/}
if [ x"$outdir" = x"$path" ]; then
outdir=
elif [ -n "$outdir" ]; then
outdir="$outdir/"
fi
if [ ! -d "$outdir.$arch" ]; then
mkdir -p "$outdir.$arch" || Exit
fi
mangled_path="${outdir}.$arch/$outbas"
}
mktemper() {
"$COSMO/o//tool/build/mktemper.com" \
"$TMPDIR/fatcosmocc.XXXXXXXXXXXXX$1"
}
build_object() {
out2=$(mktemper .txt) || Exit
TEMP_FILES="${TEMP_FILES} $out2"
(
set -- \
"$CC_X86_64" \
-o"$OUTPUT_X86_64" \
$PLATFORM \
$PREDEF \
$CFLAGS_X86_64 \
$CPPFLAGS_X86_64 \
"$@" \
$PRECIOUS
log_command "$@"
"$@" || exit
set -- \
"$COSMO/o//tool/build/fixupobj.com" \
"$OUTPUT_X86_64"
log_command "$@"
exec "$@"
) &
pid1=$!
(
set -- \
"$CC_AARCH64" \
-o"$OUTPUT_AARCH64" \
$PLATFORM \
$PREDEF \
$CFLAGS_AARCH64 \
$CPPFLAGS_AARCH64 \
"$@" \
$PRECIOUS &&
log_command "$@"
"$@" || exit
set -- \
"$COSMO/o//tool/build/fixupobj.com" \
"$OUTPUT_AARCH64"
log_command "$@"
exec "$@"
) 2>"$out2" &
pid2=$!
if ! wait $pid1; then
kill $pid2 2>/dev/null
wait
Exit 1
fi
if ! wait $pid2; then
echo "$PROG: x86_64 succeeded but aarch64 failed to build object" >&2
cat "$out2" >&2
Exit 1
fi
}
# turn source files into objects
LDARGS_X86_64=
LDARGS_AARCH64=
for x in $ARGS; do
if [ x"$x" != x"-" ] && # is alias for stdin
[ x"$x" != x"${x#-*}" ]; then # startswith(x, "-")
# this argument is a flag
LDARGS_X86_64="${LDARGS_X86_64} $x"
if [ x"$x" != x"${x#-L}" ]; then # startswith(x, "-L")
x="$x/.aarch64"
fi
LDARGS_AARCH64="${LDARGS_AARCH64} $x"
else
# this argument is an input file
if [ x"$x" != x"${x%.o}" ] ||
[ x"$x" != x"${x%.a}" ]; then
if [ $INTENT = cc ]; then
show_warning "$x: linker input file unused because linking not done"
else
mangle_object_path "$x" aarch64
if [ ! -f "$mangled_path" ]; then
fatal_error "$x: linker input missing concomitant $mangled_path file"
fi
LDARGS_X86_64="${LDARGS_X86_64} $x"
LDARGS_AARCH64="${LDARGS_AARCH64} $mangled_path"
fi
elif [ $INTENT = cc ]; then
if [ -n "$OUTPUT" ]; then
# e.g. `cc -c -o bar.o foo.c` is specified by user
OUTPUT_X86_64=$OUTPUT
mangle_object_path "$OUTPUT" aarch64
OUTPUT_AARCH64="$mangled_path"
build_object $FLAGS -c "$x"
else
# e.g. `cc -c dir/foo.c` builds foo.o
o=${x##*/}
OUTPUT_X86_64="${o%.*}.o"
mangle_object_path "${o%.*}.o" aarch64
OUTPUT_AARCH64="$mangled_path"
build_object $FLAGS -c "$x"
fi
else
# e.g. `cc foo.c` should build a.out
if [ -z "$OUTPUT" ]; then
OUTPUT=a.out
fi
# e.g. `cc -o foo foo.c` should *not* build foo.o
OUTPUT_X86_64=$(mktemper .o) || Exit
OUTPUT_AARCH64=$(mktemper .o) || Exit
TEMP_FILES="${TEMP_FILES} ${OUTPUT_X86_64} ${OUTPUT_AARCH64}"
build_object $FLAGS -c "$x"
LDARGS_X86_64="${LDARGS_X86_64} ${OUTPUT_X86_64}"
LDARGS_AARCH64="${LDARGS_AARCH64} ${OUTPUT_AARCH64}"
fi
fi
done
if [ $INTENT != ld ]; then
Exit
fi
OUTPUT_X86_64=$(mktemper ".com.dbg") || Exit
OUTPUT_AARCH64=$(mktemper ".aarch64.elf") || Exit
out2=$(mktemper .txt) || Exit
TEMP_FILES="${TEMP_FILES} $out2"
(
set -- \
"$CC_X86_64" \
-o"$OUTPUT_X86_64"\
$CRT_X86_64 \
$LDFLAGS_X86_64 \
$LDARGS_X86_64 \
$LDLIBS_X86_64
log_command "$@"
"$@" || exit
set -- \
"$COSMO/o//tool/build/fixupobj.com" \
"$OUTPUT_X86_64"
log_command "$@"
exec "$@"
) &
pid1=$!
(
set -- \
"$CC_AARCH64" \
-o"$OUTPUT_AARCH64"\
$CRT_AARCH64 \
$LDFLAGS_AARCH64 \
$LDARGS_AARCH64 \
$LDLIBS_AARCH64
log_command "$@"
"$@" || exit
set -- \
"$COSMO/o//tool/build/fixupobj.com" \
"$OUTPUT_AARCH64"
log_command "$@"
exec "$@"
) 2>"$out2" &
pid2=$!
if ! wait $pid1; then
kill $pid2 2>/dev/null
wait
Exit 1
fi
if ! wait $pid2; then
echo "$PROG: x86_64 succeeded but aarch64 failed to link executable" >&2
cat "$out2" >&2
Exit 1
fi
set -- \
"$COSMO/o//tool/build/apelink.com" \
-l "$COSMO/o/$MODE/ape/ape.elf" \
-l "$COSMO/o/$MODE_AARCH64/ape/ape.elf" \
-M "$COSMO/ape/ape-m1.c" \
-o "$OUTPUT" \
$APELINKFLAGS \
"$OUTPUT_X86_64" \
"$OUTPUT_AARCH64"
log_command "$@"
"$@" || Exit
set -- \
"$COSMO/o//tool/build/pecheck.com" "$OUTPUT"
log_command "$@"
"$@" || Exit
if [ $INTENT = ld ] && [ $SAVE_TEMPS -eq 0 ]; then
mv -f "$OUTPUT_X86_64" "${OUTPUT%.com}.com.dbg" || Exit
mv -f "$OUTPUT_AARCH64" "${OUTPUT%.com}.aarch64.elf" || Exit
fi
Exit
+100
View File
@@ -0,0 +1,100 @@
#!/bin/sh
# the `install -s` strip flag provided by gnu coreutils surprisngly
# enough understands the portable executable format however it will
# silently corrupt any actually portable executable it touches, due
# to how it overwrites the whole mz header and removes the dos stub
FIRST=1
for x; do
if [ $FIRST -eq 1 ]; then
set --
FIRST=0
fi
if [ x"$x" = x"-s" ]; then
continue
fi
set -- "$@" "$x"
done
# now magically copy multi-architecture build artifacts
if [ $# -gt 1 ]; then
# parse `install [$flags]... $src $dst`
i=0
n=$(( $# - 2 ))
dst=
src=
flags=
for x; do
if [ x"$x" != x"${x#* }" ]; then
# give up if any arguments contain spaces
exec install "$@"
fi
src=$dst
dst=$x
if [ $i -lt $n ]; then
flags="$flags $x"
fi
i=$(( i + 1 ))
done
arch=aarch64
# turn:
# install -c -m 644 foo.a /usr/lib/foo.a
# into:
# install -c -m 644 foo.a /usr/lib/foo.a
# install -c -m 644 .aarch/foo.a /usr/lib/.aarch/foo.a
if [ x"${dst##*/}" = x"${src##*/}" ] && # basenames are equal
[ x"$dst" != x"${dst%.a}" ]; then # specifies static archive
srcdir=${src%/*}
srcbas=${dst##*/}
if [ x"$srcdir" = x"$src" ]; then
srcdir=
elif [ -n "$srcdir" ]; then
srcdir="$srcdir/"
fi
if [ -f "$srcdir.$arch/$srcbas" ]; then
dstdir=${dst%/*}
dstbas=${dst##*/}
if [ x"$dstdir" = x"$dst" ]; then
dstdir=
elif [ -n "$dstdir" ]; then
dstdir="$dstdir/"
fi
if [ ! -d "$dstdir.$arch" ]; then
mkdir -p "$dstdir.$arch" || exit
fi
install $flags $src $dst || exit
exec install $flags \
"$srcdir.$arch/$srcbas" \
"$dstdir.$arch/$dstbas"
fi
fi
# turn:
# install -c -m 644 foo.a /usr/lib
# into:
# install -c -m 644 foo.a /usr/lib
# install -c -m 644 .aarch/foo.a /usr/lib/.aarch
arch=aarch64
if [ x"$src" != x"${src%.a}" ] && [ -d "$dst" ]; then
srcdir=${src%/*}
srcbas=${dst##*/}
if [ x"$srcdir" = x"$src" ]; then
srcdir=
elif [ -n "$srcdir" ]; then
srcdir="$srcdir/"
fi
if [ -f "$srcdir.$arch/$srcbas" ]; then
if [ ! -d "$dst/.$arch" ]; then
mkdir -p "$dst/.$arch" || exit
fi
install $flags $src $dst || exit
exec install $flags "$srcdir.$arch/$srcbas" "$dst/.$arch"
fi
fi
fi
exec install "$@"
+1
View File
@@ -0,0 +1 @@
fatcosmoar
+1
View File
@@ -0,0 +1 @@
fatcosmocc
+1
View File
@@ -0,0 +1 @@
fatcosmocc
+1
View File
@@ -0,0 +1 @@
fatcosmoinstall
+19
View File
@@ -0,0 +1,19 @@
#!/bin/sh
MODE=${MODE:-$m}
COSMO=${COSMO:-/opt/cosmo}
COSMOS=${COSMOS:-/opt/cosmos}
if [ -x "$COSMO/o/third_party/gcc/bin/x86_64-linux-musl-gcc" ]; then
BRAND=musl
else
BRAND=cosmo
fi
TOOL="$COSMO/o/third_party/gcc/bin/x86_64-linux-$BRAND-addr2line"
if [ ! -x "$TOOL" ]; then
echo "$0: you need to run: x86_64-unknown-cosmo-cc --update" >&2
exit 1
fi
exec "$TOOL" "$@"
+1
View File
@@ -0,0 +1 @@
apecopy
+22
View File
@@ -0,0 +1,22 @@
#!/bin/sh
PROG=${0##*/}
MODE=${MODE:-$m}
COSMO=${COSMO:-/opt/cosmo}
fatal_error() {
echo "$PROG: $1" >&2
exit 1
}
if [ -x "$COSMO/o/third_party/gcc/bin/x86_64-linux-musl-gcc" ]; then
BRAND=musl
else
BRAND=cosmo
fi
if [ ! -x "$COSMO/o/third_party/gcc/bin/x86_64-linux-$BRAND-ar" ]; then
fatal_error "you need to run: x86_64-unknown-cosmo-cc --update"
fi
exec "$COSMO/o/third_party/gcc/bin/x86_64-linux-$BRAND-ar" "$@"
+19
View File
@@ -0,0 +1,19 @@
#!/bin/sh
MODE=${MODE:-$m}
COSMO=${COSMO:-/opt/cosmo}
COSMOS=${COSMOS:-/opt/cosmos}
if [ -x "$COSMO/o/third_party/gcc/bin/x86_64-linux-musl-gcc" ]; then
BRAND=musl
else
BRAND=cosmo
fi
TOOL="$COSMO/o/third_party/gcc/bin/x86_64-linux-$BRAND-as"
if [ ! -x "$TOOL" ]; then
echo "$0: you need to run: x86_64-unknown-cosmo-cc --update" >&2
exit 1
fi
exec "$TOOL" "$@"
+1
View File
@@ -0,0 +1 @@
cosmocc
+1
View File
@@ -0,0 +1 @@
cosmocc
+15
View File
@@ -0,0 +1,15 @@
#!/bin/sh
FIRST=1
for x; do
if [ $FIRST -eq 1 ]; then
set --
FIRST=0
fi
if [ x"$x" = x"-s" ]; then
continue
fi
set -- "$@" "$x"
done
exec install "$@"
+19
View File
@@ -0,0 +1,19 @@
#!/bin/sh
MODE=${MODE:-$m}
COSMO=${COSMO:-/opt/cosmo}
COSMOS=${COSMOS:-/opt/cosmos}
if [ -x "$COSMO/o/third_party/gcc/bin/x86_64-linux-musl-gcc" ]; then
BRAND=musl
else
BRAND=cosmo
fi
TOOL="$COSMO/o/third_party/gcc/bin/x86_64-linux-$BRAND-nm"
if [ ! -x "$TOOL" ]; then
echo "$0: you need to run: x86_64-unknown-cosmo-cc --update" >&2
exit 1
fi
exec "$TOOL" "$@"
+19
View File
@@ -0,0 +1,19 @@
#!/bin/sh
MODE=${MODE:-$m}
COSMO=${COSMO:-/opt/cosmo}
COSMOS=${COSMOS:-/opt/cosmos}
if [ -x "$COSMO/o/third_party/gcc/bin/x86_64-linux-musl-gcc" ]; then
BRAND=musl
else
BRAND=cosmo
fi
TOOL="$COSMO/o/third_party/gcc/bin/x86_64-linux-$BRAND-objcopy"
if [ ! -x "$TOOL" ]; then
echo "$0: you need to run: x86_64-unknown-cosmo-cc --update" >&2
exit 1
fi
exec "$TOOL" "$@"
+19
View File
@@ -0,0 +1,19 @@
#!/bin/sh
MODE=${MODE:-$m}
COSMO=${COSMO:-/opt/cosmo}
COSMOS=${COSMOS:-/opt/cosmos}
if [ -x "$COSMO/o/third_party/gcc/bin/x86_64-linux-musl-gcc" ]; then
BRAND=musl
else
BRAND=cosmo
fi
TOOL="$COSMO/o/third_party/gcc/bin/x86_64-linux-$BRAND-objdump"
if [ ! -x "$TOOL" ]; then
echo "$0: you need to run: x86_64-unknown-cosmo-cc --update" >&2
exit 1
fi
exec "$TOOL" "$@"
+19
View File
@@ -0,0 +1,19 @@
#!/bin/sh
MODE=${MODE:-$m}
COSMO=${COSMO:-/opt/cosmo}
COSMOS=${COSMOS:-/opt/cosmos}
if [ -x "$COSMO/o/third_party/gcc/bin/x86_64-linux-musl-gcc" ]; then
BRAND=musl
else
BRAND=cosmo
fi
TOOL="$COSMO/o/third_party/gcc/bin/x86_64-linux-$BRAND-strip"
if [ ! -x "$TOOL" ]; then
echo "$0: you need to run: x86_64-unknown-cosmo-cc --update" >&2
exit 1
fi
exec "$TOOL" "$@"
BIN
View File
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
BIN
View File
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
BIN
View File
Binary file not shown.
Binary file not shown.
BIN
View File
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
BIN
View File
Binary file not shown.
Loaded 100 of 16338 files, more files were not shown because too many files have changed in this diff. Show more