Compare commits

...
999 Commits
Author SHA1 Message Date
Justine Tunney 8621034d42 Release Cosmopolitan v3.6.2 2024-07-27 20:20:54 -07:00
Justine Tunney 32292aee84 Fix build determinism issue 2024-07-27 19:03:32 -07:00
Justine Tunney 81dd9639ed Fix bug in tool/decode/zip 2024-07-27 18:34:21 -07:00
Justine Tunney f147d3dde9 Fix some static analysis issues 2024-07-27 09:16:54 -07:00
Justine Tunney fb54604b31 Upgrade to superconfigure z0.0.48
Our cosmocc binaries are now built with GCC 14.1, using the Cosmo commit
efb3a34608 from yesterday.

GCC is now configured using --enable-analyzer so you can use -fanalyzer.
2024-07-27 08:35:36 -07:00
Justine Tunney cdfcee51ca Properly serialize fork() operations
This change solves an issue where many threads attempting to spawn forks
at once would cause fork() performance to degrade with the thread count.
Things got real nasty on NetBSD, which slowed down the whole test fleet,
because there's no vfork() and we're forced to use fork() in our server.

   threads      count task
         1       1062 fork+exit+wait
         2        668 fork+exit+wait
         4         66 fork+exit+wait
         8         19 fork+exit+wait
        16         22 fork+exit+wait
        32         16 fork+exit+wait

Things are now much less bad on NetBSD, but not great, since it does not
have futexes; we rely on its semaphore file descriptors to do conditions

   threads      count task
         1       1085 fork+exit+wait
         2        842 fork+exit+wait
         4        532 fork+exit+wait
         8        400 fork+exit+wait
        16        276 fork+exit+wait
        32         66 fork+exit+wait

With OpenBSD which also lacks vfork(), things were just as bad as NetBSD

   threads      count task
         1        584 fork+exit+wait
         2        687 fork+exit+wait
         4        206 fork+exit+wait
         8         24 fork+exit+wait
        16         33 fork+exit+wait
        32         26 fork+exit+wait

But since OpenBSD has futexes fork() works terrifically thanks to *NSYNC

   threads      count task
         1        525 fork+exit+wait
         2        580 fork+exit+wait
         4        451 fork+exit+wait
         8        479 fork+exit+wait
        16        408 fork+exit+wait
        32        373 fork+exit+wait

This issue would most likely only manifest itself, when pthread_atfork()
callers manage to slip a spin lock into the outermost position of fork's
list of locks. Since fork() is very slow, a spin lock can be devastating

Needless to say vfork() rules and anyone who says differently is kidding
themselves. Look at what a FreeBSD 14.1 virtual machine with equal specs
can do over the course of three hundred milliseconds.

   threads      count task
         1       2559 vfork+exit+wait
         2       5389 vfork+exit+wait
         4      34933 vfork+exit+wait
         8      43273 vfork+exit+wait
        16      49648 vfork+exit+wait
        32      40247 vfork+exit+wait

So it's a shame that so few OSes support vfork(). It creates an unsavory
situation, where someone wanting to build a server that spawns processes
would be better served to not use threads and favor a multiprocess model
2024-07-27 08:23:44 -07:00
Justine Tunney 18a620cc1a Make some improvements of little consequence 2024-07-27 08:20:18 -07:00
Justine Tunney 690d3df66e Expand the virtual address space on Windows 2024-07-27 08:19:05 -07:00
Justine Tunney efb3a34608 Fix package.sh build error 2024-07-26 06:57:24 -07:00
Justine Tunney 642e9cb91a Introduce cosmocc flags -mdbg -mtiny -moptlinux
The cosmocc.zip toolchain will now include four builds of the libcosmo.a
runtime libraries. You can pass the -mdbg flag if you want to debug your
cosmopolitan runtime. You can pass the -moptlinux flag if you don't want
windows code lurking in your binary. See tool/cosmocc/README.md for more
details on how these flags may be used and their important implications.
2024-07-26 05:10:25 -07:00
Justine Tunney 59692b0882 Make spinlocks faster (take two)
This change is green on x86 and arm test fleet.
2024-07-26 00:45:24 -07:00
Justine Tunney 02e1cbcd00 Revert "Make spin locks go faster"
This reverts commit c8e25d811c.
2024-07-25 22:24:32 -07:00
Justine Tunney 0679cfeb41 Fix build 2024-07-25 22:12:08 -07:00
Justine Tunney edd5e2c8e3 Expose gethostbyname() 2024-07-25 19:21:35 -07:00
Justine Tunney c8e25d811c Make spin locks go faster 2024-07-25 17:37:11 -07:00
Justine Tunney a31d5ea399 Remove cosmoc++ compiler warning 2024-07-25 15:20:41 -07:00
Justine Tunney 7d88343973 Release Cosmopolitan v3.6.1 2024-07-25 13:34:02 -07:00
Justine Tunney 3915ca0f71 Have cosmocc define -Wno-implicit-int
GCC 13+ changed its policies to be very aggressive about breaking builds
that have anything resembling K&R C. I very strongly disagree with these
decisions. Users who think their compiler should should also be a linter
are perfectly welcome to opt-in to -Wimplicit-int.
2024-07-25 05:51:57 -07:00
Justine Tunney 2c4b88753b Add special errno handling to libcxx 2024-07-25 01:23:02 -07:00
Justine Tunney 0f486a13c8 Fix fdlibm license 2024-07-24 20:42:08 -07:00
Justine Tunney 1020dd41cc bzero() should be defined without special defines 2024-07-24 16:15:30 -07:00
Justine Tunney d3a13e8d70 Improve lock hierarchy
- NetBSD no longer needs a spin lock to create semaphores
- Windows fork() now locks process manager in correct order
2024-07-24 16:05:48 -07:00
Justine Tunney 7ba9a73840 Remove more _Atomic keywords from public headers
It's been thirteen years and C++ still hasn't implemented this wonderful
simple builtin keyword. In C++23 a solution was provided for making this
work in C++ which is libcxx's stdatomic.h. Including that header schleps
in literally 253 unique header files!! Many of the header files it needs
are libc header files like pthread.h where we need to have the _Atomic()
keyword, but since <atomic> depends on pthreads we can't have it include
the <stdatomic.h> header that defines _Atomic for C++ users, and instead
we simply make the type non-atomic, hoping and praying only C code shall
use those internal data structures. This just shows how STL clowns can't
be trusted to define the innermost primitives of a language. They should
instead be focusing on being the best at algorithms and data structures.
2024-07-24 13:56:03 -07:00
Justine Tunney 4a1ae86124 Make some code build faster 2024-07-24 12:26:52 -07:00
Justine Tunney 5dd7ddb9ea Remove bad defines from early days of project
These definitions were causing issues with building LLVM. It is possible
they also caused crashes we've seen with our MacOS ARM64 OpenMP support.
2024-07-24 12:11:21 -07:00
Justine Tunney f25fbbaaeb Use libcxx abi v1 2024-07-24 09:49:48 -07:00
Justine Tunney fbc4b03d4c Restore support for AMD K8 2024-07-24 08:59:29 -07:00
Justine Tunney e398f3887c Make more improvements to threads and mappings
- NetBSD should now have faster synchronization
- POSIX barriers may now be shared across processes
- An edge case with memory map tracking has been fixed
- Grand Central Dispatch is no longer used on MacOS ARM64
- POSIX mutexes in normal mode now use futexes across processes
2024-07-24 01:19:54 -07:00
Justine Tunney 2187d6d2dd Fix MODE=optlinux for GitHub Actions 2024-07-23 04:17:22 -07:00
Justine Tunney 0602ff6bab Fix MODE=optlinux and MODE=tiny builds 2024-07-23 04:04:19 -07:00
Justine Tunney 5660ec4741 Release Cosmopolitan v3.6.0
This release is an atomic upgrade to GCC 14.1.0 with C23 and C++23
2024-07-23 03:28:19 -07:00
Justine Tunney 62ace3623a Release Cosmopolitan v3.5.9 2024-07-22 21:02:40 -07:00
Justine Tunney 6e809ee49b Add unit test for process shared conditions 2024-07-22 18:48:54 -07:00
Justine Tunney 61c36c1dd6 Allow pthread_condattr_setpshared() to set shared 2024-07-22 18:41:45 -07:00
Justine Tunney 0a9a6f86bb Support process shared condition variables 2024-07-22 16:35:29 -07:00
Justine Tunney 3de6632be6 Graduate some clock_gettime() constants to #define
- CLOCK_THREAD_CPUTIME_ID
- CLOCK_PROCESS_CPUTIME_ID

Cosmo now supports the above constants universally across supported OSes
therefore it's now safe to let programs detect their presence w/ #ifdefs
2024-07-22 07:14:35 -07:00
Justine Tunney c83ec5fdd9 Fix more issues 2024-07-22 02:28:41 -07:00
Justine Tunney 23611cd854 Talk more about alignment 2024-07-22 02:11:09 -07:00
Justine Tunney 62a97c919f Fix typos in APE specification
Fixes #1244
2024-07-22 01:41:44 -07:00
Justine Tunney 16d244614e Mention red zone in APE ABI specification 2024-07-21 21:18:12 -07:00
Justine Tunney 41cc053419 Add more content to APE specification 2024-07-21 20:47:24 -07:00
Justine Tunney 5d2d9e9640 Add back missing TlsAlloc() call
Cosmopolitan Libc once called this important function although somewhere
along the way, possibly in a refactoring, it got removed and __tls_alloc
has always been zero ever since.
2024-07-21 20:45:27 -07:00
Justine Tunney e08a4cd99e Release Cosmopolitan v3.5.8 2024-07-21 17:01:33 -07:00
Justine Tunney 7ebaff34c6 Fix ctype.h and wctype.h 2024-07-21 15:54:17 -07:00
Justine Tunney e7be5a5e2b Upgrade to latest superconfigure 2024-07-21 15:30:47 -07:00
Justine Tunney 30afd6ddbb Improve multithreading 2024-07-21 14:40:45 -07:00
Justine Tunney d3167126aa Fix regression with last commit 2024-07-20 16:43:48 -07:00
Justine Tunney d3f87f4c64 Upgrade to cosmocc v3.5.7 2024-07-20 11:21:26 -07:00
Justine Tunney 29ce25c767 Start writing formal specification for APE 2024-07-20 10:04:22 -07:00
Justine Tunney 7996bf67b5 Release Cosmopolitan v3.5.7 2024-07-20 03:48:57 -07:00
Justine Tunney 626a5d02ee Add missing lock statement 2024-07-20 03:47:22 -07:00
Justine Tunney 527aaa41eb Prevent MODE=tiny ShowCrashReports() looping 2024-07-20 03:34:37 -07:00
Justine Tunney 421a819d88 Fix bug in munmap_test 2024-07-20 03:23:37 -07:00
Justine Tunney 3374cbba73 Release Cosmopolitan v3.5.6 2024-07-20 02:43:10 -07:00
Justine Tunney 2018cac11f Use better memory strategy on Windows
Rather than using the the rollo global to pick addresses, we select them
randomly now using a conservative vaspace.
2024-07-20 02:20:03 -07:00
Justine Tunney 6a5d4ed65b Fix bug with disabling sigaltstack() 2024-07-20 01:00:16 -07:00
Justine Tunney 493ffc9b7f Release Cosmopolitan v3.5.5 2024-07-19 22:33:17 -07:00
Justine Tunney 101fb3d9b3 Make some new Windows 10 memory APIs available 2024-07-19 22:26:49 -07:00
Justine Tunney f6ba270ff3 Add last commit to .git-blame-ignore-revs 2024-07-19 19:38:53 -07:00
Justine Tunney 86d884cce2 Get rid of .internal.h convention in LIBC_INTRIN 2024-07-19 19:38:00 -07:00
Justine Tunney 0ed916ad5c Fix a bug in example code 2024-07-19 19:11:28 -07:00
Justine Tunney 1029dcc597 Reduce default stack size from 256kb to 81kb
This is the same as Musl Libc. Please note it only applies to threads.
2024-07-19 14:18:06 -07:00
Ikko Eltociear Ashimine c697133a2d Fix typo in accept4-sysv.c (#1235) 2024-07-19 05:46:29 -07:00
Justine Tunney 1ff037df3c Add some documentation 2024-07-19 04:46:26 -07:00
Justine Tunney 567d8fe32d Create variables for page size 2024-07-18 21:16:53 -07:00
Justine Tunney 23dfb79d33 Fix minor suboptimalities in memory manager 2024-07-18 19:19:51 -07:00
Justine Tunney 76cea6c687 Squeeze more performance out of memory manager 2024-07-08 03:08:42 -07:00
Justine Tunney 63065cdd70 Make a test less intensive by default 2024-07-07 19:35:09 -07:00
Justine Tunney 3f2a1b696e Fix greenbean example
The memory leak detector was crashing. When using gc() you shouldn't use
the CheckForMemoryLeaks() function from inside the same function, due to
how it runs the atexit handlers.
2024-07-07 17:52:33 -07:00
Justine Tunney f590e96abd Work around QEMU bugs 2024-07-07 15:42:46 -07:00
Justine Tunney f7780de24b Make realloc() go 100x faster on Linux/NetBSD
Cosmopolitan now supports mremap(), which is only supported on Linux and
NetBSD. First, it allows memory mappings to be relocated without copying
them; this can dramatically speed up data structures like std::vector if
the array size grows larger than 256kb. The mremap() system call is also
10x faster than munmap() when shrinking large memory mappings.

There's now two functions, getpagesize() and getgransize() which help to
write portable code that uses mmap(MAP_FIXED). Alternative sysconf() may
be called with our new _SC_GRANSIZE. The madvise() system call now has a
better wrapper with improved documentation.
2024-07-07 12:40:30 -07:00
Justine Tunney 196942084b Recomment out accidental code 2024-07-06 19:57:47 -07:00
Justine Tunney 6be030cd7c Fix MODE=tinylinux build 2024-07-06 01:51:08 -07:00
Justine Tunney 8c645fa1ee Make mmap() scalable
It's now possible to create thousands of thousands of sparse independent
memory mappings, without any slowdown. The memory manager is better with
tracking memory protection now, particularly on Windows in a precise way
that can be restored during fork(). You now have the highest quality mem
manager possible. It's even better than some OSes like XNU, where mmap()
is implemented as an O(n) operation which means sadly things aren't much
improved over there. With this change the llamafile HTTP server endpoint
at /tokenize with a prompt of 50 tokens is now able to handle 2.6m r/sec
2024-07-05 23:26:00 -07:00
Justine Tunney 3756870635 Implement new red-black tree 2024-07-05 12:56:03 -07:00
Justine Tunney fc65422660 Remove __mmap() and __munmap() 2024-07-05 12:55:46 -07:00
Justine Tunney 01587de761 Simplify memory manager 2024-07-05 05:47:15 -07:00
Justine Tunney 5a9a08d1cf Fix regression in elf2pe program 2024-07-04 04:02:20 -07:00
Justine Tunney bd6d9ff99a Get deathstar demo working again on metal 2024-07-04 03:44:17 -07:00
Justine Tunney 15ea0524b3 Reduce code size of mandatory runtime
This change reduces o/tiny/examples/life from 44kb to 24kb in size since
it avoids linking mmap() when unnecessary. This is important, to helping
cosmo not completely lose touch with its roots.
2024-07-04 02:50:20 -07:00
Justine Tunney fdab49b30e Demonstrate signal safety of recursive mutexes 2024-07-04 02:47:52 -07:00
Justine Tunney 135d538b1d Make ctl::set use 30% less memory than libcxx 2024-07-04 02:46:27 -07:00
Justine Tunney 6dbc3fba18 Add AMD cache sizes to o//tool/viz/cpuid 2024-07-04 02:45:45 -07:00
Justine Tunney 70f77aad33 Release Cosmopolitan v3.5.4 2024-07-01 07:17:57 -07:00
Justine Tunney d0cd719375 Make more CTL fixes 2024-07-01 07:17:57 -07:00
Justine Tunney 61370983e1 Complete the Windows TLS fix made in e437bed00 2024-07-01 07:17:57 -07:00
Terror 72511ff0ac [Redbean] Add UuidV7 method (#1213)
To Complete #1140 add UUID version 7 to Redbean
2024-07-01 06:06:56 -07:00
Justine Tunney c1f8d0678c Mark ctl::to_string() noexcept 2024-07-01 05:54:59 -07:00
Justine Tunney e627bfa359 Introduce ctl::to_string() 2024-07-01 05:40:38 -07:00
Justine Tunney acbabedf27 Make CTL definitions less ambiguous 2024-07-01 03:48:28 -07:00
Justine Tunney 239f8ce76e Release Cosmopolitan v3.5.3 2024-07-01 02:07:56 -07:00
Justine Tunney ca4cf67eb8 Include more programs in cosmocc
The Cosmopolitan Compiler Collection now includes the following programs

- `ar.ape` is a faster alternative to `ar rcsD` for creating determistic
  static archives. It's ~10x faster than GNU because it isn't quadratic.
  It'll even outperform LLVM ar by 2x, thanks to writev/copy_file_range.

- `sha256sum.ape` is a faster alternative to the `sha256sum` command. It
  goes 2x faster since it leverages vectorized assembly implementations.

- `resymbol` is a brand new program we invented, like objcopy, that lets
  you rename all the global symbols in a .o file to have a new suffix or
  prefix. In the future, this will be used by cosmocc automatically when
  building -O3 math kernels, that need to be vectorized for all hardware

- `gzip.ape` is a faster version of the `gzip` command, that is included
  by most Linux distros. It gains better performance using Chromium Zlib
  which, once again, includes highly optimized assembly, that Mark Adler
  won't merge into the official MS-DOS compatible zlib codebase.

- `cocmd` is the cosmopolitan shell. It can function as a faster `sh -c`
  alternative than bash and dash as the `SHELL = /opt/cosmocc/bin/cocmd`
  at the top of your Makefile. Please note you should be using the cosmo
  fork of GNU make (already included), since normal make won't recognize
  this as a bourne-compatible shell and remove the execve() optimization
  which makes things slower. In some ways that's true. This doesn't have
  a complete POSIX shell implementation. However it's enough for cosmo's
  mono repo. It also implements faster behaviors in some respects.

The following programs are also introduced, which aren't as interesting.
The main reason why they're here is so Cosmopolitan's mono repo shall be
able to remove build/bootstrap/ in future editions. That way we can keep
build utilities better up to date, without bloating the git history much

- `chmod.ape` for hermeticity
- `cp.ape` for hermeticity
- `echo.ape` for hermeticity
- `objbincopy` is an objcopy-like tool that's used to build ape loader
- `package.ape` is used for strict dependency checking of object graph
- `rm.ape` for hermeticity
- `touch.ape` for hermeticity
2024-07-01 02:05:25 -07:00
Justine Tunney 78d3b86ec7 Fix Android support
Thanks to @aj47 (techfren.net) the new Cosmo memory manager is confirmed
to be working on Android!! The only issue turned out to be forgetting to
update the program address in the linker script. We now know w/ absolute
certainty that APE binaries as complex as llamafile, now work correctly.
2024-07-01 01:06:47 -07:00
Justine Tunney 44191b3f50 Add more type traits to CTL 2024-06-30 20:59:38 -07:00
Justine Tunney e437bed006 Fix crash caused when Windows needs a lot of TLS 2024-06-30 20:53:43 -07:00
Justine Tunney 76957983cf Make POSIX threads improvements
- Ensure SIGTHR isn't blocked in newly created threads
- Use TIB rather than thread_local for thread atexits
- Make POSIX thread keys atomic within thread
- Don't bother logging prctl() to --strace
- Log thread destructor names to --strace
2024-06-30 15:38:59 -07:00
Justine Tunney 387310c659 Fix issue with ctl::vector constructor 2024-06-30 02:26:38 -07:00
Justine Tunney 4cb5e21ba8 Introduce pthread_decimate_np() api
This is useful with CheckForMemoryLeaks().
2024-06-30 02:26:06 -07:00
Justine Tunney 1bf2d8e308 Further improve mmap() locking story
The way to use double linked lists, is to remove all the things you want
to work on, insert them into a new list on the stack. Then once you have
all the work items, you release the lock, do your work, and then lock it
again, to add the shelled out items back to a global freelist.
2024-06-29 17:12:43 -07:00
Justine Tunney 98e684622b Add iostream to CTL 2024-06-29 15:45:09 -07:00
Justine Tunney 617ddfee93 Release Cosmopolitan v3.5.2 2024-06-29 10:58:47 -07:00
Justine Tunney 464858dbb4 Fix bugs with new memory manager
This fixes a regression in mmap(MAP_FIXED) on Windows caused by a recent
revision. This change also fixes ZipOS so it no longer needs a MAP_FIXED
mapping to open files from the PKZIP store. The memory mapping mutex was
implemented incorrectly earlier which meant that ftrace and strace could
cause cause crashes. This lock and other recursive mutexes are rewritten
so that it should be provable that recursive mutexes in cosmopolitan are
asynchronous signal safe.
2024-06-29 10:53:57 -07:00
Justine Tunney 6de12c1032 Upgrade to superconfigure z0.0.44 2024-06-29 05:07:25 -07:00
Justine Tunney a16eb76f5e Fix build break 2024-06-29 04:34:27 -07:00
Justine Tunney 021c53ba32 Add more CTL content 2024-06-28 19:09:54 -07:00
Justine Tunney 38921dc46b Introduce more CTL content
This change introduces accumulate, addressof, advance, all_of, distance,
array, enable_if, allocator_traits, back_inserter, bad_alloc, is_signed,
any_of, copy, exception, fill, fill_n, is_same, is_same_v, out_of_range,
lexicographical_compare, is_integral, uninitialized_fill_n, is_unsigned,
numeric_limits, uninitialized_fill, iterator_traits, move_backward, min,
max, iterator_tag, move_iterator, reverse_iterator, uninitialized_move_n

This change experiments with rewriting the ctl::vector class to make the
CTL design more similar to the STL. So far it has not slowed things down
to have 42 #include lines rather than 2, since it's still almost nothing
compared to LLVM's code. In fact the closer we can flirt with being just
like libcxx, the better chance we might have of discovering exactly what
makes it so slow to compile. It would be an enormous discovery if we can
find one simple trick to solving the issue there instead.

This also fixes a bug in `ctl::string(const string &s)` when `s` is big.
2024-06-27 22:42:32 -07:00
Steven Dee (Jōshin) 054da021d0 ctl::string benchmarking code (#1200) 2024-06-26 21:30:05 -04:00
Justine Tunney 199662071a Make std::random_device use getentropy() 2024-06-24 07:32:07 -07:00
Justine Tunney 572ac7d100 Release Cosmopolitan v3.5.1 2024-06-24 06:54:15 -07:00
Justine Tunney d461c6f47d Do more quality assurance work 2024-06-24 06:53:49 -07:00
Justine Tunney 67b19ae733 Release Cosmopolitan v3.5.0 2024-06-23 22:45:14 -07:00
Justine Tunney c4c812c154 Introduce ctl::set and ctl::map
We now have a C++ red-black tree implementation that implements standard
template library compatible APIs while compiling 10x faster than libcxx.
It's not as beautiful as the red-black tree implementation in Plinko but
this will get the job done and the test proves it upholds all invariants

This change also restores CheckForMemoryLeaks() support and fixes a real
actual bug I discovered with Doug Lea's dlmalloc_inspect_all() function.
2024-06-23 22:27:11 -07:00
Justine Tunney 388e236360 Revert misguided dlmalloc optimization 2024-06-22 09:55:02 -07:00
Justine Tunney f2c8ddbbe3 Fix --strace use-after-free in pthread_join() 2024-06-22 06:05:52 -07:00
Justine Tunney d1d4388201 Delete ASAN
It hasn't been helpful enough to be justify the maintenance burden. What
actually does help is mprotect(), kprintf(), --ftrace and --strace which
can always be counted upon to work correctly. We aren't losing much with
this change. Support for ASAN on AARCH64 was never implemented. Applying
ASAN to the core libc runtimes was disabled many months ago. If there is
some way to have an ASAN runtime for user programs that is less invasive
we can potentially consider reintroducing support. But now is premature.
2024-06-22 05:45:49 -07:00
Justine Tunney 6ffed14b9c Rewrite memory manager
Actually Portable Executable now supports Android. Cosmo's old mmap code
required a 47 bit address space. The new implementation is very agnostic
and supports both smaller address spaces (e.g. embedded) and even modern
56-bit PML5T paging for x86 which finally came true on Zen4 Threadripper

Cosmopolitan no longer requires UNIX systems to observe the Windows 64kb
granularity; i.e. sysconf(_SC_PAGE_SIZE) will now report the host native
page size. This fixes a longstanding POSIX conformance issue, concerning
file mappings that overlap the end of file. Other aspects of conformance
have been improved too, such as the subtleties of address assignment and
and the various subtleties surrounding MAP_FIXED and MAP_FIXED_NOREPLACE

On Windows, mappings larger than 100 megabytes won't be broken down into
thousands of independent 64kb mappings. Support for MAP_STACK is removed
by this change; please use NewCosmoStack() instead.

Stack overflow avoidance is now being implemented using the POSIX thread
APIs. Please use GetStackBottom() and GetStackAddr(), instead of the old
error-prone GetStackAddr() and HaveStackMemory() APIs which are removed.
2024-06-22 05:45:11 -07:00
Justine Tunney 7f6d0b8709 Support -x LANG flag in cosmocc 2024-06-20 23:55:46 -07:00
Steven Dee (Jōshin) d7b1919b29 ctl::unique_ptr improvements and cleanup (#1221)
Explicitly value-initializes the deleter, even though I have not found a
way to get the deleter to act like it’s been default-initialized in unit
tests so far.

Uses auto in reset. The static cast is apparently not needed (unless I’m
missing some case I didn’t think of.)

Implements the general move constructor - turns out that the reason this
didn’t work before was that default_delete<U> was not move constructible
from default_delete<T>.

Drop inline specifiers from functions defined entirely inside the struct
definition since they are implicitly inline.

* Cleans up reset to match spec

Remove the variants from the T[] specialization. Also follow the spec on
the order of operations in reset, which may matter if we are deleting an
object that has a reference to the unique_ptr that is being reset. (?)

* Tests Base/Derived reset.

* Adds some constexpr declarations.

* Adds default_delete specialization for T[].

* Makes parameters const.
2024-06-20 18:44:31 -04:00
Steven Dee (Jōshin) f86e6f8eb0 Make new.cc definitions weak (#1233)
The STL says that these should be replaceable by user code.

new.cc now defines only a few direct functions (including a free wrapper
that perplexingly is needed since g++ didn’t want to alias "free".) Now,
all of the operators are weak references to those functions.
2024-06-20 15:20:54 -04:00
Steven Dee (Jōshin) 7e780e57d4 More ctl::string optimization (#1232)
Moves some isbig checks into string.h, enabling smarter optimizations to
be made on small strings. Also we no longer zero out our string prior to
calling the various constructors, buying back the performance we lost on
big strings when we made the small-string optimization. We further add a
little optimization to the big_string copy constructor: if the string is
using half or more of its capacity, then we don’t recompute capacity and
just take the old string’s. As well, the copy constructor always makes a
small string when it will fit, even if copied from a big string that got
truncated.

This also reworks the test to follow the idiom adopted elsewhere re stl,
and adds a helper function to tell if a string is small based on data().
2024-06-20 14:52:12 -04:00
Steven Dee (Jōshin) 9a5a13854d CTL: utility.h, use ctl::swap in string (#1227)
* Add ctl utility.h

Implements forward, move, swap, and declval. This commit also adds a def
for nullptr_t to cxx.inc. We need it now because the CTL headers stopped
including anything from libc++, so we no longer get their basic types.

* Use ctl::swap in string

The STL spec says that swap is located in the string_view header anyawy.
Performance-wise this is a noop, but it’s slightly cleaner.
2024-06-19 01:00:59 -04:00
Steven Dee (Jōshin) a795017416 Fix c.inc _Atomic define for C++ (#1231)
c.inc (AFAICT erroneously) defined _Atomic(t) as `volatile t *`, when it
should have just said `volatile t`, when __STDC_VERSION__ was too small.
This happens when we’re compiling C++, but in C++11, _Atomic is a define
supplied by the STL rather than a keyword supplied by the compiler. Wait
though, it gets better: in C++11, _Atomic hooks you into the morass that
is stdatomic.h, and ultimately refers everything back to std::atomic<T>.

The gory, horrifying details are in libcxx's __atomic/cxx_atomic_impl.h.
The tldr is that for our purposes it’s fine to just say volatile and use
the normal libc/intrin/atomic.h functions.
2024-06-17 21:12:02 -07:00
Brian 8e3b361aeb Remove testing label from labeler.yml (#1220)
#1219 had an issue with noisy testing label. Investigated if there is a
syntax to make it more exclusive, but turns out there isn't one yet. So
let's remove an manually add it in as needed.

Fixes #1219.
2024-06-16 11:45:57 -04:00
Steven Dee (Jōshin) f9dd5683a4 Implement ctl::unique_ptr (#1216)
The way unique_ptr is supposed to work is as a purely compile-time check
that your raw pointers are getting deleted when they go out of scope. It
should ideally emit the same exact machine code as if you were using raw
pointers with manual deletes.

Part of what this means is that under normal circumstances, a unique_ptr
shouldn’t take up more space than a raw pointer - in other words, sizeof
unique_ptr<T> should == sizeof(T*).

The present PR doesn’t bother with the specialization for array types. I
also left a couple other parts of the STL API unimplemented. I’d love to
see someone else implement these, or I’ll get to them at some point.
2024-06-15 22:54:52 -04:00
Steven Dee (Jōshin) e38a6e7996 ctl string const/value tweaks (#1218)
The mangled name of a C++ function will typically not vary by const-ness
of a by-value parameter; in other words, there is no meaning to a const-
qualified by-value parameter in a function prototype. However, the const
keyword _does_ matter at function _definition_ time, like it does with a
variable declared in the body. So for prototypes, we strip out const for
by-value parameters; but for definitions, we leave them alone.

At function definition (as opposed to prototype), we add const to values
in parameters by default, unless we’re going to mutate them.

This commit also changes a couple of const string_view& to be simply by-
value string_view. A string_view is only two words; it rarely ever makes
sense to pass one by reference if it’s not going to be mutated.
2024-06-15 18:09:30 -07:00
Steven Dee (Jōshin) ddfaf3fee0 Explain the clang-format off in new.h 2024-06-15 17:17:04 -07:00
Michael Lenaghan 0dbf01bf1d Bring Lua to 5.4.6. (#1214)
This essentially re-does the work of #875 on top of master.

This is what I did to check that Cosmo's Lua extensions still worked:

```
$ build/bootstrap/make MODE=aarch64 o/aarch64/third_party/lua/lua
$ ape o/aarch64/third_party/lua/lua
>: 10
10
>: 010
8
>: 0b10
2
>: string.byte("\e")
27
>: "Hello, %s" % {"world"}
Hello, world
>: "*" * 3
***
```

`luaL_traceback2` was used to show the stack trace with parameter
values; it's used in `LuaCallWithTrace`, which is used in Redbean to run
Lua code. You should be able to see the extended stack trace by running
something like this: `redbean -e "function a(b)c()end a(2)"` (with
"params" indicating the extended stack trace):

```
stack traceback:
 [string "function a(b)c()end a(2)"]:1: in function 'a', params: b = 2;
 [string "function a(b)c()end a(2)"]:1: in main chunk
```
@pkulchenko confirmed that I get the expected result with the updated
code.

This is what I did to check that Lua itself still worked:

```
$ cd third_party/lua/test/
$ ape ../../../o/aarch64/third_party/lua/lua all.lua
```

There's one test failure, in `files.lua`:

```
***** FILE 'files.lua'*****
testing i/o
../../../o/aarch64/third_party/lua/lua: files.lua:84: assertion failed!
stack traceback:
[C]: in function 'assert'
files.lua:84: in main chunk
(...tail calls...)
all.lua:195: in main chunk
[C]: in ?
.>>> closing state <<<
```

That isn't a result of these changes; the same test is failing in
master.

The failure is here:

```lua
if not _port then   -- invalid seek
  local status, msg, code = io.stdin:seek("set", 1000)
  assert(not status and type(msg) == "string" and type(code) == "number")
end
```

The test expects a seek to offset 1,000 on stdin to fail — but it
doesn't. `status` ends up being the new offset rather than `nil`.

If I comment out that one test, the remaining tests succeed.
2024-06-15 20:13:08 -04:00
Jōshin 3a599bfbe1 Update .git-blame-ignore-revs 2024-06-15 13:36:07 -07:00
Jōshin 89fc95fefd Rerun clang-format on the repo (#1217)
🚨 clang-format changes output per version!

This is with version 19.0.0. The modifications seem to be fixing the old
version’s errors - mainly involving omitted whitespace around binary ops
and inserted whitespace between goto labels and colons (if followed by a
curly brace.)

Also fixes a few mistakes made by e.g. someone (ahem) forgetting to pass
his ctl/string.h modifications through it.

We should add this to .git-blame-ignore-revs once we have its final hash
on master.
2024-06-15 16:34:48 -04:00
Jōshin 8e37ee2598 ctl::string cleanup (#1215)
We now fully initialize a ctl::string’s memory, so that it is always set
to a well-defined value, thus making it always safe to memcpy out of it.
This incidentally makes our string::swap function legal, which it wasn’t
before. This also saves us a store in string::reserve.

Now that we have made both big_string and small_string POD, I believe it
is safe to elide the launder calls, and have done so, thus cleaning up a
lot of the blob-related code.

I also got rid of set_big_capacity and replaced it with a set_big_string
that leaves us in a well-defined state afterwards. This function also is
able to be somewhat simpler; rather than delicate bit-twiddling, it just
reaches straight into blob and rewrites it wholesale.

Overall, this shaves about 1–2ns off of most benchmarks, and adds 1ns to
only one of them - creating a string from a char *.
2024-06-15 10:49:52 -07:00
Jōshin d9b4f647d8 Uncomment swap test (#1210) 2024-06-10 21:51:19 -07:00
Jōshin 0dde3a0e70 Get rid of preprocessor stuff in test (#1202)
Also it's a bit more idiomatic to say s.npos rather than string::npos.
2024-06-10 07:00:37 -07:00
Jōshin 32643e9fa7 Decouple swap from std (#1211)
This allows you to implement your own swap function without it having to
be part of the std namespace. std::swap is still used if it's available.
2024-06-10 03:40:17 -07:00
Jōshin 0a92c78035 Remove erroneous ctl:: prefixes 2024-06-08 15:07:39 -07:00
Jōshin 118db71121 Provide a minimal new.h for CTL (#1205)
This replaces the STL <new> header. Mainly, it defines a global operator
new and operator delete, as well as the placement versions of these. The
placement versions are required to not get compile errors when trying to
write a placement new statement.

Each of these operators is defined with many, many different variants. A
glance at new.cc is recommended followed by a chaser of the Alexandrescu
talk "std::allocator is to Allocation as std::vector is to Vexation". We
must provide a global-namespace source-level definition of each operator
and it is illegal for any of them to be marked inline, so here we are.

The upshot is that we no longer need to include <new>, and our optional/
vector headers are self-contained.
2024-06-08 15:05:38 -07:00
Alkis Evlogimenos a0410f0170 Make big_string pod (#1204)
`big_string` is not pod which means it needs to be properly constructed
and destroyed. Instead make it POD and destroy it manually in `string`
destructor.
2024-06-08 10:02:33 -07:00
Alkis Evlogimenos d44a7dc603 Fix bugs in in ctl::optional (#1203)
Manually manage the lifetime of `value_` by using an anonymous
`union`. This fixes a bunch of double-frees and double-constructs.

Additionally move the `present_` flag last. When `T` has padding
`present_` will be placed there saving `alignof(T)` bytes from
`sizeof(optional<T>)`.
2024-06-07 20:47:24 -04:00
Jōshin 2ba6b0158f Fix some memory issues with ctl::string (#1201)
There were a few errors in how capacity and memory was being handled for
small strings. The capacity errors meant that small strings would become
big strings too soon, and the memory error introduced undefined behavior
that was caught by CheckMemoryLeaks in our test file but only sometimes.

The crucial change is in reserve: we only copy n bytes into p2, and then
we manually set the null terminator instead of expecting it to have been
there already. (E.g. it might not be there for an empty small string.)

We also fix one other doozy in append when we were exactly at the small-
to-big string boundary: we set the last byte (i.e., the remainder field)
to 0, then decremented it, giving us size_t max. Whoops. We boneheadedly
fix this by setting the 0 byte after we've fixed up the remainder, so it
is at worst a no-op.

Otherwise, capacity now works the same for small strings as it does with
big strings: it's the amount of space available including the null byte.

We test all of this with a new test that only gets included if our class
under test is not std::string (presumably meaning it's ctl::string.) The
test manually verifies that the small string optimization behaves how we
expect.

Since this test checks against std::string, we go ahead and include that
other header from the STL.

Also modifies the new test we introduced to also run on std::string, but
it just does the append without expecting anything about how its data is
stored. We also check that the string has the right value afterwards.
2024-06-07 01:15:37 -04:00
Jōshin f3effcb703 One more SSO erratum from #1199
Making a string_view from a string appears to take about 1.3ns no matter
what. 100% definitely no point deviating from the STL API over that.
2024-06-06 18:01:26 -07:00
Jōshin 03b476f943 Minor small-string errata from #1199
These commits were sitting on a local branch that I neglected to push
before merging. :(

* Use memcpy for string::reserve

* Remove fence comments
2024-06-06 17:56:30 -07:00
Jōshin 8b3e368e9a ctl::string small-string optimization (#1199)
A small-string optimization is a way of reusing inline storage space for
sufficiently small strings, rather than allocating them on the heap. The
current approach takes after an old Facebook string class: it reuses the
highest-order byte for flags and small-string size, in such a way that a
maximally-sized small string will have its last byte zeroed, making it a
null terminator for the C string.

The only flag we have is in the highest-order bit, that says whether the
string is big (set) or small (cleared.) Most of the logic switches based
on the value of this bit; e.g. data() returns big()->p if it's set, else
small()->buf if it's cleared. For a small string, the capacity is always
fixed at sizeof(string) - 1 bytes; we store the length in the last byte,
but we store it as the number of remaining bytes of capacity, so that at
max size, the last byte will read zero and serve as our null terminator.

Morally speaking, our class's storage is a union over two POD C structs.
For now I gravitated towards a slightly more obtuse approach: the string
class itself contains a blob of the right size, and we alias that blob's
pointer for the two structs, taking some care not to run afoul of object
lifetime rules in C++. If anyone wants to improve on this, contributions
are welcome.

This commit also introduces the `ctl::__` namespace. It can't be legally
spelled by library users, and serves as our version of boost's "detail".

We introduced a string::swap function, and we now use that in operator=.
operator= now takes its argument by value, so we never need to check for
the case where the pointers are equal and can just swap the entire store
of the argument with our own, leaving the C++ destructor to free our old
storage afterwards.

There are probably still a few places where our capacity is slightly off
and we grow too fast, although there don't appear to be any where we are
too slow. I will leave these to be fixed in future changes.
2024-06-06 20:50:51 -04:00
Brian df6b384e31 github: add labeler action (#1196)
Bit easier to do this as everything seems to be sorted into logical
folders. You may need to add new labels to support this however.
2024-06-06 05:51:36 -07:00
Brian 280bdec817 github: add issue template (#1195)
copy paste and adapted from llamafile but added a research template as
well for the more technical research tickets
2024-06-06 05:51:06 -07:00
Jōshin 2c5e7ec547 Add terminating :vi on some modelines
Noticed because the settings they specified weren't getting picked up by
editor sessions in those files.
2024-06-05 20:36:55 -07:00
Jōshin fdcb8b2f7e add formatting commit 2024-06-05 16:36:34 -07:00
Jōshin 04c6bc478e vim C++ filetype is still spelled "cpp" 2024-06-05 16:34:47 -07:00
Justine Tunney cc2c1893c5 Fix some nits 2024-06-05 04:05:49 -07:00
Justine Tunney 3093f0e467 Release Cosmopolitan v3.4.0 2024-06-05 03:07:03 -07:00
Justine Tunney 3609f65de3 Make malloc() go 200x faster
If pthread_create() is linked into the binary, then the cosmo runtime
will create an independent dlmalloc arena for each core. Whenever the
malloc() function is used it will index `g_heaps[sched_getcpu() / 2]`
to find the arena with the greatest hyperthread / numa locality. This
may be configured via an environment variable. For example if you say
`export COSMOPOLITAN_HEAP_COUNT=1` then you can restore the old ways.
Your process may be configured to have anywhere between 1 - 128 heaps

We need this revision because it makes multithreaded C++ applications
faster. For example, an HTTP server I'm working on that makes extreme
use of the STL went from 16k to 2000k requests per second, after this
change was made. To understand why, try out the malloc_test benchmark
which calls malloc() + realloc() in a loop across many threads, which
sees a a 250x improvement in process clock time and 200x on wall time

The tradeoff is this adds ~25ns of latency to individual malloc calls
compared to MODE=tiny, once the cosmo runtime has transitioned into a
fully multi-threaded state. If you don't need malloc() to be scalable
then cosmo provides many options for you. For starters the heap count
variable above can be set to put the process back in single heap mode
plus you can go even faster still, if you include tinymalloc.inc like
many of the programs in tool/build/.. are already doing since that'll
shave tens of kb off your binary footprint too. Theres also MODE=tiny
which is configured to use just 1 plain old dlmalloc arena by default

Another tradeoff is we need more memory now (except in MODE=tiny), to
track the provenance of memory allocation. This is so allocations can
be freely shared across threads, and because OSes can reschedule code
to different CPUs at any time.
2024-06-05 02:02:14 -07:00
Justine Tunney 9906f299bb Refactor and improve CTL and other code 2024-06-04 05:45:48 -07:00
Justine Tunney 1d8f37a2f0 Fix the MODE=tiny builds 2024-06-03 10:36:38 -07:00
Justine Tunney e677460d14 Delete .vscode folder
It hasn't been maintained in years. I'm tired of the root level of our
project having an advertisement for Microsoft Visual Studio Code. Your
preferred editor should be Emacs or Vim.
2024-06-03 09:40:45 -07:00
Justine Tunney 4937843f70 Introduce Cosmopolitan Templates Library (CTL) 2024-06-03 09:21:59 -07:00
Justine Tunney b003888696 Make __demangle() heap 10% more compact 2024-06-02 16:18:55 -07:00
Justine Tunney 2ca491dc56 Write more __demangle() tests 2024-06-02 07:37:15 -07:00
Justine Tunney 9aa353d88b Document __demangle() and fix a const func ptr bug 2024-06-02 04:15:48 -07:00
Justine Tunney c67faf61df Delete some unintentional code 2024-06-01 20:36:58 -07:00
Justine Tunney 165c6b37e2 Add C++ demangling to privileged runtime
Cosmo will now print C++ symbols correctly in --ftrace logs and
backtraces. Doing this required reducing the memory requirement
of the __demangle() function by 3x. This was accomplished using
16-bit indices and 16-bit malloc granularity. That puts a limit
on the longest symbol we can successfully decode, which I think
would be around 6553 characters long, given a 65536-byte buffer
2024-06-01 20:10:58 -07:00
Jōshin dcd626edf8 Add .git-blame-ignore-revs
If you follow the directions in that file then git blame will ignore the
listed commits. A commit should only go in that file if its only changes
were to formatting, particularly on a large part of the codebase (like a
change to .clang-format getting applied to the repo.)

Cribbed from here:

https://www.stefanjudis.com/today-i-learned/how-to-exclude-commits-from-git-blame/
2024-06-01 20:10:48 -07:00
Jōshin f032b5570b Run clang-format (#1197) 2024-06-01 16:30:43 -04:00
Justine Tunney ea081b262c Add some noexcept annotations 2024-06-01 03:19:53 -07:00
Justine Tunney fae1c32267 Encode ±INFINITY as ±1e5000
The V8 behavior of encoding infinity as null doesn't make sense to me.
Using ±1e5000 is better, because JSON.parse decodes it as INFINITY and
the information is preserved. This could be a breaking change for some
2024-06-01 03:19:50 -07:00
Justine Tunney 9b6718ac99 Improve backtraces
We're now able to rewind the instruction pointer in x86 backtraces. This
helps ensure addr2line cannot print information about unrelated adjacent
code. I've restored -fno-schedule-insns2 in most cases because it really
does cause unpredictable breakage for backtraces.
2024-05-30 15:23:11 -07:00
Justine Tunney cd672e251f Improve crash signal reporting on Windows
This change fixes a bug where exiting a crash signal handler on Windows
after adding the signal to uc_sigmask, but not correcting the CPU state
would cause the signal handler to loop infinitely, causing process hang

Another issue is that very tiny programs, that don't link posix signals
would not have their SIGILL / SIGSEGV / etc. status reported to Cosmo's
bash shell when terminating on crash. That's fixed by a tiny handler in
WinMain() that knows how to map WIN32 crash codes to the POSIX flavors.
2024-05-30 14:04:10 -07:00
Justine Tunney 500a47bc2f Fix undefined behavior in unit test
Fixes #1194
2024-05-29 20:31:46 -07:00
Justine Tunney e4d25d68e4 Drop support for Windows 8
Microsoft caused some very gentle breakages for Cosmopolitan. They
removed the version information from the PEB which caused uname to
report WINDOWS 0.0.0. We should have called GetVersionExW but that
doesn't really exist anymore either. Windows policy is now to give
whatever version we used in ape/ape.S. Windows8 has been EOL since
2023-01-10 so lets avoid our modern executables being relegated to
legacy infrastructure. Requiring Windows 10+ going forward lets us
remove runtime compatibility bloat from the codebase. Further note
Cosmopolitan maintains a Windows Vista branch on GitHub, so anyone
preferring the older versions, can still have a future with Cosmo.

Another neat thing this fixes is UTF-8 support in the console. The
changes Microsoft made broke the if statement that enabled UTF8 in
terminals. This explains why bug reports had broken arrows. In the
future this should be less of an issue, since the PEB code is gone
which means we more strictly conform to only Microsoft's WIN32 API
2024-05-29 19:37:47 -07:00
Justine Tunney f31a98d50a Fix bug with realpath() on Windows 2024-05-29 18:47:01 -07:00
Justine Tunney 2816df59b2 Increase tinymalloc granularity 2024-05-29 18:26:01 -07:00
Justine Tunney a05ce3ad9d Support avx512f + vpclmulqdq crc32() acceleration
Cosmo's _Cz_crc32() function now goes 73 GiB/s on Threadripper. This
will significantly improve the performance of the PKZIP file format.
This algorithm is also used by apelink, to create deterministic ids.
2024-05-29 10:13:37 -07:00
Justine Tunney 7c8df05042 Improve -march=native micro-architecture detection 2024-05-29 10:12:49 -07:00
Justine Tunney 4c77acdfcf Add LoadZipArgs() to <cosmo.h> 2024-05-29 10:12:20 -07:00
Justine Tunney b74b974cfd Introduce #include <tinygetopt.h>
The normal getopt() function is bloated because it links printf(). This
change exports the original authentic bsd getopt function, that cosmo's
always used internally so cosmocc users don't need to include internals
2024-05-29 10:11:17 -07:00
Justine Tunney 07cef612c3 Make dlmalloc 2.4x faster for multithreading
This change adds a TLS freelist for small dynamic memory allocations.
Cosmopolitan's TIB is now 512 bytes in size. Single-threaded malloc()
performance isn't impacted by this, until pthread_create() is called.
Single-threaded programs may also want to consider using:

    #include "libc/mem/tinymalloc.inc"

Which will shave 30k off the executable size and sometimes go faster.
2024-05-28 11:18:34 -07:00
Justine Tunney deaef81463 Favor siginfo_t over struct siginfo 2024-05-28 02:34:17 -07:00
Justine Tunney c638eabfe0 Fix compiler warning 2024-05-27 02:23:24 -07:00
Justine Tunney 8e68384e15 Upgrade to 2022-era LLVM LIBCXX 2024-05-27 02:12:27 -07:00
Justine Tunney 2f4ca71f26 Release Cosmopolitan v3.3.10 2024-05-26 22:13:45 -07:00
Justine Tunney 07004ebf04 Upgrade to superconfigure z0.0.42 2024-05-26 22:12:25 -07:00
Justine Tunney 086d7006da Improve crash handler on XNU
This avoids an issue where a crash signal could cause the MacOS process
to freeze and consume all CPU rather than dying as it rightfully should
2024-05-26 18:42:09 -07:00
Gavin Hayes 0a51241f7a ntspawn: fix initializing NtStartupInfoEx (#1190) 2024-05-26 20:54:09 -04:00
Justine Tunney c68f6599e5 Fix definition of getpeername on FreeBSD
We were using the COMPAT magic number, which was recently removed.
2024-05-26 17:03:22 -07:00
Justine Tunney af3f62a71a Ensure io requests are always capped at 0x7ffff000
This gives us the Linux behavior across platforms.

Fixes #1189
2024-05-26 16:53:13 -07:00
Justine Tunney 6cf9b9e0fc Release Cosmopolitan v3.3.9 2024-05-26 15:28:03 -07:00
Justine Tunney 1d4b452839 Refactor some code 2024-05-26 06:03:50 -07:00
Justine Tunney c2db3b703a Introduce --timelog=FILE flag to GNU Make 2024-05-25 14:50:20 -07:00
Justine Tunney edb03b89d8 Make stdin unbuffered when appropriate 2024-05-25 07:57:13 -07:00
Justine Tunney 7724664b13 Release Cosmopolitan v3.3.8 2024-05-25 05:59:20 -07:00
Justine Tunney 1df4296208 Fix stdio for character device regression
Caused by ed93fc3dd7
2024-05-25 05:58:09 -07:00
Justine Tunney ce9aeb2aed Release Cosmopolitan v3.3.7 2024-05-24 19:37:21 -07:00
Justine Tunney ed93fc3dd7 Fix fread() with 2gb+ sizes 2024-05-24 19:28:23 -07:00
Justine Tunney 5f61d273e4 Add hwap constants to sys/auxv.h 2024-05-24 11:44:44 -07:00
Justine Tunney bf3531de81 Make crash reports reliable in multithreaded case 2024-05-24 11:44:44 -07:00
Justine Tunney 3b0ea2db4d Remove -fno-schedule-insns2
This was originally added when we were using the mcount hooking
technique, to prevent the nop from floating above the frame pointer
creation. Now we use a better codegen technique for hooking so it's no
longer needed to sacrifice performance by using this flag.
2024-05-24 11:44:44 -07:00
Justine Tunney f029375d39 Introduce MAP_HUGETLB 2024-05-24 11:44:44 -07:00
Justine Tunney 9b87dd2b87 Refactor some code 2024-05-24 11:44:44 -07:00
Jōshin 787b04f752 Run all BLAKE2B256 test vectors (#1185) 2024-05-24 10:59:23 -07:00
Jōshin 0768807935 Rename python -> python3 (closes #1144) (#1187)
When we removed the com suffix from ape binaries, we broke the build for
ape's python for any case-insensitive file system, i.e. Windows and XNU,
because there is a third_party/python/Python that gets mirrored in the o
directory with the python object files and clashes with the binary name.
This patch hacks around this by renaming the binary to "python3" so that
it no longer clashes with that directory.
2024-05-24 10:56:33 -07:00
Justine Tunney 0b59f01b43 Put confstr() in unistd.h
Fixes #1184
2024-05-21 15:35:06 -07:00
Justine Tunney cf70a44756 Support shebang on Windows
Fixes #1010
2024-05-20 22:11:42 -07:00
Justine Tunney a52792c59f Add some missing build dependencies 2024-05-20 17:23:25 -07:00
Jōshin 65c9b28e99 Fix buffer overflow in os.tmpname (#1180)
At least on macOS, `strlen(getenv("TMPDIR"))` is 50. We now allow a /tmp
that takes up to 120 or so bytes to spell. Instead of overflowing, we do
a bounds check and the function fails successfully on even longer /tmps.

Fixes #1108 (os.tmpname crashes redbean)
2024-05-20 03:46:27 -04:00
Jōshin 4292348707 Import libbsd readpassphrase (#1182)
Included from:

https://gitlab.freedesktop.org/libbsd/libbsd

At commit 04a24db27ad1572f766bad772cdd9c146e6d9cf0.
2024-05-20 03:37:30 -04:00
Jōshin 2ec20b6c4c Fix #922 (Duplicated line) (#1178) 2024-05-20 00:33:50 -07:00
Jōshin 47183551d6 Fix sleb64 (#1179)
Fixes #920
2024-05-20 00:33:17 -07:00
Jōshin 42891e82bb Fix #959 (typo in cosmocc --version) 2024-05-18 15:20:27 -07:00
Gavin Hayes 624119ea38 Fix NT accept/connect not initializing with SO_UPDATE_*_CONTEXT (#1164) 2024-05-17 02:45:30 -07:00
Justine Tunney 2f3c6e7cc3 Revert "Remove zlib namespacing (#1142)"
This reverts commit 5488f0b2ca which was a
good experiment to try, that didn't work out due to #1176

Fixes #1176
2024-05-14 20:45:23 -07:00
Justine Tunney 13b9ecd537 Remove outdated restriction on -Werror on cosmocc
This was a good idea back when we were only using it to build various
open source projects. However it no longer makes sense that many more
people are depending on cosmocc, to develop new software. Our tooling
shouldn't be making these kinds of decisions for the user.
2024-05-09 14:03:51 -07:00
Justine Tunney 6659981457 Release Cosmopolitan v3.3.6 2024-05-08 04:20:22 -07:00
Justine Tunney ae2a7ac844 Fix thread-local storage bugs on aarch64
This change fixes an issue where .tbss memory might not be initialized.
2024-05-08 04:20:22 -07:00
Justine Tunney 793393a341 Polyfill SA_RESETHAND on MacOS ARM64
This change solves the XNU crash loop mystery. Apple's documentation
claims to support this feature, but they only define the constant in
their header files. The kernel acknowledges thi SA_RESETHAND bit, by
clearing it from the sa_flags state, returns zero, and does nothing.
2024-05-08 04:20:22 -07:00
Justine Tunney 952b9009e8 Avoid crash looping on AARCH64 2024-05-08 04:20:22 -07:00
Jōshin 8f18b3ad65 Reapply apeinstall.sh binfmt flags (#1171)
7d31fc3 made it safe to register ape with the binfmt P flag. Older files
will still get their paths passed via argv[0] and everything should just
work. This also restores the F flag that was rolled back alongside the P
flag; this seems like a good idea to me now. It makes it so /usr/bin/ape
is kind of part of the kernel; simply replacing the file will not change
the loader used. The binfmt will have to be reregistered as well.

ape/apeinstall.sh will already nag you if there's an existing binfmt for
ape. It might be nice to make it actually replace the registration.

This reverts commit df648fb174.
2024-05-08 01:48:48 -04:00
Jōshin 56e0fd9d1d Use latest cosmocc (#1175) 2024-05-08 00:08:26 -04:00
Justine Tunney df68a6362b Release Cosmopolitan v3.3.5 2024-05-07 18:18:10 -07:00
Justine Tunney 19c81863a3 Improve crash backtrace reliability
We're now able to pretty print a C++ backtrace upon crashing in pretty
much any runtime execution scenario. The default pledge sandbox policy
on Linux is now to return EPERM. If you call pledge and have debugging
functions linked (e.g. GetSymbolTable) then the symbol table shall get
loaded before any security policy is put in place. This change updates
build/bootstrap/fixupobj too and fixes some other sneaky build errors.
2024-05-07 18:10:28 -07:00
Jōshin 7d31fc311a Loaders rewrite argv[0] for old binaries (#1170)
For this to work, a loader has to be able to tell the difference between
an ‘old’ and a ‘new’ binary. This is achieved via a repurposing of ELF’s
e_flags field. We previously tried to use the padding in e_ident for it,
but binutils was resetting it to zero in e.g. strip.

This introduces one new ELF flag for cosmopolitan binaries. It is called
`EF_APE_MODERN`. We choose 0x101ca75, "lol cat 5".

It should now be safe to install the ape loader binfmt registration with
the `P` flag.
2024-05-07 20:42:18 -04:00
Justine Tunney 57c0b065c8 Make old C++ demangler asynchronous signal safe
It's now possible to safely print C++ backtraces from signal handlers.
This symbol demangler doesn't need malloc, tls, or even static memory.
Additionally, this change makes it 2x faster and adds test cases. It's
almost as performant and accurate as the libcxxabi implementation now.
2024-05-07 03:41:33 -07:00
Justine Tunney a6ecbb747d Introduce libc/mem/tinymalloc.inc
This allocator shaves ~20kb off single-threaded tool programs and is
slightly faster than proper malloc for simple non-demanding programs
2024-05-07 03:19:49 -07:00
Justine Tunney 3bf95ae7ec Invent tool for renaming ELF object symbols 2024-05-07 03:19:49 -07:00
Justine Tunney 5fd7b07fac Improve AVX512 feature detection 2024-05-07 03:19:49 -07:00
Jōshin 317c8bc312 Update MODE=tiny time zone list (#1167)
I took one canonical IANA zone ID from each of the different colored
regions in this article, except those that do not observe DST and do
not have a Google office. See the "Time in Europe" Wikipedia article.

As to which canonical ID to use, this was somewhat arbitrary. Brussels
was obvious, as the de facto capital of the EU. For the rest, I mostly
just went with lexicographic ordering of the most recognizable options.

I've sorted the American zones. This Keeps the U.S. ones together but
does everything alphabetically otherwise. I've added the remaining
Canadian zones These have DST (and Newfoundland is off by a half-
hour from a UTC interval) so they cannot use Etc/. The Pacific/ zones
are sort of sorted. The Chathan Islands have been added. This is the
last of the zones I believe with a non-integer hour offset from UTC.
2024-05-06 16:48:49 -07:00
Jōshin 2a1f352622 Tell vim not to mess with ape.S 2024-05-06 15:36:35 -07:00
Justine Tunney 06d916b449 Add VirtualAlloc2 WIN32 API 2024-05-04 23:26:40 -07:00
Justine Tunney 9ea64725b6 Fix build error 2024-05-04 23:23:56 -07:00
Justine Tunney f9fc7eb49f Fix MODE=dbg build errors 2024-05-04 23:20:12 -07:00
Justine Tunney b0df6c1fce Implement proper time zone support
Cosmopolitan now supports 104 time zones. They're embedded inside any
binary that links the localtime() function. Doing so adds about 100kb
to the binary size. This change also gets time zones working properly
on Windows for the first time. It's not needed to have /etc/localtime
exist on Windows, since we can get this information from WIN32. We're
also now updated to the latest version of Paul Eggert's TZ library.
2024-05-04 23:06:37 -07:00
Justine Tunney d5ebb1fa5b Add MapViewOfFile3 WIN32 API 2024-05-04 12:25:41 -07:00
Justine Tunney 130fd66f9e Release Cosmopolitan v3.3.4 2024-05-03 09:25:02 -07:00
Justine Tunney 8a44f913ae Delete flaky tests
Signals are extremely difficult to unit test reliably. This is why
functions like sigsuspend() exist. When testing something else and
portably it becomes impossible without access to kernel internals.

OpenMP flakes in QEMU on one of my workstations. I don't think the
support is production worthy, because there's been issues on MacOS
additionally. It works great for every experiment I've used it for
though. However a flaky test is worse than no test at all. So it's
removed until someone takes an interest in productionizing it.
2024-05-03 09:11:04 -07:00
Gautham 5488f0b2ca Remove zlib namespacing (#1142)
We have an optimized version of zlib from the Chromium project.
We need it for a lot of our libc services. It would be nice to export
this to user applications if we can, since projects like llamafile are
already depending on it under the private namespace, to avoid
needing to link zlib twice.
2024-05-03 08:07:25 -07:00
Gavin Hayes deff138e7e recvfrom: don't convert address if addrsize is 0 (#1153) 2024-05-03 08:03:57 -07:00
Gavin Hayes b6e40a3a58 Add /dev/(u)random on NT (#1163) 2024-05-03 07:59:51 -07:00
Cadence Ember 8f6bc9dabc Let signals interrupt fgets unless SA_RESTART set (#1152) 2024-05-03 07:49:41 -07:00
Justine Tunney 181cd4cbe8 Add sysctlbyname() for MacOS 2024-05-02 23:21:43 -07:00
Justine Tunney 5c6877b02b Introduce support for trapping math
The feenableexcept() and fedisableexcept() APIs are now provided which
let you detect when NaNs appear the moment it happens from anywhere in
your program. Tests have also been added for the mission critical math
functions expf() and erff(), whose perfect operation has been assured.
See examples/trapping.c to see how to use this powerful functionality.
2024-04-30 13:38:43 -07:00
Justine Tunney 403bc25412 Make MXCSR editable by signal handlers on Windows
It's now possible to recover from floating point exception traps.
2024-04-30 13:38:43 -07:00
Matt Colyer 3bcd40be12 Fix regression in apeinstall.sh (#1161)
This should have been a part of a6baba1.
2024-04-29 20:40:38 -07:00
Paul Kulchenko fcfdd26b72 Fix wrong parameter type in redbean documentation (#1162) 2024-04-29 20:09:25 -04:00
Justine Tunney 2b00ee8723 Fix errno.h documentation typos
Closes #1156
2024-04-28 22:57:21 -07:00
Justine Tunney ff2b6fc0b0 Add munlock() and mlock2() to pledge 2024-04-28 16:53:23 -07:00
Justine Tunney 0eef971494 Add much of C11 threads.h API 2024-04-28 07:04:08 -07:00
Gavin Hayes 2bfd6b37c1 Various paginate improvements (#1148)
* start on improving __paginate

* make __paginate more robust

* add __paginate_file

* cleanup __paginate unlinking
2024-04-26 23:12:25 -04:00
Gavin Hayes 69db501c68 Fix fork locking on win32 (#1141)
* Fix fork locking on win32

- __enable_threads / set __threaded in __proc_setup as threads are required for
  win32 subprocess management
- move mmi/fds locking out of pthread_atfork.c into fork.c so it's done anytime
  __threaded is set instead of being dependent of pthreads
- explicitly yoink _pthread_onfork_prepare, _pthread_onfork_parent, and
  _pthread_onfork_child in pthread_create.c so they are linked in in-case they
  are separated from _pthread_atfork

Big Thanks to @dfyz for help with locating the issue, testing, and devising a fix!

* fix child processes not being able to open files, initialize all necessary locks on fork
2024-04-25 23:01:27 -04:00
Jōshin 6e6fc38935 Apply clang-format update to repo (#1154)
Commit bc6c183 introduced a bunch of discrepancies between what files
look like in the repo and what clang-format says they should look like.
However, there were already a few discrepancies prior to that. Most of
these discrepancies seemed to be unintentional, but a few of them were
load-bearing (e.g., a #include that violated header ordering needing
something to have been #defined by a 'later' #include.)

I opted to take what I hope is a relatively smooth-brained approach: I
reverted the .clang-format change, ran clang-format on the whole repo,
reapplied the .clang-format change, reran clang-format again, and then
reverted the commit that contained the first run. Thus the full effect
of this PR should only be to apply the changed formatting rules to the
repo, and from skimming the results, this seems to be the case.

My work can be checked by applying the short, manual commits, and then
rerunning the command listed in the autogenerated commits (those whose
messages I have prefixed auto:) and seeing if your results agree.

It might be that the other diffs should be fixed at some point but I'm
leaving that aside for now.

fd '\.c(c|pp)?$' --print0| xargs -0 clang-format -i
2024-04-25 10:38:00 -07:00
Jōshin 342d0c81e5 vim spells the c++ filetype 'cpp' 2024-04-24 13:56:37 -07:00
Jōshin cec4c04aa1 Remove .com logic from GetProgramExecutableName (#1134)
The `com` parameter to `TryPath` was always 1, so there was no reason to
have it. This patch changes the logic to be as though `com` was 0, which
provides a possible answer to the TODO question -- the answer is no.

If we never care about appending `.com`, then `CopyWithCwd` doesn't need
to return anything beyond a boolean success value.
2024-04-23 22:07:19 -04:00
Alexey Izbyshev f8c0186221 Fix calling __dns_parse with potentially too large rlen
__res_send returns the full answer length even if it didn't fit the
buffer, but __dns_parse expects the length of the filled part of the
buffer.

Analogous to Musl commit 77327ed064bd57b0e1865cd0e0364057ff4a53b4 which
fixed the only other __dns_parse call site.
2024-04-23 09:36:07 -07:00
Quentin Rameau 6992d8c195 Remove arbitrary limit from DNS result parsing
The name resolution would abort when getting more than 63 records per
request, due to what seems to be a left-over from the original code.
This check was non-breaking but spurious prior to TCP fallback
support, since any 512-byte packet with more than 63 records was
necessarily malformed. But now, it wrongly rejects valid results.

Reported by Daniel Stefanik in Alpine Linux aports issue 15320.
2024-04-23 09:33:02 -07:00
Justine Tunney 1a6b4ab627 Import mntent bug fixes from Musl Libc
f314e133929b6379eccc632bef32eaebb66a7335
Author: Rich Felker <[email protected]>
Date:   Thu Nov 16 12:55:21 2023 -0500

    mntent: fields are delimited only by tabs or spaces, not general whitespace

    this matters because the kernel-provided mtab only escapes tabs,
    spaces, newlines, and backslashes. it leaves carriage returns, form
    feeds, and vertical tabs literal.

commit ee1d39bc1573c1ae49ee6b658938b56bbef95a6c
Author: q66 <[email protected]>
Date:   Thu Nov 9 20:48:44 2023 +0100

    mntent: unescape octal sequences

    As entries in mtab are delimited by spaces, whitespace characters
    are escaped as octal sequences. When reading them out, we have to
    unescape these sequences to get the proper string.
2024-04-23 09:29:56 -07:00
Justine Tunney 9e848abad9 Add missing Musl license headers 2024-04-23 09:29:46 -07:00
Justine Tunney 223eb358c4 Import compiler warning fix from Musl 2024-04-23 09:29:28 -07:00
Szabolcs Nagy ea3cc4ea0b math: fix fma(x,y,0) when x*y rounds to -0
if x!=0, y!=0, z==0 then

  fma(x,y,z) == x*y

in all rounding modes, while adding z can ruin the sign of 0 if x*y
rounds to -0.
2024-04-23 09:22:33 -07:00
Paul Kulchenko 01267ea0f5 Update GetRemoteAddr documentation to clarify that it may return nil. (#1149) 2024-04-22 01:55:08 -04:00
Justine Tunney bc6c18368f Update .clang-format 2024-04-19 17:11:08 -07:00
Jōshin 06839ab301 Change loop bound on uuidv4 test (#1143)
A half second (on my machine) is too long for a unit test. 1000
iterations is probably still overkill, but 0.01 seconds is fine.

Also made `y` local.
2024-04-12 12:22:34 -04:00
BONNAURE Olivier 39dde41516 [Redbean] Add UuidV4 method (#1140) 2024-04-12 11:10:27 -04:00
Jōshin 3e16e59f72 Move store after __asan_unpoison in __zipos_alloc (#1135)
Previously, the atomic store looked like it was happening while the
struct's memory was still poisoned. I was unable to observe any issues
with this, but this change seems to make the code more obviously correct
(at the cost of a redundant atomic store to zeroed space in case the map
needed to be extended.)
2024-04-10 23:04:03 -04:00
Justine Tunney cf9a1f7f33 Fix MODE=optlinux build 2024-04-06 21:02:19 -07:00
Justine Tunney 49a32136f8 Upgrade the One True Awk 2024-04-06 19:21:48 -07:00
Justine Tunney b9d6e6e348 Standardize on -fsigned-char
MaGuess on Discord pointed out the fact that cosmocc contradicts itself
on the signedness of `char`. It's up to each platform to choose one, so
the cosmo platform shall choose signed. The rationale is it makes the C
language syntax more internally similar. `char` should be `signed char`
for the same reason `int` means `signed int`. It's recommended that you
still assume `char` could go either way since that's portable thinking.
But if you want to assume we'll always have signed char, that's ok too.
2024-03-30 21:46:08 -07:00
Justine Tunney b431ca4449 Upgrade to cosmocc v3.3.3 2024-03-30 20:17:14 -07:00
Justine Tunney 98d06b23cc Release Cosmopolitan v3.3.3 2024-03-30 19:24:04 -07:00
Justine Tunney 9ff65cff63 Fix ZipOS extraction of ZIP64 assets 2024-03-30 19:00:40 -07:00
Justine Tunney bef2cb05ff Fix AVX512_BF16 2024-03-26 00:00:46 -07:00
mierenhoop 43885a76e4 Fix binary formatting for integers 2 and 3 (#1123) 2024-03-25 21:11:09 -07:00
Justine Tunney 74a85087de Support -S in cosmocross
You can now run commands like `x86_64-unknown-cosmo-c++ -S` when using
your cosmocc toolchain. Please note the S flag isn't supported for the
cosmocc command itself.
2024-03-24 15:49:55 -07:00
Justine Tunney 045632a743 Make fixupobj work on Windows
This fixes a regression with recent cosmocc toolchain releases that
causes fixupobj to fail with "ifunc ftruncate #1 failed" on Windows
2024-03-24 03:14:25 -07:00
Justine Tunney 9a10adac35 Remove .com from README 2024-03-22 19:38:38 -07:00
Justine Tunney 0c12c26e1d Update Emacs config 2024-03-22 19:27:11 -07:00
Justine Tunney 40b7da8422 Speed up fmaf() on x86 2024-03-22 19:26:56 -07:00
Justine Tunney 640668931d Clarify MacOS supported versions
We technically support versions much earlier, possibly dating back to
2018 or earlier, but they're not officially supported since I have no
way of testing them.

Closes #1129
2024-03-22 19:25:07 -07:00
Justine Tunney 1c34088ba0 Ignore -pie and -fpie in cosmocc
Fixes #1126
2024-03-22 03:55:01 -07:00
Justine Tunney bb92347158 Add deprecated stubs for _bsf / _bsr 2024-03-05 09:45:51 -08:00
Justine Tunney 8bfd56b59e Rename _bsr/_bsf to bsr/bsf
Now that these functions are behind _COSMO_SOURCE there's no reason for
having the ugly underscore anymore. To use these functions, you need to
pass -mcosmo to cosmocc.
2024-03-04 17:33:26 -08:00
Justine Tunney a6baba1b07 Stop using .com extension in monorepo
The WIN32 CreateProcess() function does not require an .exe or .com
suffix in order to spawn an executable. Now that we have Cosmo bash
we're no longer so dependent on the cmd.exe prompt.
2024-03-03 03:12:19 -08:00
Justine Tunney c8383f25b4 Release Cosmopolitan v3.3.2 2024-02-27 09:15:59 -08:00
Justine Tunney 64a9e6fe56 Fix compiler runtime for _Float16 type 2024-02-27 09:06:23 -08:00
Justine Tunney 0ef36489c8 Walk back most uses of __STRICT_ANSI__ 2024-02-27 04:09:49 -08:00
Justine Tunney 7c7bf4bb90 Release Cosmopolitan v3.3.1 2024-02-27 00:31:36 -08:00
Justine Tunney 592f6ebc20 Make quality improvements
- Write some more unit tests
- memcpy() on ARM is now faster
- Address the Musl complex math FIXME comments
- Some libm funcs like pow() now support setting errno
- Import the latest and greatest math functions from ARM
- Use more accurate atan2f() and log1pf() implementations
- atoi() and atol() will no longer saturate or clobber errno
2024-02-25 19:08:43 -08:00
Justine Tunney af8f2bd19f Shave 4kb off each binary 2024-02-25 11:11:34 -08:00
Justine Tunney 38bceaeb8f Give up on making clock_nanosleep() precise
Multiple projects I care about make the assumption that this isn't a
system call that sleeps for a particular number of nanonseconds, but
rather a function that parks processes on kernel scheduler quantums.
Anyone who wants the old behavior should use cosmo_clock_nanosleep()
2024-02-24 10:17:47 -08:00
Ivan Komarov 99f0491f04 Fix invalid XNU binaries generated by apelink in some edge cases (#1106)
* Fix `if...fi` generation in the generated APE shell script

A shell will fail with a syntax error on an empty `if` or `else` body.
That is, neither of these is allowed:

    # Empty `if`
    if [ ... ]; then
    fi

    # Empty `else`
    if [ ... ]; then
    ...
    else
    fi

There were two places where `apelink` could generate problematic `if`'s:

1. The XNU shell generation for aarch64 binaries when no loaders (either
   binary or source) are provided. They can't assimilate, so the resulting
   `else` body becomes empty.
   There is actually a code path guarded by the `gotsome` variable that
   inserts an extra `true` in this case, but the variable was never
   initialized, so in practice this code path didn't activate in my
   tests. This is fixed by initializing the variable.
2. The loader extraction code when no loaders are provided and XNU
   support is requested. This is fixed by adding a simliar code path
   that prevents an empty body from being generated.

* Update the apelink manual after commit d53c335

The `-s` option changed its meaning, but the docs weren't updated.
2024-02-23 08:11:27 -08:00
Justine Tunney e72a88ea70 Make fixups for libcrypt 2024-02-23 07:39:44 -08:00
Ivan Komarov 9b4701b3bc Don't include <crypt.h> from <stdlib.h> (#1112) 2024-02-23 07:18:35 -08:00
Ivan Komarov f7ff515961 *scanf() fixes to make TeX work (#1109)
* Fix reading the same symbol twice when using `{f,}scanf()`

PR #924 appears to use `unget()` subtly incorrectly when parsing
floating point numbers. The rest of the code only uses `unget()`
immediately followed by `goto Done;` to return back the symbol that
can't possibly belong to the directive we're processing.

With floating-point, however, the ungot characters could very well
be valid for the *next* directive, so we will essentially read them
twice. It can't be seen in `sscanf()` tests because `unget()` is a
no-op there, but the test I added for `fscanf()` fails like this:

        ...
        EXPECT_EQ(0xDEAD, i1)
                need 57005 (or 0xdead) =
                 got 908973 (or 0x000ddead)
        ...
        EXPECT_EQ(0xBEEF, i2)
                need 48879 (or 0xbeef) =
                 got 769775 (or 0x000bbeef)

This means we read 0xDDEAD instead of 0xDEAD and 0xBBEEF instead of
0xBEEF. I checked that both musl and glibc read 0xDEAD/0xBEEF, as
expected.

Fix the failing test by removing the unneeded `unget()` calls.

* Don't read invalid floating-point numbers in `*scanf()`

Currently, we just ignore any errors from `strtod()`. They can
happen either because no valid float can be parsed at all, or
because the state machine recognizes only a prefix of a valid
floating-point number.

Fix this by making sure `strtod()` parses everything we recognized,
provided it's non-empty. This requires to pop the last character
off the FP buffer, which is supposed to be parsed by the next
`*scanf()` directive.

* Make `%c` parsing in `*scanf()` respect the C standard

Currently, `%c`-style directives always succeed even if there
are actually fewer characters in the input than requested.

Before the fix, the added test fails like this:

        ...
        EXPECT_EQ(2, sscanf("ab", "%c %c %c", &c2, &c3, &c4))
                need 2 (or 0x02 or '\2' or ENOENT) =
                 got 3 (or 0x03 or '\3' or ESRCH)
        ...
        EXPECT_EQ(0, sscanf("abcd", "%5c", s2))
                need 0 (or 0x0 or '\0') =
                 got 1 (or 0x01 or '\1' or EPERM)

musl and glibc pass this test.
2024-02-23 07:15:30 -08:00
Justine Tunney 3afe3a3646 Fix broken link 2024-02-23 07:11:44 -08:00
Justine Tunney b4faaa7c57 Relax style guide 2024-02-23 06:55:37 -08:00
Justine Tunney c4991e53ac Document copyright policy exceptions 2024-02-23 06:53:46 -08:00
Justine Tunney ad3944a3b6 Support any HTTP method
It's now possible to use redbean Fetch() with arbitrary HTTP methods,
e.g. LIST which is used by Hashicorp. There's an eight char limit and
uppercase canonicalization still happens. This change also includes a
better function for launching a browser tab, that won't deadlock on a
headless workstation running Debian.

Closes #1107
2024-02-22 14:12:18 -08:00
Justine Tunney 29eac8e2a2 Fix regression in logger functions
The recent release broke things so that the logger wasn't logging by
default. This ensures that `__log_file` initializes in correct order
2024-02-22 14:08:20 -08:00
Justine Tunney a5a3e78fcf Make ftrace/strace C APIs work with cosmocc 2024-02-22 12:13:16 -08:00
Justine Tunney e9a756e78b Make --ftrace more crash proof with foreign code
Sometimes we need to interact with code that wasn't compiled using
`-fno-omit-frame-pointer`. For example, if a function pointer gets
passed and called by a foreign function, linked by cosmo_dlopen().
Function call tracing will now detect backtrace pointer corruption
and simply reduce the indentation level back to zero, as a result.
2024-02-22 11:06:31 -08:00
Justine Tunney 38af54a2dd Fix GCC version in cosmocc wrapper script
See #1111
2024-02-21 19:16:39 -08:00
Justine Tunney 77a92f517b Introduce getcpu() system call from glibc 2024-02-21 18:17:20 -08:00
Justine Tunney 3eb405e0e2 Resurrect <windows.h> as <windowsesque.h> 2024-02-21 16:41:11 -08:00
Justine Tunney 68b9479f0c Make cosmo.h work a little better 2024-02-21 16:40:09 -08:00
Justine Tunney d9df41c4d5 Add example email to CONTRIBUTING.md 2024-02-21 16:36:52 -08:00
Justine Tunney fc3c536ec2 Make --ftrace work better with cosmocc
This change causes cosmocc to use -fno-inline-functions-called-once by
default, unless -Os or -finline-functions-called-once is defined. This
is important since I believe it generally makes code go faster, and it
most importantly makes --ftrace output much more understandable, since
the trace will be more likely to reflect the actual shape of the code.
We've always used this flag in the mono repo when ftracing is enabled,
but it slipped my mind to incorporate this into the cosmocc toolchain.
2024-02-21 13:20:21 -08:00
Justine Tunney 77ddf4cb07 Fix fixupobj regression on MacOS ARM64 2024-02-21 12:13:54 -08:00
Justine Tunney 4d018306b3 Fix MODE=optlinux build 2024-02-20 14:24:56 -08:00
Justine Tunney b3bb93d1d9 Fix MODE=opt build 2024-02-20 14:12:12 -08:00
Justine Tunney 980af27bbf Fix documentation build 2024-02-20 13:40:46 -08:00
Justine Tunney 957c61cbbf Release Cosmopolitan v3.3
This change upgrades to GCC 12.3 and GNU binutils 2.42. The GNU linker
appears to have changed things so that only a single de-duplicated str
table is present in the binary, and it gets placed wherever the linker
wants, regardless of what the linker script says. To cope with that we
need to stop using .ident to embed licenses. As such, this change does
significant work to revamp how third party licenses are defined in the
codebase, using `.section .notice,"aR",@progbits`.

This new GCC 12.3 toolchain has support for GNU indirect functions. It
lets us support __target_clones__ for the first time. This is used for
optimizing the performance of libc string functions such as strlen and
friends so far on x86, by ensuring AVX systems favor a second codepath
that uses VEX encoding. It shaves some latency off certain operations.
It's a useful feature to have for scientific computing for the reasons
explained by the test/libcxx/openmp_test.cc example which compiles for
fifteen different microarchitectures. Thanks to the upgrades, it's now
also possible to use newer instruction sets, such as AVX512FP16, VNNI.

Cosmo now uses the %gs register on x86 by default for TLS. Doing it is
helpful for any program that links `cosmo_dlopen()`. Such programs had
to recompile their binaries at startup to change the TLS instructions.
That's not great, since it means every page in the executable needs to
be faulted. The work of rewriting TLS-related x86 opcodes, is moved to
fixupobj.com instead. This is great news for MacOS x86 users, since we
previously needed to morph the binary every time for that platform but
now that's no longer necessary. The only platforms where we need fixup
of TLS x86 opcodes at runtime are now Windows, OpenBSD, and NetBSD. On
Windows we morph TLS to point deeper into the TIB, based on a TlsAlloc
assignment, and on OpenBSD/NetBSD we morph %gs back into %fs since the
kernels do not allow us to specify a value for the %gs register.

OpenBSD users are now required to use APE Loader to run Cosmo binaries
and assimilation is no longer possible. OpenBSD kernel needs to change
to allow programs to specify a value for the %gs register, or it needs
to stop marking executable pages loaded by the kernel as mimmutable().

This release fixes __constructor__, .ctor, .init_array, and lastly the
.preinit_array so they behave the exact same way as glibc.

We no longer use hex constants to define math.h symbols like M_PI.
2024-02-20 13:27:59 -08:00
BONNAURE Olivier d3ff48c63f [Redbean] Feature / OnError(status, message) hook (#1103) 2024-02-14 01:55:50 -08:00
Justine Tunney 2ab9e9f7fd Make improvements
- Introduce portable sched_getcpu() api
- Support GCC's __target_clones__ feature
- Make fma() go faster on x86 in default mode
- Remove some asan checks from core libraries
- WinMain() now ensures $HOME and $USER are defined
2024-02-12 10:23:00 -08:00
Ivan Komarov d5225a693b libc headers: make <complex.h> work, add struct ipv6_mreq (#1100)
1. `libc/isystem/complex.h` (included when you do `#include <complex.h>`)
   defines `_COMPLEX_H`, and then proceeds to include `libc/complex.h`,
   which contains the actual complex-related declarations. However, they
   are *also* guarded by `_COMPLEX_H` and hence effectively ignored.
   Fix this by changing `_COMPLEX_H` to `COSMOPOLITAN_LIBC_COMPLEX_H_`,
   which is consistent with what the other headers (such as `math.h`) do.
2. Cosmopolitan could only support IPv4 multicast requests for sockets,
   since a declaration for `struct ipv6_mreq` was missing. Add support
   for IPv6, too, by adding the missing declaration.
2024-02-05 17:22:56 -05:00
Paul Kulchenko c72904b2f6 Fix redbean Fetch redirect with relative URL (#1034) 2024-02-02 10:02:29 -05:00
Justine Tunney 616717fa82 Fine tune OpenMP some more 2024-01-30 06:30:24 -08:00
Justine Tunney 369aebfc48 Make improvements
- Let OpenMP be usable via cosmocc
- Let libunwind be usable via cosmocc
- Make X86_HAVE(AVXVNNI) work correctly
- Avoid using MAP_GROWSDOWN on qemu-aarch64
- Introduce in6addr_any and in6addr_loopback
- Have thread stacks use MAP_GROWSDOWN by default
- Ask OpenMP to not use filesystem to manage threads
- Make NI_MAXHOST and NI_MAXSERV available w/o _GNU_SOURCE
2024-01-29 16:31:58 -08:00
Justine Tunney 5f8e9f14c1 Add OpenMP support 2024-01-28 22:39:02 -08:00
Justine Tunney c1e18e7903 Restore MODE=dbg support
We recently broke MODE=dbg support when we added C++ exception support.
This change adds the missing UBSAN interfaces, needed to get it working
again. Some of the ASAN checking in the SJLJ guts needed to be disabled
since I doubt anyone's combined the two features until now.
2024-01-26 23:07:18 -08:00
Justine Tunney f27808c4d2 Remove feature for embedding blink in ape scripts
Embedding Blink builds in Cosmo executables was a failed experiment. It
turned out to be easier than expected to let the mono repo have support
for multiple architectures. Blink still works great; it's supported and
recommended; just please use it as a separate program. For example, you
can use Blink to run Cosmo binaries on architectures like i486 / s390x.
2024-01-26 22:30:56 -08:00
Justine Tunney 2719080986 Add program for printing Windows network dev info 2024-01-26 22:29:23 -08:00
Trung Nguyen 33418f6742 libc: Fix __cxa_thread_atexit prototype (#1088)
The first parameter should be a function pointer.

Otherwise, this would conflict with the definition provided by
`libcxxabi`.
2024-01-22 10:23:28 -08:00
Trung Nguyen 83a8686c06 tool/cosmocc: Properly handle dependency output (#1091)
a2753de contains some regressions, causing `fixupobj` to be
inappropriately suppressed when `-MD` or `-MMD` is passed.

This commit reverts most changes by a2753de, and:
- Treats all invocations of the compiler with `-M` and `-MM` as with the
`cpp` intent, since these flags imply `-E`.
- Handle the dependency output path specified by `-MF`.
  + This is trivial for `cosmocross` since the script does not throw
  objects to and from temporary directories.
  + For `cosmocc`, the file names are calculated based on the `-MF`
  value provided by the user. If this flag is not specified, the script
  generates the file name based on the output file using GCC rules.
  Then, before calling the real compilers, an additional `-MF` flag is
  passed to override the dependency outputs with mangled file names.
2024-01-22 10:22:16 -08:00
Justine Tunney 8ab3a545c6 Increase build memory quota
If you install qemu-user from apt then glibc links a lot of address
space bloat that causes pthread_create() to ENOMEM (a.k.a. EAGAIN).
Boosting the virtual memory quota from 512m to 2048m will hopefully
future proof the build for the future, as Linux distros get fatter.
Please note this only applies to MODE=aarch64 on x86_64 builds when
you're using QEMU from Debian/Ubuntu rather than installing the one
cosmo provides in third_party/qemu/qemu-aarch64.gz. This change may
also be useful to people who are using the host compiler toolchain.
2024-01-22 10:02:30 -08:00
Justine Tunney 8ebe2e9020 Fix assertion error in pthread_create() 2024-01-22 10:02:30 -08:00
BONNAURE Olivier 51cd83f674 Add Curve25519 method in help.txt + add new specs file (#1096) 2024-01-21 06:58:48 -08:00
Justine Tunney 39b0a9c03e Fix scanf() %n off by one at eof
Fixes #1094
2024-01-20 15:06:16 -08:00
Justine Tunney d50064a779 Add Curve25519() API to Redbean 2024-01-20 01:06:19 -08:00
Justine Tunney 1226eb7a5e Fix gcc warning 2024-01-18 16:48:02 -08:00
Trung Nguyen 8834dde0c2 libcxx: Add missing implementation source files (#1089)
Added the implementation for `std::bad_any_cast` from upstream
`any.cpp`, and `std::bad_variant_access` from upstream `variant.cpp`.

This fixes missing `vtable` and `typeinfo` symbols when trying to link
code referencing these exception types.
2024-01-18 08:20:25 -08:00
Justine Tunney 1ef63eb206 Retire third_party/quickjs/
QuickJS cosmocc binaries are now being distributed on
https://bellard.org/quickjs/
2024-01-17 12:35:46 -08:00
Brian 08793aa143 Make ... optional in .args (#1086) 2024-01-15 16:20:50 -08:00
Trung Nguyen 6db1200a7e libc: Fix cxxabi.h header (#1087)
- `__cxa_*` runtime functions are expected to be in the `abi` namespace,
which is currently an alias for `__cxxabiv1`.
- Rely on the header provided by `libcxxabi` for functions that we do
not implement ourselves anymore.
2024-01-15 11:50:38 -08:00
Trung Nguyen a2753de7fd tool/cosmocc: Do not run fixupobj for text output (#1084)
Some compiler flags (such as -E or -MM) instruct GCC to only run the
preprocessor and produce certain text files.

In this case, we do not want to run `fixupobj` and make the tool fail
because the input is not an ELF64 binary.
2024-01-15 07:16:13 -08:00
Jōshin 6715b670b1 Skip the CAS on new_pos < 0 (#1080) 2024-01-12 21:08:28 -08:00
Justine Tunney fd75fd1467 Make std::pair trivial 2024-01-09 09:51:59 -08:00
Justine Tunney 5d80e8dbf6 Add more Intel microarchitectures 2024-01-09 01:54:26 -08:00
Justine Tunney eeb20775d2 Add dontthrow attribute to most libc functions
This will help C++ code that uses exceptions to be tinier. For example,
this change shaves away 1000 lines of assembly code from LLVM's libcxx,
which is 0.7% of all assembly instructions in the entire library.
2024-01-09 01:26:03 -08:00
Justine Tunney cb19e172da Release Cosmopolitan v3.2.4 2024-01-08 19:37:59 -08:00
Jōshin df648fb174 Revert apeinstall.sh binfmt flags (#1072)
The P flag breaks backwards compatibility with older binaries. The idea
is to revert this commit after that break has been resolved.
2024-01-08 14:21:21 -08:00
Justine Tunney 9a6af92bed Fix libcxxabi tests in MODE=tiny and MODE=rel
See #1076
2024-01-08 13:49:06 -08:00
Justine Tunney 6ab01716ce Fix aarch64 setjmp abi
We now store values in jmp_buf where the compiler wants them to be. This
fixes code that calls __builtin_setjmp() and __builtin_longjmp() such as
libunwind. All libcxxabi tests are now passing on ARM64.

See #1076
2024-01-08 13:26:28 -08:00
Justine Tunney 21093044c0 Rebuild build/bootstrap/chmod.com 2024-01-08 12:22:12 -08:00
Justine Tunney b8a9fbbd01 Review catch_multi_level_pointer test failure
This test was was failing on GitHub Actions because GA uses Linux and
Linux supports resource usage accounting. Cosmo's compile.com program
imposes CPU, memory and file size limits on both the compiler and the
test programs themselves.

See #1076
2024-01-08 12:10:29 -08:00
Justine Tunney 81ce2e4cbc Call thread finalizers on exit()
While we don't call POSIX thread key destructors from exit(), we do need
to call these, since C++ uses it for TLS object destructors.

See #1076
2024-01-08 11:45:02 -08:00
Justine Tunney 07db3004d6 Disable libcxxabi/test/guard_threaded_test
It works, but exceeds the CPU quota granted to tests, by a lot.
2024-01-08 11:19:09 -08:00
Trung Nguyen b0566348b2 third_party/libcxxabi: Add test suite (#1076)
Added the `libcxxabi` test suite as found in LLVM 17.0.6.

Some tests that do not apply to the current configuration of
comsopolitan are not added. These include:
- `backtrace_test`, `forced_unwind*`: Use unwind function unsupported in
SjLj mode.
- `noexception*`: Designed to test `libcxxabi` in no exceptions mode.

Some tests are added but not enabled due to bugs specific to GCC or
cosmopolitan. These are clearly indicated in the `BUILD.mk` file.
2024-01-08 10:50:06 -08:00
Justine Tunney 8fb24a8f88 Turn exceptions back off by default 2024-01-08 10:47:42 -08:00
Justine Tunney a4b455185b Bring back gc() function
Renaming gc() to _gc() was a mistake since the better thing to do is put
it behind the _COSMO_SOURCE macro. We need this change because I haven't
wanted to use my amazing garbage collector ever since we renamed it. You
now need to define _COSMO_SOURCE yourself when using amalgamation header
and cosmocc users need to pass the -mcosmo flag to get the gc() function

Some other issues relating to cancelation have been fixed along the way.
We're also now putting cosmocc in a folder named `.cosmocc` so it can be
more safely excluded by grep --exclude-dir=.cosmocc --exclude-dir=o etc.
2024-01-08 10:26:28 -08:00
Jōshin 6cb0354e19 mkofs improvements
- Always use $PWD/o. if it doesn't exist, say how to create it.

- return 2 on unsupported platforms.
2024-01-08 17:23:09 +00:00
Trung Nguyen b580080af1 tool/cosmocc: Allow exceptions and rtti (#1074)
With `libunwind` and `libcxxabi` included in `libcosmo`, we can now
allow users to build C++ applications with exceptions and RTTI enabled.

The default is still disabling these two to avoid bloating the binary.

Closes #1065
2024-01-08 08:47:53 -08:00
Trung Nguyen 8b33204f37 Add LLVM libcxxabi (#1063)
* third_party: Add libcxxabi

Added libcxxabi from LLVM 17.0.6
The library implements the Itanium C++ exception handling ABI.

* third_party/libcxxabi: Enable __cxa_thread_atexit

Enable `__cxa_thread_atexit` from libcxxabi.
`__cxa_thread_atexit_impl` is still implemented by the cosmo libc.
The original `__cxa_thread_atexit` has been removed.

* third_party/libcxx: Build with exceptions

Build libcxx with exceptions enabled.

- Removed `_LIBCPP_NO_EXCEPTIONS` from `__config`.
- Switched the exception implementation to `libcxxabi`. These two files
are taken from the same `libcxx` version as mentioned in `README.cosmo`.
- Removed `new_handler_fallback` in favor of `libcxxabi` implementation.
- Enable `-fexceptions` and `-frtti` for `libcxx`.
- Removed `THIRD_PARTY_LIBCXX` dependency from `libcxxabi` and
`libunwind`. These libraries do not use any runtime `libcxx` functions,
just headers.

* libc: Remove remaining redundant cxa functions

- `__cxa_pure_virtual` in `libcxxabi` is also a stub similar to the
existing one.
- `__cxa_guard_*` from `libcxxabi` is used instead of the ones from
Android.

Now there should be no more duplicate implementations.
`__cxa_thread_atexit_impl`, `__cxa_atexit`, and related supporting
functions, are still left to other libraries as in `libcxxabi`.

`libcxxabi` is also now added to `cosmopolitan.a` to make up for the
removed functions.

Affected in-tree libraries (`third_party/double-conversion`) have been
updated.
2024-01-08 08:45:10 -08:00
Trung Nguyen 94bab1618d Add fixes to libunwind (#1069) 2024-01-08 08:31:13 -08:00
Jōshin cafea9a0a5 Flesh out tool/zsh/mkofs
It now prevents double-mounting.

Also adds vim filetype modelines.
2024-01-08 05:18:18 +00:00
Jōshin 16099801cc zsh mkofs tool 2024-01-08 03:43:09 +00:00
Justine Tunney 4cd02c29ed Release Cosmopolitan v3.2.3 2024-01-07 12:39:09 -08:00
Justine Tunney 5ae2554c10 Have cosmo_dlopen() request -z execstack on Linux
The AMD HIP SDK for Linux ships prebuilt DSOs with an RWX PT_GNU_STACK
since old versions of GCC made it nearly impossible to build artifacts
where that wasn't the case, however modern glibc systems will flat out
refuse to link RWX DSOs from an execuatble that uses PT_GNU_STACK = RW
2024-01-07 11:37:18 -08:00
Justine Tunney 531ac85923 Disable Python test_resource
It fails strangely on some Linux systems due to constraints imposed by
the build system. We'll try to learn more about this later on.
2024-01-07 11:36:31 -08:00
Jōshin 8d9fcb5e5a Fix ape-m1.c usage
It's not about `$0` anymore.
2024-01-07 10:35:50 -05:00
Jōshin aa37a327ea Make $prog.ape more reliable on Apple Silicon (#1071)
Now it doesn't matter what argv `$prog.ape` is invoked with. We just get
our executable path the Apple way.
2024-01-07 07:13:20 -08:00
Justine Tunney c4205f8305 Remove lingering dlopen thunk 2024-01-07 02:14:41 -08:00
Justine Tunney b3fb6cff43 Add /dev/fd support to Windows
GNU bash needs this functionality, otherwise it can't do <(cmd...).
2024-01-06 18:20:21 -08:00
Jōshin 2e5f662dfe mmake: different cosmocc regime
Now that cosmocc is unpacked into a version-specific directory under
cosmocc/, it makes more sense to put the versions in /opt/cosmocc and
maintain a symlink to the currently active one.
2024-01-06 19:14:13 -05:00
Justine Tunney 5e7137097d Make breaking change to cosmo_dlsym()
The cosmo_dlsym() function now returns the raw function address. You
need to call cosmo_dltramp() on the result, to make it safe to call.
This change is important, because cosmo_dltramp() magic can't always
work; for some tricky functions, you need to translate ABIs by hand.
2024-01-06 15:45:26 -08:00
Justine Tunney f224a55d57 Fix MODE=dbg build
Fixes #1067
2024-01-06 15:22:19 -08:00
Justine Tunney 0c51995b2b Add missing sha256sum.c file 2024-01-06 14:47:14 -08:00
Jōshin a9f23ccd17 Fix download-cosmocc.sh on Mac (#1066)
shasum behaves enough like sha256sum to work in this case.
2024-01-06 14:46:31 -08:00
Justine Tunney 57d1bf7a9c Clarify CONTRIBUTING.md 2024-01-06 14:18:01 -08:00
Justine Tunney ce17ed60df Support -MF and -MT in cosmocc 2024-01-06 12:10:06 -08:00
Jōshin d27a47b0e2 Bugfix: ape --help should exit 0 (#1060) 2024-01-06 12:07:32 -08:00
Jōshin 390335eb45 apeinstall/uninstall.sh can use doas (#1062) 2024-01-06 12:06:21 -08:00
Jōshin 492a8f4f53 Fix make o/aarch64/ape on aarch64 (#1064) 2024-01-06 11:42:39 -08:00
Jōshin 636bc4007b Enable argv[0] tests in more places (#1061)
Now we do them for assimilated binaries (except on OpenBSD or XNU
non-Silicon), for XnuSilicon, and for binaries with the preserve-
argv[0] auxv flag set. We check whether to pass the argv[0] value
at the test site rather than the Child site. We move a lot of the
test initialization into Child in the non-child case, in order to
get at the pre-init value of `__program_executable_name`. Finally,
we print out info about what we are skipping.
2024-01-06 11:42:03 -08:00
Trung Nguyen b09096691a third_party: Add libunwind (#1053)
Added libunwind from LLVM 17.0.6.
The library includes functions required for C++ exception handling.
2024-01-06 15:04:30 +07:00
Justine Tunney 91de6f1f5d Release Cosmopolitan 3.2.2 2024-01-05 21:00:22 -08:00
Justine Tunney fad1279c61 Make cosmo_dlopen() safer and faster
If cosmo_dlopen() is linked on AMD64 then the runtime will switch to
using %gs for thread-local storage. This eliminates the need for the
imported symbol trampoline. It's now safer to pass function pointers
back and forth with imported libraries. Your program gets recompiled
at runtime to make it happen and the overhead is a few milliseconds.
2024-01-05 20:42:19 -08:00
Jōshin c0eacf2eb1 clang-format 2024-01-05 23:21:52 -05:00
Justine Tunney c60b150fcf Release Cosmopolitan v3.2.1 2024-01-05 15:36:57 -08:00
Jōshin 80ec6c9283 Try to detect kernel version for P flag (#1059) 2024-01-05 15:19:46 -08:00
Justine Tunney 2d93788ce3 Fix --ftrace with cosmo_dlopen()
This change ensures function call logging won't crash the process when
cosmo_dlopen() is called.
2024-01-05 15:13:07 -08:00
Jōshin 15548b523c Cleanup apeinstall.sh (#1057) 2024-01-05 13:45:14 -08:00
Jōshin 412a200ae4 Support binfmt_misc P flag in APE loader (#1058)
This allows ape to automatically preserve `argv[0]` [as of Linux kernel
5.12][0] if the [binfmt_misc][1] registration contains the P flag.

This also removes may_path_search, which was identical to the literally
flag in usage. As a result, FindCommand is subsumed into Commandv.

[0]: https://patchew.org/QEMU/[email protected]/
[1]: https://www.kernel.org/doc/html/latest/admin-guide/binfmt-misc.html
2024-01-05 12:35:01 -08:00
Justine Tunney 44a463e4d2 Invent *cosmo_dltramp() function 2024-01-05 09:11:51 -08:00
Justine Tunney 0de6a08988 Upgrade mono repo to cosmocc 3.2
The toolchain will now be downloaded going forward from multiple pinned
URLs which have shasums. Either wget or curl must be installed.

This change unblocks #1053
2024-01-05 08:02:04 -08:00
Paul Kulchenko a334f9cc33 Update redbean ProgramDirectory to return a list of previously set directories (#1021) 2024-01-05 02:17:55 -08:00
Jōshin 3315b6ef11 Store StrlenDevFd in n (#1056) 2024-01-04 23:17:37 -05:00
Justine Tunney e93c8c4f8f Fix typo 2024-01-04 13:59:36 -08:00
Justine Tunney 0e49bed660 Support 40 cosmo_dlopen() function parameters
Our dynamic linking implementation is now able to support functions with
dozens of parameters. In addition to having extra integral arguments you
can now pass vector registers using intrinsic types. Lastly, you can now
return multiple values, which is useful for functions returning structs.
2024-01-04 13:41:26 -08:00
Justine Tunney a3deef70c2 Release Cosmopolitan v3.2 2024-01-04 09:39:48 -08:00
Justine Tunney 873069fcd7 Fix -Xaarch64 in cosmocc
It's now possible to pass flags like -Xaarch64-march=armv8.2-a+dotprod
so that cosmocc will use newer ARM ISAs. For AMD64 there's another one
worth mentioning, which looks like this: -Xx86_64-mssse3
2024-01-04 07:22:05 -08:00
Justine Tunney 3864f78b88 Give GNU Make limitless stacks 2024-01-04 03:26:26 -08:00
Justine Tunney 796148790f Remove hard coded paths from APE bootloader
This increases risk of fork bomb but is needed to support the NixOS.
Upstream dependencies of APE (uname, mkdir, dd, chmod, gzip, and mv)
will be removed from releases, and deleted from the cosmo.zip server

See #12
2024-01-03 17:55:19 -08:00
Justine Tunney 34ed1fcbea Fix bugs with DNS library on Windows 2024-01-03 17:39:57 -08:00
Justine Tunney 1bb52c223b Add missing build dependency 2024-01-03 17:04:21 -08:00
Justine Tunney 31dcdefaa8 Update bootstrap binaries
It's now possible to build the monorepo on FreeBSD ARM64.
2024-01-03 17:03:51 -08:00
Jōshin 2902b76168 mmake gets smarter about locating make
- Check `$COSMOCC`, defaulting to `/opt/cosmocc`.

- Try to get the full path of the repo `make.com`. I'm not aware of
  a way of getting the path that defines a zsh function, so the best
  fallback available is `$PWD`.
2024-01-01 23:34:35 -05:00
Justine Tunney 307823ae00 Upgrade to superconfigure z0.0.29 2024-01-01 19:31:58 -08:00
Jōshin 9801f2d1db Remove COSMOPOLITAN_PROGRAM_EXECUTABLE (#1047) 2024-01-01 07:25:16 -08:00
Jōshin 68dbe5312f Prepend getcwd to exename early in init (#1048) 2024-01-01 07:23:23 -08:00
Justine Tunney 2f89c2482a Delete some dead code 2024-01-01 00:13:16 -08:00
Justine Tunney 9e6faa5256 Fix --ftrace on Windows 2024-01-01 00:00:42 -08:00
Jōshin b02d13ccde Re-enable/fix non-argv[0] exe name tests (#1046) 2023-12-31 11:53:15 -08:00
Justine Tunney 81949f038e Mint APE Loader v1.10 2023-12-31 11:43:13 -08:00
Jōshin c9550afe5e Fix loader usage, shave off a few bytes (#1016)
* Remove -f from loader usage

-f was removed in 1.5. As there is now only one flag, a couple more
bytes can be shaved off as well.

* Further loader golf

Shaves off a few bytes, paying for the cost of `RealPath` and then some
on x86_64 and offsetting some of the cost to aarch64.

* Shave off a few more bytes

Removes `-h` and flags from usage. Keeps flag-parsing logic the same,
i.e. still accepts `-h` / `--help`. Only difference is what fd and rc
the usage uses.

Still over 1k north of 8192.
2023-12-31 11:33:42 -08:00
Jōshin 10b4693e37 Sanity-check set-id interpreter script paths (#1029) 2023-12-31 07:46:27 -08:00
Jōshin 14fe83facd aarch64 loader passes os (#1042)
* Reorder Launch arguments, pass aarch64 os

Third and fourth arguments are now identical between cosmo and Launch.
By passing sp as argument 4, we save a bit of register juggling.

Fourth argument (os) is now always passed by the loader on aarch64. It
is not yet processed by cosmo. Pushing this change separately, as the
cosmo side turns out to be somewhat more involved.

* cosmo2 receives os from loader

FreeBSD aarch64 now traps early rather than pretending to be Linux.
o/aarch64/examples/env.com still works on Linux and Xnu.
2023-12-31 06:42:36 -08:00
Justine Tunney d8ad34686a Implement issetugid() on NetBSD 2023-12-30 14:58:16 -08:00
Justine Tunney 83107f78ed Introduce FreeBSD ARM64 support
It's 100% passing test fleet. Solid as a rock.
2023-12-29 20:14:02 -08:00
Justine Tunney 43fe5956ad Use DNS implementation from Musl Libc
Now that our socket system call polyfills are good enough to support
Musl's DNS library we should be using that rather than the barebones
domain name system implementation we rolled on our own. There's many
benefits to making this change. So many, that I myself wouldn't feel
qualified to enumerate them all. The Musl DNS code had to be changed
in order to support Windows of course, which looks very solid so far
2023-12-28 23:04:35 -08:00
Justine Tunney 1a28e35c62 Use good locks in dlmalloc
Using mere spin locks causes runitd.com to go painstakingly slow on
NetBSD for reasons that aren't clear yet.
2023-12-28 04:57:36 -08:00
Justine Tunney 80fca1f7c3 Make getprogramexecutablename_test pass fleet 2023-12-28 04:57:36 -08:00
Justine Tunney 8b0e42a31b Make zipos_test hermetic 2023-12-28 04:57:36 -08:00
Paul Kulchenko 6be9477b9e Fix redbean to only check additional content types when Lua is present (#1033)
Closes #971
2023-12-28 01:08:19 -08:00
Justine Tunney ce0143e2a1 Fix madvise() on Windows 2023-12-27 22:41:46 -08:00
Justine Tunney f51fd97644 Fix MODE=dbg build break 2023-12-27 22:24:27 -08:00
Jōshin 5c35863524 Rename __zipos_free -> __zipos_drop (#1043)
Removes the separate decref function, uses keep/drop in the internal
API.
2023-12-26 12:08:57 -05:00
Jōshin fd772b9b2a mmake uses $MAKE if available 2023-12-26 10:27:04 -05:00
Jōshin 4c6f928fbf Disable argv[0] tests on non-XnuSilicon (#1015)
Adds a TODO explaining the correct condition.
2023-12-24 19:43:32 -05:00
Jōshin 25266b037b Save a redundant load in zipos read/seek (#1037)
When h->pos has changed to something other than SIZE_MAX, we don't need
the extra atomic load.
2023-12-24 19:40:18 -05:00
Jōshin bb2602a524 Minor mmake cleanup 2023-12-23 11:09:50 -05:00
mataha 1bc48bc8e4 Update stb (#885)
This commit and, by extension, PR attempts to update `stb` in the most
straightforward way possible as well as include fixes from main repo's
unmerged PRs for cases rearing their ugly heads during everyday usage:

 - stb#1299: stb_rect_pack: Make rect_height_compare a stable sort
 - stb#1402: stb_image: Fix "unused invalid_chunk" with STBI_FAILURE_USERMSG
 - stb#1404: stb_image: Fix gif two_back memory address
 - stb#1420: stb_image: Improve error reporting if file operations fail
   within *_from_file functions
 - stb#1445: stb_vorbis: Few static analyzers fixes
 - stb#1487: stb_vorbis: Fix residue classdata bounding for
   f->temp_memory_required
 - stb#1490: stb_vorbis: Fix broken clamp in codebook_decode_deinterleave_repeat
 - stb#1496: stb_image: Fix pnm only build
 - stb#1497: stb_image: Fix memory leaks if stbi__convert failed
 - stb#1498: stb_vorbis: Fix memory leaks in stb_vorbis
 - stb#1499: stb_vorbis: Minor change to prevent the undefined behavior -
   left shift of a negative value
 - stb#1500: stb_vorbis: Fix signed integer overflow

Includes additional small fixes that I felt didn't warrant a separate PR.
2023-12-22 21:39:27 -08:00
Jōshin 7faffde303 Fix buffer overflow in TryPath (#1035)
Missed this when changing the code back to be like the old version.
com is now a parameter.

The only plausible way to trigger this would be to pass a loader
pathname close to MAX_PATH characters long, and then remove that
path prior to the first sys_faccessat.
2023-12-18 23:02:16 -05:00
Jōshin 85e4c020a3 mmake: formatting + aarch64 tab-completion helper
o/aarch64/* is turned into o/$mode/* in a way analogous to o//, so
aarch64 systems can use tab complete for alternate modes just like
x86_64 can.
2023-12-18 21:29:32 -05:00
Jōshin 1280797db2 Fix indentation, make llvm happy (#1032)
clang says "label at end of switch statement is a C2x extension."
2023-12-18 20:48:33 -05:00
Jōshin 58b620d8f7 Add missing bash build dependency (#1031) 2023-12-18 17:07:34 -05:00
Jōshin 2a11a09d98 Remove realpath/getcwd from loaders (#1024)
This implements proposals 1 and 2a from this gist:

https://gist.github.com/mrdomino/2222cab61715fd527e82e036ba4156b1

The only reason to use realpath from the loader was to try to prevent a
TOCTOU between the loader and the binary. But this is only a real issue
in set-id contexts, and in those cases there is already a canonical way
to do it: `/dev/fd`, passed by the kernel to the loader, so all we have
to do is pass that along to the binary.

Aside from realpath, there is no reason to absolutize the path we supply
to the binary, since it can call `getcwd` as well as we can, and on non-
M1 the binary is in a much better position to make that call.

Since we no longer absolutize the path, the binary does need to do this,
so we make its argv-parsing code generic and apply that to the different
possible places the path could come from. This means that `_` is finally
usable as a relative path, as a nice side benefit.

The M1 realpath code had a significant bug - it uses the wrong offset to
truncate the `.ape` in the `$prog.ape` case.

This PR also fixes a regression in `ape $progname` out of `$PATH` on the
two BSDs (Free and Net) that did not implement `RealPath`.
2023-12-18 15:01:16 -05:00
Jōshin f73576ab8a mmake knows about some .PHONY targets 2023-12-17 17:04:57 -05:00
Jōshin dd57c685aa mmake gets smarter
`o/$mode/*` is passed through as-is. `o/*` other than `$mode` has
`$mode` inserted. `*` has `o/$mode/` prepended.

Really leveraging zsh default tab completion here; if you have built
things with `MODE=` you can leverage that for perfect tab completion
in other modes.
2023-12-17 15:42:27 -05:00
Jōshin c11c105270 mmake doesn't even run nproc unless it needs to 2023-12-17 15:09:41 -05:00
Jōshin 3654650c52 mmake only autoloads nproc if needed 2023-12-17 15:07:10 -05:00
Jōshin 77facc4f7b Cover more cases for detecting ape loader (#1022)
Fixes a regression in GetProgramExecutableName on Linux against old
loaders. In the loader case, /proc/self/exe gives the loader's path.
We tried to detect this by checking for `/usr/bin/ape`. But that is
only one of the possible places the loader could be.
2023-12-17 10:30:22 -05:00
Jōshin b08b92248b mmake: -j$(nproc), passthrough o//
Now `mmake o/ape/ape.elf` works.
2023-12-17 01:55:52 -05:00
Jōshin 2b315626f3 Revert retabbing of net/http and tinymath (#1020) 2023-12-16 23:59:11 -05:00
Jōshin 3a8e01a77a more modeline errata (#1019)
Somehow or another, I previously had missed `BUILD.mk` files.

In the process I found a few straggler cases where the modeline was
different from the file, including one very involved manual fix where a
file had been treated like it was ts=2 and ts=8 on separate occasions.

The commit history in the PR shows the gory details; the BUILD.mk was
automated, everything else was mostly manual.
2023-12-16 23:07:10 -05:00
Jōshin 60813003a3 tool/zsh/mmake cleanup
- collects all flags into their own array to support `mmake $target -j8`

- prints out the raw make invocation to be run
2023-12-16 19:22:17 -05:00
Jōshin c0bbc45268 Add mmake zsh helper function (#1017) 2023-12-16 08:47:38 -08:00
Jōshin f94c11d978 Loader path security (#1012)
The ape loader now passes the program executable name directly as a
register. `x2` is used on aarch64, `%rdx` on x86_64. This is passed
as the third argument to `cosmo()` (M1) or `Launch` (non-M1) and is
assigned to the global `__program_executable_name`.

`GetProgramExecutableName` now returns this global's value, setting
it if it is initially null. `InitProgramExecutableName` first tries
exotic, secure methods: `KERN_PROC_PATHNAME` on FreeBSD/NetBSD, and
`/proc` on Linux. If those produce a reasonable response (i.e., not
`"/usr/bin/ape"`, which happens with the loader before this change),
that is used. Otherwise, if `issetugid()`, the empty string is used.
Otherwise, the old argv/envp parsing code is run.

The value returned from the loader is always the full absolute path
of the binary to be executed, having passed through `realpath`. For
the non-M1 loader, this necessitated writing `RealPath`, which uses
`readlinkat` of `"/proc/self/fd/[progfd]"` on Linux, `F_GETPATH` on
Xnu, and the `__realpath` syscall on OpenBSD. On FreeBSD/NetBSD, it
punts to `GetProgramExecutableName`, which is secure on those OSes.

With the loader, all platforms now have a secure program executable
name. With no loader or an old loader, everything still works as it
did, but setuid/setgid is not supported if the insecure pathfinding
code would have been needed.

Fixes #991.
2023-12-15 12:23:58 -05:00
Jōshin 8a10ccf9c4 Fix ZipOS deadlock/segfault (#1011)
This change adds a new stress test for ZipOS which helped
us improve the locking semantics in open() and close().
2023-12-14 19:59:20 -08:00
Justine Tunney 897fa6ac00 Add some Emacs keywords for CUDA 2023-12-14 19:55:56 -08:00
Jōshin 8dbfb77890 tolower() on terminfo directories (#1009)
Fixes the ncurses build on my M1 Mac. Required a make clean.
2023-12-13 12:26:12 -05:00
Jōshin 7a05fb43ac Assume __zipos_close when closing a zipos fd 2023-12-13 10:07:32 -05:00
Jōshin 3b302e6379 Cleanup zipos vfork (#1004) 2023-12-13 01:36:44 -08:00
Jōshin 6cedbc746c Fix type of refs 2023-12-13 03:03:25 -05:00
Jōshin 2fc507c98f Fix more vi modelines (#1006)
* modelines: tw -> sw

shiftwidth, not textwidth.

* space-surround modelines

* fix irregular modelines

* Fix modeline in titlegen.c
2023-12-13 02:28:11 -05:00
Justine Tunney bd10cf9d38 Whip up find command for PE binaries
Using this shell script:

    #!/bin/sh
    mkdir -p exe
    for f in $(findpe); do
      if [ -e exe/${f##*/}.exe ]; then
        cp $f exe/${f##*/}-$(rand64).exe
      else
        cp $f exe/${f##*/}.exe
      fi
    done
    rm -f /mnt/videos/microsoft.zip
    zip -rj6 /mnt/videos/microsoft.zip exe
    echo /mnt/videos/microsoft.zip

Helps file reports with Microsoft about incorrect AV detections.

See #1003
2023-12-12 10:29:03 -08:00
Justine Tunney ae85fb74b2 Update README.md
Fixes #1002
2023-12-12 07:53:23 -08:00
Justine Tunney f4a2a65cc8 Add bash 5.2 to third_party 2023-12-11 07:38:17 -08:00
Justine Tunney e1f65d5f2b Add readline 8.2 to third_party 2023-12-11 05:59:20 -08:00
Justine Tunney cbe72c4339 Fix ncurses yoinks 2023-12-10 13:11:45 -08:00
Justine Tunney 6744de659e Remove awful terminfo definitions
These files collide on MacOS and Windows due to case insensitivity.
2023-12-10 11:28:22 -08:00
Justine Tunney 4db6b74de7 Fix build warning error 2023-12-10 08:26:36 -08:00
Justine Tunney 128be2bd2e Add less 643 2023-12-10 08:01:37 -08:00
Justine Tunney 0ed7309fdb Add ncurses 6.4 2023-12-10 07:33:37 -08:00
Justine Tunney 6ee5580adc Delete GGML and RadPajama
Please use https://github.com/mozilla-Ocho/llamafile which is better,
newer, and built on cosmocc. If you need the RadPajama model, file an
issue with llamafile asking for support.
2023-12-10 06:08:54 -08:00
Justine Tunney 4a63e4a27e Reduce header complexity of PCRE2 2023-12-10 06:08:25 -08:00
Justine Tunney b27e9fe845 Add PCRE2 library to third_party 2023-12-10 05:59:18 -08:00
Justine Tunney 4f66d7f2dd Add WIN32 pseudo console APIs
See #999
2023-12-10 01:29:25 -08:00
Justine Tunney 1bd69867d1 Explain how to build FOSS in cosmocc README 2023-12-09 06:58:24 -08:00
Joseph Battelle b0cced855d Move tcgetpgrp and tcsetpgrp decls to unistd.h (#996) (#997) 2023-12-08 22:54:46 -08:00
Justine Tunney 076e0bf9a9 Re-disable WIN32 call tracing 2023-12-08 20:06:16 -08:00
Justine Tunney 1a96de6eda Add libresolv from Musl Libc
Locally modified to get nameservers from Windows Registry when
`\etc\resolv.conf` isn't defined.
2023-12-08 20:04:10 -08:00
Justine Tunney 8874a37abc Add <link.h> for absl 2023-12-08 20:04:10 -08:00
Zoey Greer a11016e590 Use gid_t and uid_t in passwd.h (#995) 2023-12-08 18:27:41 -08:00
Jōshin e16a7d8f3b flip et / noet in modelines
`et` means `expandtab`.

```sh
rg 'vi: .* :vi' -l -0 | \
  xargs -0 sed -i '' 's/vi: \(.*\) et\(.*\)  :vi/vi: \1 xoet\2:vi/'
rg 'vi: .*  :vi' -l -0 | \
  xargs -0 sed -i '' 's/vi: \(.*\)noet\(.*\):vi/vi: \1et\2  :vi/'
rg 'vi: .*  :vi' -l -0 | \
  xargs -0 sed -i '' 's/vi: \(.*\)xoet\(.*\):vi/vi: \1noet\2:vi/'
```
2023-12-07 22:17:11 -05:00
Justine Tunney 90c53f3064 Fix some references to stdio/internal.h 2023-12-07 16:30:49 -08:00
Justine Tunney f8ea02d4d1 Import /etc/shadow support from Musl for Linux
Fixes #992
2023-12-07 16:26:04 -08:00
Justine Tunney 06ace4e7b4 Always use compiler builtin for offsetof()
We have received multiple reports of GCC breaking builds when compiler
flags like `-std=c11` were being passed. The workaround until the next
release is to simply not define `__STRICT_ANSI__` which is a bad idea.
2023-12-07 14:20:12 -08:00
Justine Tunney e36283f1d9 Support double parameters in dlsym() imports 2023-12-06 06:49:35 -08:00
Justine Tunney 4772796cd8 Improve cosmo_dlopen() documentation 2023-12-06 06:33:50 -08:00
Justine Tunney 7c39818c13 Simplify GNU Make build config 2023-12-06 03:25:16 -08:00
Jōshin 394d998315 Fix vi modelines (#989)
At least in neovim, `│vi:` is not recognized as a modeline because it
has no preceding whitespace. After fixing this, opening a file yields
an error because `net` is not an option. (`noet`, however, is.)
2023-12-05 14:37:54 -08:00
Justine Tunney 221a27836b Define max_align_t in <stddef.h>
It was previously only being defined in <cstddef>

See oilshell/oil#1760
2023-12-05 00:52:33 -08:00
Justine Tunney 536dfb947a Make getprogramexecutablename_test report status
Fixes #988
2023-12-04 23:19:28 -08:00
Justine Tunney cd52c59552 Allow MAP_POPULATE under pledge() 2023-12-04 23:15:41 -08:00
Jōshin 53357aa26a Fix __zipos_close (#984) 2023-12-04 20:24:26 -08:00
Jōshin 577bb180b7 benchmark, getenv, test cleanup (#987) 2023-12-04 20:01:52 -08:00
Jōshin da8baf2aa5 ape-m1 minor formatting cleanup (#986) 2023-12-04 19:58:32 -08:00
Jōshin ed8fadea37 Keep argv[0], add COSMOPOLITAN_PROGRAM_EXECUTABLE (#980)
* Introduce env.com

Handy tool for debugging environment issues.

* Inject path as COSMOPOLITAN_PROGRAM_EXECUTABLE

`argv[0]` was previously being used as a communication channel between
the loader and the binary, giving the binary its full path for use e.g.
in `GetProgramExecutableName`. But `argv[0]` is not a good channel for
this; much of what made 2a3813c6 so gross is due to that.

This change fixes the issue by preserving `argv[0]` and establishing a
new communication channel: `COSMOPOLITAN_PROGRAM_EXECUTABLE`.

The M1 loader will always set this as the first variable. Linux should
soon follow. On the other side, `GetProgramExecutableName` checks that
variable first. If it sees it, it trusts it as-is.

A lot of the churn in `ape/ape-m1.c` in this change is actually backing
out hacks introduced in 2a3813c6; the best comparison is:

    git diff 2a3813c6^..
2023-12-04 12:45:46 -08:00
Jōshin 2a3813c6cf $prog.ape support (#977)
* ape loader: $prog.ape + login shell support

If the ape loader is invoked with `$0 = $prog.ape`, then it searches for
a `$prog` in the same directory as it and loads that. In particular, the
loader searches the `PATH` for an executable named `$prog.ape`, then for
an executable named `$prog` in the same directory. If the former but not
the latter is found, the search terminates with an error.

It also handles the special case of getting started as `-$SHELL`, which
getty uses to indicate that the shell is a login shell. The path is not
searched in this case, and the program location is read straight out of
the `SHELL` variable.

It is now possible to have `/usr/local/bin/zsh.ape` act as a login shell
for a `/usr/local/bin/zsh` αpε, insofar as the program will get started
with the 'correct' args. Unfortunately, many things break if `$0` is not
the actual full path of the executable being run; for example, backspace
does not update the display properly.

To work around the brokenness introduced by not having `$0` be the full
path of the binary, we cut the leading `-` out of `argv[0]` if present.
This gets the loader's behavior with `$prog.ape` up to par, but doesn't
tell login shells that they are login shells.

So we introduce a hack to accomplish that: if ape is run as `-$prog.ape`
and the shell is `$prog`, the binary that is loaded has a `-l` flag put
into its first argument.

As of this commit, αpε binaries can be used as login shells on OSX.

* if islogin, execfn = shell

Prior to this, execfn was not being properly set for login shells that
did not receive `$_`, which was the case for iTerm2 on Mac. There were
no observable consequences of this, but fixing it seems good anyway.

* Fix auxv location calculation

In the non-login-shell case, it was leaving a word of uninitialized
memory at `envp[i] + 1`. This reuses the previous calculation based
on `envp`.
2023-12-03 19:39:32 -08:00
Jōshin 8dd348067b refs is atomic_size_t (#976)
The unassert covers all invalid values rather than a third of them.
2023-12-02 16:48:58 -08:00
Justine Tunney 450d9b1d43 Release Cosmopolitan v3.1.3 2023-12-01 10:29:59 -08:00
Justine Tunney 69ba516466 Revert "Re-enable apelink size optimization"
This reverts commit 7b3d7ee07b.

It broke llava-v1.5-7b-q4-server.llamafile on Windows.
2023-12-01 10:26:17 -08:00
Justine Tunney 9e9d2c0006 Release Cosmopolitan v3.1.2 2023-12-01 06:47:57 -08:00
Jōshin a0f599ce12 Fix memory_order_release in __zipos_seek (#975) 2023-12-01 02:28:15 -08:00
Jōshin d95d61b1af Better zipos refcounts and atomic reads/seeks (#973)
* Better refcounting

Cribbed from [Rust Arc][1] and the [Boost docs][2]:

"""
Increasing the reference counter can always be done with
memory_order_relaxed: New references to an object can only be formed
from an existing reference, and passing an existing reference from one
thread to another must already provide any required synchronization.

It is important to enforce any possible access to the object in one
thread (through an existing reference) to happen before deleting the
object in a different thread. This is achieved by a "release" operation
after dropping a reference (any access to the object through this
reference must obviously happened before), and an "acquire" operation
before deleting the object.

It would be possible to use memory_order_acq_rel for the fetch_sub
operation, but this results in unneeded "acquire" operations when the
reference counter does not yet reach zero and may impose a performance
penalty.
"""

[1] https://moshg.github.io/rust-std-ja/src/alloc/arc.rs.html
[2] https://www.boost.org/doc/libs/1_55_0/doc/html/atomic/usage_examples.html

* Make ZiposHandle's pos atomic

Implements a somewhat stronger guarantee than POSIX specifies: reads and
seeks are atomic. They may be arbitrarily reordered between threads, but
each one happens all the way and leaves the fd in a consistent state.

This is achieved by "locking" pos in __zipos_read by storing SIZE_MAX to
pos during the operation, so only one can be in-flight at a time. Seeks,
on the other hand, just update pos in one go, and rerun if it changed in
the meantime.

I used `LIKELY` / `UNLIKELY` to pessimize the concurrent case; hopefully
that buys back some performance.
2023-12-01 01:01:03 -08:00
Jōshin f0bfabba07 Revert "Remove noop __zipos_postdup" (#974)
`close` does not modify the fd table, so `rc` can potentially have a
stale zipos object sitting on it. As such, we need `__zipos_postdup`
there.
2023-12-01 00:51:01 -08:00
Jōshin 7845495a93 Get tool/cosmocc/package.sh working on M1 (#970)
On aarch64 hosts, MODE= is changed to MODE=aarch64, so o// targets don't
work. So On aarch64, get apelink.com out of o/aarch64/. Also prepend ape
when calling it. And finally, fetch with curl when wget isn't installed.
2023-12-01 00:08:55 -08:00
Jōshin d1a745c17c Implement __zipos_dup (#972)
* Implement __zipos_dup

Makes ZiposHandle reference-counted by an `rc` field in a union with its
freelist `next` pointer. The functions `__zipos_free` and `__zipos_keep`
function as incref/decref for it. Adds `__zipos_postdup` to fix metadata
on file descriptors after dup-like operations, and adds zipos support to
`sys_dup_nt` + `sys_close_nt`.

* Remove noop __zipos_postdup

rc is never a zipos file because it is always a previously unused file
descriptor. fd is never a zipos file because that case has been handled
above by __zipos_fcntl.
2023-12-01 00:08:30 -08:00
Justine Tunney 6556dd2673 Add $(uniq token...) native function to Make
This reduces mono repo `make` latency from 336ms to 226ms.
2023-11-30 21:02:16 -08:00
Justine Tunney 14bf57180f Import GNU Make 4.4.1
Landlock Make hasn't been working well on AARCH64 systems. Let's do this
over the right way, using our new build tools.
2023-11-30 20:54:52 -08:00
Justine Tunney 9315ebbfd9 Fix MODE=tinylinux build 2023-11-30 15:38:08 -08:00
mataha 879bb84244 Update quickjs (#890)
Includes additional fixes from main repo's unmerged PRs:

 - quickjs#132: Fix undefined behavior: shift 32 bits for uint32_t in bf_set_ui
 - quickjs#171: Fix locale-aware representation of hours in Date class
 - quickjs#182: Fix stack overflow in CVE-2023-31922
2023-11-30 10:51:16 -08:00
Justine Tunney 4b7ba9a4c5 Fix some bugs with dup2() and ZipOS
On UNIX if dup2(newfd) was a ZipOS file descriptor, then its resources
weren't being released, and the newly created file descriptor would be
mistaken for ZipOS due to its memory not being cleared. On Windows, an
issue also existed relating to newfd resources not being released.
2023-11-30 10:10:02 -08:00
Justine Tunney 26c70e08bf Make it possible to package.sh cosmocc w/ modes 2023-11-30 07:12:15 -08:00
Justine Tunney 20938a45e6 Add appropriate chmod to README instructions
See Mozilla-Ocho/llamafile#13
2023-11-30 03:35:17 -08:00
Justine Tunney 7b3d7ee07b Re-enable apelink size optimization
I'm no longer able to reproduce the PE import table corruption that was
previously observed. Since this optimization shaves up to 64kb off each
fat APE binary we build, it's worth turning this back on again, to wait
and see if something breaks, and if so, fix it. At least until the next
release is shipped.

See #965
2023-11-29 05:41:23 -08:00
Justine Tunney ff955aaa01 Make memcmp() and memchr() go fast again
Readahead within the specified size is legal, even if it overlaps a page
boundary; it's the fault of the caller if that causes a segfault.
2023-11-29 05:17:21 -08:00
Justine Tunney 70155df7a9 Avoid linking win32 signals impl
This shaves ~4kb off o/tiny/examples/hello2.com by avoiding linking the
WIN32 signals polyfill unless sigaction() is being used.

See #965
2023-11-29 04:09:31 -08:00
Justine Tunney 2b960bb249 Exclude strace from MODE=tiny builds
This change gets o/tinylinux/examples/hello2.com back down to 8kb in
size which had been unintentionally bloated to 40kb in recent months

See #965
2023-11-29 03:45:54 -08:00
Justine Tunney b7e1dc81c2 Release Cosmopolitan v3.1.1 2023-11-29 02:19:56 -08:00
Justine Tunney 41c06c6325 Make cosmocc 100x faster on MacOS
This change upgrades to superconfigure z0.0.23 which fixes an issue
where the compiler had harmless /home/... paths baked-in, which are
normally only present in the build environment, and usually skipped
over. Sadly on MacOS calling fstatat() on these paths would lead to
cloud file system ops that caused system calls to take a long time.
That's problematic, since cosmocc needs to be a 100% local command.
2023-11-29 01:48:17 -08:00
Justine Tunney 4427581a05 Cleanup inline locking code 2023-11-29 00:36:22 -08:00
Justine Tunney 96185e1ac0 Fix MODE=aarch64 cross-compiling
This change fixes a regression that happened some time ago when building
for AARCH64 using the vendored toolchain rather than cosmocc. The errors
that would show up `Relocations in generic ELF (EM: 62)` have been fixed
2023-11-28 22:40:06 -08:00
Justine Tunney ca5e361919 Don't show (win32 error 0) 2023-11-28 21:30:20 -08:00
Justine Tunney 9cad66aa11 Freshen bootstrap binaries for MacOS + SIP 2023-11-28 21:18:31 -08:00
Justine Tunney 03de761890 Update cosmocc to use superconfigure z0.0.23 2023-11-28 20:04:15 -08:00
Justine Tunney 43de12db55 Introduce forkpty() 2023-11-28 19:59:57 -08:00
Justine Tunney fa20edc44d Reduce header complexity
- Remove most __ASSEMBLER__ __LINKER__ ifdefs
- Rename libc/intrin/bits.h to libc/serialize.h
- Block pthread cancelation in fchmodat() polyfill
- Remove `clang-format off` statements in third_party
2023-11-28 14:39:42 -08:00
Justine Tunney 96f979dfc5 Rename makefiles BUILD.mk
This way they appear at the top of directory listings.
2023-11-28 11:21:08 -08:00
Paul Kulchenko 0bffd09433 Allow redbean to accept --assimilate option without complaining (#916)
This allows using --assimilate on Windows or already assimilated binary.
Ref #356.
2023-11-19 19:02:30 -08:00
Stephen Gregoratto cc5c5319bf Linux: Add cachestat, fchmodat2 syscalls (#958) 2023-11-19 19:01:20 -08:00
Justine Tunney 69faf1b403 Release Cosmpolitan v3.1 2023-11-18 20:40:35 -08:00
Justine Tunney e4dea37b8e Make clock_nanosleep() cancel faster 2023-11-18 18:12:09 -08:00
Justine Tunney 0c89516ac5 Fix unzip warning with apelink generated symtabs 2023-11-18 17:15:22 -08:00
Justine Tunney 3e6d536822 Correct misunderstanding with zip64 extra records 2023-11-18 14:35:57 -08:00
Justine Tunney dbd8176ea8 Add CPU_COUNT_S() 2023-11-18 12:38:30 -08:00
Justine Tunney 545a8f4cb0 Add more sched.h content 2023-11-18 08:08:15 -08:00
Justine Tunney f7cfe03888 Fix dlopen() for FreeBSD and NetBSD 2023-11-18 04:35:48 -08:00
Justine Tunney 2c1efd3d78 Fix Python build break 2023-11-18 02:32:01 -08:00
Matheus Moreira 3ac473df3b Floating point parsing support for scanf family (#924) 2023-11-18 02:25:36 -08:00
Justine Tunney 8caf1b48a9 Improve time/sleep accuracy on Windows
It's now almost as good as Linux thanks to a Windows 8+ API.
2023-11-18 01:57:44 -08:00
Justine Tunney 72ac5f18d9 Fix dash compatible syntax issue 2023-11-17 16:47:47 -08:00
Justine Tunney 2c9d2943d6 Introduce AddDllDirectory() 2023-11-17 10:35:34 -08:00
Justine Tunney 32b97f2d25 Improve execve() path argument munging
Munging of paths passed inside the system() interpreter command is no
longer supported. You have to pass your paths to posix_spawn() or the
execve() family of functions if you want them to be munged. The first
three characters must match `^/[a-z]/` in which case, it'll be turned
into a DOS-style drive path with backslashes.
2023-11-17 09:59:03 -08:00
Justine Tunney 529cb4817c Improve dlopen() on Apple Silicon
- Introduce MAP_JIT which is zero on other platforms
- Invent __jit_begin() and __jit_end() which wrap Apple's APIs
- Runtime dispatch to sys_icache_invalidate() in __clear_cache()
2023-11-17 02:33:14 -08:00
Justine Tunney 7a9e176ecf Improve debug binary location detection 2023-11-17 00:07:39 -08:00
Justine Tunney 68c7c9c1e0 Clean up some code
- Use good ELF technique in cosmo_dlopen()
- Make strerror() conform more to other libc impls
- Introduce __clear_cache() and use it in cosmo_dlopen()
- Remove libc/fmt/fmt.h header (trying to kill off LIBC_FMT)
2023-11-16 17:31:07 -08:00
tkchia 7010a8081e Fix regression when building deathstar.com for aarch64 (#957)
It turns out my earlier commit ddc08dc974 caused a build with
MODE=aarch64 to fail.  The commit changed deathstar.c to link
in code to support a VGA console, but this is not implemented
yet for AArch64.  Thanks to @ahgamut for spotting this issue.
2023-11-16 12:13:17 -08:00
tkchia 665747a319 Make test/libc/calls/pledge_test.c work with "nonstandard" umask (#956)
The test was failing if the process's umask happened to be
0077, for example.  The file `foo` was then created with a
file mode of 0100600, rather than the expected 0100644.
2023-11-16 00:47:21 -08:00
Justine Tunney e05933c641 Mention zsh and fish in readme 2023-11-15 23:25:47 -08:00
Justine Tunney 8f5e516b39 Remove sync_file_range()
After hearing horror stories from a trusted colleague, I don't think
this is the kind of API we want to be supporting. Also SQLite wisdom
regarding fdatasync() has been added to the documentation.
2023-11-15 23:21:22 -08:00
Justine Tunney 8318d67503 Fix regression in cosmocc toolchain 2023-11-15 21:41:53 -08:00
Justine Tunney 1351d3cede Remove bool from public headers 2023-11-15 20:58:46 -08:00
Justine Tunney dffee606cf Run host compiler to create dlopen helper 2023-11-15 20:58:46 -08:00
tkchia ddc08dc974 [metal] Make deathstar.com draw onto both VGA and serial consoles (#955)
In addition, deathstar.com can now be booted up via UEFI.
2023-11-15 19:40:54 -08:00
Justine Tunney e4584ace81 Get cosmo_dlopen() working better on System Five
Imported functions are now aspected with a trampoline that blocks
signals and changes the thread-local storage register. This means
bigger more complicated libraries can now be imported even though
the whole technique remains fundamentally unsafe.
2023-11-15 10:56:30 -08:00
Justine Tunney 3a470ed356 Remove some exploratory some tests
These were reported as failing on some Apple Arm64 systems.
2023-11-14 21:21:22 -08:00
Justine Tunney 9f9aec013a Do better job documenting platform issues 2023-11-14 18:15:58 -08:00
tkchia eea601f346 [metal] Some minor fixes and tweaks (#933)
* [metal] Ensure DF is clear when calling C from exception handler
* [metal] Mark some internal routines and declarations as `@internal`
* [metal] Fix crash under UEFI when command line string is NULL
* [metal] Fix argc & argv[] setting, & VM page freeing, for UEFI

Part of the memory occupied by the argv[] contents was
erroneously used for page tables & then later erroneously
freed.  The symptom was that argv[0] would show up as an
empty string ("").
2023-11-14 15:26:59 -08:00
Justine Tunney 1c2e7c1333 Introduce SIP_DISABLED compile option for ape-m1.c
Systems that don't use SIP can now build APE Loader with this flag to
get a performance speedup.
2023-11-13 22:04:05 -08:00
Justine Tunney 0283f2772c Fix build in MODE=dbg 2023-11-13 15:13:45 -08:00
Justine Tunney 751d20d98d Fix nsync_mu_unlock_slow_() on Apple Silicon
We torture test dlmalloc() in test/libc/stdio/memory_test.c. That test
was crashing on occasion on Apple M1 microprocessors when dlmalloc was
using *NSYNC locks. It was relatively easy to spot the cause, which is
this one particular compare and swap operation, which needed to change
to use sequentially-consistent ordering rather than an acquire barrier
2023-11-13 11:07:13 -08:00
Justine Tunney 3b15d31247 Fix #include <shared_mutex> 2023-11-13 08:30:17 -08:00
Justine Tunney f047172bd3 Fix uname assignment in Makefile 2023-11-13 05:53:10 -08:00
mattx 5d5b282d12 Inform user when wsl2 interop causes build issues (#952) 2023-11-13 05:29:25 -08:00
Justine Tunney 25a892d6ae Add some platform notes to the README file 2023-11-13 02:26:34 -08:00
Justine Tunney d1e764b3cf Modernize README.md 2023-11-12 22:04:07 -08:00
Paul Kulchenko 5c1fdc9819 Fix session changeset procesing (reverting some of 0d748ad5 changes) (#949)
This removes strdup from Lua SQLite wrapper, as it's applied to binary changeset
strings, thus cutting them short (because of strlen).
2023-11-12 14:25:50 -08:00
Justine Tunney f25beb341c Add architecture flag to zipobj 2023-11-12 09:35:59 -08:00
Justine Tunney bed77186c3 Use simple locks in dlmalloc 2023-11-12 09:00:49 -08:00
Justine Tunney bcf268adf8 Don't modify argument block on MacOS Arm64
Some dynamic library had access to this information somehow and was
crashing when it didn't have the expected structure.
2023-11-12 05:59:03 -08:00
Justine Tunney 0863427b3a Make development more pleasant on MacOS Arm64 2023-11-12 05:42:17 -08:00
Justine Tunney f7cad70da1 Introduce posix tests package 2023-11-12 05:41:48 -08:00
Justine Tunney 48bd3d85df Fix sigaction() return handler address on XNU/BSD 2023-11-12 05:40:06 -08:00
Justine Tunney bd56a9cf51 Rename dlopen() to cosmo_dlopen() 2023-11-12 01:19:04 -08:00
Justine Tunney c6d3802d3a Add more fixes for new cosmocc toolchain
We now have an `#include <cxxabi.h>` header which defines all the APIs
Cosmopolitan's implemented so far. The `cosmocc` README.md file is now
greatly expanded with documentation.
2023-11-11 23:28:19 -08:00
Gautham 95124cacbe Fixes for building superconfigure (#948) 2023-11-11 22:11:11 -08:00
Justine Tunney b320321319 Add build/bootstrap/rollup.com 2023-11-11 19:58:45 -08:00
Justine Tunney 291103ad8d Redesign cosmocc toolchain
The `cosmocc` compiler is now being distributed as a self-contained
toolchain that's path-agnostic and it no longer requires you clone the
Cosmop repo to use it. The bin/ folder has been deleted from the mono
repo. The `fatcosmocc` command has been renamed to `cosmocc`. MacOS
support now works very well.
2023-11-11 14:18:08 -08:00
Justine Tunney 3802428026 Fix missing .ok file make warning 2023-11-11 02:32:22 -08:00
Justine Tunney d0ad2694ed Iterate more on recent changes 2023-11-11 00:28:22 -08:00
Paul Kulchenko 7138399f96 Close Lua state before exiting redbean worker to allow GC to collect (#941) 2023-11-10 18:37:00 -08:00
Justine Tunney cdbd10c9d0 Introduce X86_CHECK() macro 2023-11-10 15:41:55 -08:00
Justine Tunney afbef6c361 Freshen build/bootstrap/mkdeps.com 2023-11-10 12:39:52 -08:00
Justine Tunney 3d3cdcf276 Fix APE uname direction
Fixes #942
2023-11-10 12:34:25 -08:00
Justine Tunney d2f49ca175 Improve mkdeps
Our makefile generator now accepts badly formatted include lines. It's
now more hermetic with better error checking in the cosmo repo, and it
can be configured to not be hermetic at all.
2023-11-10 04:14:27 -08:00
Justine Tunney 241f949540 Use dynamic memory for *NSYNC waiters 2023-11-10 01:42:06 -08:00
Justine Tunney 15af5c2d7e Fix ucontext_t alignment in strict ansi mode
Using `cosmocc -std=c11` was causing `ucontext_t` to become misaligned.
This change also adds the GNU constants on x86_64 for accessing general
registers, so you will not need `#ifdef`s to support both Cosmo and GNU
2023-11-09 12:48:29 -08:00
Justine Tunney b8d1377ae1 Give new wait4() another review pass 2023-11-09 11:23:30 -08:00
Justine Tunney ea28f93a26 Support <isystem> includes in monorepo 2023-11-08 19:14:56 -08:00
Justine Tunney 5330442d85 Avoid APE fork bomb
This change addresses a $PATH resolution issue where APE depends on
uname and uname is an APE program. So sorry to anyone this impacted
we'll get a release out soon.
2023-11-08 11:37:42 -08:00
mataha cc3e3de044 Make args replace ... only at the end of file (#909)
So that the string '...' can be used verbatim elsewhere in the file.
2023-11-08 10:00:52 -08:00
Justine Tunney ac125d3e1f Fix console copy/paste issue
Fixes #936
2023-11-08 09:29:45 -08:00
Justine Tunney 956e68be59 Revert "Use %gs as TLS register when dlopen() is linked"
This reverts commit d71da7fc72.
2023-11-08 01:33:01 -08:00
Justine Tunney d71da7fc72 Use %gs as TLS register when dlopen() is linked
Fixes #938
2023-11-08 01:11:17 -08:00
Justine Tunney e961385e55 Put more thought into i/o polyfills
wait4() is now solid enough to run `make -j100` on Windows. You can now
use MSG_DONTWAIT on Windows. There was a handle leak in accept() that's
been fixed. Our WIN32 overlapped i/o code has been simplified. Priority
class now inherits into subprocesses, so the verynice command will work
and the signal mask will now be inherited by execve() and posix_spawn()
2023-11-07 18:32:35 -08:00
Justine Tunney 736fdb757a Implement raise() with getcontext() / setcontext() 2023-11-05 18:04:36 -08:00
Justine Tunney dd83db9567 Improve makefile for uname 2023-11-05 16:21:09 -08:00
Justine Tunney 23d812f116 Release Cosmopolitan v3.0.2
It's recommended that all users, especially Apple Arm, re-run:

    ape/apeinstall.sh

Within the Cosmopolitan monorepo to refresh your APE loader.
2023-11-05 15:54:47 -08:00
Justine Tunney c39eb244d4 Disable an mprotect() test for now due to SIP 2023-11-05 15:38:16 -08:00
Justine Tunney f63c4d4f52 Use /usr/local/bin/ape on Apple Silicon 2023-11-05 14:52:27 -08:00
Paul Kulchenko b893479417 Add database and statement readonly checks to sqlite in redbean (#914) 2023-11-05 13:29:09 -08:00
Justine Tunney d02cba2451 Mint APE Loader v1.9 2023-11-05 13:11:34 -08:00
Justine Tunney 4cf987ddb1 Fix usleep() definition 2023-11-05 13:06:15 -08:00
Justine Tunney 20c794a353 Add strace to aarch64 vfork() 2023-11-05 13:06:15 -08:00
Gregory Haerr 4f100d2aad Fix stack alignment and auxv for macho APE loader (#931) 2023-11-05 13:06:10 -08:00
Justine Tunney a12ad17291 Get APE Loader working on MacOS with SIP enabled 2023-11-05 01:36:47 -08:00
Justine Tunney 653c7b2f87 Fatten build/bootstrap/echo.com 2023-11-04 21:01:43 -07:00
Justine Tunney d7917ea076 Make win32 i/o signals atomic and longjmp() safe 2023-11-04 20:33:29 -07:00
Justine Tunney 585c86e2a4 Support \n in /zip/.args files 2023-11-04 07:25:20 -07:00
Justine Tunney 48e260e653 Introduce NO_ADDRESS constant 2023-11-03 13:56:17 -07:00
Justine Tunney aa106c9356 Fix Lua test suite regression 2023-11-03 07:01:46 -07:00
Justine Tunney 5e8c928f1a Introduce dlopen() support
Every program built using Cosmopolitan is statically-linked. However
there are some cases, e.g. GUIs and video drivers, where linking the
host platform libraries is desirable. So what we do in such cases is
launch a stub executable using the host platform's libc, and longjmp
back into this executable. The stub executable passes back to us the
platform-specific dlopen() implementation, which we shall then wrap.

Here's the list of platforms that are supported so far:

- x86-64 Linux w/ Glibc
- x86-64 Linux w/ Musl Libc
- x86-64 FreeBSD
- x86-64 Windows
- aarch64 Linux w/ Glibc
- aarch64 MacOS

What this means is your Cosmo programs can call foreign functions on
your host operating system. However, it's important to note that any
foreign library you link won't have the ability to call functions in
your Cosmopolitan program. For example it's now technically possible
that Lua can load a module, however that almost certainly won't work
since the Lua module won't have access to Cosmo's Lua API.

Kudos to @jacereda for figuring out how to do this.
2023-11-03 06:37:18 -07:00
Justine Tunney 1eb6484c9c Rewrite getcwd()
This change addresses a bug that was reported in #923 where bash on
Windows behaved strangely. It turned out that our weak linking of
malloc() caused bash's configure script to favor its own getcwd()
function, which is implemented in the most astonishing way, using
opendir() and readdir() to recursively construct the current path.

This change moves getcwd() into LIBC_STDIO so it can strongly link
malloc(). A new __getcwd() function is now introduced, so all the
low-level runtime services can still use the actual system call. It
provides the Linux Kernel API convention across platforms, and is
overall a higher-quality implementation than what we had before.

In the future, we should probably take a closer look into why bash's
getcwd() polyfill wasn't working as intended on Windows, since there
might be a potential opportunity there to improve our readdir() too.
2023-11-02 13:16:42 -07:00
Justine Tunney a46ec61787 Make cosmocc -dumpmachine work correctly 2023-11-02 09:52:25 -07:00
Justine Tunney 024be3b009 Introduce getifaddrs()
This function was invented by the BSDs (it's not in POSIX.1). It
provides a high-level interface into ioctl(SIOCGIFCONF) which is
comparatively clumsy to use. We already made the ioctls portable
across our entire support vector back in 2021, so this interface
is portable too. See o//tool/viz/getifaddrs.com for our demo app
2023-11-02 08:33:03 -07:00
Justine Tunney 6ca5ab4680 Mention cosmocc-0.0.16 in Makefile 2023-11-01 12:08:50 -07:00
Justine Tunney 55d5d6bbc0 Release Cosmopolitan v3.0.1 2023-11-01 09:23:43 -07:00
Justine Tunney d259e573b6 Remove *NSYNC WIN32 semaphores
This implementation doesn't work as well as WIN32 futexes. This code
path was only added back when we were having issues with set context
however that's been solved so we can go back to the much better code
2023-11-01 07:18:58 -07:00
Justine Tunney 7b284f6bda Fix bugs and regressions in the pledge command
This change gets the pledge (formerly pledge.com) command back in tip
top shape for a 3.0.1 cosmos release. It now runs on all platforms, even
though it's mostly a no-op on ones that lack the kernel security stuff.
The binary footprint is now smaller, since it no longer needs to link
malloc. It's also now able to be built as a fat binary.
2023-11-01 06:08:58 -07:00
Justine Tunney b0e3d89942 Work around qemu-aarch64 bug
Qemu appears to define O_LARGEFILE as having its x86-64 value, which is
an easy mistake to make since this is one of the few magic numbers that
Linux special-cases for AARCH64.
2023-11-01 00:24:48 -07:00
Justine Tunney 0b1acce680 Introduce shm_open() and shm_unlink() 2023-10-31 23:57:52 -07:00
Justine Tunney fadb64a2bf Introduce pthread_rwlock_try{rd,wr}lock
This also changes recursive mutexes to favor cpu over scheduler yield.
2023-10-31 22:13:08 -07:00
Justine Tunney a1e1e821cb Introduce GNU/BSD timeval macros 2023-10-31 21:58:49 -07:00
Justine Tunney 2af7c802b6 Stub out getutline and pututline 2023-10-31 21:58:39 -07:00
Justine Tunney 3b791d2f44 Introduce lgammal, tgammal, erfl, and erfcl
git://git.musl-libc.org/musl
79bdacff83a6bd5b70ff5ae5eb8b6de82c2f7c30
2023-10-31 21:58:29 -07:00
Justine Tunney b4084dd6c6 Introduce libcxx <shared_mutex>
[email protected]:llvm-mirror/libcxx.git
78d6a7767ed57b50122a161b91f59f19c9bd0d19
2023-10-31 21:57:28 -07:00
Justine Tunney ee82f90bba Introduce __cxa_thread_atexit() 2023-10-31 20:04:31 -07:00
Justine Tunney a699cda5c8 Make cosmocc -Wl,--version do the right thing
Our `cosmocc` wrapper script will now report the linker version, the
same way as the `cc` command. The `fatcosmocc` command won't do this
because we wouldn't want programs to parse out the path of the amd64
linker command and use it explicitly.

Fixes #917
2023-10-31 18:20:28 -07:00
Justine Tunney f86ffa1a25 Fix aarch64 build breakage caused by ed17d3008 2023-10-31 18:17:32 -07:00
Justine Tunney a09f62141d Fix bug in login_tty() 2023-10-31 18:03:13 -07:00
tkchia ed17d3008b [metal] Add a uprintf() routine, for non-emergency boot logging (#905)
* [metal] Add a uprintf() routine, for non-emergency boot logging
* [metal] _Really_ push forward timing of VGA TTY initialization
* [metal] Do something useful with uprintf()
* [metal] Locate some ACPI tables, for later hardware detection

Specifically the code now tries to find the ACPI RSDP,
RSDT/XSDT, FADT, & MADT tables, whether in legacy BIOS
bootup mode or in a UEFI bootup.  These are useful for
figuring out how to (re)enable asynchronous interrupts
in legacy 8259 PIC mode.
2023-10-25 14:32:20 -07:00
Justine Tunney 062b2d776e Mention cosmocc v0.0.14 in Makefile
https://justine.lol/cosmocc-0.0.14.zip
https://github.com/ahgamut/superconfigure/releases/tag/z0.0.14
2023-10-16 00:07:33 -07:00
Justine Tunney 4ae5223eae Release Cosmopolitan v3.0 2023-10-15 23:53:36 -07:00
Justine Tunney 8f82a5f968 Fix the build 2023-10-15 20:36:12 -07:00
Justine Tunney d1300623d2 Add tree command to third_party 2023-10-15 19:46:40 -07:00
Justine Tunney 7b8024d088 Supplement _GNU_SOURCE with _COSMO_SOURCE 2023-10-15 16:59:27 -07:00
Gautham e6b59bced2 Add tsearch from Musl Libc (#912)
https://git.musl-libc.org/cgit/musl/snapshot/musl-1.2.4.tar.gz
2023-10-15 16:50:54 -07:00
Justine Tunney c9fecf3a55 Make improvements
- You can now run `make -j8 toolchain` on Windows
- You can now run `make -j` on MacOS ARM64 and BSD OSes
- You can now use our Emacs dev environment on MacOS/Windows
- Fix bug where the x16 register was being corrupted by --ftrace
- The programs under build/bootstrap/ are updated as fat binaries
- The Makefile now explains how to download cosmocc-0.0.12 toolchain
- The build scripts under bin/ now support "cosmo" branded toolchains
- stat() now goes faster on Windows (shaves 100ms off `make` latency)
- Code cleanup and added review on the Windows signal checking code
- posix_spawnattr_setrlimit() now works around MacOS ARM64 bugs
- Landlock Make now favors posix_spawn() on non-Linux/OpenBSD
- posix_spawn() now has better --strace logging on Windows
- fstatat() can now avoid EACCES in more cases on Windows
- fchmod() can now change the readonly bit on Windows
2023-10-15 16:45:00 -07:00
Justine Tunney 06c6baaf50 Fix copy/paste issue in Windows console 2023-10-14 16:14:50 -07:00
Justine Tunney bd48e6c666 Fix the Landlock Make build 2023-10-14 10:40:05 -07:00
Justine Tunney cdf556e7d2 Implement signal handler tail recursion
GNU Make on Windows now appears to be working reliably. This change also
fixes a bug where, after fork the Windows thread handle wasn't reset and
that caused undefined behavior using SetThreadContext() with our signals
2023-10-14 10:38:15 -07:00
Justine Tunney a657f3e878 Delete old wcwidth() implementation
This shaves away 144kb of bss memory from every binary linking printf at
the expense of slightly increased binary footprint. Kudos for Byron Lai.
2023-10-14 03:15:05 -07:00
Justine Tunney 2db2f40a98 Rewrite special file handling on Windows
This change gets GNU grep working. What caused it to not work, is it
wouldn't write to an output file descriptor when its dev/ino equaled
/dev/null's. So now we invent special dev/ino values for these files
2023-10-14 02:53:34 -07:00
Justine Tunney aca2261cda Don't preempt WIN32 libraries
This change refactors our POSIX signals emulation for Windows so that it
performs some additional safety checks before calling SetThreadContext()
which needs to be locked and must never ever interrupt Microsoft's code.
Kudos to the the Go developers for figuring out how to do this properly.
2023-10-13 13:59:39 -07:00
Justine Tunney d1a283a588 Fix redbean LaunchBrowser() 2023-10-13 11:39:00 -07:00
Justine Tunney 3851025b77 Fix SQLite regressions caused by 3b086af91 2023-10-13 11:00:39 -07:00
Justine Tunney bdc453b22d Fix redbean "-1 workers remain" blemish 2023-10-13 08:33:09 -07:00
Justine Tunney 0e0398cd5d Fix MODE=tinylinux build 2023-10-13 08:30:18 -07:00
Justine Tunney 4bcb107cb0 Fix ctrl-c in redbean on Windows 2023-10-13 08:10:03 -07:00
Justine Tunney d458642790 Write more tests and improve kill() on Windows 2023-10-13 04:38:45 -07:00
Justine Tunney b81a1bd9a8 Fix bug with __sig_tramp() + sigaltstack() 2023-10-13 01:24:10 -07:00
Justine Tunney 49b0eaa69f Improve threading and i/o routines
- On Windows connect() can now be interrupted by a signal; connect() w/
  O_NONBLOCK will now raise EINPROGRESS; and connect() with SO_SNDTIMEO
  will raise ETIMEDOUT after the interval has elapsed.

- We now get the AcceptEx(), ConnectEx(), and TransmitFile() functions
  from the WIN32 API the officially blessed way, using WSAIoctl().

- Do nothing on Windows when fsync() is called on a directory handle.
  This was raising EACCES earlier becaues GENERIC_WRITE is required on
  the handle. It's possible to FlushFileBuffers() a directory handle if
  it's opened with write access but MSDN doesn't document what it does.
  If you have any idea, please let us know!

- Prefer manual reset event objects for read() and write() on Windows.

- Do some code cleanup on our dlmalloc customizations.

- Fix errno type error in Windows blocking routines.

- Make the futex polyfill simpler and faster.
2023-10-12 23:13:04 -07:00
Justine Tunney f7343319cc Cull the examples folder 2023-10-11 21:45:32 -07:00
Justine Tunney 3a1f887928 Introduce posix_spawn_file_actions_addchdir_np() 2023-10-11 21:45:32 -07:00
Paul Kulchenko f92ad74e6b Update pidfile processing to truncate when writing pid (#910) 2023-10-11 21:45:27 -07:00
Paul Kulchenko 6e4b9b6515 Add EncodeBase32() to Redbean (#856) 2023-10-11 20:06:20 -07:00
Justine Tunney 3b086af91b Fix issues for latest GCC toolchain 2023-10-11 14:54:42 -07:00
Justine Tunney 5cb9b2658c Remove last remnants of redbean repl thread 2023-10-11 12:27:06 -07:00
Justine Tunney 285c565051 Clean up some code 2023-10-11 11:45:31 -07:00
Justine Tunney ec3275179f Fix MODE=tiny build 2023-10-10 00:58:47 -07:00
Justine Tunney 9cc4f33c76 Fix some todos 2023-10-09 23:12:32 -07:00
Justine Tunney 9d372f48dd Fix some issues 2023-10-09 20:19:09 -07:00
Gautham 211d5d902e Changes made for cosmocc builds (#908) 2023-10-09 14:39:02 -07:00
Justine Tunney 3b4dbc9fdd Make some more fixes
This change deletes mkfifo() so that GNU Make on Windows will work in
parallel mode using its pipe-based implementation. There's an example
called greenbean2 now, which shows how to build a scalable web server
for Windows with 10k+ threads. The accuracy of clock_nanosleep is now
significantly improved on Linux.
2023-10-09 12:22:00 -07:00
Justine Tunney 820c3599ed Make some quick fixes 2023-10-08 17:56:59 -07:00
Justine Tunney 94dc7a684e Fix MODE=tiny build 2023-10-08 09:34:15 -07:00
Justine Tunney 791f79fcb3 Make improvements
- We now serialize the file descriptor table when spawning / executing
  processes on Windows. This means you can now inherit more stuff than
  just standard i/o. It's needed by bash, which duplicates the console
  to file descriptor #255. We also now do a better job serializing the
  environment variables, so you're less likely to encounter E2BIG when
  using your bash shell. We also no longer coerce environ to uppercase

- execve() on Windows now remotely controls its parent process to make
  them spawn a replacement for itself. Then it'll be able to terminate
  immediately once the spawn succeeds, without having to linger around
  for the lifetime as a shell process for proxying the exit code. When
  process worker thread running in the parent sees the child die, it's
  given a handle to the new child, to replace it in the process table.

- execve() and posix_spawn() on Windows will now provide CreateProcess
  an explicit handle list. This allows us to remove handle locks which
  enables better fork/spawn concurrency, with seriously correct thread
  safety. Other codebases like Go use the same technique. On the other
  hand fork() still favors the conventional WIN32 inheritence approach
  which can be a little bit messy, but is *controlled* by guaranteeing
  perfectly clean slates at both the spawning and execution boundaries

- sigset_t is now 64 bits. Having it be 128 bits was a mistake because
  there's no reason to use that and it's only supported by FreeBSD. By
  using the system word size, signal mask manipulation on Windows goes
  very fast. Furthermore @asyncsignalsafe funcs have been rewritten on
  Windows to take advantage of signal masking, now that it's much more
  pleasant to use.

- All the overlapped i/o code on Windows has been rewritten for pretty
  good signal and cancelation safety. We're now able to ensure overlap
  data structures are cleaned up so long as you don't longjmp() out of
  out of a signal handler that interrupted an i/o operation. Latencies
  are also improved thanks to the removal of lots of "busy wait" code.
  Waits should be optimal for everything except poll(), which shall be
  the last and final demon we slay in the win32 i/o horror show.

- getrusage() on Windows is now able to report RUSAGE_CHILDREN as well
  as RUSAGE_SELF, thanks to aggregation in the process manager thread.
2023-10-08 08:59:53 -07:00
Justine Tunney af7cb3c82f Improve Windows keyboard translation to Linux 2023-10-04 10:33:03 -07:00
Justine Tunney 982dc4db87 Fix some issues with select() 2023-10-04 09:10:58 -07:00
Justine Tunney 6918c3ffc2 Delete non-standard broken strtonum() function 2023-10-04 08:20:06 -07:00
Justine Tunney af8236264e Fix silly execve() regression 2023-10-04 08:07:07 -07:00
Justine Tunney 4631d34d0d Improve stack overflow recovery
It's now possible to use sigaltstack() to recover from stack overflows
on Windows. Several bugs in sigaltstack() have been fixed, for all our
supported platforms. There's a newer better example showing how to use
this, along with three independent unit tests just to further showcase
the various techniques.
2023-10-04 07:35:17 -07:00
Justine Tunney 1694edf85c Fix some additional Windows TTY issues 2023-10-04 02:17:25 -07:00
Justine Tunney f26a280cda Implement basic canonical mode for Windows
The `cat` command now works properly, when run by itself on the bash
command prompt. It's working beautifully so far, and is only missing
a few keystrokes for clearing words and lines. Definitely works more
well than the one that ships with WIN32 :-)
2023-10-03 22:36:22 -07:00
Justine Tunney 4825737509 Make pledge() less strict about the API
We were previously returning EINVAL but OpenBSD allows it.
2023-10-03 17:57:26 -07:00
Justine Tunney 695f74035d Use CLK_TCK for clock_nanosleep() spin threshold
This more accurately reflects how the kernels actually implement this
function and it most importantly avoids incurring startup latency.
2023-10-03 17:26:29 -07:00
Justine Tunney 11c18fa644 Make ulock stracing less noisy 2023-10-03 15:19:20 -07:00
Justine Tunney 85f64f3851 Make futexes 100x better on x86 MacOS
Thanks to @autumnjolitz (in #876) the Cosmopolitan codebase is now
acquainted with Apple's outstanding ulock system calls which offer
something much closer to futexes than Grand Central Dispatch which
wasn't quite as good, since its wait function can't be interrupted
by signals (therefore necessitating a busy loop) and it also needs
semaphore objects to be created and freed. Even though ulock is an
internal Apple API, strictly speaking, the benefits of futexes are
so great that it's worth the risk for now especially since we have
the GCD implementation still as a quick escape hatch if it changes

Here's why this change is important for x86 XNU users. Cosmo has a
suboptimal polyfill when the operating system doesn't offer an API
that let's us implement futexes properly. Sadly we had to use that
on X86 XNU until now. The polyfill works using clock_nanosleep, to
poll the futex in a busy loop with exponential backoff. On XNU x86
clock_nanosleep suffers from us not being able to use a fast clock
gettime implementation, which had a compounding effect that's made
the polyfill function even more poorly. On X86 XNU we also need to
polyfill sched_yield() using select(), which made things even more
troublesome. Now that we have futexes we don't have any busy loops
anymore for both condition variables and thread joining so optimal
performance is attained. To demonstrate, consider these benchmarks

Before:

    $ ./lockscale_test.com -b
    consumed 38.8377   seconds real time and
              0.087131 seconds cpu time

After:

    $ ./lockscale_test.com -b
    consumed 0.007955 seconds real time and
             0.011515 seconds cpu time

Fixes #876
2023-10-03 15:15:43 -07:00
Justine Tunney ff250a0c10 Revert "Rewrite ZipOS"
This reverts commit b01282e23e. Some tests
are broken. It's not clear how it'll impact metal yet. Let's revisit the
memory optimization benefits of this change again sometime soon.
2023-10-03 14:40:03 -07:00
Justine Tunney ee8a861635 Fix semaphore deadlock on Apple Silicon 2023-10-03 09:26:46 -07:00
Paul Kulchenko e8ecf31ad5 Fix handling of redbean assets without extension
Fixes #902
2023-10-03 08:25:47 -07:00
Justine Tunney a201e694ee Include missing symbols in <cstdint> for C++
Fixes #899
2023-10-03 08:19:09 -07:00
Justine Tunney 5212f78c33 Fix redbean history
Fixes #903
2023-10-03 08:14:01 -07:00
tkchiaandtkchia fcdda40f19 [metal] Fix regression causing early crash in __new_page( ) (#900)
There was a glitch in the refactoring of __map_phdrs( ) in
commit ec480f5aa0, which caused it to try to map the PT_NOTE
program segment into virtual memory, which in turn caused the
memory page at BANE to be wrongly remapped to the start of
the program image (0x100000) rather than physical address 0.
This affected subsequent page allocation operations because
the `struct mman` was located at BANE + 0x0500.

Co-authored-by: tkchia <[email protected]>
2023-10-03 07:48:55 -07:00
Justine Tunney b01282e23e Rewrite ZipOS
This reduces the virtual memory usage of Emacs for me by 30%. We now
have a simpler implementation that uses read(), rather mmap()ing the
whole executable.
2023-10-03 07:27:25 -07:00
Justine Tunney ff77f2a6af Make improvements
- This change fixes a bug that allowed unbuffered printf() output (to
  streams like stderr) to be truncated. This regression was introduced
  some time between now and the last release.

- POSIX specifies all functions as thread safe by default. This change
  works towards cleaning up our use of the @threadsafe / @threadunsafe
  documentation annotations to reflect that. The goal is (1) to use
  @threadunsafe to document functions which POSIX say needn't be thread
  safe, and (2) use @threadsafe to document functions that we chose to
  implement as thread safe even though POSIX didn't mandate it.

- Tidy up the clock_gettime() implementation. We're now trying out a
  cleaner approach to system call support that aims to maintain the
  Linux errno convention as long as possible. This also fixes bugs that
  existed previously, where the vDSO errno wasn't being translated
  properly. The gettimeofday() system call is now a wrapper for
  clock_gettime(), which reduces bloat in apps that use both.

- The recently-introduced improvements to the execute bit on Windows has
  had bugs fixed. access(X_OK) on a directory on Windows now succeeds.
  fstat() will now perform the MZ/#! ReadFile() operation correctly.

- Windows.h is no longer included in libc/isystem/, because it confused
  PCRE's build system into thinking Cosmopolitan is a WIN32 platform.
  Cosmo's Windows.h polyfill was never even really that good, since it
  only defines a subset of the subset of WIN32 APIs that Cosmo defines.

- The setlongerjmp() / longerjmp() APIs are removed. While they're nice
  APIs that are superior to the standardized setjmp / longjmp functions,
  they weren't superior enough to not be dead code in the monorepo. If
  you use these APIs, please file an issue and they'll be restored.

- The .com appending magic has now been removed from APE Loader.
2023-10-03 06:17:16 -07:00
Justine Tunney b99512ac58 Temporarily disable test_suite_x509parse.com
This test broke itself due to relying on the current time. Mocking out
gettimeofday() confirms this. However, two ipv4 subject alt name tests
still surprisingly fail even with a fake current time. We'll want this
investigated further soon.
2023-10-02 13:49:16 -07:00
Gavin Hayes c315315896 fleaks.c: initialize command buffer before calling system (#904) 2023-10-02 13:31:15 -07:00
Gavin Hayes cc8c56e814 Fix strtod NaN handling / fix SIGSEGV in testlib/showerror (#901) 2023-09-27 00:16:36 -07:00
Justine Tunney 3ffc17c50e Add Cosmopolitan to uname() 2023-09-21 23:51:55 -07:00
Justine Tunney ed316491ca Fix issue with pasting text into console 2023-09-21 13:28:52 -07:00
Justine Tunney 4f5d5a6813 Fix some more issues
- ARM Neon headers are now exported in libc/isystem/

- stat() and access() now do a better job reporting which files are
  executable which ones aren't. They do this by reading the first two
  bytes in a file to see if it's `MZ` or `#!`.
2023-09-21 11:41:42 -07:00
Justine Tunney 22cf6e11eb Add siglongjmp() for aarch64 2023-09-21 10:10:20 -07:00
Justine Tunney c88f95a892 Remove Windows executable path guessing logic
Unlike CMD.EXE, CreateProcess() doesn't care if an executable name ends
with .COM or .EXE. We now have the unbourne shell and bash working well
on Windows, so we don't need DOS anymore. Making this change will grant
us better performance, particularly for builds, because commandv() will
need to make fewer system calls. Path mangling magic still happens with
WinMain() and ntspawn() in order to do things like turn \ into / so the
interop works well at the borders. But all the code in libraries, which
did that, has been removed. It's not possible for libraries to abstract
the differences between paths.
2023-09-21 08:13:50 -07:00
Justine Tunney 0c5dd7b342 Make improvements
- Improved async signal safety of read() particularly for longjmp()
- Started adding cancel cleanup handlers for locks / etc on Windows
- Make /dev/tty work better particularly for uses like `foo | less`
- Eagerly read console input into a linked list, so poll can signal
- Fix some libc definitional bugs, which configure scripts detected
2023-09-21 07:30:39 -07:00
Justine Tunney d6c2830850 Rewrite Windows console input handling
This change removes our use of ENABLE_VIRTUAL_TERMINAL_INPUT (which
isn't very good) in favor of having read() translate Windows Console
input events to ANSI/XTERM sequences by hand. This makes it possible to
capture important keystrokes (e.g. ctrl-space) that weren't possible
before. Most importantly this change also removes the stdin/sigwinch
worker threads, which never really worked that well. Interactive TTY
sessions will now work reliably when a Cosmo process spawns or forks
another Cosmo process, e.g. unbourne.com launching emacs.com.
2023-09-19 11:53:27 -07:00
Justine Tunney ececec4c94 Fix some zipos directory related bugs 2023-09-19 02:30:42 -07:00
Justine Tunney ec480f5aa0 Make improvements
- Every unit test now passes on Apple Silicon. The final piece of this
  puzzle was porting our POSIX threads cancelation support, since that
  works differently on ARM64 XNU vs. AMD64. Our semaphore support on
  Apple Silicon is also superior now compared to AMD64, thanks to the
  grand central dispatch library which lets *NSYNC locks go faster.

- The Cosmopolitan runtime is now more stable, particularly on Windows.
  To do this, thread local storage is mandatory at all runtime levels,
  and the innermost packages of the C library is no longer being built
  using ASAN. TLS is being bootstrapped with a 128-byte TIB during the
  process startup phase, and then later on the runtime re-allocates it
  either statically or dynamically to support code using _Thread_local.
  fork() and execve() now do a better job cooperating with threads. We
  can now check how much stack memory is left in the process or thread
  when functions like kprintf() / execve() etc. call alloca(), so that
  ENOMEM can be raised, reduce a buffer size, or just print a warning.

- POSIX signal emulation is now implemented the same way kernels do it
  with pthread_kill() and raise(). Any thread can interrupt any other
  thread, regardless of what it's doing. If it's blocked on read/write
  then the killer thread will cancel its i/o operation so that EINTR can
  be returned in the mark thread immediately. If it's doing a tight CPU
  bound operation, then that's also interrupted by the signal delivery.
  Signal delivery works now by suspending a thread and pushing context
  data structures onto its stack, and redirecting its execution to a
  trampoline function, which calls SetThreadContext(GetCurrentThread())
  when it's done.

- We're now doing a better job managing locks and handles. On NetBSD we
  now close semaphore file descriptors in forked children. Semaphores on
  Windows can now be canceled immediately, which means mutexes/condition
  variables will now go faster. Apple Silicon semaphores can be canceled
  too. We're now using Apple's pthread_yield() funciton. Apple _nocancel
  syscalls are now used on XNU when appropriate to ensure pthread_cancel
  requests aren't lost. The MbedTLS library has been updated to support
  POSIX thread cancelations. See tool/build/runitd.c for an example of
  how it can be used for production multi-threaded tls servers. Handles
  on Windows now leak less often across processes. All i/o operations on
  Windows are now overlapped, which means file pointers can no longer be
  inherited across dup() and fork() for the time being.

- We now spawn a thread on Windows to deliver SIGCHLD and wakeup wait4()
  which means, for example, that posix_spawn() now goes 3x faster. POSIX
  spawn is also now more correct. Like Musl, it's now able to report the
  failure code of execve() via a pipe although our approach favors using
  shared memory to do that on systems that have a true vfork() function.

- We now spawn a thread to deliver SIGALRM to threads when setitimer()
  is used. This enables the most precise wakeups the OS makes possible.

- The Cosmopolitan runtime now uses less memory. On NetBSD for example,
  it turned out the kernel would actually commit the PT_GNU_STACK size
  which caused RSS to be 6mb for every process. Now it's down to ~4kb.
  On Apple Silicon, we reduce the mandatory upstream thread size to the
  smallest possible size to reduce the memory overhead of Cosmo threads.
  The examples directory has a program called greenbean which can spawn
  a web server on Linux with 10,000 worker threads and have the memory
  usage of the process be ~77mb. The 1024 byte overhead of POSIX-style
  thread-local storage is now optional; it won't be allocated until the
  pthread_setspecific/getspecific functions are called. On Windows, the
  threads that get spawned which are internal to the libc implementation
  use reserve rather than commit memory, which shaves a few hundred kb.

- sigaltstack() is now supported on Windows, however it's currently not
  able to be used to handle stack overflows, since crash signals are
  still generated by WIN32. However the crash handler will still switch
  to the alt stack, which is helpful in environments with tiny threads.

- Test binaries are now smaller. Many of the mandatory dependencies of
  the test runner have been removed. This ensures many programs can do a
  better job only linking the the thing they're testing. This caused the
  test binaries for LIBC_FMT for example, to decrease from 200kb to 50kb

- long double is no longer used in the implementation details of libc,
  except in the APIs that define it. The old code that used long double
  for time (instead of struct timespec) has now been thoroughly removed.

- ShowCrashReports() is now much tinier in MODE=tiny. Instead of doing
  backtraces itself, it'll just print a command you can run on the shell
  using our new `cosmoaddr2line` program to view the backtrace.

- Crash report signal handling now works in a much better way. Instead
  of terminating the process, it now relies on SA_RESETHAND so that the
  default SIG_IGN behavior can terminate the process if necessary.

- Our pledge() functionality has now been fully ported to AARCH64 Linux.
2023-09-18 21:04:47 -07:00
Gavin Hayes c4eb838516 README: note version zsh and fish were patched to support ape (#896) 2023-09-15 08:10:11 -07:00
Justine Tunney 81f391dd22 Rewrite Windows signal delivery system 2023-09-12 11:38:34 -07:00
Justine Tunney 00084577a3 Improve posix_spawn() some more 2023-09-12 08:58:57 -07:00
Justine Tunney 6430e474b4 Remove VM variable
You need to use qemu-user in binfmt_misc. For non-Linux we'll update
execve() to spawn under the appropriate blink or qemu when needed.
2023-09-12 01:27:30 -07:00
Justine Tunney 20c77338e6 Remove IMAGE_BASE_VIRTUAL 2023-09-12 01:21:36 -07:00
Justine Tunney 8a0008d985 Avoid leaking handles across processes 2023-09-12 01:07:51 -07:00
Justine Tunney a359de7893 Get rid of kmalloc()
This changes *NSYNC to allocate waiters on the stack so our locks don't
need to depend on dynamic memory. This make our runtiem simpler, and it
also fixes bugs with thread cancellation support.
2023-09-11 21:56:00 -07:00
Justine Tunney 77a7873057 Improve AARCH64 execution
This change fixes bugs in the APE loader. The execve() unit tests are
now enabled for MODE=aarch64. See the README for how you need to have
binfmt_misc configured with Qemu to run them. Apple Silicon bugs have
been fixed too, e.g. tkill() now works.
2023-09-11 14:46:46 -07:00
Justine Tunney 1965d7488e Improve performance and remove fd leaks 2023-09-10 11:52:03 -07:00
Justine Tunney 26e254fb4d Overhaul process spawning 2023-09-10 08:17:44 -07:00
Justine Tunney 99dc1281f5 Overhaul Windows signal handling
The new asynchronous signal delivery technique is now also being used
for tkill(), raise(), etc. Many subtle issues have been addresesd. We
now signal handling on Windows that's remarkably similar to the POSIX
behaviors. However that's just across threads. We're lacking a way to
have the signal semantics work well, across multiple WIN32 processes.
2023-09-08 01:49:41 -07:00
Justine Tunney 8bdaddd81d Make the Windows Console work better
The stdio reader thread now appears to be working recursively along
cosmopolitan subprocesses. For example, it's now possible to launch
vim.com from the unbourne.com bestline repl, thanks to hacks plus a
bug fix to select() timeouts.
2023-09-07 18:27:22 -07:00
Justine Tunney 032b1f3449 Implement thread cancellation for aarch64 2023-09-07 08:48:38 -07:00
Justine Tunney dcda6f7d8d Simplify the rusage.com command 2023-09-07 05:41:18 -07:00
Justine Tunney b592716d1c Reduce mandatory stack rss by 256kb 2023-09-07 04:33:01 -07:00
Justine Tunney 0e087143fd Make greenbean web server better
- Remove misguided __assert_disabled variable
- Change EPROCLIM to be EAGAIN on BSD distros
- Improve quality of greenbean with cancellations
- Fix thread race condition crash with file descriptors
2023-09-07 03:44:50 -07:00
Justine Tunney 6cff3137c5 Fix GitHub Actions 2023-09-06 23:39:47 -07:00
Justine Tunney 425c055116 Make improvements
- Polyfill readlink("foo/") dir check on Windows
- Support asynchronous signal delivery on Windows
- Restore Windows Console from execve() daisy chain
- Work around bug in AARCH64 Optimized Routines memcmp()
- Disable unbourne.com shell completion on Windows for now
- Don't always set virtual terminal input state on console
- Remove Musl Libc's unusual preservation of realpath("//")
- Make realpath() strongly link malloc() to pass configure test
- Delete cosh.com shell, now that unbourne.com works on Windows!
2023-09-06 22:48:05 -07:00
Justine Tunney f9c9a323fe Fix Landlock Make crash when TMPDIR isn't defined 2023-09-06 12:49:52 -07:00
Justine Tunney f531acc8f9 Make improvements
- Invent openatemp() API
- Invent O_UNLINK open flag
- Introduce getenv_secure() API
- Remove `git pull` from cosmocc
- Fix utimes() when path is NULL
- Fix mktemp() to never return NULL
- Fix utimensat() UTIME_OMIT on XNU
- Improve utimensat() code for RHEL5
- Turn `argv[0]` C:/ to /C/ on Windows
- Introduce tmpnam() and tmpnam_r() APIs
- Fix more const issues with internal APIs
- Permit utimes() on WIN32 in O_RDONLY mode
- Fix fdopendir() to check fd is a directory
- Fix recent crash regression in landlock make
- Fix futimens(AT_FDCWD, NULL) to return EBADF
- Use workaround so `make -j` doesn't fork bomb
- Rename dontdiscard to __wur (just like glibc)
- Fix st_size for WIN32 symlinks containing UTF-8
- Introduce stdio ext APIs needed by GNU coreutils
- Fix lstat() on WIN32 for symlinks to directories
- Move some constants from normalize.inc to limits.h
- Fix segv with memchr() and memcmp() overlapping page
- Implement POSIX fflush() behavior for reader streams
- Implement AT_SYMLINK_NOFOLLOW for utimensat() on WIN32
- Don't change read-only status of existing files on WIN32
- Correctly handle `0x[^[:xdigit:]]` case in strtol() functions
2023-09-06 12:34:59 -07:00
michalbiesek 8596e83cce Linux add definition for madvise flag (#883) 2023-09-06 12:34:54 -07:00
tkchiaandtkchia b21842ed7a [metal] Fix video mode filtering & frame buffer ref-counting (#889)
Co-authored-by: tkchia <[email protected]>
2023-09-06 03:41:07 -07:00
Justine Tunney 0d748ad58e Fix warnings
This change fixes Cosmopolitan so it has fewer opinions about compiler
warnings. The whole repository had to be cleaned up to be buildable in
-Werror -Wall mode. This lets us benefit from things like strict const
checking. Some actual bugs might have been caught too.
2023-09-01 20:50:18 -07:00
Justine Tunney e2b3c3618e Move zlib down 2023-08-31 15:17:57 -07:00
stellartux 4021cd0c1e Update redbean definitions (#887)
- Add OnServerReload signature
- Fix Pledge, Unveil, Slurp, ProgramLogPath signatures
- Add pledge flags
2023-08-26 18:40:07 -07:00
Justine Tunney 4c74f09393 Fix socket() EPROTONOSUPPORT fallback on MacOS 2023-08-22 14:34:37 -07:00
Justine Tunney 58ef4e6df8 Add i/o statistics to wait4() about child process 2023-08-21 21:25:33 -07:00
Justine Tunney b9eb656e41 Fix Windows stdin regression
The non-blocking standard input feature was broken by ec957491e.
2023-08-21 21:04:05 -07:00
Justine Tunney 7e08a97cea Further improve scanf 2023-08-21 16:55:29 -07:00
Justine Tunney 6ef2a471e4 Get GNU MPFR and MPC tests to pass
This change fixes more issues with our scanf() function.
2023-08-21 15:05:10 -07:00
Justine Tunney 63a1636e1f Get GNU GMP test suite fully passing
- Fix stdio fmemopen() buffer behaviors
- Fix scanf() to return EOF when appropriate
- Prefer fseek/ftell names over fseeko/ftello
- Ensure locale field is always set in the TIB
- Fix recent regression in vfprintf() return count
- Make %n directive in scanf() have standard behavior
2023-08-21 10:16:42 -07:00
Justine Tunney 755ae64e73 Remove long path tests for Windows 2023-08-21 04:43:53 -07:00
Justine Tunney fffcd98b0e Fix msync() flags on FreeBSD 2023-08-21 04:15:05 -07:00
Justine Tunney ebf784d4f5 Make improvements
- Introduce ualarm() function
- Make rename() report EISEMPTY on Windows
- Always raise EINVAL upon open(O_RDONLY|O_TRUNC)
- Add macro so ./configure will detect SOCK_CLOEXEC
- Fix O_TRUNC without O_CREAT not working on Windows
- Let fcntl(F_SETFL) change O_APPEND status on Windows
- Make sure pwrite() / pread() report ESPIPE on sockets
- Raise ESPIPE on Windows when pwrite() is used on pipe
- Properly compute O_APPEND CreateFile() flags on Windows
- Don't require O_DIRECTORY to open directories on Windows
- Fix more instances of Windows reporting EISDIR and ENOTDIR
- Normalize EFTYPE and EMLINK to ELOOP on NetBSD and FreeBSD
- Make unlink() / rmdir() work on read-only files on Windows
- Validate UTF-8 on Windows paths to fix bug with overlong NUL
- Always print signal name to stderr when crashing due to SIG_DFL
- Fix Windows bug where denormalized paths >260 chars didn't work
- Block signals on BSDs when thread exits before trashing its own stack
2023-08-21 02:34:17 -07:00
Justine Tunney ec957491ea Make posix_spawn faster on Windows 2023-08-20 02:50:07 -07:00
michalbiesek d6f72aa4a6 Extend pr.h header with recent options (#881) 2023-08-20 02:49:58 -07:00
michalbiesek 4f517503e4 Add PTRACE_SECCOMP_GET_METADATA (#882)
see
  linux commit: 26500475ac1b499d8636ff281311d633909f5d20
  ptrace, seccomp: add support for retrieving seccomp metadata
2023-08-20 02:49:22 -07:00
Justine Tunney 610e0d95cb Fix the MODE=tinylinux build 2023-08-19 06:56:05 -07:00
Justine Tunney 965516e313 Make improvements for Actually Portable Emacs
- Get SIGWINCH working again on the New Technology
- Correctly handle O_NOFOLLOW in open() on Windows
- Implement synthetic umask() functionality on Windows
- Do a better job managing file execute access on Windows
- Fill in `st_uid` and `st_gid` with username hash on Windows
- Munge UNICODE control pictures into control codes on Windows
- Do a better job ensuring Windows console settings are restored
- Introduce KPRINTF_LOG environment variable to log kprintf to a file
2023-08-19 06:44:58 -07:00
Justine Tunney 9c7b81ee0f Give Emacs another performance boost 2023-08-18 09:34:14 -07:00
Justine Tunney 5b42c810a5 Add filesystem index to ZipOS
This change brings the /zip/... read-only filesystem into performance
parity with the native Linux filesystem which doesn't use compression
therefore, imagine how much faster this could be with bloom filtering
rather than simple binary search, and if we used zstd instead of zlib
2023-08-18 07:11:48 -07:00
Justine Tunney 7100b1cf91 Get Fat Emacs working in Windows Console 2023-08-18 05:00:30 -07:00
Justine Tunney bf835de612 Get Fat Emacs working on Apple Silicon 2023-08-17 22:01:42 -07:00
Justine Tunney 3f9b39883f Make fat ape binaries smaller again 2023-08-17 11:15:58 -07:00
Justine Tunney 1d8937d528 Mint APE Loader v1.7
This change reduces the memory requirements of your APE Loader by 10x,
in terms of virtual memory size, thanks to the help of alloca(). We're
also now creating argument blocks with the same layout across systems.
2023-08-17 09:04:50 -07:00
Paul Kulchenko d967a94c9a Update ProgramDirectory to prepend Lua path instead of appending (#862). (#864)
* Update ProgramDirectory to prepend Lua path instead of appending (#862).

* Update redbean ProgramDirectory to raise Lua error for easier handling

* Update redbean default Lua path handling to simplify
2023-08-17 00:43:11 -07:00
Paul Kulchenko d0d027810a Add OnServerReload to replace /.reload.lua with a bit more flexibility (#851) 2023-08-17 00:42:23 -07:00
Justine Tunney 3a9cac4892 Fix small matters and improve sysconf()
- Fix mkdeps.com out of memory error
- Remove static memory from __get_cpu_count()
- Add support for passing hyphen to cat in cocmd
- Change more ZipOS errors from ENOTSUP to EROFS
- Specify mem_unit in sysinfo() output on BSD OSes
2023-08-17 00:32:11 -07:00
Gautham eebc24b9cd get rsync to build (#886) 2023-08-16 21:41:36 -07:00
Justine Tunney 04bd488da7 Introduce COSMOPOLITAN_DISABLE_ZIPOS environ var 2023-08-16 21:04:16 -07:00
Justine Tunney 8d1c81ac9f Emulate ENOTDIR better 2023-08-16 20:11:23 -07:00
Justine Tunney b76b2be2d0 Improve zip read-only filesystem
readdir() will now always yield an inode that's consistent with stat()
on ZipOS and Windows in general. More APIs have been updated to return
the appropriate error code when inappropriately trying to do ops, like
sockets, with a zip file descriptor. The path normalization algorithms
are now fully fleshed out. Some socket APIs have been fixed so they'll
raise EBADF vs. ENOTSOCK appropriately. Lastly seekdir() will now work
properly on NetBSD and FreeBSD (not sure why anyone would even use it)
2023-08-16 17:52:12 -07:00
Justine Tunney dc6c67256f Remove old stack code and improve dirstream 2023-08-16 07:54:40 -07:00
Justine Tunney 74caabb823 Fix getopt again 2023-08-15 20:06:19 -07:00
Justine Tunney 507d7a0b0b Fix stack memory, undefined behavior, etc. 2023-08-15 19:10:08 -07:00
Justine Tunney 110559ce6a Make ZipOS and Qemu work better
This change improves the dirstream library in a lot of respects,
especially for /zip/... files. Also turn off MAP_STACK on Aarch64
because Qemu seems to implement it differently than Linux and it's
probably responsible for a lot of mysterious crashes.
2023-08-15 18:32:50 -07:00
Justine Tunney 4658ae539f Remove old Windows thread hack from Redbean 2023-08-15 06:06:43 -07:00
Justine Tunney de3f3a9e5a Allocate explicit stack on aarch64 2023-08-15 04:40:19 -07:00
Justine Tunney 0e586c834a Refactor fatcosmocc into a single file 2023-08-14 22:26:17 -07:00
Justine Tunney b494d56549 Fix the cosmocc ar commands 2023-08-14 20:12:51 -07:00
Gautham f5b39e9f31 Fix typo in cosmocc (#884) 2023-08-13 23:31:39 -07:00
Justine Tunney 2b67e15b3d Fix MODE=tinylinux build 2023-08-13 22:59:56 -07:00
Justine Tunney 9c0821def7 Make stdin pollable on Windows
You can now play Super Mario Bros in CMD.EXE using Cosmopolitan! This is
thanks to a new worker thread that's spawned on Windows whenever any one
of poll(), select(), or ioctl(FIONREAD) is linked.
2023-08-13 22:42:25 -07:00
Justine Tunney ef6387ee5e Polyfill the 32-bit _bsr() function 2023-08-13 21:13:32 -07:00
Justine Tunney c776a32f75 Replace COSMO define with _COSMO_SOURCE
This change might cause ABI breakages for /opt/cosmos. It's needed to
help us better conform to header declaration practices.
2023-08-13 20:55:04 -07:00
Justine Tunney a033b65a33 Fix apelink shell script corruption bug
We were using a shell heredoc value '@' to terminate the dos stub, but
that's not sufficiently safe. We found out sh doesn't consider control
characters as contributing to the start of a line, and had the unlucky
chance of the linker choosing the number 2624 for e_lfanew, and that's
"@\n" in ASCII, which compromised the APE shell script.

We now use the heredoc 'justineXXXXXX' with 31 bits of entropy, that's
determistically generated by hashing apelink inputs w/ crc32 / blake2b
2023-08-13 17:42:11 -07:00
Justine Tunney 2366848db5 Perform some more touchups to toolchain 2023-08-13 15:03:09 -07:00
Justine Tunney d1b937bf1d Make cosmocc capable of cross compilation 2023-08-13 14:40:22 -07:00
Justine Tunney 504a4bbf84 Fix --ftrace with cosmocc 2023-08-13 08:25:36 -07:00
Justine Tunney 5d5b4ce691 Add ANSI version of bsr/bsf 2023-08-13 08:19:00 -07:00
Justine Tunney b40d41085d Fix the build 2023-08-13 07:17:40 -07:00
Justine Tunney 6942d7b820 Fix bug with temporary files on Windows 2023-08-13 07:10:33 -07:00
Justine Tunney ab9a284640 Further improve fatcosmocc 2023-08-13 01:51:39 -07:00
Justine Tunney 3f2f0e3a74 Make fatcosmocc good enough to build ncurses 6.4 2023-08-12 22:30:05 -07:00
Justine Tunney 399d14aadf Make fatcosmocc good enough to build Lua 5.4.6
make all test CC=fatcosmocc AR='fatcosmoar rcu'

This change introduces a program named mktemper.com which provides more
reliable and secure temporary file name generation for scripts. It also
makes our ar.com program more permissive in what commands it'll accept.
The cosmocc command is improved by this change too.
2023-08-12 16:44:04 -07:00
Justine Tunney 566cb5963f Make assimilate.com better
It's now safer to run. It'll now remove FreeBSD from the ELF os/abi so
that GDB is happier.
2023-08-12 07:46:24 -07:00
Justine Tunney f491276b62 Add support for C++ thread safe statics 2023-08-12 07:45:32 -07:00
Justine Tunney 60f2ea36ca Make default localtime() zone /etc/localtime 2023-08-12 07:44:15 -07:00
Justine Tunney e11fa30791 Move zipos into runtime package
This way complex runtime features (e.g. ftrace, symbol tables) can
always yoink zipos support. This is important now that apelink.com
automates embedding symbol tables for multiple cpus.
2023-08-11 23:14:02 -07:00
Justine Tunney d53c335a45 Introduce new fatcosmocc command
This new script is an alternative to the `cosmocc` command. It's still a
work in progress. It abstracts all the gory details of building separate
copies of your executable and then running the apelink.com program.
2023-08-11 22:52:11 -07:00
Justine Tunney 8fc778162e Make stderr go faster
This change makes _IONBF (unbuffered) stdio handles go 20x faster for
certain kinds of formatting directives by being smarter about buffers
2023-08-11 11:56:35 -07:00
Justine Tunney 2cbd09b4d4 Fix the build 2023-08-11 09:49:39 -07:00
Justine Tunney 0105e3e2b6 Introduce new linker for fat ape binaries 2023-08-11 04:39:19 -07:00
Michael Lenaghan e3c456d23a Don’t lowercase the test filename (#871)
This test fails if Cosmo builds from a path that contains an uppercase character.

Paths with uppercase characters aren’t so common in server Linux. But they are in *desktop* Linux. Guess how I…?

But I digress.

The real problem is that the path is lowercased on one line, but not the next:

```
        self.file_name = support.TESTFN.lower()
        self.file_path = FakePath(support.TESTFN)
```

Given that no other test in the suite lowercases `support.TESTFN`, I opted to remove it from the first line rather than adding it to the second.
2023-08-09 21:13:03 -07:00
Justine Tunney dd53f31147 Introduce post-linker that converts ELF to PE
If you build a static ELF executable in `ld -q` mode (which leaves rela
sections inside the binary) then you can run it through the elf2pe.com
program afterwards, which will turn it into a PE executable. We have a
new trick for defining WIN32 DLL imports in C without any assembly code.
This also achieves the optimally tiny and perfect PE binary structure.

We need this because it isn't possible to have a GNU ld linker script
generate a PE file where the virtual pointer and the file pointer can
drift apart. This post-linker can do that. One cool benefit is we can
now use a smaller 512-byte alignment in the file, and an even bigger
64kb alignment for the segment virtual addresses, and the executable
ends up being smaller.

Another program introduced by this change is pecheck.com which can do
extensive linting of PE static executables to help explain why Windows
won't load it.
2023-08-09 18:46:06 -07:00
Justine Tunney 2f35bbf046 Fix Windows stack size 2023-08-09 00:51:30 -07:00
Justine Tunney 9dd50f7dfc Better document PE headers 2023-08-09 00:29:01 -07:00
Justine Tunney 50394064d7 Invent systemvpe() function
It goes 5x faster than system() and it's safer too.
2023-08-09 00:27:55 -07:00
Justine Tunney 1a5ef5ba13 Mint APE Loader 1.6
This change fixes a bug with loading pure bss program headers.
2023-08-09 00:27:26 -07:00
Justine Tunney c48ee8e4fe Fix POLLIN|POLLOUT when together on Windows 2023-08-08 10:33:52 -07:00
Justine Tunney 33d280c8ba Improve Windows Console I/O
- Blocking read operations on the Windows Console can now EINTR
- Blocking read operations on Windows pipes now EINTR more reliably
- setitimer() will no longer be inherited across fork() on Windows
- It's now possible to use ECHO when the console is in raw mode
- The ECHOCTL flag now works correctly on the Windows Console
- The ICRNL flag now works correctly on the Windows Console
- pread() and pwrite() will now raise ESPIPE on Windows
- Opening /dev/tty on Windows is improved (untested)
- Overlapped I/O is now implemented in a better way
2023-08-08 05:44:40 -07:00
Justine Tunney decf216655 Perform inconsequential code cleanup 2023-08-07 20:24:50 -07:00
Justine Tunney 929478c524 Clean up more WIN32 APIs 2023-08-01 14:17:03 -07:00
Gautham 44b88d659d Fix x-axis blurring in nesemu1 tty conversion (#878) 2023-08-01 08:39:27 -07:00
Joel Yliluoma f729810f85 Losslessly recompress PNG files (#877)
Original sizes:

 639082 libc/dns/dns.png
  21281 usr/share/img/honeybadger.png
   4710 third_party/python/Lib/idlelib/Icons/idle_48.png
   2542 third_party/python/Lib/idlelib/Icons/idle_32.png
   1264 third_party/python/Lib/idlelib/Icons/idle_16.png
   1020 third_party/python/Lib/test/imghdrdata/python.png

Updated sizes:

 484841 libc/dns/dns.png
  17737 usr/share/img/honeybadger.png
   3041 third_party/python/Lib/idlelib/Icons/idle_48.png
   1606 third_party/python/Lib/idlelib/Icons/idle_32.png
    701 third_party/python/Lib/idlelib/Icons/idle_16.png
    602 third_party/python/Lib/test/imghdrdata/python.png
2023-07-31 17:59:16 -07:00
Justine Tunney 712b1aab3c Make quick fix for aarch64 build 2023-07-30 15:06:28 -07:00
Justine Tunney c8aa33e0e2 Improve wait statuses
This change has the insight that dwExitCode isn't an exit code but
rather should be used to pass the wait status. This lets us report
killing as a termination status, similar to UNIX. This change also
fixes the fact that exit(259) on Windows will break the parent due
way WIN32 is designed. We now work around that.

It turns out that NetBSD and OpenBSD, will let you have exit codes
beyond 255. This change will let you use them when it's possible.
2023-07-30 14:51:37 -07:00
Justine Tunney d9d5f45e2d Restore missing cosmopolitan documentation on website 2023-07-30 11:07:53 -07:00
Justine Tunney 58352df0a4 Make forking off threads reliable on Windows
This change makes posix_spawn_test no longer flaky on Windows, by (1)
fixing a race condition in wait(), and (2) removing a misguided vfork
implementation which was letting Windows bypass pthread_atfork().
2023-07-30 09:32:41 -07:00
Justine Tunney 2ebc5781a1 Fix flakes in runitd and popen_test 2023-07-30 04:26:34 -07:00
Justine Tunney 801224df67 Support symbol tables with arch specific name 2023-07-29 23:50:15 -07:00
Justine Tunney bd49ea1c3a Fix tests on aarch64 2023-07-29 19:02:25 -07:00
Justine Tunney 18bb5888e1 Make more fixes and improvements
- Remove PAGESIZE constant
- Fix realloc() documentation
- Fix ttyname_r() error reporting
- Make forking more reliable on Windows
- Make execvp() a few microseconds faster
- Make system() a few microseconds faster
- Tighten up the socket-related magic numbers
- Loosen restrictions on mmap() offset alignment
- Improve GetProgramExecutableName() with getenv("_")
- Use mkstemp() as basis for mktemp(), tmpfile(), tmpfd()
- Fix flakes in pthread_cancel_test, unix_test, fork_test
- Fix recently introduced futex stack overflow regression
- Let sockets be passed as stdio to subprocesses on Windows
- Improve security of bind() on Windows w/ SO_EXCLUSIVEADDRUSE
2023-07-29 18:44:15 -07:00
Justine Tunney 140a8a52e5 Remove 5000 WIN32 APIs 2023-07-28 14:32:34 -07:00
Justine Tunney 6c7b4fcbd3 Delete reference to old echo program 2023-07-28 14:10:22 -07:00
Justine Tunney 06082c7d37 Polyfill fchmodat() 2023-07-28 07:41:43 -07:00
Justine Tunney 7926aa8bfa Remove ELF binaries from tests 2023-07-28 07:20:57 -07:00
Justine Tunney 5018171fa5 Fix a bunch of Windows bugs reported on Discord
This change addresses everything from stack smashing to %SYSTEMROOT%
breaking socket(). Issues relating to compile.com not reporting text
printed to stderr has been resolved for Windows builds.
2023-07-28 06:17:34 -07:00
Justine Tunney b0e3258709 Fix close(1) bug on Windows 2023-07-28 03:56:56 -07:00
Justine Tunney 83341a4269 Remove hints from Windows imports 2023-07-27 14:09:07 -07:00
Justine Tunney 7e0a09feec Mint APE Loader v1.5
This change ports APE Loader to Linux AARCH64, so that Raspberry Pi
users can run programs like redbean, without the executable needing
to modify itself. Progress has also slipped into this change on the
issue of making progress better conforming to user expectations and
industry standards regarding which symbols we're allowed to declare
2023-07-26 13:54:49 -07:00
Justine Tunney 6843150e0c Mint APE Loader v1.4
This change also incorporates more bug fixes and improvements to a wide
variety of small things. For example this fixes #860 so Windows console
doesn't get corrupted after exit. An system stack memory map issue with
aarch64 has been fixed. We no longer use O_NONBLOCK on AF_UNIX sockets.
Crash reports on Arm64 will now demangle C++ symbols, even when c++filt
isn't available. Most importantly the Apple M1 version of APE Loader is
brought up to date by this change. A prebuilt unsigned binary for it is
being included in build/bootstrap/. One more thing: retrieving the term
dimensions under --strace was causing the stack to become corrupted and
now that's been solved too. PSS: We're now including an ELF PT_NOTE for
APE in the binaries we build, that has the APE Loader version.
2023-07-25 05:48:08 -07:00
Michael Lenaghan 53d3f9d9c5 Change the default TZ value from GST to GMT #861 (#867)
With this change, `unix.localtime()` will match `unix.gmtime()`
when `TZ` hasn't been defined.
2023-07-24 20:40:15 -07:00
Andy George 9c79c5682c Add sh code tagging (#855) 2023-07-24 20:15:18 -07:00
Justine Tunney e0c2b91b3e Remove _Hide keyword
It never did anything and isn't worthwhile as documentation.
2023-07-24 08:34:58 -07:00
Justine Tunney 4fb6cbc1fe Revert "Polyfill ENOTSOCK in getsockname() and getpeername()"
This reverts commit 925219bdf3.
2023-07-24 03:09:43 -07:00
Justine Tunney 925219bdf3 Polyfill ENOTSOCK in getsockname() and getpeername() 2023-07-24 02:11:25 -07:00
Justine Tunney eb84a25994 Check for interrupts more eagerly on Windows 2023-07-24 02:03:29 -07:00
Justine Tunney 94ea34367a Tune the page sizes 2023-07-24 00:49:06 -07:00
Justine Tunney ff198e7577 Introduce GetElfSegmentAddress() function 2023-07-23 17:36:18 -07:00
Justine Tunney 3d172c99fe Mint APE Loader v1.3
This version has better error messages and safety checks. It supports
loading static position-independent executables. It correctly handles
more kinds of weird ELF program header layouts. A force flag has been
added to avoid system execve(). Finally the longstanding misalignment
with our ELF PT_NOTE section has been addressed.
2023-07-23 17:08:14 -07:00
Justine Tunney 82b1e61443 Fix Landlock Make build config issue 2023-07-23 16:43:22 -07:00
Justine Tunney 0ba3199915 Fix some more socket bugs
- The functions that return a sockaddr now do so the same way the Linux
  Kernel does across platforms, e.g. getpeername(), accept4()

- Socket system calls on Windows will now only check for interrupts when
  a blocking operation needs to be performed.

- Write tests for recvfrom() system call
2023-07-23 16:31:10 -07:00
Danny Robinson ac92f25296 Update definitions.lua with latest progress (#852)
* Update definitions.lua with latest progress

* Remove duplicate definition
2023-07-23 11:14:55 -07:00
tkchiaandtkchia 0ffc0dd461 Let lz4toasm accept LZ4 files that lack extracted-size field (#858)
lz4toasm should now more easily accept LZ4 files output by
compressor programs that do not support the extracted-size
field, such as Stephan Brumme's smallz4.

This patch also proposes to add a new lz4len() function to
the libc: it parses an LZ4 compressed block to compute the
unpacked content size, without really unpacking the block.

Co-authored-by: tkchia <[email protected]>
2023-07-23 11:12:22 -07:00
Gautham 3eec69f9c3 Simplify cosmocc builds (#863)
- tcgetpgrp(STDIN_FILENO) should be equal to getpgrp() on Windows also,
  found while reading wget source code which uses this check to decide
  whether to print to stderr or to a file
- IN6_ADDR_ARE_EQUAL is a comparison macro used when IPV6 is allowed,
  found while reading CPython3.11 source code
- the changes in signal.h and addition of ucontext.h are because
  CPython3.11 source code expect sigaltstack to be available
- the sqlite3.mk change is because CPython3.11 requires sqlite3 to be
  built with -DOMIT_SHARED_CACHE
- unistd.h has getopt.h now, because some libraries like it there
2023-07-23 11:11:08 -07:00
Justine Tunney f83eb440f7 Incorporate more small improvements 2023-07-23 10:57:18 -07:00
Justine Tunney 1d4eb08fa1 Support non-blocking i/o across platforms
This change introduces new tests for `O_NONBLOCK` and `SOCK_NONBLOCK` to
confirm that non-blocking i/o is now working on all supported platforms,
including Windows. For example, you can now say on Windows, MacOS, etc.:

    socket(AF_INET, SOCK_STREAM | SOCK_NONBLOCK, IPPROTO_TCP);

To create a non-blocking IPv4 TCP socket. Or you can enable non-blocking
i/o on an existing socket / pipe / etc. file descriptor by calling fcntl

    fcntl(fd, F_SETFL, fcntl(fd, F_GETFL) | O_NONBLOCK);

This functionality is polyfilled on older Linux kernels too, e.g. RHEL5.
Now that fcntl() support is much better the FIOCLEX / FIONCLEX polyfills
for ioctl() have been removed since they're ugly non-POSIX diameond APIs

This change fixes a weakness in kprintf() that was causing Windows trace
tools to frequently crash.
2023-07-23 02:56:47 -07:00
Justine Tunney 5c9e03e3e0 Temporarily turn down redbean StoreAsset() feature 2023-07-22 15:51:44 -07:00
Justine Tunney 18536950b3 Fix the build 2023-07-11 05:48:39 -07:00
Justine Tunney 1ee2e89326 Make improvements
- This commit mints a new release of APE Loader v1.2 which supports
  loading ELF programs with a non-contiguous virtual address layout
  even though we've never been able to take advantage of it, due to
  how `objcopy -SO binary` fills any holes left by PT_LOAD. This'll
  change soon, since we'll have a new way of creating APE binaries.

- The undiamonding trick with our ioctl() implementation is removed
  since POSIX has been killing ioctl() for years and they've done a
  much better job. One problem it resolves, is that ioctl(FIONREAD)
  wasn't working earlier and that caused issues when building Emacs
2023-07-11 04:41:50 -07:00
Justine Tunney a1b1fdd1a4 Reconfigure GitHub Actions 2023-07-10 12:17:18 -07:00
Justine Tunney a2d269dc38 Brush up some more code 2023-07-10 10:17:26 -07:00
Justine Tunney ee6566a152 Further improve usability of cosmocc
- Support -s flag in cosmocc
- Support posix_spawn() setsid() feature
- Disable monorepo debug path prefix stripping
2023-07-10 05:55:00 -07:00
Justine Tunney f7ae50462a Make improvements
- Fix unused local variable errors
- Remove yoinks from sigaction() header
- Add nox87 and aarch64 to github actions
- Fix cosmocc -fportcosmo in linking mode
- It's now possible to build `make m=llvm o/llvm/libc`
2023-07-10 04:35:14 -07:00
Justine Tunney 3dc86ce154 Fix breakages in Linux-only build modes
- compile.com now polyfills -march=native which gcc/clang removed
- Guarantee zero Windows code is linked into non-Windows binaries
- MODE=tinylinux binaries are now back to being as tiny as ~4kb
- Improve the runtime's stack allocation / alignment hack
- GitHub Actions now tests Linux modes for assurance
2023-07-09 19:51:44 -07:00
Justine Tunney 0e4c828a8e Fix alignment of ftrace hook 2023-07-09 16:52:08 -07:00
Justine Tunney e1316d6565 Fix chibicc build breakage 2023-07-09 15:00:33 -07:00
Justine Tunney d3cf9d4ef1 Use long as implicit type in chibicc
This makes it possible to write 64-bit C without headers and prototypes.
2023-07-09 10:11:06 -07:00
Justine Tunney 42ba9901e4 Fix some behavioral issues on Windows 2023-07-09 09:59:22 -07:00
Justine Tunney 41396ff48a Make fixes and improvements
- Fix handling of precision in hex float formatting
- Enhance the cocmd interpreter for system() and popen()
- Manually ran the Lua unit tests, which are now passing
- Let stdio i/o operations happen when file is in error state
- We're now saving and restoring xmm in ftrace out of paranoia
2023-07-09 05:21:11 -07:00
Justine Tunney 95fbdb4f76 Have function tracer save all registers 2023-07-08 12:04:44 -07:00
Justine Tunney 1cc8ff6a55 Fix some aarch64 build issues 2023-07-08 10:47:45 -07:00
Justine Tunney 9ae230afad Fix more build configuration errors 2023-07-08 09:08:13 -07:00
Justine Tunney a75175fe94 Make build hermetically sealed again
It turned out that Landlock Make hasn't been applying sandboxing for a
while, due to a mistyped if statement for `$(USE_SYSTEM_TOOLCHAIN)` it
should have had the opposite meaning. Regressions in the build configs
have been fixed. The rmrf() function works better now. The rm.com tool
works according to POSIX with the exception of supporting prompts.
2023-07-08 07:06:25 -07:00
Justine Tunney 0c43c98de1 Add libcxx filesystem modules 2023-07-08 02:40:17 -07:00
Justine Tunney a092fda388 Make some fixes for libcxx
This change figures out some of the build configuration issues we've
been having with libcxx. The c++ span header is added. Per a Discord
discussion we're now turning off `-g` for the default build mode, so
consider using `make MODE=dbg` or `make MODE=zero` for GDB debugging
which works much better than `MODE=` ever has. Note that the default
build mode has always had very good function call / system call logs
plus you can still use ShowCrashReports() for backtrace. Making this
change ensures cosmocc will better conform to FOSS norms. Lastly the
LoadZipArgs() API has been added to cosmopolitan.a and <cosmo.h>.
2023-07-07 19:35:58 -07:00
Justine Tunney 3a8579252d Move demangling utility to stdio
Closes #848
2023-07-07 10:47:50 -07:00
Farid Zakaria a1dd777f22 add support for __cxa_demangle
This adds support for __cxa_demangle through the cxxabi.h file.

At the moment it is the only symbol included.
The source was taken from FreeBSD
contrib/libcxxrt/libelftc_dem_gnu3.c
2176c9ab71c85efd90a6c7af4a9e04fe8e3d49ca

FreeBSD does say this is also almost verbatim from ELFToolkit
2023-07-07 10:29:03 -07:00
Justine Tunney 0d3c1c8b1a Do work on curl/mbedtls/zstd
This change fixes stderr to be unbuffered. Added hardware AES on ARM64
to help safeguard against timing attacks. The curl.com command will be
somewhat more pleasant to use.
2023-07-07 10:13:35 -07:00
Justine Tunney a186143f62 Add EncodeHex() and DecodeHex() to Redbean 2023-07-06 15:38:08 -07:00
Justine Tunney 00acd81b2f Delete more dead code 2023-07-06 09:12:28 -07:00
Justine Tunney 0a24b4fc3c Clean up more code
The *NSYNC linked list API is good enough that it deserves to be part of
the C libray, so this change writes an improved version of it which uses
that offsetof() trick from the Linux Kernel. We vendor all of the *NSYNC
tests in third_party which helped confirm the needed refactoring is safe

This change also deletes more old code that didn't pan out. My goal here
is to work towards a vision where the Cosmopolitan core libraries become
less experimental and more focused on curation. This better reflects the
current level of quality we've managed to achieve.
2023-07-06 08:03:24 -07:00
Justine Tunney 88612a2cd7 Restore original redbean build linking 2023-07-05 20:04:43 -07:00
Justine Tunney 97b7116953 Hunt down more bugs
After going through the MODE=dbg and MODE=zero build modes, a bunch of
little issues were identified, which have been addressed. Fixing those
issues created even more troubles for the project, because it improved
our ability to detect latent problems which are getting fixed so fast.
2023-07-03 18:43:29 -07:00
Justine Tunney 73c0faa1b5 Remove some dead code 2023-07-03 02:48:29 -07:00
Justine Tunney 168d1c157e Fix build breakage under special strace config 2023-07-03 00:05:51 -07:00
Justine Tunney fe044e22cc Switch public headers to getopt_long() entirely
Cosmopolitan's getopt() is now redefined as __getopt().
2023-07-02 19:57:43 -07:00
Justine Tunney 0c630d95b5 Rewrite Cosmopolitan Ar
The build/bootstrap/ar.com program is now tinier. This change reduces
its size from 140kb to 53kb. Nothing was traded away. Cosmopolitan Ar
performance is now 2x better than llvm-ar largely thanks to using the
copy_file_range() system call. This change homebrews a new allocation
API that addresses the shortcomings of the C standard library design.
Using these new balloc() and reballoc() functions I managed to reduce
memory consumption so much that Cosmpolitan Ar should now use roughly
100x fewer bytes of peak resident memory compared to llvm-ar. Correct
behavior with better compatibility has been assured. Binary output is
now pretty much bit-identical to llvm-ar, as of this change. This can
and should be the living proof we need to show that a better world is
possible for software.
2023-07-02 10:19:16 -07:00
Justine Tunney 197aa0d465 Implement swapcontext() and makecontext()
This change introduces support for Linux-style uc_context manipulation
that's fast and works well on all supported OSes and architectures. It
also integrates with the Cosmpolitan runtime which can show backtraces
comprised of multiple stacks and fibers. See the test and example code
for further details. This will be used by Mold once it's been vendored
2023-07-02 09:01:44 -07:00
Justine Tunney 7ec84655b4 Get setcontext() and getcontext() working on Aarch64
This change also adds the missing code for getting and restoring the
thread's signal mask, since that's explicitly listed by the man page
2023-07-01 22:53:23 -07:00
Farid Zakaria 8b62bff364 Introduce xxhash into third_party (#837) 2023-07-01 20:56:53 -07:00
Farid Zakaria 5f1e129222 Introduce zstd into cosmopolitan (#843) 2023-07-01 20:30:38 -07:00
Paul Kulchenko c0348330a5 Add path information in logged error messages (#832) 2023-07-01 19:47:59 -07:00
Paul Kulchenko dc8fbf38ad Update Windows IP_MULTICAST constants to use Ws2tcpip.h values (#839) (#842) 2023-07-01 19:44:27 -07:00
Justine Tunney ce0a3065f3 Update build/bootstrap/make.com
This is important, because the execve() workflow has been upgraded for
launching APE binaries. Programs that call execve() need to be remade,
otherwise they might attempt to use ~/.ape rather than ~/ape-1.1.
2023-07-01 18:37:19 -07:00
Justine Tunney 40eb3b9d5d Fully support OpenBSD 7.3
This change (1) upgrades to OpenBSD's newer kernel ABIs, and (2)
modifies APE to have a read-only data segment. Doing this required
creating APE Loader v1.1, which is backwards and forwards compatible
with the previous version.

If you've run the following commands in the past to install your APE
Loader systemwide, then you need to run them again. Ad-hoc installations
shouldn't be impacted. It's also recommended that APE binaries be remade
after upgrading, since they embed old versions of the APE Loader.

    ape/apeuninstall.sh
    ape/apeinstall.sh

This change does more than just fix OpenBSD. The new loader is smarter
and more reliable. We're now able create much tinier ELF and Mach-O data
structures than we could before. Both APE Loader and execvpe() will now
normalize ambiguous argv[0] resolution the same way as the UNIX shell.
Badness with TLS linkage has been solved.

Fixes #826
2023-07-01 18:14:27 -07:00
Justine Tunney 963e10b9bf Introduce new tinyprint() api 2023-07-01 05:09:32 -07:00
Justine Tunney 053681cb97 Fix BSD regressions
The recent change to crt.S that aggressively aligns the system-provided
stack has been rolled back on non-Linux until we can find a better way,
since it can cause a segfault early in execution on several platforms.

This change fixes a regression in tcgetattr() and tcsetattr() on OpenBSD
and NetBSD caused by 4778cd4d27.

This change has been tested across the runitd test fleet which is green.
2023-07-01 00:17:33 -07:00
Paul Kulchenko 5a77db2cd5 Fix recvfrom on Windows (closes #841) (#844) 2023-06-30 23:39:55 -07:00
Justine Tunney 48b2afb192 Address weakness with new pledge("anet") promise
The intent with pledge("anet") has been to prevent outbound connections.
However we were only doing that for TCP sockets, and outbound UDP could
still get through, by using socket() plus sendto(). This change fixed
that by preventing UDP sockets from being created.

Credit goes to chc4 on Hacker News for finding this.
2023-06-18 18:06:47 -07:00
Justine Tunney fb2bd313ae Fix redbean unix.poll() with -1 timeout 2023-06-18 18:05:26 -07:00
Justine Tunney f10845ab9f Add blackhole support to Python tokenbucket module 2023-06-18 07:50:11 -07:00
Justine Tunney 461c835784 Remove removed GCC symlinks 2023-06-18 05:41:55 -07:00
Justine Tunney 226375933a Implement more toolchain fixes 2023-06-18 05:39:31 -07:00
Justine Tunney 0409096658 Get us closer to building busybox
This change undefines __linux__ and adds APIs like clock_settime(). The
gosh darned getopt_long() API has been reintroduced, thanks to OpenBSD.
2023-06-18 04:13:45 -07:00
Justine Tunney 5ac522f0de Add make clean build rule 2023-06-18 01:23:54 -07:00
Justine Tunney 8514933b60 Delete itoa() prototype 2023-06-18 01:17:15 -07:00
Justine Tunney d7c79f43ef Clean up more code
- Found some bugs in LLVM compiler-rt library
- The useless LIBC_STUBS package is now deleted
- Improve the overflow checking story even further
- Get chibicc tests working in MODE=dbg mode again
- The libc/isystem/ headers now have correctly named guards
2023-06-18 01:00:05 -07:00
Justine Tunney afc58a8b41 Upgrade cosmopolitan gcc and binutils to 2.35.2
This change upgrades to the latest portcosmo gcc patch
6728fe1a25185560603ca312a8d4352af2a4e515 which lets us avoid needing to
define __tmpcosmo_FOO constants. We're now using an appropriate binutils
version for GCC 11. The older binutils sometimes wasn't able to print
backtraces, due to not being able to find a .debug_ranges section.

This is breaking change for /opt/cosmos libraries :'( due to this weird
"error: need linked-to section for --gc-sections" that pops up.

Please run `make clean` in the monorepo before rebuilding.
2023-06-18 00:54:14 -07:00
Justine Tunney 2c0f0bab0b Fix mkfifo() on Linux 2023-06-17 16:04:08 -07:00
Justine Tunney 12452976bd Remove getopt_long() from headers
We removed this API a long time ago and it was breaking the GNU make
open source build.
2023-06-17 16:03:16 -07:00
Justine Tunney b881c0ec9e Remove printf() linking hack 2023-06-17 10:13:50 -07:00
Justine Tunney ba03cd95c5 Modernize the README file 2023-06-17 07:55:48 -07:00
Justine Tunney 562a1384cd Make blink support conditionally linkable into APE 2023-06-17 07:55:35 -07:00
Justine Tunney 52d28966f7 Remove old zip base skew hack
Since 8ff48201ca we no longer need the
hack where, when running .com.dbg files, we scanned for the embedded
.com file offset, and then computed zip offsets realtive to that. It
wasn't very reliable in the first place, and was causing issues with
running our new .com.dbg executables, which are true zip files.
2023-06-17 04:20:16 -07:00
Justine Tunney 207e18a060 Unbreak the x86-64 build 2023-06-16 20:05:24 -07:00
Justine Tunney 7bc20bff3e Resolve argv[0] based on path search in ape-m1 2023-06-16 16:58:30 -07:00
Justine Tunney 1353db7d3f Fix rwx .rodata in python.com on aarch64 2023-06-16 16:25:04 -07:00
Justine Tunney 4eebd6b9dc Improve new C23 checked arithmetic feature 2023-06-16 15:32:18 -07:00
Justine Tunney 2a1c588826 Add tokenbucket module to python.com 2023-06-16 15:32:09 -07:00
Justine Tunney e6b7c16a53 Make changes needed for new demo 2023-06-15 23:22:49 -07:00
Justine Tunney c3440d040c Make improvements
- More timspec_*() and timeval_*() APIs have been introduced.
- The copyfd() function is now simplified thanks to POSIX rules.
- More Cosmo-specific APIs have been moved behind the COSMO define.
- The setitimer() polyfill for Windows NT is now much higher quality.
- Fixed build error for MODE=aarch64 due to -mstringop-strategy=loop.
- This change introduces `make MODE=nox87 toolchain` which makes it
  possible to build programs using your cosmocc toolchain that don't
  have legacy fpu instructions. This is useful, for example, if you
  want to have a ~22kb tinier blink virtual machine.
2023-06-15 14:50:53 -07:00
Justine Tunney 8dc11afcf6 Upgrade portcosmo patch and GCC third party docs 2023-06-14 19:53:06 -07:00
Justine Tunney 4778cd4d27 Fix bugs in termios library and cleanup code
This change fixes an issue with the tcflow() magic numbers that was
causing bash to freeze up on Linux. While auditing termios polyfills,
several other issues were identified with XNU/BSD compatibility.

Out of an abundance of caution this change undefines as much surface
area from libc/calls/struct/termios.h as possible, so that autoconf
scripts are less likely to detect non-POSIX teletypewriter APIs that
haven't been polyfilled by Cosmopolitan.

This is a *breaking change* for your static archives in /opt/cosmos if
you use the cosmocc toolchain. That's because this change disables the
ioctl() undiamonding trick for code outside the monorepo, specifically
because it'll lead to brittle ABI breakages like this. If you're using
the cosmocc toolchain, you'll need to rebuild libraries like ncurses,
readline, etc. Yes diamonds cause bloat. To work around that, consider
using tcgetwinsize() instead of ioctl(TIOCGWINSZ) since it'll help you
avoid pulling every single ioctl-related polyfill into the linkage.

The cosmocc script was specifying -DNDEBUG for some reason. It's fixed.
2023-06-14 19:30:52 -07:00
Justine Tunney 06b749ae03 Remove blinkenlights
New home: https://github.com/jart/blink
2023-06-14 19:30:52 -07:00
Gabriel Ravier 50d8d953ce Fix syscall(2) returning -errno instead of using POSIX errno scheme (#830)
As of now, the syscall function is implemented as alike to how the
linux kernel sycall ABI works, returning -errno upon errors without
setting the value of errno. However, this does not conform to the
expectations of most software, which expect it to return -1 and set
errno, which is how it works on other libcs, which document it
accordingly:

> The return value is defined by the system call being invoked.  In
> general, a 0 return value indicates success.  A -1 return value
> indicates an error, and an error number is stored in errno.
- Linux man-pages, syscall(2)

> The return value is the return value from the system call, unless
> the system call failed.  In that case, ‘syscall’ returns ‘-1’ and
> sets ‘errno’ to an error code that the system call returned.
- glibc manual, (libc)System Calls

> When the C-bit is set, syscall returns -1 and sets the external
> variable errno (see intro(2)).
- 4BSD manual, syscall(2)

> A -1 return value indicates an error, and an error code is stored in
> errno.
- 4.4BSD, FreeBSD, OpenBSD and NetBSD manuals (same quote is found in
  all of them), syscall(2)

This patch corrects the syscall function to work in the same way as in
other libcs.
2023-06-11 10:33:28 -07:00
tkchiaandtkchia e47c0cc929 [metal] Clean up code for getting pointer to struct mman (#828)
Co-authored-by: tkchia <[email protected]>
2023-06-11 10:32:39 -07:00
Justine Tunney 5a6a4819d4 Fix -iquote in cosmocc toolchain 2023-06-10 16:35:53 -07:00
Justine Tunney 2676ec55de Make improvements to cosmocc toolchain 2023-06-10 15:50:01 -07:00
Justine Tunney 8ff48201ca Rewrite .zip.o file linker
This change takes an entirely new approach to the incremental linking of
pkzip executables. The assets created by zipobj.com are now treated like
debug data. After a .com.dbg is compiled, fixupobj.com should be run, so
it can apply fixups to the offsets and move the zip directory to the end
of the file. Since debug data doesn't get objcopy'd, a new tool has been
introduced called zipcopy.com which should be run after objcopy whenever
a .com file is created. This is all automated by the `cosmocc` toolchain
which is rapidly becoming the new recommended approach.

This change also introduces the new C23 checked arithmetic macros.
2023-06-10 09:29:44 -07:00
Justine Tunney f6407d5f7c Fix missing includes 2023-06-09 18:11:53 -07:00
Justine Tunney 4a59210008 Introduce #include <cosmo.h> to toolchain users
This change improves the way internal APIs are being hidden behind the
`COSMO` define. The cosmo.h header will take care of defining that, so
that a separate define statement isn't needed. This change also does a
lot more to define which APIs are standard, and which belong to Cosmo.
2023-06-09 18:03:05 -07:00
Justine Tunney 9b55dbe417 Get GCC to mostly build with Cosmo 2023-06-09 06:41:34 -07:00
Justine Tunney 2eb7148474 Create a fake syscall() function due to Python 2023-06-09 05:18:38 -07:00
Justine Tunney 4b2023ffab Disable linker map generation and improve tinyness 2023-06-09 03:29:26 -07:00
Justine Tunney 23e235b7a5 Fix bugs in cosmocc toolchain
This change integrates e58abc1110b335a3341e8ad5821ad8e3880d9bb2 from
https://github.com/ahgamut/musl-cross-make/ which fixes the issues we
were having with our C language extension for symbolic constants. This
change also performs some code cleanup and bug fixes to getaddrinfo().
It's now possible to compile projects like ncurses, readline and python
without needing to patch anything upstream, except maybe a line or two.
Pretty soon it should be possible to build a Linux distro on Cosmo.
2023-06-08 23:44:03 -07:00
Justine Tunney 22f81a8d50 Improve cosmocc / cosmoc++ toolchain scripts
- Get out of the red zone
- Generate --ftrace nops unless -Os is passed
- Intercept -o path to generate .com / .com.dbg appropriately
2023-06-08 14:29:22 -07:00
Justine Tunney 25678db2a0 Fix remaining regressions in ASAN support
See #829
2023-06-08 08:00:50 -07:00
Justine Tunney 4d629fd424 Fix stack abuse in llama.cc
This change also incorporates improvements for MODE=asan. It's been
confirmed that o/asan/third_party/ggml/llama.com will work.

Fixes #829
2023-06-08 07:12:26 -07:00
Justine Tunney 32682f0ce7 Remove some problematic APIs
In order to improve our chances of success building other open source
projects we shouldn't define APIs that'll lead any ./configure script
astray. For example:

- brk() and sbrk() can break mac/windows support
- syscall() is a superb way to break portability
- arch_prctl() is the greatest of all horror shows
2023-06-08 06:12:26 -07:00
Justine Tunney 7512318a2a Fix MODE=aarch64 build 2023-06-08 05:17:37 -07:00
Justine Tunney 8767e9ad6a Fix --ftrace regression 2023-06-08 04:46:11 -07:00
Justine Tunney daf4454a06 Validate privileged code relationships
- Work towards improving non-optimized build support
- Introduce MODE=zero which is -O0 without ASAN/UBSAN
- Use system GCC when ~/.cosmo.mk has USE_SYSTEM_TOOLCHAIN=1
- Have package.com check .privileged code doesn't call non-privileged
2023-06-08 04:38:06 -07:00
Justine Tunney 01fd655097 Get garbage collector working on aarch64
Garbage collection will now happen on arm64 when a function returns,
rather than kicking the can down the road to when the process exits.
This change also does some code cleanup and incorporates suggestions
2023-06-07 03:34:45 -07:00
Justine Tunney 9793d3524f Remove superfluous RET instructions 2023-06-06 19:46:56 -07:00
Justine Tunney b6182db813 Simplify ftrace_hook()
We now have a test to prove that its transitive closure doesn't perform
floating point computations.
2023-06-06 11:10:38 -07:00
Justine Tunney 61b9677c05 Make improvements
- Get mprotect_test working on aarch64
- Get completion working on python.com repl again
- Improve quality of printvideo.com and printimage.com
- Fix bug in openpty() so examples/script.c works again
2023-06-06 09:12:30 -07:00
Justine Tunney b94b29d79c Prevent ftrace from misaligning functions 2023-06-06 06:00:31 -07:00
Justine Tunney b8a6a989c0 Create ELF aliases for identical symbols
This change greatly reduces the number of modules that need to be
compiled. The only issue right now is that sometimes when viewing
symbol table entries, the aliased symbol is chosen.
2023-06-06 03:33:49 -07:00
Justine Tunney e1b83399bd Fix memmove() alias on aarch64 with --ftrace 2023-06-06 00:39:55 -07:00
Justine Tunney e9e8bbe6da Avoid /etc/services in whois command
Apparently IANA has abolished the WHOIS protocol and no longer lists it
as a service. Therefore distros which naively create /etc/services from
IANA's braindead recommendation will inadvertently break any tools that
rely on /etc/services to determine this well-known Internet port.
2023-06-06 00:11:41 -07:00
Justine Tunney eb40cb371d Get --ftrace working on aarch64
This change implements a new approach to function call logging, that's
based on the GCC flag: -fpatchable-function-entry. Read the commentary
in build/config.mk to learn how it works.
2023-06-05 23:35:31 -07:00
Justine Tunney 5b908bc756 Fix some build errors 2023-06-05 15:53:44 -07:00
Justine Tunney 7558549d44 Test m=aarch64 on GitHub Actions 2023-06-05 15:23:00 -07:00
Justine Tunney 42b9b75749 Fix aarch64 build 2023-06-05 05:57:44 -07:00
Justine Tunney f554dd800e Make it possible to log kprintf() to file
It's now possible to compile Emacs using cosmocc. However we need to
troubleshoot why it's event loop isn't working correctly at runtime.
2023-06-05 04:16:15 -07:00
Justine Tunney 9cc3e37263 Upgrade to Cosmopolitan GCC 11.2.0 for aarch64 2023-06-05 02:07:28 -07:00
Justine Tunney 39f20dbb13 Upgrade to Cosmopolitan GCC 11.2.0 for x86_64 2023-06-05 02:06:18 -07:00
Justine Tunney 682b74ed88 Incorporate Blink 1.0.0 release 2023-06-04 11:07:31 -07:00
Justine Tunney 35dcaca53c Fix test fleet errors 2023-06-04 10:57:11 -07:00
Justine Tunney a91a945b88 Fix build breakage 2023-06-04 09:33:48 -07:00
Justine Tunney 4aa1d09b9e Improve aarch64 native support some more
This change introduces partial support for automating remote testing of
aarch64 binaries on Raspberry Pi and Apple Silicon.
2023-06-04 08:58:47 -07:00
Justine Tunney fc34ba2596 Fix Linenoise REPL on AARCH64 2023-06-04 02:57:17 -07:00
Justine Tunney bcf9af94bf Get threads working well on MacOS Arm64
- Now using 10x better GCD semaphores
- We now generate Linux-like thread ids
- We now use fast system clock / sleep libraries
- The APE M1 loader now generates Linux-like stacks
2023-06-04 01:57:10 -07:00
Justine Tunney b5eab2b0b7 Get POSIX threads working on Apple Silicon
It's now possible to run a working

    ape-m1 o/aarch64/third_party/ggml/llama.com

on Apple M1 hardware running XNU!
2023-06-03 18:33:01 -07:00
Justine Tunney 8fdb31681a Introduce support for GGJT v3 file format
llama.com can now load weights that use the new file format which was
introduced a few weeks ago. Note that, unlike llama.cpp, we will keep
support for old file formats in our tool so you don't need to convert
your weights when the upstream project makes breaking changes. Please
note that using ggjt v3 does make avx2 inference go 5% faster for me.
2023-06-03 15:46:21 -07:00
Justine Tunney 6ae18a10ba Fix MODE=tiny build 2023-06-03 10:30:48 -07:00
Justine Tunney 1904a3cae8 Sync llama.cpp to 6986c7835adc13ba3f9d933b95671bb1f3984dc6 2023-06-03 10:29:12 -07:00
Justine Tunney 8f522cb702 Make improvements
This change progresses our AARCH64 support:

- The AARCH64 build and tests are now passing
- Add 128-bit floating-point support to printf()
- Fix clone() so it initializes cosmo's x28 TLS register
- Fix TLS memory layout issue with aarch64 _Alignas vars
- Revamp microbenchmarking tools so they work on aarch64
- Make some subtle improvements to aarch64 crash reporting
- Make kisdangerous() memory checks more accurate on aarch64
- Remove sys_open() since it's not available on Linux AARCH64

This change makes general improvements to Cosmo and Redbean:

- Introduce GetHostIsa() function in Redbean
- You can now feature check using pledge(0, 0)
- You can now feature check using unveil("",0)
- Refactor some more x86-specific asm comments
- Refactor and write docs for some libm functions
- Make the mmap() API behave more similar to Linux
- Fix WIFSIGNALED() which wrongly returned true for zero
- Rename some obscure cosmo keywords from noFOO to dontFOO
2023-06-03 08:12:22 -07:00
Paul Kulchenko 5655c9a4e7 Extend Pledge with anet (same as inet, but with no connect) (#827)
* Add `anet` pledge for `inet` without connect

This is useful for configurations where it's desirable to start redbean
under these restrictions, but not to allow `connect` socket calls.

* Update message on protected/unpledged syscalls for clarity

* Update redbean to add reporting for unpledged sigaction

Previously it would abort without indicating what signal it failed to
install when sigaction is not pledged (although it fails all of them).

* Move GetHostIps before processing command line options

This allows using unix.pledge as part of the options without affecting
retrieving host IP addresses (which requires `connect`). It may still
fail under external `pledge` command as expected; in this case IPs
would need to be passed manually.

* Update tests for pledge anet promise
2023-06-03 07:50:29 -07:00
Justine Tunney 72f8bd10b7 Perform some code cleanup 2023-05-28 19:42:00 -07:00
Connor 992a4638ae Fix rgbtoxterm tool to output the proper colors (#819) 2023-05-26 18:30:24 -07:00
tkchiaandtkchia 90645f4dac [metal] Fix some issues that were breaking UEFI boot (#822)
Co-authored-by: tkchia <[email protected]>
2023-05-26 18:29:25 -07:00
Paul Kulchenko 5c40800d69 Update redbean ProgramDirectory to update package.path (#823) 2023-05-26 18:29:00 -07:00
Justine Tunney 1422e96b4e Introduce native support for MacOS ARM64
There's a new program named ape/ape-m1.c which will be used to build an
embeddable binary that can load ape and elf executables. The support is
mostly working so far, but still chasing down ABI issues.
2023-05-20 04:17:03 -07:00
Justine Tunney b852650c08 Fiddle around with Mach-O 2023-05-20 04:13:49 -07:00
Paul Kulchenko 6881a2ecea Extend redbean Fetch to add option to keeping connection open (#818) 2023-05-17 20:49:26 -07:00
Paul Kulchenko 021c71b651 Fix redbean to avoid compressing the first yielded chunk. (#813)
This may happen if no Status or Content-Type is set and if the written
content is longer than 100B, the compression is going to be applied
after coroutine.yield, even though it shouldn't be.
2023-05-17 20:46:27 -07:00
Justine Tunney e7eb0b3070 Make more ML improvements
- Fix UX issues with llama.com
- Do housekeeping on libm code
- Add more vectorization to GGML
- Get GGJT quantizer programs working well
- Have the quantizer keep the output layer as f16c
- Prefetching improves performance 15% if you use fewer threads
2023-05-16 08:07:23 -07:00
Justine Tunney 80db9de173 Make the intrinsics more readable 2023-05-15 23:12:11 -07:00
Justine Tunney 210187cf77 Perform some code cleanup 2023-05-15 16:32:10 -07:00
Justine Tunney cc1732bc42 Make AARCH64 harder, better, faster, stronger
- Perform some housekeeping on scalar math function code
- Import ARM's Optimized Routines for SIMD string processing
- Upgrade to latest Chromium zlib and enable more SIMD optimizations
2023-05-15 02:15:34 -07:00
Justine Tunney 550b52abf6 Port a lot more code to AARCH64
- Introduce epoll_pwait()
- Rewrite -ftrapv and ffs() libraries in C code
- Use more FreeBSD code in math function library
- Get significantly more tests passing on qemu-aarch64
- Fix many Musl long double functions that were broken on AARCH64
2023-05-14 09:37:26 -07:00
Ariel Núñez 91791e9f38 Started removing features from RedPajama to make it easier to understand for beginners (#817) 2023-05-14 09:16:22 -07:00
Justine Tunney 89d1fad7ee Enable crash reports for radpajama executables 2023-05-13 21:16:03 -07:00
Justine Tunney 296ee3ec58 Make some other fixes to radpajama build config 2023-05-13 21:09:28 -07:00
Justine Tunney 282dd8e7b7 Get radpajama to build
make -j8 o//third_party/radpajama/radpajama.com
    make -j8 o//third_party/radpajama/radpajama-chat.com

This change gets the radpajama.mk config working. This package depends
on THIRD_PARTY_GGML but it's configured to call ggjt_v1(), so that the
library will provide the old quantizers. The ggml_quantize_chunk() API
will now dispatch to older quantizers based on the configured version.
2023-05-13 20:44:36 -07:00
Justine Tunney 410c8785c9 Fix the AARCH64 build 2023-05-13 08:19:44 -07:00
Justine Tunney 5a4cf9560f Add support for new GGJT v2 quantizers
This change makes quantized models (e.g. q4_0) go 10% faster on Macs
however doesn't offer much improvement for Intel PC hardware.

This change syncs llama.cpp 699b1ad7fe6f7b9e41d3cb41e61a8cc3ea5fc6b5
which recently made a breaking change to nearly all its file formats
without any migration. Since that'll break hundreds upon hundreds of
models on websites like HuggingFace llama.com will support both file
formats because llama.com will never ever break the GGJT file format
2023-05-13 08:08:32 -07:00
Justine Tunney ba49e86e20 Get TEST_LIBC_CALLS passing on AARCH64 2023-05-13 02:41:41 -07:00
Justine Tunney 802e7eb4ef Mop up more test regressions 2023-05-13 01:09:44 -07:00
Justine Tunney 4a8a81eb9f Fix llama.com interactive mode regressions 2023-05-13 00:09:38 -07:00
Justine Tunney fd34ef732d Make considerably more progress on AARCH64
- Utilities like pledge.com now build
- kprintf() will no longer balk at 48-bit addresses
- There's a new aarch64-dbg build mode that should work
- gc() and defer() are mostly pacified; avoid using them on aarch64
- THIRD_PART_STB now has Arm Neon intrinsics for fast image handling
2023-05-12 22:42:57 -07:00
Justine Tunney 1bfb3aab1b Make Arm Neon intrinsics work with make tags 2023-05-12 18:32:53 -07:00
Justine Tunney 414667b1c9 Get TEST_LIBC_STR passing on AARCH64
It's now possible to run commands like:

    make -j8 m=aarch64 o/aarch64/test/libc/str

Which will cross-compile and run the test suites in a qemu-aarch64
binary that's vendored in the third_party/qemu/ folder within your
x86_64 build environment.
2023-05-12 18:09:23 -07:00
Justine Tunney 45186c74ac Introduce -q (quiet flag) and improve ctrl-c ux 2023-05-12 09:46:07 -07:00
Justine Tunney e8de1e4766 Fix subtoken antiprompt scanning 2023-05-12 08:55:40 -07:00
Justine Tunney 80c174d494 Clean up llama.com anti/stop/reverse-prompt code
Example use case for JSON completion:

    $ m=opt
    $ make -j16 m=$m o/$m/third_party/ggml/llama.com
    $ o/$m/third_party/ggml/llama.com -m llama.bin -p '{"key": "life", "val": ' -r '}'
    42}

This provides better control. More sophisticated facilities for
controlling text generation will be provided soon enough.
2023-05-12 08:20:58 -07:00
Justine Tunney bbfe4fbd11 Make llama.com n_predict be -1 by default 2023-05-12 08:20:34 -07:00
Justine Tunney ca19ecf49c Fine tune crash reports for llama.com 2023-05-12 06:24:26 -07:00
Justine Tunney 1f2a5a8fc1 Implement crash reporting for AARCH64
The ShowCrashReports() feature for aarch64 should work even better than
the x86 crash reports. Thanks to the benefit of hindsight these reports
should be rock solid reliable and beautiful to read.

This change also improves the syscall polyfills for aarch64. Some of the
sys_foo() functions have been removed, usually because they're legacy or
downright footguns not worth building.
2023-05-12 05:47:54 -07:00
Justine Tunney 285e8a2348 Get backtraces working on AARCH64 2023-05-11 22:12:41 -07:00
Justine Tunney 4edbc98811 Get MbedTLS and its unit tests passing AARCH64 2023-05-11 21:53:15 -07:00
Justine Tunney 5e2f7f7ced Get LIBC_TESTLIB building on AARCH64 2023-05-11 19:57:09 -07:00
Justine Tunney 95fab334e4 Use yield on aarch in spin locks 2023-05-11 19:57:09 -07:00
Ariel Núñez b3e3359d22 Import radpajama (a redpajama.cpp fork) (#814)
This is the relevant commit: https://github.com/togethercomputer/redpajama.cpp/commit/bfa6466199b8ef92185ecb72e2a550e12baf6602

Model download links:
https://huggingface.co/ceonlabs/radpajama/tree/main
2023-05-11 07:12:08 -07:00
Justine Tunney cba15bc77a Fix regression in DNS client
Calling FreeResolvConf() would crash, due to a recent change from
yesterday where free_s() was removed.
2023-05-10 23:27:10 -07:00
Justine Tunney 8d24a18c62 Tune magic division for size
This helps it avoid generating a branch instruction.
2023-05-10 21:32:04 -07:00
Justine Tunney 1f6f9e6701 Remove division from matrix multiplication
This change reduces llama.com CPU cycles systemically by 2.5% according
to the Linux Kernel `perf stat -Bddd` utility.
2023-05-10 21:19:54 -07:00
Justine Tunney a88290e595 Make sure llama.com terminal cleanup happens 2023-05-10 15:56:01 -07:00
Justine Tunney 5250feb7ad There must only be one strerror() 2023-05-10 15:34:13 -07:00
Justine Tunney c91de6a356 Avoid running clang-format on ggml.c 2023-05-10 14:59:33 -07:00
Justine Tunney ea607781f8 Fix the build 2023-05-10 10:14:15 -07:00
Justine Tunney bb3ebedfce Fix load time measurement 2023-05-10 07:54:21 -07:00
Justine Tunney 290a49952e Fix some more issues with aarch64 and llama.cpp 2023-05-10 07:34:26 -07:00
Justine Tunney 64aca4dc4f Fix weird behavior issue w/ kprintf() on aarch64 2023-05-10 06:17:33 -07:00
Justine Tunney 4e81d3277c Do some basic housekeeping in LIBC_STR 2023-05-10 06:17:20 -07:00
Justine Tunney 12a33858c9 There must be only one clock() 2023-05-10 06:16:01 -07:00
Justine Tunney 6cb9553706 Fix alignment bug in llama.com 2023-05-10 06:15:32 -07:00
Justine Tunney ca990ef091 Make llama.com -h print to stdout 2023-05-10 04:55:59 -07:00
Justine Tunney f312f706f4 Bring MODE=tiny binary sizes down to 20kb minimum
aarch64 binaries start at 4kb.
2023-05-10 04:20:48 -07:00
Justine Tunney 5f57fc1f59 Upgrade llama.cpp to e6a46b0ed1884c77267dc70693183e3b7164e0e0 2023-05-10 04:20:48 -07:00
Justine Tunney 5a455eaa0b Work on magic numbers for aarch64 2023-05-10 04:20:48 -07:00
Justine Tunney 59766efd3e Do some more aarch64 fixups 2023-05-10 04:20:47 -07:00
Justine Tunney 86d9323a43 Remove sys_getrandom() on NetBSD
This fixes an apparent regression caused by
3f0bcdc3ef where getrandom() on NetBSD 9.2
doesn't appear to work; ktrace oddly reports:

    1446      1 .ape     CALL  #91 (unimplemented getdopt)
    1446      1 .ape     RET   #91 (unimplemented getdopt) -1 errno 78
    Function not implemented
    1446      1 .ape     PSIG  SIGSYS SIG_DFL: code=SI_NOINFO
2023-05-10 04:20:47 -07:00
Justine Tunney a0237a017c Get llama.com working on aarch64 2023-05-10 04:20:47 -07:00
Justine Tunney 4c093155a3 Get llama.com building as an aarch64 native binary 2023-05-10 04:20:47 -07:00
Justine Tunney d04430f4ef Get LIBC_MEM and LIBC_STDIO building with aarch64 2023-05-10 04:20:47 -07:00
Justine Tunney ae0ee59614 Get aarch64 hello world working
$ m=aarch64-tiny
    $ make -j8 m=$m o/$m/tool/hello/hello.com o/third_party/qemu/qemu-aarch64
    $ o/third_party/qemu/qemu-aarch64 o/$m/tool/hello/hello.com
    hello world
    $ ls -hal o/$m/tool/hello/hello.com
    -rwxr-xr-x 1 jart jart 4.0K May  9 05:04 o/aarch64-tiny/tool/hello/hello.com
2023-05-10 04:20:47 -07:00
Justine Tunney e5e3cdf447 Get LIBC_RUNTIME and LIBC_CALLS building on aarch64 2023-05-10 04:20:47 -07:00
Justine Tunney 7e46645193 Get LIBC_FMT to build for aarch64 2023-05-10 04:20:47 -07:00
Justine Tunney 036b9a0002 Make further progress on non-x86 support 2023-05-10 04:20:47 -07:00
Justine Tunney aef9a69a60 Make more progress on aarch64 2023-05-10 04:20:47 -07:00
Justine Tunney 135080fd3e Get libc/tinymath/ compiling on aarch64 2023-05-10 04:20:46 -07:00
Justine Tunney 2b73e72d59 Make more code aarch64 friendly 2023-05-10 04:20:46 -07:00
Justine Tunney ca2860947f Make progress towards aarch64 build 2023-05-10 04:20:46 -07:00
Justine Tunney 08ff26c817 Add qemu-aarch64 2023-05-10 04:20:46 -07:00
Justine Tunney 8303e23b3a Do some basic build tuning 2023-05-10 04:20:46 -07:00
Justine Tunney 57cc257f58 Vendor musl-cross-make gcc 9.2.0 aarch64 2023-05-10 04:20:46 -07:00
Paul Kulchenko a493b64233 Merge pull request #807 from jart/evalarg
Fix the behavior of `redbean -i -e CODE`
2023-05-02 07:48:43 -07:00
Justine Tunney b008950d90 Add libcxx to make toolchain 2023-05-01 14:12:46 -07:00
Justine Tunney 12438cce16 Fix regression with Python linker eaxmples
We can once again create 2mb statically-linked Python binaries:

    $ make -j8 m=tiny o/tiny/examples/pyapp/pyapp.com
    $ ls -hal o/tiny/examples/pyapp/pyapp.com
    -rwxr-xr-x 1 jart jart 2.1M May  1 14:04 o/tiny/examples/pyapp/pyapp.com
    $ o/tiny/examples/pyapp/pyapp.com
    cosmopolitan is cool!

The regression was caused by Python thread support in b15f9eb58
2023-05-01 14:12:15 -07:00
Justine Tunney fc82f77a46 Fix bug in cosmocc / cosmoc++ scripts
The necessary flags weren't being passed when compiling programs as a
single step (i.e. not using `-c` to make intermediate objects).
2023-05-01 13:46:15 -07:00
Justine Tunney 3dac9f8999 Use Companion AI in llama.com by default 2023-04-30 23:08:15 -07:00
Justine Tunney d9e27203d4 Incorporate some fixes and updates for GGML 2023-04-28 20:24:55 -07:00
Justine Tunney b31ba86ace Introduce prompt caching so prompts load instantly
This change also introduces an ephemeral status line in non-verbose mode
to display a load percentage status when slow operations are happening.
2023-04-28 16:15:26 -07:00
Justine Tunney bf6459e324 Fix bugs with cli flags in gzip.com 2023-04-28 05:15:58 -07:00
Justine Tunney 1c2da3a55a Make shell usability improvements to llama.cpp
- Introduce -v and --verbose flags
- Don't print stats / diagnostics unless -v is passed
- Reduce --top_p default from 0.95 to 0.70
- Change --reverse-prompt to no longer imply --interactive
- Permit --reverse-prompt specifying custom EOS if non-interactive
2023-04-28 02:54:11 -07:00
Justine Tunney 420f889ac3 Further optimize the math library
The sincosf() function is now twice as fast, thanks to ARM Limited. The
same might also be true of logf() and expm1f() which have been updated.
2023-04-28 01:20:47 -07:00
Justine Tunney e8b43903b2 Import llama.cpp
https://github.com/ggerganov/llama.cpp
0b2da20538d01926b77ea237dd1c930c4d20b686
See third_party/ggml/README.cosmo for changes
2023-04-27 14:37:14 -07:00
Justine Tunney f42089d5c6 Be more lenient about stack memory
The new stack size is 256kb in order to compromise with llama.cpp's
aggressive use of stack memory, which can't be easily patched. This
change disables the dynamic alloca() and VLA warnings for now, plus
frame sizes for individual functions may be <=50% of the stack size

This only applies to code in the cosmo monorepo. Open source builds
should already be using an 8mb stack by default, like everyone else
2023-04-27 10:45:47 -07:00
Justine Tunney fec139f423 Add mlock() and munlock() 2023-04-27 10:42:52 -07:00
Justine Tunney 7c9ef924bf Add more sorting algorithms 2023-04-27 05:44:44 -07:00
Justine Tunney b7bf052a4b Add Intel intrinsics headers 2023-04-27 05:44:38 -07:00
Justine Tunney 369f9740de Run clang-format on most sources 2023-04-27 05:44:32 -07:00
25352 changed files with 1132447 additions and 484148 deletions

No files matched your search

+2
View File
@@ -6,6 +6,8 @@ AlignConsecutiveMacros: true
AlignConsecutiveDeclarations: false
AlwaysBreakBeforeMultilineStrings: false
AllowShortFunctionsOnASingleLine: false
AllowShortLoopsOnASingleLine: false
AllowShortIfStatementsOnASingleLine: false
KeepEmptyLinesAtTheStartOfBlocks: true
ConstructorInitializerAllOnOneLineOrOnePerLine: true
IncludeBlocks: Merge
+29
View File
@@ -0,0 +1,29 @@
# Run this command to always ignore formatting commits in git blame
# git config blame.ignoreRevsFile .git-blame-ignore-revs
# Upgraded clang-format to 19.0.0git
89fc95fefd40413706ff1bdaac2e8d98b328dbf1
# vim c++ filetype in modelines
04c6bc478e082263d67c41bedbd033dde2d429eb
# Ran clang-format
f032b5570b4cd87c6bb4abb54c0b98e69c939955
# Applied clang-format update to repo
6e6fc38935054db0534d5af4fb99c6193305b946
# revert retabbing
2b315626f3af765cdfbc61114647412cdb798b3a
# more modeline errata
3a8e01a77a7c97af0b16fb1651b230cee7f7d4c6
# fix more vi modelines
2fc507c98f53a76718f61f9a36602f86b5ac0cc9
# flip et/noet in modelines
e16a7d8f3b8f906c3ef76e79f57f3adfc7f25186
# fix vi modelines
394d998315f613a888cc6b6c051d4163bdf5cd6f
# clang-format
c0eacf2eb1e1c0b3bd4f71f12fef258f5b249c3f
# ape-m1 formatting cleanup
da8baf2aa5ce93b958aca90a0ae69f537806324b
# Run clang-format on most sources
369f9740de4534c28d0e81ab2afc99decbb9a3e6
# Get rid of .internal.h convention in LIBC_INTRIN
86d884cce24d773e298a2714c1e3d91ecab9be45
+1
View File
@@ -1,3 +1,4 @@
# -*- conf -*-
*.gz binary
/build/bootstrap/*.com binary
/usr/share/zoneinfo/* binary
+59
View File
@@ -0,0 +1,59 @@
name: Low Severity Bugs
description: Used to report low severity bugs in cosmopolitan (e.g. cosmetic issues, non critical UI glitches)
title: "Bug: "
labels: ["bug", "low severity"]
body:
- type: markdown
attributes:
value: |
Thanks for taking the time to fill out this bug report!
Please include information about your system, the steps to reproduce the bug,
and the version of cosmopolitan that you are using.
If possible, please provide a minimal code example that reproduces the bug.
You may also consider using function call tracing `--ftrace` or the lighter system call tracing `--strace`
for additional technical logging that may allow us to narrow down where the fault occurred.
- type: input
id: contact
attributes:
label: Contact Details
description: How can we get in touch with you if we need more info?
placeholder: ex. [email protected]
validations:
required: false
- type: textarea
id: what-happened
attributes:
label: What happened?
description: Also tell us, what did you expect to happen?
placeholder: Tell us what you see!
validations:
required: true
- type: textarea
id: version
attributes:
label: Version
description: What version of our software are you running? (use `--version` to get a version string)
placeholder: "cosmocc (GCC) 12.3.0"
validations:
required: true
- type: dropdown
id: operating-system
attributes:
label: What operating system are you seeing the problem on?
multiple: true
options:
- Linux
- Mac
- Windows
- FreeBSD
- OpenBSD
- NetBSD
- BIOS
validations:
required: false
- type: textarea
id: logs
attributes:
label: Relevant log output
description: Please copy and paste any relevant log output. This will be automatically formatted into code, so no need for backticks.
render: shell
+59
View File
@@ -0,0 +1,59 @@
name: Medium Severity Bug
description: Used to report medium severity bugs in cosmopolitan (e.g. Malfunctioning Features but generally still useable)
title: "Bug: "
labels: ["bug", "medium severity"]
body:
- type: markdown
attributes:
value: |
Thanks for taking the time to fill out this bug report!
Please include information about your system, the steps to reproduce the bug,
and the version of cosmopolitan that you are using.
If possible, please provide a minimal code example that reproduces the bug.
You may also consider using function call tracing `--ftrace` or the lighter system call tracing `--strace`
for additional technical logging that may allow us to narrow down where the fault occurred.
- type: input
id: contact
attributes:
label: Contact Details
description: How can we get in touch with you if we need more info?
placeholder: ex. [email protected]
validations:
required: false
- type: textarea
id: what-happened
attributes:
label: What happened?
description: Also tell us, what did you expect to happen?
placeholder: Tell us what you see!
validations:
required: true
- type: textarea
id: version
attributes:
label: Version
description: What version of our software are you running? (use `--version` to get a version string)
placeholder: "cosmocc (GCC) 12.3.0"
validations:
required: true
- type: dropdown
id: operating-system
attributes:
label: What operating system are you seeing the problem on?
multiple: true
options:
- Linux
- Mac
- Windows
- FreeBSD
- OpenBSD
- NetBSD
- BIOS
validations:
required: false
- type: textarea
id: logs
attributes:
label: Relevant log output
description: Please copy and paste any relevant log output. This will be automatically formatted into code, so no need for backticks.
render: shell
+59
View File
@@ -0,0 +1,59 @@
name: High Severity Bug
description: Used to report high severity bugs in cosmopolitan (e.g. Malfunctioning features hindering important common workflow)
title: "Bug: "
labels: ["bug", "high severity"]
body:
- type: markdown
attributes:
value: |
Thanks for taking the time to fill out this bug report!
Please include information about your system, the steps to reproduce the bug,
and the version of cosmopolitan that you are using.
If possible, please provide a minimal code example that reproduces the bug.
You may also consider using function call tracing `--ftrace` or the lighter system call tracing `--strace`
for additional technical logging that may allow us to narrow down where the fault occurred.
- type: input
id: contact
attributes:
label: Contact Details
description: How can we get in touch with you if we need more info?
placeholder: ex. [email protected]
validations:
required: false
- type: textarea
id: what-happened
attributes:
label: What happened?
description: Also tell us, what did you expect to happen?
placeholder: Tell us what you see!
validations:
required: true
- type: textarea
id: version
attributes:
label: Version
description: What version of our software are you running? (use `--version` to get a version string)
placeholder: "cosmocc (GCC) 12.3.0"
validations:
required: true
- type: dropdown
id: operating-system
attributes:
label: What operating system are you seeing the problem on?
multiple: true
options:
- Linux
- Mac
- Windows
- FreeBSD
- OpenBSD
- NetBSD
- BIOS
validations:
required: false
- type: textarea
id: logs
attributes:
label: Relevant log output
description: Please copy and paste any relevant log output. This will be automatically formatted into code, so no need for backticks.
render: shell
@@ -0,0 +1,59 @@
name: Critical Severity Bug
description: Used to report critical severity bugs in cosmopolitan (e.g. Crashing, Corrupted, Dataloss)
title: "Bug: "
labels: ["bug", "critical severity"]
body:
- type: markdown
attributes:
value: |
Thanks for taking the time to fill out this bug report!
Please include information about your system, the steps to reproduce the bug,
and the version of cosmopolitan that you are using.
If possible, please provide a minimal code example that reproduces the bug.
You may also consider using function call tracing `--ftrace` or the lighter system call tracing `--strace`
for additional technical logging that may allow us to narrow down where the fault occurred.
- type: input
id: contact
attributes:
label: Contact Details
description: How can we get in touch with you if we need more info?
placeholder: ex. [email protected]
validations:
required: false
- type: textarea
id: what-happened
attributes:
label: What happened?
description: Also tell us, what did you expect to happen?
placeholder: Tell us what you see!
validations:
required: true
- type: textarea
id: version
attributes:
label: Version
description: What version of our software are you running? (use `--version` to get a version string)
placeholder: "cosmocc (GCC) 12.3.0"
validations:
required: true
- type: dropdown
id: operating-system
attributes:
label: What operating system are you seeing the problem on?
multiple: true
options:
- Linux
- Mac
- Windows
- FreeBSD
- OpenBSD
- NetBSD
- BIOS
validations:
required: false
- type: textarea
id: logs
attributes:
label: Relevant log output
description: Please copy and paste any relevant log output. This will be automatically formatted into code, so no need for backticks.
render: shell
+51
View File
@@ -0,0 +1,51 @@
name: Enhancement template
description: Used to request enhancements for cosmopolitan
title: "Feature Request: "
labels: ["enhancement"]
body:
- type: markdown
attributes:
value: |
[Please post your idea first in Discussion if there is not yet a consensus for this enhancement request. This will help to keep this issue tracker focused on enhancements that the community has agreed needs to be implemented.](https://github.com/jart/cosmopolitan/discussions/categories/ideas)
- type: checkboxes
id: prerequisites
attributes:
label: Prerequisites
description: Please confirm the following before submitting your enhancement request.
options:
- label: I am running the latest code. Mention the version if possible as well.
required: true
- label: I carefully followed the [README.md](https://github.com/jart/cosmopolitan/blob/master/README.md).
required: true
- label: I searched using keywords relevant to my issue to make sure that I am creating a new issue that is not already open (or closed).
required: true
- label: I reviewed the [Discussions](https://github.com/jart/cosmopolitan/discussions), and have a new and useful enhancement to share.
required: true
- type: textarea
id: feature-description
attributes:
label: Feature Description
description: Please provide a detailed written description of what you were trying to do, and what you expected `cosmopolitan` to do as an enhancement.
placeholder: Detailed description of the enhancement
validations:
required: true
- type: textarea
id: motivation
attributes:
label: Motivation
description: Please provide a detailed written description of reasons why this feature is necessary and how it is useful to `cosmopolitan` users.
placeholder: Explanation of why this feature is needed and its benefits
validations:
required: true
- type: textarea
id: possible-implementation
attributes:
label: Possible Implementation
description: If you have an idea as to how it can be implemented, please write a detailed description. Feel free to give links to external sources or share visuals that might be helpful to understand the details better.
placeholder: Detailed description of potential implementation
validations:
required: false
+52
View File
@@ -0,0 +1,52 @@
name: Research
description: Track new technical research area
title: "Research: "
labels: ["research"]
body:
- type: markdown
attributes:
value: |
Don't forget to check for any [duplicate research issue tickets](https://github.com/jart/cosmopolitan/issues?q=is%3Aopen+is%3Aissue+label%3A%22research+%F0%9F%94%AC%22)
- type: checkboxes
id: research-stage
attributes:
label: Research Stage
description: Track general state of this research ticket
options:
- label: Background Research (Let's try to avoid reinventing the wheel)
- label: Hypothesis Formed (How do you think this will work and it's effect?)
- label: Strategy / Implementation Forming
- label: Analysis of results
- label: Debrief / Documentation (So people in the future can learn from us)
- type: textarea
id: background
attributes:
label: Previous existing literature and research
description: Whats the current state of the art and whats the motivation for this research?
- type: textarea
id: hypothesis
attributes:
label: Hypothesis
description: How do you think this will work and it's effect?
- type: textarea
id: implementation
attributes:
label: Implementation
description: Got an approach? e.g. a PR ready to go?
- type: textarea
id: analysis
attributes:
label: Analysis
description: How does the proposed implementation behave?
- type: textarea
id: logs
attributes:
label: Relevant log output
description: Please copy and paste any relevant log output. This will be automatically formatted into code, so no need for backticks.
render: shell
+28
View File
@@ -0,0 +1,28 @@
name: Refactor (Maintainers)
description: Used to track refactoring opportunities
title: "Refactor: "
labels: ["refactor"]
body:
- type: markdown
attributes:
value: |
Don't forget to [check for existing refactor issue tickets](https://github.com/jart/cosmopolitan/issues?q=is%3Aopen+is%3Aissue+label%3Arefactoring) in case it's already covered.
Also you may want to check [Pull request refactor label as well](https://github.com/jart/cosmopolitan/pulls?q=is%3Aopen+is%3Apr+label%3Arefactoring) for duplicates too.
- type: textarea
id: background-description
attributes:
label: Background Description
description: Please provide a detailed written description of the pain points you are trying to solve.
placeholder: Detailed description behind your motivation to request refactor
validations:
required: true
- type: textarea
id: possible-approaches
attributes:
label: Possible Refactor Approaches
description: If you have some idea of possible approaches to solve this problem. You may want to make it a todo list.
placeholder: Your idea of possible refactoring opportunity/approaches
validations:
required: false
+15
View File
@@ -0,0 +1,15 @@
blank_issues_enabled: true
contact_links:
- name: FAQ
url: https://github.com/jart/cosmopolitan/wiki/FAQ
about: Is your question a common one? You may want to check here first.
- name: Got an idea?
url: https://github.com/jart/cosmopolitan/discussions/categories/ideas
about: Pop it there. It may then become an enhancement ticket.
- name: Got a question?
url: https://github.com/jart/cosmopolitan/discussions/categories/q-a
about: Ask a question there!
- name: Want to contribute?
url: https://github.com/jart/cosmopolitan/wiki/contribute
about: Head to the contribution guide page of the wiki for areas you can help with
+49
View File
@@ -0,0 +1,49 @@
# https://github.com/actions/labeler
documentation:
- changed-files:
- any-glob-to-any-file:
- README.md
- LICENSE
- CONTRIBUTING.md
- libc/README.md
- tool/cosmocc/README.md
- third_party/getopt/README.txt
build:
- changed-files:
- any-glob-to-any-file:
- build/**
- Makefile
- '*/*.mk'
examples:
- changed-files:
- any-glob-to-any-file: examples/**
devops:
- changed-files:
- any-glob-to-any-file:
- .github/**
- .clang-format
dsp:
- changed-files:
- any-glob-to-any-file:
- dsp/**
ape:
- changed-files:
- any-glob-to-any-file:
- ape/**
libc:
- changed-files:
- any-glob-to-any-file:
- libc/**
net:
- changed-files:
- any-glob-to-any-file:
- net/**
third_party:
- changed-files:
- any-glob-to-any-file:
- third_party/**
tool:
- changed-files:
- any-glob-to-any-file:
- tool/**
+1 -1
View File
@@ -17,7 +17,7 @@ jobs:
runs-on: ubuntu-latest
strategy:
matrix:
mode: ["", tiny, opt, rel]
mode: ["", tiny, rel, tinylinux, optlinux]
steps:
- uses: actions/checkout@v3
+17
View File
@@ -0,0 +1,17 @@
name: "Pull Request Labeler"
on:
- pull_request_target
jobs:
labeler:
permissions:
contents: read
pull-requests: write
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
repository: "jart/cosmopolitan"
- uses: actions/labeler@v5
with:
configuration-path: '.github/labeler.yml'
+2
View File
@@ -1,6 +1,8 @@
# -*- conf -*-
/o
/cosmocc
/.cosmocc
# TODO: Find some way to have Python write to o/
__pycache__
-112
View File
@@ -1,112 +0,0 @@
{
"configurations": [
{
"name": "Win32",
"includePath": [
"${workspaceFolder}"
],
"cStandard": "c11",
"cppStandard": "c++11",
"forcedInclude": ["${workspaceFolder}/.vscode/vscode.h"],
"defines": [
"libcesque=",
"pureconst=",
"paramsnonnull(x)=",
"alignas(x)",
"alignof(x)",
"artificial=",
"dontdiscard=",
"mayalias=",
"forceinline=",
"forcealign(x)=",
"scanfesque(x)=",
"strftimeesque(x)=",
"wontreturn=",
"textreal=",
"mallocesque=",
"callocesque=",
"vallocesque=",
"reallocesque=",
"strlenesque=",
"memcpyesque=",
"hasatleast=",
"noinline=",
"textexit=",
"returnstwice=",
"textwindows=",
"privileged=",
"compatfn=",
"noinstrument=",
"nodebuginfo=",
"interruptfn=",
"optimizespeed=",
"forcealignargpointer=",
"noasan=",
"noubsan=",
"donothing=",
"nosideeffect=",
"unreachable=",,
"notpossible=",
"thatispacked=",
"dontthrow=",
"nocallback=",
"relegated=",
"hidden=",
"textstartup=",
"initarray=",
"returnsnonnull=",
"returnspointerwithnoaliases=",
"printfesque(x)=",
"attributeallocsize(x)=",
"returnsaligned(x)=",
"attributeallocalign(x)=",
"nullterminated(x)="
]
},
{
"name": "Linux",
"includePath": [
"${workspaceFolder}"
],
"cStandard": "gnu17",
"compilerPath": "${workspaceFolder}/o/third_party/gcc/bin/x86_64-linux-musl-gcc",
"compilerArgs": [
"-Wall",
"-Werror",
"-fdebug-prefix-map=${workspaceFolder}=",
"-frecord-gcc-switches",
"-Wa,-W",
"-Wa,-I.",
"-Wa,--noexecstack",
"-Wa,--nocompress-debug-sections",
"-Og",
"-g",
"-gdescribe-dies",
"-msse3",
"-mno-red-zone",
"-fno-math-errno",
"-fno-trapping-math",
"-fno-fp-int-builtin-inexact",
"-fno-ident",
"-fno-common",
"-fno-gnu-unique",
"-fstrict-aliasing",
"-fstrict-overflow",
"-fno-semantic-interposition",
"-mno-omit-leaf-frame-pointer",
"-fno-jump-tables",
"-nostdinc",
"-iquote."
],
"forcedInclude": [
"libc/integral/normalize.inc"
],
"defines": [
"COSMO",
"MODE=",
"IMAGE_BASE_VIRTUAL=0x400000"
]
}
],
"version": 4
}
-2
View File
@@ -1,2 +0,0 @@
#define __VSCODE_INTELLISENSE__ 1
#include "libc/integral/normalize.inc"
+69 -45
View File
@@ -2,23 +2,77 @@
We'd love to accept your patches! Please read this guide first.
## Identity Disclosure
This project does not accept anonymous contributions. Justine Tunney
won't merge pull requests from strangers. In order to change the Cosmo
codebase, and have your changes be upstreamed, she has to know who you
are. You're encouraged to disclose your full name and email address to
the public too, by including them in your git commit messages; however
that's not a requirement; as we're happy to respect the wishes of
contributors who prefer to remain anonymous to the public.
## Copyright Assignment
Please send an email to Justine Tunney <jtunney@gmail.com> stating that
you intend to assign her the copyright to the changes you contribute to
Cosmopolitan. Please use the same email address you use for git commits
which should only contain original source code from you or other people
who are also assigning copyright. Please note that, if you're employed,
you may need to get your employer's approval beforehand. If you can not
assign copyright due to local laws, then you may alternatively consider
disclaiming it using the language in [Unlicense](https://unlicense.org)
or [CC-0](http://creativecommons.org/share-your-work/public-domain/cc0)
The first time you send a pull request, you need to send an email to
Justine Tunney <jtunney@gmail.com> stating that you intend to assign her
the copyright to the changes you contribute to Cosmopolitan. It only
needs to happen once. This only applies to the code you *choose* to
contribute. The email should be sent from an email address associated
with your identity. Your email should link to your pull request.
This is important because we can't produce 12kb single-file executables
that comply with license requirements if we have to embed lots of them.
Although that's less of an issue depending on the purpose of the files.
For example, ownership is much less of a concern in the unit test files
so you're encouraged to put your copyright on those, provided it's ISC.
To make things easy, here's an example of a good email you can use:
> **From**: YOUR NAME ([email protected])
> **To**: Justine Tunney ([email protected])
> **Subject**: Cosmopolitan Copyright Assignment for YOUR NAME
>
> Hi Justine,
>
> I made my first contribution to Cosmopolitan in
> https://github.com/jart/cosmopolitan/pull/XXXX could you please take a
> look? I intend to assign you the copyright to the changes I contribute
> to Cosmopolitan.
>
> Thanks!
Please note that in order to give Justine the copyright, it has to be
yours to give in the first place. If you're employed, then you should
get your employer's approval to do this beforehand. Even with big
companies like Google, this process is quick and painless. Usually we
see employers granting authorization in less than one day.
If you live in a country that doesn't recognize one's ability to assign
copyright, then you may alternatively consider disclaiming it using the
language in [Unlicense](https://unlicense.org) or
[CC-0](http://creativecommons.org/share-your-work/public-domain/cc0).
If you're checking-in third party code, then you need to have headers at
the top of each source file (but never header files) documenting its
owners and the code should go in the `third_party/` folder. Every third
party project should have a `README.cosmo` file that documents its
provenance as well as any local changes you've made.
## Copyright Policy Exceptions
### Tests
You're encoraged to claim ownership of your test code. If you add a new
file under the `test/` directory, then you should put your name in the
ISC license header at the top of the file. If you add new test cases to
an existing unit test file, then you're encouraged to append a line with
your name to the existing copyright header of that file.
### Exceptional Features
Let's say you discovered a faster better way to implement `log10()` and
you want to give it to Cosmopolitan. In cases like this, it really isn't
appropriate for Justine to own your code. What you could do instead, is
write your own new and improved `log10.c` from scratch, put your name on
the top with the ISC license, and then add a `__notice()` directive so
that your name will be embedded inside every executable that links the
`log10()` function. This will help you get your name out there. Please
note you need get approval from Justine each time you want to do this.
## Style Guide
@@ -29,34 +83,4 @@ clang-format -i -style=file tool/net/redbean.c
```
If you use Emacs this can be automated on save for Cosmopolitan using
[tool/emacs/cosmo-format.el]([tool/emacs/cosmo-format.el]).
### Source Files
- Must use include paths relative to the root of the repository
- Must have comment at top of file documenting copyright and license
- Must have notice embedding if not owned by Justine (exception: tests)
- May use language extensions that are supported by both GCC and Clang
- Should use Google indentation (otherwise use `/* clang-format off */`)
- Should use asm() instead of compiler APIs (exception: ctz, clz, memcpy)
### Header Files
- Must not have copyright or license comments
- Must have once guards (otherwise change `.h` to `.inc`)
- Must be ANSI C89 compatible to be included in the amalgamation header
- Must include its dependencies (exception: libc/integral/normalize.inc)
- Must not define objects (i.e. `cc -c -xc foo.h` will produce empty `.o`)
- Should not use typedefs
- Should not use forward declarations
- Should not include documentation comments
- Should not include parameter names in prototypes
- Should not pose problems if included by C++ or Assembly sources
- Should not declare non-ANSI code, at all, when the user requests ANSI
### Build Config
- Must not write files outside `o/`
- Must not communicate with Internet
- Must not depend on system libraries
- Must not depend on system commands (exception: sh, make, gzip, zip)
[tool/emacs/cosmo-format.el](tool/emacs/cosmo-format.el).
+397 -202
View File
@@ -1,5 +1,5 @@
#-*-mode:makefile-gmake;indent-tabs-mode:t;tab-width:8;coding:utf-8-*-┐
#───vi: set et ft=make ts=8 tw=8 fenc=utf-8 :vi───────────────────────┘
#── vi: set et ft=make ts=8 sw=8 fenc=utf-8 :vi ──────────────────────┘
#
# SYNOPSIS
#
@@ -21,23 +21,23 @@
# make -j8 -O MODE=tiny
#
# # build individual target
# make -j8 -O o//examples/hello.com
# o//examples/hello.com
# make -j8 -O o//examples/hello
# o//examples/hello
#
# # view source
# less examples/hello.c
#
# # view binary
# o//tool/viz/bing.com o//examples/hello.com |
# o//tool/viz/fold.com
# o//tool/viz/bing o//examples/hello |
# o//tool/viz/fold
#
# # view transitive closure of legalese
# o//tool/viz/bing.com -n o//examples/hello.com |
# o//tool/viz/fold.com
# o//tool/viz/bing -n o//examples/hello |
# o//tool/viz/fold
#
# # basic debugging
# make -j8 -O MODE=dbg o/dbg/examples/crashreport.com
# o/examples/crashreport.com
# make -j8 -O MODE=dbg o/dbg/examples/crashreport
# o/examples/crashreport
# less examples/crashreport.c
#
# # extremely tiny binaries
@@ -51,7 +51,7 @@
#
# TROUBLESHOOTING
#
# make -j8 -O V=1 o//examples/hello.com
# make -j8 -O V=1 o//examples/hello
# make o//examples/life.elf -pn |& less
# etc.
#
@@ -59,14 +59,13 @@
#
# build/config.mk
SHELL = build/bootstrap/cocmd.com
HOSTS ?= freebsd openbsd netbsd rhel7 rhel5 xnu win10
SHELL = build/bootstrap/cocmd
MAKEFLAGS += --no-builtin-rules
.SUFFIXES:
.DELETE_ON_ERROR:
.FEATURES: output-sync
.PHONY: all o bins check test depend tags
.PHONY: all o bins check test depend tags aarch64 clean
ifneq ($(m),)
ifeq ($(MODE),)
@@ -74,11 +73,110 @@ MODE := $(m)
endif
endif
COMMA := ,
PWD := $(shell pwd)
# detect wsl2 running cosmopolitan binaries on the host by checking whether:
# - user ran build/bootstrap/make, in which case make's working directory is in wsl
# - user ran make, in which case cocmd's working directory is in wsl
ifneq ($(findstring //wsl.localhost/,$(CURDIR) $(PWD)),)
$(warning wsl2 interop is enabled)
$(error you need to run sudo sh -c 'echo -1 > /proc/sys/fs/binfmt_misc/WSLInterop')
endif
UNAME_M := $(shell uname -m)
UNAME_S := $(shell uname -s)
# apple still distributes a 17 year old version of gnu make
ifeq ($(MAKE_VERSION), 3.81)
$(error please use build/bootstrap/make)
endif
LC_ALL = C
SOURCE_DATE_EPOCH = 0
ARFLAGS = rcsD
ZFLAGS ?=
XARGS ?= xargs -P4 -rs8000
DOT ?= dot
CLANG = clang
TMPDIR = o/tmp
AR = $(BOOTSTRAP)/ar.ape
CP = $(BOOTSTRAP)/cp.ape
RM = $(BOOTSTRAP)/rm.ape -f
GZIP = $(BOOTSTRAP)/gzip.ape
ECHO = $(BOOTSTRAP)/echo.ape
CHMOD = $(BOOTSTRAP)/chmod.ape
TOUCH = $(BOOTSTRAP)/touch.ape
PKG = $(BOOTSTRAP)/package.ape
MKDEPS = $(BOOTSTRAP)/mkdeps
ZIPOBJ = $(BOOTSTRAP)/zipobj
ZIPCOPY = $(BOOTSTRAP)/zipcopy
PECHECK = $(BOOTSTRAP)/pecheck
FIXUPOBJ = $(BOOTSTRAP)/fixupobj
OBJBINCOPY = $(BOOTSTRAP)/objbincopy
MKDIR = build/bootstrap/mkdir -p
COMPILE = build/bootstrap/compile -V9 -M2048m -P8192 $(QUOTA)
IGNORE := $(shell $(MKDIR) $(TMPDIR))
# the default build modes is empty string
# on x86_64 hosts, MODE= is the same as MODE=x86_64
# on aarch64 hosts, MODE= is changed to MODE=aarch64
ifeq ($(MODE),)
ifeq ($(UNAME_M),arm64)
MODE := aarch64
endif
ifeq ($(UNAME_M),aarch64)
MODE := aarch64
endif
endif
ifneq ($(findstring aarch64,$(MODE)),)
ARCH = aarch64
HOSTS ?= pi studio freebsdarm
else
ARCH = x86_64
HOSTS ?= freebsd rhel7 xnu openbsd netbsd win10
endif
ZIPOBJ_FLAGS += -a$(ARCH)
IGNORE := $(shell $(MKDIR) $(TMPDIR))
export ADDR2LINE
export LC_ALL
export MKDIR
export MODE
export SOURCE_DATE_EPOCH
export TMPDIR
COSMOCC = .cosmocc/3.6.0
BOOTSTRAP = $(COSMOCC)/bin
TOOLCHAIN = $(COSMOCC)/bin/$(ARCH)-linux-cosmo-
DOWNLOAD := $(shell build/download-cosmocc.sh $(COSMOCC) 3.6.0 4918c45ac3e0972ff260e2a249e25716881e39fb679d5e714ae216a2ef6c3f7e)
AS = $(TOOLCHAIN)as
CC = $(TOOLCHAIN)gcc
CXX = $(TOOLCHAIN)g++
CXXFILT = $(TOOLCHAIN)c++filt
LD = $(TOOLCHAIN)ld.bfd
NM = $(TOOLCHAIN)nm
GCC = $(TOOLCHAIN)gcc
STRIP = $(TOOLCHAIN)strip
OBJCOPY = $(TOOLCHAIN)objcopy
OBJDUMP = $(TOOLCHAIN)objdump
ifneq ($(wildcard $(PWD)/$(TOOLCHAIN)addr2line),)
ADDR2LINE = $(PWD)/$(TOOLCHAIN)addr2line
else
ADDR2LINE = $(TOOLCHAIN)addr2line
endif
# primary build rules
all: o
o: o/$(MODE)
o/$(MODE): \
o/$(MODE)/ape \
o/$(MODE)/ctl \
o/$(MODE)/dsp \
o/$(MODE)/net \
o/$(MODE)/libc \
@@ -87,20 +185,43 @@ o/$(MODE): \
o/$(MODE)/examples \
o/$(MODE)/third_party
# TODO(jart): Make Emacs `C-c C-c` shortcut not need this.
.PHONY: o/$(MODE)/ o/$(MODE)/.
o/$(MODE)/: o/$(MODE)
o/$(MODE)/.: o/$(MODE)
# check if we're using o//third_party/make/make
# we added sandboxing to guarantee cosmo's makefile is hermetic
# it also shaves away 200ms of startup latency with native $(uniq)
ifneq ($(LANDLOCKMAKE_VERSION),)
ifeq ($(UNAME_S),Linux)
ifeq ($(wildcard /usr/bin/ape),)
$(warning please run ape/apeinstall.sh if you intend to use landlock make)
$(shell sleep .5)
endif
endif
ifneq ($(TOOLCHAIN),)
.STRICT = 1
.PLEDGE = stdio rpath wpath cpath fattr proc
.UNVEIL = \
libc/integral \
libc/disclaimer.inc \
rwc:/dev/shm \
rx:build/bootstrap \
rx:o/third_party/gcc \
/proc/stat \
rw:/dev/null \
w:o/stack.log \
/etc/hosts \
~/.runit.psk \
/proc/self/status \
endif
endif
.PLEDGE += stdio rpath wpath cpath fattr proc
.UNVEIL += \
libc/integral \
libc/stdbool.h \
rwc:/dev/shm \
rx:.cosmocc \
rx:build/bootstrap \
r:build/portcosmo.h \
/proc/stat \
rw:/dev/null \
rw:/dev/full \
w:o/stack.log \
/etc/hosts \
~/.runit.psk \
/proc/self/status \
rx:/usr/bin/qemu-aarch64 \
rx:o/third_party/qemu/qemu-aarch64 \
/sys/devices/system/cpu/cpu0/cpufreq/scaling_governor
PKGS =
@@ -110,145 +231,160 @@ include build/functions.mk #─┐
include build/definitions.mk # ├──META
include build/config.mk # │ You can build
include build/rules.mk # │ You can topologically order
include build/online.mk # │
include libc/stubs/stubs.mk #─┘
include libc/nexgen32e/nexgen32e.mk #─┐
include libc/sysv/sysv.mk # ├──SYSTEM SUPPORT
include libc/nt/nt.mk # │ You can do math
include libc/intrin/intrin.mk # │ You can use the stack
include libc/linux/linux.mk # │ You can manipulate arrays
include libc/tinymath/tinymath.mk # │ You can issue raw system calls
include third_party/compiler_rt/compiler_rt.mk # │
include libc/str/str.mk # │
include third_party/xed/xed.mk # │
include third_party/puff/puff.mk # │
include third_party/zlib/zlib.mk # │
include third_party/double-conversion/dc.mk # │
include libc/elf/elf.mk # │
include ape/ape.mk # │
include libc/fmt/fmt.mk # │
include libc/vga/vga.mk #─┘
include libc/calls/calls.mk #─┐
include third_party/getopt/getopt.mk # │
include libc/runtime/runtime.mk # ├──SYSTEMS RUNTIME
include libc/crt/crt.mk # │ You can issue system calls
include third_party/nsync/nsync.mk # │
include third_party/dlmalloc/dlmalloc.mk #─┘
include libc/mem/mem.mk #─┐
include third_party/gdtoa/gdtoa.mk # ├──DYNAMIC RUNTIME
include third_party/nsync/mem/mem.mk # │ You can now use stdio
include libc/thread/thread.mk # │ You can finally call malloc()
include libc/zipos/zipos.mk # │
include libc/time/time.mk # │
include libc/stdio/stdio.mk # │
include third_party/libcxx/libcxx.mk # │
include net/net.mk # │
include libc/log/log.mk # │
include third_party/bzip2/bzip2.mk # │
include dsp/core/core.mk # │
include libc/x/x.mk # │
include third_party/stb/stb.mk # │
include dsp/scale/scale.mk # │
include dsp/mpeg/mpeg.mk # │
include dsp/dsp.mk # │
include third_party/zlib/gz/gz.mk # │
include third_party/musl/musl.mk # │
include libc/libc.mk #─┘
include libc/sock/sock.mk #─┐
include dsp/tty/tty.mk # ├──ONLINE RUNTIME
include libc/dns/dns.mk # │ You can communicate with the network
include net/http/http.mk # │
include third_party/mbedtls/mbedtls.mk # │
include net/https/https.mk # │
include third_party/regex/regex.mk #─┘
include third_party/tidy/tidy.mk
include third_party/third_party.mk
include third_party/nsync/testing/testing.mk
include libc/testlib/testlib.mk
include tool/viz/lib/vizlib.mk
include tool/args/args.mk
include test/tool/args/test.mk
include third_party/linenoise/linenoise.mk
include third_party/maxmind/maxmind.mk
include net/finger/finger.mk
include third_party/double-conversion/test/test.mk
include third_party/lua/lua.mk
include third_party/tr/tr.mk
include third_party/sed/sed.mk
include third_party/awk/awk.mk
include third_party/hiredis/hiredis.mk
include third_party/make/make.mk
include third_party/ctags/ctags.mk
include third_party/finger/finger.mk
include third_party/argon2/argon2.mk
include third_party/smallz4/smallz4.mk
include third_party/sqlite3/sqlite3.mk
include third_party/mbedtls/test/test.mk
include third_party/quickjs/quickjs.mk
include third_party/lz4cli/lz4cli.mk
include third_party/zip/zip.mk
include third_party/unzip/unzip.mk
include tool/build/lib/buildlib.mk
include third_party/chibicc/chibicc.mk
include third_party/chibicc/test/test.mk
include third_party/python/python.mk
include tool/build/emucrt/emucrt.mk
include tool/build/emubin/emubin.mk
include tool/build/build.mk
include tool/curl/curl.mk
include examples/examples.mk
include examples/pyapp/pyapp.mk
include examples/pylife/pylife.mk
include tool/decode/lib/decodelib.mk
include tool/decode/decode.mk
include tool/lambda/lib/lib.mk
include tool/lambda/lambda.mk
include tool/plinko/lib/lib.mk
include tool/plinko/plinko.mk
include test/tool/plinko/test.mk
include tool/hash/hash.mk
include tool/net/net.mk
include tool/viz/viz.mk
include tool/tool.mk
include net/turfwar/turfwar.mk
include test/libc/tinymath/test.mk
include test/libc/intrin/test.mk
include test/libc/mem/test.mk
include test/libc/nexgen32e/test.mk
include test/libc/runtime/test.mk
include test/libc/thread/test.mk
include test/libc/sock/test.mk
include test/libc/str/test.mk
include test/libc/log/test.mk
include test/libc/str/test.mk
include test/libc/calls/test.mk
include test/libc/x/test.mk
include test/libc/xed/test.mk
include test/libc/fmt/test.mk
include test/libc/dns/test.mk
include test/libc/time/test.mk
include test/libc/stdio/test.mk
include test/libc/zipos/test.mk
include test/libc/release/test.mk
include test/libc/test.mk
include test/net/http/test.mk
include test/net/https/test.mk
include test/net/finger/test.mk
include test/net/test.mk
include test/tool/build/lib/test.mk
include test/tool/build/test.mk
include test/tool/viz/lib/test.mk
include test/tool/viz/test.mk
include test/tool/net/test.mk
include test/tool/test.mk
include test/dsp/core/test.mk
include test/dsp/scale/test.mk
include test/dsp/tty/test.mk
include test/dsp/test.mk
include examples/package/lib/build.mk
include examples/package/build.mk
include build/online.mk #─┘
include libc/nexgen32e/BUILD.mk #─┐
include libc/sysv/BUILD.mk # ├──SYSTEM SUPPORT
include libc/nt/BUILD.mk # │ You can do math
include libc/intrin/BUILD.mk # │ You can use the stack
include third_party/compiler_rt/BUILD.mk # │ You can manipulate arrays
include libc/tinymath/BUILD.mk # │ You can issue raw system calls
include libc/str/BUILD.mk # │
include third_party/xed/BUILD.mk # │
include third_party/puff/BUILD.mk # │
include libc/elf/BUILD.mk # │
include ape/BUILD.mk # │
include libc/fmt/BUILD.mk # │
include libc/vga/BUILD.mk # │
include libc/irq/BUILD.mk #─┘
include libc/calls/BUILD.mk #─┐
include libc/irq/BUILD.mk # ├──SYSTEMS RUNTIME
include third_party/nsync/BUILD.mk # │ You can issue system calls
include libc/runtime/BUILD.mk # │
include libc/crt/BUILD.mk # │
include third_party/dlmalloc/BUILD.mk #─┘
include libc/mem/BUILD.mk #─┐
include third_party/gdtoa/BUILD.mk # ├──DYNAMIC RUNTIME
include third_party/nsync/mem/BUILD.mk # │ You can now use stdio
include libc/proc/BUILD.mk # │ You can now use threads
include libc/dlopen/BUILD.mk # │ You can now use processes
include libc/thread/BUILD.mk # │ You can finally call malloc()
include third_party/zlib/BUILD.mk # │
include libc/stdio/BUILD.mk # │
include tool/hello/BUILD.mk # │
include third_party/tz/BUILD.mk # │
include net/BUILD.mk # │
include third_party/vqsort/BUILD.mk # │
include libc/log/BUILD.mk # │
include third_party/getopt/BUILD.mk # │
include third_party/bzip2/BUILD.mk # │
include dsp/core/BUILD.mk # │
include third_party/zlib/gz/BUILD.mk # │
include third_party/intel/BUILD.mk # │
include third_party/aarch64/BUILD.mk # │
include libc/BUILD.mk #─┘
include libc/sock/BUILD.mk #─┐
include net/http/BUILD.mk # ├──ONLINE RUNTIME
include third_party/musl/BUILD.mk # │ You can communicate with the network
include libc/x/BUILD.mk # │
include dsp/scale/BUILD.mk # │
include dsp/mpeg/BUILD.mk # │
include dsp/tty/BUILD.mk # │
include dsp/BUILD.mk # │
include third_party/stb/BUILD.mk # │
include third_party/mbedtls/BUILD.mk # │
include third_party/ncurses/BUILD.mk # │
include third_party/readline/BUILD.mk # │
include third_party/libunwind/BUILD.mk # |
include third_party/libcxxabi/BUILD.mk # |
include third_party/double-conversion/BUILD.mk # │
include ctl/BUILD.mk # │
include third_party/libcxx/BUILD.mk # │
include third_party/openmp/BUILD.mk # │
include third_party/pcre/BUILD.mk # │
include third_party/less/BUILD.mk # │
include net/https/BUILD.mk # │
include third_party/regex/BUILD.mk #─┘
include third_party/tidy/BUILD.mk
include third_party/BUILD.mk
include third_party/nsync/testing/BUILD.mk
include libc/testlib/BUILD.mk
include tool/viz/lib/BUILD.mk
include tool/args/BUILD.mk
include test/math/BUILD.mk
include test/posix/BUILD.mk
include test/ctl/BUILD.mk
include test/libcxx/BUILD.mk
include test/tool/args/BUILD.mk
include third_party/linenoise/BUILD.mk
include third_party/maxmind/BUILD.mk
include net/finger/BUILD.mk
include third_party/double-conversion/test/BUILD.mk
include third_party/lua/BUILD.mk
include third_party/tree/BUILD.mk
include third_party/zstd/BUILD.mk
include third_party/tr/BUILD.mk
include third_party/sed/BUILD.mk
include third_party/awk/BUILD.mk
include third_party/hiredis/BUILD.mk
include third_party/make/BUILD.mk
include third_party/ctags/BUILD.mk
include third_party/finger/BUILD.mk
include third_party/argon2/BUILD.mk
include third_party/smallz4/BUILD.mk
include third_party/sqlite3/BUILD.mk
include third_party/mbedtls/test/BUILD.mk
include third_party/lz4cli/BUILD.mk
include third_party/zip/BUILD.mk
include third_party/xxhash/BUILD.mk
include third_party/unzip/BUILD.mk
include tool/build/lib/BUILD.mk
include third_party/chibicc/BUILD.mk
include third_party/chibicc/test/BUILD.mk
include third_party/python/BUILD.mk
include tool/build/BUILD.mk
include tool/curl/BUILD.mk
include third_party/qemu/BUILD.mk
include third_party/libcxxabi/test/BUILD.mk
include examples/BUILD.mk
include examples/pyapp/BUILD.mk
include examples/pylife/BUILD.mk
include tool/decode/lib/BUILD.mk
include tool/decode/BUILD.mk
include tool/lambda/lib/BUILD.mk
include tool/lambda/BUILD.mk
include tool/plinko/lib/BUILD.mk
include tool/plinko/BUILD.mk
include test/tool/plinko/BUILD.mk
include tool/net/BUILD.mk
include tool/viz/BUILD.mk
include tool/BUILD.mk
include net/turfwar/BUILD.mk
include test/libc/tinymath/BUILD.mk
include test/libc/intrin/BUILD.mk
include test/libc/mem/BUILD.mk
include test/libc/nexgen32e/BUILD.mk
include test/libc/runtime/BUILD.mk
include test/libc/thread/BUILD.mk
include test/libc/sock/BUILD.mk
include test/libc/str/BUILD.mk
include test/libc/log/BUILD.mk
include test/libc/str/BUILD.mk
include test/libc/calls/BUILD.mk
include test/libc/x/BUILD.mk
include test/libc/xed/BUILD.mk
include test/libc/fmt/BUILD.mk
include test/libc/time/BUILD.mk
include test/libc/proc/BUILD.mk
include test/libc/stdio/BUILD.mk
include test/libc/BUILD.mk
include test/net/http/BUILD.mk
include test/net/https/BUILD.mk
include test/net/finger/BUILD.mk
include test/net/BUILD.mk
include test/tool/build/lib/BUILD.mk
include test/tool/build/BUILD.mk
include test/tool/viz/lib/BUILD.mk
include test/tool/viz/BUILD.mk
include test/tool/net/BUILD.mk
include test/tool/BUILD.mk
include test/dsp/core/BUILD.mk
include test/dsp/scale/BUILD.mk
include test/dsp/tty/BUILD.mk
include test/dsp/BUILD.mk
include examples/package/lib/BUILD.mk
include examples/package/BUILD.mk
#-φ-examples/package/new.sh
include test/test.mk
include test/BUILD.mk
OBJS = $(foreach x,$(PKGS),$($(x)_OBJS))
SRCS := $(foreach x,$(PKGS),$($(x)_SRCS))
@@ -267,14 +403,14 @@ tags: TAGS HTAGS
o/$(MODE)/.x:
@$(COMPILE) -AMKDIR -tT$@ $(MKDIR) $(@D)
o/$(MODE)/srcs.txt: o/$(MODE)/.x $(MAKEFILES) $(call uniq,$(foreach x,$(SRCS),$(dir $(x))))
o/$(MODE)/srcs.txt: o/$(MODE)/.x $(MAKEFILES) $(call uniq,$(foreach x,$(SRCS),$(dir $(x)))) $(SRCS)
$(file >$@,$(SRCS))
o/$(MODE)/hdrs.txt: o/$(MODE)/.x $(MAKEFILES) $(call uniq,$(foreach x,$(HDRS) $(INCS),$(dir $(x))))
o/$(MODE)/hdrs.txt: o/$(MODE)/.x $(MAKEFILES) $(call uniq,$(foreach x,$(HDRS) $(INCS),$(dir $(x)))) $(HDRS) $(INCS)
$(file >$@,$(HDRS) $(INCS))
o/$(MODE)/incs.txt: o/$(MODE)/.x $(MAKEFILES) $(call uniq,$(foreach x,$(INCS) $(INCS),$(dir $(x))))
o/$(MODE)/incs.txt: o/$(MODE)/.x $(MAKEFILES) $(call uniq,$(foreach x,$(INCS) $(INCS),$(dir $(x)))) $(INCS) $(INCS)
$(file >$@,$(INCS))
o/$(MODE)/depend: o/$(MODE)/.x o/$(MODE)/srcs.txt o/$(MODE)/hdrs.txt o/$(MODE)/incs.txt $(SRCS) $(HDRS) $(INCS)
@$(COMPILE) -AMKDEPS -L320 $(MKDEPS) -o $@ -r o/$(MODE)/ @o/$(MODE)/srcs.txt @o/$(MODE)/hdrs.txt @o/$(MODE)/incs.txt
$(COMPILE) -AMKDEPS -L320 $(MKDEPS) -o $@ -s -r o/$(MODE)/ @o/$(MODE)/srcs.txt @o/$(MODE)/hdrs.txt @o/$(MODE)/incs.txt
o/$(MODE)/srcs-old.txt: o/$(MODE)/.x $(MAKEFILES) $(call uniq,$(foreach x,$(SRCS),$(dir $(x))))
$(file >$@) $(foreach x,$(SRCS),$(file >>$@,$(x)))
@@ -282,45 +418,52 @@ o/$(MODE)/hdrs-old.txt: o/$(MODE)/.x $(MAKEFILES) $(call uniq,$(foreach x,$(HDRS
$(file >$@) $(foreach x,$(HDRS) $(INCS),$(file >>$@,$(x)))
TAGS: private .UNSANDBOXED = 1
TAGS: o/$(MODE)/srcs-old.txt $(SRCS) o/$(MODE)/third_party/ctags/ctags.com
TAGS: o/$(MODE)/srcs-old.txt $(SRCS) #o/$(MODE)/third_party/ctags/ctags
@$(RM) $@
@o/$(MODE)/third_party/ctags/ctags.com $(TAGSFLAGS) -L $< -o $@
@o/$(MODE)/third_party/ctags/ctags $(TAGSFLAGS) -L $< -o $@
HTAGS: private .UNSANDBOXED = 1
HTAGS: o/$(MODE)/hdrs-old.txt $(HDRS) o/$(MODE)/third_party/ctags/ctags.com
HTAGS: o/$(MODE)/hdrs-old.txt $(filter-out third_party/libcxx/%,$(HDRS)) #o/$(MODE)/third_party/ctags/ctags
@$(RM) $@
@build/htags o/$(MODE)/third_party/ctags/ctags.com -L $< -o $@
@build/htags o/$(MODE)/third_party/ctags/ctags -L $< -o $@
loc: private .UNSANDBOXED = 1
loc: o/$(MODE)/tool/build/summy.com
loc: o/$(MODE)/tool/build/summy
find -name \*.h -or -name \*.c -or -name \*.S | \
$(XARGS) wc -l | grep total | awk '{print $$1}' | $<
# PLEASE: MAINTAIN TOPOLOGICAL ORDER
# FROM HIGHEST LEVEL TO LOWEST LEVEL
COSMOPOLITAN_OBJECTS = \
CTL \
THIRD_PARTY_DOUBLECONVERSION \
THIRD_PARTY_OPENMP \
TOOL_ARGS \
NET_HTTP \
LIBC_DNS \
LIBC_SOCK \
LIBC_NT_WS2_32 \
LIBC_NT_IPHLPAPI \
LIBC_NT_MSWSOCK \
LIBC_X \
THIRD_PARTY_GETOPT \
LIBC_LOG \
LIBC_TIME \
LIBC_ZIPOS \
THIRD_PARTY_TZ \
THIRD_PARTY_MUSL \
THIRD_PARTY_ZLIB_GZ \
THIRD_PARTY_LIBCXXABI \
THIRD_PARTY_LIBUNWIND \
LIBC_STDIO \
THIRD_PARTY_GDTOA \
THIRD_PARTY_REGEX \
LIBC_THREAD \
LIBC_PROC \
THIRD_PARTY_NSYNC_MEM \
LIBC_MEM \
THIRD_PARTY_DLMALLOC \
LIBC_DLOPEN \
LIBC_RUNTIME \
THIRD_PARTY_NSYNC \
LIBC_ELF \
LIBC_IRQ \
LIBC_CALLS \
LIBC_SYSV_CALLS \
LIBC_VGA \
@@ -329,12 +472,12 @@ COSMOPOLITAN_OBJECTS = \
LIBC_NT_PDH \
LIBC_NT_GDI32 \
LIBC_NT_COMDLG32 \
LIBC_NT_URL \
LIBC_NT_USER32 \
LIBC_NT_NTDLL \
LIBC_NT_ADVAPI32 \
LIBC_NT_SYNCHRONIZATION \
LIBC_FMT \
THIRD_PARTY_ZLIB \
THIRD_PARTY_PUFF \
THIRD_PARTY_COMPILER_RT \
LIBC_TINYMATH \
@@ -342,16 +485,17 @@ COSMOPOLITAN_OBJECTS = \
LIBC_STR \
LIBC_SYSV \
LIBC_INTRIN \
LIBC_NT_BCRYPTPRIMITIVES \
LIBC_NT_KERNEL32 \
LIBC_NEXGEN32E
COSMOPOLITAN_HEADERS = \
COSMOPOLITAN_H_PKGS = \
APE \
LIBC \
LIBC_CALLS \
LIBC_DNS \
LIBC_ELF \
LIBC_FMT \
LIBC_DLOPEN \
LIBC_INTRIN \
LIBC_LOG \
LIBC_MEM \
@@ -360,44 +504,60 @@ COSMOPOLITAN_HEADERS = \
LIBC_RUNTIME \
LIBC_SOCK \
LIBC_STDIO \
LIBC_PROC \
THIRD_PARTY_NSYNC \
THIRD_PARTY_XED \
LIBC_STR \
LIBC_SYSV \
LIBC_THREAD \
LIBC_TIME \
LIBC_TINYMATH \
LIBC_X \
LIBC_ZIPOS \
LIBC_VGA \
NET_HTTP \
TOOL_ARGS \
THIRD_PARTY_DLMALLOC \
THIRD_PARTY_GDTOA \
THIRD_PARTY_GETOPT \
THIRD_PARTY_MUSL \
THIRD_PARTY_ZLIB \
THIRD_PARTY_ZLIB_GZ \
THIRD_PARTY_REGEX
COSMOCC_PKGS = \
$(COSMOPOLITAN_H_PKGS) \
CTL \
THIRD_PARTY_AARCH64 \
THIRD_PARTY_LIBCXX \
THIRD_PARTY_LIBCXXABI \
THIRD_PARTY_LIBUNWIND \
THIRD_PARTY_OPENMP \
THIRD_PARTY_INTEL
o/$(MODE)/cosmopolitan.a: \
$(foreach x,$(COSMOPOLITAN_OBJECTS),$($(x)_A_OBJS))
o/cosmopolitan.h: \
o/$(MODE)/tool/build/rollup.com \
libc/integral/normalize.inc \
$(foreach x,$(COSMOPOLITAN_HEADERS),$($(x)_HDRS)) \
$(foreach x,$(COSMOPOLITAN_HEADERS),$($(x)_INCS))
$(file >$(TMPDIR)/$(subst /,_,$@),libc/integral/normalize.inc $(foreach x,$(COSMOPOLITAN_HEADERS),$($(x)_HDRS)))
@$(COMPILE) -AROLLUP -T$@ o/$(MODE)/tool/build/rollup.com @$(TMPDIR)/$(subst /,_,$@) >$@
COSMOCC_HDRS = \
$(wildcard libc/integral/*) \
$(foreach x,$(COSMOCC_PKGS),$($(x)_HDRS)) \
$(foreach x,$(COSMOCC_PKGS),$($(x)_INCS))
o/cosmocc.h.txt: Makefile
$(file >$@, $(call uniq,$(COSMOCC_HDRS)))
COSMOPOLITAN_H_ROOT_HDRS = \
libc/integral/normalize.inc \
$(foreach x,$(COSMOPOLITAN_H_PKGS),$($(x)_HDRS))
o/cosmopolitan.html: private .UNSANDBOXED = 1
o/cosmopolitan.html: \
o/$(MODE)/third_party/chibicc/chibicc.com.dbg \
o/$(MODE)/third_party/chibicc/chibicc.dbg \
$(filter-out %.s,$(foreach x,$(COSMOPOLITAN_OBJECTS),$($(x)_SRCS))) \
$(SRCS) \
$(filter-out %.cpp,$(filter-out %.cc,$(SRCS))) \
$(HDRS)
$(file >$(TMPDIR)/$(subst /,_,$@),$(filter-out %.s,$(foreach x,$(COSMOPOLITAN_OBJECTS),$($(x)_SRCS))))
o/$(MODE)/third_party/chibicc/chibicc.com.dbg -J \
$(file >$(TMPDIR)/$(subst /,_,$@),$(filter-out %.cpp,$(filter-out %.cc,$(filter-out %.s,$(foreach x,$(COSMOPOLITAN_OBJECTS),$($(x)_SRCS))))))
o/$(MODE)/third_party/chibicc/chibicc.dbg -J \
-fno-common -include libc/integral/normalize.inc -o $@ \
@$(TMPDIR)/$(subst /,_,$@)
-DCOSMO @$(TMPDIR)/$(subst /,_,$@)
$(SRCS): \
libc/integral/normalize.inc \
@@ -407,14 +567,49 @@ $(SRCS): \
libc/integral/lp64arg.inc \
libc/integral/lp64.inc
ifeq ($(ARCH), x86_64)
TOOLCHAIN_ARTIFACTS = \
o/$(MODE)/ape/ape.lds \
o/$(MODE)/libc/crt/crt.o \
o/$(MODE)/ape/ape.elf \
o/$(MODE)/ape/ape.o \
o/$(MODE)/ape/ape-copy-self.o \
o/$(MODE)/ape/ape-no-modify-self.o \
o/$(MODE)/cosmopolitan.a \
o/$(MODE)/third_party/libcxx/libcxx.a \
o/$(MODE)/tool/build/march-native \
o/$(MODE)/tool/build/ar \
o/$(MODE)/tool/build/mktemper \
o/$(MODE)/tool/build/fixupobj \
o/$(MODE)/tool/build/zipcopy \
o/$(MODE)/tool/build/apelink \
o/$(MODE)/tool/build/pecheck
else
TOOLCHAIN_ARTIFACTS = \
o/$(MODE)/ape/ape.elf \
o/$(MODE)/ape/aarch64.lds \
o/$(MODE)/libc/crt/crt.o \
o/$(MODE)/cosmopolitan.a \
o/$(MODE)/third_party/libcxx/libcxx.a \
o/$(MODE)/tool/build/march-native \
o/$(MODE)/tool/build/fixupobj \
o/$(MODE)/tool/build/zipcopy
endif
.PHONY: toolchain
toolchain: o/cosmopolitan.h \
o/$(MODE)/ape/public/ape.lds \
o/$(MODE)/libc/crt/crt.o \
o/$(MODE)/ape/ape.o \
o/$(MODE)/ape/ape-copy-self.o \
o/$(MODE)/ape/ape-no-modify-self.o \
o/$(MODE)/cosmopolitan.a
toolchain: $(TOOLCHAIN_ARTIFACTS)
.PHONY: clean_toolchain
clean_toolchain:
$(RM) $(TOOLCHAIN_ARTIFACTS)
aarch64: private .INTERNET = true
aarch64: private .UNSANDBOXED = true
aarch64:
$(MAKE) m=aarch64
clean:
$(RM) -r o
# UNSPECIFIED PREREQUISITES TUTORIAL
#
+152 -160
View File
@@ -12,206 +12,149 @@ possible performance and the tiniest footprint imaginable.
## Background
For an introduction to this project, please read the [αcτµαlly pδrταblε
εxεcµταblε](https://justine.lol/ape.html) blog post and [cosmopolitan
For an introduction to this project, please read the [actually portable
executable](https://justine.lol/ape.html) blog post and [cosmopolitan
libc](https://justine.lol/cosmopolitan/index.html) website. We also have
[API documentation](https://justine.lol/cosmopolitan/documentation.html).
[API
documentation](https://justine.lol/cosmopolitan/documentation.html).
## Getting Started
If you're doing your development work on Linux or BSD then you need just
five files to get started. Here's what you do on Linux:
You can start by obtaining a release of our `cosmocc` compiler from
<https://cosmo.zip/pub/cosmocc/>.
```sh
wget https://justine.lol/cosmopolitan/cosmopolitan-amalgamation-2.2.zip
unzip cosmopolitan-amalgamation-2.2.zip
printf 'main() { printf("hello world\\n"); }\n' >hello.c
gcc -g -Os -static -nostdlib -nostdinc -fno-pie -no-pie -mno-red-zone \
-fno-omit-frame-pointer -pg -mnop-mcount -mno-tls-direct-seg-refs -gdwarf-4 \
-o hello.com.dbg hello.c -fuse-ld=bfd -Wl,-T,ape.lds -Wl,--gc-sections \
-include cosmopolitan.h crt.o ape-no-modify-self.o cosmopolitan.a
objcopy -S -O binary hello.com.dbg hello.com
mkdir -p cosmocc
cd cosmocc
wget https://cosmo.zip/pub/cosmocc/cosmocc.zip
unzip cosmocc.zip
```
You now have a portable program.
Here's an example program we can write:
```c
// hello.c
#include <stdio.h>
int main() {
printf("hello world\n");
}
```
It can be compiled as follows:
```sh
./hello.com
bash -c './hello.com' # zsh/fish workaround (we patched them in 2021)
cosmocc -o hello hello.c
./hello
```
If `./hello.com` executed on Linux throws an error about not finding an
interpreter, it should be fixed by running the following command (although
note that it may not survive a system restart):
The Cosmopolitan Libc runtime links some heavyweight troubleshooting
features by default, which are very useful for developers and admins.
Here's how you can log system calls:
```sh
sudo sh -c "echo ':APE:M::MZqFpD::/bin/sh:' >/proc/sys/fs/binfmt_misc/register"
./hello --strace
```
If the same command produces puzzling errors on WSL or WINE when using
Redbean 2.x, they may be fixed by disabling binfmt_misc:
Here's how you can get a much more verbose log of function calls:
```sh
sudo sh -c 'echo -1 >/proc/sys/fs/binfmt_misc/status'
./hello --ftrace
```
Since we used the `ape-no-modify-self.o` bootloader (rather than
`ape.o`) your executable will not modify itself when it's run. What
it'll instead do, is extract a 4kb program (the [APE loader](https://justine.lol/apeloader/))
to `${TMPDIR:-${HOME:-.}}` that maps your program into memory without
needing to copy it. The APE loader must be in an executable location
(e.g. not stored on a `noexec` mount) for it to run. See below for
alternatives:
It's possible to install the APE loader systemwide as follows.
You can use the Cosmopolitan's toolchain to build conventional open
source projects which use autotools. This strategy normally works:
```sh
# System-Wide APE Install
# for Linux, Darwin, and BSDs
# 1. Copies APE Loader to /usr/bin/ape
# 2. Registers w/ binfmt_misc too if Linux
ape/apeinstall.sh
# System-Wide APE Uninstall
# for Linux, Darwin, and BSDs
ape/apeuninstall.sh
export CC=x86_64-unknown-cosmo-cc
export CXX=x86_64-unknown-cosmo-c++
./configure --prefix=/opt/cosmos/x86_64
make -j
make install
```
It's also possible to convert APE binaries into the system-local format
by using the `--assimilate` flag. Please note that if binfmt_misc is in
play, you'll need to unregister it temporarily before doing this, since
the assimilate feature is part of the shell script header.
## Cosmopolitan Source Builds
```sh
$ file hello.com
hello.com: DOS/MBR boot sector
./hello.com --assimilate
$ file hello.com
hello.com: ELF 64-bit LSB executable
```
Cosmopolitan can be compiled from source on any of our supported
platforms. The Makefile will download cosmocc automatically.
Now that you're up and running with Cosmopolitan Libc and APE, here's
some of the most important troubleshooting tools APE offers that you
should know, in case you encounter any issues:
```sh
./hello.com --strace # log system calls to stderr
./hello.com --ftrace # log function calls to stderr
```
Do you love tiny binaries? If so, you may not be happy with Cosmo adding
heavyweight features like tracing to your binaries by default. In that
case, you may want to consider using our build system:
```sh
make m=tiny
```
Which will cause programs such as `hello.com` and `life.com` to shrink
from 60kb in size to about 16kb. There's also a prebuilt amalgamation
online <https://justine.lol/cosmopolitan/cosmopolitan-tiny.zip> hosted
on our download page <https://justine.lol/cosmopolitan/download.html>.
### MacOS
If you're developing on MacOS you can install the GNU compiler
collection for x86_64-elf via homebrew:
```sh
brew install x86_64-elf-gcc
```
Then in the above scripts just replace `gcc` and `objcopy` with
`x86_64-elf-gcc` and `x86_64-elf-objcopy` to compile your APE binary.
### Windows
If you're developing on Windows then you need to download an
x86_64-pc-linux-gnu toolchain beforehand. See the [Compiling on
Windows](https://justine.lol/cosmopolitan/windows-compiling.html)
tutorial. It's needed because the ELF object format is what makes
universal binaries possible.
Cosmopolitan officially only builds on Linux. However, one highly
experimental (and currently broken) thing you could try, is building the
entire cosmo repository from source using the cross9 toolchain.
```
mkdir -p o/third_party
rm -rf o/third_party/gcc
wget https://justine.lol/linux-compiler-on-windows/cross9.zip
unzip cross9.zip
mv cross9 o/third_party/gcc
build/bootstrap/make.com
```
## Source Builds
Cosmopolitan can be compiled from source on any Linux distro. First, you
need to download or clone the repository.
```sh
wget https://justine.lol/cosmopolitan/cosmopolitan.tar.gz
tar xf cosmopolitan.tar.gz # see releases page
cd cosmopolitan
```
This will build the entire repository and run all the tests:
```sh
build/bootstrap/make.com
o//examples/hello.com
find o -name \*.com | xargs ls -rShal | less
```
If you get an error running make.com then it's probably because you have
WINE installed to `binfmt_misc`. You can fix that by installing the the
APE loader as an interpreter. It'll improve build performance too!
It's recommended that you install a systemwide APE Loader. This command
requires `sudo` access to copy the `ape` command to a system folder and
register with binfmt_misc on Linux, for even more performance.
```sh
ape/apeinstall.sh
```
You can now build the mono repo with any modern version of GNU Make. To
make life easier, we've included one in the cosmocc toolchain, which is
guaranteed to be compatible and furthermore includes our extensions for
doing build system sandboxing.
```sh
build/bootstrap/make -j8
o//examples/hello
```
Since the Cosmopolitan repository is very large, you might only want to
build a particular thing. Cosmopolitan's build config does a good job at
having minimal deterministic builds. For example, if you wanted to build
only hello.com then you could do that as follows:
build one particular thing. Here's an example of a target that can be
compiled relatively quickly, which is a simple POSIX test that only
depends on core LIBC packages.
```sh
build/bootstrap/make.com o//examples/hello.com
rm -rf o//libc o//test
build/bootstrap/make o//test/posix/signal_test
o//test/posix/signal_test
```
Sometimes it's desirable to build a subset of targets, without having to
list out each individual one. You can do that by asking make to build a
directory name. For example, if you wanted to build only the targets and
subtargets of the chibicc package including its tests, you would say:
list out each individual one. For example if you wanted to build and run
all the unit tests in the `TEST_POSIX` package, you could say:
```sh
build/bootstrap/make.com o//third_party/chibicc
o//third_party/chibicc/chibicc.com --help
build/bootstrap/make o//test/posix
```
Cosmopolitan provides a variety of build modes. For example, if you want
really tiny binaries (as small as 12kb in size) then you'd say:
```sh
build/bootstrap/make.com m=tiny
build/bootstrap/make m=tiny
```
Here's some other build modes you can try:
You can furthermore cut out the bloat of other operating systems, and
have Cosmopolitan become much more similar to Musl Libc.
```sh
build/bootstrap/make.com m=dbg # asan + ubsan + debug
build/bootstrap/make.com m=asan # production memory safety
build/bootstrap/make.com m=opt # -march=native optimizations
build/bootstrap/make.com m=rel # traditional release binaries
build/bootstrap/make.com m=optlinux # optimal linux-only performance
build/bootstrap/make.com m=fastbuild # build 28% faster w/o debugging
build/bootstrap/make.com m=tinylinux # tiniest linux-only 4kb binaries
build/bootstrap/make m=tinylinux
```
For further details, see [//build/config.mk](build/config.mk).
## Debugging
To print a log of system calls to stderr:
```sh
cosmocc -o hello hello.c
./hello --strace
```
To print a log of function calls to stderr:
```sh
cosmocc -o hello hello.c
./hello --ftrace
```
Both strace and ftrace use the unbreakable kprintf() facility, which is
able to be sent to a file by setting an environment variable.
```sh
export KPRINTF_LOG=log
./hello --strace
```
## GDB
Here's the recommended `~/.gdbinit` config:
@@ -236,11 +179,60 @@ end
src
```
You normally run the `.com.dbg` file under gdb. If you need to debug the
`.com` file itself, then you can load the debug symbols independently as
You normally run the `.dbg` file under gdb. If you need to debug the
`` file itself, then you can load the debug symbols independently as
```sh
gdb foo -ex 'add-symbol-file foo.dbg 0x401000'
```
gdb foo.com -ex 'add-symbol-file foo.com.dbg 0x401000'
## Platform Notes
### Shells
If you use zsh and have trouble running APE programs try `sh -c ./prog`
or simply upgrade to zsh 5.9+ (since we patched it two years ago). The
same is the case for Python `subprocess`, old versions of fish, etc.
### Linux
Some Linux systems are configured to launch MZ executables under WINE.
Other distros configure their stock installs so that APE programs will
print "run-detectors: unable to find an interpreter". For example:
```sh
jart@ubuntu:~$ wget https://cosmo.zip/pub/cosmos/bin/dash
jart@ubuntu:~$ chmod +x dash
jart@ubuntu:~$ ./dash
run-detectors: unable to find an interpreter for ./dash
```
You can fix that by registering APE with `binfmt_misc`:
```sh
sudo wget -O /usr/bin/ape https://cosmo.zip/pub/cosmos/bin/ape-$(uname -m).elf
sudo chmod +x /usr/bin/ape
sudo sh -c "echo ':APE:M::MZqFpD::/usr/bin/ape:' >/proc/sys/fs/binfmt_misc/register"
sudo sh -c "echo ':APE-jart:M::jartsr::/usr/bin/ape:' >/proc/sys/fs/binfmt_misc/register"
```
You should be good now. APE will not only work, it'll launch executables
400µs faster now too. However if things still didn't work out, it's also
possible to disable `binfmt_misc` as follows:
```sh
sudo sh -c 'echo -1 > /proc/sys/fs/binfmt_misc/cli' # remove Ubuntu's MZ interpreter
sudo sh -c 'echo -1 > /proc/sys/fs/binfmt_misc/status' # remove ALL binfmt_misc entries
```
### WSL
It's normally unsafe to use APE in a WSL environment, because it tries
to run MZ executables as WIN32 binaries within the WSL environment. In
order to make it safe to use Cosmopolitan software on WSL, run this:
```sh
sudo sh -c "echo -1 > /proc/sys/fs/binfmt_misc/WSLInterop"
```
## Discord Chatroom
@@ -250,16 +242,16 @@ server. You're welcome to join us! <https://discord.gg/FwAVVu7eJ4>
## Support Vector
| Platform | Min Version | Circa |
| :--- | ---: | ---: |
| AMD | K8 Venus | 2005 |
| Intel | Core | 2006 |
| Linux | 2.6.18 | 2007 |
| Windows | 8 [1] | 2012 |
| Mac OS X | 15.6 | 2018 |
| OpenBSD | 6.4 | 2018 |
| FreeBSD | 13 | 2020 |
| NetBSD | 9.2 | 2021 |
| Platform | Min Version | Circa |
| :--- | ---: | ---: |
| AMD | K8 | 2003 |
| Intel | Core | 2006 |
| Linux | 2.6.18 | 2007 |
| Windows | 8 [1] | 2012 |
| Darwin (macOS) | 23.1.0+ | 2023 |
| OpenBSD | 7 | 2021 |
| FreeBSD | 13 | 2020 |
| NetBSD | 9.2 | 2021 |
[1] See our [vista branch](https://github.com/jart/cosmopolitan/tree/vista)
for a community supported version of Cosmopolitan that works on Windows
+289
View File
@@ -0,0 +1,289 @@
#-*-mode:makefile-gmake;indent-tabs-mode:t;tab-width:8;coding:utf-8-*-┐
#── vi: set noet ft=make ts=8 sw=8 fenc=utf-8 :vi ────────────────────┘
#
# OVERVIEW
#
# αcτµαlly pδrταblε εxεcµταblε
#
# DESCRIPTION
#
# This file defines the libraries, runtimes, and build rules needed to
# create executables from your Linux workstation that'll run anywhere.
# Loading this package will make certain systemic modifications to the
# build like turning off the System V "Red Zone" optimization, because
# αcτµαlly pδrταblε εxεcµταblεs need to be able to run in kernelspace.
PKGS += APE
APE_FILES := $(wildcard ape/*.*)
APE_HDRS = $(filter %.h,$(APE_FILES))
APE_INCS = $(filter %.inc,$(APE_FILES))
ifeq ($(ARCH), aarch64)
APE = o/$(MODE)/ape/aarch64.lds
APELINK = \
$(COMPILE) \
-ALINK.ape \
$(LINK) \
$(LINKARGS) \
$(OUTPUT_OPTION) && \
$(COMPILE) \
-AFIXUP.ape \
-wT$@ \
$(FIXUPOBJ) \
$@
APE_SRCS = ape/ape.S ape/start.S ape/launch.S ape/systemcall.S
APE_OBJS = o/$(MODE)/ape/ape.o
APE_NO_MODIFY_SELF = $(APE)
APE_COPY_SELF = $(APE)
.PHONY: o/$(MODE)/ape
o/$(MODE)/ape: $(APE)
o/$(MODE)/ape/aarch64.lds: \
ape/aarch64.lds \
libc/zip.internal.h \
libc/thread/tls.h \
libc/calls/struct/timespec.h \
libc/macros.internal.h \
libc/str/str.h
APE_LOADER_LDFLAGS = \
-pie \
-static \
-nostdlib \
--no-dynamic-linker \
-z norelro \
-z common-page-size=0x4000 \
-z max-page-size=0x4000
APE_LOADER_FLAGS = \
-DNDEBUG \
-iquote. \
-Wall \
-Wextra \
-fpie \
-Os \
-ffreestanding \
-mgeneral-regs-only \
-fno-asynchronous-unwind-tables \
-fno-stack-protector \
-fno-ident \
-fno-gnu-unique \
-c \
$(OUTPUT_OPTION) \
$<
o/$(MODE)/ape/ape.elf: o/$(MODE)/ape/ape.elf.dbg
@$(COMPILE) -AOBJCOPY -T$@ $(OBJCOPY) -g $< $@
@$(COMPILE) -AFIXUPOBJ -wT$@ $(FIXUPOBJ) $@
o/$(MODE)/ape/ape.elf.dbg: \
o/$(MODE)/ape/start.o \
o/$(MODE)/ape/loader.o \
o/$(MODE)/ape/launch.o \
o/$(MODE)/ape/systemcall.o
@$(COMPILE) -ALINK.elf $(LD) $(APE_LOADER_LDFLAGS) -o $@ $(patsubst %.lds,-T %.lds,$^)
o/$(MODE)/ape/loader.o: ape/loader.c ape/ape.h
@$(COMPILE) -AOBJECTIFY.c $(CC) -DSUPPORT_VECTOR=33 -g $(APE_LOADER_FLAGS)
o/$(MODE)/ape/start.o: ape/start.S
@$(COMPILE) -AOBJECTIFY.S $(OBJECTIFY.S) $(OUTPUT_OPTION) -c $<
o/$(MODE)/ape/launch.o: ape/launch.S
@$(COMPILE) -AOBJECTIFY.S $(OBJECTIFY.S) $(OUTPUT_OPTION) -c $<
o/$(MODE)/ape/systemcall.o: ape/systemcall.S
@$(COMPILE) -AOBJECTIFY.S $(OBJECTIFY.S) $(OUTPUT_OPTION) -c $<
.PHONY: o/$(MODE)/ape
o/$(MODE)/ape: o/$(MODE)/ape/ape.elf
else
APE = o/$(MODE)/ape/ape.o \
o/$(MODE)/ape/ape.lds
APELINK = \
$(COMPILE) \
-ALINK.ape \
$(LINK) \
$(LINKARGS) \
$(OUTPUT_OPTION) && \
$(COMPILE) \
-AFIXUP.ape \
-wT$@ \
$(FIXUPOBJ) \
$@
APE_NO_MODIFY_SELF = \
o/$(MODE)/ape/ape.lds \
o/$(MODE)/ape/ape-no-modify-self.o
APE_COPY_SELF = \
o/$(MODE)/ape/ape.lds \
o/$(MODE)/ape/ape-copy-self.o
APE_LOADER_FLAGS = \
-DNDEBUG \
-iquote. \
-Wall \
-Wextra \
-Werror \
-pedantic-errors \
-fpie \
-Os \
-ffreestanding \
-mgeneral-regs-only \
-fno-stack-protector \
-fno-ident \
-fno-gnu-unique \
-c \
$(OUTPUT_OPTION) \
$<
APE_SRCS_C = ape/loader.c
APE_SRCS_S = $(filter %.S,$(APE_FILES))
APE_SRCS = $(APE_SRCS_C) $(APE_SRCS_S)
APE_OBJS = $(APE_SRCS_S:%.S=o/$(MODE)/%.o)
APE_CHECKS = $(APE_HDRS:%=o/%.ok)
o/ape/idata.inc: \
ape/idata.internal.h \
ape/relocations.h
o/$(MODE)/ape/ape-no-modify-self.o: \
ape/ape.S \
ape/ape.h \
ape/macros.internal.h \
ape/relocations.h \
ape/ape.internal.h \
libc/dce.h \
libc/elf/def.h \
libc/thread/tls.h \
libc/macho.internal.h \
libc/macros.internal.h \
libc/nexgen32e/uart.internal.h \
libc/calls/metalfile.internal.h \
libc/nt/pedef.internal.h \
libc/runtime/e820.internal.h \
libc/runtime/mman.internal.h \
libc/runtime/pc.internal.h \
libc/sysv/consts/prot.h \
o/$(MODE)/ape/ape.elf
@$(COMPILE) \
-AOBJECTIFY.S \
$(OBJECTIFY.S) \
$(OUTPUT_OPTION) \
-DAPE_NO_MODIFY_SELF \
-DAPE_LOADER='"o/$(MODE)/ape/ape.elf"' $<
o/$(MODE)/ape/ape-copy-self.o: \
ape/ape.S \
ape/ape.h \
ape/macros.internal.h \
ape/relocations.h \
ape/ape.internal.h \
libc/dce.h \
libc/elf/def.h \
libc/thread/tls.h \
libc/macho.internal.h \
libc/macros.internal.h \
libc/nexgen32e/uart.internal.h \
libc/calls/metalfile.internal.h \
libc/nt/pedef.internal.h \
libc/runtime/e820.internal.h \
libc/runtime/mman.internal.h \
libc/runtime/pc.internal.h \
libc/sysv/consts/prot.h
@$(COMPILE) \
-AOBJECTIFY.S \
$(OBJECTIFY.S) \
$(OUTPUT_OPTION) \
-DAPE_NO_MODIFY_SELF $<
o/$(MODE)/ape/loader.o: ape/loader.c
@$(COMPILE) -AOBJECTIFY.c $(CC) -DSUPPORT_VECTOR=121 -g $(APE_LOADER_FLAGS)
o/$(MODE)/ape/loader-gcc.asm: ape/loader.c
@$(COMPILE) -AOBJECTIFY.c $(CC) -DSUPPORT_VECTOR=121 -S -g0 $(APE_LOADER_FLAGS)
o/$(MODE)/ape/loader-clang.asm: ape/loader.c
@$(COMPILE) -AOBJECTIFY.c $(CLANG) -DSUPPORT_VECTOR=121 -S -g0 $(APE_LOADER_FLAGS)
o/$(MODE)/ape/loader-xnu.o: ape/loader.c
@$(COMPILE) -AOBJECTIFY.c $(CC) -DSUPPORT_VECTOR=8 -g $(APE_LOADER_FLAGS)
o/$(MODE)/ape/loader-xnu-gcc.asm: ape/loader.c
@$(COMPILE) -AOBJECTIFY.c $(CC) -DSUPPORT_VECTOR=8 -S -g0 $(APE_LOADER_FLAGS)
o/$(MODE)/ape/loader-xnu-clang.asm: ape/loader.c
@$(COMPILE) -AOBJECTIFY.c $(CLANG) -DSUPPORT_VECTOR=8 -S -g0 $(APE_LOADER_FLAGS)
o/$(MODE)/ape/ape.elf: o/$(MODE)/ape/ape.elf.dbg
@$(COMPILE) -AOBJBINCOPY -w $(OBJBINCOPY) -f -o $@ $<
o/$(MODE)/ape/ape.macho: o/$(MODE)/ape/ape.elf.dbg
@$(COMPILE) -AOBJBINCOPY -w $(OBJBINCOPY) -fm -o $@ $<
APE_LOADER_LDFLAGS = \
-static \
-nostdlib \
--no-dynamic-linker \
-z separate-code \
-z common-page-size=0x1000 \
-z max-page-size=0x10000
o/$(MODE)/ape/ape.elf.dbg: \
o/$(MODE)/ape/loader-macho.o \
o/$(MODE)/ape/start.o \
o/$(MODE)/ape/loader.o \
o/$(MODE)/ape/launch.o \
o/$(MODE)/ape/systemcall.o \
ape/loader.lds
@$(COMPILE) -ALINK.elf $(LD) $(APE_LOADER_LDFLAGS) -o $@ $(patsubst %.lds,-T %.lds,$^)
.PHONY: o/$(MODE)/ape
o/$(MODE)/ape: $(APE_CHECKS) \
o/$(MODE)/ape/ape.o \
o/$(MODE)/ape/ape.lds \
o/$(MODE)/ape/ape.elf \
o/$(MODE)/ape/ape.macho \
o/$(MODE)/ape/ape-copy-self.o \
o/$(MODE)/ape/ape-no-modify-self.o
endif
# these assembly files are safe to build on aarch64
o/$(MODE)/ape/ape.o: \
ape/ape.S \
ape/ape.h \
libc/dce.h \
libc/elf/def.h \
ape/relocations.h \
libc/thread/tls.h \
ape/ape.internal.h \
ape/macros.internal.h \
libc/macho.internal.h \
libc/macros.internal.h \
libc/sysv/consts/prot.h \
libc/nt/pedef.internal.h \
libc/runtime/pc.internal.h \
libc/runtime/e820.internal.h \
libc/runtime/mman.internal.h \
libc/nexgen32e/uart.internal.h \
libc/calls/metalfile.internal.h
@$(COMPILE) -AOBJECTIFY.S $(OBJECTIFY.S) $(OUTPUT_OPTION) -c $<
o/$(MODE)/ape/ape.lds: \
ape/ape.lds \
ape/macros.internal.h \
ape/relocations.h \
ape/ape.internal.h \
libc/thread/tls.h \
libc/calls/struct/timespec.h \
libc/thread/thread.h \
libc/dce.h \
libc/elf/def.h \
libc/elf/pf2prot.internal.h \
libc/macros.internal.h \
libc/nt/pedef.internal.h \
libc/str/str.h \
libc/zip.internal.h
+298
View File
@@ -0,0 +1,298 @@
/*-*- mode: ld-script; indent-tabs-mode: nil; tab-width: 2; coding: utf-8 -*-│
│ vi: set et sts=2 sw=2 fenc=utf-8 :vi │
╚─────────────────────────────────────────────────────────────────────────────*/
#include "libc/sysv/consts/prot.h"
#include "libc/thread/tls.h"
ENTRY(_start)
OUTPUT_ARCH(aarch64)
OUTPUT_FORMAT("elf64-littleaarch64",
"elf64-bigaarch64",
"elf64-littleaarch64")
SECTIONS {
. = SEGMENT_START("text-segment", 0x000800000000);
__executable_start = .;
. += SIZEOF_HEADERS;
.interp : { *(.interp) }
.note.gnu.build-id : { *(.note.gnu.build-id) }
.hash : { *(.hash) }
.gnu.hash : { *(.gnu.hash) }
.dynsym : { *(.dynsym) }
.dynstr : { *(.dynstr) }
.gnu.version : { *(.gnu.version) }
.gnu.version_d : { *(.gnu.version_d) }
.gnu.version_r : { *(.gnu.version_r) }
.rela.init : { *(.rela.init) }
.rela.text : { *(.rela.text .rela.text.* .rela.gnu.linkonce.t.*) }
.rela.fini : { *(.rela.fini) }
.rela.rodata : { *(.rela.rodata .rela.rodata.* .rela.gnu.linkonce.r.*) }
.rela.data.rel.ro : { *(.rela.data.rel.ro .rela.data.rel.ro.* .rela.gnu.linkonce.d.rel.ro.*) }
.rela.data : { *(.rela.data .rela.data.* .rela.gnu.linkonce.d.*) }
.rela.tdata : { *(.rela.tdata .rela.tdata.* .rela.gnu.linkonce.td.*) }
.rela.tbss : { *(.rela.tbss .rela.tbss.* .rela.gnu.linkonce.tb.*) }
.rela.ctors : { *(.rela.ctors) }
.rela.dtors : { *(.rela.dtors) }
.rela.got : { *(.rela.got) }
.rela.bss : { *(.rela.bss .rela.bss.* .rela.gnu.linkonce.b.*) }
.rela.ifunc : { *(.rela.ifunc) }
.rela.plt : {
*(.rela.plt)
__rela_iplt_start = .;
*(.rela.iplt)
__rela_iplt_end = .;
}
.init : {
*(.start)
KEEP(*(.initprologue))
KEEP(*(SORT_NONE(.init)))
KEEP(*(.initepilogue))
} =0x1f2003d5
.plt : ALIGN(16) {
*(.plt)
*(.iplt)
}
.text : {
*(.text.unlikely .text.*_unlikely .text.unlikely.*)
*(.text.antiquity .text.antiquity.*)
*(.text.exit .text.exit.*)
*(.text.startup .text.startup.*)
*(.text.hot .text.hot.*)
*(.text.modernity .text.modernity.*)
*(.text .stub .text.* .gnu.linkonce.t.*)
*(.gnu.warning)
} =0x1f2003d5
.fini : {
KEEP(*(SORT_NONE(.fini)))
} =0x1f2003d5
. += CONSTANT(MAXPAGESIZE);
.privileged : {
__privileged_start = ABSOLUTE(.) & -CONSTANT(MAXPAGESIZE);
*(.privileged*)
} =0x1f2003d6
.rodata : {
KEEP(*(.rodata.pytab.0));
KEEP(*(.rodata.pytab.1));
KEEP(*(.rodata.pytab.2));
KEEP(*(SORT_BY_NAME(.sort.rodata.*)))
*(.rodata .rodata.* .gnu.linkonce.r.*)
*(.ubsan.types)
*(.ubsan.data)
}
.notice : {
__notices = .;
KEEP(*(.notice))
BYTE(0);
BYTE(10);
BYTE(10);
}
.eh_frame_hdr : {
*(.eh_frame_hdr)
*(.eh_frame_entry .eh_frame_entry.*)
}
.eh_frame : ONLY_IF_RO {
KEEP(*(.eh_frame))
*(.eh_frame.*)
}
.gcc_except_table : ONLY_IF_RO {
*(.gcc_except_table .gcc_except_table.*)
}
.gnu_extab : ONLY_IF_RO {
*(.gnu_extab*)
}
.exception_ranges : ONLY_IF_RO {
*(.exception_ranges*)
}
__etext = .;
_etext = .;
PROVIDE(etext = .);
. += CONSTANT(MAXPAGESIZE);
. = DATA_SEGMENT_ALIGN(CONSTANT(MAXPAGESIZE), CONSTANT(COMMONPAGESIZE));
.eh_frame : ONLY_IF_RW {
KEEP(*(.eh_frame))
*(.eh_frame.*)
}
.gnu_extab : ONLY_IF_RW {
*(.gnu_extab)
}
.gcc_except_table : ONLY_IF_RW {
*(.gcc_except_table .gcc_except_table.*)
}
.exception_ranges : ONLY_IF_RW {
*(.exception_ranges*)
}
.tdata : {
_tdata_start = .;
__tdata_start = .;
*(.tdata .tdata.* .gnu.linkonce.td.*)
_tdata_end = .;
}
.tbss : {
_tbss_start = .;
*(.tbss .tbss.* .gnu.linkonce.tb.*)
*(.tcommon)
_tbss_end = .;
}
.init_array : {
__init_array_start = .;
KEEP(*(.preinit_array))
KEEP(*(SORT_BY_INIT_PRIORITY(.init_array.*)
SORT_BY_INIT_PRIORITY(.ctors.*)))
KEEP(*(.init_array))
KEEP(*(.ctors))
__init_array_end = .;
}
.fini_array : {
__fini_array_start = .;
KEEP(*(SORT_BY_INIT_PRIORITY(.fini_array.*) SORT_BY_INIT_PRIORITY(.dtors.*)))
KEEP(*(.fini_array EXCLUDE_FILE(*crtbegin.o *crtbegin?.o *crtend.o *crtend?.o ) .dtors))
__fini_array_end = .;
}
.data.rel.ro : {
KEEP(*(SORT_BY_NAME(.piro.relo.sort.*)))
*(.data.rel.ro.local* .gnu.linkonce.d.rel.ro.local.*)
*(.data.rel.ro .data.rel.ro.* .gnu.linkonce.d.rel.ro.*)
}
.dynamic : {
*(.dynamic)
}
.got : {
*(.got)
*(.igot)
}
. = DATA_SEGMENT_RELRO_END(24, .);
.got.plt : {
*(.got.plt)
*(.igot.plt)
}
.data : {
__data_start = .;
KEEP(*(SORT_BY_NAME(.piro.data.sort.*)))
*(.data .data.* .gnu.linkonce.d.*)
KEEP(*(SORT_BY_NAME(.sort.data.*)))
SORT(CONSTRUCTORS)
}
_edata = .;
PROVIDE(edata = .);
. = .;
__bss_start = .;
__bss_start__ = .;
.bss : {
*(.dynbss)
*(.bss .bss.* .gnu.linkonce.b.*)
KEEP(*(SORT_BY_NAME(.piro.bss.sort.*)))
*(COMMON)
. = ALIGN(CONSTANT(COMMONPAGESIZE));
}
_bss_end__ = .;
__bss_end__ = .;
. = ALIGN(64 / 8);
. = SEGMENT_START("ldata-segment", .);
. = ALIGN(64 / 8);
__end__ = .;
_end = .;
PROVIDE(end = .);
. = DATA_SEGMENT_END(.);
.stab 0 : { *(.stab) }
.stabstr 0 : { *(.stabstr) }
.stab.excl 0 : { *(.stab.excl) }
.stab.exclstr 0 : { *(.stab.exclstr) }
.stab.index 0 : { *(.stab.index) }
.stab.indexstr 0 : { *(.stab.indexstr) }
.gnu.build.attributes : { *(.gnu.build.attributes .gnu.build.attributes.*) }
.debug 0 : { *(.debug) }
.line 0 : { *(.line) }
.debug_srcinfo 0 : { *(.debug_srcinfo) }
.debug_sfnames 0 : { *(.debug_sfnames) }
.debug_aranges 0 : { *(.debug_aranges) }
.debug_pubnames 0 : { *(.debug_pubnames) }
.debug_info 0 : { *(.debug_info .gnu.linkonce.wi.*) }
.debug_abbrev 0 : { *(.debug_abbrev) }
.debug_line 0 : { *(.debug_line .debug_line.* .debug_line_end) }
.debug_frame 0 : { *(.debug_frame) }
.debug_str 0 : { *(.debug_str) }
.debug_loc 0 : { *(.debug_loc) }
.debug_macinfo 0 : { *(.debug_macinfo) }
.debug_weaknames 0 : { *(.debug_weaknames) }
.debug_funcnames 0 : { *(.debug_funcnames) }
.debug_typenames 0 : { *(.debug_typenames) }
.debug_varnames 0 : { *(.debug_varnames) }
.debug_pubtypes 0 : { *(.debug_pubtypes) }
.debug_ranges 0 : { *(.debug_ranges) }
.debug_macro 0 : { *(.debug_macro) }
.debug_addr 0 : { *(.debug_addr) }
.ARM.attributes 0 : { KEEP(*(.ARM.attributes)) KEEP(*(.gnu.attributes)) }
.note.gnu.arm.ident 0 : { KEEP(*(.note.gnu.arm.ident)) }
.zip 0 : {
KEEP(*(.zip.file))
__zip_cdir_start = .;
KEEP(*(.zip.cdir))
__zip_cdir_size = . - __zip_cdir_start;
KEEP(*(.zip.eocd))
}
/DISCARD/ : {
*(__patchable_function_entries)
*(.GCC.command.line)
*(.note.GNU-stack)
*(.gnu_debuglink)
*(.text.windows)
*(.gnu.lto_*)
*(.eh_frame)
*(.idata.*)
*(.yoink)
*(.head)
}
}
ape_stack_vaddr = DEFINED(ape_stack_vaddr) ? ape_stack_vaddr : 0x700000000000;
ape_stack_memsz = DEFINED(ape_stack_memsz) ? ape_stack_memsz : 8 * 1024 * 1024;
ape_stack_prot = PROT_READ | PROT_WRITE;
_tls_size = _tbss_end - _tdata_start;
_tdata_size = _tdata_end - _tdata_start;
_tbss_size = _tbss_end - _tbss_start;
_tbss_offset = _tbss_start - _tdata_start;
_tls_content = (_tdata_end - _tdata_start) + (_tbss_end - _tbss_start);
_tdata_align = ALIGNOF(.tdata);
_tbss_align = ALIGNOF(.tbss);
_tls_align = MAX(TLS_ALIGNMENT, MAX(ALIGNOF(.tdata), ALIGNOF(.tbss)));
+1133
View File
File diff suppressed because it is too large. Load diff
+512 -608
View File
File diff suppressed because it is too large. Load diff
+13
View File
@@ -0,0 +1,13 @@
#ifndef COSMOPOLITAN_APE_APE_H_
#define COSMOPOLITAN_APE_APE_H_
#define APE_VERSION_MAJOR 1
#define APE_VERSION_MINOR 10
#define APE_VERSION_STR APE_VERSION_STR_(APE_VERSION_MAJOR, APE_VERSION_MINOR)
#define APE_VERSION_NOTE APE_VERSION_NOTE_(APE_VERSION_MAJOR, APE_VERSION_MINOR)
#define APE_VERSION_STR__(x, y) #x "." #y
#define APE_VERSION_STR_(x, y) APE_VERSION_STR__(x, y)
#define APE_VERSION_NOTE_(x, y) (100000000 * (x) + 1000000 * (y))
#endif /* COSMOPOLITAN_APE_APE_H_ */
+9
View File
@@ -0,0 +1,9 @@
#ifndef COSMOPOLITAN_APE_APE_INTERNAL_H_
#define COSMOPOLITAN_APE_APE_INTERNAL_H_
#include "libc/dce.h"
#if SupportsWindows() || SupportsMetal() || SupportsXnu()
#define APE_IS_SHELL_SCRIPT
#endif
#endif /* COSMOPOLITAN_APE_APE_INTERNAL_H_ */
+230 -251
View File
@@ -1,5 +1,5 @@
/*-*- mode: ld-script; indent-tabs-mode: nil; tab-width: 2; coding: utf-8 -*-│
│vi: set et sts=2 tw=2 fenc=utf-8 :vi│
│ vi: set et sts=2 sw=2 fenc=utf-8 :vi │
╞══════════════════════════════════════════════════════════════════════════════╡
│ Copyright 2020 Justine Alexandra Roberts Tunney │
│ │
@@ -181,16 +181,39 @@
#include "libc/elf/pf2prot.internal.h"
#include "libc/nt/pedef.internal.h"
#include "libc/thread/tls.h"
#include "libc/zip.h"
#include "ape/ape.internal.h"
/* uncomment if .com.dbg won't execute on your kernel (will break .com file) */
/* #define APE_FIX_COM_DBG */
#ifdef __x86__
#define CODE_GRANULE 1
#else
#define CODE_GRANULE 4
#endif
#ifdef APE_FIX_COM_DBG
#define SKEW SIZEOF_HEADERS
#else
#define SKEW 0
#endif
#ifndef IMAGE_BASE_VIRTUAL
#define IMAGE_BASE_VIRTUAL 0x400000
#endif
#if IMAGE_BASE_VIRTUAL > 0xffffffff
#error "please use 32-bit addresses for image data"
#endif
ENTRY(_start)
PHDRS {
Head PT_LOAD FLAGS(PF_X|PF_R);
Rom PT_LOAD FLAGS(PF_X|PF_R);
Ram PT_LOAD FLAGS(PF_W|PF_R);
Cod PT_LOAD FLAGS(PF_X|PF_R);
Rom PT_LOAD FLAGS(PF_R);
Tls PT_TLS FLAGS(PF_W|PF_R);
Bss PT_LOAD FLAGS(PF_W|PF_R);
Ram PT_LOAD FLAGS(PF_W|PF_R);
stack PT_GNU_STACK FLAGS(PF_W|PF_R);
}
@@ -201,44 +224,44 @@ SECTIONS {
/*BEGIN: linux addressability guarantee */
/*BEGIN: bsd addressability guarantee */
.head SEGMENT_START("text-segment", IMAGE_BASE_VIRTUAL) : AT(IMAGE_BASE_REAL) {
HIDDEN(_base = .);
.head SEGMENT_START("text-segment", IMAGE_BASE_VIRTUAL) + SKEW : AT(IMAGE_BASE_REAL) {
__executable_start = .;
/* Real Mode */
KEEP(*(.head))
KEEP(*(.apesh))
KEEP(*(.head2))
KEEP(*(.text.head))
/* Executable & Linkable Format */
. = ALIGN(__SIZEOF_POINTER__);
HIDDEN(ape_phdrs = .);
. = ALIGN(. != 0 ? __SIZEOF_POINTER__ : 0);
ape_phdrs = .;
KEEP(*(.elf.phdrs))
HIDDEN(ape_phdrs_end = .);
KEEP(*(.emush))
ape_phdrs_end = .;
/* OpenBSD */
. = ALIGN(__SIZEOF_POINTER__);
HIDDEN(ape_note = .);
. = ALIGN(. != 0 ? __SIZEOF_POINTER__ : 0);
ape_note = .;
KEEP(*(.note.openbsd.ident))
KEEP(*(.note.netbsd.ident))
HIDDEN(ape_note_end = .);
ape_note_end = .;
/* Portable Executable */
KEEP(*(.pe.header))
HIDDEN(ape_pe_sections = .);
ape_pe_sections = .;
KEEP(*(.pe.sections))
HIDDEN(ape_pe_sections_end = .);
ape_pe_sections_end = .;
/* Mach-O */
KEEP(*(.macho))
. = ALIGN(__SIZEOF_POINTER__);
HIDDEN(ape_macho_end = .);
. = ALIGN(. != 0 ? __SIZEOF_POINTER__ : 0);
ape_macho_end = .;
/* APE loader */
KEEP(*(.ape.loader))
. = ALIGN(. != 0 ? CODE_GRANULE : 0);
KEEP(*(.ape.pad.head))
. = ALIGN(SupportsWindows() || SupportsMetal() ? PAGESIZE : 16);
HIDDEN(_ehead = .);
. = ALIGN(. != 0 ? (SupportsWindows() || SupportsMetal() ? CONSTANT(COMMONPAGESIZE) : 16) : 0);
_ehead = .;
} :Head
/*BEGIN: nt addressability guarantee */
@@ -248,12 +271,10 @@ SECTIONS {
/* Code that needs to be addressable in Real Mode */
*(.text.real)
KEEP(*(SORT_BY_NAME(.sort.text.real.*)))
/* Code we want earlier in the binary w/o modifications */
KEEP(*(.ape.loader))
HIDDEN(_ereal = .);
_ereal = .;
/*END: realmode addressability guarantee */
/*BEGIN: morphable code */
. += 1;
. += CODE_GRANULE;
/* Normal Code */
*(.start)
@@ -261,12 +282,9 @@ SECTIONS {
KEEP(*(SORT_BY_NAME(.init.*)))
KEEP(*(.init))
KEEP(*(.initepilogue))
KEEP(*(.pltprologue))
*(.plt)
KEEP(*(.pltepilogue))
KEEP(*(.pltgotprologue))
*(.plt.got)
KEEP(*(.pltgotepilogue))
*(.iplt)
*(.text.startup .text.startup.*)
*(.text.exit .text.exit.*)
*(.text.unlikely .text.*_unlikely .text.unlikely.*)
@@ -275,7 +293,6 @@ SECTIONS {
KEEP(*(.textwindowsprologue))
*(.text.windows)
KEEP(*(.textwindowsepilogue))
KEEP(*(.blink))
*(SORT_BY_ALIGNMENT(.text.modernity))
*(SORT_BY_ALIGNMENT(.text.modernity.*))
*(SORT_BY_ALIGNMENT(.text.hot))
@@ -283,7 +300,6 @@ SECTIONS {
KEEP(*(.keep.text))
*(.text .stub .text.*)
KEEP(*(SORT_BY_NAME(.sort.text.*)))
*(.subrs)
KEEP(*(.ape.pad.test));
*(.test.unlikely)
@@ -291,17 +307,20 @@ SECTIONS {
/* Privileged code invulnerable to magic */
KEEP(*(.ape.pad.privileged));
. += . > 0 ? 1 : 0;
. = ALIGN(__privileged_end > __privileged_start ? CONSTANT(COMMONPAGESIZE) : 0);
/*END: morphable code */
HIDDEN(__privileged_start = .);
. += . > 0 ? 1 : 0;
__privileged_start = .;
*(.privileged)
HIDDEN(__privileged_end = .);
. += . > 0 ? 1 : 0;
__privileged_end = .;
KEEP(*(.ape.pad.text))
. = ALIGN(. != 0 ? CONSTANT(COMMONPAGESIZE) : 0);
/*END: Read Only Data (only needed for initialization) */
} :Cod
/*BEGIN: Read Only Data */
KEEP(*(.ape.pad.rodata));
.rodata ALIGN(CONSTANT(COMMONPAGESIZE)) : {
KEEP(*(.rodata.pytab.0));
KEEP(*(.rodata.pytab.1));
KEEP(*(.rodata.pytab.2));
@@ -310,48 +329,30 @@ SECTIONS {
*(.ubsan.types)
*(.ubsan.data)
/* Unit Test & Fixture Registry */
/*BEGIN: Read only data that needn't be mapped after initialization */
/* Legal Notices */
#if !defined(IM_FEELING_NAUGHTY) || defined(EMBED_NOTICES)
KEEP(*(.commentprologue))
KEEP(*(.comment))
KEEP(*(.commentepilogue))
#endif
__notices = .;
KEEP(*(.notice))
BYTE(0);
BYTE(10);
BYTE(10);
/*BEGIN: read-only data that's only needed for initialization */
#if SupportsWindows()
/* Windows DLL Import Directory */
KEEP(*(.idata.ro));
KEEP(*(SORT_BY_NAME(.idata.ro.*)))
. = ALIGN(__SIZEOF_POINTER__);
PROVIDE_HIDDEN(__init_array_start = .);
KEEP(*(SORT_BY_INIT_PRIORITY(.init_array.*)
SORT_BY_INIT_PRIORITY(.ctors.*)))
KEEP(*(.ctors))
KEEP(*(.init_array))
KEEP(*(.preinit_array))
PROVIDE_HIDDEN(__init_array_end = .);
. = ALIGN(__SIZEOF_POINTER__);
PROVIDE_HIDDEN(__fini_array_start = .);
KEEP(*(SORT_BY_INIT_PRIORITY(.fini_array.*)
SORT_BY_INIT_PRIORITY(.dtors.*)))
KEEP(*(.dtors))
PROVIDE_HIDDEN(__fini_array_end = .);
#endif
/* Encoded Data Structures w/ Linear Initialization Order */
KEEP(*(.initroprologue))
KEEP(*(SORT_BY_NAME(.initro.*)))
KEEP(*(.initroepilogue))
KEEP(*(SORT_BY_NAME(.sort.rodata.*)))
. = ALIGN(. != 0 ? CONSTANT(COMMONPAGESIZE) : 0); /* don't delete this line :o */
/*END: read-only data that's only needed for initialization */
KEEP(*(.ape.pad.text))
. = ALIGN(PAGESIZE);
HIDDEN(_etext = .);
PROVIDE_HIDDEN(etext = .);
/*END: Read Only Data (only needed for initialization) */
} :Rom
/* initialization image for thread-local storage, this is copied */
@@ -361,66 +362,96 @@ SECTIONS {
*(SORT_BY_ALIGNMENT(.tdata))
*(SORT_BY_ALIGNMENT(.tdata.*))
_tdata_end = .;
. = ALIGN(PAGESIZE);
KEEP(*(.ape.pad.rodata))
. = ALIGN(. != 0 ? CONSTANT(COMMONPAGESIZE) : 0);
_etext = .;
PROVIDE(etext = .);
} :Tls :Rom
/*END: Read Only Data */
. = ALIGN(CONSTANT(COMMONPAGESIZE));
/* this only tells the linker about the layout of uninitialized */
/* TLS data, and does not advance the linker's location counter */
.tbss . : {
.tbss : {
_tbss_start = .;
*(SORT_BY_ALIGNMENT(.tbss))
*(SORT_BY_ALIGNMENT(.tbss.*))
. = ALIGN(TLS_ALIGNMENT);
KEEP(*(.fstls))
/* the %fs register is based on this location */
_tbss_end = .;
} :Tls
.data ALIGN(PAGESIZE) : {
.data . : {
/*BEGIN: Read/Write Data */
#if SupportsWindows()
KEEP(*(SORT_BY_NAME(.piro.data.sort.iat.*)))
#endif
/*BEGIN: NT FORK COPYING */
KEEP(*(.dataprologue))
*(.data .data.*)
*(.gnu_extab)
*(.gcc_except_table .gcc_except_table.*)
*(.exception_ranges*)
*(.PyRuntime) /* for python */
*(.subrs) /* for emacs */
KEEP(*(SORT_BY_NAME(.sort.data.*)))
. += . > 0 ? 1 : 0;
. += . > 0 ? CODE_GRANULE : 0;
KEEP(*(.gotprologue))
. = ALIGN(. != 0 ? __SIZEOF_POINTER__ : 0);
__got_start = .;
*(.got)
KEEP(*(.gotepilogue))
__got_end = .;
KEEP(*(.gotpltprologue))
*(.got.plt)
KEEP(*(.gotpltepilogue))
. = ALIGN(. != 0 ? __SIZEOF_POINTER__ : 0);
__init_array_start = .;
KEEP(*(.preinit_array))
KEEP(*(SORT_BY_INIT_PRIORITY(.init_array.*)
SORT_BY_INIT_PRIORITY(.ctors.*)))
KEEP(*(.init_array))
KEEP(*(.ctors))
__init_array_end = .;
. = ALIGN(. != 0 ? __SIZEOF_POINTER__ : 0);
__fini_array_start = .;
KEEP(*(SORT_BY_INIT_PRIORITY(.fini_array.*)
SORT_BY_INIT_PRIORITY(.dtors.*)))
KEEP(*(.fini_array))
KEEP(*(.dtors))
__fini_array_end = .;
/*BEGIN: Post-Initialization Read-Only */
. = ALIGN(__SIZEOF_POINTER__);
. = ALIGN(. != 0 ? __SIZEOF_POINTER__ : 0);
KEEP(*(SORT_BY_NAME(.piro.relo.sort.*)))
PROVIDE_HIDDEN(__relo_end = .);
. = ALIGN(__SIZEOF_POINTER__);
. = ALIGN(. != 0 ? __SIZEOF_POINTER__ : 0);
KEEP(*(SORT_BY_NAME(.piro.data.sort.*)))
KEEP(*(.piro.pad.data))
*(.igot.plt)
KEEP(*(.dataepilogue))
. = ALIGN(. != 0 ? CONSTANT(COMMONPAGESIZE) : 0);
/*END: NT FORK COPYING */
HIDDEN(_edata = .);
PROVIDE_HIDDEN(edata = .);
KEEP(*(SORT_BY_NAME(.zip.*)))
HIDDEN(_ezip = .);
. = ALIGN(PAGESIZE);
_edata = .;
PROVIDE(edata = .);
_ezip = .; /* <-- very deprecated */
} :Ram
. = ALIGN(CONSTANT(COMMONPAGESIZE));
/*END: file content that's loaded by o/s */
/*END: file content */
/*BEGIN: bss memory that's addressable */
.bss ALIGN(64) : {
.bss : {
/*BEGIN: NT FORK COPYING */
KEEP(*(.bssprologue))
KEEP(*(SORT_BY_NAME(.piro.bss.init.*)))
*(.piro.bss)
KEEP(*(SORT_BY_NAME(.piro.bss.sort.*)))
HIDDEN(__piro_end = .);
. += . > 0 ? 1 : 0;
__piro_end = .;
. += . > 0 ? CODE_GRANULE : 0;
/*END: Post-Initialization Read-Only */
/* Statically Allocated Empty Space */
@@ -431,11 +462,15 @@ SECTIONS {
KEEP(*(SORT_BY_NAME(.sort.bss.*)))
KEEP(*(.bssepilogue))
. = ALIGN(. != 0 ? CONSTANT(COMMONPAGESIZE) : 0);
/*END: NT FORK COPYING */
. = ALIGN(PAGESIZE);
HIDDEN(_end = .);
PROVIDE_HIDDEN(end = .);
} :Bss
} :Ram
. = ALIGN(CONSTANT(COMMONPAGESIZE));
_end = .;
PROVIDE(end = .);
/*END: nt addressability guarantee */
/*END: bsd addressability guarantee */
@@ -478,14 +513,34 @@ SECTIONS {
.gnu.attributes 0 : { KEEP(*(.gnu.attributes)) }
.GCC.command.line 0 : { *(.GCC.command.line) }
.zip 0 : {
KEEP(*(.zip.file))
__zip_cdir_start = .;
KEEP(*(.zip.cdir))
__zip_cdir_size = . - __zip_cdir_start;
KEEP(*(.zip.eocd))
}
/DISCARD/ : {
#if !SupportsWindows()
*(.idata.ro);
*(.idata.ro.*)
*(.piro.data.sort.iat.*)
#endif
*(__patchable_function_entries)
*(.note.gnu.property)
*(__mcount_loc)
*(.rela.dyn)
*(.discard)
*(.yoink)
*(.*)
}
}
ape_pe_filealignment = 512;
ape_pe_sectionalignment = 4096;
ape_pe_sizeofheaders = SIZEOF(.head);
ape_pe_sizeofimage = ROUNDUP(_end - __executable_start, ape_pe_sectionalignment);
PFSTUB8(ape_elf_entry, _start);
PFSTUB8(ape_elf_phoff, RVA(ape_phdrs));
PFSTUB8(ape_elf_shoff, 0);
@@ -493,70 +548,58 @@ PFSTUB4(ape_elf_phnum, (ape_phdrs_end - ape_phdrs) / 56);
PFSTUB4(ape_elf_shnum, 0);
PFSTUB4(ape_elf_shstrndx, 0);
HIDDEN(_tls_size = _tbss_end - _tdata_start);
HIDDEN(_tdata_size = _tdata_end - _tdata_start);
HIDDEN(_tbss_size = _tbss_end - _tbss_start);
HIDDEN(_tbss_offset = _tbss_start - _tdata_start);
HIDDEN(_tls_content = (_tdata_end - _tdata_start) + (_tbss_end - _tbss_start));
_tls_size = _tbss_end - _tdata_start;
_tdata_size = _tdata_end - _tdata_start;
_tbss_size = _tbss_end - _tbss_start;
_tbss_offset = _tbss_start - _tdata_start;
_tls_content = (_tdata_end - _tdata_start) + (_tbss_end - _tbss_start);
_tdata_align = ALIGNOF(.tdata);
_tbss_align = ALIGNOF(.tbss);
_tls_align = MAX(TLS_ALIGNMENT, MAX(ALIGNOF(.tdata), ALIGNOF(.tbss)));
HIDDEN(__privileged_addr = ROUNDDOWN(__privileged_start, PAGESIZE));
HIDDEN(__privileged_size = (ROUNDUP(__privileged_end, PAGESIZE) -
ROUNDDOWN(__privileged_start, PAGESIZE)));
ape_cod_offset = 0;
ape_cod_vaddr = ADDR(.head);
ape_cod_paddr = LOADADDR(.head);
ape_cod_filesz = ADDR(.rodata) - ADDR(.head);
ape_cod_memsz = ape_cod_filesz;
ape_cod_align = CONSTANT(COMMONPAGESIZE);
ape_cod_rva = RVA(ape_cod_vaddr);
HIDDEN(ape_rom_offset = 0);
HIDDEN(ape_rom_vaddr = ADDR(.head));
HIDDEN(ape_rom_paddr = LOADADDR(.head));
HIDDEN(ape_rom_filesz = LOADADDR(.data) - ape_rom_paddr);
HIDDEN(ape_rom_memsz = ADDR(.data) - ADDR(.head));
HIDDEN(ape_rom_align = PAGESIZE);
HIDDEN(ape_rom_rva = RVA(ape_rom_vaddr));
ape_rom_vaddr = ADDR(.rodata);
ape_rom_offset = ape_rom_vaddr - __executable_start;
ape_rom_paddr = LOADADDR(.rodata);
ape_rom_filesz = ADDR(.tbss) - ADDR(.rodata);
ape_rom_memsz = ape_rom_filesz;
ape_rom_align = CONSTANT(COMMONPAGESIZE);
ape_rom_rva = RVA(ape_rom_vaddr);
HIDDEN(ape_ram_offset = ape_rom_offset + ape_rom_filesz);
HIDDEN(ape_ram_vaddr = ADDR(.data));
HIDDEN(ape_ram_paddr = LOADADDR(.data));
HIDDEN(ape_ram_filesz = SIZEOF(.data));
HIDDEN(ape_ram_memsz = ADDR(.bss) + SIZEOF(.bss) - ape_ram_vaddr);
HIDDEN(ape_ram_align = PAGESIZE);
HIDDEN(ape_ram_rva = RVA(ape_ram_vaddr));
ape_ram_vaddr = ADDR(.data);
ape_ram_offset = ape_ram_vaddr - __executable_start;
ape_ram_paddr = LOADADDR(.data);
ape_ram_filesz = ADDR(.bss) - ADDR(.data);
ape_ram_memsz = _end - ADDR(.data);
ape_ram_align = CONSTANT(COMMONPAGESIZE);
ape_ram_rva = RVA(ape_ram_vaddr);
HIDDEN(ape_stack_pf = DEFINED(ape_stack_pf) ? ape_stack_pf : PF_R | PF_W);
HIDDEN(ape_stack_prot = _PF2PROT(ape_stack_pf));
HIDDEN(ape_stack_offset = ape_ram_offset + ape_ram_filesz);
HIDDEN(ape_stack_vaddr = DEFINED(ape_stack_vaddr) ? ape_stack_vaddr : 0x700000000000);
HIDDEN(ape_stack_paddr = ape_ram_paddr + ape_ram_filesz);
HIDDEN(ape_stack_filesz = 0);
HIDDEN(ape_stack_memsz = DEFINED(ape_stack_memsz) ? ape_stack_memsz : STACKSIZE);
HIDDEN(ape_stack_align = 16);
ape_stack_pf = DEFINED(ape_stack_pf) ? ape_stack_pf : PF_R | PF_W;
ape_stack_prot = _PF2PROT(ape_stack_pf);
ape_stack_offset = 0;
ape_stack_vaddr = DEFINED(ape_stack_vaddr) ? ape_stack_vaddr : 0x700000000000;
ape_stack_paddr = ape_ram_paddr + ape_ram_filesz;
ape_stack_filesz = 0;
ape_stack_memsz = DEFINED(ape_stack_memsz) ? ape_stack_memsz : 8 * 1024 * 1024;
HIDDEN(ape_note_offset = ape_rom_offset + (ape_note - ape_rom_vaddr));
HIDDEN(ape_note_vaddr = ape_note);
HIDDEN(ape_note_paddr = ape_rom_paddr + ape_note_offset);
HIDDEN(ape_note_filesz = ape_note_end - ape_note);
HIDDEN(ape_note_memsz = ape_note_filesz);
HIDDEN(ape_note_align = __SIZEOF_POINTER__);
ape_note_offset = ape_cod_offset + (ape_note - ape_cod_vaddr);
ape_note_filesz = ape_note_end - ape_note;
ape_note_memsz = ape_note_filesz;
HIDDEN(ape_text_offset = ape_rom_offset + LOADADDR(.text) - ape_rom_paddr);
HIDDEN(ape_text_paddr = LOADADDR(.text));
HIDDEN(ape_text_vaddr = ADDR(.text));
HIDDEN(ape_text_filesz = SIZEOF(.text) + SIZEOF(.tdata));
HIDDEN(ape_text_memsz = SIZEOF(.text) + SIZEOF(.tdata));
HIDDEN(ape_text_align = PAGESIZE);
HIDDEN(ape_text_rva = RVA(ape_text_vaddr));
HIDDEN(ape_data_offset = ape_ram_offset + LOADADDR(.data) - ape_ram_paddr);
HIDDEN(ape_data_paddr = LOADADDR(.data));
HIDDEN(ape_data_vaddr = ADDR(.data));
HIDDEN(ape_data_filesz = SIZEOF(.data));
HIDDEN(ape_data_memsz = SIZEOF(.data));
HIDDEN(ape_data_align = PAGESIZE);
HIDDEN(ape_data_rva = RVA(ape_data_vaddr));
HIDDEN(ape_bss_offset = ape_ram_offset + LOADADDR(.bss) - ape_ram_paddr);
HIDDEN(ape_bss_paddr = LOADADDR(.bss));
HIDDEN(ape_bss_vaddr = ADDR(.bss));
HIDDEN(ape_bss_filesz = 0);
HIDDEN(ape_bss_memsz = SIZEOF(.bss));
HIDDEN(ape_bss_align = PAGESIZE);
ape_text_vaddr = ADDR(.text);
ape_text_offset = ape_text_vaddr - __executable_start;
ape_text_paddr = LOADADDR(.text);
ape_text_filesz = ADDR(.rodata) - ADDR(.text);
ape_text_memsz = ape_text_filesz;
ape_text_align = CONSTANT(COMMONPAGESIZE);
ape_text_rva = RVA(ape_text_vaddr);
/* we roundup here because xnu wants the file load segments page-aligned */
/* but we don't want to add the nop padding to the ape program, so we'll */
@@ -564,72 +607,19 @@ HIDDEN(ape_bss_align = PAGESIZE);
SHSTUB2(ape_loader_dd_skip, DEFINED(ape_loader) ? RVA(ape_loader) / 64 : 0);
SHSTUB2(ape_loader_dd_count,
DEFINED(ape_loader_end)
? ROUNDUP(ape_loader_end - ape_loader, PAGESIZE) / 64
? ROUNDUP(ape_loader_end - ape_loader, CONSTANT(COMMONPAGESIZE)) / 64
: 0);
HIDDEN(blink_aarch64_b0 = RVA(blink_aarch64) % 10 + 48);
HIDDEN(blink_aarch64_b1 = RVA(blink_aarch64) < 10 ? 32 : RVA(blink_aarch64) / 10 % 10 + 48);
HIDDEN(blink_aarch64_b2 = RVA(blink_aarch64) < 100 ? 32 : RVA(blink_aarch64) / 100 % 10 + 48);
HIDDEN(blink_aarch64_b3 = RVA(blink_aarch64) < 1000 ? 32 : RVA(blink_aarch64) / 1000 % 10 + 48);
HIDDEN(blink_aarch64_b4 = RVA(blink_aarch64) < 10000 ? 32 : RVA(blink_aarch64) / 10000 % 10 + 48);
HIDDEN(blink_aarch64_b5 = RVA(blink_aarch64) < 100000 ? 32 : RVA(blink_aarch64) / 100000 % 10 + 48);
HIDDEN(blink_aarch64_b6 = RVA(blink_aarch64) < 1000000 ? 32 : RVA(blink_aarch64) / 1000000 % 10 + 48);
HIDDEN(blink_aarch64_b7 = RVA(blink_aarch64) < 10000000 ? 32 : RVA(blink_aarch64) / 10000000 % 10 + 48);
HIDDEN(blink_aarch64_b8 = RVA(blink_aarch64) < 100000000 ? 32 : RVA(blink_aarch64) / 100000000 % 10 + 48);
HIDDEN(blink_aarch64_b9 = RVA(blink_aarch64) < 1000000000 ? 32 : RVA(blink_aarch64) / 1000000000 % 10 + 48);
HIDDEN(blink_aarch64_size_b0 = blink_aarch64_size % 10 + 48);
HIDDEN(blink_aarch64_size_b1 = blink_aarch64_size < 10 ? 32 : blink_aarch64_size / 10 % 10 + 48);
HIDDEN(blink_aarch64_size_b2 = blink_aarch64_size < 100 ? 32 : blink_aarch64_size / 100 % 10 + 48);
HIDDEN(blink_aarch64_size_b3 = blink_aarch64_size < 1000 ? 32 : blink_aarch64_size / 1000 % 10 + 48);
HIDDEN(blink_aarch64_size_b4 = blink_aarch64_size < 10000 ? 32 : blink_aarch64_size / 10000 % 10 + 48);
HIDDEN(blink_aarch64_size_b5 = blink_aarch64_size < 100000 ? 32 : blink_aarch64_size / 100000 % 10 + 48);
HIDDEN(blink_aarch64_size_b6 = blink_aarch64_size < 1000000 ? 32 : blink_aarch64_size / 1000000 % 10 + 48);
HIDDEN(blink_aarch64_size_b7 = blink_aarch64_size < 10000000 ? 32 : blink_aarch64_size / 10000000 % 10 + 48);
HIDDEN(blink_aarch64_size_b8 = blink_aarch64_size < 100000000 ? 32 : blink_aarch64_size / 100000000 % 10 + 48);
HIDDEN(blink_aarch64_size_b9 = blink_aarch64_size < 1000000000 ? 32 : blink_aarch64_size / 1000000000 % 10 + 48);
HIDDEN(blink_darwin_arm64_b0 = RVA(blink_darwin_arm64) % 10 + 48);
HIDDEN(blink_darwin_arm64_b1 = RVA(blink_darwin_arm64) < 10 ? 32 : RVA(blink_darwin_arm64) / 10 % 10 + 48);
HIDDEN(blink_darwin_arm64_b2 = RVA(blink_darwin_arm64) < 100 ? 32 : RVA(blink_darwin_arm64) / 100 % 10 + 48);
HIDDEN(blink_darwin_arm64_b3 = RVA(blink_darwin_arm64) < 1000 ? 32 : RVA(blink_darwin_arm64) / 1000 % 10 + 48);
HIDDEN(blink_darwin_arm64_b4 = RVA(blink_darwin_arm64) < 10000 ? 32 : RVA(blink_darwin_arm64) / 10000 % 10 + 48);
HIDDEN(blink_darwin_arm64_b5 = RVA(blink_darwin_arm64) < 100000 ? 32 : RVA(blink_darwin_arm64) / 100000 % 10 + 48);
HIDDEN(blink_darwin_arm64_b6 = RVA(blink_darwin_arm64) < 1000000 ? 32 : RVA(blink_darwin_arm64) / 1000000 % 10 + 48);
HIDDEN(blink_darwin_arm64_b7 = RVA(blink_darwin_arm64) < 10000000 ? 32 : RVA(blink_darwin_arm64) / 10000000 % 10 + 48);
HIDDEN(blink_darwin_arm64_b8 = RVA(blink_darwin_arm64) < 100000000 ? 32 : RVA(blink_darwin_arm64) / 100000000 % 10 + 48);
HIDDEN(blink_darwin_arm64_b9 = RVA(blink_darwin_arm64) < 1000000000 ? 32 : RVA(blink_darwin_arm64) / 1000000000 % 10 + 48);
HIDDEN(blink_darwin_arm64_size_b0 = blink_darwin_arm64_size % 10 + 48);
HIDDEN(blink_darwin_arm64_size_b1 = blink_darwin_arm64_size < 10 ? 32 : blink_darwin_arm64_size / 10 % 10 + 48);
HIDDEN(blink_darwin_arm64_size_b2 = blink_darwin_arm64_size < 100 ? 32 : blink_darwin_arm64_size / 100 % 10 + 48);
HIDDEN(blink_darwin_arm64_size_b3 = blink_darwin_arm64_size < 1000 ? 32 : blink_darwin_arm64_size / 1000 % 10 + 48);
HIDDEN(blink_darwin_arm64_size_b4 = blink_darwin_arm64_size < 10000 ? 32 : blink_darwin_arm64_size / 10000 % 10 + 48);
HIDDEN(blink_darwin_arm64_size_b5 = blink_darwin_arm64_size < 100000 ? 32 : blink_darwin_arm64_size / 100000 % 10 + 48);
HIDDEN(blink_darwin_arm64_size_b6 = blink_darwin_arm64_size < 1000000 ? 32 : blink_darwin_arm64_size / 1000000 % 10 + 48);
HIDDEN(blink_darwin_arm64_size_b7 = blink_darwin_arm64_size < 10000000 ? 32 : blink_darwin_arm64_size / 10000000 % 10 + 48);
HIDDEN(blink_darwin_arm64_size_b8 = blink_darwin_arm64_size < 100000000 ? 32 : blink_darwin_arm64_size / 100000000 % 10 + 48);
HIDDEN(blink_darwin_arm64_size_b9 = blink_darwin_arm64_size < 1000000000 ? 32 : blink_darwin_arm64_size / 1000000000 % 10 + 48);
#if SupportsMetal()
HIDDEN(v_ape_realsectors =
MIN(0x70000 - IMAGE_BASE_REAL, ROUNDUP(RVA(_ezip), 512)) / 512);
HIDDEN(v_ape_realbytes = v_ape_realsectors * 512);
HIDDEN(v_ape_realdwords = v_ape_realsectors * (512 / 4));
HIDDEN(v_ape_allsectors = ROUNDUP(RVA(_ezip), 512) / 512);
HIDDEN(v_ape_allbytes = v_ape_allsectors * 512);
HIDDEN(v_ape_highsectors = MIN(0xffff, v_ape_allsectors - v_ape_realsectors));
v_ape_realsectors = MIN(0x70000 - IMAGE_BASE_REAL, ROUNDUP(RVA(_ezip), 512)) / 512;
v_ape_realbytes = v_ape_realsectors * 512;
v_ape_realdwords = v_ape_realsectors * (512 / 4);
v_ape_allsectors = ROUNDUP(RVA(_ezip), 512) / 512;
v_ape_allbytes = v_ape_allsectors * 512;
v_ape_highsectors = MIN(0xffff, v_ape_allsectors - v_ape_realsectors);
TSSDESCSTUB2(_tss, _tss, _tss_end ? _tss_end - _tss - 1 : 0);
#endif
/* ZIP End of Central Directory header */
#define ZIPCONST(NAME, VAL) HIDDEN(NAME = DEFINED(__zip_start) ? VAL : 0);
ZIPCONST(v_zip_cdoffset, __zip_start - IMAGE_BASE_VIRTUAL);
ZIPCONST(v_zip_cdirsize, __zip_end - __zip_start);
ASSERT(v_zip_cdirsize % kZipCdirHdrLinkableSize == 0, "bad zip cdir");
ZIPCONST(v_zip_records, v_zip_cdirsize / kZipCdirHdrLinkableSize);
ZIPCONST(v_zip_commentsize, _ezip - __zip_end - kZipCdirHdrMinSize);
#if SupportsXnu()
/* Generates deterministic ID. */
#define PHI 0x9e3779b9925d4c17
@@ -648,19 +638,8 @@ ZIPCONST(v_zip_commentsize, _ezip - __zip_end - kZipCdirHdrMinSize);
KMH(ape_uuid1, X); \
XORSHIFT(ape_uuid2, X); \
KMH(ape_uuid2, X)
HIDDEN(ape_uuid1 = 88172645463325252);
HIDDEN(ape_uuid2 = 88172645463325252);
CHURN(ape_bss_align);
CHURN(ape_bss_filesz);
CHURN(ape_bss_memsz);
CHURN(ape_bss_offset);
CHURN(ape_bss_paddr);
CHURN(ape_data_filesz);
CHURN(ape_data_memsz);
CHURN(ape_data_offset);
CHURN(ape_data_paddr);
CHURN(ape_data_rva);
CHURN(ape_data_vaddr);
ape_uuid1 = 88172645463325252;
ape_uuid2 = 88172645463325252;
CHURN(ape_elf_entry);
CHURN(ape_elf_phnum);
CHURN(ape_elf_phoff);
@@ -669,13 +648,10 @@ CHURN(ape_elf_shoff);
CHURN(ape_elf_shstrndx);
CHURN(ape_macho_end);
CHURN(ape_note);
CHURN(ape_note_align);
CHURN(ape_note_end);
CHURN(ape_note_filesz);
CHURN(ape_note_memsz);
CHURN(ape_note_offset);
CHURN(ape_note_paddr);
CHURN(ape_note_vaddr);
CHURN(ape_ram_align);
CHURN(ape_ram_filesz);
CHURN(ape_ram_memsz);
@@ -689,7 +665,14 @@ CHURN(ape_rom_memsz);
CHURN(ape_rom_offset);
CHURN(ape_rom_paddr);
CHURN(ape_rom_rva);
CHURN(ape_rom_vaddr);
CHURN(ape_cod_vaddr);
CHURN(ape_cod_align);
CHURN(ape_cod_filesz);
CHURN(ape_cod_memsz);
CHURN(ape_cod_offset);
CHURN(ape_cod_paddr);
CHURN(ape_cod_rva);
CHURN(ape_cod_vaddr);
CHURN(ape_text_align);
CHURN(ape_text_filesz);
CHURN(ape_text_memsz);
@@ -722,22 +705,22 @@ CHURN(WinMain);
#if SupportsWindows() || SupportsMetal()
#define LINK_WINDOWS (SupportsWindows() && !DEFINED(EfiMain))
PFSTUB4(ape_pe_offset, ape_pe - ape_mz);
HIDDEN(ape_pe_optsz = ape_pe_sections - (ape_pe + 24));
HIDDEN(ape_pe_shnum = (ape_pe_sections_end - ape_pe_sections) / 40);
HIDDEN(ape_pe_base = IMAGE_BASE_VIRTUAL);
HIDDEN(ape_idata = LINK_WINDOWS ? RVA(ape_idata_iat) : 0);
HIDDEN(ape_idata_iatsize = LINK_WINDOWS ? ape_idata_iatend - ape_idata_iat : 0);
HIDDEN(ape_idata = LINK_WINDOWS ? RVA(ape_idata_idt) : 0);
HIDDEN(ape_idata_idtsize = LINK_WINDOWS ? ape_idata_idtend - ape_idata_idt : 0);
HIDDEN(v_ntversion = LINK_WINDOWS ? 6 : 1);
HIDDEN(v_ntdllchar = LINK_WINDOWS ? 288 : 0);
HIDDEN(v_ntsubversion = LINK_WINDOWS ? 6 : 5);
HIDDEN(v_ntsubsystem = (LINK_WINDOWS
? (DEFINED(GetMessage)
? kNtImageSubsystemWindowsGui
: kNtImageSubsystemWindowsCui)
: kNtImageSubsystemEfiApplication));
HIDDEN(ape_pe_entry = LINK_WINDOWS ? WinMain : EfiMain);
ape_pe_optsz = ape_pe_sections - (ape_pe + 24);
ASSERT(ape_pe_optsz % 8 == 0, "SizeOfOptionalHeader must be multiple of 8");
ape_pe_shnum = (ape_pe_sections_end - ape_pe_sections) / 40;
ape_pe_base = IMAGE_BASE_VIRTUAL;
ape_idataz = LINK_WINDOWS ? RVA(ape_idata_iat) : 0;
ape_idata_iatsize = LINK_WINDOWS ? ape_idata_iatend - ape_idata_iat : 0;
ape_idata = LINK_WINDOWS ? RVA(ape_idata_idt) : 0;
ape_idata_idtsize = LINK_WINDOWS ? ape_idata_idtend - ape_idata_idt : 0;
v_ntdllchar = LINK_WINDOWS ? 288 : 0;
v_ntsubversion = LINK_WINDOWS ? 6 : 5;
v_ntsubsystem = (LINK_WINDOWS
? (DEFINED(GetMessage)
? kNtImageSubsystemWindowsGui
: kNtImageSubsystemWindowsCui)
: kNtImageSubsystemEfiApplication);
ape_pe_entry = LINK_WINDOWS ? WinMain : EfiMain;
#endif
#if SupportsXnu()
@@ -755,12 +738,6 @@ ASSERT(((DEFINED(__init_rodata_end) ? __init_rodata_end : 0) %
ASSERT((!DEFINED(ape_grub) ? 1 : RVA(ape_grub) < 8192),
"grub stub needs to be in first 8kb of image");
ASSERT(DEFINED(_start) || DEFINED(_start16),
"please link a _start() or _start16() entrypoint");
ASSERT(!DEFINED(_start16) || REAL(_end) < 65536,
"ape won't support non-tiny real mode programs");
ASSERT(IS2POW(ape_stack_memsz),
"ape_stack_memsz must be a two power");
@@ -770,6 +747,8 @@ ASSERT(ape_stack_vaddr % ape_stack_memsz == 0,
ASSERT(ALIGNOF(.tdata) <= TLS_ALIGNMENT && ALIGNOF(.tbss) <= TLS_ALIGNMENT,
"_Thread_local _Alignof can't exceed TLS_ALIGNMENT");
ASSERT(DEFINED(main), "main() function not defined");
/* Let's not be like Knight Capital. */
/* NOCROSSREFS_TO(.test .text) */
-196
View File
@@ -1,196 +0,0 @@
#-*-mode:makefile-gmake;indent-tabs-mode:t;tab-width:8;coding:utf-8-*-┐
#───vi: set et ft=make ts=8 tw=8 fenc=utf-8 :vi───────────────────────┘
#
# OVERVIEW
#
# αcτµαlly pδrταblε εxεcµταblε
#
# DESCRIPTION
#
# This file defines the libraries, runtimes, and build rules needed to
# create executables from your Linux workstation that'll run anywhere.
# Loading this package will make certain systemic modifications to the
# build like turning off the System V "Red Zone" optimization, because
# αcτµαlly pδrταblε εxεcµταblεs need to be able to run in kernelspace.
PKGS += APE
APE = o/$(MODE)/ape/ape.o \
o/$(MODE)/ape/ape.lds
APE_NO_MODIFY_SELF = \
o/$(MODE)/ape/ape.lds \
o/$(MODE)/ape/ape-no-modify-self.o
APE_COPY_SELF = \
o/$(MODE)/ape/ape.lds \
o/$(MODE)/ape/ape-copy-self.o
APELINK = \
$(COMPILE) \
-ALINK.ape \
$(LINK) \
$(LINKARGS) \
$(OUTPUT_OPTION)
APE_LOADER_FLAGS = \
-DNDEBUG \
-iquote. \
-Wall \
-Wextra \
-fpie \
-Os \
-ffreestanding \
-mgeneral-regs-only \
-mno-red-zone \
-fno-ident \
-fno-gnu-unique \
-c \
$(OUTPUT_OPTION) \
$<
APE_FILES := $(wildcard ape/*.*)
APE_HDRS = $(filter %.h,$(APE_FILES))
APE_INCS = $(filter %.inc,$(APE_FILES))
APE_SRCS_C = ape/loader.c
APE_SRCS_S = $(filter %.S,$(APE_FILES))
APE_SRCS = $(APE_SRCS_C) $(APE_SRCS_S)
APE_OBJS = $(APE_SRCS_S:%.S=o/$(MODE)/%.o)
APE_CHECKS = $(APE_HDRS:%=o/%.ok)
o/$(MODE)/ape/ape.lds: \
ape/ape.lds \
ape/macros.internal.h \
ape/relocations.h \
libc/intrin/bits.h \
libc/thread/tls.h \
libc/calls/struct/timespec.h \
libc/thread/thread.h \
libc/dce.h \
libc/elf/def.h \
libc/elf/pf2prot.internal.h \
libc/macros.internal.h \
libc/nt/pedef.internal.h \
libc/str/str.h \
libc/zip.h
o/$(MODE)/ape/public/ape.lds: OVERRIDE_CPPFLAGS += -UCOSMO
o/$(MODE)/ape/public/ape.lds: \
ape/public/ape.lds \
ape/ape.lds \
ape/macros.internal.h \
ape/relocations.h \
libc/intrin/bits.h \
libc/thread/tls.h \
libc/calls/struct/timespec.h \
libc/thread/thread.h \
libc/dce.h \
libc/elf/def.h \
libc/elf/pf2prot.internal.h \
libc/macros.internal.h \
libc/nt/pedef.internal.h \
libc/str/str.h \
libc/zip.h
o/ape/idata.inc: \
ape/idata.internal.h \
ape/relocations.h
o/$(MODE)/ape/ape.o: \
ape/blink-aarch64.gz \
ape/blink-darwin-arm64.gz
o/$(MODE)/ape/ape-no-modify-self.o: \
ape/ape.S \
ape/macros.internal.h \
ape/notice.inc \
ape/relocations.h \
libc/dce.h \
libc/elf/def.h \
libc/intrin/asancodes.h \
libc/macho.internal.h \
libc/macros.internal.h \
libc/nexgen32e/uart.internal.h \
libc/calls/metalfile.internal.h \
libc/nexgen32e/vidya.internal.h \
libc/nt/pedef.internal.h \
libc/runtime/e820.internal.h \
libc/runtime/mman.internal.h \
libc/runtime/pc.internal.h \
libc/runtime/symbolic.h \
libc/sysv/consts/prot.h \
ape/blink-aarch64.gz \
ape/blink-darwin-arm64.gz \
o/$(MODE)/ape/ape.elf
@$(COMPILE) \
-AOBJECTIFY.S \
$(OBJECTIFY.S) \
$(OUTPUT_OPTION) \
-DAPE_NO_MODIFY_SELF \
-DAPE_LOADER='"o/$(MODE)/ape/ape.elf"' $<
o/$(MODE)/ape/ape-copy-self.o: \
ape/ape.S \
ape/macros.internal.h \
ape/notice.inc \
ape/relocations.h \
libc/dce.h \
libc/elf/def.h \
libc/intrin/asancodes.h \
libc/macho.internal.h \
libc/macros.internal.h \
libc/nexgen32e/uart.internal.h \
libc/calls/metalfile.internal.h \
libc/nexgen32e/vidya.internal.h \
libc/nt/pedef.internal.h \
libc/runtime/e820.internal.h \
libc/runtime/mman.internal.h \
libc/runtime/pc.internal.h \
libc/runtime/symbolic.h \
libc/sysv/consts/prot.h \
ape/blink-aarch64.gz \
ape/blink-darwin-arm64.gz
@$(COMPILE) \
-AOBJECTIFY.S \
$(OBJECTIFY.S) \
$(OUTPUT_OPTION) \
-DAPE_NO_MODIFY_SELF $<
o/$(MODE)/ape/loader.o: ape/loader.c ape/loader.h
@$(COMPILE) -AOBJECTIFY.c $(CC) -DSUPPORT_VECTOR=0b01111001 -g $(APE_LOADER_FLAGS)
o/$(MODE)/ape/loader-gcc.asm: ape/loader.c
@$(COMPILE) -AOBJECTIFY.c $(CC) -DSUPPORT_VECTOR=0b01111001 -S -g0 $(APE_LOADER_FLAGS)
o/$(MODE)/ape/loader-clang.asm: ape/loader.c
@$(COMPILE) -AOBJECTIFY.c $(CLANG) -DSUPPORT_VECTOR=0b01111001 -S -g0 $(APE_LOADER_FLAGS)
o/$(MODE)/ape/loader-xnu.o: ape/loader.c ape/loader.h
@$(COMPILE) -AOBJECTIFY.c $(CC) -DSUPPORT_VECTOR=0b00001000 -g $(APE_LOADER_FLAGS)
o/$(MODE)/ape/loader-xnu-gcc.asm: ape/loader.c
@$(COMPILE) -AOBJECTIFY.c $(CC) -DSUPPORT_VECTOR=0b00001000 -S -g0 $(APE_LOADER_FLAGS)
o/$(MODE)/ape/loader-xnu-clang.asm: ape/loader.c
@$(COMPILE) -AOBJECTIFY.c $(CLANG) -DSUPPORT_VECTOR=0b00001000 -S -g0 $(APE_LOADER_FLAGS)
o/$(MODE)/ape/ape.elf: o/$(MODE)/ape/ape.elf.dbg
o/$(MODE)/ape/ape.macho: o/$(MODE)/ape/ape.macho.dbg
o/$(MODE)/ape/ape.elf.dbg: \
o/$(MODE)/ape/loader.o \
o/$(MODE)/ape/loader-elf.o \
ape/loader.lds
@$(ELFLINK) -z max-page-size=0x10
o/$(MODE)/ape/ape.macho.dbg: \
o/$(MODE)/ape/loader-xnu.o \
o/$(MODE)/ape/loader-macho.o \
ape/loader-macho.lds
@$(ELFLINK) -z max-page-size=0x10
.PHONY: o/$(MODE)/ape
o/$(MODE)/ape: $(APE_CHECKS) \
o/$(MODE)/ape/ape.o \
o/$(MODE)/ape/ape.lds \
o/$(MODE)/ape/public/ape.lds \
o/$(MODE)/ape/ape.elf \
o/$(MODE)/ape/ape.macho \
o/$(MODE)/ape/ape-copy-self.o \
o/$(MODE)/ape/ape-no-modify-self.o
+91 -39
View File
@@ -1,62 +1,107 @@
#!/bin/sh
PROG=${0##*/}
MODE=${MODE:-$m}
TMPDIR=${TMPDIR:-/tmp}
COSMO=${COSMO:-/opt/cosmo}
COSMOS=${COSMOS:-/opt/cosmos}
if [ ! -f ape/loader.c ]; then
cd "$COSMO" || exit
fi
if [ -x build/bootstrap/make ]; then
MAKE=build/bootstrap/make
else
MAKE=make
fi
if [ "$(id -u)" -eq 0 ]; then
SUDO=
else
elif command -v sudo >/dev/null 2>&1; then
SUDO=sudo
elif command -v doas >/dev/null 2>&1; then
SUDO=doas
else
echo "need root or sudo" >&2
exit
fi
if command -v install >/dev/null 2>&1; then
if [ x"$(uname -s)" = xLinux ]; then
INSTALL="install -o root -g root -m 755"
else
INSTALL="install -o root -g wheel -m 755"
fi
else
INSTALL="cp -f"
fi
echo "Actually Portable Executable (APE) Installer" >&2
echo "Author: Justine Tunney <[email protected]>" >&2
# special installation process for apple silicon
if [ x"$(uname -s)" = xDarwin ] && [ x"$(uname -m)" = xarm64 ]; then
echo "cc -O -o $TMPDIR/ape.$$ ape/ape-m1.c" >&2
cc -O -o "$TMPDIR/ape.$$" ape/ape-m1.c || exit
trap 'rm "$TMPDIR/ape.$$"' EXIT
if [ ! -d /usr/local/bin ]; then
echo "$SUDO mkdir -p /usr/local/bin" >&2
$SUDO mkdir -p /usr/local/bin || exit
fi
echo "$SUDO $INSTALL $TMPDIR/ape.$$ /usr/local/bin/ape" >&2
$SUDO $INSTALL "$TMPDIR/ape.$$" /usr/local/bin/ape || exit
exit
fi
if [ x"$(uname -m)" = xarm64 ] || [ x"$(uname -m)" = xaarch64 ]; then
MODE=aarch64
EXT=elf
BEXT=aarch64
elif [ x"$(uname -m)" = xx86_64 ]; then
MODE=
if [ x"$(uname -s)" = xDarwin ]; then
EXT=macho
else
EXT=elf
fi
BEXT=$EXT
else
echo "unsupported architecture $(uname -m)" >&2
exit
fi
################################################################################
# INSTALL APE LOADER SYSTEMWIDE
if [ -f o/depend ]; then
# mkdeps.com build was successfully run so assume we can build
echo >&2
echo "recompiling ape loader" >&2
echo "running: make -j8 o//ape" >&2
make -j8 o//ape || exit
echo "done" >&2
if [ -f o/$MODE/depend ] && $MAKE -j8 o/$MODE/ape; then
echo "successfully recompiled ape loader" >&2
elif [ -x o/$MODE/ape/ape.$EXT ]; then
echo "using ape loader you compiled earlier" >&2
elif [ -d build/bootstrap ]; then
# if make isn't being used then it's unlikely the user changed the sources
# in that case the prebuilt binaries should be completely up-to-date
echo "using prebuilt ape loader from cosmo repo" >&2
mkdir -p o//ape || exit
cp -af build/bootstrap/ape.elf o//ape/ape.elf || exit
cp -af build/bootstrap/ape.macho o//ape/ape.macho || exit
mkdir -p o/$MODE/ape || exit
cp -af build/bootstrap/ape.$BEXT o/$MODE/ape/ape.$EXT || exit
else
echo "no cosmopolitan libc repository here" >&2
echo "fetching ape loader from justine.lol" >&2
mkdir -p o//ape || exit
echo "fetching ape loader from justine.lol" >&2
mkdir -p o/$MODE/ape || exit
if command -v wget >/dev/null 2>&1; then
wget -qO o//ape/ape.elf https://justine.lol/ape.elf || exit
wget -qO o//ape/ape.macho https://justine.lol/ape.macho || exit
wget -qO o/$MODE/ape/ape.$EXT https://justine.lol/ape.$BEXT || exit
else
curl -Rso o//ape/ape.elf https://justine.lol/ape.elf || exit
curl -Rso o//ape/ape.macho https://justine.lol/ape.macho || exit
curl -Rso o/$MODE/ape/ape.$EXT https://justine.lol/ape.$BEXT || exit
fi
chmod +x o//ape/ape.elf || exit
chmod +x o//ape/ape.macho || exit
chmod +x o/$MODE/ape/ape.$EXT || exit
fi
if [ "$(uname -s)" = "Darwin" ]; then
if ! [ /usr/bin/ape -nt o//ape/ape.macho ]; then
echo >&2
echo "installing o//ape/ape.elf to /usr/bin/ape" >&2
echo "$SUDO mv -f o//ape/ape.elf /usr/bin/ape" >&2
$SUDO cp -f o//ape/ape.macho /usr/bin/ape || exit
echo "done" >&2
fi
else
if ! [ /usr/bin/ape -nt o//ape/ape.elf ]; then
echo >&2
echo "installing o//ape/ape.elf to /usr/bin/ape" >&2
echo "$SUDO mv -f o//ape/ape.elf /usr/bin/ape" >&2
$SUDO cp -f o//ape/ape.elf /usr/bin/ape || exit
echo "done" >&2
fi
if ! [ /usr/bin/ape -nt o/$MODE/ape/ape.$EXT ]; then
echo >&2
echo "installing o/$MODE/ape/ape.$EXT to /usr/bin/ape" >&2
echo "$SUDO $INSTALL o/$MODE/ape/ape.$EXT /usr/bin/ape" >&2
$SUDO $INSTALL o/$MODE/ape/ape.$EXT /usr/bin/ape || exit
echo "done" >&2
fi
################################################################################
@@ -92,13 +137,20 @@ if [ x"$(uname -s)" = xLinux ]; then
echo done >&2
fi
uname_r="$(uname -r)"
if printf '%s\n%s\n' 5.12 "$uname_r" | sort -CV; then
FLAGS=FP
else
FLAGS=F
fi
echo >&2
echo registering APE with binfmt_misc >&2
echo you may need to edit configs to persist across reboot >&2
echo '$SUDO sh -c "echo '"'"':APE:M::MZqFpD::/usr/bin/ape:'"'"' >/proc/sys/fs/binfmt_misc/register"' >&2
$SUDO sh -c "echo ':APE:M::MZqFpD::/usr/bin/ape:' >/proc/sys/fs/binfmt_misc/register" || exit
echo '$SUDO sh -c "echo '"'"':APE-jart:M::jartsr::/usr/bin/ape:'"'"' >/proc/sys/fs/binfmt_misc/register"' >&2
$SUDO sh -c "echo ':APE-jart:M::jartsr::/usr/bin/ape:' >/proc/sys/fs/binfmt_misc/register" || exit
echo '$SUDO sh -c "echo '"'"':APE:M::MZqFpD::/usr/bin/ape:'"$FLAGS'"' >/proc/sys/fs/binfmt_misc/register"' >&2
$SUDO sh -c "echo ':APE:M::MZqFpD::/usr/bin/ape:$FLAGS' >/proc/sys/fs/binfmt_misc/register" || exit
echo '$SUDO sh -c "echo '"'"':APE-jart:M::jartsr::/usr/bin/ape:'"$FLAGS'"' >/proc/sys/fs/binfmt_misc/register"' >&2
$SUDO sh -c "echo ':APE-jart:M::jartsr::/usr/bin/ape:$FLAGS' >/proc/sys/fs/binfmt_misc/register" || exit
echo done >&2
if [ x"$(cat /proc/sys/fs/binfmt_misc/status)" = xdisabled ]; then
+45 -3
View File
@@ -1,9 +1,23 @@
#!/bin/sh
PROG=${0##*/}
MODE=${MODE:-$m}
COSMO=${COSMO:-/opt/cosmo}
COSMOS=${COSMOS:-/opt/cosmos}
if [ ! -f ape/loader.c ]; then
cd "$COSMO" || exit
fi
if [ "$UID" = "0" ]; then
SUDO=
else
elif command -v sudo >/dev/null 2>&1; then
SUDO=sudo
elif command -v doas >/dev/null 2>&1; then
SUDO=doas
else
echo "need root or sudo" >&2
exit
fi
{
@@ -11,7 +25,7 @@ fi
echo "APE Uninstaller intends to run (in pseudo-shell)"
echo
echo " sudo echo -1 into /proc/sys/fs/binfmt_misc/APE*"
echo " sudo rm -f /usr/bin/ape ~/.ape o/tmp/.ape /tmp/.ape"
echo " sudo rm -f /usr/bin/ape ~/.ape /tmp/.ape # etc."
echo
echo "You may then use ape/apeinstall.sh to reinstall it"
echo
@@ -23,4 +37,32 @@ for f in /proc/sys/fs/binfmt_misc/APE*; do
$SUDO sh -c "echo -1 >$f" || exit
fi
done
$SUDO rm -f /usr/bin/ape ~/.ape o/tmp/.ape o/tmp/ape /tmp/.ape /tmp/ape || exit
# system installation
if [ -f /usr/bin/ape ]; then
$SUDO rm -f /usr/bin/ape
fi
if [ -f /usr/local/bin/ape ]; then
$SUDO rm -f /usr/local/bin/ape
fi
# legacy installations
rm -f o/tmp/ape /tmp/ape "${TMPDIR:-/tmp}/ape"
# ad-hoc installations
for x in .ape \
.ape-1.1 \
.ape-1.3 \
.ape-1.4 \
.ape-1.5 \
.ape-1.6 \
.ape-1.7 \
.ape-1.8 \
.ape-1.9 \
.ape-1.10; do
rm -f \
~/$x \
/tmp/$x \
o/tmp/$x \
"${TMPDIR:-/tmp}/$x"
done
Binary file not shown.
Binary file not shown.
-10
View File
@@ -1,10 +0,0 @@
#ifndef COSMOPOLITAN_APE_CONFIG_H_
#define COSMOPOLITAN_APE_CONFIG_H_
#if !(__ASSEMBLER__ + __LINKER__ + 0)
#define STATIC_SYMBOL(NAME, VALUE) \
asm(".equ\t" NAME "," VALUE "\n\t" \
".globl\t" NAME)
#endif /* !(__ASSEMBLER__ + __LINKER__ + 0) */
#endif /* COSMOPOLITAN_APE_CONFIG_H_ */
+23 -19
View File
@@ -1,5 +1,5 @@
/*-*- mode:unix-assembly; indent-tabs-mode:t; tab-width:8; coding:utf-8 -*-│
│vi: set et ft=asm ts=8 sw=8 fenc=utf-8 :vi│
│ vi: set noet ft=asm ts=8 sw=8 fenc=utf-8 :vi │
╞══════════════════════════════════════════════════════════════════════════════╡
│ Copyright 2020 Justine Alexandra Roberts Tunney │
│ │
@@ -31,7 +31,8 @@
// @see libc/nt/master.sh
// @see ape/ape.lds
// @see winimp
.macro .imp dll:req fn:req actual:req hint
.macro .imp dll:req fn:req actual:req
#ifdef __x86_64__
.dll "\dll"
.section ".piro.data.sort.iat.2.\dll\().2.\actual","aw",@progbits
.type \fn,@object
@@ -42,25 +43,28 @@
.hidden \fn
.previous
.section ".idata.ro.ilt.\dll\().2.\actual","a",@progbits
".Lidata.ilt.\dll\().\actual":
"idata.ilt.\dll\().\actual":
.quad RVA("\dll\().\actual")
.type ".Lidata.ilt.\dll\().\actual",@object
.size ".Lidata.ilt.\dll\().\actual",.-".Lidata.ilt.\dll\().\actual"
.type "idata.ilt.\dll\().\actual",@object
.size "idata.ilt.\dll\().\actual",.-"idata.ilt.\dll\().\actual"
.previous
.section ".idata.ro.hnt.\dll\().2.\actual","a",@progbits
"\dll\().\actual":
.ifnb \hint # hint i.e. guess function ordinal
.short \hint
.else
.short 0
.endif
.short 0 // hint
.asciz "\actual"
.align 2 # documented requirement
.align 2 // documented requirement
.globl "\dll\().\actual"
.hidden "\dll\().\actual"
.type "\dll\().\actual",@object
.size "\dll\().\actual",.-"\dll\().\actual"
.previous
#else
.section ".data.nt.\actual","aw",@progbits
.globl "\fn"
.balign 8
.weak "\actual"
"\fn": .quad "\actual"
#endif
.endm
// Defines DLL import.
@@ -68,16 +72,16 @@
.macro .dll name:req
.section ".idata.ro.idt.2.\name","aG",@progbits,"\name",comdat
.equ ".Lidata.idt.\name",.
.long RVA("idata.ilt.\name") # ImportLookupTable
.long 0 # TimeDateStamp
.long 0 # ForwarderChain
.long RVA(".Lidata.str.\name") # DllNameRva
.long RVA("idata.iat.\name") # ImportAddressTable
.long RVA("idata.ilt.\name") // ImportLookupTable
.long 0 // TimeDateStamp
.long 0 // ForwarderChain
.long RVA(".Lidata.str.\name") // DllNameRva
.long RVA("idata.iat.\name") // ImportAddressTable
.type ".Lidata.idt.\name",@object
.size ".Lidata.idt.\name",.-".Lidata.idt.\name"
.previous
.section ".idata.ro.ilt.\name\().1","aG",@progbits,"\name",comdat
.align __SIZEOF_POINTER__
.balign __SIZEOF_POINTER__
.type "idata.ilt.\name",@object
"idata.ilt.\name":
.previous/*
@@ -88,12 +92,12 @@
.quad 0
.previous
.section ".idata.ro.hnt.\name\().1","aG",@progbits,"\name",comdat
.align __SIZEOF_POINTER__
.balign __SIZEOF_POINTER__
.type "idata.hnt.\name",@object
.equ "idata.hnt.\name",.
.previous
.section ".piro.data.sort.iat.2.\name\().1","awG",@progbits,"\name",comdat
.align __SIZEOF_POINTER__
.balign __SIZEOF_POINTER__
.type "idata.iat.\name",@object
"idata.iat.\name":
.previous/*
+90
View File
@@ -0,0 +1,90 @@
/*-*- mode:unix-assembly; indent-tabs-mode:t; tab-width:8; coding:utf-8 -*-│
│ vi: set noet ft=asm ts=8 sw=8 fenc=utf-8 :vi │
╞══════════════════════════════════════════════════════════════════════════════╡
│ Copyright 2023 Justine Alexandra Roberts Tunney │
│ │
│ Permission to use, copy, modify, and/or distribute this software for │
│ any purpose with or without fee is hereby granted, provided that the │
│ above copyright notice and this permission notice appear in all copies. │
│ │
│ THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL │
│ WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED │
│ WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE │
│ AUTHOR BE LIABLE FOR ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL │
│ DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR │
│ PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER │
│ TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR │
│ PERFORMANCE OF THIS SOFTWARE. │
╚─────────────────────────────────────────────────────────────────────────────*/
#include "libc/macros.internal.h"
// Calls _start() function of loaded program.
//
// When the program entrypoint is called, all registers shall be
// cleared, with the exception of (1) %rdi will be equal to %rsp
// on FreeBSD and (2) %cl will contain the detected host OS code
//
// We clear all the general registers we can to have some wiggle
// room, to extend the behavior of this loader in the future. We
// don't need to clear the XMM registers because your APE loader
// should be compiled using gcc/clang's -mgeneral-regs-only flag
//
// @param rdi is passed through as-is
// @param rsi is address of entrypoint (becomes zero)
// @param rdx is passed through as-is
// @param rcx is passed through as-is
// @param r8 is stack pointer (becomes zero)
// @noreturn
Launch:
#ifdef __aarch64__
mov x16,x1
mov sp,x4
mov x1,0
mov x4,0
mov x5,0
mov x6,0
mov x7,0
mov x8,0
mov x9,0
mov x10,0
mov x11,0
mov x12,0
mov x13,0
mov x14,0
mov x15,0
mov x17,0
mov x19,0
mov x20,0
mov x21,0
mov x22,0
mov x23,0
mov x24,0
mov x25,0
mov x26,0
mov x27,0
mov x28,0
mov x29,0
mov x30,0
br x16
#else
mov %r8,%rsp
xor %r8d,%r8d
xor %r9d,%r9d
xor %r10d,%r10d
xor %r11d,%r11d
xor %r12d,%r12d
xor %r13d,%r13d
xor %r14d,%r14d
xor %r15d,%r15d
push %rsi
xor %esi,%esi
xor %ebp,%ebp
xor %ebx,%ebx
xor %eax,%eax
ret
#endif
.endfn Launch,globl
-249
View File
@@ -1,249 +0,0 @@
/*-*- mode:unix-assembly; indent-tabs-mode:t; tab-width:8; coding:utf-8 -*-│
│vi: set et ft=asm ts=8 tw=8 fenc=utf-8 :vi│
╞══════════════════════════════════════════════════════════════════════════════╡
│ Copyright 2021 Justine Alexandra Roberts Tunney │
│ │
│ Permission to use, copy, modify, and/or distribute this software for │
│ any purpose with or without fee is hereby granted, provided that the │
│ above copyright notice and this permission notice appear in all copies. │
│ │
│ THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL │
│ WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED │
│ WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE │
│ AUTHOR BE LIABLE FOR ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL │
│ DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR │
│ PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER │
│ TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR │
│ PERFORMANCE OF THIS SOFTWARE. │
╚─────────────────────────────────────────────────────────────────────────────*/
#include "libc/elf/def.h"
#include "libc/sysv/consts/prot.h"
#include "libc/macho.internal.h"
#include "libc/dce.h"
#include "libc/macros.internal.h"
// APE Loader Executable Structure
// Linux, FreeBSD, NetBSD, OpenBSD
.align 8
ehdr: .ascii "\177ELF"
.byte ELFCLASS64
.byte ELFDATA2LSB
.byte 1
.byte ELFOSABI_FREEBSD
.quad 0
.word ET_EXEC # e_type
.word EM_NEXGEN32E # e_machine
.long 1 # e_version
.quad _start # e_entry
.quad phdrs - ehdr # e_phoff
.quad 0 # e_shoff
.long 0 # e_flags
.word 64 # e_ehsize
.word 56 # e_phentsize
.word 4 # e_phnum
.word 0 # e_shentsize
.word 0 # e_shnum
.word 0 # e_shstrndx
.endobj ehdr,globl
.align 8
phdrs: .long PT_LOAD # p_type
.long PF_R|PF_X # p_flags
.quad 0 # p_offset
.quad ehdr # p_vaddr
.quad ehdr # p_paddr
.quad filesz # p_filesz
.quad filesz # p_memsz
.quad PAGESIZE # p_align
.long PT_LOAD # p_type
.long PF_R|PF_W # p_flags
.quad 0 # p_offset
.quad bss # p_vaddr
.quad bss # p_paddr
.quad 0 # p_filesz
.quad bsssize # p_memsz
.quad PAGESIZE # p_align
.long PT_GNU_STACK # p_type
.long PF_R|PF_W # p_flags
.quad 0 # p_offset
.quad 0 # p_vaddr
.quad 0 # p_paddr
.quad 0 # p_filesz
.quad 0 # p_memsz
.quad 16 # p_align
.long PT_NOTE # p_type
.long PF_R # p_flags
.quad note - ehdr # p_offset
.quad note # p_vaddr
.quad note # p_paddr
.quad notesize # p_filesz
.quad notesize # p_memsz
.quad 8 # p_align
.endobj phdrs
note: .long 2f-1f
.long 4f-3f
.long 1
1: .asciz "OpenBSD"
2: .align 4
3: .long 0
4: .long 2f-1f
.long 4f-3f
.long 1
1: .asciz "NetBSD"
2: .align 4
3: .long 901000000
4: .endobj note
notesize = . - note
.align 64,0 # for ape.S dd
.org 0x180 # for ape.S dd
// APE Loader XNU Header
//
// This header is dd'd backwards by the APE shell script when
// running on Mac OS X.
//
// @see ape/ape.S
macho: .long 0xFEEDFACE+1
.long MAC_CPU_NEXGEN32E
.long MAC_CPU_NEXGEN32E_ALL
.long MAC_EXECUTE
.long 5 # number of load commands
.long 60f-10f # size of all load commands
.long MAC_NOUNDEFS # flags
.long 0 # reserved
10: .long MAC_LC_SEGMENT_64
.long 20f-10b # unmaps first page dir
.ascin "__PAGEZERO",16 # consistent with linux
.quad 0,0x200000,0,0 # which forbids mem <2m
.long 0,0,0,0
20: .long MAC_LC_SEGMENT_64
.long 30f-20b
.ascin "__TEXT",16
.quad ehdr # vaddr
.quad 4096 # memsz
.quad 0 # file offset
.quad filesz # file size
.long PROT_EXEC|PROT_READ|PROT_WRITE # maxprot
.long PROT_EXEC|PROT_READ # initprot
.long 1 # segment section count
.long 0 # flags
210: .ascin "__text",16 # section name (.text)
.ascin "__TEXT",16
.quad _start # vaddr
.quad textsz # memsz
.long textoff # offset
.long 6 # align 2**6 = 64
.long 0 # reloc table offset
.long 0 # relocation count
.long MAC_S_ATTR_SOME_INSTRUCTIONS # section type & attributes
.long 0,0,0 # reserved
30: .long MAC_LC_SEGMENT_64
.long 40f-30b
.ascin "__DATA",16
.quad bss # vaddr
.quad bsssize # memsz
.quad 0 # offset
.quad 0 # file size
.long PROT_EXEC|PROT_READ|PROT_WRITE # maxprot
.long PROT_READ|PROT_WRITE # initprot
.long 1 # segment section count
.long 0 # flags
310: .ascin "__bss",16 # section name (.bss)
.ascin "__DATA",16
.quad bss # vaddr
.quad bsssize # memsz
.long 0 # offset
.long 12 # align 2**12 = 4096
.long 0 # reloc table offset
.long 0 # relocation count
.long MAC_S_ZEROFILL # section type & attributes
.long 0,0,0 # reserved
40: .long MAC_LC_UUID
.long 50f-40b
.quad 0x3fb29ee4ac6c87aa # uuid1
.quad 0xdd2c9bb866d9eef8 # uuid2
50: .long MAC_LC_UNIXTHREAD
.long 60f-50b # cmdsize
.long MAC_THREAD_NEXGEN32E # flavaflav
.long (520f-510f)/4 # count
510: .quad 0 # rax
.quad 0 # rbx
.quad 0 # rcx
.quad _HOSTXNU # rdx
.quad 0 # rdi
.quad 0 # rsi
.quad 0 # rbp
.quad 0 # rsp
.quad 0 # r8
.quad 0 # r9
.quad 0 # r10
.quad 0 # r11
.quad 0 # r12
.quad 0 # r13
.quad 0 # r14
.quad 0 # r15
.quad _start # rip
.quad 0 # rflags
.quad 0 # cs
.quad 0 # fs
.quad 0 # gs
520:
60:
.endobj macho
.align 64,0 # for ape.S dd
.org 0x400 # for ape.S dd
// Ape Loader Entrpoint
//
// This is normally called by the operating system. However it may
// be called by the Actually Portable Executables themselves, when
// re-executing a program. Just do this:
//
// memcpy(0x200000, loader)
// lea handoff(%rip),%rcx
// lea argblock(%rip),%rsp
// jmp 0x200400
//
// @see APE_LOADER_ENTRY
// @see ape/loader.h
_start:
// Hack for detecting M1 Rosetta environment.
// https://github.com/jart/cosmopolitan/issues/429#issuecomment-1166704377
cmp $-1,%ebx
jne 0f
cmp $+1,%edx
jne 0f
mov $_HOSTXNU,%dl
xor %ecx,%ecx
0: mov %rsp,%rsi
jmp ApeLoader
.endfn _start,globl
// System Call Entrpoint
//
// This function is used by the APE loader to make system calls.
// We also pass a reference to this function to the APE binary's
// _start() function. It's needed because on OpenBSD, msyscall()
// restricts which pages can issue system calls, and it can only
// be called once. Therefore if we want to be load and re-load a
// binary multiple times without calling the system execve(), we
// need to be able to handover the SYSCALL function. We hardcode
// this to a fixed address, but that shouldn't be used, since we
// would ideally want to move it to a random page in the future.
__syscall_loader:
clc
syscall
jc 1f
ret
1: neg %rax
ret
.endfn __syscall_loader,globl
+57 -100
View File
@@ -1,5 +1,5 @@
/*-*- mode:unix-assembly; indent-tabs-mode:t; tab-width:8; coding:utf-8 -*-│
│vi: set et ft=asm ts=8 tw=8 fenc=utf-8 :vi│
│ vi: set noet ft=asm ts=8 sw=8 fenc=utf-8 :vi │
╞══════════════════════════════════════════════════════════════════════════════╡
│ Copyright 2021 Justine Alexandra Roberts Tunney │
│ │
@@ -21,118 +21,75 @@
#include "libc/dce.h"
#include "libc/macros.internal.h"
// APE Loader Executable Structure for XNU
// Apple Mach-O Executable Headers
// Fixups are applied by objbincopy
// There must exist a MAC_LC_SEGMENT_64 for every PT_LOAD
.align 4096
macho: .long 0xFEEDFACE+1
.section .macho,"a",@progbits
.balign 64
.long 0xFEEDFACE+1
.long MAC_CPU_NEXGEN32E
.long MAC_CPU_NEXGEN32E_ALL
.long MAC_EXECUTE
.long 5 # number of load commands
.long 60f-10f # size of all load commands
.long MAC_NOUNDEFS # flags
.long 0 # reserved
.long 5 // number of load commands
.long 60f-10f // size of all load commands
.long MAC_NOUNDEFS|MAC_SPLIT_SEGS // flags
.long 0 // reserved
10: .long MAC_LC_SEGMENT_64
.long 20f-10b # unmaps first page dir
.ascin "__PAGEZERO",16 # consistent with linux
.quad 0,0x200000,0,0 # which forbids mem <2m
.long 20f-10b // unmaps first page dir
.ascin "__PAGEZERO",16 // consistent with linux
.quad 0,0x200000,0,0 // which forbids mem <2m
.long 0,0,0,0
20: .long MAC_LC_SEGMENT_64
.long 30f-20b
.ascin "__TEXT",16
.quad macho # vaddr
.quad 4096 # memsz
.quad 0 # file offset
.quad filesz # file size
.long PROT_EXEC|PROT_READ|PROT_WRITE # maxprot
.long PROT_EXEC|PROT_READ # initprot
.long 1 # segment section count
.long 0 # flags
210: .ascin "__text",16 # section name (.text)
.ascin "__TEXT",16
.quad _start # vaddr
.quad textsz # memsz
.long textoff # offset
.long 6 # align 2**3 = 64
.long 0 # reloc table offset
.long 0 # relocation count
.long MAC_S_ATTR_SOME_INSTRUCTIONS # section type & attributes
.long 0,0,0 # reserved
.quad 0 // vaddr
.quad 0 // memsz
.quad 0 // file offset
.quad 0 // file size
.long 0 // maxprot
.long 0 // initprot
.long 0 // segment section count
.long 0 // flags
30: .long MAC_LC_SEGMENT_64
.long 40f-30b
.ascin "__DATA",16
.quad bss # vaddr
.quad bsssize # memsz
.quad 0 # offset
.quad 0 # file size
.long PROT_EXEC|PROT_READ|PROT_WRITE # maxprot
.long PROT_READ|PROT_WRITE # initprot
.long 1 # segment section count
.long 0 # flags
310: .ascin "__bss",16 # section name (.bss)
.ascin "__DATA",16
.quad bss # vaddr
.quad bsssize # memsz
.long 0 # offset
.long 12 # align 2**12 = 4096
.long 0 # reloc table offset
.long 0 # relocation count
.long MAC_S_ZEROFILL # section type & attributes
.long 0,0,0 # reserved
.ascin "__RODATA",16
.quad 0 // vaddr
.quad 0 // memsz
.quad 0 // file offset
.quad 0 // file size
.long 0 // maxprot
.long 0 // initprot
.long 0 // segment section count
.long 0 // flags
40: .long MAC_LC_UUID
.long 50f-40b
.quad 0x3fb29ee4ac6c87aa # uuid1
.quad 0xdd2c9bb866d9eef8 # uuid2
.quad 0x4527148ba7a513ef // uuid1
.quad 0x56fa865940665e8f // uuid2
50: .long MAC_LC_UNIXTHREAD
.long 60f-50b # cmdsize
.long MAC_THREAD_NEXGEN32E # flavaflav
.long (520f-510f)/4 # count
510: .quad 0 # rax
.quad 0 # rbx
.quad 0 # rcx
.quad _HOSTXNU # rdx
.quad 0 # rdi
.quad 0 # rsi
.quad 0 # rbp
.quad 0 # rsp
.quad 0 # r8
.quad 0 # r9
.quad 0 # r10
.quad 0 # r11
.quad 0 # r12
.quad 0 # r13
.quad 0 # r14
.quad 0 # r15
.quad _start # rip
.quad 0 # rflags
.quad 0 # cs
.quad 0 # fs
.quad 0 # gs
.long 60f-50b // cmdsize
.long MAC_THREAD_NEXGEN32E // flavaflav
.long (520f-510f)/4 // count
510: .quad 0 // rax
.quad 0 // rbx
.quad 0 // rcx
.quad 0 // rdx
.quad 0 // rdi
.quad 0 // rsi
.quad 0 // rbp
.quad 0 // rsp
.quad 0 // r8
.quad 0 // r9
.quad 0 // r10
.quad 0 // r11
.quad 0 // r12
.quad 0 // r13
.quad 0 // r14
.quad 0 // r15
.quad XnuEntrypoint // rip
.quad 0 // rflags
.quad 0 // cs
.quad 0 // fs
.quad 0 // gs
520:
60:
.endobj macho,globl
.align 64
_start:
// Hack for detecting M1 Rosetta environment.
// https://github.com/jart/cosmopolitan/issues/429#issuecomment-1166704377
cmp $-1,%ebx
jne 0f
cmp $+1,%edx
jne 0f
mov $_HOSTXNU,%dl
xor %ecx,%ecx
0: mov %rsp,%rsi
jmp ApeLoader
.endfn _start,globl
__syscall_loader:
clc
syscall
jc 1f
ret
1: neg %rax
ret
.endfn __syscall_loader,globl
-42
View File
@@ -1,42 +0,0 @@
/*-*- mode: ld-script; indent-tabs-mode: nil; tab-width: 2; coding: utf-8 -*-│
│vi: set et sts=2 tw=2 fenc=utf-8 :vi│
╞══════════════════════════════════════════════════════════════════════════════╡
│ Copyright 2021 Justine Alexandra Roberts Tunney │
│ │
│ Permission to use, copy, modify, and/or distribute this software for │
│ any purpose with or without fee is hereby granted, provided that the │
│ above copyright notice and this permission notice appear in all copies. │
│ │
│ THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL │
│ WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED │
│ WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE │
│ AUTHOR BE LIABLE FOR ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL │
│ DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR │
│ PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER │
│ TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR │
│ PERFORMANCE OF THIS SOFTWARE. │
╚─────────────────────────────────────────────────────────────────────────────*/
ENTRY(_start)
SECTIONS {
. = 0x200000;
.text : {
*(.text)
*(.rodata .rodata.*)
. = ALIGN(4096);
}
filesz = . - macho;
textsz = . - _start;
.bss ALIGN(4096) : {
bss = .;
*(.bss)
. = ALIGN(4096);
}
memsz = . - macho;
/DISCARD/ : {
*(.*)
}
}
bsssize = SIZEOF(.bss);
textoff = _start - macho;
+820 -484
View File
File diff suppressed because it is too large. Load diff
-20
View File
@@ -1,20 +0,0 @@
#ifndef COSMOPOLITAN_APE_LOADER_H_
#define COSMOPOLITAN_APE_LOADER_H_
#define APE_LOADER_BASE 0x200000
#define APE_LOADER_SIZE 0x200000
#define APE_LOADER_ENTRY 0x200400
#define APE_LOADER_BSS (PAGESIZE * 2)
#define APE_LOADER_STACK 0x7f0000000000
#define APE_BLOCK_BASE 0x7e0000000000
#define APE_BLOCK_SIZE 0x000200000000
struct ApeLoader {
int fd;
int os;
char *prog;
char *page;
void *systemcall;
};
#endif /* COSMOPOLITAN_APE_LOADER_H_ */
+71 -19
View File
@@ -1,7 +1,7 @@
/*-*- mode: ld-script; indent-tabs-mode: nil; tab-width: 2; coding: utf-8 -*-│
│vi: set et sts=2 tw=2 fenc=utf-8 :vi│
│ vi: set et sts=2 sw=2 fenc=utf-8 :vi │
╞══════════════════════════════════════════════════════════════════════════════╡
│ Copyright 2021 Justine Alexandra Roberts Tunney │
│ Copyright 2023 Justine Alexandra Roberts Tunney │
│ │
│ Permission to use, copy, modify, and/or distribute this software for │
│ any purpose with or without fee is hereby granted, provided that the │
@@ -16,28 +16,80 @@
│ TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR │
│ PERFORMANCE OF THIS SOFTWARE. │
╚─────────────────────────────────────────────────────────────────────────────*/
ENTRY(_start)
ENTRY(ElfEntrypoint)
PHDRS {
text PT_LOAD FLAGS(1) FILEHDR PHDRS; /* PF_X */
rodata PT_LOAD FLAGS(4); /* PF_R */
stack PT_GNU_STACK FLAGS(6); /* PF_W|PF_R */
note PT_NOTE FLAGS(4); /* PF_R */
}
SECTIONS {
. = 0x200000;
. = SEGMENT_START("text-segment", 0x7f000000);
__executable_start = .;
. += SIZEOF_HEADERS;
.macho : {
KEEP(*(.macho))
} :text
.note : {
KEEP(*(.note))
} :text :note
.text : {
*(.text)
*(.rodata .rodata.*)
. = ALIGN(64);
}
filesz = . - ehdr;
textsz = . - _start;
.bss ALIGN(4096) : {
bss = .;
*(.bss)
. = ALIGN(4096);
}
*(.text .text.* .gnu.linkonce.t.*)
} :text
.rodata ALIGN(CONSTANT(COMMONPAGESIZE)) : {
*(.rodata .rodata.* .gnu.linkonce.r.*)
} :rodata
.stack : {
*(.stack)
} :stack
_end = .;
memsz = _end - ehdr;
.stab 0 : { *(.stab) }
.stabstr 0 : { *(.stabstr) }
.stab.excl 0 : { *(.stab.excl) }
.stab.exclstr 0 : { *(.stab.exclstr) }
.stab.index 0 : { *(.stab.index) }
.stab.indexstr 0 : { *(.stab.indexstr) }
.gnu.build.attributes : { *(.gnu.build.attributes .gnu.build.attributes.*) }
.debug 0 : { *(.debug) }
.line 0 : { *(.line) }
.debug_srcinfo 0 : { *(.debug_srcinfo) }
.debug_sfnames 0 : { *(.debug_sfnames) }
.debug_aranges 0 : { *(.debug_aranges) }
.debug_pubnames 0 : { *(.debug_pubnames) }
.debug_info 0 : { *(.debug_info .gnu.linkonce.wi.*) }
.debug_abbrev 0 : { *(.debug_abbrev) }
.debug_line 0 : { *(.debug_line .debug_line.* .debug_line_end) }
.debug_frame 0 : { *(.debug_frame) }
.debug_str 0 : { *(.debug_str) }
.debug_loc 0 : { *(.debug_loc) }
.debug_macinfo 0 : { *(.debug_macinfo) }
.debug_weaknames 0 : { *(.debug_weaknames) }
.debug_funcnames 0 : { *(.debug_funcnames) }
.debug_typenames 0 : { *(.debug_typenames) }
.debug_varnames 0 : { *(.debug_varnames) }
.debug_pubtypes 0 : { *(.debug_pubtypes) }
.debug_ranges 0 : { *(.debug_ranges) }
.debug_macro 0 : { *(.debug_macro) }
.debug_addr 0 : { *(.debug_addr) }
.debug_loclists 0 : { *(.debug_loclists) }
.debug_rnglists 0 : { *(.debug_rnglists) }
.debug_line_str 0 : { *(.debug_line_str) }
.debug_extra 0 : { *(.debug_line_str) }
.debug_pubtypes 0 : { *(.debug_pubtypes) }
.gnu.attributes 0 : { KEEP(*(.gnu.attributes)) }
/DISCARD/ : {
*(.*)
}
}
bsssize = SIZEOF(.bss);
textoff = _start - ehdr;
+22 -145
View File
@@ -1,5 +1,5 @@
/*-*- mode:unix-assembly; indent-tabs-mode:t; tab-width:8; coding:utf-8 -*-│
│vi: set et ft=asm ts=8 sw=8 fenc=utf-8 :vi│
│ vi: set noet ft=asm ts=8 sw=8 fenc=utf-8 :vi │
╞══════════════════════════════════════════════════════════════════════════════╡
│ Copyright 2020 Justine Alexandra Roberts Tunney │
│ │
@@ -22,119 +22,6 @@
#ifdef __ASSEMBLER__
/* clang-format off */
/**
* @fileoverview Macros relevant to αcτµαlly pδrταblε εxεcµταblε.
*/
// Calls near (i.e. pc+pcrel<64kB) FUNCTION.
// @mode long,legacy,real
// @cost 9 bytes overhead
.macro rlcall function:req
.byte 0x50 # push %[er]ax
.byte 0xb8,0,0 # mov $?,%[e]ax
jmp 911f
.byte 0x58 # pop %[er]ax
.byte 0xe8 # call Jvds
.long \function\()-.-4
jmp 912f
911: .byte 0x58 # pop %[er]ax
.byte 0xe8 # call Jvds
.short \function\()-.-2
912:
.endm
// Loads far (i.e. <1mb) abs constexpr ADDRESS into ES:DI+EDX+RDX.
// @mode long,legacy,real
.macro movesdi address:req
.byte 0xbf # mov $0x????xxxx,%[e]di
.short \address>>4
.byte 0x8e,0xc7 # mov %di,%es
.byte 0xbf # mov $0x????xxxx,%[e]di
.short \address&0xf
jmp 297f
.byte 0xbf # mov $0x????xxxx,%edi
.long \address
297:
.endm
// Loads 16-bit CONSTEXPR into Qw-register w/ optional zero-extend.
// @mode long,legacy,real
.macro bbmov constexpr:req abcd abcd.hi:req abcd.lo:req
.ifnb \abcd
.if (\constexpr)<128 && (\constexpr)>=0
pushpop \constexpr,\abcd
.exitm
.endif
.endif
movb $(\constexpr)>>8&0xff,\abcd.hi
movb $(\constexpr)&0xff,\abcd.lo
.endm
// Compares 16-bit CONSTEXPR with Qw-register.
// @mode long,legacy,real
.macro bbcmp constexpr:req abcd.hi:req abcd.lo:req
cmpb $(\constexpr)>>8&0xff,\abcd.hi
jnz 387f
cmpb $(\constexpr)&0xff,\abcd.lo
387:
.endm
// Adds 16-bit CONSTEXPR to Qw-register.
// @mode long,legacy,real
.macro bbadd constexpr:req abcd.hi:req abcd.lo:req
addb $(\constexpr)&0xff,\abcd.lo
.if (\constexpr) != 0
adcb $(\constexpr)>>8&0xff,\abcd.hi
.endif
.endm
// Subtracts 16-bit CONSTEXPR from Qw-register.
// @mode long,legacy,real
.macro bbsub constexpr:req abcd.hi:req abcd.lo:req
subb $(\constexpr)&0xff,\abcd.lo
.if (\constexpr) != 0
sbbb $(\constexpr)>>8&0xff,\abcd.hi
.endif
.endm
// Ands Qw-register with 16-bit CONSTEXPR.
// @mode long,legacy,real
.macro bband constexpr:req abcd.hi:req abcd.lo:req
.if ((\constexpr)&0xff) != 0xff || ((\constexpr)>>8&0xff) == 0xff
andb $(\constexpr)&0xff,\abcd.lo
.endif
.if ((\constexpr)>>8&0xff) != 0xff
andb $(\constexpr)>>8&0xff,\abcd.hi
.endif
.endm
// Ors Qw-register with 16-bit CONSTEXPR.
// @mode long,legacy,real
.macro bbor constexpr:req abcd.hi:req abcd.lo:req
.if ((\constexpr)&0xff) != 0 || ((\constexpr)>>8&0xff) != 0
orb $(\constexpr)&0xff,\abcd.lo
.endif
.if ((\constexpr)>>8&0xff) != 0
orb $(\constexpr)>>8&0xff,\abcd.hi
.endif
.endm
// Performs ACTION only if in real mode.
// @mode long,legacy,real
.macro rlo clobber:req action:vararg
990: mov $0,\clobber
.if .-990b!=3
.error "bad clobber or assembler mode"
.endif
991: \action
.rept 2-(.-991b)
nop
.endr
.if .-991b!=2
.error "ACTION must be 1-2 bytes"
.endif
.endm
// Initializes real mode stack.
// The most holiest of holy code.
// @mode real
@@ -197,9 +84,10 @@
/* clang-format on */
#elif defined(__LINKER__)
#define BCX_NIBBLE(X) ((((X)&0xf) > 0x9) ? ((X)&0xf) + 0x37 : ((X)&0xf) + 0x30)
#define BCX_OCTET(X) ((BCX_NIBBLE((X) >> 4) << 8) | (BCX_NIBBLE((X) >> 0) << 0))
#define BCX_INT16(X) ((BCX_OCTET((X) >> 8) << 16) | (BCX_OCTET((X) >> 0) << 0))
#define BCX_NIBBLE(X) \
((((X) & 0xf) > 0x9) ? ((X) & 0xf) + 0x37 : ((X) & 0xf) + 0x30)
#define BCX_OCTET(X) ((BCX_NIBBLE((X) >> 4) << 8) | (BCX_NIBBLE((X) >> 0) << 0))
#define BCX_INT16(X) ((BCX_OCTET((X) >> 8) << 16) | (BCX_OCTET((X) >> 0) << 0))
#define BCXSTUB(SYM, X) \
HIDDEN(SYM##_bcx0 = BCX_INT16((X) >> 48)); \
HIDDEN(SYM##_bcx1 = BCX_INT16((X) >> 32)); \
@@ -211,12 +99,12 @@
*
* <p>This allows linker scripts to generate printf commands.
*/
#define BCO_OCTET(X) (((X)&0x7) + 0x30)
#define BCO_OCTET(X) (((X) & 0x7) + 0x30)
#define BCOB_UNIT(X) \
((BCO_OCTET((X) >> 0) << 24) | (BCO_OCTET((X) >> 3) << 16) | \
(BCO_OCTET(((X)&0xff) >> 6) << 8) | 0x5c)
(BCO_OCTET(((X) & 0xff) >> 6) << 8) | 0x5c)
#define PFBYTE(SYM, X, I) HIDDEN(SYM##_bcs##I = BCOB_UNIT((X) >> ((I)*8)))
#define PFBYTE(SYM, X, I) HIDDEN(SYM##_bcs##I = BCOB_UNIT((X) >> ((I) * 8)))
#define PFSTUB2(SYM, X) \
HIDDEN(SYM = (X)); \
PFBYTE(SYM, X, 0); \
@@ -253,23 +141,16 @@
: (X) < 10000000 ? BCD_RIGHT(BCD_SMEAR((X) / 1000)) \
: (X) < 100000000 ? BCD_RIGHT(BCD_SMEAR((X) / 10000)) \
: 0xffffffffffffffff)
#define BCD_RIGHT(X) \
(((X)) < 10000 ? 0x20202020 \
: (X) < 100000 ? 0x20202030 + \
(X) % 10 \
: (X) < 1000000 ? 0x20203030 + \
((X) / 10) % 10 + \
(X) % 10 * 0x100 \
: (X) < 10000000 ? 0x20303030 + \
((X) / 100) % 10 + \
((X) / 10) % 10 * 0x100 + \
(X) % 10 * 0x10000 \
: (X) < 100000000 ? 0x30303030 + \
((X) / 1000) % 10 + \
((X) / 100) % 10 * 0x100 + \
((X) / 10) % 10 * 0x10000 + \
(X) % 10 * 0x1000000 \
: 0xffffffffffffffff)
#define BCD_RIGHT(X) \
(((X)) < 10000 ? 0x20202020 \
: (X) < 100000 ? 0x20202030 + (X) % 10 \
: (X) < 1000000 ? 0x20203030 + ((X) / 10) % 10 + (X) % 10 * 0x100 \
: (X) < 10000000 ? 0x20303030 + ((X) / 100) % 10 + \
((X) / 10) % 10 * 0x100 + (X) % 10 * 0x10000 \
: (X) < 100000000 \
? 0x30303030 + ((X) / 1000) % 10 + ((X) / 100) % 10 * 0x100 + \
((X) / 10) % 10 * 0x10000 + (X) % 10 * 0x1000000 \
: 0xffffffffffffffff)
/**
* Laying out the GDT entries for a TSS for bare metal operation.
@@ -278,15 +159,11 @@
HIDDEN(SYM##_desc_ent0 = TSSDESC_ENT0(BASE, LIM)); \
HIDDEN(SYM##_desc_ent1 = TSSDESC_ENT1(BASE)); \
ASSERT((LIM) >= 0 && (LIM) <= 0xffff, "bare metal TSS is suspiciously fat")
#define TSSDESC_ENT0(BASE, LIM) \
(((LIM) << 0 & 0x000000000000ffff) | \
((BASE) << 16 & 0x000000ffffff0000) | \
0x89 << 40 | \
((LIM) >> 16 << 48 & 0x000f000000000000) | \
0x2 << 52 | \
#define TSSDESC_ENT0(BASE, LIM) \
(((LIM) << 0 & 0x000000000000ffff) | ((BASE) << 16 & 0x000000ffffff0000) | \
0x89 << 40 | ((LIM) >> 16 << 48 & 0x000f000000000000) | 0x2 << 52 | \
((BASE) >> 24 << 56 & 0xff00000000000000))
#define TSSDESC_ENT1(BASE) \
((BASE) >> 32 << 0 & 0x00000000ffffffff)
#define TSSDESC_ENT1(BASE) ((BASE) >> 32 << 0 & 0x00000000ffffffff)
#endif /* __ASSEMBLER__ */
#endif /* APE_MACROS_H_ */
+25 -27
View File
@@ -1,34 +1,32 @@
#ifndef COSMOPOLITAN_APE_SECTIONS_INTERNAL_H_
#define COSMOPOLITAN_APE_SECTIONS_INTERNAL_H_
#if !(__ASSEMBLER__ + __LINKER__ + 0)
COSMOPOLITAN_C_START_
extern unsigned char _base[];
extern unsigned char _ehead[];
extern unsigned char _etext[];
extern unsigned char _edata[];
extern unsigned char _ezip[];
extern unsigned char _end[];
extern unsigned char _ereal[];
extern unsigned char _tdata_start[];
extern unsigned char _tdata_end[];
extern unsigned char _tbss_start[];
extern unsigned char _tbss_end[];
extern unsigned char __privileged_start[];
extern unsigned char __privileged_addr[];
extern unsigned char __privileged_size[];
extern unsigned char __privileged_end[];
extern unsigned char __test_start[];
extern unsigned char __ro[];
extern unsigned char *__relo_start[];
extern unsigned char *__relo_end[];
extern uint8_t __zip_start[];
extern uint8_t __zip_end[];
extern uint8_t __data_start[];
extern uint8_t __data_end[];
extern uint8_t __bss_start[];
extern uint8_t __bss_end[];
extern const char __notices[] __attribute__((__weak__));
extern unsigned char __executable_start[] __attribute__((__weak__));
extern unsigned char __privileged_start[] __attribute__((__weak__));
extern unsigned char _ehead[] __attribute__((__weak__));
extern unsigned char _etext[] __attribute__((__weak__));
extern unsigned char _edata[] __attribute__((__weak__));
extern unsigned char _ezip[] __attribute__((__weak__));
extern unsigned char _end[] __attribute__((__weak__));
extern unsigned char _ereal[] __attribute__((__weak__));
extern unsigned char _tdata_start[] __attribute__((__weak__));
extern unsigned char _tdata_end[] __attribute__((__weak__));
extern unsigned char _tbss_start[] __attribute__((__weak__));
extern unsigned char _tbss_end[] __attribute__((__weak__));
extern unsigned char _tls_align[] __attribute__((__weak__));
extern unsigned char _tdata_align[] __attribute__((__weak__));
extern unsigned char _tbss_align[] __attribute__((__weak__));
extern unsigned char __test_start[] __attribute__((__weak__));
extern unsigned char __ro[] __attribute__((__weak__));
extern unsigned char __data_start[] __attribute__((__weak__));
extern unsigned char __data_end[] __attribute__((__weak__));
extern unsigned char __bss_start[] __attribute__((__weak__));
extern unsigned char __bss_end[] __attribute__((__weak__));
extern unsigned long __got_start[] __attribute__((__weak__));
extern unsigned long __got_end[] __attribute__((__weak__));
extern unsigned char ape_phdrs[] __attribute__((__weak__));
COSMOPOLITAN_C_END_
#endif /* !(__ASSEMBLER__ + __LINKER__ + 0) */
#endif /* COSMOPOLITAN_APE_SECTIONS_INTERNAL_H_ */
+703
View File
@@ -0,0 +1,703 @@
# Actually Portable Executable Specification v0.1
Actually Portable Executable (APE) is an executable file format that
polyglots the Windows Portable Executable (PE) format with a UNIX Sixth
Edition style shell script that doesn't have a shebang. This makes it
possible to produce a single file binary that executes on the stock
installations of the many OSes and architectures.
## Supported OSes and Architectures
- AMD64
- Linux
- MacOS
- Windows
- FreeBSD
- OpenBSD
- NetBSD
- BIOS
- ARM64
- Linux
- MacOS
- FreeBSD
- Windows (non-native)
## File Header
APE defines three separate file magics, all of which are 8 characters
long. Any file that starts with one of these magic values can be
considered an APE program.
### (1) APE MZ Magic
- ASCII: `MZqFpD='`
- Hex: 4d 5a 71 46 70 44 3d 27
This is the canonical magic used by almost all APE programs. It enables
maximum portability between OSes. When interpreted as a shell script, it
is assigning a single quoted string to an unused variable. The shell
will then ignore subsequent binary content that's placed inside the
string.
It is strongly recommended that this magic value be immediately followed
by a newline (\n or hex 0a) character. Some shells, e.g. FreeBSD SH and
Zsh impose a binary safety check before handing off files that don't
have a shebang to `/bin/sh`. That check applies to the first line, which
can't contain NUL characters.
The letters were carefully chosen so as to be valid x86 instructions in
all operating modes. This makes it possible to store a BIOS bootloader
disk image inside an APE binary. For example, simple CLI programs built
with Cosmopolitan Libc will boot from BIOS into long mode if they're
treated as a floppy disk image.
The letters also allow for the possibility of being treated on x86-64 as
a flat executable, where the PE / ELF / Mach-O executable structures are
ignored, and execution simply begins at the beginning of the file,
similar to how MS-DOS .COM binaries work.
The 0x4a relative offset of the magic causes execution to jump into the
MS-DOS stub defined by Portable Executable. APE binaries built by Cosmo
Libc use tricks in the MS-DOS stub to check the operating mode and then
jump to the appropriate entrypoint, e.g. `_start()`.
#### Decoded as i8086
```asm
dec %bp
pop %dx
jno 0x4a
jo 0x4a
```
#### Decoded as i386
```asm
push %ebp
pop %edx
jno 0x4a
jo 0x4a
```
#### Decoded as x86-64
```asm
rex.WRB
pop %r10
jno 0x4a
jo 0x4a
```
### (2) APE UNIX-Only Magic
- ASCII: `jartsr='`
- Hex: 6a 61 72 74 73 72 3d 27
Being a novel executable format that was first published in 2020, the
APE file format is less understood by industry tools compared to the PE,
ELF, and Mach-O executable file formats, which have been around for
decades. For this reason, APE programs that use the MZ magic above can
attract attention from Windows AV software, which may be unwanted by
developers who aren't interested in targeting the Windows platform.
Therefore the `jartsr='` magic is defined which enables the creation of
APE binaries that can safely target all non-Windows platforms. Even
though this magic is less common, APE interpreters and binfmt-misc
installations MUST support this.
It is strongly recommended that this magic value be immediately followed
by a newline (\n or hex 0a) character. Some shells, e.g. FreeBSD SH and
Zsh impose a binary safety check before handing off files that don't
have a shebang to `/bin/sh`. That check applies to the first line, which
can't contain NUL characters.
The letters were carefully chosen so as to be valid x86 instructions in
all operating modes. This makes it possible to store a BIOS bootloader
disk image inside an APE binary. For example, simple CLI programs built
with Cosmopolitan Libc will boot from BIOS into long mode if they're
treated as a floppy disk image.
The letters also allow for the possibility of being treated on x86-64 as
a flat executable, where the PE / ELF / Mach-O executable structures are
ignored, and execution simply begins at the beginning of the file,
similar to how MS-DOS .COM binaries work.
The 0x78 relative offset of the magic causes execution to jump into the
MS-DOS stub defined by Portable Executable. APE binaries built by Cosmo
Libc use tricks in the MS-DOS stub to check the operating mode and then
jump to the appropriate entrypoint, e.g. `_start()`.
#### Decoded as i8086 / i386 / x86-64
```asm
push $0x61
jb 0x78
jae 0x78
```
### (3) APE Debug Magic
- ASCII: `APEDBG='`
- Hex: 41 50 45 44 42 47 3d 27
While APE files must be valid shell scripts, in practice, UNIX systems
will oftentimes be configured to provide a faster safer alternative to
loading an APE binary through `/bin/sh`. The Linux Kernel can be patched
to have execve() recognize the APE format and directly load its embedded
ELF header. Linux systems can also use binfmt-misc to recognize APE's MZ
and jartsr magic, and pass them to a userspace program named `ape` that
acts as an interpreter. In such environments, the need sometimes arises
to be able to test that the `/bin/sh` is working correctly, in which
case the `APEDBG='` magic is RECOMMENDED.
APE interpreters, execve() implementations, and binfmt-misc installs
MUST ignore this magic. If necessary, steps can be taken to help files
with this magic be passed to `/bin/sh` like a normal shebang-less shell
script for execution.
## Embedded ELF Header
APE binaries MAY embed an ELF header inside them. Unlike conventional
executable file formats, this header is not stored at a fixed offset.
It's instead encoded as octal escape codes in a shell script `printf`
statement. For example:
```
printf '\177ELF\2\1\1\011\0\0\0\0\0\0\0\0\2\0\076\0\1\0\0\0\166\105\100\000\000\000\000\000\060\013\000\000\000\000\000\000\000\000\000\000\000\000\000\000\165\312\1\1\100\0\070\0\005\000\0\0\000\000\000\000'
```
This `printf` statement MUST appear in the first 8192 bytes of the APE
executable, so as to limit how much of the initial portion of a file an
interpreter must load.
Multiple such `printf` statements MAY appear in the first 8192 bytes, in
order to specify multiple architectures. For example, fat binaries built
by the `apelink` program (provided by Cosmo Libc) will have two encoded
ELF headers, for AMD64 and ARM64, each of which point into the proper
file offsets for their respective native code. Therefore, kernels and
interpreters which load the APE format directly MUST check the
`e_machine` field of the `Elf64_Ehdr` that's decoded from the octal
codes, before accepting a `printf` shell statement as valid.
These printf statements MUST always use only unescaped ASCII characters
or octal escape codes. These printf statements MUST NOT use space saving
escape codes such as `\n`. For example, rather than saying `\n` it would
be valid to say `\012` instead. It's also valid to say `\12` but only if
the encoded characters that follow aren't an octal digit.
For example, the following algorithm may be used for parsing octal:
```c
static int ape_parse_octal(const unsigned char page[8192], int i, int *pc)
{
int c;
if ('0' <= page[i] && page[i] <= '7') {
c = page[i++] - '0';
if ('0' <= page[i] && page[i] <= '7') {
c *= 8;
c += page[i++] - '0';
if ('0' <= page[i] && page[i] <= '7') {
c *= 8;
c += page[i++] - '0';
}
}
*pc = c;
}
return i;
}
```
APE aware interpreters SHOULD only take `e_machine` into consideration.
It is the responsibility of the `_start()` function to detect the OS.
Therefore, multiple `printf` statements are only embedded in the shell
script for different CPU architectures.
The OS ABI field of an APE embedded `Elf64_Ehdr` SHOULD be set to
`ELFOSABI_FREEBSD`, since it's the only UNIX OS APE supports that
actually checks the field. However different values MAY be chosen for
binaries that don't intend to have FreeBSD in their support vector.
Counter-intuitively, the ARM64 ELF header is used on the MacOS ARM64
platform when loading from fat binaries.
## Embedded Mach-O Header (x86-64 only)
APE shell scripts that support MacOS on AMD64 must use the `dd` command
in a very specific way to specify how the embedded binary Macho-O header
is copied backward to the start of the file. For example:
```
dd if="$o" of="$o" bs=8 skip=433 count=66 conv=notrunc
```
These `dd` statements have traditionally been generated by the GNU as
and ld.bfd programs by encoding ASCII into 64-bit linker relocations,
which necessitated a fixed width for integer values. It took several
iterations over APE's history before we eventually got it right:
- `arg=" 9293"` is how we originally had ape do it
- `arg=$(( 9293))` b/c busybox sh disliked quoted space
- `arg=9293 ` is generated by modern apelink program
Software that parses the APE file format, which needs to extract the
Macho-O x86-64 header SHOULD support the old binaries that use the
previous encodings. To make backwards compatibility simple the following
regular expression may be used, which generalizes to all defined
formats:
```c
regcomp(&rx,
"bs=" // dd block size arg
"(['\"] *)?" // #1 optional quote w/ space
"(\\$\\(\\( *)?" // #2 optional math w/ space
"([[:digit:]]+)" // #3
"( *\\)\\))?" // #4 optional math w/ space
"( *['\"])?" // #5 optional quote w/ space
" +" //
"skip=" // dd skip arg
"(['\"] *)?" // #6 optional quote w/ space
"(\\$\\(\\( *)?" // #7 optional math w/ space
"([[:digit:]]+)" // #8
"( *\\)\\))?" // #9 optional math w/ space
"( *['\"])?" // #10 optional quote w/ space
" +" //
"count=" // dd count arg
"(['\"] *)?" // #11 optional quote w/ space
"(\\$\\(\\( *)?" // #12 optional math w/ space
"([[:digit:]]+)", // #13
REG_EXTENDED);
```
For further details, see the canonical implementation in
`cosmopolitan/tool/build/assimilate.c`.
## Static Linking
Actually Portable Executables are always statically linked. This
revision of the specification does not define any facility for storing
code in dynamic shared objects.
Cosmopolitan Libc provides a solution that enables APE binaries have
limited access to dlopen(). By manually loading a platform-specific
executable and asking the OS-specific libc's dlopen() to load
OS-specific libraries, it becomes possible to use GPUs and GUIs. This
has worked great for AI projects like llamafile.
There is no way for an Actually Portable Executable to interact with
OS-specific dynamic shared object extension modules to programming
languages. For example, a Lua interpreter compiled as an Actually
Portable Executable would have no way of linking extension libraries
downloaded from the Lua Rocks package manager. This is primarily because
different OSes define incompatible ABIs.
While it was possible to polyglot PE+ELF+MachO to create multi-OS
executables, it simply isn't possible to do that same thing for
DLL+DYLIB+SO. Therefore, in order to have DSOs, APE would need to either
choose one of the existing formats or invent one of its own, and then
develop its own parallel ecosystem of extension software. In the future,
the APE specification may expand to encompass this. However the focus to
date has been exclusively on executables with limited dlopen() support.
## Application Binary Interface (ABI)
APE binaries use the System V ABI, as defined by:
- [System V ABI - AMD64 Architecture Processor Supplement](https://gitlab.com/x86-psABIs/x86-64-ABI)
- AARCH64 has a uniform consensus defined by ARM Limited
There are however a few changes we've had to make.
### No Red Zone
Actually Portable Executables that have Windows and/or bare metal in
their support vector MUST be compiled using `-mno-red-zone`. This is
because, on Windows, DLLs and other software lurking in the va-space
might use tricks like SetThreadContext() to take control of a thread
whereas on bare metal, it's also generally accepted that kernel-mode
code cannot assume a red zone either due to hardware interrupts that
pull the exact same kinds of stunts.
APE software that only has truly System V ABI conformant OSes (e.g.
Linux) in their support vector MAY use the red zone optimization.
### Thread Local Storage
#### aarch64
Here's the TLS memory layout on aarch64:
```
x28
%tpidr_el0
│
│ _Thread_local
┌───┼───┬──────────┬──────────┐
│tib│dtv│ .tdata │ .tbss │
├───┴───┴──────────┴──────────┘
│
__get_tls()
```
The ARM64 code in actually portable executables use the `x28` register
to store the address of the thread information block. All aarch64 code
linked into these executables SHOULD be compiled with `-ffixed-x28`
which is supported by both Clang and GCC.
The runtime library for an actually portable executables MAY choose to
use `tpidr_el0` instead, if OSes like MacOS aren't being targeted. For
example, if the goal is to create a Linux-only fat binary linker program
for Musl Libc, then choosing to use the existing `tpidr_el0` convention
would be friction-free alternative.
It's not possible for an APE runtime that targets the full range of OSes
to use the `tpidr_el0` register for TLS because Apple won't allow it. On
MacOS ARM64 systems, this register can only be used by a runtime to
implement the `sched_getcpu()` system call. It's reserved by MacOS.
#### x86-64
Here's the TLS memory layout on x86_64:
```
__get_tls()
│
%fs OpenBSD/NetBSD
_Thread_local │
┌───┬──────────┬──────────┼───┐
│pad│ .tdata │ .tbss │tib│
└───┴──────────┴──────────┼───┘
│
Linux/FreeBSD/Windows/Mac %gs
```
Quite possibly the greatest challenge in Actually Portable Executable
working, has been overcoming the incompatibilities between OSes in how
thread-local storage works on x86-64. The AMD64 architecture defines two
special segment registers. Every OS uses one of these segment registers
to implement TLS support. However not all OSes agree on which register
to use. Some OSes grant userspace the power to define either of these
registers to hold any value that is desired. Some OSes only effectively
allow a single one of them to be changed. Lastly, some OSes, e.g.
Windows, claim ownership of the memory layout these registers point
towards too.
Here's a breakdown on how much power is granted to userspace runtimes by
each OS when it comes to changing amd64 segment registers.
| | %fs | %gs |
|---------|--------------|--------------|
| Linux | unrestricted | unrestricted |
| MacOS | inaccessible | unrestricted |
| Windows | inaccessible | restricted |
| FreeBSD | unrestricted | unrestricted |
| NetBSD | unrestricted | broken |
| OpenBSD | unrestricted | inaccessible |
Therefore, regardless of which register one we choose, some OSes are
going to be incompatible.
APE binaries are always built with a Linux compiler. So another issue
arises in the fact that our Linux-flavored GCC and Clang toolchains
(which are used to produce cross-OS binaries) are also only capable of
producing TLS instructions that use the %fs convention.
To solve these challenges, the `cosmocc` compiler will rewrite binary
objects after they've been compiled by GCC, so that the `%gs` register
is used, rather than `%fs`. Morphing x86-64 binaries after they've been
compiled is normally difficult, due to the complexity of the machine
instruction language. However GCC provides `-mno-tls-direct-seg-refs`
which greatly reduces the complexity of this task. This flag forgoes
some optimizations to make the generated code simpler. Rather than doing
clever arithmetic with `%fs` prefixes, the compiler will always generate
the thread information block address load as a separate instruction.
```c
// Change AMD code to use %gs:0x30 instead of %fs:0
// We assume -mno-tls-direct-seg-refs has been used
static void ChangeTlsFsToGs(unsigned char *p, size_t n) {
unsigned char *e = p + n - 9;
while (p <= e) {
// we're checking for the following expression:
// 0144 == p[0] && // %fs
// 0110 == (p[1] & 0373) && // rex.w (and ignore rex.r)
// (0213 == p[2] || // mov reg/mem → reg (word-sized)
// 0003 == p[2]) && // add reg/mem → reg (word-sized)
// 0004 == (p[3] & 0307) && // mod/rm (4,reg,0) means sib → reg
// 0045 == p[4] && // sib (5,4,0) → (rbp,rsp,0) → disp32
// 0000 == p[5] && // displacement (von Neumann endian)
// 0000 == p[6] && // displacement
// 0000 == p[7] && // displacement
// 0000 == p[8] // displacement
uint64_t w = READ64LE(p) & READ64LE("\377\373\377\307\377\377\377\377");
if ((w == READ64LE("\144\110\213\004\045\000\000\000") ||
w == READ64LE("\144\110\003\004\045\000\000\000")) &&
!p[8]) {
p[0] = 0145; // change %fs to %gs
p[5] = 0x30; // change 0 to 0x30
p += 9;
} else {
++p;
}
}
}
```
By favoring `%gs` we've now ensured friction-free compatibility for the
APE runtime on MacOS, Linux, and FreeBSD which are all able to conform
easily to this convention. However additional work needs to be done at
runtime when an APE program is started on Windows, OpenBSD, and NetBSD.
On these platforms, all executable pages must be faulted and morphed to
fixup the TLS instructions.
On OpenBSD and NetBSD, this is as simple as undoing the example
operation above. Earlier at compile-time we turned `%fs` into `%gs`.
Now, at runtime, `%gs` must be turned back into `%fs`. Since the
executable is morphing itself, this is easier said than done.
OpenBSD for example enforces a `W^X` invariant. Code that's executing
can't modify itself at the same time. The way Cosmopolitan solves this
is by defining a special part of the binary called `.text.privileged`.
This section is aligned to page boundaries. A GNU ld linker script is
used to ensure that code which morphs code is placed into this section,
through the use of a header-defined cosmo-specific keyword `privileged`.
Additionally, the `fixupobj` program is used by the Cosmo build system
to ensure that compiled objects don't contain privileged functions that
call non-privileged functions. Needless to say, `mprotect()` needs to be
a privileged function, so that it can be used to disable the execute bit
on all other parts of the executable except for the privileged section,
thereby making it writable. Once this has been done, code can change.
On Windows the displacement bytes of the TLS instruction are changed to
use the `%gs:0x1480+i*8` ABI where `i` is a number assigned by the WIN32
`TlsAlloc()` API. This avoids the need to call `TlsGetValue()` which is
implemented this exact same way under the hood. Even though 0x1480 isn't
explicitly documented by MSDN, this ABI is believed to be stable because
MSVC generates binaries that use this offset directly. The only caveat
is that `TlsAlloc()` must be called as early in the runtime init as
possible, to ensure an index less than 64 is returned.
### Thread Information Block (TIB)
The Actually Portable Executable Thread Information Block (TIB) is
defined by this version of the specification as follows:
- The 64-bit TIB self-pointer is stored at offset 0x00.
- The 64-bit TIB self-pointer is also stored at offset 0x30.
- The 32-bit `errno` value is stored at offset 0x3c.
All other parts of the thread information block should be considered
unspecified and therefore reserved for future specifications.
The APE thread information block is aligned on a 64-byte boundary.
Cosmopolitan Libc v3.5.8 (c. 2024-07-21) currently implements a thread
information block that's 512 bytes in size.
### Foreign Function Calls
Even though APE programs always use the System V ABI, there arises the
occasional need to interface with foreign functions, e.g. WIN32. The
`__attribute__((__ms_abi__))` annotation introduced by GCC v6 is used
for this purpose.
The ability to change a function's ABI on a case-by-case basis is
surprisingly enough supported by GCC, Clang, NVCC, and even the AMD HIP
compilers for both UNIX systems and Windows. All of these compilers
support both the System V ABI and the Microsoft x64 ABI.
APE binaries will actually favor the Microsoft ABI even when running on
UNIX OSes for certain dlopen() use-cases. For example, if we control the
code to a CUDA module, which we compile on each OS separately from our
main APE binary, then any function that's inside the APE binary whose
pointer may be passed into a foreign module SHOULD be compiled to use
the Microsoft ABI. This is because in practice the OS-specific module
may need to be compiled by MSVC, where MS ABI is the *only* ABI, which
forces our UNIX programs to partially conform. Thankfully, all UNIX
compilers support doing it on a case-by-case basis.
### Char Signedness
Actually Portable Executable defines `char` as signed.
Therefore conformant APE software MUST use `-fsigned-char` when building
code for aarch64, as well as any other architecture that (unlike x86-64)
would otherwise define `char` as being `unsigned char` by default.
This decision was one of the cases where it made sense to offer a more
consistent runtime experience for fat multi-arch binaries. However you
SHOULD still write code to assume `char` can go either way. But if all
you care about is using APE, then you CAN assume `char` is signed.
### Long Double
On AMD64 platforms, APE binaries define `long double` as 80-bit.
On ARM64 platforms, APE binaries define `long double` as 128-bit.
We accept inconsistency in this case, because hardware acceleration is
far more valuable than stylistic consistency in the case of mathematics.
One challenge arises on AMD64 for supporting `long double` across OSes.
Unlike UNIX systems, the Windows Executive on x86-64 initializes the x87
FPU to have double (64-bit) precision rather than 80-bit. That's because
code compiled by MSVC treats `long double` as though it were `double` to
prefer always using the more modern SSE instructions. However System V
requires genuine 80-bit `long double` support on AMD64.
Therefore, if an APE program detects that it's been started on a Windows
x86-64 system, then it SHOULD use the following assembly to initialize
the x87 FPU in System V ABI mode.
```asm
fldcw 1f(%rip)
.rodata
.balign 2
// 8087 FPU Control Word
// IM: Invalid Operation ───────────────┐
// DM: Denormal Operand ───────────────┐│
// ZM: Zero Divide ───────────────────┐││
// OM: Overflow ─────────────────────┐│││
// UM: Underflow ───────────────────┐││││
// PM: Precision ──────────────────┐│││││
// PC: Precision Control ───────┐ ││││││
// {float,∅,double,long double}│ ││││││
// RC: Rounding Control ──────┐ │ ││││││
// {even, →-∞, →+∞, →0} │┌┤ ││││││
// ┌┤││ ││││││
// d││││rr││││││
1: .short 0b00000000000000000001101111111
.previous
```
## Executable File Alignment
Actually Portable Executable is a statically-linked flat executable file
format that is, as a thing in itself, agnostic to file alignments. For
example, the shell script payload at the beginning of the file and its
statements have no such requirements. Alignment requirements are however
imposed by the executable formats that APE wraps.
1. ELF requires that file offsets be congruent with virtual addresses
modulo the CPU page size. So when we add a shell script to the start
of an executable, we need to round up to the page size in order to
maintain ELF's invariant. Although no such roundup is required on the
program segments once the invariant is restored. ELF loaders will
happily map program headers from arbitrary file intervals (which may
overlap) onto arbitrarily virtual intervals (which don't need to be
contiguous). In order to do that, the loaders will generally use
UNIX's mmap() function which is more restrictive and only accepts
addresses and offsets that are page aligned. To make it possible to
map an unaligned ELF program header that could potentially start and
stop at any byte, ELF loaders round-out the intervals, which means
adjacent unrelated data might also get mapped, which may need to be
explicitly zero'd. Thanks to the cleverness of ELF, it's possible to
have an executable file be very tiny, without needing any alignment
bytes, and it'll be loaded into a properly aligned virtual space
where segments can be as sparse as we want them to be.
2. PE doesn't care about congruence and instead defines two separate
kinds of alignment. First, PE requires that the layout of segment
memory inside the file be aligned on at minimum the classic 512 byte
MS-DOS page size. This means that, unlike ELF, some alignment padding
may need to be encoded into the file, making it slightly larger. Next
PE imposes an alignment restriction on segments once they've been
mapped into the virtual address space, which must be rounded to the
system page size. Like ELF, PE segments need to be properly ordered
but they're allowed to drift apart once mapped in a non-contiguous
sparsely mapped way. When inserting shell script content at the start
of a PE file, the most problematic thing is the need to round up to
the 64kb system granularity, which results in a lot of needless bytes
of padding being inserted by a naive second-pass linker.
3. Apple's Mach-O format is the strictest of them all. While both ELF
and PE are defined in such a way that invites great creativity, XNU
will simply refuse to an executable that does anything creative with
alignment. All loaded segments need to both start and end on a page
aligned address. XNU also wants segments to be contiguous similar to
portable executable, except it applies to both the file and virtual
spaces, which must follow the same structure.
Actually Portable Executables must conform to the strictest requirements
demanded by the support vector. Therefore an APE binary that has headers
for all three of the above executable formats MUST conform to the Apple
way of doing things. GNU ld linker scripts aren't very good at producing
ELF binaries that rigidly conform to this simple naive layout. There are
so many ways things can go wrong, where third party code might slip its
own custom section name in-between the linker script sections that are
explicitly defined, thereby causing ELF's powerful features to manifest
and the resulting content overlapping. The best `ld` flag that helps is
`--orphan-handling=error` which can help with explaining such mysteries.
While Cosmopolitan was originally defined to just use stock GNU tools,
this proved intractable over time, and the project has been evolving in
the direction of building its own. Inventing the `apelink` program was
what enabled the project to achieve multi-architecture binaries whereas
previously it was only possible to do multi-OS binaries. In the future,
our hope is that a fast power linker like Mold can be adapted to produce
fat APE binaries directly from object files in one pass.
## Position Independent Code
APE doesn't currently support position independent executable formats.
This is because APE was originally written for the GNU linker, where PIC
and PIE were after-thoughts and never fully incorporated with the older
more powerful linker script techniques upon which APE relies. Future
iterations of this specification are intended to converge on modern
standards, as our tooling becomes developed enough to support it.
However this only applies to the wrapped executable formats themselves.
While our convention to date has been to always load ELF programs at the
4mb mark, this is not guaranteed across OSes and architectures. Programs
should have no expectations that a program will be loaded to any given
address. For example, Cosmo currently implements APE on AARCH64 as
loading executables to a starting address of 0x000800000000. This
address occupies a sweet spot of requirements.
## Address Space
In order to create a single binary that supports as many platforms as
possible without needing to be recompiled, there's a very narrow range
of addresses that can be used. That range is somewhere between 32 bits
and 39 bits.
- Embedded devices that claim to be 64-bit will oftentimes only support
a virtual address space that's 39 bits in size.
- We can't load executable images on AARCH64 beneath 0x100000000 (4gb)
because Apple forbids doing that, possibly in an effort to enforce a
best practice for spotting 32-bit to 64-bit transition bugs. Please
note that this restriction only applies to Apple ARM64 systems. The
x86-64 version of XNU will happily load APE binaries to 0x00400000.
- The AMD64 architecture on desktops and servers can usually be counted
upon to provide a 47-bit address space. The Linux Kernel for instance
grants each userspace program full dominion over addresses 0x00200000
through 0x00007fffffffffff provided the hardware supports this. On
modern workstations supporting Intel and AMD's new PML5T feature which
virtualizes memory using a radix trie that's five layers deep, Linux
is able to offer userspace its choice of fixed addresses from
0x00200000 through 0x00ffffffffffffff. The only exception to this rule
we've encountered so far is that Windows 7 and Windows Vista behaved
similar to embedded devices in reducing the number of va bits.
## Page Size
APE software MUST be page size agnostic. For many years the industry had
converged on a strong consensus of having a page size that's 4096 bytes.
However this convention was never guaranteed. New computers have become
extremely popular, such as Apple Silicon, that use a 16kb page size.
By convention, Cosmopolitan Libc currently generates ELF headers for
x86-64 that are strictly aligned on a 4096-byte page size. On ARM64
Cosmopolitan is currently implemented to always generate ELF headers
aligned on a 16kb page size.
In addition to being page size agnostic, APE software that cares about
working correctly on Windows needs to be aware of the concept of
allocation granularity. While the page size on Windows is generally 4kb
in size, memory mappings can only be created on addresses that aligned
to the system allocation granularity, which is generally 64kb. If you
use a function like mmap() with Cosmopolitan Libc, then the `addr` and
`offset` parameters need to be aligned to `sysconf(_SC_GRANSIZE)` or
else your software won't work on Windows. Windows has other limitations
too, such as lacking the ability to carve or punch holes in mappings.
+74
View File
@@ -0,0 +1,74 @@
/*-*- mode:unix-assembly; indent-tabs-mode:t; tab-width:8; coding:utf-8 -*-│
│ vi: set noet ft=asm ts=8 sw=8 fenc=utf-8 :vi │
╞══════════════════════════════════════════════════════════════════════════════╡
│ Copyright 2023 Justine Alexandra Roberts Tunney │
│ │
│ Permission to use, copy, modify, and/or distribute this software for │
│ any purpose with or without fee is hereby granted, provided that the │
│ above copyright notice and this permission notice appear in all copies. │
│ │
│ THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL │
│ WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED │
│ WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE │
│ AUTHOR BE LIABLE FOR ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL │
│ DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR │
│ PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER │
│ TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR │
│ PERFORMANCE OF THIS SOFTWARE. │
╚─────────────────────────────────────────────────────────────────────────────*/
#include "libc/dce.h"
#include "ape/ape.h"
#include "libc/macros.internal.h"
#ifdef __aarch64__
_start: mov x1,sp
and sp,x1,#-16
mov x29,0
bl ApeLoader
.endfn _start,globl
#else
XnuEntrypoint:
mov $_HOSTXNU,%dl // xnu's not unix!
ElfEntrypoint:
mov %rsp,%rsi // save real stack
andq $-16,%rsp // force SSE alignment
call ApeLoader
.endfn ElfEntrypoint,globl
.endfn XnuEntrypoint,globl
.section .note,"a",@progbits
.balign 4
openbsd.ident:
.long 2f-1f
.long 4f-3f
.long 1
1: .asciz "OpenBSD"
2: .balign 4
3: .long 0
4: .size openbsd.ident,.-openbsd.ident
.type openbsd.ident,@object
.balign 4
netbsd.ident:
.long 2f-1f
.long 4f-3f
.long 1
1: .asciz "NetBSD"
2: .balign 4
3: .long 901000000
4: .size netbsd.ident,.-netbsd.ident
.type netbsd.ident,@object
.balign 4
ape.ident:
.long 2f-1f
.long 4f-3f
.long 1
1: .asciz "APE"
2: .balign 4
3: .long APE_VERSION_NOTE
4: .size ape.ident,.-ape.ident
.type ape.ident,@object
#endif
+49
View File
@@ -0,0 +1,49 @@
/*-*- mode:unix-assembly; indent-tabs-mode:t; tab-width:8; coding:utf-8 -*-│
│ vi: set noet ft=asm ts=8 sw=8 fenc=utf-8 :vi │
╞══════════════════════════════════════════════════════════════════════════════╡
│ Copyright 2023 Justine Alexandra Roberts Tunney │
│ │
│ Permission to use, copy, modify, and/or distribute this software for │
│ any purpose with or without fee is hereby granted, provided that the │
│ above copyright notice and this permission notice appear in all copies. │
│ │
│ THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL │
│ WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED │
│ WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE │
│ AUTHOR BE LIABLE FOR ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL │
│ DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR │
│ PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER │
│ TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR │
│ PERFORMANCE OF THIS SOFTWARE. │
╚─────────────────────────────────────────────────────────────────────────────*/
#include "libc/macros.internal.h"
// Invokes system call.
//
// This function has eight parameters. The first seven are for
// arguments passed along to the system call. The eight is for
// the magic number that indicates which system call is called
//
// The return value follows the Linux kernel convention, where
// errors are returned as `-errno`. BSD systems are normalized
// to follow this convention automatically.
SystemCall:
#ifdef __aarch64__
mov x8,x7
mov x9,0
adds x9,x9,0
svc 0
bcs 1f
ret
1: neg x0,x0
ret
#else
mov %rcx,%r10
mov 16(%rsp),%eax
clc
syscall
jnc 1f
neg %rax
1: ret
#endif
.endfn SystemCall,globl
BIN
View File
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
BIN
View File
Binary file not shown.
Binary file not shown.
BIN
View File
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
+58
View File
@@ -0,0 +1,58 @@
--nocompress-debug-sections
--noexecstack
-Wa,--nocompress-debug-sections
-Wa,--noexecstack
-Wa,-msse2avx
-Werror=maybe-uninitialized
-Wno-literal-suffix
-Wno-unused-but-set-variable
-Wunsafe-loop-optimizations
-fbranch-target-load-optimize
-fcx-limited-range
-fdelete-dead-exceptions
-femit-struct-debug-baseonly
-ffp-int-builtin-inexact
-finline-functions-called-once
-fipa-pta
-fivopts
-flimit-function-alignment
-fmerge-constants
-fmodulo-sched
-fmodulo-sched-allow-regmoves
-fno-align-jumps
-fno-align-labels
-fno-align-loops
-fno-code-hoisting
-fno-cx-limited-range
-fno-fp-int-builtin-inexact
-fno-gnu-unique
-fno-inline-functions-called-once
-fno-instrument-functions
-fno-schedule-insns2
-fno-whole-program
-fopt-info-vec
-fopt-info-vec-missed
-freg-struct-return
-freschedule-modulo-scheduled-loops
-frounding-math
-fsched2-use-superblocks
-fschedule-insns
-fschedule-insns2
-fshrink-wrap
-fshrink-wrap-separate
-fsignaling-nans
-fstack-clash-protection
-ftracer
-ftrapv
-ftree-loop-im
-ftree-loop-vectorize
-funsafe-loop-optimizations
-fversion-loops-for-strides
-fwhole-program
-gdescribe-dies
-gstabs
-mcall-ms2sysv-xlogues
-mdispatch-scheduler
-mfpmath=sse+387
-mmitigate-rop
-mno-fentry
Binary file not shown.
BIN
View File
Binary file not shown.
Binary file not shown.
Binary file not shown.
BIN
View File
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
+224 -107
View File
@@ -1,20 +1,52 @@
#-*-mode:makefile-gmake;indent-tabs-mode:t;tab-width:8;coding:utf-8-*-┐
#───vi: set et ft=make ts=8 tw=8 fenc=utf-8 :vi───────────────────────┘
#── vi: set et ft=make ts=8 sw=8 fenc=utf-8 :vi ──────────────────────┘
# Default Mode
#
# - `make`
# - Optimized
# - Backtraces
# - Debuggable
# - Syscall tracing
# - Function tracing
# - Reasonably small
#
ifeq ($(MODE),)
CONFIG_CCFLAGS += $(BACKTRACES) $(FTRACE) -O2
ENABLE_FTRACE = 1
CONFIG_OFLAGS ?= -g -ggdb
CONFIG_CCFLAGS += -O2 $(BACKTRACES)
CONFIG_CPPFLAGS += -DSYSDEBUG
endif
ifeq ($(MODE), x86_64)
ENABLE_FTRACE = 1
CONFIG_OFLAGS ?= -g -ggdb
CONFIG_CCFLAGS += -O2 $(BACKTRACES)
CONFIG_CPPFLAGS += -DSYSDEBUG
endif
ifeq ($(MODE), aarch64)
ENABLE_FTRACE = 1
CONFIG_OFLAGS ?= -g -ggdb
CONFIG_CCFLAGS += -O2 $(BACKTRACES)
CONFIG_CPPFLAGS += -DSYSDEBUG
endif
# Zero Optimization Mode
#
# - Goes 2x slower
# - Supports --strace
# - Unsupports --ftrace
# - Better GDB debugging
#
ifeq ($(MODE), zero)
ENABLE_FTRACE = 1
CONFIG_OFLAGS ?= -g -ggdb
OVERRIDE_CFLAGS += -O0
OVERRIDE_CXXFLAGS += -O0
CONFIG_CPPFLAGS += -DSYSDEBUG
endif
ifeq ($(MODE), aarch64-zero)
CONFIG_OFLAGS ?= -g -ggdb
OVERRIDE_CFLAGS += -O0 -fdce
OVERRIDE_CXXFLAGS += -O0 -fdce
CONFIG_CPPFLAGS += -DSYSDEBUG
TARGET_ARCH ?= -msse3
endif
# Fast Build Mode
@@ -25,14 +57,12 @@ endif
# - Function tracing
# - Some optimizations
# - Limited Backtraces
# - Compiles 28% faster
#
ifeq ($(MODE),fastbuild)
CONFIG_CCFLAGS += $(BACKTRACES) $(FTRACE) -O
ifeq ($(MODE), fastbuild)
ENABLE_FTRACE = 1
CONFIG_CCFLAGS += $(BACKTRACES) -O
CONFIG_CPPFLAGS += -DSYSDEBUG -DDWARFLESS
CONFIG_OFLAGS += -g0
CONFIG_LDFLAGS += -S
TARGET_ARCH ?= -msse3
endif
# Optimized Mode
@@ -48,12 +78,15 @@ endif
# - GCC 8+ hoists check fails into .text.cold, thus minimizing impact
#
ifeq ($(MODE), opt)
CONFIG_CPPFLAGS += -DNDEBUG -DSYSDEBUG -msse2avx -Wa,-msse2avx
CONFIG_CCFLAGS += $(BACKTRACES) $(FTRACE) -O3 -fmerge-all-constants
TARGET_ARCH ?= -march=native
ENABLE_FTRACE = 1
CONFIG_OFLAGS ?= -g -ggdb
CONFIG_CPPFLAGS += -DNDEBUG -DSYSDEBUG
CONFIG_CCFLAGS += $(BACKTRACES) -O3 -fmerge-all-constants
CONFIG_TARGET_ARCH ?= -march=native
endif
# Optimized Linux Mode
# The Fastest Mode of All
#
# - `make MODE=optlinux`
# - Turns on red zone
@@ -63,10 +96,24 @@ endif
# - Turns off support for other operating systems
#
ifeq ($(MODE), optlinux)
CONFIG_CPPFLAGS += -DNDEBUG -msse2avx -Wa,-msse2avx -DSUPPORT_VECTOR=1
CONFIG_OFLAGS ?= -g -ggdb
CONFIG_CPPFLAGS += -DNDEBUG -DSYSDEBUG -DSUPPORT_VECTOR=1
CONFIG_CCFLAGS += -O3 -fmerge-all-constants
DEFAULT_COPTS += -mred-zone
TARGET_ARCH ?= -march=native
CONFIG_COPTS += -mred-zone
CONFIG_TARGET_ARCH ?= -march=native
endif
ifeq ($(MODE), x86_64-optlinux)
CONFIG_OFLAGS ?= -g -ggdb
CONFIG_CPPFLAGS += -DNDEBUG -DSYSDEBUG -DSUPPORT_VECTOR=1
CONFIG_CCFLAGS += -O3 -fmerge-all-constants
CONFIG_COPTS += -mred-zone
CONFIG_TARGET_ARCH ?= -march=native
endif
ifeq ($(MODE), aarch64-optlinux)
CONFIG_OFLAGS ?= -g -ggdb
CONFIG_CPPFLAGS += -DNDEBUG -DSYSDEBUG -DSUPPORT_VECTOR=1
CONFIG_CCFLAGS += -O3 -fmerge-all-constants
CONFIG_COPTS += -mred-zone
endif
# Release Mode
@@ -77,6 +124,7 @@ endif
# - More optimized
# - Reasonably small
# - Numeric backtraces
# - No DWARF data bloat
# - Toilsome debuggability
# - assert() statements removed
# - DCHECK_xx() statements removed
@@ -84,45 +132,51 @@ endif
# - CHECK_xx() won't leak strings into binary
#
ifeq ($(MODE), rel)
CONFIG_CPPFLAGS += -DNDEBUG
CONFIG_CPPFLAGS += -DNDEBUG -DDWARFLESS
CONFIG_CCFLAGS += $(BACKTRACES) -O2
TARGET_ARCH ?= -msse3
PYFLAGS += -O1
endif
# Asan Mode
#
# Safer binaries good for backend production serving.
#
# - `make MODE=asan`
# - Memory safety
# - Production worthy
# - Backtraces
# - Debuggability
# - Larger binaries
#
ifeq ($(MODE), asan)
CONFIG_CCFLAGS += $(BACKTRACES) -O2 -DSYSDEBUG
CONFIG_COPTS += -fsanitize=address
TARGET_ARCH ?= -msse3
endif
# Debug Mode
#
# - `make MODE=dbg`
# - Backtraces
# - Enables asan
# - Enables ubsan (TODO)
# - Enables ubsan
# - Stack canaries
# - No optimization (TODO)
# - No optimization
# - Enormous binaries
#
ifeq ($(MODE), dbg)
CONFIG_CPPFLAGS += -DMODE_DBG
CONFIG_CCFLAGS += $(BACKTRACES) $(FTRACE) -DSYSDEBUG -O -fno-inline
CONFIG_COPTS += -fsanitize=address -fsanitize=undefined
TARGET_ARCH ?= -msse3
ENABLE_FTRACE = 1
CONFIG_OFLAGS ?= -g -ggdb
OVERRIDE_CFLAGS += -O0
OVERRIDE_CXXFLAGS += -O0
CONFIG_CPPFLAGS += -DMODE_DBG -D__SANITIZE_UNDEFINED__ -Wno-unused-variable -Wno-unused-but-set-variable
CONFIG_CCFLAGS += $(BACKTRACES) -DSYSDEBUG
CONFIG_COPTS += -fsanitize=undefined
OVERRIDE_CCFLAGS += -fno-pie
QUOTA ?= -C64 -L300
endif
ifeq ($(MODE), x86_64-dbg)
ENABLE_FTRACE = 1
CONFIG_OFLAGS ?= -g -ggdb
OVERRIDE_CFLAGS += -O0
OVERRIDE_CXXFLAGS += -O0
CONFIG_CPPFLAGS += -DMODE_DBG -D__SANITIZE_UNDEFINED__ -Wno-unused-variable -Wno-unused-but-set-variable
CONFIG_CCFLAGS += $(BACKTRACES) -DSYSDEBUG
CONFIG_COPTS += -fsanitize=undefined
OVERRIDE_CCFLAGS += -fno-pie
QUOTA ?= -C64 -L300
endif
ifeq ($(MODE), aarch64-dbg)
ENABLE_FTRACE = 1
CONFIG_OFLAGS ?= -g -ggdb
OVERRIDE_CFLAGS += -O0 -fdce
OVERRIDE_CXXFLAGS += -O0 -fdce
CONFIG_CPPFLAGS += -DMODE_DBG -D__SANITIZE_UNDEFINED__ -Wno-unused-variable -Wno-unused-but-set-variable
CONFIG_CCFLAGS += $(BACKTRACES) -DSYSDEBUG
CONFIG_COPTS += -fsanitize=undefined
QUOTA ?= -C64 -L300
endif
# System Five Mode
@@ -136,9 +190,10 @@ endif
# - No Windows bloat!
#
ifeq ($(MODE), sysv)
CONFIG_CCFLAGS += $(BACKTRACES) $(FTRACE) -O2
ENABLE_FTRACE = 1
CONFIG_OFLAGS ?= -g -ggdb
CONFIG_CCFLAGS += $(BACKTRACES) -O2
CONFIG_CPPFLAGS += -DSYSDEBUG -DSUPPORT_VECTOR=121
TARGET_ARCH ?= -msse3
endif
# Tiny Mode
@@ -165,16 +220,48 @@ CONFIG_CCFLAGS += \
-fno-align-labels \
-fno-align-loops \
-fschedule-insns2 \
-fomit-frame-pointer \
-momit-leaf-frame-pointer \
-foptimize-sibling-calls \
-DDWARFLESS
CONFIG_OFLAGS += \
-g0
CONFIG_LDFLAGS += \
-S
TARGET_ARCH ?= \
-msse3
PYFLAGS += \
-O2 \
-B
endif
ifeq ($(MODE), x86_64-tiny)
CONFIG_CPPFLAGS += \
-DTINY \
-DNDEBUG \
-DTRUSTWORTHY
CONFIG_CCFLAGS += \
-Os \
-fno-align-functions \
-fno-align-jumps \
-fno-align-labels \
-fno-align-loops \
-fschedule-insns2 \
-momit-leaf-frame-pointer \
-foptimize-sibling-calls \
-DDWARFLESS
PYFLAGS += \
-O2 \
-B
endif
ifeq ($(MODE), aarch64-tiny)
# TODO(jart): -mcmodel=tiny
CONFIG_CPPFLAGS += \
-DTINY \
-DNDEBUG \
-DTRUSTWORTHY
CONFIG_CCFLAGS += \
-Os \
-fno-align-functions \
-fno-align-jumps \
-fno-align-labels \
-fno-align-loops \
-fschedule-insns2 \
-momit-leaf-frame-pointer \
-foptimize-sibling-calls \
-DDWARFLESS
PYFLAGS += \
-O2 \
-B
@@ -200,20 +287,12 @@ CONFIG_CPPFLAGS += \
-DTRUSTWORTHY \
-DSUPPORT_VECTOR=1 \
-DDWARFLESS
DEFAULT_COPTS += \
-mred-zone
CONFIG_OFLAGS += \
-g0
CONFIG_LDFLAGS += \
-S
CONFIG_CCFLAGS += \
-Os \
-fno-align-functions \
-fno-align-jumps \
-fno-align-labels \
-fno-align-loops
TARGET_ARCH ?= \
-msse3
endif
# Linux+BSD Tiny Mode
@@ -237,20 +316,12 @@ CONFIG_CPPFLAGS += \
-DTRUSTWORTHY \
-DSUPPORT_VECTOR=113 \
-DDWARFLESS
DEFAULT_COPTS += \
-mred-zone
CONFIG_OFLAGS += \
-g0
CONFIG_LDFLAGS += \
-S
CONFIG_CCFLAGS += \
-Os \
-fno-align-functions \
-fno-align-jumps \
-fno-align-labels \
-fno-align-loops
TARGET_ARCH ?= \
-msse3
endif
# Unix Tiny Mode
@@ -273,20 +344,12 @@ CONFIG_CPPFLAGS += \
-DTRUSTWORTHY \
-DSUPPORT_VECTOR=121 \
-DDWARFLESS
DEFAULT_COPTS += \
-mred-zone
CONFIG_CCFLAGS += \
-Os \
-fno-align-functions \
-fno-align-jumps \
-fno-align-labels \
-fno-align-loops
CONFIG_OFLAGS += \
-g0
CONFIG_LDFLAGS += \
-S
TARGET_ARCH ?= \
-msse3
endif
# Tiny Metallic Unix Mode
@@ -315,40 +378,40 @@ CONFIG_CCFLAGS += \
-fno-align-jumps \
-fno-align-labels \
-fno-align-loops
CONFIG_OFLAGS += \
-g0
CONFIG_LDFLAGS += \
-S
TARGET_ARCH ?= \
-msse3
endif
# GCC11 Mode
# https://justine.lol/compilers/x86_64-linux-musl__x86_64-linux-musl__g++-11.2.0.tar.xz
ifeq ($(MODE), gcc11)
.UNVEIL += rx:/opt/gcc11
CONFIG_CCFLAGS += $(BACKTRACES) $(FTRACE) -DSYSDEBUG -O2
AS = /opt/gcc11/bin/x86_64-linux-musl-as
CC = /opt/gcc11/bin/x86_64-linux-musl-gcc
CXX = /opt/gcc11/bin/x86_64-linux-musl-g++
CXXFILT = /opt/gcc11/bin/x86_64-linux-musl-c++filt
LD = /opt/gcc11/bin/x86_64-linux-musl-ld.bfd
NM = /opt/gcc11/bin/x86_64-linux-musl-nm
GCC = /opt/gcc11/bin/x86_64-linux-musl-gcc
STRIP = /opt/gcc11/bin/x86_64-linux-musl-strip
OBJCOPY = /opt/gcc11/bin/x86_64-linux-musl-objcopy
OBJDUMP = /opt/gcc11/bin/x86_64-linux-musl-objdump
ADDR2LINE = /opt/gcc11/bin/x86_64-linux-musl-addr2line
CONFIG_CCFLAGS += $(BACKTRACES) $(FTRACE) -O2 -Wno-stringop-overread
CONFIG_CFLAGS += -Wno-old-style-definition
CONFIG_CPPFLAGS += -DNDEBUG -DSYSDEBUG
TARGET_ARCH ?= -msse3
# no x87 instructions mode
#
# export MODE=nox87
# make -j8 toolchain
# cosmocc -o /tmp/hello.com hello.c
#
# lets you shave ~23kb off blink
#
# git clone https://github.com/jart/blink
# cd blink
# ./configure --disable-x87
# make -j8
# o//blink/blink /tmp/hello.com
#
ifeq ($(MODE), nox87)
ENABLE_FTRACE = 1
CONFIG_COPTS += -mlong-double-64
CONFIG_CCFLAGS += $(BACKTRACES) -O2
CONFIG_CPPFLAGS += -DSYSDEBUG -DNOX87
endif
# LLVM Mode
#
# We aim to support:
#
# make -j8 m=llvm o/llvm/libc
#
# The rest of the monorepo may not work with llvm.
#
ifeq ($(MODE), llvm)
TARGET_ARCH ?= -msse3
CONFIG_CCFLAGS += $(BACKTRACES) $(FTRACE) -DSYSDEBUG -O2
.STRICT = 0
CONFIG_CCFLAGS += $(BACKTRACES) -DSYSDEBUG -O2
AS = clang
CC = clang
CXX = clang++
@@ -386,12 +449,66 @@ endif
# such as MSVC or XCode. You can run your binary objects through a tool
# like objconv to convert them to COFF or MachO. Then use ANSI mode to
# rollup one header file that'll enable linkage with minimal issues.
ifeq ($(MODE), ansi)
CONFIG_CFLAGS += -std=c11
#CONFIG_CPPFLAGS += -ansi
CONFIG_CXXFLAGS += -std=c++11
TARGET_ARCH ?= -msse3
endif
ifneq ($(ENABLE_FTRACE),)
CONFIG_CPPFLAGS += -DFTRACE
FTRACE_CCFLAGS = -fno-inline-functions-called-once
OVERRIDE_CFLAGS += $(FTRACE_CCFLAGS)
OVERRIDE_CXXFLAGS += $(FTRACE_CCFLAGS)
# function prologue nops for --ftrace
ifeq ($(ARCH), x86_64)
# this flag causes gcc to generate functions like this
#
# nop nop nop nop nop nop nop nop nop
# func:
# nop nop
# ...
#
# which tool/build/fixupobj.c improves at build time like this
#
# nop nop nop nop nop nop nop nop nop
# func:
# xchg %ax,%ax
# ...
#
# which --ftrace morphs at runtime like this
#
# ud2 # 2 bytes
# call ftrace_hook # 5 bytes
# jmp +2 # 2 bytes
# func:
# jmp -7 # 2 bytes
# ...
#
CONFIG_CCFLAGS += -fpatchable-function-entry=18,16
endif
ifeq ($(ARCH), aarch64)
# this flag causes gcc to generate functions like this
#
# nop nop nop nop nop nop
# func:
# nop
# ...
#
# which --ftrace morphs at runtime like this
#
# brk #31337
# stp x29,x30,[sp,#-16]!
# mov x29,sp
# bl ftrace_hook
# ldp x29,x30,[sp],#16
# b +1
# func:
# b -5
# ...
#
CONFIG_CCFLAGS += -fpatchable-function-entry=7,6
endif
endif
TARGET_ARCH ?= $(CONFIG_TARGET_ARCH)
+85 -180
View File
@@ -1,5 +1,5 @@
#-*-mode:makefile-gmake;indent-tabs-mode:t;tab-width:8;coding:utf-8-*-┐
#───vi: set et ft=make ts=8 tw=8 fenc=utf-8 :vi───────────────────────┘
#── vi: set et ft=make ts=8 sw=8 fenc=utf-8 :vi ──────────────────────┘
#
# SYNOPSIS
#
@@ -33,94 +33,43 @@
#
# VARIABLES
#
# CCFLAGS gcc frontend flags (.i, .c, .cc, .f, .S, .lds, etc.)
# Our configuration variables, ordered by increasing preference:
#
# CCFLAGS frontend flags (.i, .c, .cc, .f, .S, .lds, etc.)
# OFLAGS objectify flags (precludes -S and -E)
# CPPFLAGS preprocessor flags (.h, .c, .cc, .S, .inc, .lds, etc.)
# TARGET_ARCH microarchitecture flags (e.g. -march=native)
# COPTS c/c++ flags (.c, .cc)
# CFLAGS c flags (.c only)
# CXXFLAGS c++ flags (.cc only)
# COPTS c/c++ flags (.c, .cc)
# LDFLAGS linker flags (don't use -Wl, frontend prefix)
# ASFLAGS assembler flags (don't use -Wa, frontend prefix)
# TARGET_ARCH microarchitecture flags (e.g. -march=native)
LC_ALL = C
SOURCE_DATE_EPOCH = 0
ARFLAGS = rcsD
ZFLAGS ?=
XARGS ?= xargs -P4 -rs8000
DOT ?= dot
CLANG = clang
FC = gfortran #/opt/cross9f/bin/x86_64-linux-musl-gfortran
TMPDIR = o/tmp
AR = build/bootstrap/ar.com
CP = build/bootstrap/cp.com
RM = build/bootstrap/rm.com -f
ECHO = build/bootstrap/echo.com
TOUCH = build/bootstrap/touch.com
PKG = build/bootstrap/package.com
MKDEPS = build/bootstrap/mkdeps.com
ZIPOBJ = build/bootstrap/zipobj.com
FIXUPOBJ = build/bootstrap/fixupobj.com
MKDIR = build/bootstrap/mkdir.com -p
COMPILE = build/bootstrap/compile.com -V9 -P4096 $(QUOTA)
COMMA := ,
PWD := $(shell build/bootstrap/pwd.com)
IMAGE_BASE_VIRTUAL ?= 0x400000
IGNORE := $(shell $(ECHO) -2 ♥cosmo)
IGNORE := $(shell $(MKDIR) o/tmp)
ifneq ("$(wildcard o/third_party/gcc/bin/x86_64-pc-linux-gnu-*)","")
PREFIX = o/third_party/gcc/bin/x86_64-pc-linux-gnu-
else
IGNORE := $(shell build/bootstrap/unbundle.com)
PREFIX = o/third_party/gcc/bin/x86_64-linux-musl-
endif
AS = $(PREFIX)as
CC = $(PREFIX)gcc
CXX = $(PREFIX)g++
CXXFILT = $(PREFIX)c++filt
LD = $(PREFIX)ld.bfd
NM = $(PREFIX)nm
GCC = $(PREFIX)gcc
STRIP = $(PREFIX)strip
OBJCOPY = $(PREFIX)objcopy
OBJDUMP = $(PREFIX)objdump
ADDR2LINE = $(PWD)/$(PREFIX)addr2line
export ADDR2LINE
export LC_ALL
export MKDIR
export MODE
export SOURCE_DATE_EPOCH
export TMPDIR
#
# For each FOO above, there exists (by increasing preference)
#
# DEFAULT_FOO see build/definitions.mk
# CONFIG_FOO see build/config.mk
# FOO set ~/.cosmo.mk and target-specific
# OVERRIDE_FOO set ~/.cosmo.mk and target-specific (use rarely)
#
ifeq ($(LANDLOCKMAKE_VERSION),)
TMPSAFE = $(TMPDIR)/$(subst /,_,$@).tmp
TMPSAFE = $(join $(TMPDIR)/,$(subst /,_,$@)).tmp
else
TMPSAFE = $(TMPDIR)/
endif
FTRACE = \
-pg
BACKTRACES = \
-fno-schedule-insns2 \
-fno-omit-frame-pointer \
-fno-optimize-sibling-calls \
-mno-omit-leaf-frame-pointer
SANITIZER = \
-fsanitize=address
NO_MAGIC = \
-mno-fentry \
-ffreestanding \
-fno-stack-protector \
-fwrapv \
-fno-sanitize=all
-fno-sanitize=all \
-fpatchable-function-entry=0,0
OLD_CODE = \
-fno-strict-aliasing \
@@ -131,78 +80,108 @@ TRADITIONAL = \
-Wno-return-type \
-Wno-pointer-sign
DEFAULT_CCFLAGS = \
DEFAULT_CCFLAGS += \
-Wall \
-Werror \
-fdebug-prefix-map='$(PWD)'= \
-fno-omit-frame-pointer \
-frecord-gcc-switches
DEFAULT_OFLAGS = \
-g \
-gdwarf-4 \
-gdescribe-dies
DEFAULT_COPTS = \
-mno-red-zone \
-fno-math-errno \
-fno-trapping-math \
-fno-fp-int-builtin-inexact \
DEFAULT_COPTS ?= \
-fno-ident \
-fno-common \
-fno-gnu-unique \
-fstrict-aliasing \
-fstrict-overflow \
-fno-semantic-interposition \
-fno-dwarf2-cfi-asm \
-fno-unwind-tables \
-fno-asynchronous-unwind-tables
ifeq ($(ARCH), x86_64)
# Microsoft says "[a]ny memory below the stack beyond the red zone
# [note: Windows defines the x64 red zone size as 0] is considered
# volatile and may be modified by the operating system at any time."
# https://devblogs.microsoft.com/oldnewthing/20190111-00/?p=100685
DEFAULT_COPTS += \
-mno-red-zone \
-mno-tls-direct-seg-refs
endif
ifeq ($(ARCH), aarch64)
#
# - Apple says in "Writing ARM64 code for Apple platforms" that we're
# not allowed to use the x18 register.
#
# - Cosmopolitan Libc uses x28 for thread-local storage because Apple
# forbids us from using tpidr_el0 too.
#
# - Cosmopolitan currently lacks an implementation of the runtime
# libraries needed by the -moutline-atomics flag
#
DEFAULT_COPTS += \
-ffixed-x18 \
-ffixed-x28 \
-fsigned-char \
-mno-outline-atomics
endif
MATHEMATICAL = \
-O3 \
-fwrapv
DEFAULT_CPPFLAGS = \
-DCOSMO \
DEFAULT_CPPFLAGS += \
-D_COSMO_SOURCE \
-DMODE='"$(MODE)"' \
-DIMAGE_BASE_VIRTUAL=$(IMAGE_BASE_VIRTUAL) \
-Wno-prio-ctor-dtor \
-Wno-unknown-pragmas \
-nostdinc \
-iquote .
-iquote. \
-isystem libc/isystem
DEFAULT_CFLAGS = \
-std=gnu2x
-std=gnu23
DEFAULT_CXXFLAGS = \
-std=gnu++23 \
-fno-rtti \
-fno-exceptions \
-fuse-cxa-atexit \
-fno-threadsafe-statics \
-Wno-int-in-bool-context \
-Wno-narrowing \
-Wno-literal-suffix
-Wno-literal-suffix \
-isystem third_party/libcxx
DEFAULT_ASFLAGS = \
-W \
-I. \
--noexecstack \
--nocompress-debug-sections
--noexecstack
DEFAULT_LDFLAGS = \
-static \
-nostdlib \
-melf_x86_64 \
-znorelro \
--gc-sections \
-z noexecstack \
--build-id=none \
--no-dynamic-linker \
-zmax-page-size=0x1000 #--cref [email protected]
--no-dynamic-linker
ZIPOBJ_FLAGS = \
-b$(IMAGE_BASE_VIRTUAL)
# # generate linker report files
# DEFAULT_LDFLAGS += --cref [email protected]
PYFLAGS = \
-b$(IMAGE_BASE_VIRTUAL)
ifeq ($(ARCH), aarch64)
DEFAULT_LDFLAGS += \
-zmax-page-size=0x4000 \
-zcommon-page-size=0x4000 \
-znorelro
else
DEFAULT_LDFLAGS += \
-zmax-page-size=0x4000 \
-zcommon-page-size=0x1000
endif
ASONLYFLAGS = \
-c \
-g \
--debug-prefix-map='$(PWD)'=
-g
DEFAULT_LDLIBS =
@@ -276,40 +255,31 @@ LD.libs = \
$(CONFIG_LIBS) \
$(LIBS)
COMPILE.c.flags = $(cc.flags) $(cpp.flags) $(copt.flags) $(c.flags)
COMPILE.cxx.flags = $(cc.flags) $(cpp.flags) $(copt.flags) $(cxx.flags)
COMPILE.f.flags = $(cc.flags) $(copt.flags) $(f.flags)
COMPILE.F.flags = $(cc.flags) $(cpp.flags) $(copt.flags) $(f.flags)
COMPILE.c.flags = $(cc.flags) $(copt.flags) $(cpp.flags) $(c.flags)
COMPILE.cxx.flags = $(cc.flags) $(copt.flags) $(cxx.flags) $(cpp.flags)
COMPILE.i.flags = $(cc.flags) $(copt.flags) $(c.flags)
COMPILE.ii.flags = $(cc.flags) $(copt.flags) $(cxx.flags)
LINK.flags = $(DEFAULT_LDFLAGS) $(CONFIG_LDFLAGS) $(LDFLAGS)
OBJECTIFY.c.flags = $(OBJECTIFY.S.flags) $(copt.flags) $(c.flags)
OBJECTIFY.cxx.flags = $(OBJECTIFY.S.flags) $(copt.flags) $(cxx.flags)
OBJECTIFY.c.flags = $(cc.flags) $(o.flags) $(S.flags) $(cpp.flags) $(copt.flags) $(c.flags)
OBJECTIFY.cxx.flags = $(cc.flags) $(o.flags) $(S.flags) $(cxx.flags) $(cpp.flags) $(copt.flags)
OBJECTIFY.s.flags = $(ASONLYFLAGS) $(s.flags)
OBJECTIFY.S.flags = $(copt.flags) $(cc.flags) $(o.flags) $(cpp.flags) $(S.flags)
OBJECTIFY.f.flags = $(copt.flags) $(cc.flags) $(o.flags) $(copt.flags) $(S.flags) $(f.flags)
OBJECTIFY.F.flags = $(OBJECTIFY.f.flags) $(cpp.flags)
OBJECTIFY.S.flags = $(cc.flags) $(o.flags) $(S.flags) $(cpp.flags)
PREPROCESS.flags = -E $(copt.flags) $(cc.flags) $(cpp.flags)
PREPROCESS.lds.flags = -D__LINKER__ $(filter-out -g%,$(PREPROCESS.flags)) -P -xc
COMPILE.c = $(CC) -S $(COMPILE.c.flags)
COMPILE.cxx = $(CXX) -S $(COMPILE.cxx.flags)
COMPILE.i = $(CC) -S $(COMPILE.i.flags)
COMPILE.f = $(FC) -S $(COMPILE.f.flags)
COMPILE.F = $(FC) -S $(COMPILE.F.flags)
OBJECTIFY.s = $(AS) $(OBJECTIFY.s.flags)
OBJECTIFY.S = $(CC) $(OBJECTIFY.S.flags) -c
OBJECTIFY.f = $(FC) $(OBJECTIFY.f.flags) -c
OBJECTIFY.F = $(FC) $(OBJECTIFY.F.flags) -c
OBJECTIFY.c = $(CC) $(OBJECTIFY.c.flags) -c
OBJECTIFY.cxx = $(CXX) $(OBJECTIFY.cxx.flags) -c
PREPROCESS = $(CC) $(PREPROCESS.flags)
PREPROCESS.lds = $(CC) $(PREPROCESS.lds.flags)
LINK = $(LD) $(LINK.flags)
ELF = o/libc/elf/elf.lds
ELFLINK = $(COMPILE) -ALINK.elf $(LINK) $(LINKARGS) $(OUTPUT_OPTION)
ELFLINK = $(COMPILE) -ALINK.elf $(LINK) $(LINKARGS) $(OUTPUT_OPTION) && $(COMPILE) -AFIXUP.ape -T$@ $(FIXUPOBJ) $@
LINKARGS = $(patsubst %.lds,-T %.lds,$(call uniqr,$(LD.libs) $(filter-out %.pkg,$^)))
LOLSAN = build/lolsan -b $(IMAGE_BASE_VIRTUAL)
# The compiler won't generate %xmm code for sources extensioned .greg.c,
# which is needed for C modules wanting to run at the executive level or
@@ -324,74 +294,9 @@ OBJECTIFY.greg.c = \
-fno-optimize-sibling-calls \
-fno-sanitize=all \
-ffreestanding \
-mno-fentry \
-fwrapv \
-c
OBJECTIFY.ansi.c = $(CC) $(OBJECTIFY.c.flags) -ansi -Wextra -Werror -pedantic-errors -c
OBJECTIFY.c99.c = $(CC) $(OBJECTIFY.c.flags) -std=c99 -Wextra -Werror -pedantic-errors -c
OBJECTIFY.c11.c = $(CC) $(OBJECTIFY.c.flags) -std=c11 -Wextra -Werror -pedantic-errors -c
OBJECTIFY.c2x.c = $(CC) $(OBJECTIFY.c.flags) -std=c2x -Wextra -Werror -pedantic-errors -c
OBJECTIFY.real.c = \
$(GCC) \
-x-no-pg \
$(OBJECTIFY.c.flags) \
-wrapper build/realify.sh \
-D__REAL_MODE__ \
-ffixed-r8 \
-ffixed-r9 \
-ffixed-r10 \
-ffixed-r11 \
-ffixed-r12 \
-ffixed-r13 \
-ffixed-r14 \
-ffixed-r15 \
-mno-red-zone \
-fcall-used-rbx \
-fno-jump-tables \
-fno-shrink-wrap \
-fno-schedule-insns2 \
-flive-range-shrinkage \
-fno-omit-frame-pointer \
-momit-leaf-frame-pointer \
-mpreferred-stack-boundary=3 \
-fno-delete-null-pointer-checks \
-c
OBJECTIFY.ncabi.c = \
$(GCC) \
$(OBJECTIFY.c.flags) \
-mno-sse \
-mfpmath=387 \
-mno-fentry \
-fno-stack-protector \
-fno-instrument-functions \
-fno-optimize-sibling-calls \
-fno-sanitize=all \
-fcall-saved-rcx \
-fcall-saved-rdx \
-fcall-saved-rdi \
-fcall-saved-rsi \
-fcall-saved-r8 \
-fcall-saved-r9 \
-fcall-saved-r10 \
-fcall-saved-r11 \
-c \
-xc
OBJECTIFY.initabi.c = \
$(GCC) \
$(OBJECTIFY.c.flags) \
-mno-fentry \
-fno-stack-protector \
-fno-instrument-functions \
-fno-optimize-sibling-calls \
-fno-sanitize=all \
-fcall-saved-rdi \
-fcall-saved-rsi \
-c
TAGSFLAGS = \
-e \
-a \
+101
View File
@@ -0,0 +1,101 @@
#!/bin/sh
# cosmocc downloader script
# https://justine.lol/cosmo3/#install
# https://github.com/jart/cosmopolitan/blob/master/tool/cosmocc/README.md
# collect arguments
OUTPUT_DIR=${1:?OUTPUT_DIR}
COSMOCC_VERSION=${2:?COSMOCC_VERSION}
COSMOCC_SHA256SUM=${3:?COSMOCC_SHA256SUM}
URL1="https://github.com/jart/cosmopolitan/releases/download/${COSMOCC_VERSION}/cosmocc-${COSMOCC_VERSION}.zip"
URL2="https://cosmo.zip/pub/cosmocc/cosmocc-${COSMOCC_VERSION}.zip"
# helper function
abort() {
printf '%s\n' "download terminated." >&2
exit 1
}
# exit if already downloaded
# we need it because directory timestamps work wierdly
OUTPUT_DIR=${OUTPUT_DIR%/}
if [ -d "${OUTPUT_DIR}" ]; then
exit 0
fi
# find commands we need to securely download cosmocc
if ! UNZIP=$(command -v unzip 2>/dev/null); then
printf '%s\n' "$0: fatal error: you need the unzip command" >&2
printf '%s\n' "please download https://cosmo.zip/pub/cosmos/bin/unzip and put it on the system path" >&2
abort
fi
if command -v sha256sum >/dev/null 2>&1; then
# can use system sha256sum
true
elif command -v shasum >/dev/null 2>&1; then
sha256sum() {
shasum -a 256 "$@"
}
else
if [ ! -f build/sha256sum.c ]; then
printf '%s\n' "$0: fatal error: you need to install sha256sum" >&2
printf '%s\n' "please download https://cosmo.zip/pub/cosmos/bin/sha256sum and put it on the system path" >&2
abort
fi
if ! SHA256SUM=$(command -v "$PWD/o/build/sha256sum" 2>/dev/null); then
if ! CC=$(command -v "$CC" 2>/dev/null); then
if ! CC=$(command -v cc 2>/dev/null); then
if ! CC=$(command -v cosmocc 2>/dev/null); then
printf '%s\n' "$0: fatal error: you need to install either sha256sum, cc, or cosmocc" >&2
printf '%s\n' "please download https://cosmo.zip/pub/cosmos/bin/sha256sum and put it on the system path" >&2
abort
fi
fi
fi
mkdir -p o/build || abort
SHA256SUM="$PWD/o/build/sha256sum"
printf '%s\n' "${CC} -w -O2 -o ${SHA256SUM} build/sha256sum.c" >&2
"${CC}" -w -O2 -o "${SHA256SUM}.$$" build/sha256sum.c || abort
mv -f "${SHA256SUM}.$$" "${SHA256SUM}" || abort
fi
sha256sum() {
"${SHA256SUM}" "$@"
}
fi
if WGET=$(command -v wget 2>/dev/null); then
DOWNLOAD=$WGET
DOWNLOAD_ARGS=-O
elif CURL=$(command -v curl 2>/dev/null); then
DOWNLOAD=$CURL
DOWNLOAD_ARGS=-fLo
else
printf '%s\n' "$0: fatal error: you need to install either wget or curl" >&2
printf '%s\n' "please download https://cosmo.zip/pub/cosmos/bin/wget and put it on the system path" >&2
abort
fi
# create temporary output directory
OLDPWD=$PWD
OUTPUT_TMP="${OUTPUT_DIR}.tmp.$$/"
mkdir -p "${OUTPUT_TMP}" || abort
cd "${OUTPUT_TMP}"
die() {
cd "${OLDPWD}"
rm -rf "${OUTPUT_TMP}"
abort
}
# download cosmocc toolchain
# multiple urls avoids outages and national firewalls
if ! "${DOWNLOAD}" ${DOWNLOAD_ARGS} cosmocc.zip "${URL1}"; then
rm -f cosmocc.zip
"${DOWNLOAD}" ${DOWNLOAD_ARGS} cosmocc.zip "${URL2}" || die
fi
printf '%s\n' "${COSMOCC_SHA256SUM} *cosmocc.zip" >cosmocc.zip.sha256sum
sha256sum -c cosmocc.zip.sha256sum || die
"${UNZIP}" cosmocc.zip || die
rm -f cosmocc.zip cosmocc.zip.sha256sum
# commit output directory
cd "${OLDPWD}" || die
mv "${OUTPUT_TMP}" "${OUTPUT_DIR}" || die
+1 -1
View File
@@ -1,5 +1,5 @@
#-*-mode:makefile-gmake;indent-tabs-mode:t;tab-width:8;coding:utf-8-*-┐
#───vi: set et ft=make ts=8 tw=8 fenc=utf-8 :vi───────────────────────┘
#── vi: set et ft=make ts=8 sw=8 fenc=utf-8 :vi ──────────────────────┘
#
# SYNOPSIS
#
+3 -3
View File
@@ -1,6 +1,6 @@
#!/bin/sh
#-*-mode:sh;indent-tabs-mode:nil;tab-width:2;coding:utf-8-*-┐
#───vi: set net ft=sh ts=2 sts=2 fenc=utf-8 :vi─────────────┘
#── vi: set et ft=sh ts=2 sts=2 fenc=utf-8 :vi ─────────────┘
#
# OVERVIEW
#
@@ -55,7 +55,7 @@ set -- --regex-c='/_Atomic(\([^)]*\))/\1/b' "$@"
set -- --regex-c='/^\(\(hidden\|extern\|const\) \)*[_[:alpha:]][_[:alnum:]]*[ *][ *]*\([_[:alpha:]][_[:alnum:]]*[ *][ *]*\)*\([_[:alpha:]][_$[:alnum:]]*\)/\4/b' "$@"
# ctags doesn't understand function prototypes, e.g.
# bool isheap(void *p) dontthrow nocallback;
# bool isheap(void *p) dontthrow dontcallback;
set -- --regex-c='/^[_[:alpha:]][_[:alnum:]]*[ *][ *]*\([_[:alpha:]][_[:alnum:]]*[ *][ *]*\)*\([_[:alpha:]][_$[:alnum:]]*\)(.*/\2/b' "$@"
# ctags doesn't understand function pointers, e.g.
@@ -66,7 +66,7 @@ set -- --regex-c='/^extern [^(]*(\*const \([^)]*\))(/\1/b' "$@"
# struct WorstSoftwareEver;
set -- --regex-c='/^struct.*;$/uehocruehcroue/b' "$@"
exec $TAGS \
build/run $TAGS \
-e \
--langmap=c:.c.h \
--exclude=libc/nt/struct/imagefileheader.internal.h \
Executable
+29
View File
@@ -0,0 +1,29 @@
#!/bin/sh
COSMO=${COSMO:-/opt/cosmo}
if [ -n "$OBJDUMP" ]; then
exec "$OBJDUMP" "$@"
fi
find_objdump() {
if [ -x .cosmocc/3.6.0/bin/$1-linux-cosmo-objdump ]; then
OBJDUMP=.cosmocc/3.6.0/bin/$1-linux-cosmo-objdump
elif [ -x .cosmocc/3.6.0/bin/$1-linux-musl-objdump ]; then
OBJDUMP=.cosmocc/3.6.0/bin/$1-linux-musl-objdump
elif [ -x "$COSMO/.cosmocc/3.6.0/bin/$1-linux-cosmo-objdump" ]; then
OBJDUMP="$COSMO/.cosmocc/3.6.0/bin/$1-linux-cosmo-objdump"
elif [ -x "$COSMO/.cosmocc/3.6.0/bin/$1-linux-musl-objdump" ]; then
OBJDUMP="$COSMO/.cosmocc/3.6.0/bin/$1-linux-musl-objdump"
else
echo "error: toolchain not found (try running 'cosmocc --update' or 'make' in the cosmo monorepo)" >&2
exit 1
fi
}
if printf '%s\n' "$*" | grep aarch64 >/dev/null 2>&1; then
find_objdump aarch64
exec "$OBJDUMP" "$@"
else
find_objdump x86_64
exec "$OBJDUMP" "$@"
fi
+11 -7
View File
@@ -1,5 +1,5 @@
#-*-mode:makefile-gmake;indent-tabs-mode:t;tab-width:8;coding:utf-8-*-┐
#───vi: set et ft=make ts=8 tw=8 fenc=utf-8 :vi───────────────────────┘
#── vi: set et ft=make ts=8 sw=8 fenc=utf-8 :vi ──────────────────────┘
#
# SYNOPSIS
#
@@ -22,10 +22,14 @@
# - tool/build/runit.c
# - tool/build/runitd.c
.PRECIOUS: o/$(MODE)/%.com.ok
o/$(MODE)/%.com.ok: .PLEDGE = stdio rpath wpath cpath proc fattr inet
o/$(MODE)/%.com.ok: \
o/$(MODE)/tool/build/runit.com \
o/$(MODE)/tool/build/runitd.com \
o/$(MODE)/%.com
.PRECIOUS: o/$(MODE)/%.ok
o/$(MODE)/%.ok: private .PLEDGE = stdio rpath wpath cpath proc fattr inet dns
o/$(MODE)/%.ok: private .UNVEIL += r:/etc/resolv.conf
o/$(MODE)/%.ok: \
o/$(MODE)/tool/build/runit \
o/$(MODE)/tool/build/runitd \
o/$(MODE)/%
@$(COMPILE) -wATEST -tT$@ $^ $(HOSTS)
.PHONY:
o/tiny/tool/build/runit:
+1 -1
View File
@@ -1,5 +1,5 @@
#-*-mode:sed;indent-tabs-mode:t;tab-width:8;coding:utf-8-*-┐
#───vi: et ft=sed ts=8 tw=8 fenc=utf-8 :vi─────────────────┘
#── vi: et ft=sed ts=8 sw=8 fenc=utf-8 :vi ────────────────┘
#
# SYNOPSIS
#
+54 -74
View File
@@ -1,5 +1,5 @@
#-*-mode:makefile-gmake;indent-tabs-mode:t;tab-width:8;coding:utf-8-*-┐
#───vi: set et ft=make ts=8 tw=8 fenc=utf-8 :vi───────────────────────┘
#── vi: set et ft=make ts=8 sw=8 fenc=utf-8 :vi ──────────────────────┘
#
# SYNOPSIS
#
@@ -16,40 +16,15 @@
MAKEFLAGS += --no-builtin-rules
o/%.o: %.s ; @$(COMPILE) -AOBJECTIFY.s $(OBJECTIFY.s) $(OUTPUT_OPTION) $<
o/%.o: o/%.s ; @$(COMPILE) -AOBJECTIFY.s $(OBJECTIFY.s) $(OUTPUT_OPTION) $<
o/%.s: %.S ; @$(COMPILE) -APREPROCESS $(PREPROCESS) $(OUTPUT_OPTION) $<
o/%.s: o/%.S ; @$(COMPILE) -APREPROCESS $(PREPROCESS) $(OUTPUT_OPTION) $<
o/%.o: %.S ; @$(COMPILE) -AOBJECTIFY.S $(OBJECTIFY.S) $(OUTPUT_OPTION) $<
o/%.o: o/%.S ; @$(COMPILE) -AOBJECTIFY.S $(OBJECTIFY.S) $(OUTPUT_OPTION) $<
o/%.lds: %.lds ; @$(COMPILE) -APREPROCESS $(PREPROCESS.lds) $(OUTPUT_OPTION) $<
o/%.inc: %.h ; @$(COMPILE) -APREPROCESS $(PREPROCESS) $(OUTPUT_OPTION) -D__ASSEMBLER__ -P $<
o/%.greg.o: %.greg.c ; @$(COMPILE) -AOBJECTIFY.greg $(OBJECTIFY.greg.c) $(OUTPUT_OPTION) $<
MAKE_ZIPCOPY = $(COMPILE) -AZIPCOPY -wT$@ $(ZIPCOPY) $< $@
ifneq ($(ARCH), aarch64)
MAKE_OBJCOPY = $(COMPILE) -AOBJCOPY -T$@ $(OBJCOPY) -S -O binary $< $@ && $(MAKE_ZIPCOPY)
else
MAKE_OBJCOPY = $(COMPILE) -AOBJCOPY -T$@ $(OBJCOPY) -S $< $@ && $(MAKE_ZIPCOPY)
endif
o/$(MODE)/%: o/$(MODE)/%.dbg ; @$(COMPILE) -AOBJCOPY -T$@ $(OBJCOPY) -S -O binary $< $@
o/$(MODE)/%.o: %.s ; @$(COMPILE) -AOBJECTIFY.s $(OBJECTIFY.s) $(OUTPUT_OPTION) $<
o/$(MODE)/%.o: o/$(MODE)/%.s ; @$(COMPILE) -AOBJECTIFY.s $(OBJECTIFY.s) $(OUTPUT_OPTION) $<
o/$(MODE)/%.o: %.f ; @$(COMPILE) -AOBJECTIFY.f $(OBJECTIFY.f) $(OUTPUT_OPTION) $<
o/$(MODE)/%.o: %.F ; @$(COMPILE) -AOBJECTIFY.F $(OBJECTIFY.F) $(OUTPUT_OPTION) $<
o/$(MODE)/%.h: %.c ; @$(COMPILE) -AAMALGAMATE $(PREPROCESS) $(OUTPUT_OPTION) -fdirectives-only -P $<
o/$(MODE)/%.h: o/$(MODE)/%.c ; @$(COMPILE) -AAMALGAMATE $(PREPROCESS) $(OUTPUT_OPTION) -fdirectives-only -P $<
o/$(MODE)/%.o: %.S ; @$(COMPILE) -AOBJECTIFY.S $(OBJECTIFY.S) $(OUTPUT_OPTION) $<
o/$(MODE)/%.o: o/$(MODE)/%.S ; @$(COMPILE) -AOBJECTIFY.S $(OBJECTIFY.S) $(OUTPUT_OPTION) $<
o/$(MODE)/%.o: %.cc ; @$(COMPILE) -AOBJECTIFY.cxx $(OBJECTIFY.cxx) $(OUTPUT_OPTION) $<
o/$(MODE)/%.lds: %.lds ; @$(COMPILE) -APREPROCESS $(PREPROCESS.lds) $(OUTPUT_OPTION) $<
o/$(MODE)/%.o: %.greg.c ; @$(COMPILE) -AOBJECTIFY.greg $(OBJECTIFY.greg.c) $(OUTPUT_OPTION) $<
o/$(MODE)/%.greg.o: %.greg.c ; @$(COMPILE) -AOBJECTIFY.greg $(OBJECTIFY.greg.c) $(OUTPUT_OPTION) $<
o/$(MODE)/%.initabi.o: %.initabi.c ; @$(COMPILE) -AOBJECTIFY.init $(OBJECTIFY.initabi.c) $(OUTPUT_OPTION) $<
o/$(MODE)/%.ncabi.o: %.ncabi.c ; @$(COMPILE) -AOBJECTIFY.nc $(OBJECTIFY.ncabi.c) $(OUTPUT_OPTION) $<
o/$(MODE)/%.real.o: %.c ; @$(COMPILE) -AOBJECTIFY.real $(OBJECTIFY.real.c) $(OUTPUT_OPTION) $<
o/%.o: %.cc
@$(COMPILE) -AOBJECTIFY.cxx $(OBJECTIFY.cxx) $(OUTPUT_OPTION) $<
@$(COMPILE) -AFIXUPOBJ -wT$@ $(FIXUPOBJ) $@
o/%.o: o/%.cc
@$(COMPILE) -AOBJECTIFY.cxx $(OBJECTIFY.cxx) $(OUTPUT_OPTION) $<
@$(COMPILE) -AFIXUPOBJ -wT$@ $(FIXUPOBJ) $@
o/$(MODE)/%: o/$(MODE)/%.dbg
@$(MAKE_OBJCOPY)
o/$(MODE)/%.o: %.c
@$(COMPILE) -AOBJECTIFY.c $(OBJECTIFY.c) $(OUTPUT_OPTION) $<
@@ -59,10 +34,39 @@ o/$(MODE)/%.o: o/$(MODE)/%.c
@$(COMPILE) -AOBJECTIFY.c $(OBJECTIFY.c) $(OUTPUT_OPTION) $<
@$(COMPILE) -AFIXUPOBJ -wT$@ $(FIXUPOBJ) $@
o/$(MODE)/%.o: o/$(MODE)/%.cc
o/$(MODE)/%.h: %.c
@$(COMPILE) -AAMALGAMATE $(PREPROCESS) $(OUTPUT_OPTION) -fdirectives-only -P $<
o/$(MODE)/%.o: %.cc
@$(COMPILE) -AOBJECTIFY.cxx $(OBJECTIFY.cxx) $(OUTPUT_OPTION) $<
@$(COMPILE) -AFIXUPOBJ -wT$@ $(FIXUPOBJ) $@
o/$(MODE)/%.o: %.cpp
@$(COMPILE) -AOBJECTIFY.cxx $(OBJECTIFY.cxx) $(OUTPUT_OPTION) $<
o/$(MODE)/%.lds: %.lds
@$(COMPILE) -APREPROCESS $(PREPROCESS.lds) $(OUTPUT_OPTION) $<
ifneq ($(ARCH), aarch64)
o/$(MODE)/%.o: %.S
@$(COMPILE) -AOBJECTIFY.S $(OBJECTIFY.S) $(OUTPUT_OPTION) $<
o/$(MODE)/%.o: o/$(MODE)/%.S
@$(COMPILE) -AOBJECTIFY.S $(OBJECTIFY.S) $(OUTPUT_OPTION) $<
else
o/$(MODE)/%.o: %.S libc/empty.s
@$(COMPILE) -AOBJECTIFY.S $(OBJECTIFY.s) $(OUTPUT_OPTION) libc/empty.s
o/$(MODE)/%.o: o/$(MODE)/%.S libc/empty.s
@$(COMPILE) -AOBJECTIFY.S $(OBJECTIFY.s) $(OUTPUT_OPTION) libc/empty.s
endif
o/$(MODE)/%.o: %.greg.c
@$(COMPILE) -AOBJECTIFY.greg $(OBJECTIFY.greg.c) $(OUTPUT_OPTION) $<
@$(COMPILE) -AFIXUPOBJ -wT$@ $(FIXUPOBJ) $@
o/$(MODE)/%.greg.o: %.greg.c
@$(COMPILE) -AOBJECTIFY.greg $(OBJECTIFY.greg.c) $(OUTPUT_OPTION) $<
@$(COMPILE) -AFIXUPOBJ -wT$@ $(FIXUPOBJ) $@
o/%.a:
$(file >$(TMPDIR)/$(subst /,_,$@),$^)
@$(COMPILE) -AARCHIVE -wT$@ $(AR) $(ARFLAGS) $@ @$(TMPDIR)/$(subst /,_,$@)
@@ -75,21 +79,17 @@ o/$(MODE)/%.pkg:
$(file >$(TMPSAFE).args,$(filter %.o,$^))
@$(COMPILE) -APACKAGE -wT$@ $(PKG) $(OUTPUT_OPTION) $(addprefix -d,$(filter %.pkg,$^)) @$(TMPSAFE).args
o/$(MODE)/%.o: %.py o/$(MODE)/third_party/python/pyobj.com
@$(COMPILE) -wAPYOBJ o/$(MODE)/third_party/python/pyobj.com $(PYFLAGS) -o $@ $<
o/$(MODE)/%.o: %.py o/$(MODE)/third_party/python/pyobj
@$(COMPILE) -wAPYOBJ o/$(MODE)/third_party/python/pyobj $(PYFLAGS) -o $@ $<
o/$(MODE)/%.pyc: %.py o/$(MODE)/third_party/python/pycomp.com
@$(COMPILE) -wAPYCOMP o/$(MODE)/third_party/python/pycomp.com $(PYCFLAGS) -o $@ $<
o/$(MODE)/%.pyc: %.py o/$(MODE)/third_party/python/pycomp
@$(COMPILE) -wAPYCOMP o/$(MODE)/third_party/python/pycomp $(PYCFLAGS) -o $@ $<
o/$(MODE)/%.lua: %.lua o/$(MODE)/third_party/lua/luac.com
@$(COMPILE) -wALUAC o/$(MODE)/third_party/lua/luac.com -s -o $@ $<
o/$(MODE)/%.lua: %.lua o/$(MODE)/third_party/lua/luac
@$(COMPILE) -wALUAC o/$(MODE)/third_party/lua/luac -s -o $@ $<
o/$(MODE)/%.lua.runs: %.lua o/$(MODE)/tool/net/redbean.com
@$(COMPILE) -wALUA -tT$@ o/$(MODE)/tool/net/redbean.com $(LUAFLAGS) -i $<
o/$(MODE)/%: o/$(MODE)/%.com o/$(MODE)/tool/build/cp.com o/$(MODE)/tool/build/assimilate.com
@$(COMPILE) -wACP -T$@ o/$(MODE)/tool/build/cp.com $< $@
@$(COMPILE) -wAASSIMILATE -T$@ o/$(MODE)/tool/build/assimilate.com $@
o/$(MODE)/%.lua.runs: %.lua o/$(MODE)/tool/net/redbean
@$(COMPILE) -wALUA -tT$@ o/$(MODE)/tool/net/redbean $(LUAFLAGS) -i $<
################################################################################
# LOCAL UNIT TESTS
@@ -108,22 +108,22 @@ o/$(MODE)/%: o/$(MODE)/%.com o/$(MODE)/tool/build/cp.com o/$(MODE)/tool/build/as
#
# You could then run a command like:
#
# make -j8 o//test/libc/calls/openbsd_test.com.runs
# make -j8 o//test/libc/calls/openbsd_test.runs
#
# You need PROT_EXEC permission since ftrace morphs the binary. It's
# also worth mentioning that the pledge.com command can simulate what
# also worth mentioning that the pledge command can simulate what
# Landlock Make does:
#
# o//tool/build/pledge.com \
# o//tool/build/pledge \
# -v. -p 'stdio rpath wpath cpath tty prot_exec' \
# o//test/libc/calls/openbsd_test.com \
# o//test/libc/calls/openbsd_test \
# ----ftrace
#
# This is useful in the event a test binary should run by itself, but
# fails to run beneath Landlock Make. It's also useful sometimes to
# override the verbosity when running tests:
#
# make V=5 TESTARGS=-b o//test/libc/calls/openbsd_test.com.runs
# make V=5 TESTARGS=-b o//test/libc/calls/openbsd_test.runs
#
# This way, if for some reason a test should fail but calls exit(0),
# then the stdout/stderr output, which would normally be suppressed,
@@ -135,7 +135,7 @@ o/$(MODE)/%.runs: o/$(MODE)/%
################################################################################
# ELF ZIP FILES
#
# zipobj.com lets us do fast incremental linking of compressed data.
# zipobj lets us do fast incremental linking of compressed data.
# it's nice because if we link a hundred binaries that use the time zone
# database, then that database only needs to be DEFLATE'd once.
@@ -183,27 +183,6 @@ o/$(MODE)/%.okk: private .UNSANDBOXED = 1
o/$(MODE)/%.okk: %
@$(COMPILE) -ACHECK.h $(COMPILE.cxx) -xc++ -g0 -o $@ $<
################################################################################
# EXECUTABLE HELPERS
MAKE_OBJCOPY = \
$(COMPILE) -AOBJCOPY -T$@ \
$(OBJCOPY) -S -O binary $< $@
MAKE_SYMTAB_CREATE = \
$(COMPILE) -wASYMTAB \
o/$(MODE)/tool/build/symtab.com \
-o $(TMPSAFE)/.symtab \
$<
MAKE_SYMTAB_ZIP = \
$(COMPILE) -AZIP -T$@ \
o/$(MODE)/third_party/zip/zip.com \
-b$(TMPDIR) \
-9qj \
$@ \
$(TMPSAFE)/.symtab
################################################################################
# EMACS ASSEMBLY GENERATION
@@ -219,6 +198,7 @@ o/$(MODE)/%-clang.asm: private .UNSANDBOXED = 1
o/$(MODE)/%-clang.asm: %.c
@$(COMPILE) -AOBJECTIFY.c $(OBJECTIFY.c) -S -g0 $(OUTPUT_OPTION) $<
# TODO(jart): Make intrinsics support Clang.
o/$(MODE)/%-clang.asm: CC = $(CLANG)
o/$(MODE)/%-clang.asm: private .UNSANDBOXED = 1
o/$(MODE)/%-clang.asm: %.cc
Executable
+8
View File
@@ -0,0 +1,8 @@
#!/bin/sh
UNAMEM=$(uname -m)
UNAMES=$(uname -s)
if [ x"$UNAMES" = x"Darwin" ] && [ x"$UNAMEM" = x"arm64" ]; then
exec ape "$@"
else
exec "$@"
fi
+1 -1
View File
@@ -31,7 +31,7 @@ if [ ! -f /proc/sys/fs/binfmt_misc/status ]; then
exit 0
fi
if ! build/bootstrap/echo.com -n; then
if ! build/bootstrap/echo -n; then
cat <<'EOF' >&2
ERROR
+455
View File
@@ -0,0 +1,455 @@
/*-*- mode:c;indent-tabs-mode:nil;c-basic-offset:2;tab-width:8;coding:utf-8 -*-│
│ vi: set et ft=c ts=2 sts=2 sw=2 fenc=utf-8 :vi │
╞══════════════════════════════════════════════════════════════════════════════╡
│ Copyright 2022 Justine Alexandra Roberts Tunney │
│ │
│ Permission to use, copy, modify, and/or distribute this software for │
│ any purpose with or without fee is hereby granted, provided that the │
│ above copyright notice and this permission notice appear in all copies. │
│ │
│ THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL │
│ WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED │
│ WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE │
│ AUTHOR BE LIABLE FOR ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL │
│ DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR │
│ PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER │
│ TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR │
│ PERFORMANCE OF THIS SOFTWARE. │
╚─────────────────────────────────────────────────────────────────────────────*/
#include <errno.h>
#include <limits.h>
#include <stdarg.h>
#include <stdbool.h>
#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
// this file should not have dependencies, because everything will be
// re-downloaded if the o/tool/sha256sum artifact becomes invalidated
#define PROG "sha256sum"
#define USAGE \
"\
Usage: " PROG " [-?hbctw] [PATH...]\n\
-h help\n\
-c check mode\n\
-b binary mode\n\
-t textual mode\n\
-w warning mode\n"
#define ROTR(a, b) (((a) >> (b)) | ((a) << (32 - (b))))
#define CH(x, y, z) (((x) & (y)) ^ (~(x) & (z)))
#define MAJ(x, y, z) (((x) & (y)) ^ ((x) & (z)) ^ ((y) & (z)))
#define EP0(x) (ROTR(x, 2) ^ ROTR(x, 13) ^ ROTR(x, 22))
#define EP1(x) (ROTR(x, 6) ^ ROTR(x, 11) ^ ROTR(x, 25))
#define SIG0(x) (ROTR(x, 7) ^ ROTR(x, 18) ^ ((x) >> 3))
#define SIG1(x) (ROTR(x, 17) ^ ROTR(x, 19) ^ ((x) >> 10))
struct Sha256Ctx {
uint8_t data[64];
uint32_t datalen;
uint64_t bitlen;
uint32_t state[8];
};
static const uint32_t kSha256Tab[64] = {
0x428a2f98, 0x71374491, 0xb5c0fbcf, 0xe9b5dba5, //
0x3956c25b, 0x59f111f1, 0x923f82a4, 0xab1c5ed5, //
0xd807aa98, 0x12835b01, 0x243185be, 0x550c7dc3, //
0x72be5d74, 0x80deb1fe, 0x9bdc06a7, 0xc19bf174, //
0xe49b69c1, 0xefbe4786, 0x0fc19dc6, 0x240ca1cc, //
0x2de92c6f, 0x4a7484aa, 0x5cb0a9dc, 0x76f988da, //
0x983e5152, 0xa831c66d, 0xb00327c8, 0xbf597fc7, //
0xc6e00bf3, 0xd5a79147, 0x06ca6351, 0x14292967, //
0x27b70a85, 0x2e1b2138, 0x4d2c6dfc, 0x53380d13, //
0x650a7354, 0x766a0abb, 0x81c2c92e, 0x92722c85, //
0xa2bfe8a1, 0xa81a664b, 0xc24b8b70, 0xc76c51a3, //
0xd192e819, 0xd6990624, 0xf40e3585, 0x106aa070, //
0x19a4c116, 0x1e376c08, 0x2748774c, 0x34b0bcb5, //
0x391c0cb3, 0x4ed8aa4a, 0x5b9cca4f, 0x682e6ff3, //
0x748f82ee, 0x78a5636f, 0x84c87814, 0x8cc70208, //
0x90befffa, 0xa4506ceb, 0xbef9a3f7, 0xc67178f2, //
};
static bool g_warn;
static char g_mode;
static bool g_check;
static int g_mismatches;
static void Sha256Transform(uint32_t state[8], const uint8_t data[64]) {
unsigned i;
uint32_t a, b, c, d, e, f, g, h, t1, t2, m[64];
for (i = 0; i < 16; ++i, data += 4) {
m[i] = (uint32_t)data[0] << 24 | data[1] << 16 | data[2] << 8 | data[3];
}
for (; i < 64; ++i) {
m[i] = SIG1(m[i - 2]) + m[i - 7] + SIG0(m[i - 15]) + m[i - 16];
}
a = state[0];
b = state[1];
c = state[2];
d = state[3];
e = state[4];
f = state[5];
g = state[6];
h = state[7];
for (i = 0; i < 64; ++i) {
t1 = h + EP1(e) + CH(e, f, g) + kSha256Tab[i] + m[i];
t2 = EP0(a) + MAJ(a, b, c);
h = g;
g = f;
f = e;
e = d + t1;
d = c;
c = b;
b = a;
a = t1 + t2;
}
state[0] += a;
state[1] += b;
state[2] += c;
state[3] += d;
state[4] += e;
state[5] += f;
state[6] += g;
state[7] += h;
}
static void Sha256Init(struct Sha256Ctx *ctx) {
ctx->datalen = 0;
ctx->bitlen = 0;
ctx->state[0] = 0x6a09e667;
ctx->state[1] = 0xbb67ae85;
ctx->state[2] = 0x3c6ef372;
ctx->state[3] = 0xa54ff53a;
ctx->state[4] = 0x510e527f;
ctx->state[5] = 0x9b05688c;
ctx->state[6] = 0x1f83d9ab;
ctx->state[7] = 0x5be0cd19;
}
static void Sha256Update(struct Sha256Ctx *ctx, const uint8_t *data,
long size) {
long i;
for (i = 0; i < size; ++i) {
ctx->data[ctx->datalen] = data[i];
ctx->datalen++;
if (ctx->datalen == 64) {
Sha256Transform(ctx->state, ctx->data);
ctx->bitlen += 512;
ctx->datalen = 0;
}
}
}
static void Sha256Final(struct Sha256Ctx *ctx, uint8_t *hash) {
long i;
i = ctx->datalen;
ctx->data[i++] = 0x80;
if (ctx->datalen < 56) {
memset(ctx->data + i, 0, 56 - i);
} else {
memset(ctx->data + i, 0, 64 - i);
Sha256Transform(ctx->state, ctx->data);
memset(ctx->data, 0, 56);
}
ctx->bitlen += ctx->datalen * 8;
ctx->data[63] = ctx->bitlen;
ctx->data[62] = ctx->bitlen >> 8;
ctx->data[61] = ctx->bitlen >> 16;
ctx->data[60] = ctx->bitlen >> 24;
ctx->data[59] = ctx->bitlen >> 32;
ctx->data[58] = ctx->bitlen >> 40;
ctx->data[57] = ctx->bitlen >> 48;
ctx->data[56] = ctx->bitlen >> 56;
Sha256Transform(ctx->state, ctx->data);
for (i = 0; i < 4; ++i) {
hash[i] = (ctx->state[0] >> (24 - i * 8)) & 0xff;
hash[i + 4] = (ctx->state[1] >> (24 - i * 8)) & 0xff;
hash[i + 8] = (ctx->state[2] >> (24 - i * 8)) & 0xff;
hash[i + 12] = (ctx->state[3] >> (24 - i * 8)) & 0xff;
hash[i + 16] = (ctx->state[4] >> (24 - i * 8)) & 0xff;
hash[i + 20] = (ctx->state[5] >> (24 - i * 8)) & 0xff;
hash[i + 24] = (ctx->state[6] >> (24 - i * 8)) & 0xff;
hash[i + 28] = (ctx->state[7] >> (24 - i * 8)) & 0xff;
}
}
static char *FormatUint32(char *p, uint32_t x) {
char t;
size_t i, a, b;
i = 0;
do {
p[i++] = x % 10 + '0';
x = x / 10;
} while (x > 0);
p[i] = '\0';
if (i) {
for (a = 0, b = i - 1; a < b; ++a, --b) {
t = p[a];
p[a] = p[b];
p[b] = t;
}
}
return p + i;
}
static char *FormatInt32(char *p, int32_t x) {
if (x < 0)
*p++ = '-', x = -(uint32_t)x;
return FormatUint32(p, x);
}
static size_t StrCat(char *dst, const char *src, size_t dsize) {
size_t m, n = dsize;
const char *p = dst;
const char *q = src;
while (n-- != 0 && *dst != '\0')
dst++;
m = dst - p;
n = dsize - m;
if (n-- == 0) {
return m + strlen(src);
}
while (*src != '\0') {
if (n != 0) {
*dst++ = *src;
n--;
}
src++;
}
*dst = '\0';
return m + (src - q);
}
static void GetOpts(int argc, char *argv[]) {
int opt;
g_mode = ' ';
while ((opt = getopt(argc, argv, "?hbctw")) != -1) {
switch (opt) {
case 'w':
g_warn = true;
break;
case 'c':
g_check = true;
break;
case 't':
g_mode = ' ';
break;
case 'b':
g_mode = '*';
break;
case 'h':
case '?':
(void)write(1, USAGE, sizeof(USAGE) - 1);
exit(0);
default:
(void)write(2, USAGE, sizeof(USAGE) - 1);
exit(64);
}
}
}
static void Write(int fd, const char *s, ...) {
va_list va;
char buf[512];
buf[0] = 0;
va_start(va, s);
do {
StrCat(buf, s, sizeof(buf));
} while ((s = va_arg(va, const char *)));
va_end(va);
(void)write(fd, buf, strlen(buf));
}
static bool IsModeCharacter(char c) {
switch (c) {
case ' ':
case '*':
return true;
default:
return false;
}
}
static bool IsSupportedPath(const char *path) {
size_t i;
for (i = 0;; ++i) {
switch (path[i]) {
case 0:
if (i)
return true;
// fallthrough
case '\r':
case '\n':
case '\\':
Write(2, PROG, ": ", path, ": unsupported path\n", NULL);
return false;
default:
break;
}
}
}
static bool GetDigest(const char *path, FILE *f, uint8_t digest[32]) {
size_t got;
uint8_t buf[512];
struct Sha256Ctx ctx;
Sha256Init(&ctx);
while ((got = fread(buf, 1, sizeof(buf), f))) {
Sha256Update(&ctx, buf, got);
}
if (ferror(f)) {
Write(2, PROG, ": ", path, ": ", strerror(errno), "\n", NULL);
return false;
}
Sha256Final(&ctx, digest);
return true;
}
static char *CopyHex(char *s, const void *p, size_t n) {
const char *d, *e;
for (d = (const char *)p, e = d + n; d < e; ++d) {
*s++ = "0123456789abcdef"[(*d >> 4) & 15];
*s++ = "0123456789abcdef"[(*d >> 0) & 15];
}
*s = 0;
return s;
}
static bool ProduceDigest(const char *path, FILE *f) {
char hexdigest[65];
char mode[2] = {g_mode};
unsigned char digest[32];
if (!IsSupportedPath(path))
return false;
if (!GetDigest(path, f, digest))
return false;
CopyHex(hexdigest, digest, 32);
Write(1, hexdigest, " ", mode, path, "\n", NULL);
return true;
}
static char *Chomp(char *line) {
size_t i;
if (line) {
for (i = strlen(line); i--;) {
if (line[i] == '\r' || line[i] == '\n') {
line[i] = '\0';
} else {
break;
}
}
}
return line;
}
static int HexToInt(int c) {
if ('0' <= c && c <= '9') {
return c - '0';
} else if ('a' <= c && c <= 'f') {
return c - 'a' + 10;
} else if ('A' <= c && c <= 'F') {
return c - 'A' + 10;
} else {
return -1;
}
}
static bool CheckDigests(const char *path, FILE *f) {
FILE *f2;
bool k = true;
int a, b, i, line;
const char *path2, *status;
uint8_t wantdigest[32], gotdigest[32];
char buf[64 + 2 + PATH_MAX + 1 + 1], *p;
for (line = 0; fgets(buf, sizeof(buf), f); ++line) {
if (!*Chomp(buf))
continue;
for (p = buf, i = 0; i < 32; ++i) {
if ((a = HexToInt(*p++ & 255)) == -1)
goto InvalidLine;
if ((b = HexToInt(*p++ & 255)) == -1)
goto InvalidLine;
wantdigest[i] = a << 4 | b;
}
if (*p++ != ' ')
goto InvalidLine;
if (!IsModeCharacter(*p++))
goto InvalidLine;
path2 = p;
if (!*path2)
goto InvalidLine;
if (!IsSupportedPath(path2))
continue;
if ((f2 = fopen(path2, "rb"))) {
if (GetDigest(path2, f2, gotdigest)) {
if (!memcmp(wantdigest, gotdigest, 32)) {
status = "OK";
} else {
status = "FAILED";
++g_mismatches;
k = false;
}
Write(1, path2, ": ", status, "\n", NULL);
} else {
k = false;
}
fclose(f2);
} else {
Write(2, PROG, ": ", path2, ": ", strerror(errno), "\n", NULL);
k = false;
}
continue;
InvalidLine:
if (g_warn) {
char linestr[12];
FormatInt32(linestr, line + 1);
Write(2, PROG, ": ", path, ":", linestr, ": ",
"improperly formatted checksum line", "\n", NULL);
}
}
if (ferror(f)) {
Write(2, PROG, ": ", path, ": ", strerror(errno), "\n", NULL);
k = false;
}
return k;
}
static bool Process(const char *path, FILE *f) {
if (g_check) {
return CheckDigests(path, f);
} else {
return ProduceDigest(path, f);
}
}
int main(int argc, char *argv[]) {
int i;
FILE *f;
bool k = true;
GetOpts(argc, argv);
if (optind == argc) {
f = stdin;
k &= Process("-", f);
} else {
for (i = optind; i < argc; ++i) {
if ((f = fopen(argv[i], "rb"))) {
k &= Process(argv[i], f);
fclose(f);
} else {
Write(2, PROG, ": ", argv[i], ": ", strerror(errno), "\n", NULL);
k = false;
}
}
}
if (g_mismatches) {
char ibuf[12];
FormatInt32(ibuf, g_mismatches);
Write(2, PROG, ": WARNING: ", ibuf, " computed checksum did NOT match\n",
NULL);
}
return !k;
}
+13
View File
@@ -0,0 +1,13 @@
---
BasedOnStyle: Mozilla
IndentWidth: 4
ColumnLimit: 80
---
Language: Cpp
AllowShortFunctionsOnASingleLine: false
AlignTrailingComments: false
AlignEscapedNewlines: DontAlign
AlwaysBreakTemplateDeclarations: true
ConstructorInitializerAllOnOneLineOrOnePerLine: true
FixNamespaceComments: true
---
+55
View File
@@ -0,0 +1,55 @@
#-*-mode:makefile-gmake;indent-tabs-mode:t;tab-width:8;coding:utf-8-*-┐
#── vi: set noet ft=make ts=8 sw=8 fenc=utf-8 :vi ────────────────────┘
PKGS += CTL
CTL_ARTIFACTS += CTL_A
CTL = $(CTL_A_DEPS) $(CTL_A)
CTL_A = o/$(MODE)/ctl/ctl.a
CTL_A_FILES := $(wildcard ctl/*)
CTL_A_HDRS = $(filter %.h,$(CTL_A_FILES))
CTL_A_SRCS = $(filter %.cc,$(CTL_A_FILES))
CTL_A_OBJS = $(CTL_A_SRCS:%.cc=o/$(MODE)/%.o)
CTL_A_CHECKS = \
$(CTL_A).pkg \
$(CTL_A_HDRS:%=o/$(MODE)/%.okk) \
CTL_A_DIRECTDEPS = \
LIBC_INTRIN \
LIBC_MEM \
LIBC_NEXGEN32E \
LIBC_STDIO \
LIBC_STR \
THIRD_PARTY_DOUBLECONVERSION \
THIRD_PARTY_GDTOA \
THIRD_PARTY_LIBCXXABI \
THIRD_PARTY_LIBUNWIND \
CTL_A_DEPS := $(call uniq,$(foreach x,$(CTL_A_DIRECTDEPS),$($(x))))
$(CTL_A): ctl/ \
$(CTL_A).pkg \
$(CTL_A_OBJS)
$(CTL_A).pkg: \
$(CTL_A_OBJS) \
$(foreach x,$(CTL_A_DIRECTDEPS),$($(x)_A).pkg)
$(CTL_A_OBJS): private \
OVERRIDE_CXXFLAGS += \
-Wframe-larger-than=4096 \
-Walloca-larger-than=4096 \
-ffunction-sections \
-fdata-sections \
-fexceptions \
CTL_LIBS = $(foreach x,$(CTL_ARTIFACTS),$($(x)))
CTL_SRCS = $(foreach x,$(CTL_ARTIFACTS),$($(x)_SRCS))
CTL_HDRS = $(foreach x,$(CTL_ARTIFACTS),$($(x)_HDRS))
CTL_CHECKS = $(foreach x,$(CTL_ARTIFACTS),$($(x)_CHECKS))
CTL_OBJS = $(foreach x,$(CTL_ARTIFACTS),$($(x)_OBJS))
$(CTL_OBJS): $(BUILD_FILES) ctl/BUILD.mk
.PHONY: o/$(MODE)/ctl
o/$(MODE)/ctl: $(CTL_CHECKS)
+5015
View File
File diff suppressed because it is too large. Load diff
+28
View File
@@ -0,0 +1,28 @@
// -*-mode:c++;indent-tabs-mode:nil;c-basic-offset:4;tab-width:8;coding:utf-8-*-
// vi: set et ft=cpp ts=4 sts=4 sw=4 fenc=utf-8 :vi
#ifndef CTL_ACCUMULATE_H_
#define CTL_ACCUMULATE_H_
namespace ctl {
template<class InputIt, class T>
constexpr T
accumulate(InputIt first, InputIt last, T init)
{
for (; first != last; ++first)
init = init + *first;
return init;
}
template<class InputIt, class T, class BinaryOperation>
constexpr T
accumulate(InputIt first, InputIt last, T init, BinaryOperation op)
{
for (; first != last; ++first)
init = op(init, *first);
return init;
}
} // namespace ctl
#endif // CTL_ACCUMULATE_H_
+55
View File
@@ -0,0 +1,55 @@
// -*-mode:c++;indent-tabs-mode:nil;c-basic-offset:4;tab-width:8;coding:utf-8-*-
// vi: set et ft=cpp ts=4 sts=4 sw=4 fenc=utf-8 :vi
#ifndef CTL_ADD_LVALUE_REFERENCE_H_
#define CTL_ADD_LVALUE_REFERENCE_H_
namespace ctl {
template<typename T>
struct add_lvalue_reference
{
typedef T& type;
};
template<typename T>
struct add_lvalue_reference<T&>
{
typedef T& type;
};
template<typename T>
struct add_lvalue_reference<T&&>
{
typedef T& type;
};
template<>
struct add_lvalue_reference<void>
{
typedef void type;
};
template<>
struct add_lvalue_reference<const void>
{
typedef const void type;
};
template<>
struct add_lvalue_reference<volatile void>
{
typedef volatile void type;
};
template<>
struct add_lvalue_reference<const volatile void>
{
typedef const volatile void type;
};
template<typename T>
using add_lvalue_reference_t = typename add_lvalue_reference<T>::type;
} // namespace ctl
#endif // CTL_ADD_LVALUE_REFERENCE_H_
+43
View File
@@ -0,0 +1,43 @@
// -*-mode:c++;indent-tabs-mode:nil;c-basic-offset:4;tab-width:8;coding:utf-8-*-
// vi: set et ft=cpp ts=4 sts=4 sw=4 fenc=utf-8 :vi
#ifndef CTL_ADD_POINTER_H_
#define CTL_ADD_POINTER_H_
#include "remove_reference.h"
namespace ctl {
// Primary template
template<typename T>
struct add_pointer
{
typedef typename remove_reference<T>::type* type;
};
// Specialization for functions
template<typename R, typename... Args>
struct add_pointer<R(Args...)>
{
typedef R (*type)(Args...);
};
// Specialization for function references
template<typename R, typename... Args>
struct add_pointer<R (&)(Args...)>
{
typedef R (*type)(Args...);
};
// Specialization for rvalue references to functions
template<typename R, typename... Args>
struct add_pointer<R (&&)(Args...)>
{
typedef R (*type)(Args...);
};
// Helper alias template (C++14 and later)
template<typename T>
using add_pointer_t = typename add_pointer<T>::type;
} // namespace ctl
#endif // CTL_ADD_POINTER_H_
+29
View File
@@ -0,0 +1,29 @@
// -*-mode:c++;indent-tabs-mode:nil;c-basic-offset:4;tab-width:8;coding:utf-8-*-
// vi: set et ft=cpp ts=4 sts=4 sw=4 fenc=utf-8 :vi
#ifndef CTL_ADDRESSOF_H_
#define CTL_ADDRESSOF_H_
namespace ctl {
template<typename T>
T*
addressof(T& arg) noexcept
{
return reinterpret_cast<T*>(
&const_cast<char&>(reinterpret_cast<const volatile char&>(arg)));
}
template<typename R, typename... Args>
R (*addressof(R (*&arg)(Args...)) noexcept)
(Args...)
{
return arg;
}
template<typename T>
T*
addressof(T&&) = delete;
} // namespace ctl
#endif // CTL_ADDRESSOF_H_
+24
View File
@@ -0,0 +1,24 @@
// -*-mode:c++;indent-tabs-mode:nil;c-basic-offset:4;tab-width:8;coding:utf-8-*-
// vi: set et ft=cpp ts=4 sts=4 sw=4 fenc=utf-8 :vi
#ifndef CTL_ADVANCE_H_
#define CTL_ADVANCE_H_
namespace ctl {
template<class InputIt, class Distance>
constexpr void
advance(InputIt& it, Distance n)
{
while (n > 0) {
--n;
++it;
}
while (n < 0) {
++n;
--it;
}
}
} // namespace ctl
#endif // CTL_ADVANCE_H_
+20
View File
@@ -0,0 +1,20 @@
// -*-mode:c++;indent-tabs-mode:nil;c-basic-offset:4;tab-width:8;coding:utf-8-*-
// vi: set et ft=cpp ts=4 sts=4 sw=4 fenc=utf-8 :vi
#ifndef CTL_ALL_OF_H_
#define CTL_ALL_OF_H_
namespace ctl {
template<class InputIt, class UnaryPredicate>
constexpr bool
all_of(InputIt first, InputIt last, UnaryPredicate p)
{
for (; first != last; ++first)
if (!p(*first))
return false;
return true;
}
} // namespace ctl
#endif // CTL_ALL_OF_H_
+94
View File
@@ -0,0 +1,94 @@
// -*-mode:c++;indent-tabs-mode:nil;c-basic-offset:4;tab-width:8;coding:utf-8-*-
// vi: set et ft=cpp ts=4 sts=4 sw=4 fenc=utf-8 :vi
#ifndef CTL_ALLOCATOR_H_
#define CTL_ALLOCATOR_H_
#include "bad_alloc.h"
#include "integral_constant.h"
#include "new.h"
#include "utility.h"
namespace ctl {
template<typename T>
class allocator
{
public:
using value_type = T;
using size_type = size_t;
using difference_type = ptrdiff_t;
using propagate_on_container_move_assignment = ctl::true_type;
using is_always_equal = ctl::true_type;
using pointer = T*;
using const_pointer = const T*;
using reference = T&;
using const_reference = const T&;
constexpr allocator() noexcept = default;
constexpr allocator(const allocator&) noexcept = default;
template<class U>
constexpr allocator(const allocator<U>&) noexcept
{
}
constexpr ~allocator() = default;
[[nodiscard]] T* allocate(size_type n)
{
if (n > __SIZE_MAX__ / sizeof(T))
throw ctl::bad_alloc();
if (auto p = static_cast<T*>(::operator new(
n * sizeof(T), ctl::align_val_t(alignof(T)), ctl::nothrow)))
return p;
throw ctl::bad_alloc();
}
void deallocate(T* p, size_type n) noexcept
{
::operator delete(p, n * sizeof(T), ctl::align_val_t(alignof(T)));
}
template<typename U, typename... Args>
void construct(U* p, Args&&... args)
{
::new (static_cast<void*>(p)) U(ctl::forward<Args>(args)...);
}
template<typename U>
void destroy(U* p)
{
p->~U();
}
size_type max_size() const noexcept
{
return __SIZE_MAX__ / sizeof(T);
}
allocator& operator=(const allocator&) = default;
template<typename U>
struct rebind
{
using other = allocator<U>;
};
};
template<class T, class U>
bool
operator==(const allocator<T>&, const allocator<U>&) noexcept
{
return true;
}
template<class T, class U>
bool
operator!=(const allocator<T>&, const allocator<U>&) noexcept
{
return false;
}
} // namespace ctl
#endif // CTL_ALLOCATOR_H_
+70
View File
@@ -0,0 +1,70 @@
// -*-mode:c++;indent-tabs-mode:nil;c-basic-offset:4;tab-width:8;coding:utf-8-*-
// vi: set et ft=cpp ts=4 sts=4 sw=4 fenc=utf-8 :vi
#ifndef CTL_ALLOCATOR_TRAITS_H_
#define CTL_ALLOCATOR_TRAITS_H_
#include "integral_constant.h"
namespace ctl {
template<typename Alloc>
struct allocator_traits
{
using allocator_type = Alloc;
using value_type = typename Alloc::value_type;
using pointer = typename Alloc::value_type*;
using const_pointer = const typename Alloc::value_type*;
using void_pointer = void*;
using const_void_pointer = const void*;
using difference_type = ptrdiff_t;
using size_type = size_t;
using propagate_on_container_copy_assignment = ctl::false_type;
using propagate_on_container_move_assignment = ctl::true_type;
using propagate_on_container_swap = ctl::false_type;
using is_always_equal = ctl::true_type;
template<typename T>
struct rebind_alloc
{
using other = typename Alloc::template rebind<T>::other;
};
template<typename T>
using rebind_traits = allocator_traits<typename rebind_alloc<T>::other>;
static pointer allocate(Alloc& a, size_type n)
{
return a.allocate(n);
}
static void deallocate(Alloc& a, pointer p, size_type n)
{
a.deallocate(p, n);
}
template<typename T, typename... Args>
static void construct(Alloc& a, T* p, Args&&... args)
{
::new ((void*)p) T(static_cast<Args&&>(args)...);
}
template<typename T>
static void destroy(Alloc& a, T* p)
{
p->~T();
}
static size_type max_size(const Alloc& a) noexcept
{
return a.max_size();
}
static Alloc select_on_container_copy_construction(const Alloc& a)
{
return a;
}
};
} // namespace ctl
#endif // CTL_ALLOCATOR_TRAITS_H_
+20
View File
@@ -0,0 +1,20 @@
// -*-mode:c++;indent-tabs-mode:nil;c-basic-offset:4;tab-width:8;coding:utf-8-*-
// vi: set et ft=cpp ts=4 sts=4 sw=4 fenc=utf-8 :vi
#ifndef CTL_ANY_OF_H_
#define CTL_ANY_OF_H_
namespace ctl {
template<class InputIt, class UnaryPredicate>
constexpr bool
any_of(InputIt first, InputIt last, UnaryPredicate p)
{
for (; first != last; ++first)
if (p(*first))
return true;
return false;
}
} // namespace ctl
#endif // CTL_ANY_OF_H_
+248
View File
@@ -0,0 +1,248 @@
// -*-mode:c++;indent-tabs-mode:nil;c-basic-offset:4;tab-width:8;coding:utf-8-*-
// vi: set et ft=cpp ts=4 sts=4 sw=4 fenc=utf-8 :vi
#ifndef CTL_ARRAY_H_
#define CTL_ARRAY_H_
#include "initializer_list.h"
#include "out_of_range.h"
#include "reverse_iterator.h"
namespace ctl {
template<typename T, size_t N>
struct array
{
using value_type = T;
using size_type = size_t;
using difference_type = ptrdiff_t;
using reference = value_type&;
using const_reference = const value_type&;
using pointer = value_type*;
using const_pointer = const value_type*;
using iterator = value_type*;
using const_iterator = const value_type*;
using reverse_iterator = ctl::reverse_iterator<iterator>;
using const_reverse_iterator = ctl::reverse_iterator<const_iterator>;
T elems[N];
constexpr array() = default;
constexpr array(std::initializer_list<T> init)
{
auto it = init.begin();
for (size_t i = 0; i < N && it != init.end(); ++i, ++it)
elems[i] = *it;
}
constexpr reference at(size_type pos)
{
if (pos >= N)
throw ctl::out_of_range();
return elems[pos];
}
constexpr const_reference at(size_type pos) const
{
if (pos >= N)
throw ctl::out_of_range();
return elems[pos];
}
constexpr reference operator[](size_type pos)
{
if (pos >= N)
__builtin_trap();
return elems[pos];
}
constexpr const_reference operator[](size_type pos) const
{
if (pos >= N)
__builtin_trap();
return elems[pos];
}
constexpr reference front()
{
return elems[0];
}
constexpr const_reference front() const
{
return elems[0];
}
constexpr reference back()
{
return elems[N - 1];
}
constexpr const_reference back() const
{
return elems[N - 1];
}
constexpr T* data() noexcept
{
return elems;
}
constexpr const T* data() const noexcept
{
return elems;
}
constexpr iterator begin() noexcept
{
return elems;
}
constexpr const_iterator begin() const noexcept
{
return elems;
}
constexpr iterator end() noexcept
{
return elems + N;
}
constexpr const_iterator end() const noexcept
{
return elems + N;
}
constexpr reverse_iterator rbegin() noexcept
{
return reverse_iterator(end());
}
constexpr const_reverse_iterator rbegin() const noexcept
{
return const_reverse_iterator(end());
}
constexpr reverse_iterator rend() noexcept
{
return reverse_iterator(begin());
}
constexpr const_reverse_iterator rend() const noexcept
{
return const_reverse_iterator(begin());
}
constexpr const_iterator cbegin() const noexcept
{
return begin();
}
constexpr const_iterator cend() const noexcept
{
return end();
}
constexpr const_reverse_iterator crbegin() const noexcept
{
return rbegin();
}
constexpr const_reverse_iterator crend() const noexcept
{
return rend();
}
constexpr bool empty() const noexcept
{
return N == 0;
}
constexpr size_type size() const noexcept
{
return N;
}
constexpr size_type max_size() const noexcept
{
return N;
}
void fill(const T& value)
{
for (size_type i = 0; i < N; ++i) {
elems[i] = value;
}
}
void swap(array& other) noexcept
{
for (size_type i = 0; i < N; ++i) {
T temp = elems[i];
elems[i] = other.elems[i];
other.elems[i] = temp;
}
}
};
template<typename T, size_t N>
bool
operator==(const array<T, N>& lhs, const array<T, N>& rhs)
{
for (size_t i = 0; i < N; ++i) {
if (!(lhs[i] == rhs[i]))
return false;
}
return true;
}
template<typename T, size_t N>
bool
operator!=(const array<T, N>& lhs, const array<T, N>& rhs)
{
return !(lhs == rhs);
}
template<typename T, size_t N>
bool
operator<(const array<T, N>& lhs, const array<T, N>& rhs)
{
for (size_t i = 0; i < N; ++i) {
if (lhs[i] < rhs[i])
return true;
if (rhs[i] < lhs[i])
return false;
}
return false;
}
template<typename T, size_t N>
bool
operator<=(const array<T, N>& lhs, const array<T, N>& rhs)
{
return !(rhs < lhs);
}
template<typename T, size_t N>
bool
operator>(const array<T, N>& lhs, const array<T, N>& rhs)
{
return rhs < lhs;
}
template<typename T, size_t N>
bool
operator>=(const array<T, N>& lhs, const array<T, N>& rhs)
{
return !(lhs < rhs);
}
template<typename T, size_t N>
void
swap(array<T, N>& lhs, array<T, N>& rhs) noexcept
{
lhs.swap(rhs);
}
} // namespace ctl
#endif // CTL_ARRAY_H_
+64
View File
@@ -0,0 +1,64 @@
// -*-mode:c++;indent-tabs-mode:nil;c-basic-offset:4;tab-width:8;coding:utf-8-*-
// vi: set et ft=cpp ts=4 sts=4 sw=4 fenc=utf-8 :vi
#ifndef CTL_BACK_INSERTER_H_
#define CTL_BACK_INSERTER_H_
#include "iterator.h"
#include "utility.h"
namespace ctl {
template<class Container>
class back_insert_iterator
{
protected:
Container* container;
public:
using iterator_category = ctl::output_iterator_tag;
using value_type = void;
using difference_type = void;
using pointer = void;
using reference = void;
explicit back_insert_iterator(Container& c) : container(&c)
{
}
back_insert_iterator& operator=(const typename Container::value_type& value)
{
container->push_back(value);
return *this;
}
back_insert_iterator& operator=(typename Container::value_type&& value)
{
container->push_back(ctl::move(value));
return *this;
}
back_insert_iterator& operator*()
{
return *this;
}
back_insert_iterator& operator++()
{
return *this;
}
back_insert_iterator operator++(int)
{
return *this;
}
};
template<class Container>
back_insert_iterator<Container>
back_inserter(Container& c)
{
return back_insert_iterator<Container>(c);
}
} // namespace ctl
#endif // CTL_BACK_INSERTER_H_
+23
View File
@@ -0,0 +1,23 @@
// -*-mode:c++;indent-tabs-mode:nil;c-basic-offset:4;tab-width:8;coding:utf-8-*-
// vi: set et ft=cpp ts=4 sts=4 sw=4 fenc=utf-8 :vi
#ifndef CTL_BAD_ALLOC_H_
#define CTL_BAD_ALLOC_H_
#include "exception.h"
namespace ctl {
class bad_alloc : public ctl::exception
{
public:
bad_alloc() noexcept = default;
~bad_alloc() override = default;
const char* what() const noexcept override
{
return "ctl::bad_alloc";
}
};
} // namespace ctl
#endif // CTL_BAD_ALLOC_H_
+22
View File
@@ -0,0 +1,22 @@
// -*-mode:c++;indent-tabs-mode:nil;c-basic-offset:4;tab-width:8;coding:utf-8-*-
// vi: set et ft=cpp ts=4 sts=4 sw=4 fenc=utf-8 :vi
#ifndef CTL_CONDITIONAL_H_
#define CTL_CONDITIONAL_H_
namespace ctl {
template<bool B, class T, class F>
struct conditional
{
typedef T type;
};
template<class T, class F>
struct conditional<false, T, F>
{
typedef F type;
};
} // namespace ctl
#endif // CTL_CONDITIONAL_H_
+22
View File
@@ -0,0 +1,22 @@
// -*-mode:c++;indent-tabs-mode:nil;c-basic-offset:4;tab-width:8;coding:utf-8-*-
// vi: set et ft=cpp ts=4 sts=4 sw=4 fenc=utf-8 :vi
#ifndef CTL_COPY_H_
#define CTL_COPY_H_
namespace ctl {
template<class InputIt, class OutputIt>
OutputIt
copy(InputIt first, InputIt last, OutputIt d_first)
{
while (first != last) {
*d_first = *first;
++d_first;
++first;
}
return d_first;
}
} // namespace ctl
#endif // CTL_COPY_H_
+36
View File
@@ -0,0 +1,36 @@
// -*-mode:c++;indent-tabs-mode:nil;c-basic-offset:4;tab-width:8;coding:utf-8-*-
// vi: set et ft=cpp ts=4 sts=4 sw=4 fenc=utf-8 :vi
#ifndef CTL_DECAY_H_
#define CTL_DECAY_H_
#include "add_pointer.h"
#include "conditional.h"
#include "is_array.h"
#include "is_function.h"
#include "remove_cv.h"
#include "remove_extent.h"
#include "remove_reference.h"
namespace ctl {
template<typename T>
struct decay
{
private:
typedef typename ctl::remove_reference<T>::type U;
public:
typedef typename ctl::conditional<
ctl::is_array<U>::value,
typename ctl::remove_extent<U>::type*,
typename ctl::conditional<ctl::is_function<U>::value,
typename ctl::add_pointer<U>::type,
typename ctl::remove_cv<U>::type>::type>::type
type;
};
template<typename T>
using decay_t = typename decay<T>::type;
} // namespace ctl
#endif // CTL_DECAY_H_
Loaded 100 of 25352 files, more files were not shown because too many files have changed in this diff. Show more