Compare commits

...
1029 Commits
Author SHA1 Message Date
Justine Tunney 67b19ae733 Release Cosmopolitan v3.5.0 2024-06-23 22:45:14 -07:00
Justine Tunney c4c812c154 Introduce ctl::set and ctl::map
We now have a C++ red-black tree implementation that implements standard
template library compatible APIs while compiling 10x faster than libcxx.
It's not as beautiful as the red-black tree implementation in Plinko but
this will get the job done and the test proves it upholds all invariants

This change also restores CheckForMemoryLeaks() support and fixes a real
actual bug I discovered with Doug Lea's dlmalloc_inspect_all() function.
2024-06-23 22:27:11 -07:00
Justine Tunney 388e236360 Revert misguided dlmalloc optimization 2024-06-22 09:55:02 -07:00
Justine Tunney f2c8ddbbe3 Fix --strace use-after-free in pthread_join() 2024-06-22 06:05:52 -07:00
Justine Tunney d1d4388201 Delete ASAN
It hasn't been helpful enough to be justify the maintenance burden. What
actually does help is mprotect(), kprintf(), --ftrace and --strace which
can always be counted upon to work correctly. We aren't losing much with
this change. Support for ASAN on AARCH64 was never implemented. Applying
ASAN to the core libc runtimes was disabled many months ago. If there is
some way to have an ASAN runtime for user programs that is less invasive
we can potentially consider reintroducing support. But now is premature.
2024-06-22 05:45:49 -07:00
Justine Tunney 6ffed14b9c Rewrite memory manager
Actually Portable Executable now supports Android. Cosmo's old mmap code
required a 47 bit address space. The new implementation is very agnostic
and supports both smaller address spaces (e.g. embedded) and even modern
56-bit PML5T paging for x86 which finally came true on Zen4 Threadripper

Cosmopolitan no longer requires UNIX systems to observe the Windows 64kb
granularity; i.e. sysconf(_SC_PAGE_SIZE) will now report the host native
page size. This fixes a longstanding POSIX conformance issue, concerning
file mappings that overlap the end of file. Other aspects of conformance
have been improved too, such as the subtleties of address assignment and
and the various subtleties surrounding MAP_FIXED and MAP_FIXED_NOREPLACE

On Windows, mappings larger than 100 megabytes won't be broken down into
thousands of independent 64kb mappings. Support for MAP_STACK is removed
by this change; please use NewCosmoStack() instead.

Stack overflow avoidance is now being implemented using the POSIX thread
APIs. Please use GetStackBottom() and GetStackAddr(), instead of the old
error-prone GetStackAddr() and HaveStackMemory() APIs which are removed.
2024-06-22 05:45:11 -07:00
Justine Tunney 7f6d0b8709 Support -x LANG flag in cosmocc 2024-06-20 23:55:46 -07:00
Steven Dee (Jōshin) d7b1919b29 ctl::unique_ptr improvements and cleanup (#1221)
Explicitly value-initializes the deleter, even though I have not found a
way to get the deleter to act like it’s been default-initialized in unit
tests so far.

Uses auto in reset. The static cast is apparently not needed (unless I’m
missing some case I didn’t think of.)

Implements the general move constructor - turns out that the reason this
didn’t work before was that default_delete<U> was not move constructible
from default_delete<T>.

Drop inline specifiers from functions defined entirely inside the struct
definition since they are implicitly inline.

* Cleans up reset to match spec

Remove the variants from the T[] specialization. Also follow the spec on
the order of operations in reset, which may matter if we are deleting an
object that has a reference to the unique_ptr that is being reset. (?)

* Tests Base/Derived reset.

* Adds some constexpr declarations.

* Adds default_delete specialization for T[].

* Makes parameters const.
2024-06-20 18:44:31 -04:00
Steven Dee (Jōshin) f86e6f8eb0 Make new.cc definitions weak (#1233)
The STL says that these should be replaceable by user code.

new.cc now defines only a few direct functions (including a free wrapper
that perplexingly is needed since g++ didn’t want to alias "free".) Now,
all of the operators are weak references to those functions.
2024-06-20 15:20:54 -04:00
Steven Dee (Jōshin) 7e780e57d4 More ctl::string optimization (#1232)
Moves some isbig checks into string.h, enabling smarter optimizations to
be made on small strings. Also we no longer zero out our string prior to
calling the various constructors, buying back the performance we lost on
big strings when we made the small-string optimization. We further add a
little optimization to the big_string copy constructor: if the string is
using half or more of its capacity, then we don’t recompute capacity and
just take the old string’s. As well, the copy constructor always makes a
small string when it will fit, even if copied from a big string that got
truncated.

This also reworks the test to follow the idiom adopted elsewhere re stl,
and adds a helper function to tell if a string is small based on data().
2024-06-20 14:52:12 -04:00
Steven Dee (Jōshin) 9a5a13854d CTL: utility.h, use ctl::swap in string (#1227)
* Add ctl utility.h

Implements forward, move, swap, and declval. This commit also adds a def
for nullptr_t to cxx.inc. We need it now because the CTL headers stopped
including anything from libc++, so we no longer get their basic types.

* Use ctl::swap in string

The STL spec says that swap is located in the string_view header anyawy.
Performance-wise this is a noop, but it’s slightly cleaner.
2024-06-19 01:00:59 -04:00
Steven Dee (Jōshin) a795017416 Fix c.inc _Atomic define for C++ (#1231)
c.inc (AFAICT erroneously) defined _Atomic(t) as `volatile t *`, when it
should have just said `volatile t`, when __STDC_VERSION__ was too small.
This happens when we’re compiling C++, but in C++11, _Atomic is a define
supplied by the STL rather than a keyword supplied by the compiler. Wait
though, it gets better: in C++11, _Atomic hooks you into the morass that
is stdatomic.h, and ultimately refers everything back to std::atomic<T>.

The gory, horrifying details are in libcxx's __atomic/cxx_atomic_impl.h.
The tldr is that for our purposes it’s fine to just say volatile and use
the normal libc/intrin/atomic.h functions.
2024-06-17 21:12:02 -07:00
Brian 8e3b361aeb Remove testing label from labeler.yml (#1220)
#1219 had an issue with noisy testing label. Investigated if there is a
syntax to make it more exclusive, but turns out there isn't one yet. So
let's remove an manually add it in as needed.

Fixes #1219.
2024-06-16 11:45:57 -04:00
Steven Dee (Jōshin) f9dd5683a4 Implement ctl::unique_ptr (#1216)
The way unique_ptr is supposed to work is as a purely compile-time check
that your raw pointers are getting deleted when they go out of scope. It
should ideally emit the same exact machine code as if you were using raw
pointers with manual deletes.

Part of what this means is that under normal circumstances, a unique_ptr
shouldn’t take up more space than a raw pointer - in other words, sizeof
unique_ptr<T> should == sizeof(T*).

The present PR doesn’t bother with the specialization for array types. I
also left a couple other parts of the STL API unimplemented. I’d love to
see someone else implement these, or I’ll get to them at some point.
2024-06-15 22:54:52 -04:00
Steven Dee (Jōshin) e38a6e7996 ctl string const/value tweaks (#1218)
The mangled name of a C++ function will typically not vary by const-ness
of a by-value parameter; in other words, there is no meaning to a const-
qualified by-value parameter in a function prototype. However, the const
keyword _does_ matter at function _definition_ time, like it does with a
variable declared in the body. So for prototypes, we strip out const for
by-value parameters; but for definitions, we leave them alone.

At function definition (as opposed to prototype), we add const to values
in parameters by default, unless we’re going to mutate them.

This commit also changes a couple of const string_view& to be simply by-
value string_view. A string_view is only two words; it rarely ever makes
sense to pass one by reference if it’s not going to be mutated.
2024-06-15 18:09:30 -07:00
Steven Dee (Jōshin) ddfaf3fee0 Explain the clang-format off in new.h 2024-06-15 17:17:04 -07:00
Michael Lenaghan 0dbf01bf1d Bring Lua to 5.4.6. (#1214)
This essentially re-does the work of #875 on top of master.

This is what I did to check that Cosmo's Lua extensions still worked:

```
$ build/bootstrap/make MODE=aarch64 o/aarch64/third_party/lua/lua
$ ape o/aarch64/third_party/lua/lua
>: 10
10
>: 010
8
>: 0b10
2
>: string.byte("\e")
27
>: "Hello, %s" % {"world"}
Hello, world
>: "*" * 3
***
```

`luaL_traceback2` was used to show the stack trace with parameter
values; it's used in `LuaCallWithTrace`, which is used in Redbean to run
Lua code. You should be able to see the extended stack trace by running
something like this: `redbean -e "function a(b)c()end a(2)"` (with
"params" indicating the extended stack trace):

```
stack traceback:
 [string "function a(b)c()end a(2)"]:1: in function 'a', params: b = 2;
 [string "function a(b)c()end a(2)"]:1: in main chunk
```
@pkulchenko confirmed that I get the expected result with the updated
code.

This is what I did to check that Lua itself still worked:

```
$ cd third_party/lua/test/
$ ape ../../../o/aarch64/third_party/lua/lua all.lua
```

There's one test failure, in `files.lua`:

```
***** FILE 'files.lua'*****
testing i/o
../../../o/aarch64/third_party/lua/lua: files.lua:84: assertion failed!
stack traceback:
[C]: in function 'assert'
files.lua:84: in main chunk
(...tail calls...)
all.lua:195: in main chunk
[C]: in ?
.>>> closing state <<<
```

That isn't a result of these changes; the same test is failing in
master.

The failure is here:

```lua
if not _port then   -- invalid seek
  local status, msg, code = io.stdin:seek("set", 1000)
  assert(not status and type(msg) == "string" and type(code) == "number")
end
```

The test expects a seek to offset 1,000 on stdin to fail — but it
doesn't. `status` ends up being the new offset rather than `nil`.

If I comment out that one test, the remaining tests succeed.
2024-06-15 20:13:08 -04:00
Jōshin 3a599bfbe1 Update .git-blame-ignore-revs 2024-06-15 13:36:07 -07:00
Jōshin 89fc95fefd Rerun clang-format on the repo (#1217)
🚨 clang-format changes output per version!

This is with version 19.0.0. The modifications seem to be fixing the old
version’s errors - mainly involving omitted whitespace around binary ops
and inserted whitespace between goto labels and colons (if followed by a
curly brace.)

Also fixes a few mistakes made by e.g. someone (ahem) forgetting to pass
his ctl/string.h modifications through it.

We should add this to .git-blame-ignore-revs once we have its final hash
on master.
2024-06-15 16:34:48 -04:00
Jōshin 8e37ee2598 ctl::string cleanup (#1215)
We now fully initialize a ctl::string’s memory, so that it is always set
to a well-defined value, thus making it always safe to memcpy out of it.
This incidentally makes our string::swap function legal, which it wasn’t
before. This also saves us a store in string::reserve.

Now that we have made both big_string and small_string POD, I believe it
is safe to elide the launder calls, and have done so, thus cleaning up a
lot of the blob-related code.

I also got rid of set_big_capacity and replaced it with a set_big_string
that leaves us in a well-defined state afterwards. This function also is
able to be somewhat simpler; rather than delicate bit-twiddling, it just
reaches straight into blob and rewrites it wholesale.

Overall, this shaves about 1–2ns off of most benchmarks, and adds 1ns to
only one of them - creating a string from a char *.
2024-06-15 10:49:52 -07:00
Jōshin d9b4f647d8 Uncomment swap test (#1210) 2024-06-10 21:51:19 -07:00
Jōshin 0dde3a0e70 Get rid of preprocessor stuff in test (#1202)
Also it's a bit more idiomatic to say s.npos rather than string::npos.
2024-06-10 07:00:37 -07:00
Jōshin 32643e9fa7 Decouple swap from std (#1211)
This allows you to implement your own swap function without it having to
be part of the std namespace. std::swap is still used if it's available.
2024-06-10 03:40:17 -07:00
Jōshin 0a92c78035 Remove erroneous ctl:: prefixes 2024-06-08 15:07:39 -07:00
Jōshin 118db71121 Provide a minimal new.h for CTL (#1205)
This replaces the STL <new> header. Mainly, it defines a global operator
new and operator delete, as well as the placement versions of these. The
placement versions are required to not get compile errors when trying to
write a placement new statement.

Each of these operators is defined with many, many different variants. A
glance at new.cc is recommended followed by a chaser of the Alexandrescu
talk "std::allocator is to Allocation as std::vector is to Vexation". We
must provide a global-namespace source-level definition of each operator
and it is illegal for any of them to be marked inline, so here we are.

The upshot is that we no longer need to include <new>, and our optional/
vector headers are self-contained.
2024-06-08 15:05:38 -07:00
Alkis Evlogimenos a0410f0170 Make big_string pod (#1204)
`big_string` is not pod which means it needs to be properly constructed
and destroyed. Instead make it POD and destroy it manually in `string`
destructor.
2024-06-08 10:02:33 -07:00
Alkis Evlogimenos d44a7dc603 Fix bugs in in ctl::optional (#1203)
Manually manage the lifetime of `value_` by using an anonymous
`union`. This fixes a bunch of double-frees and double-constructs.

Additionally move the `present_` flag last. When `T` has padding
`present_` will be placed there saving `alignof(T)` bytes from
`sizeof(optional<T>)`.
2024-06-07 20:47:24 -04:00
Jōshin 2ba6b0158f Fix some memory issues with ctl::string (#1201)
There were a few errors in how capacity and memory was being handled for
small strings. The capacity errors meant that small strings would become
big strings too soon, and the memory error introduced undefined behavior
that was caught by CheckMemoryLeaks in our test file but only sometimes.

The crucial change is in reserve: we only copy n bytes into p2, and then
we manually set the null terminator instead of expecting it to have been
there already. (E.g. it might not be there for an empty small string.)

We also fix one other doozy in append when we were exactly at the small-
to-big string boundary: we set the last byte (i.e., the remainder field)
to 0, then decremented it, giving us size_t max. Whoops. We boneheadedly
fix this by setting the 0 byte after we've fixed up the remainder, so it
is at worst a no-op.

Otherwise, capacity now works the same for small strings as it does with
big strings: it's the amount of space available including the null byte.

We test all of this with a new test that only gets included if our class
under test is not std::string (presumably meaning it's ctl::string.) The
test manually verifies that the small string optimization behaves how we
expect.

Since this test checks against std::string, we go ahead and include that
other header from the STL.

Also modifies the new test we introduced to also run on std::string, but
it just does the append without expecting anything about how its data is
stored. We also check that the string has the right value afterwards.
2024-06-07 01:15:37 -04:00
Jōshin f3effcb703 One more SSO erratum from #1199
Making a string_view from a string appears to take about 1.3ns no matter
what. 100% definitely no point deviating from the STL API over that.
2024-06-06 18:01:26 -07:00
Jōshin 03b476f943 Minor small-string errata from #1199
These commits were sitting on a local branch that I neglected to push
before merging. :(

* Use memcpy for string::reserve

* Remove fence comments
2024-06-06 17:56:30 -07:00
Jōshin 8b3e368e9a ctl::string small-string optimization (#1199)
A small-string optimization is a way of reusing inline storage space for
sufficiently small strings, rather than allocating them on the heap. The
current approach takes after an old Facebook string class: it reuses the
highest-order byte for flags and small-string size, in such a way that a
maximally-sized small string will have its last byte zeroed, making it a
null terminator for the C string.

The only flag we have is in the highest-order bit, that says whether the
string is big (set) or small (cleared.) Most of the logic switches based
on the value of this bit; e.g. data() returns big()->p if it's set, else
small()->buf if it's cleared. For a small string, the capacity is always
fixed at sizeof(string) - 1 bytes; we store the length in the last byte,
but we store it as the number of remaining bytes of capacity, so that at
max size, the last byte will read zero and serve as our null terminator.

Morally speaking, our class's storage is a union over two POD C structs.
For now I gravitated towards a slightly more obtuse approach: the string
class itself contains a blob of the right size, and we alias that blob's
pointer for the two structs, taking some care not to run afoul of object
lifetime rules in C++. If anyone wants to improve on this, contributions
are welcome.

This commit also introduces the `ctl::__` namespace. It can't be legally
spelled by library users, and serves as our version of boost's "detail".

We introduced a string::swap function, and we now use that in operator=.
operator= now takes its argument by value, so we never need to check for
the case where the pointers are equal and can just swap the entire store
of the argument with our own, leaving the C++ destructor to free our old
storage afterwards.

There are probably still a few places where our capacity is slightly off
and we grow too fast, although there don't appear to be any where we are
too slow. I will leave these to be fixed in future changes.
2024-06-06 20:50:51 -04:00
Brian df6b384e31 github: add labeler action (#1196)
Bit easier to do this as everything seems to be sorted into logical
folders. You may need to add new labels to support this however.
2024-06-06 05:51:36 -07:00
Brian 280bdec817 github: add issue template (#1195)
copy paste and adapted from llamafile but added a research template as
well for the more technical research tickets
2024-06-06 05:51:06 -07:00
Jōshin 2c5e7ec547 Add terminating :vi on some modelines
Noticed because the settings they specified weren't getting picked up by
editor sessions in those files.
2024-06-05 20:36:55 -07:00
Jōshin fdcb8b2f7e add formatting commit 2024-06-05 16:36:34 -07:00
Jōshin 04c6bc478e vim C++ filetype is still spelled "cpp" 2024-06-05 16:34:47 -07:00
Justine Tunney cc2c1893c5 Fix some nits 2024-06-05 04:05:49 -07:00
Justine Tunney 3093f0e467 Release Cosmopolitan v3.4.0 2024-06-05 03:07:03 -07:00
Justine Tunney 3609f65de3 Make malloc() go 200x faster
If pthread_create() is linked into the binary, then the cosmo runtime
will create an independent dlmalloc arena for each core. Whenever the
malloc() function is used it will index `g_heaps[sched_getcpu() / 2]`
to find the arena with the greatest hyperthread / numa locality. This
may be configured via an environment variable. For example if you say
`export COSMOPOLITAN_HEAP_COUNT=1` then you can restore the old ways.
Your process may be configured to have anywhere between 1 - 128 heaps

We need this revision because it makes multithreaded C++ applications
faster. For example, an HTTP server I'm working on that makes extreme
use of the STL went from 16k to 2000k requests per second, after this
change was made. To understand why, try out the malloc_test benchmark
which calls malloc() + realloc() in a loop across many threads, which
sees a a 250x improvement in process clock time and 200x on wall time

The tradeoff is this adds ~25ns of latency to individual malloc calls
compared to MODE=tiny, once the cosmo runtime has transitioned into a
fully multi-threaded state. If you don't need malloc() to be scalable
then cosmo provides many options for you. For starters the heap count
variable above can be set to put the process back in single heap mode
plus you can go even faster still, if you include tinymalloc.inc like
many of the programs in tool/build/.. are already doing since that'll
shave tens of kb off your binary footprint too. Theres also MODE=tiny
which is configured to use just 1 plain old dlmalloc arena by default

Another tradeoff is we need more memory now (except in MODE=tiny), to
track the provenance of memory allocation. This is so allocations can
be freely shared across threads, and because OSes can reschedule code
to different CPUs at any time.
2024-06-05 02:02:14 -07:00
Justine Tunney 9906f299bb Refactor and improve CTL and other code 2024-06-04 05:45:48 -07:00
Justine Tunney 1d8f37a2f0 Fix the MODE=tiny builds 2024-06-03 10:36:38 -07:00
Justine Tunney e677460d14 Delete .vscode folder
It hasn't been maintained in years. I'm tired of the root level of our
project having an advertisement for Microsoft Visual Studio Code. Your
preferred editor should be Emacs or Vim.
2024-06-03 09:40:45 -07:00
Justine Tunney 4937843f70 Introduce Cosmopolitan Templates Library (CTL) 2024-06-03 09:21:59 -07:00
Justine Tunney b003888696 Make __demangle() heap 10% more compact 2024-06-02 16:18:55 -07:00
Justine Tunney 2ca491dc56 Write more __demangle() tests 2024-06-02 07:37:15 -07:00
Justine Tunney 9aa353d88b Document __demangle() and fix a const func ptr bug 2024-06-02 04:15:48 -07:00
Justine Tunney c67faf61df Delete some unintentional code 2024-06-01 20:36:58 -07:00
Justine Tunney 165c6b37e2 Add C++ demangling to privileged runtime
Cosmo will now print C++ symbols correctly in --ftrace logs and
backtraces. Doing this required reducing the memory requirement
of the __demangle() function by 3x. This was accomplished using
16-bit indices and 16-bit malloc granularity. That puts a limit
on the longest symbol we can successfully decode, which I think
would be around 6553 characters long, given a 65536-byte buffer
2024-06-01 20:10:58 -07:00
Jōshin dcd626edf8 Add .git-blame-ignore-revs
If you follow the directions in that file then git blame will ignore the
listed commits. A commit should only go in that file if its only changes
were to formatting, particularly on a large part of the codebase (like a
change to .clang-format getting applied to the repo.)

Cribbed from here:

https://www.stefanjudis.com/today-i-learned/how-to-exclude-commits-from-git-blame/
2024-06-01 20:10:48 -07:00
Jōshin f032b5570b Run clang-format (#1197) 2024-06-01 16:30:43 -04:00
Justine Tunney ea081b262c Add some noexcept annotations 2024-06-01 03:19:53 -07:00
Justine Tunney fae1c32267 Encode ±INFINITY as ±1e5000
The V8 behavior of encoding infinity as null doesn't make sense to me.
Using ±1e5000 is better, because JSON.parse decodes it as INFINITY and
the information is preserved. This could be a breaking change for some
2024-06-01 03:19:50 -07:00
Justine Tunney 9b6718ac99 Improve backtraces
We're now able to rewind the instruction pointer in x86 backtraces. This
helps ensure addr2line cannot print information about unrelated adjacent
code. I've restored -fno-schedule-insns2 in most cases because it really
does cause unpredictable breakage for backtraces.
2024-05-30 15:23:11 -07:00
Justine Tunney cd672e251f Improve crash signal reporting on Windows
This change fixes a bug where exiting a crash signal handler on Windows
after adding the signal to uc_sigmask, but not correcting the CPU state
would cause the signal handler to loop infinitely, causing process hang

Another issue is that very tiny programs, that don't link posix signals
would not have their SIGILL / SIGSEGV / etc. status reported to Cosmo's
bash shell when terminating on crash. That's fixed by a tiny handler in
WinMain() that knows how to map WIN32 crash codes to the POSIX flavors.
2024-05-30 14:04:10 -07:00
Justine Tunney 500a47bc2f Fix undefined behavior in unit test
Fixes #1194
2024-05-29 20:31:46 -07:00
Justine Tunney e4d25d68e4 Drop support for Windows 8
Microsoft caused some very gentle breakages for Cosmopolitan. They
removed the version information from the PEB which caused uname to
report WINDOWS 0.0.0. We should have called GetVersionExW but that
doesn't really exist anymore either. Windows policy is now to give
whatever version we used in ape/ape.S. Windows8 has been EOL since
2023-01-10 so lets avoid our modern executables being relegated to
legacy infrastructure. Requiring Windows 10+ going forward lets us
remove runtime compatibility bloat from the codebase. Further note
Cosmopolitan maintains a Windows Vista branch on GitHub, so anyone
preferring the older versions, can still have a future with Cosmo.

Another neat thing this fixes is UTF-8 support in the console. The
changes Microsoft made broke the if statement that enabled UTF8 in
terminals. This explains why bug reports had broken arrows. In the
future this should be less of an issue, since the PEB code is gone
which means we more strictly conform to only Microsoft's WIN32 API
2024-05-29 19:37:47 -07:00
Justine Tunney f31a98d50a Fix bug with realpath() on Windows 2024-05-29 18:47:01 -07:00
Justine Tunney 2816df59b2 Increase tinymalloc granularity 2024-05-29 18:26:01 -07:00
Justine Tunney a05ce3ad9d Support avx512f + vpclmulqdq crc32() acceleration
Cosmo's _Cz_crc32() function now goes 73 GiB/s on Threadripper. This
will significantly improve the performance of the PKZIP file format.
This algorithm is also used by apelink, to create deterministic ids.
2024-05-29 10:13:37 -07:00
Justine Tunney 7c8df05042 Improve -march=native micro-architecture detection 2024-05-29 10:12:49 -07:00
Justine Tunney 4c77acdfcf Add LoadZipArgs() to <cosmo.h> 2024-05-29 10:12:20 -07:00
Justine Tunney b74b974cfd Introduce #include <tinygetopt.h>
The normal getopt() function is bloated because it links printf(). This
change exports the original authentic bsd getopt function, that cosmo's
always used internally so cosmocc users don't need to include internals
2024-05-29 10:11:17 -07:00
Justine Tunney 07cef612c3 Make dlmalloc 2.4x faster for multithreading
This change adds a TLS freelist for small dynamic memory allocations.
Cosmopolitan's TIB is now 512 bytes in size. Single-threaded malloc()
performance isn't impacted by this, until pthread_create() is called.
Single-threaded programs may also want to consider using:

    #include "libc/mem/tinymalloc.inc"

Which will shave 30k off the executable size and sometimes go faster.
2024-05-28 11:18:34 -07:00
Justine Tunney deaef81463 Favor siginfo_t over struct siginfo 2024-05-28 02:34:17 -07:00
Justine Tunney c638eabfe0 Fix compiler warning 2024-05-27 02:23:24 -07:00
Justine Tunney 8e68384e15 Upgrade to 2022-era LLVM LIBCXX 2024-05-27 02:12:27 -07:00
Justine Tunney 2f4ca71f26 Release Cosmopolitan v3.3.10 2024-05-26 22:13:45 -07:00
Justine Tunney 07004ebf04 Upgrade to superconfigure z0.0.42 2024-05-26 22:12:25 -07:00
Justine Tunney 086d7006da Improve crash handler on XNU
This avoids an issue where a crash signal could cause the MacOS process
to freeze and consume all CPU rather than dying as it rightfully should
2024-05-26 18:42:09 -07:00
Gavin Hayes 0a51241f7a ntspawn: fix initializing NtStartupInfoEx (#1190) 2024-05-26 20:54:09 -04:00
Justine Tunney c68f6599e5 Fix definition of getpeername on FreeBSD
We were using the COMPAT magic number, which was recently removed.
2024-05-26 17:03:22 -07:00
Justine Tunney af3f62a71a Ensure io requests are always capped at 0x7ffff000
This gives us the Linux behavior across platforms.

Fixes #1189
2024-05-26 16:53:13 -07:00
Justine Tunney 6cf9b9e0fc Release Cosmopolitan v3.3.9 2024-05-26 15:28:03 -07:00
Justine Tunney 1d4b452839 Refactor some code 2024-05-26 06:03:50 -07:00
Justine Tunney c2db3b703a Introduce --timelog=FILE flag to GNU Make 2024-05-25 14:50:20 -07:00
Justine Tunney edb03b89d8 Make stdin unbuffered when appropriate 2024-05-25 07:57:13 -07:00
Justine Tunney 7724664b13 Release Cosmopolitan v3.3.8 2024-05-25 05:59:20 -07:00
Justine Tunney 1df4296208 Fix stdio for character device regression
Caused by ed93fc3dd7
2024-05-25 05:58:09 -07:00
Justine Tunney ce9aeb2aed Release Cosmopolitan v3.3.7 2024-05-24 19:37:21 -07:00
Justine Tunney ed93fc3dd7 Fix fread() with 2gb+ sizes 2024-05-24 19:28:23 -07:00
Justine Tunney 5f61d273e4 Add hwap constants to sys/auxv.h 2024-05-24 11:44:44 -07:00
Justine Tunney bf3531de81 Make crash reports reliable in multithreaded case 2024-05-24 11:44:44 -07:00
Justine Tunney 3b0ea2db4d Remove -fno-schedule-insns2
This was originally added when we were using the mcount hooking
technique, to prevent the nop from floating above the frame pointer
creation. Now we use a better codegen technique for hooking so it's no
longer needed to sacrifice performance by using this flag.
2024-05-24 11:44:44 -07:00
Justine Tunney f029375d39 Introduce MAP_HUGETLB 2024-05-24 11:44:44 -07:00
Justine Tunney 9b87dd2b87 Refactor some code 2024-05-24 11:44:44 -07:00
Jōshin 787b04f752 Run all BLAKE2B256 test vectors (#1185) 2024-05-24 10:59:23 -07:00
Jōshin 0768807935 Rename python -> python3 (closes #1144) (#1187)
When we removed the com suffix from ape binaries, we broke the build for
ape's python for any case-insensitive file system, i.e. Windows and XNU,
because there is a third_party/python/Python that gets mirrored in the o
directory with the python object files and clashes with the binary name.
This patch hacks around this by renaming the binary to "python3" so that
it no longer clashes with that directory.
2024-05-24 10:56:33 -07:00
Justine Tunney 0b59f01b43 Put confstr() in unistd.h
Fixes #1184
2024-05-21 15:35:06 -07:00
Justine Tunney cf70a44756 Support shebang on Windows
Fixes #1010
2024-05-20 22:11:42 -07:00
Justine Tunney a52792c59f Add some missing build dependencies 2024-05-20 17:23:25 -07:00
Jōshin 65c9b28e99 Fix buffer overflow in os.tmpname (#1180)
At least on macOS, `strlen(getenv("TMPDIR"))` is 50. We now allow a /tmp
that takes up to 120 or so bytes to spell. Instead of overflowing, we do
a bounds check and the function fails successfully on even longer /tmps.

Fixes #1108 (os.tmpname crashes redbean)
2024-05-20 03:46:27 -04:00
Jōshin 4292348707 Import libbsd readpassphrase (#1182)
Included from:

https://gitlab.freedesktop.org/libbsd/libbsd

At commit 04a24db27ad1572f766bad772cdd9c146e6d9cf0.
2024-05-20 03:37:30 -04:00
Jōshin 2ec20b6c4c Fix #922 (Duplicated line) (#1178) 2024-05-20 00:33:50 -07:00
Jōshin 47183551d6 Fix sleb64 (#1179)
Fixes #920
2024-05-20 00:33:17 -07:00
Jōshin 42891e82bb Fix #959 (typo in cosmocc --version) 2024-05-18 15:20:27 -07:00
Gavin Hayes 624119ea38 Fix NT accept/connect not initializing with SO_UPDATE_*_CONTEXT (#1164) 2024-05-17 02:45:30 -07:00
Justine Tunney 2f3c6e7cc3 Revert "Remove zlib namespacing (#1142)"
This reverts commit 5488f0b2ca which was a
good experiment to try, that didn't work out due to #1176

Fixes #1176
2024-05-14 20:45:23 -07:00
Justine Tunney 13b9ecd537 Remove outdated restriction on -Werror on cosmocc
This was a good idea back when we were only using it to build various
open source projects. However it no longer makes sense that many more
people are depending on cosmocc, to develop new software. Our tooling
shouldn't be making these kinds of decisions for the user.
2024-05-09 14:03:51 -07:00
Justine Tunney 6659981457 Release Cosmopolitan v3.3.6 2024-05-08 04:20:22 -07:00
Justine Tunney ae2a7ac844 Fix thread-local storage bugs on aarch64
This change fixes an issue where .tbss memory might not be initialized.
2024-05-08 04:20:22 -07:00
Justine Tunney 793393a341 Polyfill SA_RESETHAND on MacOS ARM64
This change solves the XNU crash loop mystery. Apple's documentation
claims to support this feature, but they only define the constant in
their header files. The kernel acknowledges thi SA_RESETHAND bit, by
clearing it from the sa_flags state, returns zero, and does nothing.
2024-05-08 04:20:22 -07:00
Justine Tunney 952b9009e8 Avoid crash looping on AARCH64 2024-05-08 04:20:22 -07:00
Jōshin 8f18b3ad65 Reapply apeinstall.sh binfmt flags (#1171)
7d31fc3 made it safe to register ape with the binfmt P flag. Older files
will still get their paths passed via argv[0] and everything should just
work. This also restores the F flag that was rolled back alongside the P
flag; this seems like a good idea to me now. It makes it so /usr/bin/ape
is kind of part of the kernel; simply replacing the file will not change
the loader used. The binfmt will have to be reregistered as well.

ape/apeinstall.sh will already nag you if there's an existing binfmt for
ape. It might be nice to make it actually replace the registration.

This reverts commit df648fb174.
2024-05-08 01:48:48 -04:00
Jōshin 56e0fd9d1d Use latest cosmocc (#1175) 2024-05-08 00:08:26 -04:00
Justine Tunney df68a6362b Release Cosmopolitan v3.3.5 2024-05-07 18:18:10 -07:00
Justine Tunney 19c81863a3 Improve crash backtrace reliability
We're now able to pretty print a C++ backtrace upon crashing in pretty
much any runtime execution scenario. The default pledge sandbox policy
on Linux is now to return EPERM. If you call pledge and have debugging
functions linked (e.g. GetSymbolTable) then the symbol table shall get
loaded before any security policy is put in place. This change updates
build/bootstrap/fixupobj too and fixes some other sneaky build errors.
2024-05-07 18:10:28 -07:00
Jōshin 7d31fc311a Loaders rewrite argv[0] for old binaries (#1170)
For this to work, a loader has to be able to tell the difference between
an ‘old’ and a ‘new’ binary. This is achieved via a repurposing of ELF’s
e_flags field. We previously tried to use the padding in e_ident for it,
but binutils was resetting it to zero in e.g. strip.

This introduces one new ELF flag for cosmopolitan binaries. It is called
`EF_APE_MODERN`. We choose 0x101ca75, "lol cat 5".

It should now be safe to install the ape loader binfmt registration with
the `P` flag.
2024-05-07 20:42:18 -04:00
Justine Tunney 57c0b065c8 Make old C++ demangler asynchronous signal safe
It's now possible to safely print C++ backtraces from signal handlers.
This symbol demangler doesn't need malloc, tls, or even static memory.
Additionally, this change makes it 2x faster and adds test cases. It's
almost as performant and accurate as the libcxxabi implementation now.
2024-05-07 03:41:33 -07:00
Justine Tunney a6ecbb747d Introduce libc/mem/tinymalloc.inc
This allocator shaves ~20kb off single-threaded tool programs and is
slightly faster than proper malloc for simple non-demanding programs
2024-05-07 03:19:49 -07:00
Justine Tunney 3bf95ae7ec Invent tool for renaming ELF object symbols 2024-05-07 03:19:49 -07:00
Justine Tunney 5fd7b07fac Improve AVX512 feature detection 2024-05-07 03:19:49 -07:00
Jōshin 317c8bc312 Update MODE=tiny time zone list (#1167)
I took one canonical IANA zone ID from each of the different colored
regions in this article, except those that do not observe DST and do
not have a Google office. See the "Time in Europe" Wikipedia article.

As to which canonical ID to use, this was somewhat arbitrary. Brussels
was obvious, as the de facto capital of the EU. For the rest, I mostly
just went with lexicographic ordering of the most recognizable options.

I've sorted the American zones. This Keeps the U.S. ones together but
does everything alphabetically otherwise. I've added the remaining
Canadian zones These have DST (and Newfoundland is off by a half-
hour from a UTC interval) so they cannot use Etc/. The Pacific/ zones
are sort of sorted. The Chathan Islands have been added. This is the
last of the zones I believe with a non-integer hour offset from UTC.
2024-05-06 16:48:49 -07:00
Jōshin 2a1f352622 Tell vim not to mess with ape.S 2024-05-06 15:36:35 -07:00
Justine Tunney 06d916b449 Add VirtualAlloc2 WIN32 API 2024-05-04 23:26:40 -07:00
Justine Tunney 9ea64725b6 Fix build error 2024-05-04 23:23:56 -07:00
Justine Tunney f9fc7eb49f Fix MODE=dbg build errors 2024-05-04 23:20:12 -07:00
Justine Tunney b0df6c1fce Implement proper time zone support
Cosmopolitan now supports 104 time zones. They're embedded inside any
binary that links the localtime() function. Doing so adds about 100kb
to the binary size. This change also gets time zones working properly
on Windows for the first time. It's not needed to have /etc/localtime
exist on Windows, since we can get this information from WIN32. We're
also now updated to the latest version of Paul Eggert's TZ library.
2024-05-04 23:06:37 -07:00
Justine Tunney d5ebb1fa5b Add MapViewOfFile3 WIN32 API 2024-05-04 12:25:41 -07:00
Justine Tunney 130fd66f9e Release Cosmopolitan v3.3.4 2024-05-03 09:25:02 -07:00
Justine Tunney 8a44f913ae Delete flaky tests
Signals are extremely difficult to unit test reliably. This is why
functions like sigsuspend() exist. When testing something else and
portably it becomes impossible without access to kernel internals.

OpenMP flakes in QEMU on one of my workstations. I don't think the
support is production worthy, because there's been issues on MacOS
additionally. It works great for every experiment I've used it for
though. However a flaky test is worse than no test at all. So it's
removed until someone takes an interest in productionizing it.
2024-05-03 09:11:04 -07:00
Gautham 5488f0b2ca Remove zlib namespacing (#1142)
We have an optimized version of zlib from the Chromium project.
We need it for a lot of our libc services. It would be nice to export
this to user applications if we can, since projects like llamafile are
already depending on it under the private namespace, to avoid
needing to link zlib twice.
2024-05-03 08:07:25 -07:00
Gavin Hayes deff138e7e recvfrom: don't convert address if addrsize is 0 (#1153) 2024-05-03 08:03:57 -07:00
Gavin Hayes b6e40a3a58 Add /dev/(u)random on NT (#1163) 2024-05-03 07:59:51 -07:00
Cadence Ember 8f6bc9dabc Let signals interrupt fgets unless SA_RESTART set (#1152) 2024-05-03 07:49:41 -07:00
Justine Tunney 181cd4cbe8 Add sysctlbyname() for MacOS 2024-05-02 23:21:43 -07:00
Justine Tunney 5c6877b02b Introduce support for trapping math
The feenableexcept() and fedisableexcept() APIs are now provided which
let you detect when NaNs appear the moment it happens from anywhere in
your program. Tests have also been added for the mission critical math
functions expf() and erff(), whose perfect operation has been assured.
See examples/trapping.c to see how to use this powerful functionality.
2024-04-30 13:38:43 -07:00
Justine Tunney 403bc25412 Make MXCSR editable by signal handlers on Windows
It's now possible to recover from floating point exception traps.
2024-04-30 13:38:43 -07:00
Matt Colyer 3bcd40be12 Fix regression in apeinstall.sh (#1161)
This should have been a part of a6baba1.
2024-04-29 20:40:38 -07:00
Paul Kulchenko fcfdd26b72 Fix wrong parameter type in redbean documentation (#1162) 2024-04-29 20:09:25 -04:00
Justine Tunney 2b00ee8723 Fix errno.h documentation typos
Closes #1156
2024-04-28 22:57:21 -07:00
Justine Tunney ff2b6fc0b0 Add munlock() and mlock2() to pledge 2024-04-28 16:53:23 -07:00
Justine Tunney 0eef971494 Add much of C11 threads.h API 2024-04-28 07:04:08 -07:00
Gavin Hayes 2bfd6b37c1 Various paginate improvements (#1148)
* start on improving __paginate

* make __paginate more robust

* add __paginate_file

* cleanup __paginate unlinking
2024-04-26 23:12:25 -04:00
Gavin Hayes 69db501c68 Fix fork locking on win32 (#1141)
* Fix fork locking on win32

- __enable_threads / set __threaded in __proc_setup as threads are required for
  win32 subprocess management
- move mmi/fds locking out of pthread_atfork.c into fork.c so it's done anytime
  __threaded is set instead of being dependent of pthreads
- explicitly yoink _pthread_onfork_prepare, _pthread_onfork_parent, and
  _pthread_onfork_child in pthread_create.c so they are linked in in-case they
  are separated from _pthread_atfork

Big Thanks to @dfyz for help with locating the issue, testing, and devising a fix!

* fix child processes not being able to open files, initialize all necessary locks on fork
2024-04-25 23:01:27 -04:00
Jōshin 6e6fc38935 Apply clang-format update to repo (#1154)
Commit bc6c183 introduced a bunch of discrepancies between what files
look like in the repo and what clang-format says they should look like.
However, there were already a few discrepancies prior to that. Most of
these discrepancies seemed to be unintentional, but a few of them were
load-bearing (e.g., a #include that violated header ordering needing
something to have been #defined by a 'later' #include.)

I opted to take what I hope is a relatively smooth-brained approach: I
reverted the .clang-format change, ran clang-format on the whole repo,
reapplied the .clang-format change, reran clang-format again, and then
reverted the commit that contained the first run. Thus the full effect
of this PR should only be to apply the changed formatting rules to the
repo, and from skimming the results, this seems to be the case.

My work can be checked by applying the short, manual commits, and then
rerunning the command listed in the autogenerated commits (those whose
messages I have prefixed auto:) and seeing if your results agree.

It might be that the other diffs should be fixed at some point but I'm
leaving that aside for now.

fd '\.c(c|pp)?$' --print0| xargs -0 clang-format -i
2024-04-25 10:38:00 -07:00
Jōshin 342d0c81e5 vim spells the c++ filetype 'cpp' 2024-04-24 13:56:37 -07:00
Jōshin cec4c04aa1 Remove .com logic from GetProgramExecutableName (#1134)
The `com` parameter to `TryPath` was always 1, so there was no reason to
have it. This patch changes the logic to be as though `com` was 0, which
provides a possible answer to the TODO question -- the answer is no.

If we never care about appending `.com`, then `CopyWithCwd` doesn't need
to return anything beyond a boolean success value.
2024-04-23 22:07:19 -04:00
Alexey Izbyshev f8c0186221 Fix calling __dns_parse with potentially too large rlen
__res_send returns the full answer length even if it didn't fit the
buffer, but __dns_parse expects the length of the filled part of the
buffer.

Analogous to Musl commit 77327ed064bd57b0e1865cd0e0364057ff4a53b4 which
fixed the only other __dns_parse call site.
2024-04-23 09:36:07 -07:00
Quentin Rameau 6992d8c195 Remove arbitrary limit from DNS result parsing
The name resolution would abort when getting more than 63 records per
request, due to what seems to be a left-over from the original code.
This check was non-breaking but spurious prior to TCP fallback
support, since any 512-byte packet with more than 63 records was
necessarily malformed. But now, it wrongly rejects valid results.

Reported by Daniel Stefanik in Alpine Linux aports issue 15320.
2024-04-23 09:33:02 -07:00
Justine Tunney 1a6b4ab627 Import mntent bug fixes from Musl Libc
f314e133929b6379eccc632bef32eaebb66a7335
Author: Rich Felker <[email protected]>
Date:   Thu Nov 16 12:55:21 2023 -0500

    mntent: fields are delimited only by tabs or spaces, not general whitespace

    this matters because the kernel-provided mtab only escapes tabs,
    spaces, newlines, and backslashes. it leaves carriage returns, form
    feeds, and vertical tabs literal.

commit ee1d39bc1573c1ae49ee6b658938b56bbef95a6c
Author: q66 <[email protected]>
Date:   Thu Nov 9 20:48:44 2023 +0100

    mntent: unescape octal sequences

    As entries in mtab are delimited by spaces, whitespace characters
    are escaped as octal sequences. When reading them out, we have to
    unescape these sequences to get the proper string.
2024-04-23 09:29:56 -07:00
Justine Tunney 9e848abad9 Add missing Musl license headers 2024-04-23 09:29:46 -07:00
Justine Tunney 223eb358c4 Import compiler warning fix from Musl 2024-04-23 09:29:28 -07:00
Szabolcs Nagy ea3cc4ea0b math: fix fma(x,y,0) when x*y rounds to -0
if x!=0, y!=0, z==0 then

  fma(x,y,z) == x*y

in all rounding modes, while adding z can ruin the sign of 0 if x*y
rounds to -0.
2024-04-23 09:22:33 -07:00
Paul Kulchenko 01267ea0f5 Update GetRemoteAddr documentation to clarify that it may return nil. (#1149) 2024-04-22 01:55:08 -04:00
Justine Tunney bc6c18368f Update .clang-format 2024-04-19 17:11:08 -07:00
Jōshin 06839ab301 Change loop bound on uuidv4 test (#1143)
A half second (on my machine) is too long for a unit test. 1000
iterations is probably still overkill, but 0.01 seconds is fine.

Also made `y` local.
2024-04-12 12:22:34 -04:00
BONNAURE Olivier 39dde41516 [Redbean] Add UuidV4 method (#1140) 2024-04-12 11:10:27 -04:00
Jōshin 3e16e59f72 Move store after __asan_unpoison in __zipos_alloc (#1135)
Previously, the atomic store looked like it was happening while the
struct's memory was still poisoned. I was unable to observe any issues
with this, but this change seems to make the code more obviously correct
(at the cost of a redundant atomic store to zeroed space in case the map
needed to be extended.)
2024-04-10 23:04:03 -04:00
Justine Tunney cf9a1f7f33 Fix MODE=optlinux build 2024-04-06 21:02:19 -07:00
Justine Tunney 49a32136f8 Upgrade the One True Awk 2024-04-06 19:21:48 -07:00
Justine Tunney b9d6e6e348 Standardize on -fsigned-char
MaGuess on Discord pointed out the fact that cosmocc contradicts itself
on the signedness of `char`. It's up to each platform to choose one, so
the cosmo platform shall choose signed. The rationale is it makes the C
language syntax more internally similar. `char` should be `signed char`
for the same reason `int` means `signed int`. It's recommended that you
still assume `char` could go either way since that's portable thinking.
But if you want to assume we'll always have signed char, that's ok too.
2024-03-30 21:46:08 -07:00
Justine Tunney b431ca4449 Upgrade to cosmocc v3.3.3 2024-03-30 20:17:14 -07:00
Justine Tunney 98d06b23cc Release Cosmopolitan v3.3.3 2024-03-30 19:24:04 -07:00
Justine Tunney 9ff65cff63 Fix ZipOS extraction of ZIP64 assets 2024-03-30 19:00:40 -07:00
Justine Tunney bef2cb05ff Fix AVX512_BF16 2024-03-26 00:00:46 -07:00
mierenhoop 43885a76e4 Fix binary formatting for integers 2 and 3 (#1123) 2024-03-25 21:11:09 -07:00
Justine Tunney 74a85087de Support -S in cosmocross
You can now run commands like `x86_64-unknown-cosmo-c++ -S` when using
your cosmocc toolchain. Please note the S flag isn't supported for the
cosmocc command itself.
2024-03-24 15:49:55 -07:00
Justine Tunney 045632a743 Make fixupobj work on Windows
This fixes a regression with recent cosmocc toolchain releases that
causes fixupobj to fail with "ifunc ftruncate #1 failed" on Windows
2024-03-24 03:14:25 -07:00
Justine Tunney 9a10adac35 Remove .com from README 2024-03-22 19:38:38 -07:00
Justine Tunney 0c12c26e1d Update Emacs config 2024-03-22 19:27:11 -07:00
Justine Tunney 40b7da8422 Speed up fmaf() on x86 2024-03-22 19:26:56 -07:00
Justine Tunney 640668931d Clarify MacOS supported versions
We technically support versions much earlier, possibly dating back to
2018 or earlier, but they're not officially supported since I have no
way of testing them.

Closes #1129
2024-03-22 19:25:07 -07:00
Justine Tunney 1c34088ba0 Ignore -pie and -fpie in cosmocc
Fixes #1126
2024-03-22 03:55:01 -07:00
Justine Tunney bb92347158 Add deprecated stubs for _bsf / _bsr 2024-03-05 09:45:51 -08:00
Justine Tunney 8bfd56b59e Rename _bsr/_bsf to bsr/bsf
Now that these functions are behind _COSMO_SOURCE there's no reason for
having the ugly underscore anymore. To use these functions, you need to
pass -mcosmo to cosmocc.
2024-03-04 17:33:26 -08:00
Justine Tunney a6baba1b07 Stop using .com extension in monorepo
The WIN32 CreateProcess() function does not require an .exe or .com
suffix in order to spawn an executable. Now that we have Cosmo bash
we're no longer so dependent on the cmd.exe prompt.
2024-03-03 03:12:19 -08:00
Justine Tunney c8383f25b4 Release Cosmopolitan v3.3.2 2024-02-27 09:15:59 -08:00
Justine Tunney 64a9e6fe56 Fix compiler runtime for _Float16 type 2024-02-27 09:06:23 -08:00
Justine Tunney 0ef36489c8 Walk back most uses of __STRICT_ANSI__ 2024-02-27 04:09:49 -08:00
Justine Tunney 7c7bf4bb90 Release Cosmopolitan v3.3.1 2024-02-27 00:31:36 -08:00
Justine Tunney 592f6ebc20 Make quality improvements
- Write some more unit tests
- memcpy() on ARM is now faster
- Address the Musl complex math FIXME comments
- Some libm funcs like pow() now support setting errno
- Import the latest and greatest math functions from ARM
- Use more accurate atan2f() and log1pf() implementations
- atoi() and atol() will no longer saturate or clobber errno
2024-02-25 19:08:43 -08:00
Justine Tunney af8f2bd19f Shave 4kb off each binary 2024-02-25 11:11:34 -08:00
Justine Tunney 38bceaeb8f Give up on making clock_nanosleep() precise
Multiple projects I care about make the assumption that this isn't a
system call that sleeps for a particular number of nanonseconds, but
rather a function that parks processes on kernel scheduler quantums.
Anyone who wants the old behavior should use cosmo_clock_nanosleep()
2024-02-24 10:17:47 -08:00
Ivan Komarov 99f0491f04 Fix invalid XNU binaries generated by apelink in some edge cases (#1106)
* Fix `if...fi` generation in the generated APE shell script

A shell will fail with a syntax error on an empty `if` or `else` body.
That is, neither of these is allowed:

    # Empty `if`
    if [ ... ]; then
    fi

    # Empty `else`
    if [ ... ]; then
    ...
    else
    fi

There were two places where `apelink` could generate problematic `if`'s:

1. The XNU shell generation for aarch64 binaries when no loaders (either
   binary or source) are provided. They can't assimilate, so the resulting
   `else` body becomes empty.
   There is actually a code path guarded by the `gotsome` variable that
   inserts an extra `true` in this case, but the variable was never
   initialized, so in practice this code path didn't activate in my
   tests. This is fixed by initializing the variable.
2. The loader extraction code when no loaders are provided and XNU
   support is requested. This is fixed by adding a simliar code path
   that prevents an empty body from being generated.

* Update the apelink manual after commit d53c335

The `-s` option changed its meaning, but the docs weren't updated.
2024-02-23 08:11:27 -08:00
Justine Tunney e72a88ea70 Make fixups for libcrypt 2024-02-23 07:39:44 -08:00
Ivan Komarov 9b4701b3bc Don't include <crypt.h> from <stdlib.h> (#1112) 2024-02-23 07:18:35 -08:00
Ivan Komarov f7ff515961 *scanf() fixes to make TeX work (#1109)
* Fix reading the same symbol twice when using `{f,}scanf()`

PR #924 appears to use `unget()` subtly incorrectly when parsing
floating point numbers. The rest of the code only uses `unget()`
immediately followed by `goto Done;` to return back the symbol that
can't possibly belong to the directive we're processing.

With floating-point, however, the ungot characters could very well
be valid for the *next* directive, so we will essentially read them
twice. It can't be seen in `sscanf()` tests because `unget()` is a
no-op there, but the test I added for `fscanf()` fails like this:

        ...
        EXPECT_EQ(0xDEAD, i1)
                need 57005 (or 0xdead) =
                 got 908973 (or 0x000ddead)
        ...
        EXPECT_EQ(0xBEEF, i2)
                need 48879 (or 0xbeef) =
                 got 769775 (or 0x000bbeef)

This means we read 0xDDEAD instead of 0xDEAD and 0xBBEEF instead of
0xBEEF. I checked that both musl and glibc read 0xDEAD/0xBEEF, as
expected.

Fix the failing test by removing the unneeded `unget()` calls.

* Don't read invalid floating-point numbers in `*scanf()`

Currently, we just ignore any errors from `strtod()`. They can
happen either because no valid float can be parsed at all, or
because the state machine recognizes only a prefix of a valid
floating-point number.

Fix this by making sure `strtod()` parses everything we recognized,
provided it's non-empty. This requires to pop the last character
off the FP buffer, which is supposed to be parsed by the next
`*scanf()` directive.

* Make `%c` parsing in `*scanf()` respect the C standard

Currently, `%c`-style directives always succeed even if there
are actually fewer characters in the input than requested.

Before the fix, the added test fails like this:

        ...
        EXPECT_EQ(2, sscanf("ab", "%c %c %c", &c2, &c3, &c4))
                need 2 (or 0x02 or '\2' or ENOENT) =
                 got 3 (or 0x03 or '\3' or ESRCH)
        ...
        EXPECT_EQ(0, sscanf("abcd", "%5c", s2))
                need 0 (or 0x0 or '\0') =
                 got 1 (or 0x01 or '\1' or EPERM)

musl and glibc pass this test.
2024-02-23 07:15:30 -08:00
Justine Tunney 3afe3a3646 Fix broken link 2024-02-23 07:11:44 -08:00
Justine Tunney b4faaa7c57 Relax style guide 2024-02-23 06:55:37 -08:00
Justine Tunney c4991e53ac Document copyright policy exceptions 2024-02-23 06:53:46 -08:00
Justine Tunney ad3944a3b6 Support any HTTP method
It's now possible to use redbean Fetch() with arbitrary HTTP methods,
e.g. LIST which is used by Hashicorp. There's an eight char limit and
uppercase canonicalization still happens. This change also includes a
better function for launching a browser tab, that won't deadlock on a
headless workstation running Debian.

Closes #1107
2024-02-22 14:12:18 -08:00
Justine Tunney 29eac8e2a2 Fix regression in logger functions
The recent release broke things so that the logger wasn't logging by
default. This ensures that `__log_file` initializes in correct order
2024-02-22 14:08:20 -08:00
Justine Tunney a5a3e78fcf Make ftrace/strace C APIs work with cosmocc 2024-02-22 12:13:16 -08:00
Justine Tunney e9a756e78b Make --ftrace more crash proof with foreign code
Sometimes we need to interact with code that wasn't compiled using
`-fno-omit-frame-pointer`. For example, if a function pointer gets
passed and called by a foreign function, linked by cosmo_dlopen().
Function call tracing will now detect backtrace pointer corruption
and simply reduce the indentation level back to zero, as a result.
2024-02-22 11:06:31 -08:00
Justine Tunney 38af54a2dd Fix GCC version in cosmocc wrapper script
See #1111
2024-02-21 19:16:39 -08:00
Justine Tunney 77a92f517b Introduce getcpu() system call from glibc 2024-02-21 18:17:20 -08:00
Justine Tunney 3eb405e0e2 Resurrect <windows.h> as <windowsesque.h> 2024-02-21 16:41:11 -08:00
Justine Tunney 68b9479f0c Make cosmo.h work a little better 2024-02-21 16:40:09 -08:00
Justine Tunney d9df41c4d5 Add example email to CONTRIBUTING.md 2024-02-21 16:36:52 -08:00
Justine Tunney fc3c536ec2 Make --ftrace work better with cosmocc
This change causes cosmocc to use -fno-inline-functions-called-once by
default, unless -Os or -finline-functions-called-once is defined. This
is important since I believe it generally makes code go faster, and it
most importantly makes --ftrace output much more understandable, since
the trace will be more likely to reflect the actual shape of the code.
We've always used this flag in the mono repo when ftracing is enabled,
but it slipped my mind to incorporate this into the cosmocc toolchain.
2024-02-21 13:20:21 -08:00
Justine Tunney 77ddf4cb07 Fix fixupobj regression on MacOS ARM64 2024-02-21 12:13:54 -08:00
Justine Tunney 4d018306b3 Fix MODE=optlinux build 2024-02-20 14:24:56 -08:00
Justine Tunney b3bb93d1d9 Fix MODE=opt build 2024-02-20 14:12:12 -08:00
Justine Tunney 980af27bbf Fix documentation build 2024-02-20 13:40:46 -08:00
Justine Tunney 957c61cbbf Release Cosmopolitan v3.3
This change upgrades to GCC 12.3 and GNU binutils 2.42. The GNU linker
appears to have changed things so that only a single de-duplicated str
table is present in the binary, and it gets placed wherever the linker
wants, regardless of what the linker script says. To cope with that we
need to stop using .ident to embed licenses. As such, this change does
significant work to revamp how third party licenses are defined in the
codebase, using `.section .notice,"aR",@progbits`.

This new GCC 12.3 toolchain has support for GNU indirect functions. It
lets us support __target_clones__ for the first time. This is used for
optimizing the performance of libc string functions such as strlen and
friends so far on x86, by ensuring AVX systems favor a second codepath
that uses VEX encoding. It shaves some latency off certain operations.
It's a useful feature to have for scientific computing for the reasons
explained by the test/libcxx/openmp_test.cc example which compiles for
fifteen different microarchitectures. Thanks to the upgrades, it's now
also possible to use newer instruction sets, such as AVX512FP16, VNNI.

Cosmo now uses the %gs register on x86 by default for TLS. Doing it is
helpful for any program that links `cosmo_dlopen()`. Such programs had
to recompile their binaries at startup to change the TLS instructions.
That's not great, since it means every page in the executable needs to
be faulted. The work of rewriting TLS-related x86 opcodes, is moved to
fixupobj.com instead. This is great news for MacOS x86 users, since we
previously needed to morph the binary every time for that platform but
now that's no longer necessary. The only platforms where we need fixup
of TLS x86 opcodes at runtime are now Windows, OpenBSD, and NetBSD. On
Windows we morph TLS to point deeper into the TIB, based on a TlsAlloc
assignment, and on OpenBSD/NetBSD we morph %gs back into %fs since the
kernels do not allow us to specify a value for the %gs register.

OpenBSD users are now required to use APE Loader to run Cosmo binaries
and assimilation is no longer possible. OpenBSD kernel needs to change
to allow programs to specify a value for the %gs register, or it needs
to stop marking executable pages loaded by the kernel as mimmutable().

This release fixes __constructor__, .ctor, .init_array, and lastly the
.preinit_array so they behave the exact same way as glibc.

We no longer use hex constants to define math.h symbols like M_PI.
2024-02-20 13:27:59 -08:00
BONNAURE Olivier d3ff48c63f [Redbean] Feature / OnError(status, message) hook (#1103) 2024-02-14 01:55:50 -08:00
Justine Tunney 2ab9e9f7fd Make improvements
- Introduce portable sched_getcpu() api
- Support GCC's __target_clones__ feature
- Make fma() go faster on x86 in default mode
- Remove some asan checks from core libraries
- WinMain() now ensures $HOME and $USER are defined
2024-02-12 10:23:00 -08:00
Ivan Komarov d5225a693b libc headers: make <complex.h> work, add struct ipv6_mreq (#1100)
1. `libc/isystem/complex.h` (included when you do `#include <complex.h>`)
   defines `_COMPLEX_H`, and then proceeds to include `libc/complex.h`,
   which contains the actual complex-related declarations. However, they
   are *also* guarded by `_COMPLEX_H` and hence effectively ignored.
   Fix this by changing `_COMPLEX_H` to `COSMOPOLITAN_LIBC_COMPLEX_H_`,
   which is consistent with what the other headers (such as `math.h`) do.
2. Cosmopolitan could only support IPv4 multicast requests for sockets,
   since a declaration for `struct ipv6_mreq` was missing. Add support
   for IPv6, too, by adding the missing declaration.
2024-02-05 17:22:56 -05:00
Paul Kulchenko c72904b2f6 Fix redbean Fetch redirect with relative URL (#1034) 2024-02-02 10:02:29 -05:00
Justine Tunney 616717fa82 Fine tune OpenMP some more 2024-01-30 06:30:24 -08:00
Justine Tunney 369aebfc48 Make improvements
- Let OpenMP be usable via cosmocc
- Let libunwind be usable via cosmocc
- Make X86_HAVE(AVXVNNI) work correctly
- Avoid using MAP_GROWSDOWN on qemu-aarch64
- Introduce in6addr_any and in6addr_loopback
- Have thread stacks use MAP_GROWSDOWN by default
- Ask OpenMP to not use filesystem to manage threads
- Make NI_MAXHOST and NI_MAXSERV available w/o _GNU_SOURCE
2024-01-29 16:31:58 -08:00
Justine Tunney 5f8e9f14c1 Add OpenMP support 2024-01-28 22:39:02 -08:00
Justine Tunney c1e18e7903 Restore MODE=dbg support
We recently broke MODE=dbg support when we added C++ exception support.
This change adds the missing UBSAN interfaces, needed to get it working
again. Some of the ASAN checking in the SJLJ guts needed to be disabled
since I doubt anyone's combined the two features until now.
2024-01-26 23:07:18 -08:00
Justine Tunney f27808c4d2 Remove feature for embedding blink in ape scripts
Embedding Blink builds in Cosmo executables was a failed experiment. It
turned out to be easier than expected to let the mono repo have support
for multiple architectures. Blink still works great; it's supported and
recommended; just please use it as a separate program. For example, you
can use Blink to run Cosmo binaries on architectures like i486 / s390x.
2024-01-26 22:30:56 -08:00
Justine Tunney 2719080986 Add program for printing Windows network dev info 2024-01-26 22:29:23 -08:00
Trung Nguyen 33418f6742 libc: Fix __cxa_thread_atexit prototype (#1088)
The first parameter should be a function pointer.

Otherwise, this would conflict with the definition provided by
`libcxxabi`.
2024-01-22 10:23:28 -08:00
Trung Nguyen 83a8686c06 tool/cosmocc: Properly handle dependency output (#1091)
a2753de contains some regressions, causing `fixupobj` to be
inappropriately suppressed when `-MD` or `-MMD` is passed.

This commit reverts most changes by a2753de, and:
- Treats all invocations of the compiler with `-M` and `-MM` as with the
`cpp` intent, since these flags imply `-E`.
- Handle the dependency output path specified by `-MF`.
  + This is trivial for `cosmocross` since the script does not throw
  objects to and from temporary directories.
  + For `cosmocc`, the file names are calculated based on the `-MF`
  value provided by the user. If this flag is not specified, the script
  generates the file name based on the output file using GCC rules.
  Then, before calling the real compilers, an additional `-MF` flag is
  passed to override the dependency outputs with mangled file names.
2024-01-22 10:22:16 -08:00
Justine Tunney 8ab3a545c6 Increase build memory quota
If you install qemu-user from apt then glibc links a lot of address
space bloat that causes pthread_create() to ENOMEM (a.k.a. EAGAIN).
Boosting the virtual memory quota from 512m to 2048m will hopefully
future proof the build for the future, as Linux distros get fatter.
Please note this only applies to MODE=aarch64 on x86_64 builds when
you're using QEMU from Debian/Ubuntu rather than installing the one
cosmo provides in third_party/qemu/qemu-aarch64.gz. This change may
also be useful to people who are using the host compiler toolchain.
2024-01-22 10:02:30 -08:00
Justine Tunney 8ebe2e9020 Fix assertion error in pthread_create() 2024-01-22 10:02:30 -08:00
BONNAURE Olivier 51cd83f674 Add Curve25519 method in help.txt + add new specs file (#1096) 2024-01-21 06:58:48 -08:00
Justine Tunney 39b0a9c03e Fix scanf() %n off by one at eof
Fixes #1094
2024-01-20 15:06:16 -08:00
Justine Tunney d50064a779 Add Curve25519() API to Redbean 2024-01-20 01:06:19 -08:00
Justine Tunney 1226eb7a5e Fix gcc warning 2024-01-18 16:48:02 -08:00
Trung Nguyen 8834dde0c2 libcxx: Add missing implementation source files (#1089)
Added the implementation for `std::bad_any_cast` from upstream
`any.cpp`, and `std::bad_variant_access` from upstream `variant.cpp`.

This fixes missing `vtable` and `typeinfo` symbols when trying to link
code referencing these exception types.
2024-01-18 08:20:25 -08:00
Justine Tunney 1ef63eb206 Retire third_party/quickjs/
QuickJS cosmocc binaries are now being distributed on
https://bellard.org/quickjs/
2024-01-17 12:35:46 -08:00
Brian 08793aa143 Make ... optional in .args (#1086) 2024-01-15 16:20:50 -08:00
Trung Nguyen 6db1200a7e libc: Fix cxxabi.h header (#1087)
- `__cxa_*` runtime functions are expected to be in the `abi` namespace,
which is currently an alias for `__cxxabiv1`.
- Rely on the header provided by `libcxxabi` for functions that we do
not implement ourselves anymore.
2024-01-15 11:50:38 -08:00
Trung Nguyen a2753de7fd tool/cosmocc: Do not run fixupobj for text output (#1084)
Some compiler flags (such as -E or -MM) instruct GCC to only run the
preprocessor and produce certain text files.

In this case, we do not want to run `fixupobj` and make the tool fail
because the input is not an ELF64 binary.
2024-01-15 07:16:13 -08:00
Jōshin 6715b670b1 Skip the CAS on new_pos < 0 (#1080) 2024-01-12 21:08:28 -08:00
Justine Tunney fd75fd1467 Make std::pair trivial 2024-01-09 09:51:59 -08:00
Justine Tunney 5d80e8dbf6 Add more Intel microarchitectures 2024-01-09 01:54:26 -08:00
Justine Tunney eeb20775d2 Add dontthrow attribute to most libc functions
This will help C++ code that uses exceptions to be tinier. For example,
this change shaves away 1000 lines of assembly code from LLVM's libcxx,
which is 0.7% of all assembly instructions in the entire library.
2024-01-09 01:26:03 -08:00
Justine Tunney cb19e172da Release Cosmopolitan v3.2.4 2024-01-08 19:37:59 -08:00
Jōshin df648fb174 Revert apeinstall.sh binfmt flags (#1072)
The P flag breaks backwards compatibility with older binaries. The idea
is to revert this commit after that break has been resolved.
2024-01-08 14:21:21 -08:00
Justine Tunney 9a6af92bed Fix libcxxabi tests in MODE=tiny and MODE=rel
See #1076
2024-01-08 13:49:06 -08:00
Justine Tunney 6ab01716ce Fix aarch64 setjmp abi
We now store values in jmp_buf where the compiler wants them to be. This
fixes code that calls __builtin_setjmp() and __builtin_longjmp() such as
libunwind. All libcxxabi tests are now passing on ARM64.

See #1076
2024-01-08 13:26:28 -08:00
Justine Tunney 21093044c0 Rebuild build/bootstrap/chmod.com 2024-01-08 12:22:12 -08:00
Justine Tunney b8a9fbbd01 Review catch_multi_level_pointer test failure
This test was was failing on GitHub Actions because GA uses Linux and
Linux supports resource usage accounting. Cosmo's compile.com program
imposes CPU, memory and file size limits on both the compiler and the
test programs themselves.

See #1076
2024-01-08 12:10:29 -08:00
Justine Tunney 81ce2e4cbc Call thread finalizers on exit()
While we don't call POSIX thread key destructors from exit(), we do need
to call these, since C++ uses it for TLS object destructors.

See #1076
2024-01-08 11:45:02 -08:00
Justine Tunney 07db3004d6 Disable libcxxabi/test/guard_threaded_test
It works, but exceeds the CPU quota granted to tests, by a lot.
2024-01-08 11:19:09 -08:00
Trung Nguyen b0566348b2 third_party/libcxxabi: Add test suite (#1076)
Added the `libcxxabi` test suite as found in LLVM 17.0.6.

Some tests that do not apply to the current configuration of
comsopolitan are not added. These include:
- `backtrace_test`, `forced_unwind*`: Use unwind function unsupported in
SjLj mode.
- `noexception*`: Designed to test `libcxxabi` in no exceptions mode.

Some tests are added but not enabled due to bugs specific to GCC or
cosmopolitan. These are clearly indicated in the `BUILD.mk` file.
2024-01-08 10:50:06 -08:00
Justine Tunney 8fb24a8f88 Turn exceptions back off by default 2024-01-08 10:47:42 -08:00
Justine Tunney a4b455185b Bring back gc() function
Renaming gc() to _gc() was a mistake since the better thing to do is put
it behind the _COSMO_SOURCE macro. We need this change because I haven't
wanted to use my amazing garbage collector ever since we renamed it. You
now need to define _COSMO_SOURCE yourself when using amalgamation header
and cosmocc users need to pass the -mcosmo flag to get the gc() function

Some other issues relating to cancelation have been fixed along the way.
We're also now putting cosmocc in a folder named `.cosmocc` so it can be
more safely excluded by grep --exclude-dir=.cosmocc --exclude-dir=o etc.
2024-01-08 10:26:28 -08:00
Jōshin 6cb0354e19 mkofs improvements
- Always use $PWD/o. if it doesn't exist, say how to create it.

- return 2 on unsupported platforms.
2024-01-08 17:23:09 +00:00
Trung Nguyen b580080af1 tool/cosmocc: Allow exceptions and rtti (#1074)
With `libunwind` and `libcxxabi` included in `libcosmo`, we can now
allow users to build C++ applications with exceptions and RTTI enabled.

The default is still disabling these two to avoid bloating the binary.

Closes #1065
2024-01-08 08:47:53 -08:00
Trung Nguyen 8b33204f37 Add LLVM libcxxabi (#1063)
* third_party: Add libcxxabi

Added libcxxabi from LLVM 17.0.6
The library implements the Itanium C++ exception handling ABI.

* third_party/libcxxabi: Enable __cxa_thread_atexit

Enable `__cxa_thread_atexit` from libcxxabi.
`__cxa_thread_atexit_impl` is still implemented by the cosmo libc.
The original `__cxa_thread_atexit` has been removed.

* third_party/libcxx: Build with exceptions

Build libcxx with exceptions enabled.

- Removed `_LIBCPP_NO_EXCEPTIONS` from `__config`.
- Switched the exception implementation to `libcxxabi`. These two files
are taken from the same `libcxx` version as mentioned in `README.cosmo`.
- Removed `new_handler_fallback` in favor of `libcxxabi` implementation.
- Enable `-fexceptions` and `-frtti` for `libcxx`.
- Removed `THIRD_PARTY_LIBCXX` dependency from `libcxxabi` and
`libunwind`. These libraries do not use any runtime `libcxx` functions,
just headers.

* libc: Remove remaining redundant cxa functions

- `__cxa_pure_virtual` in `libcxxabi` is also a stub similar to the
existing one.
- `__cxa_guard_*` from `libcxxabi` is used instead of the ones from
Android.

Now there should be no more duplicate implementations.
`__cxa_thread_atexit_impl`, `__cxa_atexit`, and related supporting
functions, are still left to other libraries as in `libcxxabi`.

`libcxxabi` is also now added to `cosmopolitan.a` to make up for the
removed functions.

Affected in-tree libraries (`third_party/double-conversion`) have been
updated.
2024-01-08 08:45:10 -08:00
Trung Nguyen 94bab1618d Add fixes to libunwind (#1069) 2024-01-08 08:31:13 -08:00
Jōshin cafea9a0a5 Flesh out tool/zsh/mkofs
It now prevents double-mounting.

Also adds vim filetype modelines.
2024-01-08 05:18:18 +00:00
Jōshin 16099801cc zsh mkofs tool 2024-01-08 03:43:09 +00:00
Justine Tunney 4cd02c29ed Release Cosmopolitan v3.2.3 2024-01-07 12:39:09 -08:00
Justine Tunney 5ae2554c10 Have cosmo_dlopen() request -z execstack on Linux
The AMD HIP SDK for Linux ships prebuilt DSOs with an RWX PT_GNU_STACK
since old versions of GCC made it nearly impossible to build artifacts
where that wasn't the case, however modern glibc systems will flat out
refuse to link RWX DSOs from an execuatble that uses PT_GNU_STACK = RW
2024-01-07 11:37:18 -08:00
Justine Tunney 531ac85923 Disable Python test_resource
It fails strangely on some Linux systems due to constraints imposed by
the build system. We'll try to learn more about this later on.
2024-01-07 11:36:31 -08:00
Jōshin 8d9fcb5e5a Fix ape-m1.c usage
It's not about `$0` anymore.
2024-01-07 10:35:50 -05:00
Jōshin aa37a327ea Make $prog.ape more reliable on Apple Silicon (#1071)
Now it doesn't matter what argv `$prog.ape` is invoked with. We just get
our executable path the Apple way.
2024-01-07 07:13:20 -08:00
Justine Tunney c4205f8305 Remove lingering dlopen thunk 2024-01-07 02:14:41 -08:00
Justine Tunney b3fb6cff43 Add /dev/fd support to Windows
GNU bash needs this functionality, otherwise it can't do <(cmd...).
2024-01-06 18:20:21 -08:00
Jōshin 2e5f662dfe mmake: different cosmocc regime
Now that cosmocc is unpacked into a version-specific directory under
cosmocc/, it makes more sense to put the versions in /opt/cosmocc and
maintain a symlink to the currently active one.
2024-01-06 19:14:13 -05:00
Justine Tunney 5e7137097d Make breaking change to cosmo_dlsym()
The cosmo_dlsym() function now returns the raw function address. You
need to call cosmo_dltramp() on the result, to make it safe to call.
This change is important, because cosmo_dltramp() magic can't always
work; for some tricky functions, you need to translate ABIs by hand.
2024-01-06 15:45:26 -08:00
Justine Tunney f224a55d57 Fix MODE=dbg build
Fixes #1067
2024-01-06 15:22:19 -08:00
Justine Tunney 0c51995b2b Add missing sha256sum.c file 2024-01-06 14:47:14 -08:00
Jōshin a9f23ccd17 Fix download-cosmocc.sh on Mac (#1066)
shasum behaves enough like sha256sum to work in this case.
2024-01-06 14:46:31 -08:00
Justine Tunney 57d1bf7a9c Clarify CONTRIBUTING.md 2024-01-06 14:18:01 -08:00
Justine Tunney ce17ed60df Support -MF and -MT in cosmocc 2024-01-06 12:10:06 -08:00
Jōshin d27a47b0e2 Bugfix: ape --help should exit 0 (#1060) 2024-01-06 12:07:32 -08:00
Jōshin 390335eb45 apeinstall/uninstall.sh can use doas (#1062) 2024-01-06 12:06:21 -08:00
Jōshin 492a8f4f53 Fix make o/aarch64/ape on aarch64 (#1064) 2024-01-06 11:42:39 -08:00
Jōshin 636bc4007b Enable argv[0] tests in more places (#1061)
Now we do them for assimilated binaries (except on OpenBSD or XNU
non-Silicon), for XnuSilicon, and for binaries with the preserve-
argv[0] auxv flag set. We check whether to pass the argv[0] value
at the test site rather than the Child site. We move a lot of the
test initialization into Child in the non-child case, in order to
get at the pre-init value of `__program_executable_name`. Finally,
we print out info about what we are skipping.
2024-01-06 11:42:03 -08:00
Trung Nguyen b09096691a third_party: Add libunwind (#1053)
Added libunwind from LLVM 17.0.6.
The library includes functions required for C++ exception handling.
2024-01-06 15:04:30 +07:00
Justine Tunney 91de6f1f5d Release Cosmopolitan 3.2.2 2024-01-05 21:00:22 -08:00
Justine Tunney fad1279c61 Make cosmo_dlopen() safer and faster
If cosmo_dlopen() is linked on AMD64 then the runtime will switch to
using %gs for thread-local storage. This eliminates the need for the
imported symbol trampoline. It's now safer to pass function pointers
back and forth with imported libraries. Your program gets recompiled
at runtime to make it happen and the overhead is a few milliseconds.
2024-01-05 20:42:19 -08:00
Jōshin c0eacf2eb1 clang-format 2024-01-05 23:21:52 -05:00
Justine Tunney c60b150fcf Release Cosmopolitan v3.2.1 2024-01-05 15:36:57 -08:00
Jōshin 80ec6c9283 Try to detect kernel version for P flag (#1059) 2024-01-05 15:19:46 -08:00
Justine Tunney 2d93788ce3 Fix --ftrace with cosmo_dlopen()
This change ensures function call logging won't crash the process when
cosmo_dlopen() is called.
2024-01-05 15:13:07 -08:00
Jōshin 15548b523c Cleanup apeinstall.sh (#1057) 2024-01-05 13:45:14 -08:00
Jōshin 412a200ae4 Support binfmt_misc P flag in APE loader (#1058)
This allows ape to automatically preserve `argv[0]` [as of Linux kernel
5.12][0] if the [binfmt_misc][1] registration contains the P flag.

This also removes may_path_search, which was identical to the literally
flag in usage. As a result, FindCommand is subsumed into Commandv.

[0]: https://patchew.org/QEMU/[email protected]/
[1]: https://www.kernel.org/doc/html/latest/admin-guide/binfmt-misc.html
2024-01-05 12:35:01 -08:00
Justine Tunney 44a463e4d2 Invent *cosmo_dltramp() function 2024-01-05 09:11:51 -08:00
Justine Tunney 0de6a08988 Upgrade mono repo to cosmocc 3.2
The toolchain will now be downloaded going forward from multiple pinned
URLs which have shasums. Either wget or curl must be installed.

This change unblocks #1053
2024-01-05 08:02:04 -08:00
Paul Kulchenko a334f9cc33 Update redbean ProgramDirectory to return a list of previously set directories (#1021) 2024-01-05 02:17:55 -08:00
Jōshin 3315b6ef11 Store StrlenDevFd in n (#1056) 2024-01-04 23:17:37 -05:00
Justine Tunney e93c8c4f8f Fix typo 2024-01-04 13:59:36 -08:00
Justine Tunney 0e49bed660 Support 40 cosmo_dlopen() function parameters
Our dynamic linking implementation is now able to support functions with
dozens of parameters. In addition to having extra integral arguments you
can now pass vector registers using intrinsic types. Lastly, you can now
return multiple values, which is useful for functions returning structs.
2024-01-04 13:41:26 -08:00
Justine Tunney a3deef70c2 Release Cosmopolitan v3.2 2024-01-04 09:39:48 -08:00
Justine Tunney 873069fcd7 Fix -Xaarch64 in cosmocc
It's now possible to pass flags like -Xaarch64-march=armv8.2-a+dotprod
so that cosmocc will use newer ARM ISAs. For AMD64 there's another one
worth mentioning, which looks like this: -Xx86_64-mssse3
2024-01-04 07:22:05 -08:00
Justine Tunney 3864f78b88 Give GNU Make limitless stacks 2024-01-04 03:26:26 -08:00
Justine Tunney 796148790f Remove hard coded paths from APE bootloader
This increases risk of fork bomb but is needed to support the NixOS.
Upstream dependencies of APE (uname, mkdir, dd, chmod, gzip, and mv)
will be removed from releases, and deleted from the cosmo.zip server

See #12
2024-01-03 17:55:19 -08:00
Justine Tunney 34ed1fcbea Fix bugs with DNS library on Windows 2024-01-03 17:39:57 -08:00
Justine Tunney 1bb52c223b Add missing build dependency 2024-01-03 17:04:21 -08:00
Justine Tunney 31dcdefaa8 Update bootstrap binaries
It's now possible to build the monorepo on FreeBSD ARM64.
2024-01-03 17:03:51 -08:00
Jōshin 2902b76168 mmake gets smarter about locating make
- Check `$COSMOCC`, defaulting to `/opt/cosmocc`.

- Try to get the full path of the repo `make.com`. I'm not aware of
  a way of getting the path that defines a zsh function, so the best
  fallback available is `$PWD`.
2024-01-01 23:34:35 -05:00
Justine Tunney 307823ae00 Upgrade to superconfigure z0.0.29 2024-01-01 19:31:58 -08:00
Jōshin 9801f2d1db Remove COSMOPOLITAN_PROGRAM_EXECUTABLE (#1047) 2024-01-01 07:25:16 -08:00
Jōshin 68dbe5312f Prepend getcwd to exename early in init (#1048) 2024-01-01 07:23:23 -08:00
Justine Tunney 2f89c2482a Delete some dead code 2024-01-01 00:13:16 -08:00
Justine Tunney 9e6faa5256 Fix --ftrace on Windows 2024-01-01 00:00:42 -08:00
Jōshin b02d13ccde Re-enable/fix non-argv[0] exe name tests (#1046) 2023-12-31 11:53:15 -08:00
Justine Tunney 81949f038e Mint APE Loader v1.10 2023-12-31 11:43:13 -08:00
Jōshin c9550afe5e Fix loader usage, shave off a few bytes (#1016)
* Remove -f from loader usage

-f was removed in 1.5. As there is now only one flag, a couple more
bytes can be shaved off as well.

* Further loader golf

Shaves off a few bytes, paying for the cost of `RealPath` and then some
on x86_64 and offsetting some of the cost to aarch64.

* Shave off a few more bytes

Removes `-h` and flags from usage. Keeps flag-parsing logic the same,
i.e. still accepts `-h` / `--help`. Only difference is what fd and rc
the usage uses.

Still over 1k north of 8192.
2023-12-31 11:33:42 -08:00
Jōshin 10b4693e37 Sanity-check set-id interpreter script paths (#1029) 2023-12-31 07:46:27 -08:00
Jōshin 14fe83facd aarch64 loader passes os (#1042)
* Reorder Launch arguments, pass aarch64 os

Third and fourth arguments are now identical between cosmo and Launch.
By passing sp as argument 4, we save a bit of register juggling.

Fourth argument (os) is now always passed by the loader on aarch64. It
is not yet processed by cosmo. Pushing this change separately, as the
cosmo side turns out to be somewhat more involved.

* cosmo2 receives os from loader

FreeBSD aarch64 now traps early rather than pretending to be Linux.
o/aarch64/examples/env.com still works on Linux and Xnu.
2023-12-31 06:42:36 -08:00
Justine Tunney d8ad34686a Implement issetugid() on NetBSD 2023-12-30 14:58:16 -08:00
Justine Tunney 83107f78ed Introduce FreeBSD ARM64 support
It's 100% passing test fleet. Solid as a rock.
2023-12-29 20:14:02 -08:00
Justine Tunney 43fe5956ad Use DNS implementation from Musl Libc
Now that our socket system call polyfills are good enough to support
Musl's DNS library we should be using that rather than the barebones
domain name system implementation we rolled on our own. There's many
benefits to making this change. So many, that I myself wouldn't feel
qualified to enumerate them all. The Musl DNS code had to be changed
in order to support Windows of course, which looks very solid so far
2023-12-28 23:04:35 -08:00
Justine Tunney 1a28e35c62 Use good locks in dlmalloc
Using mere spin locks causes runitd.com to go painstakingly slow on
NetBSD for reasons that aren't clear yet.
2023-12-28 04:57:36 -08:00
Justine Tunney 80fca1f7c3 Make getprogramexecutablename_test pass fleet 2023-12-28 04:57:36 -08:00
Justine Tunney 8b0e42a31b Make zipos_test hermetic 2023-12-28 04:57:36 -08:00
Paul Kulchenko 6be9477b9e Fix redbean to only check additional content types when Lua is present (#1033)
Closes #971
2023-12-28 01:08:19 -08:00
Justine Tunney ce0143e2a1 Fix madvise() on Windows 2023-12-27 22:41:46 -08:00
Justine Tunney f51fd97644 Fix MODE=dbg build break 2023-12-27 22:24:27 -08:00
Jōshin 5c35863524 Rename __zipos_free -> __zipos_drop (#1043)
Removes the separate decref function, uses keep/drop in the internal
API.
2023-12-26 12:08:57 -05:00
Jōshin fd772b9b2a mmake uses $MAKE if available 2023-12-26 10:27:04 -05:00
Jōshin 4c6f928fbf Disable argv[0] tests on non-XnuSilicon (#1015)
Adds a TODO explaining the correct condition.
2023-12-24 19:43:32 -05:00
Jōshin 25266b037b Save a redundant load in zipos read/seek (#1037)
When h->pos has changed to something other than SIZE_MAX, we don't need
the extra atomic load.
2023-12-24 19:40:18 -05:00
Jōshin bb2602a524 Minor mmake cleanup 2023-12-23 11:09:50 -05:00
mataha 1bc48bc8e4 Update stb (#885)
This commit and, by extension, PR attempts to update `stb` in the most
straightforward way possible as well as include fixes from main repo's
unmerged PRs for cases rearing their ugly heads during everyday usage:

 - stb#1299: stb_rect_pack: Make rect_height_compare a stable sort
 - stb#1402: stb_image: Fix "unused invalid_chunk" with STBI_FAILURE_USERMSG
 - stb#1404: stb_image: Fix gif two_back memory address
 - stb#1420: stb_image: Improve error reporting if file operations fail
   within *_from_file functions
 - stb#1445: stb_vorbis: Few static analyzers fixes
 - stb#1487: stb_vorbis: Fix residue classdata bounding for
   f->temp_memory_required
 - stb#1490: stb_vorbis: Fix broken clamp in codebook_decode_deinterleave_repeat
 - stb#1496: stb_image: Fix pnm only build
 - stb#1497: stb_image: Fix memory leaks if stbi__convert failed
 - stb#1498: stb_vorbis: Fix memory leaks in stb_vorbis
 - stb#1499: stb_vorbis: Minor change to prevent the undefined behavior -
   left shift of a negative value
 - stb#1500: stb_vorbis: Fix signed integer overflow

Includes additional small fixes that I felt didn't warrant a separate PR.
2023-12-22 21:39:27 -08:00
Jōshin 7faffde303 Fix buffer overflow in TryPath (#1035)
Missed this when changing the code back to be like the old version.
com is now a parameter.

The only plausible way to trigger this would be to pass a loader
pathname close to MAX_PATH characters long, and then remove that
path prior to the first sys_faccessat.
2023-12-18 23:02:16 -05:00
Jōshin 85e4c020a3 mmake: formatting + aarch64 tab-completion helper
o/aarch64/* is turned into o/$mode/* in a way analogous to o//, so
aarch64 systems can use tab complete for alternate modes just like
x86_64 can.
2023-12-18 21:29:32 -05:00
Jōshin 1280797db2 Fix indentation, make llvm happy (#1032)
clang says "label at end of switch statement is a C2x extension."
2023-12-18 20:48:33 -05:00
Jōshin 58b620d8f7 Add missing bash build dependency (#1031) 2023-12-18 17:07:34 -05:00
Jōshin 2a11a09d98 Remove realpath/getcwd from loaders (#1024)
This implements proposals 1 and 2a from this gist:

https://gist.github.com/mrdomino/2222cab61715fd527e82e036ba4156b1

The only reason to use realpath from the loader was to try to prevent a
TOCTOU between the loader and the binary. But this is only a real issue
in set-id contexts, and in those cases there is already a canonical way
to do it: `/dev/fd`, passed by the kernel to the loader, so all we have
to do is pass that along to the binary.

Aside from realpath, there is no reason to absolutize the path we supply
to the binary, since it can call `getcwd` as well as we can, and on non-
M1 the binary is in a much better position to make that call.

Since we no longer absolutize the path, the binary does need to do this,
so we make its argv-parsing code generic and apply that to the different
possible places the path could come from. This means that `_` is finally
usable as a relative path, as a nice side benefit.

The M1 realpath code had a significant bug - it uses the wrong offset to
truncate the `.ape` in the `$prog.ape` case.

This PR also fixes a regression in `ape $progname` out of `$PATH` on the
two BSDs (Free and Net) that did not implement `RealPath`.
2023-12-18 15:01:16 -05:00
Jōshin f73576ab8a mmake knows about some .PHONY targets 2023-12-17 17:04:57 -05:00
Jōshin dd57c685aa mmake gets smarter
`o/$mode/*` is passed through as-is. `o/*` other than `$mode` has
`$mode` inserted. `*` has `o/$mode/` prepended.

Really leveraging zsh default tab completion here; if you have built
things with `MODE=` you can leverage that for perfect tab completion
in other modes.
2023-12-17 15:42:27 -05:00
Jōshin c11c105270 mmake doesn't even run nproc unless it needs to 2023-12-17 15:09:41 -05:00
Jōshin 3654650c52 mmake only autoloads nproc if needed 2023-12-17 15:07:10 -05:00
Jōshin 77facc4f7b Cover more cases for detecting ape loader (#1022)
Fixes a regression in GetProgramExecutableName on Linux against old
loaders. In the loader case, /proc/self/exe gives the loader's path.
We tried to detect this by checking for `/usr/bin/ape`. But that is
only one of the possible places the loader could be.
2023-12-17 10:30:22 -05:00
Jōshin b08b92248b mmake: -j$(nproc), passthrough o//
Now `mmake o/ape/ape.elf` works.
2023-12-17 01:55:52 -05:00
Jōshin 2b315626f3 Revert retabbing of net/http and tinymath (#1020) 2023-12-16 23:59:11 -05:00
Jōshin 3a8e01a77a more modeline errata (#1019)
Somehow or another, I previously had missed `BUILD.mk` files.

In the process I found a few straggler cases where the modeline was
different from the file, including one very involved manual fix where a
file had been treated like it was ts=2 and ts=8 on separate occasions.

The commit history in the PR shows the gory details; the BUILD.mk was
automated, everything else was mostly manual.
2023-12-16 23:07:10 -05:00
Jōshin 60813003a3 tool/zsh/mmake cleanup
- collects all flags into their own array to support `mmake $target -j8`

- prints out the raw make invocation to be run
2023-12-16 19:22:17 -05:00
Jōshin c0bbc45268 Add mmake zsh helper function (#1017) 2023-12-16 08:47:38 -08:00
Jōshin f94c11d978 Loader path security (#1012)
The ape loader now passes the program executable name directly as a
register. `x2` is used on aarch64, `%rdx` on x86_64. This is passed
as the third argument to `cosmo()` (M1) or `Launch` (non-M1) and is
assigned to the global `__program_executable_name`.

`GetProgramExecutableName` now returns this global's value, setting
it if it is initially null. `InitProgramExecutableName` first tries
exotic, secure methods: `KERN_PROC_PATHNAME` on FreeBSD/NetBSD, and
`/proc` on Linux. If those produce a reasonable response (i.e., not
`"/usr/bin/ape"`, which happens with the loader before this change),
that is used. Otherwise, if `issetugid()`, the empty string is used.
Otherwise, the old argv/envp parsing code is run.

The value returned from the loader is always the full absolute path
of the binary to be executed, having passed through `realpath`. For
the non-M1 loader, this necessitated writing `RealPath`, which uses
`readlinkat` of `"/proc/self/fd/[progfd]"` on Linux, `F_GETPATH` on
Xnu, and the `__realpath` syscall on OpenBSD. On FreeBSD/NetBSD, it
punts to `GetProgramExecutableName`, which is secure on those OSes.

With the loader, all platforms now have a secure program executable
name. With no loader or an old loader, everything still works as it
did, but setuid/setgid is not supported if the insecure pathfinding
code would have been needed.

Fixes #991.
2023-12-15 12:23:58 -05:00
Jōshin 8a10ccf9c4 Fix ZipOS deadlock/segfault (#1011)
This change adds a new stress test for ZipOS which helped
us improve the locking semantics in open() and close().
2023-12-14 19:59:20 -08:00
Justine Tunney 897fa6ac00 Add some Emacs keywords for CUDA 2023-12-14 19:55:56 -08:00
Jōshin 8dbfb77890 tolower() on terminfo directories (#1009)
Fixes the ncurses build on my M1 Mac. Required a make clean.
2023-12-13 12:26:12 -05:00
Jōshin 7a05fb43ac Assume __zipos_close when closing a zipos fd 2023-12-13 10:07:32 -05:00
Jōshin 3b302e6379 Cleanup zipos vfork (#1004) 2023-12-13 01:36:44 -08:00
Jōshin 6cedbc746c Fix type of refs 2023-12-13 03:03:25 -05:00
Jōshin 2fc507c98f Fix more vi modelines (#1006)
* modelines: tw -> sw

shiftwidth, not textwidth.

* space-surround modelines

* fix irregular modelines

* Fix modeline in titlegen.c
2023-12-13 02:28:11 -05:00
Justine Tunney bd10cf9d38 Whip up find command for PE binaries
Using this shell script:

    #!/bin/sh
    mkdir -p exe
    for f in $(findpe); do
      if [ -e exe/${f##*/}.exe ]; then
        cp $f exe/${f##*/}-$(rand64).exe
      else
        cp $f exe/${f##*/}.exe
      fi
    done
    rm -f /mnt/videos/microsoft.zip
    zip -rj6 /mnt/videos/microsoft.zip exe
    echo /mnt/videos/microsoft.zip

Helps file reports with Microsoft about incorrect AV detections.

See #1003
2023-12-12 10:29:03 -08:00
Justine Tunney ae85fb74b2 Update README.md
Fixes #1002
2023-12-12 07:53:23 -08:00
Justine Tunney f4a2a65cc8 Add bash 5.2 to third_party 2023-12-11 07:38:17 -08:00
Justine Tunney e1f65d5f2b Add readline 8.2 to third_party 2023-12-11 05:59:20 -08:00
Justine Tunney cbe72c4339 Fix ncurses yoinks 2023-12-10 13:11:45 -08:00
Justine Tunney 6744de659e Remove awful terminfo definitions
These files collide on MacOS and Windows due to case insensitivity.
2023-12-10 11:28:22 -08:00
Justine Tunney 4db6b74de7 Fix build warning error 2023-12-10 08:26:36 -08:00
Justine Tunney 128be2bd2e Add less 643 2023-12-10 08:01:37 -08:00
Justine Tunney 0ed7309fdb Add ncurses 6.4 2023-12-10 07:33:37 -08:00
Justine Tunney 6ee5580adc Delete GGML and RadPajama
Please use https://github.com/mozilla-Ocho/llamafile which is better,
newer, and built on cosmocc. If you need the RadPajama model, file an
issue with llamafile asking for support.
2023-12-10 06:08:54 -08:00
Justine Tunney 4a63e4a27e Reduce header complexity of PCRE2 2023-12-10 06:08:25 -08:00
Justine Tunney b27e9fe845 Add PCRE2 library to third_party 2023-12-10 05:59:18 -08:00
Justine Tunney 4f66d7f2dd Add WIN32 pseudo console APIs
See #999
2023-12-10 01:29:25 -08:00
Justine Tunney 1bd69867d1 Explain how to build FOSS in cosmocc README 2023-12-09 06:58:24 -08:00
Joseph Battelle b0cced855d Move tcgetpgrp and tcsetpgrp decls to unistd.h (#996) (#997) 2023-12-08 22:54:46 -08:00
Justine Tunney 076e0bf9a9 Re-disable WIN32 call tracing 2023-12-08 20:06:16 -08:00
Justine Tunney 1a96de6eda Add libresolv from Musl Libc
Locally modified to get nameservers from Windows Registry when
`\etc\resolv.conf` isn't defined.
2023-12-08 20:04:10 -08:00
Justine Tunney 8874a37abc Add <link.h> for absl 2023-12-08 20:04:10 -08:00
Zoey Greer a11016e590 Use gid_t and uid_t in passwd.h (#995) 2023-12-08 18:27:41 -08:00
Jōshin e16a7d8f3b flip et / noet in modelines
`et` means `expandtab`.

```sh
rg 'vi: .* :vi' -l -0 | \
  xargs -0 sed -i '' 's/vi: \(.*\) et\(.*\)  :vi/vi: \1 xoet\2:vi/'
rg 'vi: .*  :vi' -l -0 | \
  xargs -0 sed -i '' 's/vi: \(.*\)noet\(.*\):vi/vi: \1et\2  :vi/'
rg 'vi: .*  :vi' -l -0 | \
  xargs -0 sed -i '' 's/vi: \(.*\)xoet\(.*\):vi/vi: \1noet\2:vi/'
```
2023-12-07 22:17:11 -05:00
Justine Tunney 90c53f3064 Fix some references to stdio/internal.h 2023-12-07 16:30:49 -08:00
Justine Tunney f8ea02d4d1 Import /etc/shadow support from Musl for Linux
Fixes #992
2023-12-07 16:26:04 -08:00
Justine Tunney 06ace4e7b4 Always use compiler builtin for offsetof()
We have received multiple reports of GCC breaking builds when compiler
flags like `-std=c11` were being passed. The workaround until the next
release is to simply not define `__STRICT_ANSI__` which is a bad idea.
2023-12-07 14:20:12 -08:00
Justine Tunney e36283f1d9 Support double parameters in dlsym() imports 2023-12-06 06:49:35 -08:00
Justine Tunney 4772796cd8 Improve cosmo_dlopen() documentation 2023-12-06 06:33:50 -08:00
Justine Tunney 7c39818c13 Simplify GNU Make build config 2023-12-06 03:25:16 -08:00
Jōshin 394d998315 Fix vi modelines (#989)
At least in neovim, `│vi:` is not recognized as a modeline because it
has no preceding whitespace. After fixing this, opening a file yields
an error because `net` is not an option. (`noet`, however, is.)
2023-12-05 14:37:54 -08:00
Justine Tunney 221a27836b Define max_align_t in <stddef.h>
It was previously only being defined in <cstddef>

See oilshell/oil#1760
2023-12-05 00:52:33 -08:00
Justine Tunney 536dfb947a Make getprogramexecutablename_test report status
Fixes #988
2023-12-04 23:19:28 -08:00
Justine Tunney cd52c59552 Allow MAP_POPULATE under pledge() 2023-12-04 23:15:41 -08:00
Jōshin 53357aa26a Fix __zipos_close (#984) 2023-12-04 20:24:26 -08:00
Jōshin 577bb180b7 benchmark, getenv, test cleanup (#987) 2023-12-04 20:01:52 -08:00
Jōshin da8baf2aa5 ape-m1 minor formatting cleanup (#986) 2023-12-04 19:58:32 -08:00
Jōshin ed8fadea37 Keep argv[0], add COSMOPOLITAN_PROGRAM_EXECUTABLE (#980)
* Introduce env.com

Handy tool for debugging environment issues.

* Inject path as COSMOPOLITAN_PROGRAM_EXECUTABLE

`argv[0]` was previously being used as a communication channel between
the loader and the binary, giving the binary its full path for use e.g.
in `GetProgramExecutableName`. But `argv[0]` is not a good channel for
this; much of what made 2a3813c6 so gross is due to that.

This change fixes the issue by preserving `argv[0]` and establishing a
new communication channel: `COSMOPOLITAN_PROGRAM_EXECUTABLE`.

The M1 loader will always set this as the first variable. Linux should
soon follow. On the other side, `GetProgramExecutableName` checks that
variable first. If it sees it, it trusts it as-is.

A lot of the churn in `ape/ape-m1.c` in this change is actually backing
out hacks introduced in 2a3813c6; the best comparison is:

    git diff 2a3813c6^..
2023-12-04 12:45:46 -08:00
Jōshin 2a3813c6cf $prog.ape support (#977)
* ape loader: $prog.ape + login shell support

If the ape loader is invoked with `$0 = $prog.ape`, then it searches for
a `$prog` in the same directory as it and loads that. In particular, the
loader searches the `PATH` for an executable named `$prog.ape`, then for
an executable named `$prog` in the same directory. If the former but not
the latter is found, the search terminates with an error.

It also handles the special case of getting started as `-$SHELL`, which
getty uses to indicate that the shell is a login shell. The path is not
searched in this case, and the program location is read straight out of
the `SHELL` variable.

It is now possible to have `/usr/local/bin/zsh.ape` act as a login shell
for a `/usr/local/bin/zsh` αpε, insofar as the program will get started
with the 'correct' args. Unfortunately, many things break if `$0` is not
the actual full path of the executable being run; for example, backspace
does not update the display properly.

To work around the brokenness introduced by not having `$0` be the full
path of the binary, we cut the leading `-` out of `argv[0]` if present.
This gets the loader's behavior with `$prog.ape` up to par, but doesn't
tell login shells that they are login shells.

So we introduce a hack to accomplish that: if ape is run as `-$prog.ape`
and the shell is `$prog`, the binary that is loaded has a `-l` flag put
into its first argument.

As of this commit, αpε binaries can be used as login shells on OSX.

* if islogin, execfn = shell

Prior to this, execfn was not being properly set for login shells that
did not receive `$_`, which was the case for iTerm2 on Mac. There were
no observable consequences of this, but fixing it seems good anyway.

* Fix auxv location calculation

In the non-login-shell case, it was leaving a word of uninitialized
memory at `envp[i] + 1`. This reuses the previous calculation based
on `envp`.
2023-12-03 19:39:32 -08:00
Jōshin 8dd348067b refs is atomic_size_t (#976)
The unassert covers all invalid values rather than a third of them.
2023-12-02 16:48:58 -08:00
Justine Tunney 450d9b1d43 Release Cosmopolitan v3.1.3 2023-12-01 10:29:59 -08:00
Justine Tunney 69ba516466 Revert "Re-enable apelink size optimization"
This reverts commit 7b3d7ee07b.

It broke llava-v1.5-7b-q4-server.llamafile on Windows.
2023-12-01 10:26:17 -08:00
Justine Tunney 9e9d2c0006 Release Cosmopolitan v3.1.2 2023-12-01 06:47:57 -08:00
Jōshin a0f599ce12 Fix memory_order_release in __zipos_seek (#975) 2023-12-01 02:28:15 -08:00
Jōshin d95d61b1af Better zipos refcounts and atomic reads/seeks (#973)
* Better refcounting

Cribbed from [Rust Arc][1] and the [Boost docs][2]:

"""
Increasing the reference counter can always be done with
memory_order_relaxed: New references to an object can only be formed
from an existing reference, and passing an existing reference from one
thread to another must already provide any required synchronization.

It is important to enforce any possible access to the object in one
thread (through an existing reference) to happen before deleting the
object in a different thread. This is achieved by a "release" operation
after dropping a reference (any access to the object through this
reference must obviously happened before), and an "acquire" operation
before deleting the object.

It would be possible to use memory_order_acq_rel for the fetch_sub
operation, but this results in unneeded "acquire" operations when the
reference counter does not yet reach zero and may impose a performance
penalty.
"""

[1] https://moshg.github.io/rust-std-ja/src/alloc/arc.rs.html
[2] https://www.boost.org/doc/libs/1_55_0/doc/html/atomic/usage_examples.html

* Make ZiposHandle's pos atomic

Implements a somewhat stronger guarantee than POSIX specifies: reads and
seeks are atomic. They may be arbitrarily reordered between threads, but
each one happens all the way and leaves the fd in a consistent state.

This is achieved by "locking" pos in __zipos_read by storing SIZE_MAX to
pos during the operation, so only one can be in-flight at a time. Seeks,
on the other hand, just update pos in one go, and rerun if it changed in
the meantime.

I used `LIKELY` / `UNLIKELY` to pessimize the concurrent case; hopefully
that buys back some performance.
2023-12-01 01:01:03 -08:00
Jōshin f0bfabba07 Revert "Remove noop __zipos_postdup" (#974)
`close` does not modify the fd table, so `rc` can potentially have a
stale zipos object sitting on it. As such, we need `__zipos_postdup`
there.
2023-12-01 00:51:01 -08:00
Jōshin 7845495a93 Get tool/cosmocc/package.sh working on M1 (#970)
On aarch64 hosts, MODE= is changed to MODE=aarch64, so o// targets don't
work. So On aarch64, get apelink.com out of o/aarch64/. Also prepend ape
when calling it. And finally, fetch with curl when wget isn't installed.
2023-12-01 00:08:55 -08:00
Jōshin d1a745c17c Implement __zipos_dup (#972)
* Implement __zipos_dup

Makes ZiposHandle reference-counted by an `rc` field in a union with its
freelist `next` pointer. The functions `__zipos_free` and `__zipos_keep`
function as incref/decref for it. Adds `__zipos_postdup` to fix metadata
on file descriptors after dup-like operations, and adds zipos support to
`sys_dup_nt` + `sys_close_nt`.

* Remove noop __zipos_postdup

rc is never a zipos file because it is always a previously unused file
descriptor. fd is never a zipos file because that case has been handled
above by __zipos_fcntl.
2023-12-01 00:08:30 -08:00
Justine Tunney 6556dd2673 Add $(uniq token...) native function to Make
This reduces mono repo `make` latency from 336ms to 226ms.
2023-11-30 21:02:16 -08:00
Justine Tunney 14bf57180f Import GNU Make 4.4.1
Landlock Make hasn't been working well on AARCH64 systems. Let's do this
over the right way, using our new build tools.
2023-11-30 20:54:52 -08:00
Justine Tunney 9315ebbfd9 Fix MODE=tinylinux build 2023-11-30 15:38:08 -08:00
mataha 879bb84244 Update quickjs (#890)
Includes additional fixes from main repo's unmerged PRs:

 - quickjs#132: Fix undefined behavior: shift 32 bits for uint32_t in bf_set_ui
 - quickjs#171: Fix locale-aware representation of hours in Date class
 - quickjs#182: Fix stack overflow in CVE-2023-31922
2023-11-30 10:51:16 -08:00
Justine Tunney 4b7ba9a4c5 Fix some bugs with dup2() and ZipOS
On UNIX if dup2(newfd) was a ZipOS file descriptor, then its resources
weren't being released, and the newly created file descriptor would be
mistaken for ZipOS due to its memory not being cleared. On Windows, an
issue also existed relating to newfd resources not being released.
2023-11-30 10:10:02 -08:00
Justine Tunney 26c70e08bf Make it possible to package.sh cosmocc w/ modes 2023-11-30 07:12:15 -08:00
Justine Tunney 20938a45e6 Add appropriate chmod to README instructions
See Mozilla-Ocho/llamafile#13
2023-11-30 03:35:17 -08:00
Justine Tunney 7b3d7ee07b Re-enable apelink size optimization
I'm no longer able to reproduce the PE import table corruption that was
previously observed. Since this optimization shaves up to 64kb off each
fat APE binary we build, it's worth turning this back on again, to wait
and see if something breaks, and if so, fix it. At least until the next
release is shipped.

See #965
2023-11-29 05:41:23 -08:00
Justine Tunney ff955aaa01 Make memcmp() and memchr() go fast again
Readahead within the specified size is legal, even if it overlaps a page
boundary; it's the fault of the caller if that causes a segfault.
2023-11-29 05:17:21 -08:00
Justine Tunney 70155df7a9 Avoid linking win32 signals impl
This shaves ~4kb off o/tiny/examples/hello2.com by avoiding linking the
WIN32 signals polyfill unless sigaction() is being used.

See #965
2023-11-29 04:09:31 -08:00
Justine Tunney 2b960bb249 Exclude strace from MODE=tiny builds
This change gets o/tinylinux/examples/hello2.com back down to 8kb in
size which had been unintentionally bloated to 40kb in recent months

See #965
2023-11-29 03:45:54 -08:00
Justine Tunney b7e1dc81c2 Release Cosmopolitan v3.1.1 2023-11-29 02:19:56 -08:00
Justine Tunney 41c06c6325 Make cosmocc 100x faster on MacOS
This change upgrades to superconfigure z0.0.23 which fixes an issue
where the compiler had harmless /home/... paths baked-in, which are
normally only present in the build environment, and usually skipped
over. Sadly on MacOS calling fstatat() on these paths would lead to
cloud file system ops that caused system calls to take a long time.
That's problematic, since cosmocc needs to be a 100% local command.
2023-11-29 01:48:17 -08:00
Justine Tunney 4427581a05 Cleanup inline locking code 2023-11-29 00:36:22 -08:00
Justine Tunney 96185e1ac0 Fix MODE=aarch64 cross-compiling
This change fixes a regression that happened some time ago when building
for AARCH64 using the vendored toolchain rather than cosmocc. The errors
that would show up `Relocations in generic ELF (EM: 62)` have been fixed
2023-11-28 22:40:06 -08:00
Justine Tunney ca5e361919 Don't show (win32 error 0) 2023-11-28 21:30:20 -08:00
Justine Tunney 9cad66aa11 Freshen bootstrap binaries for MacOS + SIP 2023-11-28 21:18:31 -08:00
Justine Tunney 03de761890 Update cosmocc to use superconfigure z0.0.23 2023-11-28 20:04:15 -08:00
Justine Tunney 43de12db55 Introduce forkpty() 2023-11-28 19:59:57 -08:00
Justine Tunney fa20edc44d Reduce header complexity
- Remove most __ASSEMBLER__ __LINKER__ ifdefs
- Rename libc/intrin/bits.h to libc/serialize.h
- Block pthread cancelation in fchmodat() polyfill
- Remove `clang-format off` statements in third_party
2023-11-28 14:39:42 -08:00
Justine Tunney 96f979dfc5 Rename makefiles BUILD.mk
This way they appear at the top of directory listings.
2023-11-28 11:21:08 -08:00
Paul Kulchenko 0bffd09433 Allow redbean to accept --assimilate option without complaining (#916)
This allows using --assimilate on Windows or already assimilated binary.
Ref #356.
2023-11-19 19:02:30 -08:00
Stephen Gregoratto cc5c5319bf Linux: Add cachestat, fchmodat2 syscalls (#958) 2023-11-19 19:01:20 -08:00
Justine Tunney 69faf1b403 Release Cosmpolitan v3.1 2023-11-18 20:40:35 -08:00
Justine Tunney e4dea37b8e Make clock_nanosleep() cancel faster 2023-11-18 18:12:09 -08:00
Justine Tunney 0c89516ac5 Fix unzip warning with apelink generated symtabs 2023-11-18 17:15:22 -08:00
Justine Tunney 3e6d536822 Correct misunderstanding with zip64 extra records 2023-11-18 14:35:57 -08:00
Justine Tunney dbd8176ea8 Add CPU_COUNT_S() 2023-11-18 12:38:30 -08:00
Justine Tunney 545a8f4cb0 Add more sched.h content 2023-11-18 08:08:15 -08:00
Justine Tunney f7cfe03888 Fix dlopen() for FreeBSD and NetBSD 2023-11-18 04:35:48 -08:00
Justine Tunney 2c1efd3d78 Fix Python build break 2023-11-18 02:32:01 -08:00
Matheus Moreira 3ac473df3b Floating point parsing support for scanf family (#924) 2023-11-18 02:25:36 -08:00
Justine Tunney 8caf1b48a9 Improve time/sleep accuracy on Windows
It's now almost as good as Linux thanks to a Windows 8+ API.
2023-11-18 01:57:44 -08:00
Justine Tunney 72ac5f18d9 Fix dash compatible syntax issue 2023-11-17 16:47:47 -08:00
Justine Tunney 2c9d2943d6 Introduce AddDllDirectory() 2023-11-17 10:35:34 -08:00
Justine Tunney 32b97f2d25 Improve execve() path argument munging
Munging of paths passed inside the system() interpreter command is no
longer supported. You have to pass your paths to posix_spawn() or the
execve() family of functions if you want them to be munged. The first
three characters must match `^/[a-z]/` in which case, it'll be turned
into a DOS-style drive path with backslashes.
2023-11-17 09:59:03 -08:00
Justine Tunney 529cb4817c Improve dlopen() on Apple Silicon
- Introduce MAP_JIT which is zero on other platforms
- Invent __jit_begin() and __jit_end() which wrap Apple's APIs
- Runtime dispatch to sys_icache_invalidate() in __clear_cache()
2023-11-17 02:33:14 -08:00
Justine Tunney 7a9e176ecf Improve debug binary location detection 2023-11-17 00:07:39 -08:00
Justine Tunney 68c7c9c1e0 Clean up some code
- Use good ELF technique in cosmo_dlopen()
- Make strerror() conform more to other libc impls
- Introduce __clear_cache() and use it in cosmo_dlopen()
- Remove libc/fmt/fmt.h header (trying to kill off LIBC_FMT)
2023-11-16 17:31:07 -08:00
tkchia 7010a8081e Fix regression when building deathstar.com for aarch64 (#957)
It turns out my earlier commit ddc08dc974 caused a build with
MODE=aarch64 to fail.  The commit changed deathstar.c to link
in code to support a VGA console, but this is not implemented
yet for AArch64.  Thanks to @ahgamut for spotting this issue.
2023-11-16 12:13:17 -08:00
tkchia 665747a319 Make test/libc/calls/pledge_test.c work with "nonstandard" umask (#956)
The test was failing if the process's umask happened to be
0077, for example.  The file `foo` was then created with a
file mode of 0100600, rather than the expected 0100644.
2023-11-16 00:47:21 -08:00
Justine Tunney e05933c641 Mention zsh and fish in readme 2023-11-15 23:25:47 -08:00
Justine Tunney 8f5e516b39 Remove sync_file_range()
After hearing horror stories from a trusted colleague, I don't think
this is the kind of API we want to be supporting. Also SQLite wisdom
regarding fdatasync() has been added to the documentation.
2023-11-15 23:21:22 -08:00
Justine Tunney 8318d67503 Fix regression in cosmocc toolchain 2023-11-15 21:41:53 -08:00
Justine Tunney 1351d3cede Remove bool from public headers 2023-11-15 20:58:46 -08:00
Justine Tunney dffee606cf Run host compiler to create dlopen helper 2023-11-15 20:58:46 -08:00
tkchia ddc08dc974 [metal] Make deathstar.com draw onto both VGA and serial consoles (#955)
In addition, deathstar.com can now be booted up via UEFI.
2023-11-15 19:40:54 -08:00
Justine Tunney e4584ace81 Get cosmo_dlopen() working better on System Five
Imported functions are now aspected with a trampoline that blocks
signals and changes the thread-local storage register. This means
bigger more complicated libraries can now be imported even though
the whole technique remains fundamentally unsafe.
2023-11-15 10:56:30 -08:00
Justine Tunney 3a470ed356 Remove some exploratory some tests
These were reported as failing on some Apple Arm64 systems.
2023-11-14 21:21:22 -08:00
Justine Tunney 9f9aec013a Do better job documenting platform issues 2023-11-14 18:15:58 -08:00
tkchia eea601f346 [metal] Some minor fixes and tweaks (#933)
* [metal] Ensure DF is clear when calling C from exception handler
* [metal] Mark some internal routines and declarations as `@internal`
* [metal] Fix crash under UEFI when command line string is NULL
* [metal] Fix argc & argv[] setting, & VM page freeing, for UEFI

Part of the memory occupied by the argv[] contents was
erroneously used for page tables & then later erroneously
freed.  The symptom was that argv[0] would show up as an
empty string ("").
2023-11-14 15:26:59 -08:00
Justine Tunney 1c2e7c1333 Introduce SIP_DISABLED compile option for ape-m1.c
Systems that don't use SIP can now build APE Loader with this flag to
get a performance speedup.
2023-11-13 22:04:05 -08:00
Justine Tunney 0283f2772c Fix build in MODE=dbg 2023-11-13 15:13:45 -08:00
Justine Tunney 751d20d98d Fix nsync_mu_unlock_slow_() on Apple Silicon
We torture test dlmalloc() in test/libc/stdio/memory_test.c. That test
was crashing on occasion on Apple M1 microprocessors when dlmalloc was
using *NSYNC locks. It was relatively easy to spot the cause, which is
this one particular compare and swap operation, which needed to change
to use sequentially-consistent ordering rather than an acquire barrier
2023-11-13 11:07:13 -08:00
Justine Tunney 3b15d31247 Fix #include <shared_mutex> 2023-11-13 08:30:17 -08:00
Justine Tunney f047172bd3 Fix uname assignment in Makefile 2023-11-13 05:53:10 -08:00
mattx 5d5b282d12 Inform user when wsl2 interop causes build issues (#952) 2023-11-13 05:29:25 -08:00
Justine Tunney 25a892d6ae Add some platform notes to the README file 2023-11-13 02:26:34 -08:00
Justine Tunney d1e764b3cf Modernize README.md 2023-11-12 22:04:07 -08:00
Paul Kulchenko 5c1fdc9819 Fix session changeset procesing (reverting some of 0d748ad5 changes) (#949)
This removes strdup from Lua SQLite wrapper, as it's applied to binary changeset
strings, thus cutting them short (because of strlen).
2023-11-12 14:25:50 -08:00
Justine Tunney f25beb341c Add architecture flag to zipobj 2023-11-12 09:35:59 -08:00
Justine Tunney bed77186c3 Use simple locks in dlmalloc 2023-11-12 09:00:49 -08:00
Justine Tunney bcf268adf8 Don't modify argument block on MacOS Arm64
Some dynamic library had access to this information somehow and was
crashing when it didn't have the expected structure.
2023-11-12 05:59:03 -08:00
Justine Tunney 0863427b3a Make development more pleasant on MacOS Arm64 2023-11-12 05:42:17 -08:00
Justine Tunney f7cad70da1 Introduce posix tests package 2023-11-12 05:41:48 -08:00
Justine Tunney 48bd3d85df Fix sigaction() return handler address on XNU/BSD 2023-11-12 05:40:06 -08:00
Justine Tunney bd56a9cf51 Rename dlopen() to cosmo_dlopen() 2023-11-12 01:19:04 -08:00
Justine Tunney c6d3802d3a Add more fixes for new cosmocc toolchain
We now have an `#include <cxxabi.h>` header which defines all the APIs
Cosmopolitan's implemented so far. The `cosmocc` README.md file is now
greatly expanded with documentation.
2023-11-11 23:28:19 -08:00
Gautham 95124cacbe Fixes for building superconfigure (#948) 2023-11-11 22:11:11 -08:00
Justine Tunney b320321319 Add build/bootstrap/rollup.com 2023-11-11 19:58:45 -08:00
Justine Tunney 291103ad8d Redesign cosmocc toolchain
The `cosmocc` compiler is now being distributed as a self-contained
toolchain that's path-agnostic and it no longer requires you clone the
Cosmop repo to use it. The bin/ folder has been deleted from the mono
repo. The `fatcosmocc` command has been renamed to `cosmocc`. MacOS
support now works very well.
2023-11-11 14:18:08 -08:00
Justine Tunney 3802428026 Fix missing .ok file make warning 2023-11-11 02:32:22 -08:00
Justine Tunney d0ad2694ed Iterate more on recent changes 2023-11-11 00:28:22 -08:00
Paul Kulchenko 7138399f96 Close Lua state before exiting redbean worker to allow GC to collect (#941) 2023-11-10 18:37:00 -08:00
Justine Tunney cdbd10c9d0 Introduce X86_CHECK() macro 2023-11-10 15:41:55 -08:00
Justine Tunney afbef6c361 Freshen build/bootstrap/mkdeps.com 2023-11-10 12:39:52 -08:00
Justine Tunney 3d3cdcf276 Fix APE uname direction
Fixes #942
2023-11-10 12:34:25 -08:00
Justine Tunney d2f49ca175 Improve mkdeps
Our makefile generator now accepts badly formatted include lines. It's
now more hermetic with better error checking in the cosmo repo, and it
can be configured to not be hermetic at all.
2023-11-10 04:14:27 -08:00
Justine Tunney 241f949540 Use dynamic memory for *NSYNC waiters 2023-11-10 01:42:06 -08:00
Justine Tunney 15af5c2d7e Fix ucontext_t alignment in strict ansi mode
Using `cosmocc -std=c11` was causing `ucontext_t` to become misaligned.
This change also adds the GNU constants on x86_64 for accessing general
registers, so you will not need `#ifdef`s to support both Cosmo and GNU
2023-11-09 12:48:29 -08:00
Justine Tunney b8d1377ae1 Give new wait4() another review pass 2023-11-09 11:23:30 -08:00
Justine Tunney ea28f93a26 Support <isystem> includes in monorepo 2023-11-08 19:14:56 -08:00
Justine Tunney 5330442d85 Avoid APE fork bomb
This change addresses a $PATH resolution issue where APE depends on
uname and uname is an APE program. So sorry to anyone this impacted
we'll get a release out soon.
2023-11-08 11:37:42 -08:00
mataha cc3e3de044 Make args replace ... only at the end of file (#909)
So that the string '...' can be used verbatim elsewhere in the file.
2023-11-08 10:00:52 -08:00
Justine Tunney ac125d3e1f Fix console copy/paste issue
Fixes #936
2023-11-08 09:29:45 -08:00
Justine Tunney 956e68be59 Revert "Use %gs as TLS register when dlopen() is linked"
This reverts commit d71da7fc72.
2023-11-08 01:33:01 -08:00
Justine Tunney d71da7fc72 Use %gs as TLS register when dlopen() is linked
Fixes #938
2023-11-08 01:11:17 -08:00
Justine Tunney e961385e55 Put more thought into i/o polyfills
wait4() is now solid enough to run `make -j100` on Windows. You can now
use MSG_DONTWAIT on Windows. There was a handle leak in accept() that's
been fixed. Our WIN32 overlapped i/o code has been simplified. Priority
class now inherits into subprocesses, so the verynice command will work
and the signal mask will now be inherited by execve() and posix_spawn()
2023-11-07 18:32:35 -08:00
Justine Tunney 736fdb757a Implement raise() with getcontext() / setcontext() 2023-11-05 18:04:36 -08:00
Justine Tunney dd83db9567 Improve makefile for uname 2023-11-05 16:21:09 -08:00
Justine Tunney 23d812f116 Release Cosmopolitan v3.0.2
It's recommended that all users, especially Apple Arm, re-run:

    ape/apeinstall.sh

Within the Cosmopolitan monorepo to refresh your APE loader.
2023-11-05 15:54:47 -08:00
Justine Tunney c39eb244d4 Disable an mprotect() test for now due to SIP 2023-11-05 15:38:16 -08:00
Justine Tunney f63c4d4f52 Use /usr/local/bin/ape on Apple Silicon 2023-11-05 14:52:27 -08:00
Paul Kulchenko b893479417 Add database and statement readonly checks to sqlite in redbean (#914) 2023-11-05 13:29:09 -08:00
Justine Tunney d02cba2451 Mint APE Loader v1.9 2023-11-05 13:11:34 -08:00
Justine Tunney 4cf987ddb1 Fix usleep() definition 2023-11-05 13:06:15 -08:00
Justine Tunney 20c794a353 Add strace to aarch64 vfork() 2023-11-05 13:06:15 -08:00
Gregory Haerr 4f100d2aad Fix stack alignment and auxv for macho APE loader (#931) 2023-11-05 13:06:10 -08:00
Justine Tunney a12ad17291 Get APE Loader working on MacOS with SIP enabled 2023-11-05 01:36:47 -08:00
Justine Tunney 653c7b2f87 Fatten build/bootstrap/echo.com 2023-11-04 21:01:43 -07:00
Justine Tunney d7917ea076 Make win32 i/o signals atomic and longjmp() safe 2023-11-04 20:33:29 -07:00
Justine Tunney 585c86e2a4 Support \n in /zip/.args files 2023-11-04 07:25:20 -07:00
Justine Tunney 48e260e653 Introduce NO_ADDRESS constant 2023-11-03 13:56:17 -07:00
Justine Tunney aa106c9356 Fix Lua test suite regression 2023-11-03 07:01:46 -07:00
Justine Tunney 5e8c928f1a Introduce dlopen() support
Every program built using Cosmopolitan is statically-linked. However
there are some cases, e.g. GUIs and video drivers, where linking the
host platform libraries is desirable. So what we do in such cases is
launch a stub executable using the host platform's libc, and longjmp
back into this executable. The stub executable passes back to us the
platform-specific dlopen() implementation, which we shall then wrap.

Here's the list of platforms that are supported so far:

- x86-64 Linux w/ Glibc
- x86-64 Linux w/ Musl Libc
- x86-64 FreeBSD
- x86-64 Windows
- aarch64 Linux w/ Glibc
- aarch64 MacOS

What this means is your Cosmo programs can call foreign functions on
your host operating system. However, it's important to note that any
foreign library you link won't have the ability to call functions in
your Cosmopolitan program. For example it's now technically possible
that Lua can load a module, however that almost certainly won't work
since the Lua module won't have access to Cosmo's Lua API.

Kudos to @jacereda for figuring out how to do this.
2023-11-03 06:37:18 -07:00
Justine Tunney 1eb6484c9c Rewrite getcwd()
This change addresses a bug that was reported in #923 where bash on
Windows behaved strangely. It turned out that our weak linking of
malloc() caused bash's configure script to favor its own getcwd()
function, which is implemented in the most astonishing way, using
opendir() and readdir() to recursively construct the current path.

This change moves getcwd() into LIBC_STDIO so it can strongly link
malloc(). A new __getcwd() function is now introduced, so all the
low-level runtime services can still use the actual system call. It
provides the Linux Kernel API convention across platforms, and is
overall a higher-quality implementation than what we had before.

In the future, we should probably take a closer look into why bash's
getcwd() polyfill wasn't working as intended on Windows, since there
might be a potential opportunity there to improve our readdir() too.
2023-11-02 13:16:42 -07:00
Justine Tunney a46ec61787 Make cosmocc -dumpmachine work correctly 2023-11-02 09:52:25 -07:00
Justine Tunney 024be3b009 Introduce getifaddrs()
This function was invented by the BSDs (it's not in POSIX.1). It
provides a high-level interface into ioctl(SIOCGIFCONF) which is
comparatively clumsy to use. We already made the ioctls portable
across our entire support vector back in 2021, so this interface
is portable too. See o//tool/viz/getifaddrs.com for our demo app
2023-11-02 08:33:03 -07:00
Justine Tunney 6ca5ab4680 Mention cosmocc-0.0.16 in Makefile 2023-11-01 12:08:50 -07:00
Justine Tunney 55d5d6bbc0 Release Cosmopolitan v3.0.1 2023-11-01 09:23:43 -07:00
Justine Tunney d259e573b6 Remove *NSYNC WIN32 semaphores
This implementation doesn't work as well as WIN32 futexes. This code
path was only added back when we were having issues with set context
however that's been solved so we can go back to the much better code
2023-11-01 07:18:58 -07:00
Justine Tunney 7b284f6bda Fix bugs and regressions in the pledge command
This change gets the pledge (formerly pledge.com) command back in tip
top shape for a 3.0.1 cosmos release. It now runs on all platforms, even
though it's mostly a no-op on ones that lack the kernel security stuff.
The binary footprint is now smaller, since it no longer needs to link
malloc. It's also now able to be built as a fat binary.
2023-11-01 06:08:58 -07:00
Justine Tunney b0e3d89942 Work around qemu-aarch64 bug
Qemu appears to define O_LARGEFILE as having its x86-64 value, which is
an easy mistake to make since this is one of the few magic numbers that
Linux special-cases for AARCH64.
2023-11-01 00:24:48 -07:00
Justine Tunney 0b1acce680 Introduce shm_open() and shm_unlink() 2023-10-31 23:57:52 -07:00
Justine Tunney fadb64a2bf Introduce pthread_rwlock_try{rd,wr}lock
This also changes recursive mutexes to favor cpu over scheduler yield.
2023-10-31 22:13:08 -07:00
Justine Tunney a1e1e821cb Introduce GNU/BSD timeval macros 2023-10-31 21:58:49 -07:00
Justine Tunney 2af7c802b6 Stub out getutline and pututline 2023-10-31 21:58:39 -07:00
Justine Tunney 3b791d2f44 Introduce lgammal, tgammal, erfl, and erfcl
git://git.musl-libc.org/musl
79bdacff83a6bd5b70ff5ae5eb8b6de82c2f7c30
2023-10-31 21:58:29 -07:00
Justine Tunney b4084dd6c6 Introduce libcxx <shared_mutex>
[email protected]:llvm-mirror/libcxx.git
78d6a7767ed57b50122a161b91f59f19c9bd0d19
2023-10-31 21:57:28 -07:00
Justine Tunney ee82f90bba Introduce __cxa_thread_atexit() 2023-10-31 20:04:31 -07:00
Justine Tunney a699cda5c8 Make cosmocc -Wl,--version do the right thing
Our `cosmocc` wrapper script will now report the linker version, the
same way as the `cc` command. The `fatcosmocc` command won't do this
because we wouldn't want programs to parse out the path of the amd64
linker command and use it explicitly.

Fixes #917
2023-10-31 18:20:28 -07:00
Justine Tunney f86ffa1a25 Fix aarch64 build breakage caused by ed17d3008 2023-10-31 18:17:32 -07:00
Justine Tunney a09f62141d Fix bug in login_tty() 2023-10-31 18:03:13 -07:00
tkchia ed17d3008b [metal] Add a uprintf() routine, for non-emergency boot logging (#905)
* [metal] Add a uprintf() routine, for non-emergency boot logging
* [metal] _Really_ push forward timing of VGA TTY initialization
* [metal] Do something useful with uprintf()
* [metal] Locate some ACPI tables, for later hardware detection

Specifically the code now tries to find the ACPI RSDP,
RSDT/XSDT, FADT, & MADT tables, whether in legacy BIOS
bootup mode or in a UEFI bootup.  These are useful for
figuring out how to (re)enable asynchronous interrupts
in legacy 8259 PIC mode.
2023-10-25 14:32:20 -07:00
Justine Tunney 062b2d776e Mention cosmocc v0.0.14 in Makefile
https://justine.lol/cosmocc-0.0.14.zip
https://github.com/ahgamut/superconfigure/releases/tag/z0.0.14
2023-10-16 00:07:33 -07:00
Justine Tunney 4ae5223eae Release Cosmopolitan v3.0 2023-10-15 23:53:36 -07:00
Justine Tunney 8f82a5f968 Fix the build 2023-10-15 20:36:12 -07:00
Justine Tunney d1300623d2 Add tree command to third_party 2023-10-15 19:46:40 -07:00
Justine Tunney 7b8024d088 Supplement _GNU_SOURCE with _COSMO_SOURCE 2023-10-15 16:59:27 -07:00
Gautham e6b59bced2 Add tsearch from Musl Libc (#912)
https://git.musl-libc.org/cgit/musl/snapshot/musl-1.2.4.tar.gz
2023-10-15 16:50:54 -07:00
Justine Tunney c9fecf3a55 Make improvements
- You can now run `make -j8 toolchain` on Windows
- You can now run `make -j` on MacOS ARM64 and BSD OSes
- You can now use our Emacs dev environment on MacOS/Windows
- Fix bug where the x16 register was being corrupted by --ftrace
- The programs under build/bootstrap/ are updated as fat binaries
- The Makefile now explains how to download cosmocc-0.0.12 toolchain
- The build scripts under bin/ now support "cosmo" branded toolchains
- stat() now goes faster on Windows (shaves 100ms off `make` latency)
- Code cleanup and added review on the Windows signal checking code
- posix_spawnattr_setrlimit() now works around MacOS ARM64 bugs
- Landlock Make now favors posix_spawn() on non-Linux/OpenBSD
- posix_spawn() now has better --strace logging on Windows
- fstatat() can now avoid EACCES in more cases on Windows
- fchmod() can now change the readonly bit on Windows
2023-10-15 16:45:00 -07:00
Justine Tunney 06c6baaf50 Fix copy/paste issue in Windows console 2023-10-14 16:14:50 -07:00
Justine Tunney bd48e6c666 Fix the Landlock Make build 2023-10-14 10:40:05 -07:00
Justine Tunney cdf556e7d2 Implement signal handler tail recursion
GNU Make on Windows now appears to be working reliably. This change also
fixes a bug where, after fork the Windows thread handle wasn't reset and
that caused undefined behavior using SetThreadContext() with our signals
2023-10-14 10:38:15 -07:00
Justine Tunney a657f3e878 Delete old wcwidth() implementation
This shaves away 144kb of bss memory from every binary linking printf at
the expense of slightly increased binary footprint. Kudos for Byron Lai.
2023-10-14 03:15:05 -07:00
Justine Tunney 2db2f40a98 Rewrite special file handling on Windows
This change gets GNU grep working. What caused it to not work, is it
wouldn't write to an output file descriptor when its dev/ino equaled
/dev/null's. So now we invent special dev/ino values for these files
2023-10-14 02:53:34 -07:00
Justine Tunney aca2261cda Don't preempt WIN32 libraries
This change refactors our POSIX signals emulation for Windows so that it
performs some additional safety checks before calling SetThreadContext()
which needs to be locked and must never ever interrupt Microsoft's code.
Kudos to the the Go developers for figuring out how to do this properly.
2023-10-13 13:59:39 -07:00
Justine Tunney d1a283a588 Fix redbean LaunchBrowser() 2023-10-13 11:39:00 -07:00
Justine Tunney 3851025b77 Fix SQLite regressions caused by 3b086af91 2023-10-13 11:00:39 -07:00
Justine Tunney bdc453b22d Fix redbean "-1 workers remain" blemish 2023-10-13 08:33:09 -07:00
Justine Tunney 0e0398cd5d Fix MODE=tinylinux build 2023-10-13 08:30:18 -07:00
Justine Tunney 4bcb107cb0 Fix ctrl-c in redbean on Windows 2023-10-13 08:10:03 -07:00
Justine Tunney d458642790 Write more tests and improve kill() on Windows 2023-10-13 04:38:45 -07:00
Justine Tunney b81a1bd9a8 Fix bug with __sig_tramp() + sigaltstack() 2023-10-13 01:24:10 -07:00
Justine Tunney 49b0eaa69f Improve threading and i/o routines
- On Windows connect() can now be interrupted by a signal; connect() w/
  O_NONBLOCK will now raise EINPROGRESS; and connect() with SO_SNDTIMEO
  will raise ETIMEDOUT after the interval has elapsed.

- We now get the AcceptEx(), ConnectEx(), and TransmitFile() functions
  from the WIN32 API the officially blessed way, using WSAIoctl().

- Do nothing on Windows when fsync() is called on a directory handle.
  This was raising EACCES earlier becaues GENERIC_WRITE is required on
  the handle. It's possible to FlushFileBuffers() a directory handle if
  it's opened with write access but MSDN doesn't document what it does.
  If you have any idea, please let us know!

- Prefer manual reset event objects for read() and write() on Windows.

- Do some code cleanup on our dlmalloc customizations.

- Fix errno type error in Windows blocking routines.

- Make the futex polyfill simpler and faster.
2023-10-12 23:13:04 -07:00
Justine Tunney f7343319cc Cull the examples folder 2023-10-11 21:45:32 -07:00
Justine Tunney 3a1f887928 Introduce posix_spawn_file_actions_addchdir_np() 2023-10-11 21:45:32 -07:00
Paul Kulchenko f92ad74e6b Update pidfile processing to truncate when writing pid (#910) 2023-10-11 21:45:27 -07:00
Paul Kulchenko 6e4b9b6515 Add EncodeBase32() to Redbean (#856) 2023-10-11 20:06:20 -07:00
Justine Tunney 3b086af91b Fix issues for latest GCC toolchain 2023-10-11 14:54:42 -07:00
Justine Tunney 5cb9b2658c Remove last remnants of redbean repl thread 2023-10-11 12:27:06 -07:00
Justine Tunney 285c565051 Clean up some code 2023-10-11 11:45:31 -07:00
Justine Tunney ec3275179f Fix MODE=tiny build 2023-10-10 00:58:47 -07:00
Justine Tunney 9cc4f33c76 Fix some todos 2023-10-09 23:12:32 -07:00
Justine Tunney 9d372f48dd Fix some issues 2023-10-09 20:19:09 -07:00
Gautham 211d5d902e Changes made for cosmocc builds (#908) 2023-10-09 14:39:02 -07:00
Justine Tunney 3b4dbc9fdd Make some more fixes
This change deletes mkfifo() so that GNU Make on Windows will work in
parallel mode using its pipe-based implementation. There's an example
called greenbean2 now, which shows how to build a scalable web server
for Windows with 10k+ threads. The accuracy of clock_nanosleep is now
significantly improved on Linux.
2023-10-09 12:22:00 -07:00
Justine Tunney 820c3599ed Make some quick fixes 2023-10-08 17:56:59 -07:00
Justine Tunney 94dc7a684e Fix MODE=tiny build 2023-10-08 09:34:15 -07:00
Justine Tunney 791f79fcb3 Make improvements
- We now serialize the file descriptor table when spawning / executing
  processes on Windows. This means you can now inherit more stuff than
  just standard i/o. It's needed by bash, which duplicates the console
  to file descriptor #255. We also now do a better job serializing the
  environment variables, so you're less likely to encounter E2BIG when
  using your bash shell. We also no longer coerce environ to uppercase

- execve() on Windows now remotely controls its parent process to make
  them spawn a replacement for itself. Then it'll be able to terminate
  immediately once the spawn succeeds, without having to linger around
  for the lifetime as a shell process for proxying the exit code. When
  process worker thread running in the parent sees the child die, it's
  given a handle to the new child, to replace it in the process table.

- execve() and posix_spawn() on Windows will now provide CreateProcess
  an explicit handle list. This allows us to remove handle locks which
  enables better fork/spawn concurrency, with seriously correct thread
  safety. Other codebases like Go use the same technique. On the other
  hand fork() still favors the conventional WIN32 inheritence approach
  which can be a little bit messy, but is *controlled* by guaranteeing
  perfectly clean slates at both the spawning and execution boundaries

- sigset_t is now 64 bits. Having it be 128 bits was a mistake because
  there's no reason to use that and it's only supported by FreeBSD. By
  using the system word size, signal mask manipulation on Windows goes
  very fast. Furthermore @asyncsignalsafe funcs have been rewritten on
  Windows to take advantage of signal masking, now that it's much more
  pleasant to use.

- All the overlapped i/o code on Windows has been rewritten for pretty
  good signal and cancelation safety. We're now able to ensure overlap
  data structures are cleaned up so long as you don't longjmp() out of
  out of a signal handler that interrupted an i/o operation. Latencies
  are also improved thanks to the removal of lots of "busy wait" code.
  Waits should be optimal for everything except poll(), which shall be
  the last and final demon we slay in the win32 i/o horror show.

- getrusage() on Windows is now able to report RUSAGE_CHILDREN as well
  as RUSAGE_SELF, thanks to aggregation in the process manager thread.
2023-10-08 08:59:53 -07:00
Justine Tunney af7cb3c82f Improve Windows keyboard translation to Linux 2023-10-04 10:33:03 -07:00
Justine Tunney 982dc4db87 Fix some issues with select() 2023-10-04 09:10:58 -07:00
Justine Tunney 6918c3ffc2 Delete non-standard broken strtonum() function 2023-10-04 08:20:06 -07:00
Justine Tunney af8236264e Fix silly execve() regression 2023-10-04 08:07:07 -07:00
Justine Tunney 4631d34d0d Improve stack overflow recovery
It's now possible to use sigaltstack() to recover from stack overflows
on Windows. Several bugs in sigaltstack() have been fixed, for all our
supported platforms. There's a newer better example showing how to use
this, along with three independent unit tests just to further showcase
the various techniques.
2023-10-04 07:35:17 -07:00
Justine Tunney 1694edf85c Fix some additional Windows TTY issues 2023-10-04 02:17:25 -07:00
Justine Tunney f26a280cda Implement basic canonical mode for Windows
The `cat` command now works properly, when run by itself on the bash
command prompt. It's working beautifully so far, and is only missing
a few keystrokes for clearing words and lines. Definitely works more
well than the one that ships with WIN32 :-)
2023-10-03 22:36:22 -07:00
Justine Tunney 4825737509 Make pledge() less strict about the API
We were previously returning EINVAL but OpenBSD allows it.
2023-10-03 17:57:26 -07:00
Justine Tunney 695f74035d Use CLK_TCK for clock_nanosleep() spin threshold
This more accurately reflects how the kernels actually implement this
function and it most importantly avoids incurring startup latency.
2023-10-03 17:26:29 -07:00
Justine Tunney 11c18fa644 Make ulock stracing less noisy 2023-10-03 15:19:20 -07:00
Justine Tunney 85f64f3851 Make futexes 100x better on x86 MacOS
Thanks to @autumnjolitz (in #876) the Cosmopolitan codebase is now
acquainted with Apple's outstanding ulock system calls which offer
something much closer to futexes than Grand Central Dispatch which
wasn't quite as good, since its wait function can't be interrupted
by signals (therefore necessitating a busy loop) and it also needs
semaphore objects to be created and freed. Even though ulock is an
internal Apple API, strictly speaking, the benefits of futexes are
so great that it's worth the risk for now especially since we have
the GCD implementation still as a quick escape hatch if it changes

Here's why this change is important for x86 XNU users. Cosmo has a
suboptimal polyfill when the operating system doesn't offer an API
that let's us implement futexes properly. Sadly we had to use that
on X86 XNU until now. The polyfill works using clock_nanosleep, to
poll the futex in a busy loop with exponential backoff. On XNU x86
clock_nanosleep suffers from us not being able to use a fast clock
gettime implementation, which had a compounding effect that's made
the polyfill function even more poorly. On X86 XNU we also need to
polyfill sched_yield() using select(), which made things even more
troublesome. Now that we have futexes we don't have any busy loops
anymore for both condition variables and thread joining so optimal
performance is attained. To demonstrate, consider these benchmarks

Before:

    $ ./lockscale_test.com -b
    consumed 38.8377   seconds real time and
              0.087131 seconds cpu time

After:

    $ ./lockscale_test.com -b
    consumed 0.007955 seconds real time and
             0.011515 seconds cpu time

Fixes #876
2023-10-03 15:15:43 -07:00
Justine Tunney ff250a0c10 Revert "Rewrite ZipOS"
This reverts commit b01282e23e. Some tests
are broken. It's not clear how it'll impact metal yet. Let's revisit the
memory optimization benefits of this change again sometime soon.
2023-10-03 14:40:03 -07:00
Justine Tunney ee8a861635 Fix semaphore deadlock on Apple Silicon 2023-10-03 09:26:46 -07:00
Paul Kulchenko e8ecf31ad5 Fix handling of redbean assets without extension
Fixes #902
2023-10-03 08:25:47 -07:00
Justine Tunney a201e694ee Include missing symbols in <cstdint> for C++
Fixes #899
2023-10-03 08:19:09 -07:00
Justine Tunney 5212f78c33 Fix redbean history
Fixes #903
2023-10-03 08:14:01 -07:00
tkchiaandtkchia fcdda40f19 [metal] Fix regression causing early crash in __new_page( ) (#900)
There was a glitch in the refactoring of __map_phdrs( ) in
commit ec480f5aa0, which caused it to try to map the PT_NOTE
program segment into virtual memory, which in turn caused the
memory page at BANE to be wrongly remapped to the start of
the program image (0x100000) rather than physical address 0.
This affected subsequent page allocation operations because
the `struct mman` was located at BANE + 0x0500.

Co-authored-by: tkchia <[email protected]>
2023-10-03 07:48:55 -07:00
Justine Tunney b01282e23e Rewrite ZipOS
This reduces the virtual memory usage of Emacs for me by 30%. We now
have a simpler implementation that uses read(), rather mmap()ing the
whole executable.
2023-10-03 07:27:25 -07:00
Justine Tunney ff77f2a6af Make improvements
- This change fixes a bug that allowed unbuffered printf() output (to
  streams like stderr) to be truncated. This regression was introduced
  some time between now and the last release.

- POSIX specifies all functions as thread safe by default. This change
  works towards cleaning up our use of the @threadsafe / @threadunsafe
  documentation annotations to reflect that. The goal is (1) to use
  @threadunsafe to document functions which POSIX say needn't be thread
  safe, and (2) use @threadsafe to document functions that we chose to
  implement as thread safe even though POSIX didn't mandate it.

- Tidy up the clock_gettime() implementation. We're now trying out a
  cleaner approach to system call support that aims to maintain the
  Linux errno convention as long as possible. This also fixes bugs that
  existed previously, where the vDSO errno wasn't being translated
  properly. The gettimeofday() system call is now a wrapper for
  clock_gettime(), which reduces bloat in apps that use both.

- The recently-introduced improvements to the execute bit on Windows has
  had bugs fixed. access(X_OK) on a directory on Windows now succeeds.
  fstat() will now perform the MZ/#! ReadFile() operation correctly.

- Windows.h is no longer included in libc/isystem/, because it confused
  PCRE's build system into thinking Cosmopolitan is a WIN32 platform.
  Cosmo's Windows.h polyfill was never even really that good, since it
  only defines a subset of the subset of WIN32 APIs that Cosmo defines.

- The setlongerjmp() / longerjmp() APIs are removed. While they're nice
  APIs that are superior to the standardized setjmp / longjmp functions,
  they weren't superior enough to not be dead code in the monorepo. If
  you use these APIs, please file an issue and they'll be restored.

- The .com appending magic has now been removed from APE Loader.
2023-10-03 06:17:16 -07:00
Justine Tunney b99512ac58 Temporarily disable test_suite_x509parse.com
This test broke itself due to relying on the current time. Mocking out
gettimeofday() confirms this. However, two ipv4 subject alt name tests
still surprisingly fail even with a fake current time. We'll want this
investigated further soon.
2023-10-02 13:49:16 -07:00
Gavin Hayes c315315896 fleaks.c: initialize command buffer before calling system (#904) 2023-10-02 13:31:15 -07:00
Gavin Hayes cc8c56e814 Fix strtod NaN handling / fix SIGSEGV in testlib/showerror (#901) 2023-09-27 00:16:36 -07:00
Justine Tunney 3ffc17c50e Add Cosmopolitan to uname() 2023-09-21 23:51:55 -07:00
Justine Tunney ed316491ca Fix issue with pasting text into console 2023-09-21 13:28:52 -07:00
Justine Tunney 4f5d5a6813 Fix some more issues
- ARM Neon headers are now exported in libc/isystem/

- stat() and access() now do a better job reporting which files are
  executable which ones aren't. They do this by reading the first two
  bytes in a file to see if it's `MZ` or `#!`.
2023-09-21 11:41:42 -07:00
Justine Tunney 22cf6e11eb Add siglongjmp() for aarch64 2023-09-21 10:10:20 -07:00
Justine Tunney c88f95a892 Remove Windows executable path guessing logic
Unlike CMD.EXE, CreateProcess() doesn't care if an executable name ends
with .COM or .EXE. We now have the unbourne shell and bash working well
on Windows, so we don't need DOS anymore. Making this change will grant
us better performance, particularly for builds, because commandv() will
need to make fewer system calls. Path mangling magic still happens with
WinMain() and ntspawn() in order to do things like turn \ into / so the
interop works well at the borders. But all the code in libraries, which
did that, has been removed. It's not possible for libraries to abstract
the differences between paths.
2023-09-21 08:13:50 -07:00
Justine Tunney 0c5dd7b342 Make improvements
- Improved async signal safety of read() particularly for longjmp()
- Started adding cancel cleanup handlers for locks / etc on Windows
- Make /dev/tty work better particularly for uses like `foo | less`
- Eagerly read console input into a linked list, so poll can signal
- Fix some libc definitional bugs, which configure scripts detected
2023-09-21 07:30:39 -07:00
Justine Tunney d6c2830850 Rewrite Windows console input handling
This change removes our use of ENABLE_VIRTUAL_TERMINAL_INPUT (which
isn't very good) in favor of having read() translate Windows Console
input events to ANSI/XTERM sequences by hand. This makes it possible to
capture important keystrokes (e.g. ctrl-space) that weren't possible
before. Most importantly this change also removes the stdin/sigwinch
worker threads, which never really worked that well. Interactive TTY
sessions will now work reliably when a Cosmo process spawns or forks
another Cosmo process, e.g. unbourne.com launching emacs.com.
2023-09-19 11:53:27 -07:00
Justine Tunney ececec4c94 Fix some zipos directory related bugs 2023-09-19 02:30:42 -07:00
Justine Tunney ec480f5aa0 Make improvements
- Every unit test now passes on Apple Silicon. The final piece of this
  puzzle was porting our POSIX threads cancelation support, since that
  works differently on ARM64 XNU vs. AMD64. Our semaphore support on
  Apple Silicon is also superior now compared to AMD64, thanks to the
  grand central dispatch library which lets *NSYNC locks go faster.

- The Cosmopolitan runtime is now more stable, particularly on Windows.
  To do this, thread local storage is mandatory at all runtime levels,
  and the innermost packages of the C library is no longer being built
  using ASAN. TLS is being bootstrapped with a 128-byte TIB during the
  process startup phase, and then later on the runtime re-allocates it
  either statically or dynamically to support code using _Thread_local.
  fork() and execve() now do a better job cooperating with threads. We
  can now check how much stack memory is left in the process or thread
  when functions like kprintf() / execve() etc. call alloca(), so that
  ENOMEM can be raised, reduce a buffer size, or just print a warning.

- POSIX signal emulation is now implemented the same way kernels do it
  with pthread_kill() and raise(). Any thread can interrupt any other
  thread, regardless of what it's doing. If it's blocked on read/write
  then the killer thread will cancel its i/o operation so that EINTR can
  be returned in the mark thread immediately. If it's doing a tight CPU
  bound operation, then that's also interrupted by the signal delivery.
  Signal delivery works now by suspending a thread and pushing context
  data structures onto its stack, and redirecting its execution to a
  trampoline function, which calls SetThreadContext(GetCurrentThread())
  when it's done.

- We're now doing a better job managing locks and handles. On NetBSD we
  now close semaphore file descriptors in forked children. Semaphores on
  Windows can now be canceled immediately, which means mutexes/condition
  variables will now go faster. Apple Silicon semaphores can be canceled
  too. We're now using Apple's pthread_yield() funciton. Apple _nocancel
  syscalls are now used on XNU when appropriate to ensure pthread_cancel
  requests aren't lost. The MbedTLS library has been updated to support
  POSIX thread cancelations. See tool/build/runitd.c for an example of
  how it can be used for production multi-threaded tls servers. Handles
  on Windows now leak less often across processes. All i/o operations on
  Windows are now overlapped, which means file pointers can no longer be
  inherited across dup() and fork() for the time being.

- We now spawn a thread on Windows to deliver SIGCHLD and wakeup wait4()
  which means, for example, that posix_spawn() now goes 3x faster. POSIX
  spawn is also now more correct. Like Musl, it's now able to report the
  failure code of execve() via a pipe although our approach favors using
  shared memory to do that on systems that have a true vfork() function.

- We now spawn a thread to deliver SIGALRM to threads when setitimer()
  is used. This enables the most precise wakeups the OS makes possible.

- The Cosmopolitan runtime now uses less memory. On NetBSD for example,
  it turned out the kernel would actually commit the PT_GNU_STACK size
  which caused RSS to be 6mb for every process. Now it's down to ~4kb.
  On Apple Silicon, we reduce the mandatory upstream thread size to the
  smallest possible size to reduce the memory overhead of Cosmo threads.
  The examples directory has a program called greenbean which can spawn
  a web server on Linux with 10,000 worker threads and have the memory
  usage of the process be ~77mb. The 1024 byte overhead of POSIX-style
  thread-local storage is now optional; it won't be allocated until the
  pthread_setspecific/getspecific functions are called. On Windows, the
  threads that get spawned which are internal to the libc implementation
  use reserve rather than commit memory, which shaves a few hundred kb.

- sigaltstack() is now supported on Windows, however it's currently not
  able to be used to handle stack overflows, since crash signals are
  still generated by WIN32. However the crash handler will still switch
  to the alt stack, which is helpful in environments with tiny threads.

- Test binaries are now smaller. Many of the mandatory dependencies of
  the test runner have been removed. This ensures many programs can do a
  better job only linking the the thing they're testing. This caused the
  test binaries for LIBC_FMT for example, to decrease from 200kb to 50kb

- long double is no longer used in the implementation details of libc,
  except in the APIs that define it. The old code that used long double
  for time (instead of struct timespec) has now been thoroughly removed.

- ShowCrashReports() is now much tinier in MODE=tiny. Instead of doing
  backtraces itself, it'll just print a command you can run on the shell
  using our new `cosmoaddr2line` program to view the backtrace.

- Crash report signal handling now works in a much better way. Instead
  of terminating the process, it now relies on SA_RESETHAND so that the
  default SIG_IGN behavior can terminate the process if necessary.

- Our pledge() functionality has now been fully ported to AARCH64 Linux.
2023-09-18 21:04:47 -07:00
Gavin Hayes c4eb838516 README: note version zsh and fish were patched to support ape (#896) 2023-09-15 08:10:11 -07:00
Justine Tunney 81f391dd22 Rewrite Windows signal delivery system 2023-09-12 11:38:34 -07:00
Justine Tunney 00084577a3 Improve posix_spawn() some more 2023-09-12 08:58:57 -07:00
Justine Tunney 6430e474b4 Remove VM variable
You need to use qemu-user in binfmt_misc. For non-Linux we'll update
execve() to spawn under the appropriate blink or qemu when needed.
2023-09-12 01:27:30 -07:00
Justine Tunney 20c77338e6 Remove IMAGE_BASE_VIRTUAL 2023-09-12 01:21:36 -07:00
Justine Tunney 8a0008d985 Avoid leaking handles across processes 2023-09-12 01:07:51 -07:00
Justine Tunney a359de7893 Get rid of kmalloc()
This changes *NSYNC to allocate waiters on the stack so our locks don't
need to depend on dynamic memory. This make our runtiem simpler, and it
also fixes bugs with thread cancellation support.
2023-09-11 21:56:00 -07:00
Justine Tunney 77a7873057 Improve AARCH64 execution
This change fixes bugs in the APE loader. The execve() unit tests are
now enabled for MODE=aarch64. See the README for how you need to have
binfmt_misc configured with Qemu to run them. Apple Silicon bugs have
been fixed too, e.g. tkill() now works.
2023-09-11 14:46:46 -07:00
Justine Tunney 1965d7488e Improve performance and remove fd leaks 2023-09-10 11:52:03 -07:00
Justine Tunney 26e254fb4d Overhaul process spawning 2023-09-10 08:17:44 -07:00
Justine Tunney 99dc1281f5 Overhaul Windows signal handling
The new asynchronous signal delivery technique is now also being used
for tkill(), raise(), etc. Many subtle issues have been addresesd. We
now signal handling on Windows that's remarkably similar to the POSIX
behaviors. However that's just across threads. We're lacking a way to
have the signal semantics work well, across multiple WIN32 processes.
2023-09-08 01:49:41 -07:00
Justine Tunney 8bdaddd81d Make the Windows Console work better
The stdio reader thread now appears to be working recursively along
cosmopolitan subprocesses. For example, it's now possible to launch
vim.com from the unbourne.com bestline repl, thanks to hacks plus a
bug fix to select() timeouts.
2023-09-07 18:27:22 -07:00
Justine Tunney 032b1f3449 Implement thread cancellation for aarch64 2023-09-07 08:48:38 -07:00
Justine Tunney dcda6f7d8d Simplify the rusage.com command 2023-09-07 05:41:18 -07:00
Justine Tunney b592716d1c Reduce mandatory stack rss by 256kb 2023-09-07 04:33:01 -07:00
Justine Tunney 0e087143fd Make greenbean web server better
- Remove misguided __assert_disabled variable
- Change EPROCLIM to be EAGAIN on BSD distros
- Improve quality of greenbean with cancellations
- Fix thread race condition crash with file descriptors
2023-09-07 03:44:50 -07:00
Justine Tunney 6cff3137c5 Fix GitHub Actions 2023-09-06 23:39:47 -07:00
Justine Tunney 425c055116 Make improvements
- Polyfill readlink("foo/") dir check on Windows
- Support asynchronous signal delivery on Windows
- Restore Windows Console from execve() daisy chain
- Work around bug in AARCH64 Optimized Routines memcmp()
- Disable unbourne.com shell completion on Windows for now
- Don't always set virtual terminal input state on console
- Remove Musl Libc's unusual preservation of realpath("//")
- Make realpath() strongly link malloc() to pass configure test
- Delete cosh.com shell, now that unbourne.com works on Windows!
2023-09-06 22:48:05 -07:00
Justine Tunney f9c9a323fe Fix Landlock Make crash when TMPDIR isn't defined 2023-09-06 12:49:52 -07:00
Justine Tunney f531acc8f9 Make improvements
- Invent openatemp() API
- Invent O_UNLINK open flag
- Introduce getenv_secure() API
- Remove `git pull` from cosmocc
- Fix utimes() when path is NULL
- Fix mktemp() to never return NULL
- Fix utimensat() UTIME_OMIT on XNU
- Improve utimensat() code for RHEL5
- Turn `argv[0]` C:/ to /C/ on Windows
- Introduce tmpnam() and tmpnam_r() APIs
- Fix more const issues with internal APIs
- Permit utimes() on WIN32 in O_RDONLY mode
- Fix fdopendir() to check fd is a directory
- Fix recent crash regression in landlock make
- Fix futimens(AT_FDCWD, NULL) to return EBADF
- Use workaround so `make -j` doesn't fork bomb
- Rename dontdiscard to __wur (just like glibc)
- Fix st_size for WIN32 symlinks containing UTF-8
- Introduce stdio ext APIs needed by GNU coreutils
- Fix lstat() on WIN32 for symlinks to directories
- Move some constants from normalize.inc to limits.h
- Fix segv with memchr() and memcmp() overlapping page
- Implement POSIX fflush() behavior for reader streams
- Implement AT_SYMLINK_NOFOLLOW for utimensat() on WIN32
- Don't change read-only status of existing files on WIN32
- Correctly handle `0x[^[:xdigit:]]` case in strtol() functions
2023-09-06 12:34:59 -07:00
michalbiesek 8596e83cce Linux add definition for madvise flag (#883) 2023-09-06 12:34:54 -07:00
tkchiaandtkchia b21842ed7a [metal] Fix video mode filtering & frame buffer ref-counting (#889)
Co-authored-by: tkchia <[email protected]>
2023-09-06 03:41:07 -07:00
Justine Tunney 0d748ad58e Fix warnings
This change fixes Cosmopolitan so it has fewer opinions about compiler
warnings. The whole repository had to be cleaned up to be buildable in
-Werror -Wall mode. This lets us benefit from things like strict const
checking. Some actual bugs might have been caught too.
2023-09-01 20:50:18 -07:00
Justine Tunney e2b3c3618e Move zlib down 2023-08-31 15:17:57 -07:00
stellartux 4021cd0c1e Update redbean definitions (#887)
- Add OnServerReload signature
- Fix Pledge, Unveil, Slurp, ProgramLogPath signatures
- Add pledge flags
2023-08-26 18:40:07 -07:00
Justine Tunney 4c74f09393 Fix socket() EPROTONOSUPPORT fallback on MacOS 2023-08-22 14:34:37 -07:00
Justine Tunney 58ef4e6df8 Add i/o statistics to wait4() about child process 2023-08-21 21:25:33 -07:00
Justine Tunney b9eb656e41 Fix Windows stdin regression
The non-blocking standard input feature was broken by ec957491e.
2023-08-21 21:04:05 -07:00
Justine Tunney 7e08a97cea Further improve scanf 2023-08-21 16:55:29 -07:00
Justine Tunney 6ef2a471e4 Get GNU MPFR and MPC tests to pass
This change fixes more issues with our scanf() function.
2023-08-21 15:05:10 -07:00
Justine Tunney 63a1636e1f Get GNU GMP test suite fully passing
- Fix stdio fmemopen() buffer behaviors
- Fix scanf() to return EOF when appropriate
- Prefer fseek/ftell names over fseeko/ftello
- Ensure locale field is always set in the TIB
- Fix recent regression in vfprintf() return count
- Make %n directive in scanf() have standard behavior
2023-08-21 10:16:42 -07:00
Justine Tunney 755ae64e73 Remove long path tests for Windows 2023-08-21 04:43:53 -07:00
Justine Tunney fffcd98b0e Fix msync() flags on FreeBSD 2023-08-21 04:15:05 -07:00
Justine Tunney ebf784d4f5 Make improvements
- Introduce ualarm() function
- Make rename() report EISEMPTY on Windows
- Always raise EINVAL upon open(O_RDONLY|O_TRUNC)
- Add macro so ./configure will detect SOCK_CLOEXEC
- Fix O_TRUNC without O_CREAT not working on Windows
- Let fcntl(F_SETFL) change O_APPEND status on Windows
- Make sure pwrite() / pread() report ESPIPE on sockets
- Raise ESPIPE on Windows when pwrite() is used on pipe
- Properly compute O_APPEND CreateFile() flags on Windows
- Don't require O_DIRECTORY to open directories on Windows
- Fix more instances of Windows reporting EISDIR and ENOTDIR
- Normalize EFTYPE and EMLINK to ELOOP on NetBSD and FreeBSD
- Make unlink() / rmdir() work on read-only files on Windows
- Validate UTF-8 on Windows paths to fix bug with overlong NUL
- Always print signal name to stderr when crashing due to SIG_DFL
- Fix Windows bug where denormalized paths >260 chars didn't work
- Block signals on BSDs when thread exits before trashing its own stack
2023-08-21 02:34:17 -07:00
Justine Tunney ec957491ea Make posix_spawn faster on Windows 2023-08-20 02:50:07 -07:00
michalbiesek d6f72aa4a6 Extend pr.h header with recent options (#881) 2023-08-20 02:49:58 -07:00
michalbiesek 4f517503e4 Add PTRACE_SECCOMP_GET_METADATA (#882)
see
  linux commit: 26500475ac1b499d8636ff281311d633909f5d20
  ptrace, seccomp: add support for retrieving seccomp metadata
2023-08-20 02:49:22 -07:00
Justine Tunney 610e0d95cb Fix the MODE=tinylinux build 2023-08-19 06:56:05 -07:00
Justine Tunney 965516e313 Make improvements for Actually Portable Emacs
- Get SIGWINCH working again on the New Technology
- Correctly handle O_NOFOLLOW in open() on Windows
- Implement synthetic umask() functionality on Windows
- Do a better job managing file execute access on Windows
- Fill in `st_uid` and `st_gid` with username hash on Windows
- Munge UNICODE control pictures into control codes on Windows
- Do a better job ensuring Windows console settings are restored
- Introduce KPRINTF_LOG environment variable to log kprintf to a file
2023-08-19 06:44:58 -07:00
Justine Tunney 9c7b81ee0f Give Emacs another performance boost 2023-08-18 09:34:14 -07:00
Justine Tunney 5b42c810a5 Add filesystem index to ZipOS
This change brings the /zip/... read-only filesystem into performance
parity with the native Linux filesystem which doesn't use compression
therefore, imagine how much faster this could be with bloom filtering
rather than simple binary search, and if we used zstd instead of zlib
2023-08-18 07:11:48 -07:00
Justine Tunney 7100b1cf91 Get Fat Emacs working in Windows Console 2023-08-18 05:00:30 -07:00
Justine Tunney bf835de612 Get Fat Emacs working on Apple Silicon 2023-08-17 22:01:42 -07:00
Justine Tunney 3f9b39883f Make fat ape binaries smaller again 2023-08-17 11:15:58 -07:00
Justine Tunney 1d8937d528 Mint APE Loader v1.7
This change reduces the memory requirements of your APE Loader by 10x,
in terms of virtual memory size, thanks to the help of alloca(). We're
also now creating argument blocks with the same layout across systems.
2023-08-17 09:04:50 -07:00
Paul Kulchenko d967a94c9a Update ProgramDirectory to prepend Lua path instead of appending (#862). (#864)
* Update ProgramDirectory to prepend Lua path instead of appending (#862).

* Update redbean ProgramDirectory to raise Lua error for easier handling

* Update redbean default Lua path handling to simplify
2023-08-17 00:43:11 -07:00
Paul Kulchenko d0d027810a Add OnServerReload to replace /.reload.lua with a bit more flexibility (#851) 2023-08-17 00:42:23 -07:00
Justine Tunney 3a9cac4892 Fix small matters and improve sysconf()
- Fix mkdeps.com out of memory error
- Remove static memory from __get_cpu_count()
- Add support for passing hyphen to cat in cocmd
- Change more ZipOS errors from ENOTSUP to EROFS
- Specify mem_unit in sysinfo() output on BSD OSes
2023-08-17 00:32:11 -07:00
Gautham eebc24b9cd get rsync to build (#886) 2023-08-16 21:41:36 -07:00
Justine Tunney 04bd488da7 Introduce COSMOPOLITAN_DISABLE_ZIPOS environ var 2023-08-16 21:04:16 -07:00
Justine Tunney 8d1c81ac9f Emulate ENOTDIR better 2023-08-16 20:11:23 -07:00
Justine Tunney b76b2be2d0 Improve zip read-only filesystem
readdir() will now always yield an inode that's consistent with stat()
on ZipOS and Windows in general. More APIs have been updated to return
the appropriate error code when inappropriately trying to do ops, like
sockets, with a zip file descriptor. The path normalization algorithms
are now fully fleshed out. Some socket APIs have been fixed so they'll
raise EBADF vs. ENOTSOCK appropriately. Lastly seekdir() will now work
properly on NetBSD and FreeBSD (not sure why anyone would even use it)
2023-08-16 17:52:12 -07:00
Justine Tunney dc6c67256f Remove old stack code and improve dirstream 2023-08-16 07:54:40 -07:00
Justine Tunney 74caabb823 Fix getopt again 2023-08-15 20:06:19 -07:00
Justine Tunney 507d7a0b0b Fix stack memory, undefined behavior, etc. 2023-08-15 19:10:08 -07:00
Justine Tunney 110559ce6a Make ZipOS and Qemu work better
This change improves the dirstream library in a lot of respects,
especially for /zip/... files. Also turn off MAP_STACK on Aarch64
because Qemu seems to implement it differently than Linux and it's
probably responsible for a lot of mysterious crashes.
2023-08-15 18:32:50 -07:00
Justine Tunney 4658ae539f Remove old Windows thread hack from Redbean 2023-08-15 06:06:43 -07:00
Justine Tunney de3f3a9e5a Allocate explicit stack on aarch64 2023-08-15 04:40:19 -07:00
Justine Tunney 0e586c834a Refactor fatcosmocc into a single file 2023-08-14 22:26:17 -07:00
Justine Tunney b494d56549 Fix the cosmocc ar commands 2023-08-14 20:12:51 -07:00
Gautham f5b39e9f31 Fix typo in cosmocc (#884) 2023-08-13 23:31:39 -07:00
Justine Tunney 2b67e15b3d Fix MODE=tinylinux build 2023-08-13 22:59:56 -07:00
Justine Tunney 9c0821def7 Make stdin pollable on Windows
You can now play Super Mario Bros in CMD.EXE using Cosmopolitan! This is
thanks to a new worker thread that's spawned on Windows whenever any one
of poll(), select(), or ioctl(FIONREAD) is linked.
2023-08-13 22:42:25 -07:00
Justine Tunney ef6387ee5e Polyfill the 32-bit _bsr() function 2023-08-13 21:13:32 -07:00
Justine Tunney c776a32f75 Replace COSMO define with _COSMO_SOURCE
This change might cause ABI breakages for /opt/cosmos. It's needed to
help us better conform to header declaration practices.
2023-08-13 20:55:04 -07:00
Justine Tunney a033b65a33 Fix apelink shell script corruption bug
We were using a shell heredoc value '@' to terminate the dos stub, but
that's not sufficiently safe. We found out sh doesn't consider control
characters as contributing to the start of a line, and had the unlucky
chance of the linker choosing the number 2624 for e_lfanew, and that's
"@\n" in ASCII, which compromised the APE shell script.

We now use the heredoc 'justineXXXXXX' with 31 bits of entropy, that's
determistically generated by hashing apelink inputs w/ crc32 / blake2b
2023-08-13 17:42:11 -07:00
Justine Tunney 2366848db5 Perform some more touchups to toolchain 2023-08-13 15:03:09 -07:00
Justine Tunney d1b937bf1d Make cosmocc capable of cross compilation 2023-08-13 14:40:22 -07:00
Justine Tunney 504a4bbf84 Fix --ftrace with cosmocc 2023-08-13 08:25:36 -07:00
Justine Tunney 5d5b4ce691 Add ANSI version of bsr/bsf 2023-08-13 08:19:00 -07:00
Justine Tunney b40d41085d Fix the build 2023-08-13 07:17:40 -07:00
Justine Tunney 6942d7b820 Fix bug with temporary files on Windows 2023-08-13 07:10:33 -07:00
Justine Tunney ab9a284640 Further improve fatcosmocc 2023-08-13 01:51:39 -07:00
Justine Tunney 3f2f0e3a74 Make fatcosmocc good enough to build ncurses 6.4 2023-08-12 22:30:05 -07:00
Justine Tunney 399d14aadf Make fatcosmocc good enough to build Lua 5.4.6
make all test CC=fatcosmocc AR='fatcosmoar rcu'

This change introduces a program named mktemper.com which provides more
reliable and secure temporary file name generation for scripts. It also
makes our ar.com program more permissive in what commands it'll accept.
The cosmocc command is improved by this change too.
2023-08-12 16:44:04 -07:00
Justine Tunney 566cb5963f Make assimilate.com better
It's now safer to run. It'll now remove FreeBSD from the ELF os/abi so
that GDB is happier.
2023-08-12 07:46:24 -07:00
Justine Tunney f491276b62 Add support for C++ thread safe statics 2023-08-12 07:45:32 -07:00
Justine Tunney 60f2ea36ca Make default localtime() zone /etc/localtime 2023-08-12 07:44:15 -07:00
Justine Tunney e11fa30791 Move zipos into runtime package
This way complex runtime features (e.g. ftrace, symbol tables) can
always yoink zipos support. This is important now that apelink.com
automates embedding symbol tables for multiple cpus.
2023-08-11 23:14:02 -07:00
Justine Tunney d53c335a45 Introduce new fatcosmocc command
This new script is an alternative to the `cosmocc` command. It's still a
work in progress. It abstracts all the gory details of building separate
copies of your executable and then running the apelink.com program.
2023-08-11 22:52:11 -07:00
Justine Tunney 8fc778162e Make stderr go faster
This change makes _IONBF (unbuffered) stdio handles go 20x faster for
certain kinds of formatting directives by being smarter about buffers
2023-08-11 11:56:35 -07:00
Justine Tunney 2cbd09b4d4 Fix the build 2023-08-11 09:49:39 -07:00
Justine Tunney 0105e3e2b6 Introduce new linker for fat ape binaries 2023-08-11 04:39:19 -07:00
Michael Lenaghan e3c456d23a Don’t lowercase the test filename (#871)
This test fails if Cosmo builds from a path that contains an uppercase character.

Paths with uppercase characters aren’t so common in server Linux. But they are in *desktop* Linux. Guess how I…?

But I digress.

The real problem is that the path is lowercased on one line, but not the next:

```
        self.file_name = support.TESTFN.lower()
        self.file_path = FakePath(support.TESTFN)
```

Given that no other test in the suite lowercases `support.TESTFN`, I opted to remove it from the first line rather than adding it to the second.
2023-08-09 21:13:03 -07:00
Justine Tunney dd53f31147 Introduce post-linker that converts ELF to PE
If you build a static ELF executable in `ld -q` mode (which leaves rela
sections inside the binary) then you can run it through the elf2pe.com
program afterwards, which will turn it into a PE executable. We have a
new trick for defining WIN32 DLL imports in C without any assembly code.
This also achieves the optimally tiny and perfect PE binary structure.

We need this because it isn't possible to have a GNU ld linker script
generate a PE file where the virtual pointer and the file pointer can
drift apart. This post-linker can do that. One cool benefit is we can
now use a smaller 512-byte alignment in the file, and an even bigger
64kb alignment for the segment virtual addresses, and the executable
ends up being smaller.

Another program introduced by this change is pecheck.com which can do
extensive linting of PE static executables to help explain why Windows
won't load it.
2023-08-09 18:46:06 -07:00
Justine Tunney 2f35bbf046 Fix Windows stack size 2023-08-09 00:51:30 -07:00
Justine Tunney 9dd50f7dfc Better document PE headers 2023-08-09 00:29:01 -07:00
Justine Tunney 50394064d7 Invent systemvpe() function
It goes 5x faster than system() and it's safer too.
2023-08-09 00:27:55 -07:00
Justine Tunney 1a5ef5ba13 Mint APE Loader 1.6
This change fixes a bug with loading pure bss program headers.
2023-08-09 00:27:26 -07:00
Justine Tunney c48ee8e4fe Fix POLLIN|POLLOUT when together on Windows 2023-08-08 10:33:52 -07:00
Justine Tunney 33d280c8ba Improve Windows Console I/O
- Blocking read operations on the Windows Console can now EINTR
- Blocking read operations on Windows pipes now EINTR more reliably
- setitimer() will no longer be inherited across fork() on Windows
- It's now possible to use ECHO when the console is in raw mode
- The ECHOCTL flag now works correctly on the Windows Console
- The ICRNL flag now works correctly on the Windows Console
- pread() and pwrite() will now raise ESPIPE on Windows
- Opening /dev/tty on Windows is improved (untested)
- Overlapped I/O is now implemented in a better way
2023-08-08 05:44:40 -07:00
Justine Tunney decf216655 Perform inconsequential code cleanup 2023-08-07 20:24:50 -07:00
Justine Tunney 929478c524 Clean up more WIN32 APIs 2023-08-01 14:17:03 -07:00
Gautham 44b88d659d Fix x-axis blurring in nesemu1 tty conversion (#878) 2023-08-01 08:39:27 -07:00
Joel Yliluoma f729810f85 Losslessly recompress PNG files (#877)
Original sizes:

 639082 libc/dns/dns.png
  21281 usr/share/img/honeybadger.png
   4710 third_party/python/Lib/idlelib/Icons/idle_48.png
   2542 third_party/python/Lib/idlelib/Icons/idle_32.png
   1264 third_party/python/Lib/idlelib/Icons/idle_16.png
   1020 third_party/python/Lib/test/imghdrdata/python.png

Updated sizes:

 484841 libc/dns/dns.png
  17737 usr/share/img/honeybadger.png
   3041 third_party/python/Lib/idlelib/Icons/idle_48.png
   1606 third_party/python/Lib/idlelib/Icons/idle_32.png
    701 third_party/python/Lib/idlelib/Icons/idle_16.png
    602 third_party/python/Lib/test/imghdrdata/python.png
2023-07-31 17:59:16 -07:00
Justine Tunney 712b1aab3c Make quick fix for aarch64 build 2023-07-30 15:06:28 -07:00
Justine Tunney c8aa33e0e2 Improve wait statuses
This change has the insight that dwExitCode isn't an exit code but
rather should be used to pass the wait status. This lets us report
killing as a termination status, similar to UNIX. This change also
fixes the fact that exit(259) on Windows will break the parent due
way WIN32 is designed. We now work around that.

It turns out that NetBSD and OpenBSD, will let you have exit codes
beyond 255. This change will let you use them when it's possible.
2023-07-30 14:51:37 -07:00
Justine Tunney d9d5f45e2d Restore missing cosmopolitan documentation on website 2023-07-30 11:07:53 -07:00
Justine Tunney 58352df0a4 Make forking off threads reliable on Windows
This change makes posix_spawn_test no longer flaky on Windows, by (1)
fixing a race condition in wait(), and (2) removing a misguided vfork
implementation which was letting Windows bypass pthread_atfork().
2023-07-30 09:32:41 -07:00
Justine Tunney 2ebc5781a1 Fix flakes in runitd and popen_test 2023-07-30 04:26:34 -07:00
Justine Tunney 801224df67 Support symbol tables with arch specific name 2023-07-29 23:50:15 -07:00
Justine Tunney bd49ea1c3a Fix tests on aarch64 2023-07-29 19:02:25 -07:00
Justine Tunney 18bb5888e1 Make more fixes and improvements
- Remove PAGESIZE constant
- Fix realloc() documentation
- Fix ttyname_r() error reporting
- Make forking more reliable on Windows
- Make execvp() a few microseconds faster
- Make system() a few microseconds faster
- Tighten up the socket-related magic numbers
- Loosen restrictions on mmap() offset alignment
- Improve GetProgramExecutableName() with getenv("_")
- Use mkstemp() as basis for mktemp(), tmpfile(), tmpfd()
- Fix flakes in pthread_cancel_test, unix_test, fork_test
- Fix recently introduced futex stack overflow regression
- Let sockets be passed as stdio to subprocesses on Windows
- Improve security of bind() on Windows w/ SO_EXCLUSIVEADDRUSE
2023-07-29 18:44:15 -07:00
Justine Tunney 140a8a52e5 Remove 5000 WIN32 APIs 2023-07-28 14:32:34 -07:00
Justine Tunney 6c7b4fcbd3 Delete reference to old echo program 2023-07-28 14:10:22 -07:00
Justine Tunney 06082c7d37 Polyfill fchmodat() 2023-07-28 07:41:43 -07:00
Justine Tunney 7926aa8bfa Remove ELF binaries from tests 2023-07-28 07:20:57 -07:00
Justine Tunney 5018171fa5 Fix a bunch of Windows bugs reported on Discord
This change addresses everything from stack smashing to %SYSTEMROOT%
breaking socket(). Issues relating to compile.com not reporting text
printed to stderr has been resolved for Windows builds.
2023-07-28 06:17:34 -07:00
Justine Tunney b0e3258709 Fix close(1) bug on Windows 2023-07-28 03:56:56 -07:00
Justine Tunney 83341a4269 Remove hints from Windows imports 2023-07-27 14:09:07 -07:00
Justine Tunney 7e0a09feec Mint APE Loader v1.5
This change ports APE Loader to Linux AARCH64, so that Raspberry Pi
users can run programs like redbean, without the executable needing
to modify itself. Progress has also slipped into this change on the
issue of making progress better conforming to user expectations and
industry standards regarding which symbols we're allowed to declare
2023-07-26 13:54:49 -07:00
Justine Tunney 6843150e0c Mint APE Loader v1.4
This change also incorporates more bug fixes and improvements to a wide
variety of small things. For example this fixes #860 so Windows console
doesn't get corrupted after exit. An system stack memory map issue with
aarch64 has been fixed. We no longer use O_NONBLOCK on AF_UNIX sockets.
Crash reports on Arm64 will now demangle C++ symbols, even when c++filt
isn't available. Most importantly the Apple M1 version of APE Loader is
brought up to date by this change. A prebuilt unsigned binary for it is
being included in build/bootstrap/. One more thing: retrieving the term
dimensions under --strace was causing the stack to become corrupted and
now that's been solved too. PSS: We're now including an ELF PT_NOTE for
APE in the binaries we build, that has the APE Loader version.
2023-07-25 05:48:08 -07:00
Michael Lenaghan 53d3f9d9c5 Change the default TZ value from GST to GMT #861 (#867)
With this change, `unix.localtime()` will match `unix.gmtime()`
when `TZ` hasn't been defined.
2023-07-24 20:40:15 -07:00
Andy George 9c79c5682c Add sh code tagging (#855) 2023-07-24 20:15:18 -07:00
Justine Tunney e0c2b91b3e Remove _Hide keyword
It never did anything and isn't worthwhile as documentation.
2023-07-24 08:34:58 -07:00
Justine Tunney 4fb6cbc1fe Revert "Polyfill ENOTSOCK in getsockname() and getpeername()"
This reverts commit 925219bdf3.
2023-07-24 03:09:43 -07:00
Justine Tunney 925219bdf3 Polyfill ENOTSOCK in getsockname() and getpeername() 2023-07-24 02:11:25 -07:00
Justine Tunney eb84a25994 Check for interrupts more eagerly on Windows 2023-07-24 02:03:29 -07:00
Justine Tunney 94ea34367a Tune the page sizes 2023-07-24 00:49:06 -07:00
Justine Tunney ff198e7577 Introduce GetElfSegmentAddress() function 2023-07-23 17:36:18 -07:00
Justine Tunney 3d172c99fe Mint APE Loader v1.3
This version has better error messages and safety checks. It supports
loading static position-independent executables. It correctly handles
more kinds of weird ELF program header layouts. A force flag has been
added to avoid system execve(). Finally the longstanding misalignment
with our ELF PT_NOTE section has been addressed.
2023-07-23 17:08:14 -07:00
Justine Tunney 82b1e61443 Fix Landlock Make build config issue 2023-07-23 16:43:22 -07:00
Justine Tunney 0ba3199915 Fix some more socket bugs
- The functions that return a sockaddr now do so the same way the Linux
  Kernel does across platforms, e.g. getpeername(), accept4()

- Socket system calls on Windows will now only check for interrupts when
  a blocking operation needs to be performed.

- Write tests for recvfrom() system call
2023-07-23 16:31:10 -07:00
Danny Robinson ac92f25296 Update definitions.lua with latest progress (#852)
* Update definitions.lua with latest progress

* Remove duplicate definition
2023-07-23 11:14:55 -07:00
tkchiaandtkchia 0ffc0dd461 Let lz4toasm accept LZ4 files that lack extracted-size field (#858)
lz4toasm should now more easily accept LZ4 files output by
compressor programs that do not support the extracted-size
field, such as Stephan Brumme's smallz4.

This patch also proposes to add a new lz4len() function to
the libc: it parses an LZ4 compressed block to compute the
unpacked content size, without really unpacking the block.

Co-authored-by: tkchia <[email protected]>
2023-07-23 11:12:22 -07:00
Gautham 3eec69f9c3 Simplify cosmocc builds (#863)
- tcgetpgrp(STDIN_FILENO) should be equal to getpgrp() on Windows also,
  found while reading wget source code which uses this check to decide
  whether to print to stderr or to a file
- IN6_ADDR_ARE_EQUAL is a comparison macro used when IPV6 is allowed,
  found while reading CPython3.11 source code
- the changes in signal.h and addition of ucontext.h are because
  CPython3.11 source code expect sigaltstack to be available
- the sqlite3.mk change is because CPython3.11 requires sqlite3 to be
  built with -DOMIT_SHARED_CACHE
- unistd.h has getopt.h now, because some libraries like it there
2023-07-23 11:11:08 -07:00
Justine Tunney f83eb440f7 Incorporate more small improvements 2023-07-23 10:57:18 -07:00
Justine Tunney 1d4eb08fa1 Support non-blocking i/o across platforms
This change introduces new tests for `O_NONBLOCK` and `SOCK_NONBLOCK` to
confirm that non-blocking i/o is now working on all supported platforms,
including Windows. For example, you can now say on Windows, MacOS, etc.:

    socket(AF_INET, SOCK_STREAM | SOCK_NONBLOCK, IPPROTO_TCP);

To create a non-blocking IPv4 TCP socket. Or you can enable non-blocking
i/o on an existing socket / pipe / etc. file descriptor by calling fcntl

    fcntl(fd, F_SETFL, fcntl(fd, F_GETFL) | O_NONBLOCK);

This functionality is polyfilled on older Linux kernels too, e.g. RHEL5.
Now that fcntl() support is much better the FIOCLEX / FIONCLEX polyfills
for ioctl() have been removed since they're ugly non-POSIX diameond APIs

This change fixes a weakness in kprintf() that was causing Windows trace
tools to frequently crash.
2023-07-23 02:56:47 -07:00
Justine Tunney 5c9e03e3e0 Temporarily turn down redbean StoreAsset() feature 2023-07-22 15:51:44 -07:00
Justine Tunney 18536950b3 Fix the build 2023-07-11 05:48:39 -07:00
Justine Tunney 1ee2e89326 Make improvements
- This commit mints a new release of APE Loader v1.2 which supports
  loading ELF programs with a non-contiguous virtual address layout
  even though we've never been able to take advantage of it, due to
  how `objcopy -SO binary` fills any holes left by PT_LOAD. This'll
  change soon, since we'll have a new way of creating APE binaries.

- The undiamonding trick with our ioctl() implementation is removed
  since POSIX has been killing ioctl() for years and they've done a
  much better job. One problem it resolves, is that ioctl(FIONREAD)
  wasn't working earlier and that caused issues when building Emacs
2023-07-11 04:41:50 -07:00
Justine Tunney a1b1fdd1a4 Reconfigure GitHub Actions 2023-07-10 12:17:18 -07:00
Justine Tunney a2d269dc38 Brush up some more code 2023-07-10 10:17:26 -07:00
Justine Tunney ee6566a152 Further improve usability of cosmocc
- Support -s flag in cosmocc
- Support posix_spawn() setsid() feature
- Disable monorepo debug path prefix stripping
2023-07-10 05:55:00 -07:00
Justine Tunney f7ae50462a Make improvements
- Fix unused local variable errors
- Remove yoinks from sigaction() header
- Add nox87 and aarch64 to github actions
- Fix cosmocc -fportcosmo in linking mode
- It's now possible to build `make m=llvm o/llvm/libc`
2023-07-10 04:35:14 -07:00
Justine Tunney 3dc86ce154 Fix breakages in Linux-only build modes
- compile.com now polyfills -march=native which gcc/clang removed
- Guarantee zero Windows code is linked into non-Windows binaries
- MODE=tinylinux binaries are now back to being as tiny as ~4kb
- Improve the runtime's stack allocation / alignment hack
- GitHub Actions now tests Linux modes for assurance
2023-07-09 19:51:44 -07:00
Justine Tunney 0e4c828a8e Fix alignment of ftrace hook 2023-07-09 16:52:08 -07:00
Justine Tunney e1316d6565 Fix chibicc build breakage 2023-07-09 15:00:33 -07:00
Justine Tunney d3cf9d4ef1 Use long as implicit type in chibicc
This makes it possible to write 64-bit C without headers and prototypes.
2023-07-09 10:11:06 -07:00
Justine Tunney 42ba9901e4 Fix some behavioral issues on Windows 2023-07-09 09:59:22 -07:00
Justine Tunney 41396ff48a Make fixes and improvements
- Fix handling of precision in hex float formatting
- Enhance the cocmd interpreter for system() and popen()
- Manually ran the Lua unit tests, which are now passing
- Let stdio i/o operations happen when file is in error state
- We're now saving and restoring xmm in ftrace out of paranoia
2023-07-09 05:21:11 -07:00
Justine Tunney 95fbdb4f76 Have function tracer save all registers 2023-07-08 12:04:44 -07:00
Justine Tunney 1cc8ff6a55 Fix some aarch64 build issues 2023-07-08 10:47:45 -07:00
Justine Tunney 9ae230afad Fix more build configuration errors 2023-07-08 09:08:13 -07:00
Justine Tunney a75175fe94 Make build hermetically sealed again
It turned out that Landlock Make hasn't been applying sandboxing for a
while, due to a mistyped if statement for `$(USE_SYSTEM_TOOLCHAIN)` it
should have had the opposite meaning. Regressions in the build configs
have been fixed. The rmrf() function works better now. The rm.com tool
works according to POSIX with the exception of supporting prompts.
2023-07-08 07:06:25 -07:00
Justine Tunney 0c43c98de1 Add libcxx filesystem modules 2023-07-08 02:40:17 -07:00
Justine Tunney a092fda388 Make some fixes for libcxx
This change figures out some of the build configuration issues we've
been having with libcxx. The c++ span header is added. Per a Discord
discussion we're now turning off `-g` for the default build mode, so
consider using `make MODE=dbg` or `make MODE=zero` for GDB debugging
which works much better than `MODE=` ever has. Note that the default
build mode has always had very good function call / system call logs
plus you can still use ShowCrashReports() for backtrace. Making this
change ensures cosmocc will better conform to FOSS norms. Lastly the
LoadZipArgs() API has been added to cosmopolitan.a and <cosmo.h>.
2023-07-07 19:35:58 -07:00
Justine Tunney 3a8579252d Move demangling utility to stdio
Closes #848
2023-07-07 10:47:50 -07:00
Farid Zakaria a1dd777f22 add support for __cxa_demangle
This adds support for __cxa_demangle through the cxxabi.h file.

At the moment it is the only symbol included.
The source was taken from FreeBSD
contrib/libcxxrt/libelftc_dem_gnu3.c
2176c9ab71c85efd90a6c7af4a9e04fe8e3d49ca

FreeBSD does say this is also almost verbatim from ELFToolkit
2023-07-07 10:29:03 -07:00
Justine Tunney 0d3c1c8b1a Do work on curl/mbedtls/zstd
This change fixes stderr to be unbuffered. Added hardware AES on ARM64
to help safeguard against timing attacks. The curl.com command will be
somewhat more pleasant to use.
2023-07-07 10:13:35 -07:00
Justine Tunney a186143f62 Add EncodeHex() and DecodeHex() to Redbean 2023-07-06 15:38:08 -07:00
Justine Tunney 00acd81b2f Delete more dead code 2023-07-06 09:12:28 -07:00
Justine Tunney 0a24b4fc3c Clean up more code
The *NSYNC linked list API is good enough that it deserves to be part of
the C libray, so this change writes an improved version of it which uses
that offsetof() trick from the Linux Kernel. We vendor all of the *NSYNC
tests in third_party which helped confirm the needed refactoring is safe

This change also deletes more old code that didn't pan out. My goal here
is to work towards a vision where the Cosmopolitan core libraries become
less experimental and more focused on curation. This better reflects the
current level of quality we've managed to achieve.
2023-07-06 08:03:24 -07:00
Justine Tunney 88612a2cd7 Restore original redbean build linking 2023-07-05 20:04:43 -07:00
Justine Tunney 97b7116953 Hunt down more bugs
After going through the MODE=dbg and MODE=zero build modes, a bunch of
little issues were identified, which have been addressed. Fixing those
issues created even more troubles for the project, because it improved
our ability to detect latent problems which are getting fixed so fast.
2023-07-03 18:43:29 -07:00
Justine Tunney 73c0faa1b5 Remove some dead code 2023-07-03 02:48:29 -07:00
Justine Tunney 168d1c157e Fix build breakage under special strace config 2023-07-03 00:05:51 -07:00
Justine Tunney fe044e22cc Switch public headers to getopt_long() entirely
Cosmopolitan's getopt() is now redefined as __getopt().
2023-07-02 19:57:43 -07:00
Justine Tunney 0c630d95b5 Rewrite Cosmopolitan Ar
The build/bootstrap/ar.com program is now tinier. This change reduces
its size from 140kb to 53kb. Nothing was traded away. Cosmopolitan Ar
performance is now 2x better than llvm-ar largely thanks to using the
copy_file_range() system call. This change homebrews a new allocation
API that addresses the shortcomings of the C standard library design.
Using these new balloc() and reballoc() functions I managed to reduce
memory consumption so much that Cosmpolitan Ar should now use roughly
100x fewer bytes of peak resident memory compared to llvm-ar. Correct
behavior with better compatibility has been assured. Binary output is
now pretty much bit-identical to llvm-ar, as of this change. This can
and should be the living proof we need to show that a better world is
possible for software.
2023-07-02 10:19:16 -07:00
Justine Tunney 197aa0d465 Implement swapcontext() and makecontext()
This change introduces support for Linux-style uc_context manipulation
that's fast and works well on all supported OSes and architectures. It
also integrates with the Cosmpolitan runtime which can show backtraces
comprised of multiple stacks and fibers. See the test and example code
for further details. This will be used by Mold once it's been vendored
2023-07-02 09:01:44 -07:00
Justine Tunney 7ec84655b4 Get setcontext() and getcontext() working on Aarch64
This change also adds the missing code for getting and restoring the
thread's signal mask, since that's explicitly listed by the man page
2023-07-01 22:53:23 -07:00
Farid Zakaria 8b62bff364 Introduce xxhash into third_party (#837) 2023-07-01 20:56:53 -07:00
Farid Zakaria 5f1e129222 Introduce zstd into cosmopolitan (#843) 2023-07-01 20:30:38 -07:00
Paul Kulchenko c0348330a5 Add path information in logged error messages (#832) 2023-07-01 19:47:59 -07:00
Paul Kulchenko dc8fbf38ad Update Windows IP_MULTICAST constants to use Ws2tcpip.h values (#839) (#842) 2023-07-01 19:44:27 -07:00
Justine Tunney ce0a3065f3 Update build/bootstrap/make.com
This is important, because the execve() workflow has been upgraded for
launching APE binaries. Programs that call execve() need to be remade,
otherwise they might attempt to use ~/.ape rather than ~/ape-1.1.
2023-07-01 18:37:19 -07:00
Justine Tunney 40eb3b9d5d Fully support OpenBSD 7.3
This change (1) upgrades to OpenBSD's newer kernel ABIs, and (2)
modifies APE to have a read-only data segment. Doing this required
creating APE Loader v1.1, which is backwards and forwards compatible
with the previous version.

If you've run the following commands in the past to install your APE
Loader systemwide, then you need to run them again. Ad-hoc installations
shouldn't be impacted. It's also recommended that APE binaries be remade
after upgrading, since they embed old versions of the APE Loader.

    ape/apeuninstall.sh
    ape/apeinstall.sh

This change does more than just fix OpenBSD. The new loader is smarter
and more reliable. We're now able create much tinier ELF and Mach-O data
structures than we could before. Both APE Loader and execvpe() will now
normalize ambiguous argv[0] resolution the same way as the UNIX shell.
Badness with TLS linkage has been solved.

Fixes #826
2023-07-01 18:14:27 -07:00
Justine Tunney 963e10b9bf Introduce new tinyprint() api 2023-07-01 05:09:32 -07:00
Justine Tunney 053681cb97 Fix BSD regressions
The recent change to crt.S that aggressively aligns the system-provided
stack has been rolled back on non-Linux until we can find a better way,
since it can cause a segfault early in execution on several platforms.

This change fixes a regression in tcgetattr() and tcsetattr() on OpenBSD
and NetBSD caused by 4778cd4d27.

This change has been tested across the runitd test fleet which is green.
2023-07-01 00:17:33 -07:00
Paul Kulchenko 5a77db2cd5 Fix recvfrom on Windows (closes #841) (#844) 2023-06-30 23:39:55 -07:00
Justine Tunney 48b2afb192 Address weakness with new pledge("anet") promise
The intent with pledge("anet") has been to prevent outbound connections.
However we were only doing that for TCP sockets, and outbound UDP could
still get through, by using socket() plus sendto(). This change fixed
that by preventing UDP sockets from being created.

Credit goes to chc4 on Hacker News for finding this.
2023-06-18 18:06:47 -07:00
Justine Tunney fb2bd313ae Fix redbean unix.poll() with -1 timeout 2023-06-18 18:05:26 -07:00
Justine Tunney f10845ab9f Add blackhole support to Python tokenbucket module 2023-06-18 07:50:11 -07:00
Justine Tunney 461c835784 Remove removed GCC symlinks 2023-06-18 05:41:55 -07:00
Justine Tunney 226375933a Implement more toolchain fixes 2023-06-18 05:39:31 -07:00
Justine Tunney 0409096658 Get us closer to building busybox
This change undefines __linux__ and adds APIs like clock_settime(). The
gosh darned getopt_long() API has been reintroduced, thanks to OpenBSD.
2023-06-18 04:13:45 -07:00
Justine Tunney 5ac522f0de Add make clean build rule 2023-06-18 01:23:54 -07:00
Justine Tunney 8514933b60 Delete itoa() prototype 2023-06-18 01:17:15 -07:00
Justine Tunney d7c79f43ef Clean up more code
- Found some bugs in LLVM compiler-rt library
- The useless LIBC_STUBS package is now deleted
- Improve the overflow checking story even further
- Get chibicc tests working in MODE=dbg mode again
- The libc/isystem/ headers now have correctly named guards
2023-06-18 01:00:05 -07:00
Justine Tunney afc58a8b41 Upgrade cosmopolitan gcc and binutils to 2.35.2
This change upgrades to the latest portcosmo gcc patch
6728fe1a25185560603ca312a8d4352af2a4e515 which lets us avoid needing to
define __tmpcosmo_FOO constants. We're now using an appropriate binutils
version for GCC 11. The older binutils sometimes wasn't able to print
backtraces, due to not being able to find a .debug_ranges section.

This is breaking change for /opt/cosmos libraries :'( due to this weird
"error: need linked-to section for --gc-sections" that pops up.

Please run `make clean` in the monorepo before rebuilding.
2023-06-18 00:54:14 -07:00
Justine Tunney 2c0f0bab0b Fix mkfifo() on Linux 2023-06-17 16:04:08 -07:00
Justine Tunney 12452976bd Remove getopt_long() from headers
We removed this API a long time ago and it was breaking the GNU make
open source build.
2023-06-17 16:03:16 -07:00
Justine Tunney b881c0ec9e Remove printf() linking hack 2023-06-17 10:13:50 -07:00
Justine Tunney ba03cd95c5 Modernize the README file 2023-06-17 07:55:48 -07:00
Justine Tunney 562a1384cd Make blink support conditionally linkable into APE 2023-06-17 07:55:35 -07:00
Justine Tunney 52d28966f7 Remove old zip base skew hack
Since 8ff48201ca we no longer need the
hack where, when running .com.dbg files, we scanned for the embedded
.com file offset, and then computed zip offsets realtive to that. It
wasn't very reliable in the first place, and was causing issues with
running our new .com.dbg executables, which are true zip files.
2023-06-17 04:20:16 -07:00
Justine Tunney 207e18a060 Unbreak the x86-64 build 2023-06-16 20:05:24 -07:00
Justine Tunney 7bc20bff3e Resolve argv[0] based on path search in ape-m1 2023-06-16 16:58:30 -07:00
Justine Tunney 1353db7d3f Fix rwx .rodata in python.com on aarch64 2023-06-16 16:25:04 -07:00
Justine Tunney 4eebd6b9dc Improve new C23 checked arithmetic feature 2023-06-16 15:32:18 -07:00
Justine Tunney 2a1c588826 Add tokenbucket module to python.com 2023-06-16 15:32:09 -07:00
Justine Tunney e6b7c16a53 Make changes needed for new demo 2023-06-15 23:22:49 -07:00
Justine Tunney c3440d040c Make improvements
- More timspec_*() and timeval_*() APIs have been introduced.
- The copyfd() function is now simplified thanks to POSIX rules.
- More Cosmo-specific APIs have been moved behind the COSMO define.
- The setitimer() polyfill for Windows NT is now much higher quality.
- Fixed build error for MODE=aarch64 due to -mstringop-strategy=loop.
- This change introduces `make MODE=nox87 toolchain` which makes it
  possible to build programs using your cosmocc toolchain that don't
  have legacy fpu instructions. This is useful, for example, if you
  want to have a ~22kb tinier blink virtual machine.
2023-06-15 14:50:53 -07:00
Justine Tunney 8dc11afcf6 Upgrade portcosmo patch and GCC third party docs 2023-06-14 19:53:06 -07:00
Justine Tunney 4778cd4d27 Fix bugs in termios library and cleanup code
This change fixes an issue with the tcflow() magic numbers that was
causing bash to freeze up on Linux. While auditing termios polyfills,
several other issues were identified with XNU/BSD compatibility.

Out of an abundance of caution this change undefines as much surface
area from libc/calls/struct/termios.h as possible, so that autoconf
scripts are less likely to detect non-POSIX teletypewriter APIs that
haven't been polyfilled by Cosmopolitan.

This is a *breaking change* for your static archives in /opt/cosmos if
you use the cosmocc toolchain. That's because this change disables the
ioctl() undiamonding trick for code outside the monorepo, specifically
because it'll lead to brittle ABI breakages like this. If you're using
the cosmocc toolchain, you'll need to rebuild libraries like ncurses,
readline, etc. Yes diamonds cause bloat. To work around that, consider
using tcgetwinsize() instead of ioctl(TIOCGWINSZ) since it'll help you
avoid pulling every single ioctl-related polyfill into the linkage.

The cosmocc script was specifying -DNDEBUG for some reason. It's fixed.
2023-06-14 19:30:52 -07:00
Justine Tunney 06b749ae03 Remove blinkenlights
New home: https://github.com/jart/blink
2023-06-14 19:30:52 -07:00
Gabriel Ravier 50d8d953ce Fix syscall(2) returning -errno instead of using POSIX errno scheme (#830)
As of now, the syscall function is implemented as alike to how the
linux kernel sycall ABI works, returning -errno upon errors without
setting the value of errno. However, this does not conform to the
expectations of most software, which expect it to return -1 and set
errno, which is how it works on other libcs, which document it
accordingly:

> The return value is defined by the system call being invoked.  In
> general, a 0 return value indicates success.  A -1 return value
> indicates an error, and an error number is stored in errno.
- Linux man-pages, syscall(2)

> The return value is the return value from the system call, unless
> the system call failed.  In that case, ‘syscall’ returns ‘-1’ and
> sets ‘errno’ to an error code that the system call returned.
- glibc manual, (libc)System Calls

> When the C-bit is set, syscall returns -1 and sets the external
> variable errno (see intro(2)).
- 4BSD manual, syscall(2)

> A -1 return value indicates an error, and an error code is stored in
> errno.
- 4.4BSD, FreeBSD, OpenBSD and NetBSD manuals (same quote is found in
  all of them), syscall(2)

This patch corrects the syscall function to work in the same way as in
other libcs.
2023-06-11 10:33:28 -07:00
tkchiaandtkchia e47c0cc929 [metal] Clean up code for getting pointer to struct mman (#828)
Co-authored-by: tkchia <[email protected]>
2023-06-11 10:32:39 -07:00
Justine Tunney 5a6a4819d4 Fix -iquote in cosmocc toolchain 2023-06-10 16:35:53 -07:00
Justine Tunney 2676ec55de Make improvements to cosmocc toolchain 2023-06-10 15:50:01 -07:00
Justine Tunney 8ff48201ca Rewrite .zip.o file linker
This change takes an entirely new approach to the incremental linking of
pkzip executables. The assets created by zipobj.com are now treated like
debug data. After a .com.dbg is compiled, fixupobj.com should be run, so
it can apply fixups to the offsets and move the zip directory to the end
of the file. Since debug data doesn't get objcopy'd, a new tool has been
introduced called zipcopy.com which should be run after objcopy whenever
a .com file is created. This is all automated by the `cosmocc` toolchain
which is rapidly becoming the new recommended approach.

This change also introduces the new C23 checked arithmetic macros.
2023-06-10 09:29:44 -07:00
Justine Tunney f6407d5f7c Fix missing includes 2023-06-09 18:11:53 -07:00
Justine Tunney 4a59210008 Introduce #include <cosmo.h> to toolchain users
This change improves the way internal APIs are being hidden behind the
`COSMO` define. The cosmo.h header will take care of defining that, so
that a separate define statement isn't needed. This change also does a
lot more to define which APIs are standard, and which belong to Cosmo.
2023-06-09 18:03:05 -07:00
Justine Tunney 9b55dbe417 Get GCC to mostly build with Cosmo 2023-06-09 06:41:34 -07:00
Justine Tunney 2eb7148474 Create a fake syscall() function due to Python 2023-06-09 05:18:38 -07:00
Justine Tunney 4b2023ffab Disable linker map generation and improve tinyness 2023-06-09 03:29:26 -07:00
Justine Tunney 23e235b7a5 Fix bugs in cosmocc toolchain
This change integrates e58abc1110b335a3341e8ad5821ad8e3880d9bb2 from
https://github.com/ahgamut/musl-cross-make/ which fixes the issues we
were having with our C language extension for symbolic constants. This
change also performs some code cleanup and bug fixes to getaddrinfo().
It's now possible to compile projects like ncurses, readline and python
without needing to patch anything upstream, except maybe a line or two.
Pretty soon it should be possible to build a Linux distro on Cosmo.
2023-06-08 23:44:03 -07:00
Justine Tunney 22f81a8d50 Improve cosmocc / cosmoc++ toolchain scripts
- Get out of the red zone
- Generate --ftrace nops unless -Os is passed
- Intercept -o path to generate .com / .com.dbg appropriately
2023-06-08 14:29:22 -07:00
Justine Tunney 25678db2a0 Fix remaining regressions in ASAN support
See #829
2023-06-08 08:00:50 -07:00
Justine Tunney 4d629fd424 Fix stack abuse in llama.cc
This change also incorporates improvements for MODE=asan. It's been
confirmed that o/asan/third_party/ggml/llama.com will work.

Fixes #829
2023-06-08 07:12:26 -07:00
Justine Tunney 32682f0ce7 Remove some problematic APIs
In order to improve our chances of success building other open source
projects we shouldn't define APIs that'll lead any ./configure script
astray. For example:

- brk() and sbrk() can break mac/windows support
- syscall() is a superb way to break portability
- arch_prctl() is the greatest of all horror shows
2023-06-08 06:12:26 -07:00
Justine Tunney 7512318a2a Fix MODE=aarch64 build 2023-06-08 05:17:37 -07:00
Justine Tunney 8767e9ad6a Fix --ftrace regression 2023-06-08 04:46:11 -07:00
Justine Tunney daf4454a06 Validate privileged code relationships
- Work towards improving non-optimized build support
- Introduce MODE=zero which is -O0 without ASAN/UBSAN
- Use system GCC when ~/.cosmo.mk has USE_SYSTEM_TOOLCHAIN=1
- Have package.com check .privileged code doesn't call non-privileged
2023-06-08 04:38:06 -07:00
Justine Tunney 01fd655097 Get garbage collector working on aarch64
Garbage collection will now happen on arm64 when a function returns,
rather than kicking the can down the road to when the process exits.
This change also does some code cleanup and incorporates suggestions
2023-06-07 03:34:45 -07:00
Justine Tunney 9793d3524f Remove superfluous RET instructions 2023-06-06 19:46:56 -07:00
Justine Tunney b6182db813 Simplify ftrace_hook()
We now have a test to prove that its transitive closure doesn't perform
floating point computations.
2023-06-06 11:10:38 -07:00
Justine Tunney 61b9677c05 Make improvements
- Get mprotect_test working on aarch64
- Get completion working on python.com repl again
- Improve quality of printvideo.com and printimage.com
- Fix bug in openpty() so examples/script.c works again
2023-06-06 09:12:30 -07:00
Justine Tunney b94b29d79c Prevent ftrace from misaligning functions 2023-06-06 06:00:31 -07:00
Justine Tunney b8a6a989c0 Create ELF aliases for identical symbols
This change greatly reduces the number of modules that need to be
compiled. The only issue right now is that sometimes when viewing
symbol table entries, the aliased symbol is chosen.
2023-06-06 03:33:49 -07:00
Justine Tunney e1b83399bd Fix memmove() alias on aarch64 with --ftrace 2023-06-06 00:39:55 -07:00
Justine Tunney e9e8bbe6da Avoid /etc/services in whois command
Apparently IANA has abolished the WHOIS protocol and no longer lists it
as a service. Therefore distros which naively create /etc/services from
IANA's braindead recommendation will inadvertently break any tools that
rely on /etc/services to determine this well-known Internet port.
2023-06-06 00:11:41 -07:00
Justine Tunney eb40cb371d Get --ftrace working on aarch64
This change implements a new approach to function call logging, that's
based on the GCC flag: -fpatchable-function-entry. Read the commentary
in build/config.mk to learn how it works.
2023-06-05 23:35:31 -07:00
Justine Tunney 5b908bc756 Fix some build errors 2023-06-05 15:53:44 -07:00
Justine Tunney 7558549d44 Test m=aarch64 on GitHub Actions 2023-06-05 15:23:00 -07:00
Justine Tunney 42b9b75749 Fix aarch64 build 2023-06-05 05:57:44 -07:00
Justine Tunney f554dd800e Make it possible to log kprintf() to file
It's now possible to compile Emacs using cosmocc. However we need to
troubleshoot why it's event loop isn't working correctly at runtime.
2023-06-05 04:16:15 -07:00
Justine Tunney 9cc3e37263 Upgrade to Cosmopolitan GCC 11.2.0 for aarch64 2023-06-05 02:07:28 -07:00
Justine Tunney 39f20dbb13 Upgrade to Cosmopolitan GCC 11.2.0 for x86_64 2023-06-05 02:06:18 -07:00
Justine Tunney 682b74ed88 Incorporate Blink 1.0.0 release 2023-06-04 11:07:31 -07:00
Justine Tunney 35dcaca53c Fix test fleet errors 2023-06-04 10:57:11 -07:00
Justine Tunney a91a945b88 Fix build breakage 2023-06-04 09:33:48 -07:00
Justine Tunney 4aa1d09b9e Improve aarch64 native support some more
This change introduces partial support for automating remote testing of
aarch64 binaries on Raspberry Pi and Apple Silicon.
2023-06-04 08:58:47 -07:00
Justine Tunney fc34ba2596 Fix Linenoise REPL on AARCH64 2023-06-04 02:57:17 -07:00
Justine Tunney bcf9af94bf Get threads working well on MacOS Arm64
- Now using 10x better GCD semaphores
- We now generate Linux-like thread ids
- We now use fast system clock / sleep libraries
- The APE M1 loader now generates Linux-like stacks
2023-06-04 01:57:10 -07:00
Justine Tunney b5eab2b0b7 Get POSIX threads working on Apple Silicon
It's now possible to run a working

    ape-m1 o/aarch64/third_party/ggml/llama.com

on Apple M1 hardware running XNU!
2023-06-03 18:33:01 -07:00
Justine Tunney 8fdb31681a Introduce support for GGJT v3 file format
llama.com can now load weights that use the new file format which was
introduced a few weeks ago. Note that, unlike llama.cpp, we will keep
support for old file formats in our tool so you don't need to convert
your weights when the upstream project makes breaking changes. Please
note that using ggjt v3 does make avx2 inference go 5% faster for me.
2023-06-03 15:46:21 -07:00
Justine Tunney 6ae18a10ba Fix MODE=tiny build 2023-06-03 10:30:48 -07:00
Justine Tunney 1904a3cae8 Sync llama.cpp to 6986c7835adc13ba3f9d933b95671bb1f3984dc6 2023-06-03 10:29:12 -07:00
Justine Tunney 8f522cb702 Make improvements
This change progresses our AARCH64 support:

- The AARCH64 build and tests are now passing
- Add 128-bit floating-point support to printf()
- Fix clone() so it initializes cosmo's x28 TLS register
- Fix TLS memory layout issue with aarch64 _Alignas vars
- Revamp microbenchmarking tools so they work on aarch64
- Make some subtle improvements to aarch64 crash reporting
- Make kisdangerous() memory checks more accurate on aarch64
- Remove sys_open() since it's not available on Linux AARCH64

This change makes general improvements to Cosmo and Redbean:

- Introduce GetHostIsa() function in Redbean
- You can now feature check using pledge(0, 0)
- You can now feature check using unveil("",0)
- Refactor some more x86-specific asm comments
- Refactor and write docs for some libm functions
- Make the mmap() API behave more similar to Linux
- Fix WIFSIGNALED() which wrongly returned true for zero
- Rename some obscure cosmo keywords from noFOO to dontFOO
2023-06-03 08:12:22 -07:00
Paul Kulchenko 5655c9a4e7 Extend Pledge with anet (same as inet, but with no connect) (#827)
* Add `anet` pledge for `inet` without connect

This is useful for configurations where it's desirable to start redbean
under these restrictions, but not to allow `connect` socket calls.

* Update message on protected/unpledged syscalls for clarity

* Update redbean to add reporting for unpledged sigaction

Previously it would abort without indicating what signal it failed to
install when sigaction is not pledged (although it fails all of them).

* Move GetHostIps before processing command line options

This allows using unix.pledge as part of the options without affecting
retrieving host IP addresses (which requires `connect`). It may still
fail under external `pledge` command as expected; in this case IPs
would need to be passed manually.

* Update tests for pledge anet promise
2023-06-03 07:50:29 -07:00
Justine Tunney 72f8bd10b7 Perform some code cleanup 2023-05-28 19:42:00 -07:00
Connor 992a4638ae Fix rgbtoxterm tool to output the proper colors (#819) 2023-05-26 18:30:24 -07:00
tkchiaandtkchia 90645f4dac [metal] Fix some issues that were breaking UEFI boot (#822)
Co-authored-by: tkchia <[email protected]>
2023-05-26 18:29:25 -07:00
Paul Kulchenko 5c40800d69 Update redbean ProgramDirectory to update package.path (#823) 2023-05-26 18:29:00 -07:00
Justine Tunney 1422e96b4e Introduce native support for MacOS ARM64
There's a new program named ape/ape-m1.c which will be used to build an
embeddable binary that can load ape and elf executables. The support is
mostly working so far, but still chasing down ABI issues.
2023-05-20 04:17:03 -07:00
Justine Tunney b852650c08 Fiddle around with Mach-O 2023-05-20 04:13:49 -07:00
Paul Kulchenko 6881a2ecea Extend redbean Fetch to add option to keeping connection open (#818) 2023-05-17 20:49:26 -07:00
Paul Kulchenko 021c71b651 Fix redbean to avoid compressing the first yielded chunk. (#813)
This may happen if no Status or Content-Type is set and if the written
content is longer than 100B, the compression is going to be applied
after coroutine.yield, even though it shouldn't be.
2023-05-17 20:46:27 -07:00
Justine Tunney e7eb0b3070 Make more ML improvements
- Fix UX issues with llama.com
- Do housekeeping on libm code
- Add more vectorization to GGML
- Get GGJT quantizer programs working well
- Have the quantizer keep the output layer as f16c
- Prefetching improves performance 15% if you use fewer threads
2023-05-16 08:07:23 -07:00
Justine Tunney 80db9de173 Make the intrinsics more readable 2023-05-15 23:12:11 -07:00
Justine Tunney 210187cf77 Perform some code cleanup 2023-05-15 16:32:10 -07:00
Justine Tunney cc1732bc42 Make AARCH64 harder, better, faster, stronger
- Perform some housekeeping on scalar math function code
- Import ARM's Optimized Routines for SIMD string processing
- Upgrade to latest Chromium zlib and enable more SIMD optimizations
2023-05-15 02:15:34 -07:00
Justine Tunney 550b52abf6 Port a lot more code to AARCH64
- Introduce epoll_pwait()
- Rewrite -ftrapv and ffs() libraries in C code
- Use more FreeBSD code in math function library
- Get significantly more tests passing on qemu-aarch64
- Fix many Musl long double functions that were broken on AARCH64
2023-05-14 09:37:26 -07:00
Ariel Núñez 91791e9f38 Started removing features from RedPajama to make it easier to understand for beginners (#817) 2023-05-14 09:16:22 -07:00
Justine Tunney 89d1fad7ee Enable crash reports for radpajama executables 2023-05-13 21:16:03 -07:00
Justine Tunney 296ee3ec58 Make some other fixes to radpajama build config 2023-05-13 21:09:28 -07:00
Justine Tunney 282dd8e7b7 Get radpajama to build
make -j8 o//third_party/radpajama/radpajama.com
    make -j8 o//third_party/radpajama/radpajama-chat.com

This change gets the radpajama.mk config working. This package depends
on THIRD_PARTY_GGML but it's configured to call ggjt_v1(), so that the
library will provide the old quantizers. The ggml_quantize_chunk() API
will now dispatch to older quantizers based on the configured version.
2023-05-13 20:44:36 -07:00
Justine Tunney 410c8785c9 Fix the AARCH64 build 2023-05-13 08:19:44 -07:00
Justine Tunney 5a4cf9560f Add support for new GGJT v2 quantizers
This change makes quantized models (e.g. q4_0) go 10% faster on Macs
however doesn't offer much improvement for Intel PC hardware.

This change syncs llama.cpp 699b1ad7fe6f7b9e41d3cb41e61a8cc3ea5fc6b5
which recently made a breaking change to nearly all its file formats
without any migration. Since that'll break hundreds upon hundreds of
models on websites like HuggingFace llama.com will support both file
formats because llama.com will never ever break the GGJT file format
2023-05-13 08:08:32 -07:00
Justine Tunney ba49e86e20 Get TEST_LIBC_CALLS passing on AARCH64 2023-05-13 02:41:41 -07:00
Justine Tunney 802e7eb4ef Mop up more test regressions 2023-05-13 01:09:44 -07:00
Justine Tunney 4a8a81eb9f Fix llama.com interactive mode regressions 2023-05-13 00:09:38 -07:00
Justine Tunney fd34ef732d Make considerably more progress on AARCH64
- Utilities like pledge.com now build
- kprintf() will no longer balk at 48-bit addresses
- There's a new aarch64-dbg build mode that should work
- gc() and defer() are mostly pacified; avoid using them on aarch64
- THIRD_PART_STB now has Arm Neon intrinsics for fast image handling
2023-05-12 22:42:57 -07:00
Justine Tunney 1bfb3aab1b Make Arm Neon intrinsics work with make tags 2023-05-12 18:32:53 -07:00
Justine Tunney 414667b1c9 Get TEST_LIBC_STR passing on AARCH64
It's now possible to run commands like:

    make -j8 m=aarch64 o/aarch64/test/libc/str

Which will cross-compile and run the test suites in a qemu-aarch64
binary that's vendored in the third_party/qemu/ folder within your
x86_64 build environment.
2023-05-12 18:09:23 -07:00
Justine Tunney 45186c74ac Introduce -q (quiet flag) and improve ctrl-c ux 2023-05-12 09:46:07 -07:00
Justine Tunney e8de1e4766 Fix subtoken antiprompt scanning 2023-05-12 08:55:40 -07:00
Justine Tunney 80c174d494 Clean up llama.com anti/stop/reverse-prompt code
Example use case for JSON completion:

    $ m=opt
    $ make -j16 m=$m o/$m/third_party/ggml/llama.com
    $ o/$m/third_party/ggml/llama.com -m llama.bin -p '{"key": "life", "val": ' -r '}'
    42}

This provides better control. More sophisticated facilities for
controlling text generation will be provided soon enough.
2023-05-12 08:20:58 -07:00
Justine Tunney bbfe4fbd11 Make llama.com n_predict be -1 by default 2023-05-12 08:20:34 -07:00
Justine Tunney ca19ecf49c Fine tune crash reports for llama.com 2023-05-12 06:24:26 -07:00
Justine Tunney 1f2a5a8fc1 Implement crash reporting for AARCH64
The ShowCrashReports() feature for aarch64 should work even better than
the x86 crash reports. Thanks to the benefit of hindsight these reports
should be rock solid reliable and beautiful to read.

This change also improves the syscall polyfills for aarch64. Some of the
sys_foo() functions have been removed, usually because they're legacy or
downright footguns not worth building.
2023-05-12 05:47:54 -07:00
Justine Tunney 285e8a2348 Get backtraces working on AARCH64 2023-05-11 22:12:41 -07:00
Justine Tunney 4edbc98811 Get MbedTLS and its unit tests passing AARCH64 2023-05-11 21:53:15 -07:00
Justine Tunney 5e2f7f7ced Get LIBC_TESTLIB building on AARCH64 2023-05-11 19:57:09 -07:00
Justine Tunney 95fab334e4 Use yield on aarch in spin locks 2023-05-11 19:57:09 -07:00
Ariel Núñez b3e3359d22 Import radpajama (a redpajama.cpp fork) (#814)
This is the relevant commit: https://github.com/togethercomputer/redpajama.cpp/commit/bfa6466199b8ef92185ecb72e2a550e12baf6602

Model download links:
https://huggingface.co/ceonlabs/radpajama/tree/main
2023-05-11 07:12:08 -07:00
Justine Tunney cba15bc77a Fix regression in DNS client
Calling FreeResolvConf() would crash, due to a recent change from
yesterday where free_s() was removed.
2023-05-10 23:27:10 -07:00
Justine Tunney 8d24a18c62 Tune magic division for size
This helps it avoid generating a branch instruction.
2023-05-10 21:32:04 -07:00
Justine Tunney 1f6f9e6701 Remove division from matrix multiplication
This change reduces llama.com CPU cycles systemically by 2.5% according
to the Linux Kernel `perf stat -Bddd` utility.
2023-05-10 21:19:54 -07:00
Justine Tunney a88290e595 Make sure llama.com terminal cleanup happens 2023-05-10 15:56:01 -07:00
Justine Tunney 5250feb7ad There must only be one strerror() 2023-05-10 15:34:13 -07:00
Justine Tunney c91de6a356 Avoid running clang-format on ggml.c 2023-05-10 14:59:33 -07:00
Justine Tunney ea607781f8 Fix the build 2023-05-10 10:14:15 -07:00
Justine Tunney bb3ebedfce Fix load time measurement 2023-05-10 07:54:21 -07:00
Justine Tunney 290a49952e Fix some more issues with aarch64 and llama.cpp 2023-05-10 07:34:26 -07:00
Justine Tunney 64aca4dc4f Fix weird behavior issue w/ kprintf() on aarch64 2023-05-10 06:17:33 -07:00
Justine Tunney 4e81d3277c Do some basic housekeeping in LIBC_STR 2023-05-10 06:17:20 -07:00
Justine Tunney 12a33858c9 There must be only one clock() 2023-05-10 06:16:01 -07:00
Justine Tunney 6cb9553706 Fix alignment bug in llama.com 2023-05-10 06:15:32 -07:00
Justine Tunney ca990ef091 Make llama.com -h print to stdout 2023-05-10 04:55:59 -07:00
Justine Tunney f312f706f4 Bring MODE=tiny binary sizes down to 20kb minimum
aarch64 binaries start at 4kb.
2023-05-10 04:20:48 -07:00
Justine Tunney 5f57fc1f59 Upgrade llama.cpp to e6a46b0ed1884c77267dc70693183e3b7164e0e0 2023-05-10 04:20:48 -07:00
Justine Tunney 5a455eaa0b Work on magic numbers for aarch64 2023-05-10 04:20:48 -07:00
Justine Tunney 59766efd3e Do some more aarch64 fixups 2023-05-10 04:20:47 -07:00
Justine Tunney 86d9323a43 Remove sys_getrandom() on NetBSD
This fixes an apparent regression caused by
3f0bcdc3ef where getrandom() on NetBSD 9.2
doesn't appear to work; ktrace oddly reports:

    1446      1 .ape     CALL  #91 (unimplemented getdopt)
    1446      1 .ape     RET   #91 (unimplemented getdopt) -1 errno 78
    Function not implemented
    1446      1 .ape     PSIG  SIGSYS SIG_DFL: code=SI_NOINFO
2023-05-10 04:20:47 -07:00
Justine Tunney a0237a017c Get llama.com working on aarch64 2023-05-10 04:20:47 -07:00
Justine Tunney 4c093155a3 Get llama.com building as an aarch64 native binary 2023-05-10 04:20:47 -07:00
Justine Tunney d04430f4ef Get LIBC_MEM and LIBC_STDIO building with aarch64 2023-05-10 04:20:47 -07:00
Justine Tunney ae0ee59614 Get aarch64 hello world working
$ m=aarch64-tiny
    $ make -j8 m=$m o/$m/tool/hello/hello.com o/third_party/qemu/qemu-aarch64
    $ o/third_party/qemu/qemu-aarch64 o/$m/tool/hello/hello.com
    hello world
    $ ls -hal o/$m/tool/hello/hello.com
    -rwxr-xr-x 1 jart jart 4.0K May  9 05:04 o/aarch64-tiny/tool/hello/hello.com
2023-05-10 04:20:47 -07:00
Justine Tunney e5e3cdf447 Get LIBC_RUNTIME and LIBC_CALLS building on aarch64 2023-05-10 04:20:47 -07:00
Justine Tunney 7e46645193 Get LIBC_FMT to build for aarch64 2023-05-10 04:20:47 -07:00
Justine Tunney 036b9a0002 Make further progress on non-x86 support 2023-05-10 04:20:47 -07:00
Justine Tunney aef9a69a60 Make more progress on aarch64 2023-05-10 04:20:47 -07:00
Justine Tunney 135080fd3e Get libc/tinymath/ compiling on aarch64 2023-05-10 04:20:46 -07:00
Justine Tunney 2b73e72d59 Make more code aarch64 friendly 2023-05-10 04:20:46 -07:00
Justine Tunney ca2860947f Make progress towards aarch64 build 2023-05-10 04:20:46 -07:00
Justine Tunney 08ff26c817 Add qemu-aarch64 2023-05-10 04:20:46 -07:00
Justine Tunney 8303e23b3a Do some basic build tuning 2023-05-10 04:20:46 -07:00
Justine Tunney 57cc257f58 Vendor musl-cross-make gcc 9.2.0 aarch64 2023-05-10 04:20:46 -07:00
Paul Kulchenko a493b64233 Merge pull request #807 from jart/evalarg
Fix the behavior of `redbean -i -e CODE`
2023-05-02 07:48:43 -07:00
Justine Tunney b008950d90 Add libcxx to make toolchain 2023-05-01 14:12:46 -07:00
Justine Tunney 12438cce16 Fix regression with Python linker eaxmples
We can once again create 2mb statically-linked Python binaries:

    $ make -j8 m=tiny o/tiny/examples/pyapp/pyapp.com
    $ ls -hal o/tiny/examples/pyapp/pyapp.com
    -rwxr-xr-x 1 jart jart 2.1M May  1 14:04 o/tiny/examples/pyapp/pyapp.com
    $ o/tiny/examples/pyapp/pyapp.com
    cosmopolitan is cool!

The regression was caused by Python thread support in b15f9eb58
2023-05-01 14:12:15 -07:00
Justine Tunney fc82f77a46 Fix bug in cosmocc / cosmoc++ scripts
The necessary flags weren't being passed when compiling programs as a
single step (i.e. not using `-c` to make intermediate objects).
2023-05-01 13:46:15 -07:00
Justine Tunney 3dac9f8999 Use Companion AI in llama.com by default 2023-04-30 23:08:15 -07:00
Justine Tunney d9e27203d4 Incorporate some fixes and updates for GGML 2023-04-28 20:24:55 -07:00
Justine Tunney b31ba86ace Introduce prompt caching so prompts load instantly
This change also introduces an ephemeral status line in non-verbose mode
to display a load percentage status when slow operations are happening.
2023-04-28 16:15:26 -07:00
Justine Tunney bf6459e324 Fix bugs with cli flags in gzip.com 2023-04-28 05:15:58 -07:00
Justine Tunney 1c2da3a55a Make shell usability improvements to llama.cpp
- Introduce -v and --verbose flags
- Don't print stats / diagnostics unless -v is passed
- Reduce --top_p default from 0.95 to 0.70
- Change --reverse-prompt to no longer imply --interactive
- Permit --reverse-prompt specifying custom EOS if non-interactive
2023-04-28 02:54:11 -07:00
Justine Tunney 420f889ac3 Further optimize the math library
The sincosf() function is now twice as fast, thanks to ARM Limited. The
same might also be true of logf() and expm1f() which have been updated.
2023-04-28 01:20:47 -07:00
Justine Tunney e8b43903b2 Import llama.cpp
https://github.com/ggerganov/llama.cpp
0b2da20538d01926b77ea237dd1c930c4d20b686
See third_party/ggml/README.cosmo for changes
2023-04-27 14:37:14 -07:00
Justine Tunney f42089d5c6 Be more lenient about stack memory
The new stack size is 256kb in order to compromise with llama.cpp's
aggressive use of stack memory, which can't be easily patched. This
change disables the dynamic alloca() and VLA warnings for now, plus
frame sizes for individual functions may be <=50% of the stack size

This only applies to code in the cosmo monorepo. Open source builds
should already be using an 8mb stack by default, like everyone else
2023-04-27 10:45:47 -07:00
Justine Tunney fec139f423 Add mlock() and munlock() 2023-04-27 10:42:52 -07:00
Justine Tunney 7c9ef924bf Add more sorting algorithms 2023-04-27 05:44:44 -07:00
Justine Tunney b7bf052a4b Add Intel intrinsics headers 2023-04-27 05:44:38 -07:00
Justine Tunney 369f9740de Run clang-format on most sources 2023-04-27 05:44:32 -07:00
Justine Tunney 497d4fa6d1 Fix the behavior of redbean -i -e CODE
When redbean is functioning as a Lua interpreter, the `-e` flag should
behave the same way as other open source language interpreters. Namely
it should exit after evaluating the code rather than showing the REPL.
2023-04-25 01:52:13 -07:00
Justine Tunney 614229e3f4 Make some sorting functions faster 2023-04-24 23:11:18 -07:00
Gautham 1a839ba41d add os.geteuid to python (#806) 2023-04-24 20:15:24 -07:00
Justine Tunney 4a6be2154d Fix hermeticity issue with new test 2023-04-18 06:37:01 -07:00
Justine Tunney 3e702e8f65 Push new rolling release 2023-04-18 05:11:15 -07:00
Justine Tunney a8ff220654 Fix some minor issues 2023-04-18 05:02:45 -07:00
Gavin Hayes d484e1dbd4 Improve system() / popen() builtin shell (#753)
* cocmd: stop splitting redirect expressions
* cocmd: allow passing unpaired } for passing Perl test
* cocmd: propagate exit status from semicolon
2023-04-17 16:18:16 -07:00
Gabriel Ravier d5b8b644c2 unveil: Added truncate support on Linux 6.2+ (#803)
Right now, cosmopolitan uses Linux Landlock ABI version 2 on Linux,
meaning that the polyfill for unveil() cannot restrict operations such
as truncate() (a limitation of Landlock's ABI from then). This means
that to restrict truncation operations Cosmopolitan instead has to ban
the syscall through a SECCOMP BPF filter, meaning that completely
legitimate truncate() calls are blocked

However, the newest version of the Landlock ABI (version 3) introduced
in Linux 6.2, released in February 2023, implements support for controlling truncation
operations. As such, the previous SECCOMP BPF truncate() filtering is
no longer needed when the new ABI is available

This patch implements unveil truncate support for Linux Landlock ABI
version 3
2023-04-17 16:17:02 -07:00
Gabriel Ravier f7bccf5513 Fix pledge rpath support for old getdents syscall (#804)
The rpath pledge as currently implemented in cosmopolitan does not
allow for usage of the old getdents syscall (0x4e), which is different
from the newer getdents syscall (0xd9) solely in that it does not
support 64-bit filesystems.

This means that, for example, old statically linked binaries cannot
use `readdir` and other such functions which use this syscall instead
of the more modern one, even though there is no threat in allowing
that syscall alongside the more modern one (except that the binary may
have issues with 64-bit filesystems, but that's a separate problem).

This patch fixes this.
2023-04-17 16:15:01 -07:00
Gavin Hayes 6758c3f400 commandv: fix dir tests on non-NT (#764) 2023-04-15 06:26:13 -07:00
Gabriel Ravier 9a5d69c842 Fix scanf x specifier with string of 0 (#793)
The C standard states that, in the context of an x conversion
specifier given to scanf:
> Matches an optionally signed hexadecimal integer, whose format is
> the same as expected for the subject sequence of the strtoul
> function with the value 16 for the base argument.
- C standard, 7.23.6.2.11. The fscanf function

Cosmopolitan fails to do this, as 0 should be parsed as a 0 by such an
invocation of strtoul. Instead, cosmopolitan errors out as though such
input is invalid, which is wrong.

This means that a program such as this:

 #include <stdio.h>
 #undef NDEBUG
 #include <assert.h>

int main()
{
    int v = 0;
    assert(sscanf("0", "%x", &v) == 1);
}

will not run correctly on cosmpolitan, instead failing the assertion.

This patch fixes this, along with the associated GitHub issue,
https://github.com/jart/cosmopolitan/issues/778
2023-04-15 06:25:35 -07:00
Gabriel Ravier 12e07798df Fix printf precision/field width being limited by internal buffer size (#799)
The C standard, when defining field width and precision, never gives
any limit on the values used for them (except, I believe, that they
fit within an int). In other words, if the user gives a field width of
32145 and a precision of 9218, the implementation has to handle these
values correctly. However, when such kinds of high numbers are used
with integer conversions, cosmopolitan is limited by an internal
buffer size of 144, which means precisions and field widths have to
fit within this, which violates the standard.

This means that for example, the following program:

 #include <stdio.h>
 #include <string.h>

int main()
{
    char buf2[512] = {};

    int i = snprintf(buf2, sizeof(buf2), "%.9999u", 10);
    printf("%d %zu\n", i, strlen(buf2));
}

would, instead of printing "9999 511" (the correct output), instead
print "144 144" under cosmopolitan.

This patch fixes this.
2023-04-04 14:16:34 -04:00
Gabriel Ravier 9c6d78c26d Fix vfprintf and derived functions not handling write errors (#798)
The C standard states:
> The fprintf function returns the number of characters transmitted,
> or a negative value if an output or encoding error occurred or if
> the implementation does not support a specified width length
> modifier.
- C Standard, 7.23.6.1.15. The fprintf function

However, cosmopolitan fails to return a negative value in the case of
an output error, meaning that a program such as:

 #include <stdio.h>

int main()
{
    FILE *fp = fopen("/dev/full", "w");

    setbuf(fp, NULL);
    printf("fprintf: %d\n", fprintf(fp, "test\n"));
    printf("fflush: %d\n", fflush(fp));
}

will, under cosmopolitan, print that no error occured in either of the
calls to fprintf and fflush.

This patch fixes this, along with the associated GitHub issue,
https://github.com/jart/cosmopolitan/issues/784
2023-03-31 09:57:29 -04:00
Justine Tunney 221adfa606 Fix quick typo in fsync() --ftrace 2023-03-30 16:17:40 -07:00
Justine Tunney 4ad4c2625d Add Discord invite link to README 2023-03-30 15:42:58 -07:00
Justine Tunney e3eb55cc8f Add note to README about Discord chatroom 2023-03-30 10:49:19 -07:00
Gabriel Ravier 36f52ea687 Fix PFLINK mechanism for uppercase float conversion specifiers (#796)
_PFLINK is supposed to automatically pull in required functions for
specific conversion specifiers. However, it fails to do so for the F,
G and E conversion specifiers.

This means that, for example, the following program:

 #include <stdio.h>

int main()
{
    printf("%F %G %E\n", .0, .0, .0);
}

fails to run correctly, printing "? ? ?" instead of
"0.000000 0 0.000000E+00".

This patch fixes this.
2023-03-29 22:18:59 -04:00
Gavin Hayes 669b4c5f19 Add memfd fexecve zipos support (#752) 2023-03-29 18:16:46 -07:00
Gabriel Ravier ba42248575 Fix larger than WCHAR_MAX differences in wcs{,n}cmp (#795)
The C standard states:

> Unless explicitly stated otherwise, the functions described in this
> subclause order two wide characters the same way as two integers of
> the underlying integer type designated by wchar_t.
>
> [...]
>
> The wcscmp function returns an integer greater than, equal to, or
> less than zero, accordingly as the wide string pointed to by s1 is
> greater than, equal to, or less than the wide string pointed to by
> s2.
>
> [...]
>
> The wcsncmp function returns an integer greater than, equal to, or
> less than zero, accordingly as the possibly null-terminated array
> pointed to by s1 is greater than, equal to, or less than the
> possibly null-terminated array pointed to by s2.
- C Standard, 7.31.4.4. Wide string comparison functions

Cosmopolitan fails to obey this in cases where the difference between
two wide characters is larger than WCHAR_MAX.

This means that, for example, the following program:

 #include <stdio.h>
 #include <wchar.h>
 #include <limits.h>

int main()
{
    wchar_t str1[] = { WCHAR_MIN, L'\0' };
    wchar_t str2[] = { WCHAR_MAX, L'\0' };
    printf("%d\n", wcscmp(str1, str2));
    printf("%d\n", wcsncmp(str1, str2, 2));
}

will print `1` twice, instead of the negative numbers mandated by the
standard (as WCHAR_MIN is less than WCHAR_MAX)

This patch fixes this, along with the associated Github issue,
https://github.com/jart/cosmopolitan/issues/783
2023-03-29 20:28:18 -04:00
Justine Tunney 999481ace0 Fix memcpy(size=0) ubsan warning in vsnprintf()
Fixes #785
2023-03-29 01:28:10 -07:00
Gabriel Ravier 7f925e6be9 Fix issues 774, 782 and 789 (printf precision bugs) (#790)
The C standard states that, within the context of a printf-family
function, when specifying the precision of a conversion specification:

> A negative precision argument is taken as if the precision were
> omitted.
- Quoth the C Standard, 7.23.6.1. The fprintf function

Cosmopolitan instead treated negative precision arguments as
though they had a value of 0, which was non-conforming. This
change fixes that. Another issue we found relates to:

> For o conversion, it increases the precision, if and only if
> necessary, to force the first digit of the result to be a zero (if
> the value and precision are both 0, a single 0 is printed).
- Quoth the C standard, 7.23.6.1.6. The fprintf function

When printing numbers in their alternative form, with a precision and
with a conversion specifier of o (octal), Cosmopolitan wasn't following
the standard in two ways:

1. When printing a value with a precision that results in 0-padding,
   cosmopolitan would still add an extra 0 even though this should be
   done "if and only if necessary"
2. When printing a value of 0 with a precision of 0, nothing is
   printed, even though the standard specifically states that a single
   0 is printed in this case

This change fixes those issues too. Furthermore, regression tests have
been introduced to ensure Cosmopolitan continues to be conformant
going forward.

Fixes #774 
Fixes #782 
Fixes #789
2023-03-29 01:11:48 -07:00
Gabriel Ravier 2f4335e081 Fix inttypes.h FAST16 macros to have a correct definition (#791)
Cosmopolitan now conforms to the C Standard 7.8.1 specification
of the PRI and SCN macros, because this change fixes a bug where
the FAST16 ones were incorrectly using the %hd specifier.
2023-03-29 00:19:40 -07:00
Gabriel Ravier 0adefbf152 Fix the X conversion specifier's alternative form (#788)
The standard states that, when the # flag is used:
> The result is converted to an "alternative form". [...] For x (or X)
conversion, a nonzero result has 0x (or 0X) prefixed to it.
- C standard, 7.23.6.1. The fprintf function

cosmopolitan fails to use the correct alternative form (0X) when the X
conversion specifier is used, instead using 0x, which is not
capitalized.

This patch fixes this, along with the several tests that test for the
wrong behavior.
2023-03-29 00:10:53 -07:00
Gavin Hayes a412ca7a77 clang-format: merge include blocks (#792) 2023-03-29 00:05:58 -07:00
Justine Tunney 426d441994 Fix quick regression due to header line 2023-03-28 23:35:09 -07:00
Justine Tunney 4f39139fe0 Support kill(pid, 0) behavior on Windows
It's specified by POSIX as a way to check if a pid can be signalled.
2023-03-28 22:28:09 -07:00
Justine Tunney 0d084d01b9 Discern MAP_ANONYMOUS the proper way on NT
We were checking for anonymous mappings earlier on Windows by seeing if
the file descriptor argument to mmap() was supplied as -1. This was not
correct. The proper thing to do is check `flags & MAP_ANONYMOUS`.
2023-03-28 22:16:26 -07:00
Justine Tunney 390aee960a Use enum and typedef in stdatomic.h 2023-03-28 22:15:58 -07:00
Gabriel Ravier 792b1c84c0 Fix padding+minus flag on numbers for printf-family functions (#787)
The C standard states, for conversions using the d, i, b, B, o, u, x or X conversion specifiers:
> The precision specifies the minimum number of digits to appear; if
> the value being converted can be represented in fewer digits, it is
> expanded with leading zeros.
- C standard, 7.23.6.1. The fprintf function

However, cosmopolitan currently suppresses the addition of leading
zeros when the minus flag is set. This is not reflected by anything
within the C standard, meaning that behavior is incorrect.

This patch fixes this.
2023-03-25 14:39:25 -04:00
Gabriel RavierandGavin Hayes 2d6ea2fbc9 Fix issue #771 by implementing S conversion specifier for printf-related functions (#786)
* Implement S conversion specifier for printf-related functions

POSIX specifies that a conversion specifier of S must be interpreted
the same way as %ls. This patch implements this.

* clang-format

---------

Co-authored-by: Gavin Hayes <[email protected]>
2023-03-25 14:38:21 -04:00
Paul Kulchenko 58b2f9bd6b Fix missing detail in the error message in redbean (#768)
This only happens on calls that need to happen during active connection.
2023-03-20 11:10:32 -04:00
Ariel Núñez 893703a07b Fix import asyncio (#763) 2023-03-11 23:29:03 -08:00
Alexandre Gomes Gaigalas 775f456d4c Avoid matching directories when searching PATH (#717)
When searching for an executable, performs an additional check to
determine if the path is a file.
2023-03-06 11:15:32 -05:00
Paul Kulchenko e2f429b742 Redbean documentation typo fixes. (#756)
* Redbean documentation typo fixes.

* Add brief description to some unix.* calls missing it
2023-03-06 00:15:09 -08:00
Justine Tunney ba180e754d Improve cosmocc toolchain
- Decouple zlib from libc
- Add some underscores to mostly internal names
2023-03-05 23:52:49 -08:00
Justine Tunney 713d4424c6 Don't add loopback address to generated SSL cert 2023-03-05 22:57:22 -08:00
Justine Tunney e16b969829 Make intrin_test go faster 2023-03-05 22:57:07 -08:00
Justine Tunney 2112fb1736 Fix SHUFPD and SHUFPS and add fuzz tests
See: https://github.com/jart/blink/issues/72
See: https://github.com/jart/blink/issues/68
2023-03-05 16:12:28 -08:00
Justine Tunney 2aa044cb0c Interalize msr.h 2023-03-05 16:12:11 -08:00
Justine Tunney c8a9d3458e Update commentary in ape.S 2023-03-05 16:11:03 -08:00
Gavin Hayes c5de653b98 GetZipCdir: prevent integer underflow. posix_spawn_test.c: stop attempting to load zipos from /usr/bin/ape. (#758) 2023-03-05 14:29:38 -08:00
Paul Kulchenko 22fcab131c Fix redbean SQLite to report results on failure to open db (#760) 2023-03-05 14:26:56 -08:00
imawaki da7c8c7b54 Serve .mjs files as text/javascript (#761) 2023-03-05 14:26:18 -08:00
Gavin Hayes f5520209f5 execve_test.com: fix run with landlock make. Cleanup execve/fexecve tests (#755) 2023-03-01 22:32:15 -08:00
tkchia ae365928c5 [metal] Minor corrections to constants for paging structures (#751) 2023-02-24 11:49:08 -08:00
Gavin Hayes 5923d483a4 Add execve / fexecve support to ZIpOS (#727) 2023-02-24 11:48:24 -08:00
Paul Kulchenko 0312898979 Make redbean cache directive configurable (#750)
Thank you @johnmuhl for the proposal. Fixes #739
2023-02-23 22:24:40 -08:00
Paul Kulchenko db16f0129a Redbean sqlite fix multi close (#749)
* Update redbean SQLite config to handle more options

This requires moving sqlite3_initialize call to open, as configuration
should be allowed before initialization is done. This call is effective
only for the first time and then no-op after that.

* Fix redbean SQLite for closing db with already finalized statements

There is a loop in cleanupdb that finalizes all vms that are associated
with that db when it's being closed. Under some circumstances (detailed
below) that loop may contain references pointing to already collected
objects, thus leading to SIGSEGV when those references are used.

This may happen with the following sequence of events ("VM" is the name
used in lsqlite and describes the same thing as "statement"):
1. A finalized statement is created (for example, by preparing an empty
string or a string with no statement that is still grammatically valid).
2. This statement goes out of scope before the DB object it's associated
with does and is garbage collected.
3. When it's garbage collected, dbvm_gc method is called, which checks
for svm->vm being not NULL.
4. Since the VM is already finalized, cleanupvm method is not called,
so the VM reference is not removed from the table of VMs tracked for
that DB.
5. When the DB is finally closed or garbage collected, all the VMs
associated with it are accessed to be finalized, including the ones that
have been garbage collected and have invalid references (thus leading
to a memory access error).

Here is an example of a stacktrace from the resulting SIGSEGV:

70000003de20 5df71a getgeneric+26
70000003fac0 5dfc7f luaH_get+111
70000003faf0 5e06c8 luaH_set+40
70000003fb20 5c5bd7 aux_rawset+55
70000003fb50 5c70cb lua_rawset+27
70000003fb60 4fa8e7 cleanupvm+71
70000003fb80 4fa988 cleanupdb+88
70000003fbc0 4fe899 db_gc+41

One way to fix this is to use userdata references (which anchor their
targets) instead of lightuserdata references (which do not), but this
would prevent the targets (VMs) from being garbage collected until the
DB itself is garbage collected, so this needs to be combined with
weakening the keys in the DB table. The code in cleanupdb to remove the
VM references is no longer needed, as this is handled by having weak keys.

The patch also switches to using close_v2, as it is intended for use
with garbage collected languages where the order in which destructors
are called is arbitrary, as is the case here.

* Remove GC collection from redbean SQLite session

The behavior of sqlite3session_delete is undefined after the DB
connection is closed, so we need to avoid calling it from gc handler.
2023-02-23 20:10:03 -08:00
Justine Tunney c395d16230 Upgrade embedded blink to 0.9.2 2023-02-23 12:25:01 -08:00
Justine Tunney 1011db793f Bump embedded blink binary to 0.9.0
This is an official tagged version. The 1.1 version used earlier was
informal. Formal versions will always have major.minor.patch going
forward. See https://github.com/jart/blink/tags 1.0.0 coming soon.
2023-02-23 09:57:08 -08:00
Michael Ford ca88fc34c3 doc: remove "everything everything" from ParseUrl doc (#704)
Closes: #702.
2023-02-23 09:34:48 -08:00
Justine Tunney ad97d7b9db Disable Python test_sys 2023-02-23 08:56:31 -08:00
Justine Tunney 18a24935cc Fix Python stack overflow checking in MODE=dbg 2023-02-23 08:13:33 -08:00
Justine Tunney 8b69acc1cc Make mkdeps more helpful when files are missing 2023-02-23 08:11:08 -08:00
Justine Tunney 0468ca02fe Support building APE binaries greater than 30mb 2023-02-23 08:10:43 -08:00
Justine Tunney 7355499a6a Remove a misplaced include 2023-02-23 08:10:38 -08:00
Justine Tunney 1208ac9f10 Add some POSIX feature definitions
This helps Python configure itself. We should focus more on vetting
these to make 100% sure they're accurate to define.
2023-02-23 08:09:34 -08:00
Justine Tunney 298ba74a45 Make POSIX semaphores always process shared
Python triggered the undefined behavior previously since it appears to
be posting to a semaphore owned by a different process that wasn't set
to process shared mode. The performance loss to process shared futexes
is so low and semaphores are generally used for this purpose, so it'll
be much simpler to simply not impose undefined behavior here.
2023-02-23 08:07:54 -08:00
Justine Tunney b15f9eb58f Improve Python's threading story
Python threads are now generally working, however some parts of Python's
regression tests for threads are flaky. This is possibly due to needing
more locking primitives in Cosmo's IO system call wrappers, e.g. close.

    make o//third_party/python/Lib/test/test_threading.py.runs

See #747
2023-02-23 06:55:54 -08:00
ahgamut 60eb34509b quick addition of cosmo pthreads to python.com
- enable WITH_THREAD and _POSIX_THREADS
- add headers everywhere
- breaks only two tests (faulthandler and signal)
- disabled terminal completion because it causes segfaults for some
  reason (probably could not get the current thread)
2023-02-23 06:55:17 -08:00
Justine Tunney a808b3e738 Serialize ZipOS handle IO across threads 2023-02-23 06:55:10 -08:00
Justine Tunney 0f1ead8943 Fix select() on Linux
This fixes a bug where the caller's timeval will be clobbered on Linux.
The Kernel ABI *always* modifies the timeout argument but POSIX says it
should be a const parameter. The wrapper now handles the difference and
sys_select() may be used if obtaining the remainder on Linux is needed.
2023-02-23 06:55:05 -08:00
Justine Tunney dfabcd84c1 Fix sysinfo()
The system call wrapper was wrongfully reinterpreting kernel data. The
examples/sysinfo.c program is now updated to show how to correctly use
what's returned.
2023-02-23 06:54:58 -08:00
Justine Tunney 71d955be1d Add blog link for access() impl inspiration 2023-02-23 06:54:53 -08:00
James Hulce e4ee1da652 Fix filenames in help.txt (#722)
Many functions incorrectly direct users to see a .c file, when they should be .S
2023-02-22 19:00:49 -08:00
Gavin Hayes ff9c15f48a Add APE fexecve() support (#733) 2023-02-22 18:58:23 -08:00
Gavin Hayes b275e664ec Make ZipOS mmap safer (#735)
- It now runs entirely under __mmi_lock
- Hide implementation strace
2023-02-22 18:57:36 -08:00
Paul Kulchenko e323527ffa Remove slowloris handling from Redbean (#737)
It's been superseded by token bucket processing, does not take time
into considerations (only the number of fragments), and affects file
uploads that may require a large number of reads.
2023-02-22 18:56:02 -08:00
Joshua Wierenga 08fef9b277 Fix nt path format check (#744) 2023-02-22 18:42:13 -08:00
Gavin Hayes 94dcf81954 Add glob to cocmd (#740) 2023-02-21 10:31:04 -08:00
Justine Tunney 2b6261a52d Make some system call fixes
- Fix minor ABI issue with SIOCGIFCONF
- Fix ABI translation issues with statfs() on BSDs
- Fix SQLite angled header line
2023-02-12 22:16:34 -08:00
Justine Tunney 0eb621f75e Remove non-JIT'd blink builds for now 2023-02-12 22:14:16 -08:00
Justine Tunney 9634227181 Polyfill Linux unlink() EISDIR on POSIX platforms 2023-02-05 16:50:11 -08:00
Justine Tunney 9fce2751c8 Remove problematic test 2023-02-05 16:50:04 -08:00
Justine Tunney 49c1dea03b Add msync_test 2023-02-05 16:49:52 -08:00
Justine Tunney 4b8874ceb9 Make some simple fixes 2023-02-03 02:09:15 -08:00
Justine Tunney cf19e3e46a Fix SO_LINGER on Apple 2023-02-02 20:05:37 -08:00
Justine Tunney 1429a7b3ef Fix bug in ASAN reporting 2023-02-02 02:28:58 -08:00
Justine Tunney 7486a2cd51 Perform minor cleanups 2023-02-02 02:28:58 -08:00
Gavin Hayes fd0da9c0df Implement zipos mmap (#725) 2023-02-02 02:28:50 -08:00
Paul Kulchenko 6d36584ff2 Redbean lsqlite config addition (#724)
* Fix mislabeled constant in redbean SQLite code
* Simplify callback handling in redbean SQLite
* Add config options to redbean SQLite
* Update redbean SQLite API to return current error callback
2023-01-29 19:37:19 -08:00
James Hulce f422edaf15 FIx link in gensvg demo (#723)
Code link in the gensvg demo is broken, fix that
2023-01-28 09:12:13 -08:00
Justine Tunney 1473eafd1a Fix the build and introduce some APIs
The tcgetwinsize() and tcsetwinsize() APIs are now available. The
printargs.com example also now displays the baud rate.
2023-01-20 09:25:45 -08:00
Justine Tunney cac86197cb Add fix for Blink detection 2023-01-19 09:12:49 -08:00
Justine Tunney 006c44ff5d Update tests and CPU detection for Blink 2023-01-18 00:56:09 -08:00
Justine Tunney be3e109309 Fix build break 2023-01-08 15:35:08 -08:00
Justine Tunney 2bad436abf Return error on fopen(NULL)
https://sqlite.org/forum/forumpost/d1c96a9032e564f8
2023-01-08 15:17:44 -08:00
Justine Tunney 7ab4630cc9 Fix MODE=asan build regression 2023-01-08 05:47:18 -08:00
Justine Tunney 5dab97b6d4 Implement program for finding start of zip content 2023-01-06 19:55:00 -08:00
Justine Tunney b38a442386 Add ppc64le support and improve platform detection 2023-01-06 09:14:27 -08:00
Justine Tunney 0708310041 Update the blink binaries 2023-01-05 22:50:20 -08:00
Justine Tunney cd27a53345 Support aarch64, silicon, arm, i486, riscv64, mipsel, ppc and s390x
Your APE binaries will now run on eight hardware architectures. The
tradeoff is this change adds about 800kb of bloat to the executable
which to be fair is still more than half as small as Go hello world

Your APE shell script extracts an embedded build of Blink, based on
`$(uname -m)`. The Blink binaries are statically-linked (manylinux)
which means, they'll run on every distro's stock install, including
Apple M1's that haven't enabled Rosetta. Blink's emulation has been
designed to be nearly as fast as Rosetta, using a JIT technique for
faster startup time. Blink has also been tested more thoroughly, w/
emulating Cosmopolitan software, since we're the ones who wrote it.
2023-01-05 14:29:21 -08:00
Justine Tunney 416e173448 Improve the cosmocc toolchain
These changes were necessary to help Blink become buildable using
Cosmopolitan Libc.
2023-01-05 12:20:52 -08:00
Justine Tunney d73523864a Fix bugs with Redbean Fetch() uploading
This change makes Fetch() work correctly when large bodies are supplied,
or bodies containing NUL characters are supplied.

Fixes #573
2023-01-05 09:52:36 -08:00
Justine Tunney eb69a42863 Fix --strace crashing on MacOS and Windows 2023-01-04 06:46:37 -08:00
Justine Tunney 651826b660 Update assimilate to work with modern APE format 2023-01-04 06:34:48 -08:00
Justine Tunney 6dcdf91458 Add a turfwar hilbert decoration 2023-01-03 18:28:35 -08:00
Justine Tunney a6586cafb2 Retry SQLITE_BUSY when stepping in TurfWar 2023-01-03 06:10:39 -08:00
Justine Tunney e496ee7a80 Be less pushy about redbean ddos protection 2023-01-03 05:14:34 -08:00
Justine Tunney a4b6803556 Fix recursive locks when tid is huge
It turns out Linux may assign thread ids that are much larger than
previously thought. We now have fewer bits of depth so we can have
bigger tids.
2023-01-03 05:01:00 -08:00
Justine Tunney 9c72fef086 Remove affinity tuning from IPv4 Games 2023-01-03 04:48:01 -08:00
Justine Tunney 10d1c6da18 Support argv[0] munging on Windows 2023-01-03 03:22:40 -08:00
ahgamut 5ed84e04f8 remove libc/sock/if.h
libc/sock/if.h is removed because:

- it contained only IFNAMSIZ, which was also defined in
  libc/sock/struct/ifreq.h
- it was not included in any file EXCEPT libc/isystem/net/if.h, which
  also included the libc/sock/struct/ifreq.h

hence we remove this file, and move the IFNAMSIZ definition to
libc/sock/struct/ifreq.h which is included in isystem.
2023-01-01 16:51:00 -08:00
ahgamut 25fa6acc4d check existence of IFNAMSIZ
IFNAMSIZ is defined in:

- libc/sock/if.h
- libc/sock/struct/ifreq.h

we add a check in the latter to avoid a complaint when using the
amalgamated header.
2023-01-01 16:51:00 -08:00
Ronsor 9a46c5afc4 Add a new example showing how to use hiredis. 2023-01-01 16:30:10 -08:00
Ronsor 97ae5f7b1e Fix hiredis build 2023-01-01 16:30:10 -08:00
Ronsor 1c7c277432 Include a port of the hiredis library.
SSL support is not yet included; however, everything else works
perfectly fine.
2023-01-01 16:30:10 -08:00
Danny Robinson 70ca8c4016 Update redbean lsqlite definitions (#709) 2023-01-01 16:28:34 -08:00
tkchiaandtkchia 73507d5f4e Add some definitions for Linux frame buffer/console ioctl APIs (#712)
Co-authored-by: tkchia <[email protected]>
2023-01-01 16:28:03 -08:00
Ronsor d2f811eff3 Add support for the -i (include headers) and -d (POST data) options in curl.com (#708)
This commit also paves the way for initial IPv6 support in `curl.com` by specifying
`AF_UNSPEC` for the address family in the hints passed to `getaddrinfo()`.
2022-12-23 17:06:48 -08:00
Justine Tunney 2e5181666f Fix the build 2022-12-21 23:06:55 -08:00
Gavin Hayes 5c0ce6b83e Add note about APE loader and noexec mounts (#707) 2022-12-21 23:00:51 -08:00
Ronsor f6b3019796 Make curl example into a complete tool (#706)
You can now handily use `curl` with popen()/system() by
placing `STATIC_YOINK("_curl")` in your main source file.
2022-12-21 16:12:02 -08:00
Justine Tunney dd04aeba1c Increase stack size to 128k and guard size to 16k
This improves our compatibility with Apple M1.
2022-12-18 22:58:29 -08:00
Justine Tunney 57c0dcdc29 Decrease TurfWar background query intervals
The database is much larger than it was when we started, so these
background reporting queries have become much more expensive than
before. We also had someone make 750k claims in one day, and even
hundreds of thousands per hour, which broke our assumptions about
hourly / daily reports being cheap.
2022-12-18 02:23:12 -08:00
Justine Tunney 6d39c6f646 Have redbean avoid gzip when emulated without jit 2022-12-18 02:22:58 -08:00
Justine Tunney 5a3c646307 Fix sysinfo() totalram on XNU
HW_MEMSIZE is the 64-bit version of HW_PHYSMEM, which is inaccurate on
systems having more than 4gb of RAM.
2022-12-18 02:22:46 -08:00
tkchiaandtkchia 0da47c51de [metal] Allow programs larger than 440 KiB to run in bare metal mode (#685)
* [metal] Copy program pages to extended memory at startup
* [metal] Reclaim base memory pages for later app use
* [metal] Load program pages beyond 1st 440 KiB to extended memory

o//examples/hellolua.com now runs correctly under QEMU (in
legacy BIOS mode).

* [metal] Place GDT in read/write segment

The CPU absolutely needs to alter the GDT when loading the
task register (via ltr).  To account for this, I move the
GDT into a read/write data section.  There is still a "rump"
read-only GDT in the text section that is used by the real
mode bootloader.

We also delay the loading of the task register (ltr) until
after the IDT and TSS are finally set up.

* [metal] Get examples/vga2.c serial output working for UEFI boot
* [metal] Get examples/vga2.c VGA output working for UEFI boot
* [metal] Allow munmap() to reclaim dynamically allocated pages
* Place TLS sections right after .text, not after embedded zip file

Co-authored-by: tkchia <[email protected]>
2022-12-17 17:51:20 -08:00
Justine Tunney 120079b0a6 Remove evil constants from cosmopolitan.h
- `I` a.k.a. `_Complex_I` (fixes #705)
- `B0`, `B50`, etc. (fixes #403)
2022-12-17 00:42:45 -08:00
Justine Tunney 3da887c58f Introduce CPUID detection for Blink 2022-12-17 00:38:07 -08:00
Justine Tunney 4922bc4890 Fix SQLite3 build for Landlock Make 2022-12-11 14:49:53 -08:00
Justine Tunney ed161b240e Clean up some code 2022-12-11 14:30:59 -08:00
Justine Tunney 531bfbd61f Fix DNS resolution w/ commas in Windows Registry 2022-12-11 14:30:59 -08:00
Paul Kulchenko cd0b12ed1a Add diff and iterator to SQLite session (#700) 2022-11-28 13:40:04 -08:00
Paul Kulchenko 0dc0758574 Upgrade SQLite to 3.40 (#699) 2022-11-28 12:54:48 -08:00
Paul Kulchenko bcae817215 Update SQLite deserialize to allow resizing restored DB (#701)
Using either option (FREEONCLOSE or RESIZEABLE) requires using
sqlite3_malloc. Ref. #436
2022-11-28 12:53:20 -08:00
Justine Tunney b850b14300 Improve cosmocc / cosmoc++ toolchains
They'll now automatically create empty static archives for system
libraries that are provided by Cosmopolitan Libc. This helps make
configure scripts less confused. Musl does the same thing.
2022-11-14 13:45:57 -08:00
Justine Tunney a277170c9a Make kprintf_test resistant to limbo 2022-11-13 14:26:19 -08:00
Justine Tunney 711bd095db Import exuberant ctags 2022-11-13 13:26:28 -08:00
Justine Tunney d55b019284 Fix the build 2022-11-11 11:13:21 -08:00
Justine Tunney f2af97711b Make improvements
- Improve compatibility with Blink virtual machine
- Add non-POSIX APIs for joining threads and signal masks
- Never ever use anything except 32-bit integers for atomics
- Add some `#undef` statements to workaround `ctags` problems
2022-11-10 21:52:47 -08:00
Justine Tunney b46ac13504 Introduce new jartsr=' ape prefix
Files beginning with "MZqFpD" are Actually Portable Executables,
which have a printf statement in the first 4096 bytes with octal
codes that specify the ELF header. APE also specifies `jartsr='`
as an alternative prefix, intended for binaries that do not want
to be identified as Windows executables. Like the \177ELF magic,
all these prefixes decode as x86 jump instructions that could be
used for 16-bit bootloaders or 32-bit / 64-bit flat executables.
Most importantly they provide a fallback path for Thompson shell
copmatible command interpreters, which do not require a shebang,
e.g. bash, zsh, fish, bourne, almquist, etc. Please note that in
order to meet the requirements of POSIX.1, the single quote must
be followed by a newline character, before any null bytes occur.
See also: https://www.austingroupbugs.net/view.php?id=1250
2022-11-10 21:18:32 -08:00
Justine Tunney cee6871710 Make detached threads work better
This change adds a double linked list of threads, so that pthread_exit()
will know when it should call exit() from an orphaned child. This change
also improves ftrace and strace logging.
2022-11-09 03:58:57 -08:00
Justine Tunney b74d8c1acd Remove unnecessary STRACE() statement 2022-11-08 15:33:16 -08:00
Justine Tunney fc595a7111 Fix build
libc/integral changes aren't checked in the build dependency, due to
being explicitly listed in .UNVEIL, which is how this breakage ended
up accidentally slipping through the cracks.
2022-11-08 15:24:17 -08:00
Justine Tunney bf7843833f Rename hidden keyword to _Hide 2022-11-08 12:55:28 -08:00
Paul Kulchenko 251dcb07eb Add redbean sqlite session support (#680) 2022-11-08 12:53:37 -08:00
Justine Tunney b407327972 Make fixes and improvements
- clock_nanosleep() is now much faster on OpenBSD and NetBSD
- Thread joining is now much faster on NetBSD
- FreeBSD timestamps are now more accurate
- Thread spawning now goes faster on XNU
- Clean up the clone() code
2022-11-08 10:11:46 -08:00
Justine Tunney aee50b1327 Fix test breakage 2022-11-07 06:31:30 -08:00
Justine Tunney 4de2cf34e6 Allow HTTP range past EOF
Fixes #683
2022-11-07 05:52:24 -08:00
25560 changed files with 1307345 additions and 465730 deletions

No files matched your search

+3
View File
@@ -6,8 +6,11 @@ AlignConsecutiveMacros: true
AlignConsecutiveDeclarations: false
AlwaysBreakBeforeMultilineStrings: false
AllowShortFunctionsOnASingleLine: false
AllowShortLoopsOnASingleLine: false
AllowShortIfStatementsOnASingleLine: false
KeepEmptyLinesAtTheStartOfBlocks: true
ConstructorInitializerAllOnOneLineOrOnePerLine: true
IncludeBlocks: Merge
---
Language: Cpp
AllowShortFunctionsOnASingleLine: false
+27
View File
@@ -0,0 +1,27 @@
# Run this command to always ignore formatting commits in git blame
# git config blame.ignoreRevsFile .git-blame-ignore-revs
# Upgraded clang-format to 19.0.0git
89fc95fefd40413706ff1bdaac2e8d98b328dbf1
# vim c++ filetype in modelines
04c6bc478e082263d67c41bedbd033dde2d429eb
# Ran clang-format
f032b5570b4cd87c6bb4abb54c0b98e69c939955
# Applied clang-format update to repo
6e6fc38935054db0534d5af4fb99c6193305b946
# revert retabbing
2b315626f3af765cdfbc61114647412cdb798b3a
# more modeline errata
3a8e01a77a7c97af0b16fb1651b230cee7f7d4c6
# fix more vi modelines
2fc507c98f53a76718f61f9a36602f86b5ac0cc9
# flip et/noet in modelines
e16a7d8f3b8f906c3ef76e79f57f3adfc7f25186
# fix vi modelines
394d998315f613a888cc6b6c051d4163bdf5cd6f
# clang-format
c0eacf2eb1e1c0b3bd4f71f12fef258f5b249c3f
# ape-m1 formatting cleanup
da8baf2aa5ce93b958aca90a0ae69f537806324b
# Run clang-format on most sources
369f9740de4534c28d0e81ab2afc99decbb9a3e6
+1
View File
@@ -1,3 +1,4 @@
# -*- conf -*-
*.gz binary
/build/bootstrap/*.com binary
/usr/share/zoneinfo/* binary
+59
View File
@@ -0,0 +1,59 @@
name: Low Severity Bugs
description: Used to report low severity bugs in cosmopolitan (e.g. cosmetic issues, non critical UI glitches)
title: "Bug: "
labels: ["bug", "low severity"]
body:
- type: markdown
attributes:
value: |
Thanks for taking the time to fill out this bug report!
Please include information about your system, the steps to reproduce the bug,
and the version of cosmopolitan that you are using.
If possible, please provide a minimal code example that reproduces the bug.
You may also consider using function call tracing `--ftrace` or the lighter system call tracing `--strace`
for additional technical logging that may allow us to narrow down where the fault occurred.
- type: input
id: contact
attributes:
label: Contact Details
description: How can we get in touch with you if we need more info?
placeholder: ex. [email protected]
validations:
required: false
- type: textarea
id: what-happened
attributes:
label: What happened?
description: Also tell us, what did you expect to happen?
placeholder: Tell us what you see!
validations:
required: true
- type: textarea
id: version
attributes:
label: Version
description: What version of our software are you running? (use `--version` to get a version string)
placeholder: "cosmocc (GCC) 12.3.0"
validations:
required: true
- type: dropdown
id: operating-system
attributes:
label: What operating system are you seeing the problem on?
multiple: true
options:
- Linux
- Mac
- Windows
- FreeBSD
- OpenBSD
- NetBSD
- BIOS
validations:
required: false
- type: textarea
id: logs
attributes:
label: Relevant log output
description: Please copy and paste any relevant log output. This will be automatically formatted into code, so no need for backticks.
render: shell
+59
View File
@@ -0,0 +1,59 @@
name: Medium Severity Bug
description: Used to report medium severity bugs in cosmopolitan (e.g. Malfunctioning Features but generally still useable)
title: "Bug: "
labels: ["bug", "medium severity"]
body:
- type: markdown
attributes:
value: |
Thanks for taking the time to fill out this bug report!
Please include information about your system, the steps to reproduce the bug,
and the version of cosmopolitan that you are using.
If possible, please provide a minimal code example that reproduces the bug.
You may also consider using function call tracing `--ftrace` or the lighter system call tracing `--strace`
for additional technical logging that may allow us to narrow down where the fault occurred.
- type: input
id: contact
attributes:
label: Contact Details
description: How can we get in touch with you if we need more info?
placeholder: ex. [email protected]
validations:
required: false
- type: textarea
id: what-happened
attributes:
label: What happened?
description: Also tell us, what did you expect to happen?
placeholder: Tell us what you see!
validations:
required: true
- type: textarea
id: version
attributes:
label: Version
description: What version of our software are you running? (use `--version` to get a version string)
placeholder: "cosmocc (GCC) 12.3.0"
validations:
required: true
- type: dropdown
id: operating-system
attributes:
label: What operating system are you seeing the problem on?
multiple: true
options:
- Linux
- Mac
- Windows
- FreeBSD
- OpenBSD
- NetBSD
- BIOS
validations:
required: false
- type: textarea
id: logs
attributes:
label: Relevant log output
description: Please copy and paste any relevant log output. This will be automatically formatted into code, so no need for backticks.
render: shell
+59
View File
@@ -0,0 +1,59 @@
name: High Severity Bug
description: Used to report high severity bugs in cosmopolitan (e.g. Malfunctioning features hindering important common workflow)
title: "Bug: "
labels: ["bug", "high severity"]
body:
- type: markdown
attributes:
value: |
Thanks for taking the time to fill out this bug report!
Please include information about your system, the steps to reproduce the bug,
and the version of cosmopolitan that you are using.
If possible, please provide a minimal code example that reproduces the bug.
You may also consider using function call tracing `--ftrace` or the lighter system call tracing `--strace`
for additional technical logging that may allow us to narrow down where the fault occurred.
- type: input
id: contact
attributes:
label: Contact Details
description: How can we get in touch with you if we need more info?
placeholder: ex. [email protected]
validations:
required: false
- type: textarea
id: what-happened
attributes:
label: What happened?
description: Also tell us, what did you expect to happen?
placeholder: Tell us what you see!
validations:
required: true
- type: textarea
id: version
attributes:
label: Version
description: What version of our software are you running? (use `--version` to get a version string)
placeholder: "cosmocc (GCC) 12.3.0"
validations:
required: true
- type: dropdown
id: operating-system
attributes:
label: What operating system are you seeing the problem on?
multiple: true
options:
- Linux
- Mac
- Windows
- FreeBSD
- OpenBSD
- NetBSD
- BIOS
validations:
required: false
- type: textarea
id: logs
attributes:
label: Relevant log output
description: Please copy and paste any relevant log output. This will be automatically formatted into code, so no need for backticks.
render: shell
@@ -0,0 +1,59 @@
name: Critical Severity Bug
description: Used to report critical severity bugs in cosmopolitan (e.g. Crashing, Corrupted, Dataloss)
title: "Bug: "
labels: ["bug", "critical severity"]
body:
- type: markdown
attributes:
value: |
Thanks for taking the time to fill out this bug report!
Please include information about your system, the steps to reproduce the bug,
and the version of cosmopolitan that you are using.
If possible, please provide a minimal code example that reproduces the bug.
You may also consider using function call tracing `--ftrace` or the lighter system call tracing `--strace`
for additional technical logging that may allow us to narrow down where the fault occurred.
- type: input
id: contact
attributes:
label: Contact Details
description: How can we get in touch with you if we need more info?
placeholder: ex. [email protected]
validations:
required: false
- type: textarea
id: what-happened
attributes:
label: What happened?
description: Also tell us, what did you expect to happen?
placeholder: Tell us what you see!
validations:
required: true
- type: textarea
id: version
attributes:
label: Version
description: What version of our software are you running? (use `--version` to get a version string)
placeholder: "cosmocc (GCC) 12.3.0"
validations:
required: true
- type: dropdown
id: operating-system
attributes:
label: What operating system are you seeing the problem on?
multiple: true
options:
- Linux
- Mac
- Windows
- FreeBSD
- OpenBSD
- NetBSD
- BIOS
validations:
required: false
- type: textarea
id: logs
attributes:
label: Relevant log output
description: Please copy and paste any relevant log output. This will be automatically formatted into code, so no need for backticks.
render: shell
+51
View File
@@ -0,0 +1,51 @@
name: Enhancement template
description: Used to request enhancements for cosmopolitan
title: "Feature Request: "
labels: ["enhancement"]
body:
- type: markdown
attributes:
value: |
[Please post your idea first in Discussion if there is not yet a consensus for this enhancement request. This will help to keep this issue tracker focused on enhancements that the community has agreed needs to be implemented.](https://github.com/jart/cosmopolitan/discussions/categories/ideas)
- type: checkboxes
id: prerequisites
attributes:
label: Prerequisites
description: Please confirm the following before submitting your enhancement request.
options:
- label: I am running the latest code. Mention the version if possible as well.
required: true
- label: I carefully followed the [README.md](https://github.com/jart/cosmopolitan/blob/master/README.md).
required: true
- label: I searched using keywords relevant to my issue to make sure that I am creating a new issue that is not already open (or closed).
required: true
- label: I reviewed the [Discussions](https://github.com/jart/cosmopolitan/discussions), and have a new and useful enhancement to share.
required: true
- type: textarea
id: feature-description
attributes:
label: Feature Description
description: Please provide a detailed written description of what you were trying to do, and what you expected `cosmopolitan` to do as an enhancement.
placeholder: Detailed description of the enhancement
validations:
required: true
- type: textarea
id: motivation
attributes:
label: Motivation
description: Please provide a detailed written description of reasons why this feature is necessary and how it is useful to `cosmopolitan` users.
placeholder: Explanation of why this feature is needed and its benefits
validations:
required: true
- type: textarea
id: possible-implementation
attributes:
label: Possible Implementation
description: If you have an idea as to how it can be implemented, please write a detailed description. Feel free to give links to external sources or share visuals that might be helpful to understand the details better.
placeholder: Detailed description of potential implementation
validations:
required: false
+52
View File
@@ -0,0 +1,52 @@
name: Research
description: Track new technical research area
title: "Research: "
labels: ["research"]
body:
- type: markdown
attributes:
value: |
Don't forget to check for any [duplicate research issue tickets](https://github.com/jart/cosmopolitan/issues?q=is%3Aopen+is%3Aissue+label%3A%22research+%F0%9F%94%AC%22)
- type: checkboxes
id: research-stage
attributes:
label: Research Stage
description: Track general state of this research ticket
options:
- label: Background Research (Let's try to avoid reinventing the wheel)
- label: Hypothesis Formed (How do you think this will work and it's effect?)
- label: Strategy / Implementation Forming
- label: Analysis of results
- label: Debrief / Documentation (So people in the future can learn from us)
- type: textarea
id: background
attributes:
label: Previous existing literature and research
description: Whats the current state of the art and whats the motivation for this research?
- type: textarea
id: hypothesis
attributes:
label: Hypothesis
description: How do you think this will work and it's effect?
- type: textarea
id: implementation
attributes:
label: Implementation
description: Got an approach? e.g. a PR ready to go?
- type: textarea
id: analysis
attributes:
label: Analysis
description: How does the proposed implementation behave?
- type: textarea
id: logs
attributes:
label: Relevant log output
description: Please copy and paste any relevant log output. This will be automatically formatted into code, so no need for backticks.
render: shell
+28
View File
@@ -0,0 +1,28 @@
name: Refactor (Maintainers)
description: Used to track refactoring opportunities
title: "Refactor: "
labels: ["refactor"]
body:
- type: markdown
attributes:
value: |
Don't forget to [check for existing refactor issue tickets](https://github.com/jart/cosmopolitan/issues?q=is%3Aopen+is%3Aissue+label%3Arefactoring) in case it's already covered.
Also you may want to check [Pull request refactor label as well](https://github.com/jart/cosmopolitan/pulls?q=is%3Aopen+is%3Apr+label%3Arefactoring) for duplicates too.
- type: textarea
id: background-description
attributes:
label: Background Description
description: Please provide a detailed written description of the pain points you are trying to solve.
placeholder: Detailed description behind your motivation to request refactor
validations:
required: true
- type: textarea
id: possible-approaches
attributes:
label: Possible Refactor Approaches
description: If you have some idea of possible approaches to solve this problem. You may want to make it a todo list.
placeholder: Your idea of possible refactoring opportunity/approaches
validations:
required: false
+15
View File
@@ -0,0 +1,15 @@
blank_issues_enabled: true
contact_links:
- name: FAQ
url: https://github.com/jart/cosmopolitan/wiki/FAQ
about: Is your question a common one? You may want to check here first.
- name: Got an idea?
url: https://github.com/jart/cosmopolitan/discussions/categories/ideas
about: Pop it there. It may then become an enhancement ticket.
- name: Got a question?
url: https://github.com/jart/cosmopolitan/discussions/categories/q-a
about: Ask a question there!
- name: Want to contribute?
url: https://github.com/jart/cosmopolitan/wiki/contribute
about: Head to the contribution guide page of the wiki for areas you can help with
+49
View File
@@ -0,0 +1,49 @@
# https://github.com/actions/labeler
documentation:
- changed-files:
- any-glob-to-any-file:
- README.md
- LICENSE
- CONTRIBUTING.md
- libc/README.md
- tool/cosmocc/README.md
- third_party/getopt/README.txt
build:
- changed-files:
- any-glob-to-any-file:
- build/**
- Makefile
- '*/*.mk'
examples:
- changed-files:
- any-glob-to-any-file: examples/**
devops:
- changed-files:
- any-glob-to-any-file:
- .github/**
- .clang-format
dsp:
- changed-files:
- any-glob-to-any-file:
- dsp/**
ape:
- changed-files:
- any-glob-to-any-file:
- ape/**
libc:
- changed-files:
- any-glob-to-any-file:
- libc/**
net:
- changed-files:
- any-glob-to-any-file:
- net/**
third_party:
- changed-files:
- any-glob-to-any-file:
- third_party/**
tool:
- changed-files:
- any-glob-to-any-file:
- tool/**
+1 -1
View File
@@ -17,7 +17,7 @@ jobs:
runs-on: ubuntu-latest
strategy:
matrix:
mode: ["", tiny, opt, rel]
mode: ["", tiny, rel, tinylinux, optlinux]
steps:
- uses: actions/checkout@v3
+17
View File
@@ -0,0 +1,17 @@
name: "Pull Request Labeler"
on:
- pull_request_target
jobs:
labeler:
permissions:
contents: read
pull-requests: write
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
repository: "jart/cosmopolitan"
- uses: actions/labeler@v5
with:
configuration-path: '.github/labeler.yml'
+2
View File
@@ -1,6 +1,8 @@
# -*- conf -*-
/o
/cosmocc
/.cosmocc
# TODO: Find some way to have Python write to o/
__pycache__
-112
View File
@@ -1,112 +0,0 @@
{
"configurations": [
{
"name": "Win32",
"includePath": [
"${workspaceFolder}"
],
"cStandard": "c11",
"cppStandard": "c++11",
"forcedInclude": ["${workspaceFolder}/.vscode/vscode.h"],
"defines": [
"libcesque=",
"pureconst=",
"paramsnonnull(x)=",
"alignas(x)",
"alignof(x)",
"artificial=",
"dontdiscard=",
"mayalias=",
"forceinline=",
"forcealign(x)=",
"scanfesque(x)=",
"strftimeesque(x)=",
"wontreturn=",
"textreal=",
"mallocesque=",
"callocesque=",
"vallocesque=",
"reallocesque=",
"strlenesque=",
"memcpyesque=",
"hasatleast=",
"noinline=",
"textexit=",
"returnstwice=",
"textwindows=",
"privileged=",
"compatfn=",
"noinstrument=",
"nodebuginfo=",
"interruptfn=",
"optimizespeed=",
"forcealignargpointer=",
"noasan=",
"noubsan=",
"donothing=",
"nosideeffect=",
"unreachable=",,
"notpossible=",
"thatispacked=",
"dontthrow=",
"nocallback=",
"relegated=",
"hidden=",
"textstartup=",
"initarray=",
"returnsnonnull=",
"returnspointerwithnoaliases=",
"printfesque(x)=",
"attributeallocsize(x)=",
"returnsaligned(x)=",
"attributeallocalign(x)=",
"nullterminated(x)="
]
},
{
"name": "Linux",
"includePath": [
"${workspaceFolder}"
],
"cStandard": "gnu17",
"compilerPath": "${workspaceFolder}/o/third_party/gcc/bin/x86_64-linux-musl-gcc",
"compilerArgs": [
"-Wall",
"-Werror",
"-fdebug-prefix-map=${workspaceFolder}=",
"-frecord-gcc-switches",
"-Wa,-W",
"-Wa,-I.",
"-Wa,--noexecstack",
"-Wa,--nocompress-debug-sections",
"-Og",
"-g",
"-gdescribe-dies",
"-msse3",
"-mno-red-zone",
"-fno-math-errno",
"-fno-trapping-math",
"-fno-fp-int-builtin-inexact",
"-fno-ident",
"-fno-common",
"-fno-gnu-unique",
"-fstrict-aliasing",
"-fstrict-overflow",
"-fno-semantic-interposition",
"-mno-omit-leaf-frame-pointer",
"-fno-jump-tables",
"-nostdinc",
"-iquote."
],
"forcedInclude": [
"libc/integral/normalize.inc"
],
"defines": [
"COSMO",
"MODE=",
"IMAGE_BASE_VIRTUAL=0x400000"
]
}
],
"version": 4
}
-2
View File
@@ -1,2 +0,0 @@
#define __VSCODE_INTELLISENSE__ 1
#include "libc/integral/normalize.inc"
+69 -45
View File
@@ -2,23 +2,77 @@
We'd love to accept your patches! Please read this guide first.
## Identity Disclosure
This project does not accept anonymous contributions. Justine Tunney
won't merge pull requests from strangers. In order to change the Cosmo
codebase, and have your changes be upstreamed, she has to know who you
are. You're encouraged to disclose your full name and email address to
the public too, by including them in your git commit messages; however
that's not a requirement; as we're happy to respect the wishes of
contributors who prefer to remain anonymous to the public.
## Copyright Assignment
Please send an email to Justine Tunney <jtunney@gmail.com> stating that
you intend to assign her the copyright to the changes you contribute to
Cosmopolitan. Please use the same email address you use for git commits
which should only contain original source code from you or other people
who are also assigning copyright. Please note that, if you're employed,
you may need to get your employer's approval beforehand. If you can not
assign copyright due to local laws, then you may alternatively consider
disclaiming it using the language in [Unlicense](https://unlicense.org)
or [CC-0](http://creativecommons.org/share-your-work/public-domain/cc0)
The first time you send a pull request, you need to send an email to
Justine Tunney <jtunney@gmail.com> stating that you intend to assign her
the copyright to the changes you contribute to Cosmopolitan. It only
needs to happen once. This only applies to the code you *choose* to
contribute. The email should be sent from an email address associated
with your identity. Your email should link to your pull request.
This is important because we can't produce 12kb single-file executables
that comply with license requirements if we have to embed lots of them.
Although that's less of an issue depending on the purpose of the files.
For example, ownership is much less of a concern in the unit test files
so you're encouraged to put your copyright on those, provided it's ISC.
To make things easy, here's an example of a good email you can use:
> **From**: YOUR NAME ([email protected])
> **To**: Justine Tunney ([email protected])
> **Subject**: Cosmopolitan Copyright Assignment for YOUR NAME
>
> Hi Justine,
>
> I made my first contribution to Cosmopolitan in
> https://github.com/jart/cosmopolitan/pull/XXXX could you please take a
> look? I intend to assign you the copyright to the changes I contribute
> to Cosmopolitan.
>
> Thanks!
Please note that in order to give Justine the copyright, it has to be
yours to give in the first place. If you're employed, then you should
get your employer's approval to do this beforehand. Even with big
companies like Google, this process is quick and painless. Usually we
see employers granting authorization in less than one day.
If you live in a country that doesn't recognize one's ability to assign
copyright, then you may alternatively consider disclaiming it using the
language in [Unlicense](https://unlicense.org) or
[CC-0](http://creativecommons.org/share-your-work/public-domain/cc0).
If you're checking-in third party code, then you need to have headers at
the top of each source file (but never header files) documenting its
owners and the code should go in the `third_party/` folder. Every third
party project should have a `README.cosmo` file that documents its
provenance as well as any local changes you've made.
## Copyright Policy Exceptions
### Tests
You're encoraged to claim ownership of your test code. If you add a new
file under the `test/` directory, then you should put your name in the
ISC license header at the top of the file. If you add new test cases to
an existing unit test file, then you're encouraged to append a line with
your name to the existing copyright header of that file.
### Exceptional Features
Let's say you discovered a faster better way to implement `log10()` and
you want to give it to Cosmopolitan. In cases like this, it really isn't
appropriate for Justine to own your code. What you could do instead, is
write your own new and improved `log10.c` from scratch, put your name on
the top with the ISC license, and then add a `__notice()` directive so
that your name will be embedded inside every executable that links the
`log10()` function. This will help you get your name out there. Please
note you need get approval from Justine each time you want to do this.
## Style Guide
@@ -29,34 +83,4 @@ clang-format -i -style=file tool/net/redbean.c
```
If you use Emacs this can be automated on save for Cosmopolitan using
[tool/emacs/cosmo-format.el]([tool/emacs/cosmo-format.el]).
### Source Files
- Must use include paths relative to the root of the repository
- Must have comment at top of file documenting copyright and license
- Must have notice embedding if not owned by Justine (exception: tests)
- May use language extensions that are supported by both GCC and Clang
- Should use Google indentation (otherwise use `/* clang-format off */`)
- Should use asm() instead of compiler APIs (exception: ctz, clz, memcpy)
### Header Files
- Must not have copyright or license comments
- Must have once guards (otherwise change `.h` to `.inc`)
- Must be ANSI C89 compatible to be included in the amalgamation header
- Must include its dependencies (exception: libc/integral/normalize.inc)
- Must not define objects (i.e. `cc -c -xc foo.h` will produce empty `.o`)
- Should not use typedefs
- Should not use forward declarations
- Should not include documentation comments
- Should not include parameter names in prototypes
- Should not pose problems if included by C++ or Assembly sources
- Should not declare non-ANSI code, at all, when the user requests ANSI
### Build Config
- Must not write files outside `o/`
- Must not communicate with Internet
- Must not depend on system libraries
- Must not depend on system commands (exception: sh, make, gzip, zip)
[tool/emacs/cosmo-format.el](tool/emacs/cosmo-format.el).
+398 -203
View File
@@ -1,5 +1,5 @@
#-*-mode:makefile-gmake;indent-tabs-mode:t;tab-width:8;coding:utf-8-*-┐
#───vi: set et ft=make ts=8 tw=8 fenc=utf-8 :vi───────────────────────┘
#── vi: set et ft=make ts=8 sw=8 fenc=utf-8 :vi ──────────────────────┘
#
# SYNOPSIS
#
@@ -21,23 +21,23 @@
# make -j8 -O MODE=tiny
#
# # build individual target
# make -j8 -O o//examples/hello.com
# o//examples/hello.com
# make -j8 -O o//examples/hello
# o//examples/hello
#
# # view source
# less examples/hello.c
#
# # view binary
# o//tool/viz/bing.com o//examples/hello.com |
# o//tool/viz/fold.com
# o//tool/viz/bing o//examples/hello |
# o//tool/viz/fold
#
# # view transitive closure of legalese
# o//tool/viz/bing.com -n o//examples/hello.com |
# o//tool/viz/fold.com
# o//tool/viz/bing -n o//examples/hello |
# o//tool/viz/fold
#
# # basic debugging
# make -j8 -O MODE=dbg o/dbg/examples/crashreport.com
# o/examples/crashreport.com
# make -j8 -O MODE=dbg o/dbg/examples/crashreport
# o/examples/crashreport
# less examples/crashreport.c
#
# # extremely tiny binaries
@@ -51,7 +51,7 @@
#
# TROUBLESHOOTING
#
# make -j8 -O V=1 o//examples/hello.com
# make -j8 -O V=1 o//examples/hello
# make o//examples/life.elf -pn |& less
# etc.
#
@@ -59,14 +59,13 @@
#
# build/config.mk
SHELL = build/bootstrap/cocmd.com
HOSTS ?= freebsd openbsd netbsd rhel7 rhel5 xnu win10 win10:31336
SHELL = build/bootstrap/cocmd
MAKEFLAGS += --no-builtin-rules
.SUFFIXES:
.DELETE_ON_ERROR:
.FEATURES: output-sync
.PHONY: all o bins check test depend tags
.PHONY: all o bins check test depend tags aarch64 clean
ifneq ($(m),)
ifeq ($(MODE),)
@@ -74,11 +73,108 @@ MODE := $(m)
endif
endif
COMMA := ,
PWD := $(shell pwd)
# detect wsl2 running cosmopolitan binaries on the host by checking whether:
# - user ran build/bootstrap/make, in which case make's working directory is in wsl
# - user ran make, in which case cocmd's working directory is in wsl
ifneq ($(findstring //wsl.localhost/,$(CURDIR) $(PWD)),)
$(warning wsl2 interop is enabled)
$(error you need to run sudo sh -c 'echo -1 > /proc/sys/fs/binfmt_misc/WSLInterop')
endif
UNAME_M := $(shell uname -m)
UNAME_S := $(shell uname -s)
# apple still distributes a 17 year old version of gnu make
ifeq ($(MAKE_VERSION), 3.81)
$(error please use build/bootstrap/make)
endif
LC_ALL = C
SOURCE_DATE_EPOCH = 0
ARFLAGS = rcsD
ZFLAGS ?=
XARGS ?= xargs -P4 -rs8000
DOT ?= dot
CLANG = clang
TMPDIR = o/tmp
AR = build/bootstrap/ar
CP = build/bootstrap/cp
RM = build/bootstrap/rm -f
GZIP = build/bootstrap/gzip
ECHO = build/bootstrap/echo
CHMOD = build/bootstrap/chmod
TOUCH = build/bootstrap/touch
PKG = build/bootstrap/package
MKDEPS = build/bootstrap/mkdeps
ZIPOBJ = build/bootstrap/zipobj
ZIPCOPY = build/bootstrap/zipcopy
PECHECK = build/bootstrap/pecheck
FIXUPOBJ = build/bootstrap/fixupobj
MKDIR = build/bootstrap/mkdir -p
COMPILE = build/bootstrap/compile -V9 -M2048m -P8192 $(QUOTA)
IGNORE := $(shell $(MKDIR) $(TMPDIR))
# the default build modes is empty string
# on x86_64 hosts, MODE= is the same as MODE=x86_64
# on aarch64 hosts, MODE= is changed to MODE=aarch64
ifeq ($(MODE),)
ifeq ($(UNAME_M),arm64)
MODE := aarch64
endif
ifeq ($(UNAME_M),aarch64)
MODE := aarch64
endif
endif
ifneq ($(findstring aarch64,$(MODE)),)
ARCH = aarch64
HOSTS ?= pi studio freebsdarm
else
ARCH = x86_64
HOSTS ?= freebsd rhel7 xnu openbsd netbsd win10
endif
ZIPOBJ_FLAGS += -a$(ARCH)
IGNORE := $(shell $(MKDIR) $(TMPDIR))
export ADDR2LINE
export LC_ALL
export MKDIR
export MODE
export SOURCE_DATE_EPOCH
export TMPDIR
COSMOCC = .cosmocc/3.3.5
TOOLCHAIN = $(COSMOCC)/bin/$(ARCH)-linux-cosmo-
DOWNLOAD := $(shell build/download-cosmocc.sh $(COSMOCC) 3.3.5 db78fd8d3f8706e9dff4be72bf71d37a3f12062f212f407e1c33bc4af3780dd0)
AS = $(TOOLCHAIN)as
CC = $(TOOLCHAIN)gcc
CXX = $(TOOLCHAIN)g++
CXXFILT = $(TOOLCHAIN)c++filt
LD = $(TOOLCHAIN)ld.bfd
NM = $(TOOLCHAIN)nm
GCC = $(TOOLCHAIN)gcc
STRIP = $(TOOLCHAIN)strip
OBJCOPY = $(TOOLCHAIN)objcopy
OBJDUMP = $(TOOLCHAIN)objdump
ifneq ($(wildcard $(PWD)/$(TOOLCHAIN)addr2line),)
ADDR2LINE = $(PWD)/$(TOOLCHAIN)addr2line
else
ADDR2LINE = $(TOOLCHAIN)addr2line
endif
# primary build rules
all: o
o: o/$(MODE)
o/$(MODE): \
o/$(MODE)/ape \
o/$(MODE)/ctl \
o/$(MODE)/dsp \
o/$(MODE)/net \
o/$(MODE)/libc \
@@ -87,20 +183,43 @@ o/$(MODE): \
o/$(MODE)/examples \
o/$(MODE)/third_party
# TODO(jart): Make Emacs `C-c C-c` shortcut not need this.
.PHONY: o/$(MODE)/ o/$(MODE)/.
o/$(MODE)/: o/$(MODE)
o/$(MODE)/.: o/$(MODE)
# check if we're using o//third_party/make/make
# we added sandboxing to guarantee cosmo's makefile is hermetic
# it also shaves away 200ms of startup latency with native $(uniq)
ifneq ($(LANDLOCKMAKE_VERSION),)
ifeq ($(UNAME_S),Linux)
ifeq ($(wildcard /usr/bin/ape),)
$(warning please run ape/apeinstall.sh if you intend to use landlock make)
$(shell sleep .5)
endif
endif
ifneq ($(TOOLCHAIN),)
.STRICT = 1
.PLEDGE = stdio rpath wpath cpath fattr proc
.UNVEIL = \
libc/integral \
libc/disclaimer.inc \
rwc:/dev/shm \
rx:build/bootstrap \
rx:o/third_party/gcc \
/proc/stat \
rw:/dev/null \
w:o/stack.log \
/etc/hosts \
~/.runit.psk \
/proc/self/status \
endif
endif
.PLEDGE += stdio rpath wpath cpath fattr proc
.UNVEIL += \
libc/integral \
libc/stdbool.h \
rwc:/dev/shm \
rx:.cosmocc \
rx:build/bootstrap \
r:build/portcosmo.h \
/proc/stat \
rw:/dev/null \
rw:/dev/full \
w:o/stack.log \
/etc/hosts \
~/.runit.psk \
/proc/self/status \
rx:/usr/bin/qemu-aarch64 \
rx:o/third_party/qemu/qemu-aarch64 \
/sys/devices/system/cpu/cpu0/cpufreq/scaling_governor
PKGS =
@@ -110,141 +229,161 @@ include build/functions.mk #─┐
include build/definitions.mk # ├──META
include build/config.mk # │ You can build
include build/rules.mk # │ You can topologically order
include build/online.mk # │
include libc/stubs/stubs.mk #─┘
include libc/nexgen32e/nexgen32e.mk #─┐
include libc/sysv/sysv.mk # ├──SYSTEM SUPPORT
include libc/nt/nt.mk # │ You can do math
include libc/intrin/intrin.mk # │ You can use the stack
include libc/linux/linux.mk # │ You can manipulate arrays
include libc/tinymath/tinymath.mk # │ You can issue raw system calls
include third_party/compiler_rt/compiler_rt.mk # │
include libc/str/str.mk # │
include third_party/xed/xed.mk # │
include third_party/zlib/zlib.mk # │
include third_party/double-conversion/dc.mk # │
include libc/elf/elf.mk # │
include ape/ape.mk # │
include libc/fmt/fmt.mk # │
include libc/vga/vga.mk #─┘
include libc/calls/calls.mk #─┐
include third_party/getopt/getopt.mk # │
include libc/runtime/runtime.mk # ├──SYSTEMS RUNTIME
include libc/crt/crt.mk # │ You can issue system calls
include third_party/nsync/nsync.mk # │
include third_party/dlmalloc/dlmalloc.mk #─┘
include libc/mem/mem.mk #─┐
include libc/zipos/zipos.mk # ├──DYNAMIC RUNTIME
include third_party/gdtoa/gdtoa.mk # │ You can now use stdio
include libc/time/time.mk # │ You can finally call malloc()
include third_party/nsync/mem/mem.mk # │
include libc/thread/thread.mk # │
include libc/stdio/stdio.mk # │
include third_party/libcxx/libcxx.mk # │
include net/net.mk # │
include libc/log/log.mk # │
include third_party/bzip2/bzip2.mk # │
include dsp/core/core.mk # │
include libc/x/x.mk # │
include third_party/stb/stb.mk # │
include dsp/scale/scale.mk # │
include dsp/mpeg/mpeg.mk # │
include dsp/dsp.mk # │
include third_party/zlib/gz/gz.mk # │
include third_party/musl/musl.mk # │
include libc/libc.mk #─┘
include libc/sock/sock.mk #─┐
include dsp/tty/tty.mk # ├──ONLINE RUNTIME
include libc/dns/dns.mk # │ You can communicate with the network
include net/http/http.mk # │
include third_party/mbedtls/mbedtls.mk # │
include net/https/https.mk # │
include third_party/regex/regex.mk #─┘
include third_party/tidy/tidy.mk
include third_party/third_party.mk
include third_party/nsync/testing/testing.mk
include libc/testlib/testlib.mk
include tool/viz/lib/vizlib.mk
include tool/args/args.mk
include test/tool/args/test.mk
include third_party/linenoise/linenoise.mk
include third_party/maxmind/maxmind.mk
include net/finger/finger.mk
include third_party/double-conversion/test/test.mk
include third_party/lua/lua.mk
include third_party/tr/tr.mk
include third_party/sed/sed.mk
include third_party/awk/awk.mk
include third_party/make/make.mk
include third_party/finger/finger.mk
include third_party/argon2/argon2.mk
include third_party/smallz4/smallz4.mk
include third_party/sqlite3/sqlite3.mk
include third_party/mbedtls/test/test.mk
include third_party/quickjs/quickjs.mk
include third_party/lz4cli/lz4cli.mk
include third_party/zip/zip.mk
include third_party/unzip/unzip.mk
include tool/build/lib/buildlib.mk
include third_party/chibicc/chibicc.mk
include third_party/chibicc/test/test.mk
include third_party/python/python.mk
include tool/build/emucrt/emucrt.mk
include tool/build/emubin/emubin.mk
include tool/build/build.mk
include examples/examples.mk
include examples/pyapp/pyapp.mk
include examples/pylife/pylife.mk
include tool/decode/lib/decodelib.mk
include tool/decode/decode.mk
include tool/lambda/lib/lib.mk
include tool/lambda/lambda.mk
include tool/plinko/lib/lib.mk
include tool/plinko/plinko.mk
include test/tool/plinko/test.mk
include tool/hash/hash.mk
include tool/net/net.mk
include tool/viz/viz.mk
include tool/tool.mk
include net/turfwar/turfwar.mk
include test/libc/tinymath/test.mk
include test/libc/intrin/test.mk
include test/libc/mem/test.mk
include test/libc/nexgen32e/test.mk
include test/libc/runtime/test.mk
include test/libc/thread/test.mk
include test/libc/sock/test.mk
include test/libc/str/test.mk
include test/libc/log/test.mk
include test/libc/str/test.mk
include test/libc/calls/test.mk
include test/libc/x/test.mk
include test/libc/xed/test.mk
include test/libc/fmt/test.mk
include test/libc/dns/test.mk
include test/libc/time/test.mk
include test/libc/stdio/test.mk
include test/libc/zipos/test.mk
include test/libc/release/test.mk
include test/libc/test.mk
include test/net/http/test.mk
include test/net/https/test.mk
include test/net/finger/test.mk
include test/net/test.mk
include test/tool/build/lib/test.mk
include test/tool/build/test.mk
include test/tool/viz/lib/test.mk
include test/tool/viz/test.mk
include test/tool/net/test.mk
include test/tool/test.mk
include test/dsp/core/test.mk
include test/dsp/scale/test.mk
include test/dsp/tty/test.mk
include test/dsp/test.mk
include examples/package/lib/build.mk
include examples/package/build.mk
include build/online.mk #─┘
include libc/nexgen32e/BUILD.mk #─┐
include libc/sysv/BUILD.mk # ├──SYSTEM SUPPORT
include libc/nt/BUILD.mk # │ You can do math
include libc/intrin/BUILD.mk # │ You can use the stack
include third_party/compiler_rt/BUILD.mk # │ You can manipulate arrays
include libc/tinymath/BUILD.mk # │ You can issue raw system calls
include libc/str/BUILD.mk # │
include third_party/xed/BUILD.mk # │
include third_party/puff/BUILD.mk # │
include libc/elf/BUILD.mk # │
include ape/BUILD.mk # │
include libc/fmt/BUILD.mk # │
include libc/vga/BUILD.mk # │
include libc/irq/BUILD.mk #─┘
include libc/calls/BUILD.mk #─┐
include libc/irq/BUILD.mk # ├──SYSTEMS RUNTIME
include third_party/nsync/BUILD.mk # │ You can issue system calls
include libc/runtime/BUILD.mk # │
include libc/crt/BUILD.mk # │
include third_party/dlmalloc/BUILD.mk #─┘
include libc/mem/BUILD.mk #─┐
include third_party/gdtoa/BUILD.mk # ├──DYNAMIC RUNTIME
include third_party/nsync/mem/BUILD.mk # │ You can now use stdio
include libc/proc/BUILD.mk # │ You can now use threads
include libc/dlopen/BUILD.mk # │ You can now use processes
include libc/thread/BUILD.mk # │ You can finally call malloc()
include ctl/BUILD.mk # │
include third_party/zlib/BUILD.mk # │
include libc/stdio/BUILD.mk # │
include tool/hello/BUILD.mk # │
include third_party/tz/BUILD.mk # │
include net/BUILD.mk # │
include third_party/vqsort/BUILD.mk # │
include libc/log/BUILD.mk # │
include third_party/getopt/BUILD.mk # │
include third_party/bzip2/BUILD.mk # │
include dsp/core/BUILD.mk # │
include third_party/zlib/gz/BUILD.mk # │
include third_party/intel/BUILD.mk # │
include third_party/aarch64/BUILD.mk # │
include libc/BUILD.mk #─┘
include libc/sock/BUILD.mk #─┐
include net/http/BUILD.mk # ├──ONLINE RUNTIME
include third_party/musl/BUILD.mk # │ You can communicate with the network
include libc/x/BUILD.mk # │
include dsp/scale/BUILD.mk # │
include dsp/mpeg/BUILD.mk # │
include dsp/tty/BUILD.mk # │
include dsp/BUILD.mk # │
include third_party/stb/BUILD.mk # │
include third_party/mbedtls/BUILD.mk # │
include third_party/ncurses/BUILD.mk # │
include third_party/readline/BUILD.mk # │
include third_party/libunwind/BUILD.mk # |
include third_party/libcxxabi/BUILD.mk # |
include third_party/libcxx/BUILD.mk # │
include third_party/openmp/BUILD.mk # │
include third_party/double-conversion/BUILD.mk # │
include third_party/pcre/BUILD.mk # │
include third_party/less/BUILD.mk # │
include net/https/BUILD.mk # │
include third_party/regex/BUILD.mk # │
include third_party/bash/BUILD.mk #─┘
include third_party/tidy/BUILD.mk
include third_party/BUILD.mk
include third_party/nsync/testing/BUILD.mk
include libc/testlib/BUILD.mk
include tool/viz/lib/BUILD.mk
include tool/args/BUILD.mk
include test/math/BUILD.mk
include test/posix/BUILD.mk
include test/ctl/BUILD.mk
include test/libcxx/BUILD.mk
include test/tool/args/BUILD.mk
include third_party/linenoise/BUILD.mk
include third_party/maxmind/BUILD.mk
include net/finger/BUILD.mk
include third_party/double-conversion/test/BUILD.mk
include third_party/lua/BUILD.mk
include third_party/tree/BUILD.mk
include third_party/zstd/BUILD.mk
include third_party/tr/BUILD.mk
include third_party/sed/BUILD.mk
include third_party/awk/BUILD.mk
include third_party/hiredis/BUILD.mk
include third_party/make/BUILD.mk
include third_party/ctags/BUILD.mk
include third_party/finger/BUILD.mk
include third_party/argon2/BUILD.mk
include third_party/smallz4/BUILD.mk
include third_party/sqlite3/BUILD.mk
include third_party/mbedtls/test/BUILD.mk
include third_party/lz4cli/BUILD.mk
include third_party/zip/BUILD.mk
include third_party/xxhash/BUILD.mk
include third_party/unzip/BUILD.mk
include tool/build/lib/BUILD.mk
include third_party/chibicc/BUILD.mk
include third_party/chibicc/test/BUILD.mk
include third_party/python/BUILD.mk
include tool/build/BUILD.mk
include tool/curl/BUILD.mk
include third_party/qemu/BUILD.mk
include third_party/libcxxabi/test/BUILD.mk
include examples/BUILD.mk
include examples/pyapp/BUILD.mk
include examples/pylife/BUILD.mk
include tool/decode/lib/BUILD.mk
include tool/decode/BUILD.mk
include tool/lambda/lib/BUILD.mk
include tool/lambda/BUILD.mk
include tool/plinko/lib/BUILD.mk
include tool/plinko/BUILD.mk
include test/tool/plinko/BUILD.mk
include tool/net/BUILD.mk
include tool/viz/BUILD.mk
include tool/BUILD.mk
include net/turfwar/BUILD.mk
include test/libc/tinymath/BUILD.mk
include test/libc/intrin/BUILD.mk
include test/libc/mem/BUILD.mk
include test/libc/nexgen32e/BUILD.mk
include test/libc/runtime/BUILD.mk
include test/libc/thread/BUILD.mk
include test/libc/sock/BUILD.mk
include test/libc/str/BUILD.mk
include test/libc/log/BUILD.mk
include test/libc/str/BUILD.mk
include test/libc/calls/BUILD.mk
include test/libc/x/BUILD.mk
include test/libc/xed/BUILD.mk
include test/libc/fmt/BUILD.mk
include test/libc/time/BUILD.mk
include test/libc/proc/BUILD.mk
include test/libc/stdio/BUILD.mk
include test/libc/BUILD.mk
include test/net/http/BUILD.mk
include test/net/https/BUILD.mk
include test/net/finger/BUILD.mk
include test/net/BUILD.mk
include test/tool/build/lib/BUILD.mk
include test/tool/build/BUILD.mk
include test/tool/viz/lib/BUILD.mk
include test/tool/viz/BUILD.mk
include test/tool/net/BUILD.mk
include test/tool/BUILD.mk
include test/dsp/core/BUILD.mk
include test/dsp/scale/BUILD.mk
include test/dsp/tty/BUILD.mk
include test/dsp/BUILD.mk
include examples/package/lib/BUILD.mk
include examples/package/BUILD.mk
#-φ-examples/package/new.sh
include test/test.mk
include test/BUILD.mk
OBJS = $(foreach x,$(PKGS),$($(x)_OBJS))
SRCS := $(foreach x,$(PKGS),$($(x)_SRCS))
@@ -263,14 +402,14 @@ tags: TAGS HTAGS
o/$(MODE)/.x:
@$(COMPILE) -AMKDIR -tT$@ $(MKDIR) $(@D)
o/$(MODE)/srcs.txt: o/$(MODE)/.x $(MAKEFILES) $(call uniq,$(foreach x,$(SRCS),$(dir $(x))))
o/$(MODE)/srcs.txt: o/$(MODE)/.x $(MAKEFILES) $(call uniq,$(foreach x,$(SRCS),$(dir $(x)))) $(SRCS)
$(file >$@,$(SRCS))
o/$(MODE)/hdrs.txt: o/$(MODE)/.x $(MAKEFILES) $(call uniq,$(foreach x,$(HDRS) $(INCS),$(dir $(x))))
o/$(MODE)/hdrs.txt: o/$(MODE)/.x $(MAKEFILES) $(call uniq,$(foreach x,$(HDRS) $(INCS),$(dir $(x)))) $(HDRS) $(INCS)
$(file >$@,$(HDRS) $(INCS))
o/$(MODE)/incs.txt: o/$(MODE)/.x $(MAKEFILES) $(call uniq,$(foreach x,$(INCS) $(INCS),$(dir $(x))))
o/$(MODE)/incs.txt: o/$(MODE)/.x $(MAKEFILES) $(call uniq,$(foreach x,$(INCS) $(INCS),$(dir $(x)))) $(INCS) $(INCS)
$(file >$@,$(INCS))
o/$(MODE)/depend: o/$(MODE)/.x o/$(MODE)/srcs.txt o/$(MODE)/hdrs.txt o/$(MODE)/incs.txt $(SRCS) $(HDRS) $(INCS)
@$(COMPILE) -AMKDEPS -L320 $(MKDEPS) -o $@ -r o/$(MODE)/ @o/$(MODE)/srcs.txt @o/$(MODE)/hdrs.txt @o/$(MODE)/incs.txt
$(COMPILE) -AMKDEPS -L320 $(MKDEPS) -o $@ -s -r o/$(MODE)/ @o/$(MODE)/srcs.txt @o/$(MODE)/hdrs.txt @o/$(MODE)/incs.txt
o/$(MODE)/srcs-old.txt: o/$(MODE)/.x $(MAKEFILES) $(call uniq,$(foreach x,$(SRCS),$(dir $(x))))
$(file >$@) $(foreach x,$(SRCS),$(file >>$@,$(x)))
@@ -278,46 +417,51 @@ o/$(MODE)/hdrs-old.txt: o/$(MODE)/.x $(MAKEFILES) $(call uniq,$(foreach x,$(HDRS
$(file >$@) $(foreach x,$(HDRS) $(INCS),$(file >>$@,$(x)))
TAGS: private .UNSANDBOXED = 1
TAGS: o/$(MODE)/srcs-old.txt $(SRCS)
TAGS: o/$(MODE)/srcs-old.txt $(SRCS) #o/$(MODE)/third_party/ctags/ctags
@$(RM) $@
@$(TAGS) $(TAGSFLAGS) -L $< -o $@
@o/$(MODE)/third_party/ctags/ctags $(TAGSFLAGS) -L $< -o $@
HTAGS: private .UNSANDBOXED = 1
HTAGS: o/$(MODE)/hdrs-old.txt $(HDRS)
HTAGS: o/$(MODE)/hdrs-old.txt $(filter-out third_party/libcxx/%,$(HDRS)) #o/$(MODE)/third_party/ctags/ctags
@$(RM) $@
@build/htags -L $< -o $@
@build/htags o/$(MODE)/third_party/ctags/ctags -L $< -o $@
loc: private .UNSANDBOXED = 1
loc: o/$(MODE)/tool/build/summy.com
loc: o/$(MODE)/tool/build/summy
find -name \*.h -or -name \*.c -or -name \*.S | \
$(XARGS) wc -l | grep total | awk '{print $$1}' | $<
# PLEASE: MAINTAIN TOPOLOGICAL ORDER
# FROM HIGHEST LEVEL TO LOWEST LEVEL
COSMOPOLITAN_OBJECTS = \
TOOL_ARGS \
NET_HTTP \
LIBC_DNS \
LIBC_SOCK \
LIBC_NT_WS2_32 \
LIBC_NT_IPHLPAPI \
LIBC_NT_MSWSOCK \
LIBC_X \
THIRD_PARTY_GETOPT \
LIBC_LOG \
LIBC_TIME \
LIBC_ZIPOS \
THIRD_PARTY_ZLIB \
THIRD_PARTY_TZ \
THIRD_PARTY_OPENMP \
THIRD_PARTY_MUSL \
THIRD_PARTY_ZLIB_GZ \
THIRD_PARTY_LIBCXXABI \
THIRD_PARTY_LIBUNWIND \
LIBC_STDIO \
THIRD_PARTY_GDTOA \
THIRD_PARTY_REGEX \
LIBC_THREAD \
LIBC_PROC \
THIRD_PARTY_NSYNC_MEM \
CTL \
LIBC_MEM \
THIRD_PARTY_DLMALLOC \
LIBC_DLOPEN \
LIBC_RUNTIME \
THIRD_PARTY_NSYNC \
LIBC_ELF \
LIBC_IRQ \
LIBC_CALLS \
LIBC_SYSV_CALLS \
LIBC_VGA \
@@ -326,28 +470,30 @@ COSMOPOLITAN_OBJECTS = \
LIBC_NT_PDH \
LIBC_NT_GDI32 \
LIBC_NT_COMDLG32 \
LIBC_NT_URL \
LIBC_NT_USER32 \
LIBC_NT_NTDLL \
LIBC_NT_ADVAPI32 \
LIBC_NT_SYNCHRONIZATION \
LIBC_FMT \
THIRD_PARTY_ZLIB \
THIRD_PARTY_PUFF \
THIRD_PARTY_COMPILER_RT \
LIBC_TINYMATH \
THIRD_PARTY_XED \
LIBC_STR \
LIBC_SYSV \
LIBC_INTRIN \
LIBC_NT_BCRYPTPRIMITIVES \
LIBC_NT_KERNEL32 \
LIBC_NEXGEN32E
COSMOPOLITAN_HEADERS = \
COSMOPOLITAN_H_PKGS = \
APE \
LIBC \
LIBC_CALLS \
LIBC_DNS \
LIBC_ELF \
LIBC_FMT \
LIBC_DLOPEN \
LIBC_INTRIN \
LIBC_LOG \
LIBC_MEM \
@@ -356,17 +502,17 @@ COSMOPOLITAN_HEADERS = \
LIBC_RUNTIME \
LIBC_SOCK \
LIBC_STDIO \
LIBC_PROC \
THIRD_PARTY_NSYNC \
THIRD_PARTY_XED \
LIBC_STR \
LIBC_SYSV \
LIBC_THREAD \
LIBC_TIME \
LIBC_TINYMATH \
LIBC_X \
LIBC_ZIPOS \
LIBC_VGA \
NET_HTTP \
TOOL_ARGS \
THIRD_PARTY_DLMALLOC \
THIRD_PARTY_GDTOA \
THIRD_PARTY_GETOPT \
@@ -375,27 +521,41 @@ COSMOPOLITAN_HEADERS = \
THIRD_PARTY_ZLIB_GZ \
THIRD_PARTY_REGEX
o/$(MODE)/cosmopolitan.a: \
COSMOCC_PKGS = \
$(COSMOPOLITAN_H_PKGS) \
CTL \
THIRD_PARTY_AARCH64 \
THIRD_PARTY_LIBCXX \
THIRD_PARTY_LIBCXXABI \
THIRD_PARTY_LIBUNWIND \
THIRD_PARTY_OPENMP \
THIRD_PARTY_INTEL
o/$(MODE)/cosmopolitan.a: \
$(foreach x,$(COSMOPOLITAN_OBJECTS),$($(x)_A_OBJS))
o/cosmopolitan.h: \
o/$(MODE)/tool/build/rollup.com \
libc/integral/normalize.inc \
$(foreach x,$(COSMOPOLITAN_HEADERS),$($(x)_HDRS)) \
$(foreach x,$(COSMOPOLITAN_HEADERS),$($(x)_INCS))
$(file >$(TMPDIR)/$(subst /,_,$@),libc/integral/normalize.inc $(foreach x,$(COSMOPOLITAN_HEADERS),$($(x)_HDRS)))
@$(COMPILE) -AROLLUP -T$@ o/$(MODE)/tool/build/rollup.com @$(TMPDIR)/$(subst /,_,$@) >$@
COSMOCC_HDRS = \
$(wildcard libc/integral/*) \
$(foreach x,$(COSMOCC_PKGS),$($(x)_HDRS)) \
$(foreach x,$(COSMOCC_PKGS),$($(x)_INCS))
o/cosmocc.h.txt: Makefile
$(file >$@, $(call uniq,$(COSMOCC_HDRS)))
COSMOPOLITAN_H_ROOT_HDRS = \
libc/integral/normalize.inc \
$(foreach x,$(COSMOPOLITAN_H_PKGS),$($(x)_HDRS))
o/cosmopolitan.html: private .UNSANDBOXED = 1
o/cosmopolitan.html: \
o/$(MODE)/third_party/chibicc/chibicc.com.dbg \
o/$(MODE)/third_party/chibicc/chibicc.dbg \
$(filter-out %.s,$(foreach x,$(COSMOPOLITAN_OBJECTS),$($(x)_SRCS))) \
$(SRCS) \
$(filter-out %.cpp,$(filter-out %.cc,$(SRCS))) \
$(HDRS)
$(file >$(TMPDIR)/$(subst /,_,$@),$(filter-out %.s,$(foreach x,$(COSMOPOLITAN_OBJECTS),$($(x)_SRCS))))
o/$(MODE)/third_party/chibicc/chibicc.com.dbg -J \
$(file >$(TMPDIR)/$(subst /,_,$@),$(filter-out %.cpp,$(filter-out %.cc,$(filter-out %.s,$(foreach x,$(COSMOPOLITAN_OBJECTS),$($(x)_SRCS))))))
o/$(MODE)/third_party/chibicc/chibicc.dbg -J \
-fno-common -include libc/integral/normalize.inc -o $@ \
@$(TMPDIR)/$(subst /,_,$@)
-DCOSMO @$(TMPDIR)/$(subst /,_,$@)
$(SRCS): \
libc/integral/normalize.inc \
@@ -405,14 +565,49 @@ $(SRCS): \
libc/integral/lp64arg.inc \
libc/integral/lp64.inc
ifeq ($(ARCH), x86_64)
TOOLCHAIN_ARTIFACTS = \
o/$(MODE)/ape/ape.lds \
o/$(MODE)/libc/crt/crt.o \
o/$(MODE)/ape/ape.elf \
o/$(MODE)/ape/ape.o \
o/$(MODE)/ape/ape-copy-self.o \
o/$(MODE)/ape/ape-no-modify-self.o \
o/$(MODE)/cosmopolitan.a \
o/$(MODE)/third_party/libcxx/libcxx.a \
o/$(MODE)/tool/build/march-native \
o/$(MODE)/tool/build/ar \
o/$(MODE)/tool/build/mktemper \
o/$(MODE)/tool/build/fixupobj \
o/$(MODE)/tool/build/zipcopy \
o/$(MODE)/tool/build/apelink \
o/$(MODE)/tool/build/pecheck
else
TOOLCHAIN_ARTIFACTS = \
o/$(MODE)/ape/ape.elf \
o/$(MODE)/ape/aarch64.lds \
o/$(MODE)/libc/crt/crt.o \
o/$(MODE)/cosmopolitan.a \
o/$(MODE)/third_party/libcxx/libcxx.a \
o/$(MODE)/tool/build/march-native \
o/$(MODE)/tool/build/fixupobj \
o/$(MODE)/tool/build/zipcopy
endif
.PHONY: toolchain
toolchain: o/cosmopolitan.h \
o/$(MODE)/ape/public/ape.lds \
o/$(MODE)/libc/crt/crt.o \
o/$(MODE)/ape/ape.o \
o/$(MODE)/ape/ape-copy-self.o \
o/$(MODE)/ape/ape-no-modify-self.o \
o/$(MODE)/cosmopolitan.a
toolchain: $(TOOLCHAIN_ARTIFACTS)
.PHONY: clean_toolchain
clean_toolchain:
$(RM) $(TOOLCHAIN_ARTIFACTS)
aarch64: private .INTERNET = true
aarch64: private .UNSANDBOXED = true
aarch64:
$(MAKE) m=aarch64
clean:
$(RM) -r o
# UNSPECIFIED PREREQUISITES TUTORIAL
#
@@ -438,9 +633,9 @@ $(SRCS):
$(HDRS):
$(INCS):
.DEFAULT:
@$(ECHO) >&2
@$(ECHO) NOTE: deleting o/$(MODE)/depend because of an unspecified prerequisite: $@ >&2
@$(ECHO) >&2
@$(ECHO)
@$(ECHO) NOTE: deleting o/$(MODE)/depend because of an unspecified prerequisite: $@
@$(ECHO)
$(RM) o/$(MODE)/depend
-include o/$(MODE)/depend
+157 -156
View File
@@ -12,202 +12,149 @@ possible performance and the tiniest footprint imaginable.
## Background
For an introduction to this project, please read the [αcτµαlly pδrταblε
εxεcµταblε](https://justine.lol/ape.html) blog post and [cosmopolitan
For an introduction to this project, please read the [actually portable
executable](https://justine.lol/ape.html) blog post and [cosmopolitan
libc](https://justine.lol/cosmopolitan/index.html) website. We also have
[API documentation](https://justine.lol/cosmopolitan/documentation.html).
[API
documentation](https://justine.lol/cosmopolitan/documentation.html).
## Getting Started
If you're doing your development work on Linux or BSD then you need just
five files to get started. Here's what you do on Linux:
You can start by obtaining a release of our `cosmocc` compiler from
<https://cosmo.zip/pub/cosmocc/>.
```sh
wget https://justine.lol/cosmopolitan/cosmopolitan-amalgamation-2.2.zip
unzip cosmopolitan-amalgamation-2.2.zip
printf 'main() { printf("hello world\\n"); }\n' >hello.c
gcc -g -Os -static -nostdlib -nostdinc -fno-pie -no-pie -mno-red-zone \
-fno-omit-frame-pointer -pg -mnop-mcount -mno-tls-direct-seg-refs -gdwarf-4 \
-o hello.com.dbg hello.c -fuse-ld=bfd -Wl,-T,ape.lds -Wl,--gc-sections \
-include cosmopolitan.h crt.o ape-no-modify-self.o cosmopolitan.a
objcopy -S -O binary hello.com.dbg hello.com
mkdir -p cosmocc
cd cosmocc
wget https://cosmo.zip/pub/cosmocc/cosmocc.zip
unzip cosmocc.zip
```
You now have a portable program.
Here's an example program we can write:
```c
// hello.c
#include <stdio.h>
int main() {
printf("hello world\n");
}
```
It can be compiled as follows:
```sh
./hello.com
bash -c './hello.com' # zsh/fish workaround (we patched them in 2021)
cosmocc -o hello hello.c
./hello
```
If `./hello.com` executed on Linux throws an error about not finding an
interpreter, it should be fixed by running the following command (although
note that it may not survive a system restart):
The Cosmopolitan Libc runtime links some heavyweight troubleshooting
features by default, which are very useful for developers and admins.
Here's how you can log system calls:
```sh
sudo sh -c "echo ':APE:M::MZqFpD::/bin/sh:' >/proc/sys/fs/binfmt_misc/register"
./hello --strace
```
If the same command produces puzzling errors on WSL or WINE when using
Redbean 2.x, they may be fixed by disabling binfmt_misc:
Here's how you can get a much more verbose log of function calls:
```sh
sudo sh -c 'echo -1 >/proc/sys/fs/binfmt_misc/status'
./hello --ftrace
```
Since we used the `ape-no-modify-self.o` bootloader (rather than
`ape.o`) your executable will not modify itself when it's run. What
it'll instead do, is extract a 4kb program to `${TMPDIR:-${HOME:-.}}`
that maps your program into memory without needing to copy it. It's
possible to install the APE loader systemwide as follows.
You can use the Cosmopolitan's toolchain to build conventional open
source projects which use autotools. This strategy normally works:
```sh
# System-Wide APE Install
# for Linux, Darwin, and BSDs
# 1. Copies APE Loader to /usr/bin/ape
# 2. Registers w/ binfmt_misc too if Linux
ape/apeinstall.sh
# System-Wide APE Uninstall
# for Linux, Darwin, and BSDs
ape/apeuninstall.sh
export CC=x86_64-unknown-cosmo-cc
export CXX=x86_64-unknown-cosmo-c++
./configure --prefix=/opt/cosmos/x86_64
make -j
make install
```
It's also possible to convert APE binaries into the system-local format
by using the `--assimilate` flag. Please note that if binfmt_misc is in
play, you'll need to unregister it temporarily before doing this, since
the assimilate feature is part of the shell script header.
## Cosmopolitan Source Builds
```sh
$ file hello.com
hello.com: DOS/MBR boot sector
./hello.com --assimilate
$ file hello.com
hello.com: ELF 64-bit LSB executable
```
Cosmopolitan can be compiled from source on any of our supported
platforms. The Makefile will download cosmocc automatically.
Now that you're up and running with Cosmopolitan Libc and APE, here's
some of the most important troubleshooting tools APE offers that you
should know, in case you encounter any issues:
```sh
./hello.com --strace # log system calls to stderr
./hello.com --ftrace # log function calls to stderr
```
Do you love tiny binaries? If so, you may not be happy with Cosmo adding
heavyweight features like tracing to your binaries by default. In that
case, you may want to consider using our build system:
```sh
make m=tiny
```
Which will cause programs such as `hello.com` and `life.com` to shrink
from 60kb in size to about 16kb. There's also a prebuilt amalgamation
online <https://justine.lol/cosmopolitan/cosmopolitan-tiny.zip> hosted
on our download page <https://justine.lol/cosmopolitan/download.html>.
### MacOS
If you're developing on MacOS you can install the GNU compiler
collection for x86_64-elf via homebrew:
```sh
brew install x86_64-elf-gcc
```
Then in the above scripts just replace `gcc` and `objcopy` with
`x86_64-elf-gcc` and `x86_64-elf-objcopy` to compile your APE binary.
### Windows
If you're developing on Windows then you need to download an
x86_64-pc-linux-gnu toolchain beforehand. See the [Compiling on
Windows](https://justine.lol/cosmopolitan/windows-compiling.html)
tutorial. It's needed because the ELF object format is what makes
universal binaries possible.
Cosmopolitan officially only builds on Linux. However, one highly
experimental (and currently broken) thing you could try, is building the
entire cosmo repository from source using the cross9 toolchain.
```
mkdir -p o/third_party
rm -rf o/third_party/gcc
wget https://justine.lol/linux-compiler-on-windows/cross9.zip
unzip cross9.zip
mv cross9 o/third_party/gcc
build/bootstrap/make.com
```
## Source Builds
Cosmopolitan can be compiled from source on any Linux distro. First, you
need to download or clone the repository.
```sh
wget https://justine.lol/cosmopolitan/cosmopolitan.tar.gz
tar xf cosmopolitan.tar.gz # see releases page
cd cosmopolitan
```
This will build the entire repository and run all the tests:
```sh
build/bootstrap/make.com
o//examples/hello.com
find o -name \*.com | xargs ls -rShal | less
```
If you get an error running make.com then it's probably because you have
WINE installed to `binfmt_misc`. You can fix that by installing the the
APE loader as an interpreter. It'll improve build performance too!
It's recommended that you install a systemwide APE Loader. This command
requires `sudo` access to copy the `ape` command to a system folder and
register with binfmt_misc on Linux, for even more performance.
```sh
ape/apeinstall.sh
```
You can now build the mono repo with any modern version of GNU Make. To
make life easier, we've included one in the cosmocc toolchain, which is
guaranteed to be compatible and furthermore includes our extensions for
doing build system sandboxing.
```sh
build/bootstrap/make -j8
o//examples/hello
```
Since the Cosmopolitan repository is very large, you might only want to
build a particular thing. Cosmopolitan's build config does a good job at
having minimal deterministic builds. For example, if you wanted to build
only hello.com then you could do that as follows:
build one particular thing. Here's an example of a target that can be
compiled relatively quickly, which is a simple POSIX test that only
depends on core LIBC packages.
```sh
build/bootstrap/make.com o//examples/hello.com
rm -rf o//libc o//test
build/bootstrap/make o//test/posix/signal_test
o//test/posix/signal_test
```
Sometimes it's desirable to build a subset of targets, without having to
list out each individual one. You can do that by asking make to build a
directory name. For example, if you wanted to build only the targets and
subtargets of the chibicc package including its tests, you would say:
list out each individual one. For example if you wanted to build and run
all the unit tests in the `TEST_POSIX` package, you could say:
```sh
build/bootstrap/make.com o//third_party/chibicc
o//third_party/chibicc/chibicc.com --help
build/bootstrap/make o//test/posix
```
Cosmopolitan provides a variety of build modes. For example, if you want
really tiny binaries (as small as 12kb in size) then you'd say:
```sh
build/bootstrap/make.com m=tiny
build/bootstrap/make m=tiny
```
Here's some other build modes you can try:
You can furthermore cut out the bloat of other operating systems, and
have Cosmopolitan become much more similar to Musl Libc.
```sh
build/bootstrap/make.com m=dbg # asan + ubsan + debug
build/bootstrap/make.com m=asan # production memory safety
build/bootstrap/make.com m=opt # -march=native optimizations
build/bootstrap/make.com m=rel # traditional release binaries
build/bootstrap/make.com m=optlinux # optimal linux-only performance
build/bootstrap/make.com m=fastbuild # build 28% faster w/o debugging
build/bootstrap/make.com m=tinylinux # tiniest linux-only 4kb binaries
build/bootstrap/make m=tinylinux
```
For further details, see [//build/config.mk](build/config.mk).
## Debugging
To print a log of system calls to stderr:
```sh
cosmocc -o hello hello.c
./hello --strace
```
To print a log of function calls to stderr:
```sh
cosmocc -o hello hello.c
./hello --ftrace
```
Both strace and ftrace use the unbreakable kprintf() facility, which is
able to be sent to a file by setting an environment variable.
```sh
export KPRINTF_LOG=log
./hello --strace
```
## GDB
Here's the recommended `~/.gdbinit` config:
@@ -232,25 +179,79 @@ end
src
```
You normally run the `.com.dbg` file under gdb. If you need to debug the
`.com` file itself, then you can load the debug symbols independently as
You normally run the `.dbg` file under gdb. If you need to debug the
`` file itself, then you can load the debug symbols independently as
```sh
gdb foo -ex 'add-symbol-file foo.dbg 0x401000'
```
gdb foo.com -ex 'add-symbol-file foo.com.dbg 0x401000'
## Platform Notes
### Shells
If you use zsh and have trouble running APE programs try `sh -c ./prog`
or simply upgrade to zsh 5.9+ (since we patched it two years ago). The
same is the case for Python `subprocess`, old versions of fish, etc.
### Linux
Some Linux systems are configured to launch MZ executables under WINE.
Other distros configure their stock installs so that APE programs will
print "run-detectors: unable to find an interpreter". For example:
```sh
jart@ubuntu:~$ wget https://cosmo.zip/pub/cosmos/bin/dash
jart@ubuntu:~$ chmod +x dash
jart@ubuntu:~$ ./dash
run-detectors: unable to find an interpreter for ./dash
```
You can fix that by registering APE with `binfmt_misc`:
```sh
sudo wget -O /usr/bin/ape https://cosmo.zip/pub/cosmos/bin/ape-$(uname -m).elf
sudo chmod +x /usr/bin/ape
sudo sh -c "echo ':APE:M::MZqFpD::/usr/bin/ape:' >/proc/sys/fs/binfmt_misc/register"
sudo sh -c "echo ':APE-jart:M::jartsr::/usr/bin/ape:' >/proc/sys/fs/binfmt_misc/register"
```
You should be good now. APE will not only work, it'll launch executables
400µs faster now too. However if things still didn't work out, it's also
possible to disable `binfmt_misc` as follows:
```sh
sudo sh -c 'echo -1 > /proc/sys/fs/binfmt_misc/cli' # remove Ubuntu's MZ interpreter
sudo sh -c 'echo -1 > /proc/sys/fs/binfmt_misc/status' # remove ALL binfmt_misc entries
```
### WSL
It's normally unsafe to use APE in a WSL environment, because it tries
to run MZ executables as WIN32 binaries within the WSL environment. In
order to make it safe to use Cosmopolitan software on WSL, run this:
```sh
sudo sh -c "echo -1 > /proc/sys/fs/binfmt_misc/WSLInterop"
```
## Discord Chatroom
The Cosmopolitan development team collaborates on the Redbean Discord
server. You're welcome to join us! <https://discord.gg/FwAVVu7eJ4>
## Support Vector
| Platform | Min Version | Circa |
| :--- | ---: | ---: |
| AMD | K8 Venus | 2005 |
| Intel | Core | 2006 |
| Linux | 2.6.18 | 2007 |
| Windows | 8 [1] | 2012 |
| Mac OS X | 15.6 | 2018 |
| OpenBSD | 6.4 | 2018 |
| FreeBSD | 13 | 2020 |
| NetBSD | 9.2 | 2021 |
| Platform | Min Version | Circa |
| :--- | ---: | ---: |
| AMD | K8 Venus | 2005 |
| Intel | Core | 2006 |
| Linux | 2.6.18 | 2007 |
| Windows | 8 [1] | 2012 |
| Darwin (macOS) | 23.1.0+ | 2023 |
| OpenBSD | 7 | 2021 |
| FreeBSD | 13 | 2020 |
| NetBSD | 9.2 | 2021 |
[1] See our [vista branch](https://github.com/jart/cosmopolitan/tree/vista)
for a community supported version of Cosmopolitan that works on Windows
+289
View File
@@ -0,0 +1,289 @@
#-*-mode:makefile-gmake;indent-tabs-mode:t;tab-width:8;coding:utf-8-*-┐
#── vi: set noet ft=make ts=8 sw=8 fenc=utf-8 :vi ────────────────────┘
#
# OVERVIEW
#
# αcτµαlly pδrταblε εxεcµταblε
#
# DESCRIPTION
#
# This file defines the libraries, runtimes, and build rules needed to
# create executables from your Linux workstation that'll run anywhere.
# Loading this package will make certain systemic modifications to the
# build like turning off the System V "Red Zone" optimization, because
# αcτµαlly pδrταblε εxεcµταblεs need to be able to run in kernelspace.
PKGS += APE
APE_FILES := $(wildcard ape/*.*)
APE_HDRS = $(filter %.h,$(APE_FILES))
APE_INCS = $(filter %.inc,$(APE_FILES))
ifeq ($(ARCH), aarch64)
APE = o/$(MODE)/ape/aarch64.lds
APELINK = \
$(COMPILE) \
-ALINK.ape \
$(LINK) \
$(LINKARGS) \
$(OUTPUT_OPTION) && \
$(COMPILE) \
-AFIXUP.ape \
-wT$@ \
$(FIXUPOBJ) \
$@
APE_SRCS = ape/ape.S ape/start.S ape/launch.S ape/systemcall.S
APE_OBJS = o/$(MODE)/ape/ape.o
APE_NO_MODIFY_SELF = $(APE)
APE_COPY_SELF = $(APE)
.PHONY: o/$(MODE)/ape
o/$(MODE)/ape: $(APE)
o/$(MODE)/ape/aarch64.lds: \
ape/aarch64.lds \
libc/zip.internal.h \
libc/thread/tls.h \
libc/calls/struct/timespec.h \
libc/macros.internal.h \
libc/str/str.h
APE_LOADER_LDFLAGS = \
-pie \
-static \
-nostdlib \
--no-dynamic-linker \
-z norelro \
-z common-page-size=0x4000 \
-z max-page-size=0x4000
APE_LOADER_FLAGS = \
-DNDEBUG \
-iquote. \
-Wall \
-Wextra \
-fpie \
-Os \
-ffreestanding \
-mgeneral-regs-only \
-fno-asynchronous-unwind-tables \
-fno-stack-protector \
-fno-ident \
-fno-gnu-unique \
-c \
$(OUTPUT_OPTION) \
$<
o/$(MODE)/ape/ape.elf: o/$(MODE)/ape/ape.elf.dbg
@$(COMPILE) -AOBJCOPY -T$@ $(OBJCOPY) -g $< $@
@$(COMPILE) -AFIXUPOBJ -wT$@ $(FIXUPOBJ) $@
o/$(MODE)/ape/ape.elf.dbg: \
o/$(MODE)/ape/start.o \
o/$(MODE)/ape/loader.o \
o/$(MODE)/ape/launch.o \
o/$(MODE)/ape/systemcall.o
@$(COMPILE) -ALINK.elf $(LD) $(APE_LOADER_LDFLAGS) -o $@ $(patsubst %.lds,-T %.lds,$^)
o/$(MODE)/ape/loader.o: ape/loader.c ape/ape.h
@$(COMPILE) -AOBJECTIFY.c $(CC) -DSUPPORT_VECTOR=33 -g $(APE_LOADER_FLAGS)
o/$(MODE)/ape/start.o: ape/start.S
@$(COMPILE) -AOBJECTIFY.S $(OBJECTIFY.S) $(OUTPUT_OPTION) -c $<
o/$(MODE)/ape/launch.o: ape/launch.S
@$(COMPILE) -AOBJECTIFY.S $(OBJECTIFY.S) $(OUTPUT_OPTION) -c $<
o/$(MODE)/ape/systemcall.o: ape/systemcall.S
@$(COMPILE) -AOBJECTIFY.S $(OBJECTIFY.S) $(OUTPUT_OPTION) -c $<
.PHONY: o/$(MODE)/ape
o/$(MODE)/ape: o/$(MODE)/ape/ape.elf
else
APE = o/$(MODE)/ape/ape.o \
o/$(MODE)/ape/ape.lds
APELINK = \
$(COMPILE) \
-ALINK.ape \
$(LINK) \
$(LINKARGS) \
$(OUTPUT_OPTION) && \
$(COMPILE) \
-AFIXUP.ape \
-wT$@ \
$(FIXUPOBJ) \
$@
APE_NO_MODIFY_SELF = \
o/$(MODE)/ape/ape.lds \
o/$(MODE)/ape/ape-no-modify-self.o
APE_COPY_SELF = \
o/$(MODE)/ape/ape.lds \
o/$(MODE)/ape/ape-copy-self.o
APE_LOADER_FLAGS = \
-DNDEBUG \
-iquote. \
-Wall \
-Wextra \
-Werror \
-pedantic-errors \
-fpie \
-Os \
-ffreestanding \
-mgeneral-regs-only \
-fno-stack-protector \
-fno-ident \
-fno-gnu-unique \
-c \
$(OUTPUT_OPTION) \
$<
APE_SRCS_C = ape/loader.c
APE_SRCS_S = $(filter %.S,$(APE_FILES))
APE_SRCS = $(APE_SRCS_C) $(APE_SRCS_S)
APE_OBJS = $(APE_SRCS_S:%.S=o/$(MODE)/%.o)
APE_CHECKS = $(APE_HDRS:%=o/%.ok)
o/ape/idata.inc: \
ape/idata.internal.h \
ape/relocations.h
o/$(MODE)/ape/ape-no-modify-self.o: \
ape/ape.S \
ape/ape.h \
ape/macros.internal.h \
ape/relocations.h \
ape/ape.internal.h \
libc/dce.h \
libc/elf/def.h \
libc/thread/tls.h \
libc/macho.internal.h \
libc/macros.internal.h \
libc/nexgen32e/uart.internal.h \
libc/calls/metalfile.internal.h \
libc/nt/pedef.internal.h \
libc/runtime/e820.internal.h \
libc/runtime/mman.internal.h \
libc/runtime/pc.internal.h \
libc/sysv/consts/prot.h \
o/$(MODE)/ape/ape.elf
@$(COMPILE) \
-AOBJECTIFY.S \
$(OBJECTIFY.S) \
$(OUTPUT_OPTION) \
-DAPE_NO_MODIFY_SELF \
-DAPE_LOADER='"o/$(MODE)/ape/ape.elf"' $<
o/$(MODE)/ape/ape-copy-self.o: \
ape/ape.S \
ape/ape.h \
ape/macros.internal.h \
ape/relocations.h \
ape/ape.internal.h \
libc/dce.h \
libc/elf/def.h \
libc/thread/tls.h \
libc/macho.internal.h \
libc/macros.internal.h \
libc/nexgen32e/uart.internal.h \
libc/calls/metalfile.internal.h \
libc/nt/pedef.internal.h \
libc/runtime/e820.internal.h \
libc/runtime/mman.internal.h \
libc/runtime/pc.internal.h \
libc/sysv/consts/prot.h
@$(COMPILE) \
-AOBJECTIFY.S \
$(OBJECTIFY.S) \
$(OUTPUT_OPTION) \
-DAPE_NO_MODIFY_SELF $<
o/$(MODE)/ape/loader.o: ape/loader.c
@$(COMPILE) -AOBJECTIFY.c $(CC) -DSUPPORT_VECTOR=121 -g $(APE_LOADER_FLAGS)
o/$(MODE)/ape/loader-gcc.asm: ape/loader.c
@$(COMPILE) -AOBJECTIFY.c $(CC) -DSUPPORT_VECTOR=121 -S -g0 $(APE_LOADER_FLAGS)
o/$(MODE)/ape/loader-clang.asm: ape/loader.c
@$(COMPILE) -AOBJECTIFY.c $(CLANG) -DSUPPORT_VECTOR=121 -S -g0 $(APE_LOADER_FLAGS)
o/$(MODE)/ape/loader-xnu.o: ape/loader.c
@$(COMPILE) -AOBJECTIFY.c $(CC) -DSUPPORT_VECTOR=8 -g $(APE_LOADER_FLAGS)
o/$(MODE)/ape/loader-xnu-gcc.asm: ape/loader.c
@$(COMPILE) -AOBJECTIFY.c $(CC) -DSUPPORT_VECTOR=8 -S -g0 $(APE_LOADER_FLAGS)
o/$(MODE)/ape/loader-xnu-clang.asm: ape/loader.c
@$(COMPILE) -AOBJECTIFY.c $(CLANG) -DSUPPORT_VECTOR=8 -S -g0 $(APE_LOADER_FLAGS)
o/$(MODE)/ape/ape.elf: o/$(MODE)/ape/ape.elf.dbg
@$(COMPILE) -AOBJBINCOPY -w build/bootstrap/objbincopy -f -o $@ $<
o/$(MODE)/ape/ape.macho: o/$(MODE)/ape/ape.elf.dbg
@$(COMPILE) -AOBJBINCOPY -w build/bootstrap/objbincopy -fm -o $@ $<
APE_LOADER_LDFLAGS = \
-static \
-nostdlib \
--no-dynamic-linker \
-z separate-code \
-z common-page-size=0x1000 \
-z max-page-size=0x10000
o/$(MODE)/ape/ape.elf.dbg: \
o/$(MODE)/ape/loader-macho.o \
o/$(MODE)/ape/start.o \
o/$(MODE)/ape/loader.o \
o/$(MODE)/ape/launch.o \
o/$(MODE)/ape/systemcall.o \
ape/loader.lds
@$(COMPILE) -ALINK.elf $(LD) $(APE_LOADER_LDFLAGS) -o $@ $(patsubst %.lds,-T %.lds,$^)
.PHONY: o/$(MODE)/ape
o/$(MODE)/ape: $(APE_CHECKS) \
o/$(MODE)/ape/ape.o \
o/$(MODE)/ape/ape.lds \
o/$(MODE)/ape/ape.elf \
o/$(MODE)/ape/ape.macho \
o/$(MODE)/ape/ape-copy-self.o \
o/$(MODE)/ape/ape-no-modify-self.o
endif
# these assembly files are safe to build on aarch64
o/$(MODE)/ape/ape.o: \
ape/ape.S \
ape/ape.h \
libc/dce.h \
libc/elf/def.h \
ape/relocations.h \
libc/thread/tls.h \
ape/ape.internal.h \
ape/macros.internal.h \
libc/macho.internal.h \
libc/macros.internal.h \
libc/sysv/consts/prot.h \
libc/nt/pedef.internal.h \
libc/runtime/pc.internal.h \
libc/runtime/e820.internal.h \
libc/runtime/mman.internal.h \
libc/nexgen32e/uart.internal.h \
libc/calls/metalfile.internal.h
@$(COMPILE) -AOBJECTIFY.S $(OBJECTIFY.S) $(OUTPUT_OPTION) -c $<
o/$(MODE)/ape/ape.lds: \
ape/ape.lds \
ape/macros.internal.h \
ape/relocations.h \
ape/ape.internal.h \
libc/thread/tls.h \
libc/calls/struct/timespec.h \
libc/thread/thread.h \
libc/dce.h \
libc/elf/def.h \
libc/elf/pf2prot.internal.h \
libc/macros.internal.h \
libc/nt/pedef.internal.h \
libc/str/str.h \
libc/zip.internal.h
+303
View File
@@ -0,0 +1,303 @@
/*-*- mode: ld-script; indent-tabs-mode: nil; tab-width: 2; coding: utf-8 -*-│
│ vi: set et sts=2 sw=2 fenc=utf-8 :vi │
╚─────────────────────────────────────────────────────────────────────────────*/
#include "libc/sysv/consts/prot.h"
#include "libc/thread/tls.h"
ENTRY(_start)
OUTPUT_ARCH(aarch64)
OUTPUT_FORMAT("elf64-littleaarch64",
"elf64-bigaarch64",
"elf64-littleaarch64")
SECTIONS {
. = SEGMENT_START("text-segment", 0x010000000000);
__executable_start = .;
. += SIZEOF_HEADERS;
.interp : { *(.interp) }
.note.gnu.build-id : { *(.note.gnu.build-id) }
.hash : { *(.hash) }
.gnu.hash : { *(.gnu.hash) }
.dynsym : { *(.dynsym) }
.dynstr : { *(.dynstr) }
.gnu.version : { *(.gnu.version) }
.gnu.version_d : { *(.gnu.version_d) }
.gnu.version_r : { *(.gnu.version_r) }
.rela.init : { *(.rela.init) }
.rela.text : { *(.rela.text .rela.text.* .rela.gnu.linkonce.t.*) }
.rela.fini : { *(.rela.fini) }
.rela.rodata : { *(.rela.rodata .rela.rodata.* .rela.gnu.linkonce.r.*) }
.rela.data.rel.ro : { *(.rela.data.rel.ro .rela.data.rel.ro.* .rela.gnu.linkonce.d.rel.ro.*) }
.rela.data : { *(.rela.data .rela.data.* .rela.gnu.linkonce.d.*) }
.rela.tdata : { *(.rela.tdata .rela.tdata.* .rela.gnu.linkonce.td.*) }
.rela.tbss : { *(.rela.tbss .rela.tbss.* .rela.gnu.linkonce.tb.*) }
.rela.ctors : { *(.rela.ctors) }
.rela.dtors : { *(.rela.dtors) }
.rela.got : { *(.rela.got) }
.rela.bss : { *(.rela.bss .rela.bss.* .rela.gnu.linkonce.b.*) }
.rela.ifunc : { *(.rela.ifunc) }
.rela.plt : {
*(.rela.plt)
__rela_iplt_start = .;
*(.rela.iplt)
__rela_iplt_end = .;
}
.init : {
*(.start)
KEEP(*(.initprologue))
KEEP(*(SORT_NONE(.init)))
KEEP(*(.initepilogue))
} =0x1f2003d5
.plt : ALIGN(16) {
*(.plt)
*(.iplt)
}
.text : {
*(.text.unlikely .text.*_unlikely .text.unlikely.*)
*(.text.antiquity .text.antiquity.*)
*(.text.exit .text.exit.*)
*(.text.startup .text.startup.*)
*(.text.hot .text.hot.*)
*(.text.modernity .text.modernity.*)
*(.text .stub .text.* .gnu.linkonce.t.*)
*(.gnu.warning)
} =0x1f2003d5
.fini : {
KEEP(*(SORT_NONE(.fini)))
} =0x1f2003d5
. += CONSTANT(MAXPAGESIZE);
.privileged : {
__privileged_start = ABSOLUTE(.) & -CONSTANT(MAXPAGESIZE);
*(.privileged*)
} =0x1f2003d6
.rodata : {
KEEP(*(.rodata.pytab.0));
KEEP(*(.rodata.pytab.1));
KEEP(*(.rodata.pytab.2));
KEEP(*(SORT_BY_NAME(.sort.rodata.*)))
*(.rodata .rodata.* .gnu.linkonce.r.*)
*(.ubsan.types)
*(.ubsan.data)
}
.notice : {
__notices = .;
KEEP(*(.notice))
BYTE(0);
BYTE(10);
BYTE(10);
}
.eh_frame_hdr : {
*(.eh_frame_hdr)
*(.eh_frame_entry .eh_frame_entry.*)
}
.eh_frame : ONLY_IF_RO {
KEEP(*(.eh_frame))
*(.eh_frame.*)
}
.gcc_except_table : ONLY_IF_RO {
*(.gcc_except_table .gcc_except_table.*)
}
.gnu_extab : ONLY_IF_RO {
*(.gnu_extab*)
}
.exception_ranges : ONLY_IF_RO {
*(.exception_ranges*)
}
__etext = .;
_etext = .;
PROVIDE(etext = .);
. += CONSTANT(MAXPAGESIZE);
. = DATA_SEGMENT_ALIGN(CONSTANT(MAXPAGESIZE), CONSTANT(COMMONPAGESIZE));
.eh_frame : ONLY_IF_RW {
KEEP(*(.eh_frame))
*(.eh_frame.*)
}
.gnu_extab : ONLY_IF_RW {
*(.gnu_extab)
}
.gcc_except_table : ONLY_IF_RW {
*(.gcc_except_table .gcc_except_table.*)
}
.exception_ranges : ONLY_IF_RW {
*(.exception_ranges*)
}
.tdata : {
_tdata_start = .;
__tdata_start = .;
*(.tdata .tdata.* .gnu.linkonce.td.*)
_tdata_end = .;
}
.tbss : {
_tbss_start = .;
*(.tbss .tbss.* .gnu.linkonce.tb.*)
*(.tcommon)
_tbss_end = .;
}
.init_array : {
__init_array_start = .;
KEEP(*(.preinit_array))
KEEP(*(SORT_BY_INIT_PRIORITY(.init_array.*)
SORT_BY_INIT_PRIORITY(.ctors.*)))
KEEP(*(.init_array))
KEEP(*(.ctors))
__init_array_end = .;
}
.fini_array : {
__fini_array_start = .;
KEEP(*(SORT_BY_INIT_PRIORITY(.fini_array.*) SORT_BY_INIT_PRIORITY(.dtors.*)))
KEEP(*(.fini_array EXCLUDE_FILE(*crtbegin.o *crtbegin?.o *crtend.o *crtend?.o ) .dtors))
__fini_array_end = .;
}
.data.rel.ro : {
KEEP(*(SORT_BY_NAME(.piro.relo.sort.*)))
*(.data.rel.ro.local* .gnu.linkonce.d.rel.ro.local.*)
*(.data.rel.ro .data.rel.ro.* .gnu.linkonce.d.rel.ro.*)
}
.dynamic : {
*(.dynamic)
}
.got : {
*(.got)
*(.igot)
}
. = DATA_SEGMENT_RELRO_END(24, .);
.got.plt : {
*(.got.plt)
*(.igot.plt)
}
.data : {
__data_start = .;
KEEP(*(SORT_BY_NAME(.piro.data.sort.*)))
*(.data .data.* .gnu.linkonce.d.*)
KEEP(*(SORT_BY_NAME(.sort.data.*)))
SORT(CONSTRUCTORS)
}
_edata = .;
PROVIDE(edata = .);
. = .;
__bss_start = .;
__bss_start__ = .;
.bss : {
*(.dynbss)
*(.bss .bss.* .gnu.linkonce.b.*)
KEEP(*(SORT_BY_NAME(.piro.bss.sort.*)))
*(COMMON)
. = ALIGN(CONSTANT(COMMONPAGESIZE));
}
_bss_end__ = .;
__bss_end__ = .;
. = ALIGN(64 / 8);
. = SEGMENT_START("ldata-segment", .);
. = ALIGN(64 / 8);
__end__ = .;
_end = .;
PROVIDE(end = .);
. = DATA_SEGMENT_END(.);
.stab 0 : { *(.stab) }
.stabstr 0 : { *(.stabstr) }
.stab.excl 0 : { *(.stab.excl) }
.stab.exclstr 0 : { *(.stab.exclstr) }
.stab.index 0 : { *(.stab.index) }
.stab.indexstr 0 : { *(.stab.indexstr) }
.gnu.build.attributes : { *(.gnu.build.attributes .gnu.build.attributes.*) }
.debug 0 : { *(.debug) }
.line 0 : { *(.line) }
.debug_srcinfo 0 : { *(.debug_srcinfo) }
.debug_sfnames 0 : { *(.debug_sfnames) }
.debug_aranges 0 : { *(.debug_aranges) }
.debug_pubnames 0 : { *(.debug_pubnames) }
.debug_info 0 : { *(.debug_info .gnu.linkonce.wi.*) }
.debug_abbrev 0 : { *(.debug_abbrev) }
.debug_line 0 : { *(.debug_line .debug_line.* .debug_line_end) }
.debug_frame 0 : { *(.debug_frame) }
.debug_str 0 : { *(.debug_str) }
.debug_loc 0 : { *(.debug_loc) }
.debug_macinfo 0 : { *(.debug_macinfo) }
.debug_weaknames 0 : { *(.debug_weaknames) }
.debug_funcnames 0 : { *(.debug_funcnames) }
.debug_typenames 0 : { *(.debug_typenames) }
.debug_varnames 0 : { *(.debug_varnames) }
.debug_pubtypes 0 : { *(.debug_pubtypes) }
.debug_ranges 0 : { *(.debug_ranges) }
.debug_macro 0 : { *(.debug_macro) }
.debug_addr 0 : { *(.debug_addr) }
.ARM.attributes 0 : { KEEP(*(.ARM.attributes)) KEEP(*(.gnu.attributes)) }
.note.gnu.arm.ident 0 : { KEEP(*(.note.gnu.arm.ident)) }
.zip 0 : {
KEEP(*(.zip.file))
__zip_cdir_start = .;
KEEP(*(.zip.cdir))
__zip_cdir_size = . - __zip_cdir_start;
KEEP(*(.zip.eocd))
}
/DISCARD/ : {
*(__patchable_function_entries)
*(.GCC.command.line)
*(.note.GNU-stack)
*(.gnu_debuglink)
*(.text.windows)
*(.gnu.lto_*)
*(.eh_frame)
*(.idata.*)
*(.yoink)
*(.head)
}
}
ape_stack_vaddr = DEFINED(ape_stack_vaddr) ? ape_stack_vaddr : 0x700000000000;
ape_stack_memsz = DEFINED(ape_stack_memsz) ? ape_stack_memsz : 8 * 1024 * 1024;
ape_stack_align = DEFINED(ape_stack_align) ? MAX(ape_stack_align, 16) : 16;
ape_stack_prot = PROT_READ | PROT_WRITE;
ape_stack_round = -ape_stack_align;
ASSERT(ape_stack_align < 1048576, "stack align too big");
ASSERT(!(ape_stack_align & (ape_stack_align - 1)), "stack align must be two power");
_tls_size = _tbss_end - _tdata_start;
_tdata_size = _tdata_end - _tdata_start;
_tbss_size = _tbss_end - _tbss_start;
_tbss_offset = _tbss_start - _tdata_start;
_tls_content = (_tdata_end - _tdata_start) + (_tbss_end - _tbss_start);
_tdata_align = ALIGNOF(.tdata);
_tbss_align = ALIGNOF(.tbss);
_tls_align = MAX(TLS_ALIGNMENT, MAX(ALIGNOF(.tdata), ALIGNOF(.tbss)));
+1133
View File
File diff suppressed because it is too large. Load diff
+665 -524
View File
File diff suppressed because it is too large. Load diff
+13
View File
@@ -0,0 +1,13 @@
#ifndef COSMOPOLITAN_APE_APE_H_
#define COSMOPOLITAN_APE_APE_H_
#define APE_VERSION_MAJOR 1
#define APE_VERSION_MINOR 10
#define APE_VERSION_STR APE_VERSION_STR_(APE_VERSION_MAJOR, APE_VERSION_MINOR)
#define APE_VERSION_NOTE APE_VERSION_NOTE_(APE_VERSION_MAJOR, APE_VERSION_MINOR)
#define APE_VERSION_STR__(x, y) #x "." #y
#define APE_VERSION_STR_(x, y) APE_VERSION_STR__(x, y)
#define APE_VERSION_NOTE_(x, y) (100000000 * (x) + 1000000 * (y))
#endif /* COSMOPOLITAN_APE_APE_H_ */
+9
View File
@@ -0,0 +1,9 @@
#ifndef COSMOPOLITAN_APE_APE_INTERNAL_H_
#define COSMOPOLITAN_APE_APE_INTERNAL_H_
#include "libc/dce.h"
#if SupportsWindows() || SupportsMetal() || SupportsXnu()
#define APE_IS_SHELL_SCRIPT
#endif
#endif /* COSMOPOLITAN_APE_APE_INTERNAL_H_ */
+261 -227
View File
@@ -1,5 +1,5 @@
/*-*- mode: ld-script; indent-tabs-mode: nil; tab-width: 2; coding: utf-8 -*-│
│vi: set et sts=2 tw=2 fenc=utf-8 :vi│
│ vi: set et sts=2 sw=2 fenc=utf-8 :vi │
╞══════════════════════════════════════════════════════════════════════════════╡
│ Copyright 2020 Justine Alexandra Roberts Tunney │
│ │
@@ -181,16 +181,39 @@
#include "libc/elf/pf2prot.internal.h"
#include "libc/nt/pedef.internal.h"
#include "libc/thread/tls.h"
#include "libc/zip.h"
#include "ape/ape.internal.h"
/* uncomment if .com.dbg won't execute on your kernel (will break .com file) */
/* #define APE_FIX_COM_DBG */
#ifdef __x86__
#define CODE_GRANULE 1
#else
#define CODE_GRANULE 4
#endif
#ifdef APE_FIX_COM_DBG
#define SKEW SIZEOF_HEADERS
#else
#define SKEW 0
#endif
#ifndef IMAGE_BASE_VIRTUAL
#define IMAGE_BASE_VIRTUAL 0x400000
#endif
#if IMAGE_BASE_VIRTUAL > 0xffffffff
#error "please use 32-bit addresses for image data"
#endif
ENTRY(_start)
PHDRS {
Head PT_LOAD FLAGS(PF_X|PF_R);
Rom PT_LOAD FLAGS(PF_X|PF_R);
Ram PT_LOAD FLAGS(PF_W|PF_R);
Cod PT_LOAD FLAGS(PF_X|PF_R);
Rom PT_LOAD FLAGS(PF_R);
Tls PT_TLS FLAGS(PF_W|PF_R);
Bss PT_LOAD FLAGS(PF_W|PF_R);
Ram PT_LOAD FLAGS(PF_W|PF_R);
stack PT_GNU_STACK FLAGS(PF_W|PF_R);
}
@@ -201,42 +224,44 @@ SECTIONS {
/*BEGIN: linux addressability guarantee */
/*BEGIN: bsd addressability guarantee */
.head SEGMENT_START("text-segment", IMAGE_BASE_VIRTUAL) : AT(IMAGE_BASE_REAL) {
HIDDEN(_base = .);
.head SEGMENT_START("text-segment", IMAGE_BASE_VIRTUAL) + SKEW : AT(IMAGE_BASE_REAL) {
__executable_start = .;
/* Real Mode */
KEEP(*(.head))
KEEP(*(.apesh))
KEEP(*(.head2))
KEEP(*(.text.head))
/* Executable & Linkable Format */
. = ALIGN(__SIZEOF_POINTER__);
HIDDEN(ape_phdrs = .);
. = ALIGN(. != 0 ? __SIZEOF_POINTER__ : 0);
ape_phdrs = .;
KEEP(*(.elf.phdrs))
HIDDEN(ape_phdrs_end = .);
ape_phdrs_end = .;
/* OpenBSD */
. = ALIGN(__SIZEOF_POINTER__);
HIDDEN(ape_note = .);
. = ALIGN(. != 0 ? __SIZEOF_POINTER__ : 0);
ape_note = .;
KEEP(*(.note.openbsd.ident))
KEEP(*(.note.netbsd.ident))
HIDDEN(ape_note_end = .);
ape_note_end = .;
/* Portable Executable */
KEEP(*(.pe.header))
HIDDEN(ape_pe_sections = .);
ape_pe_sections = .;
KEEP(*(.pe.sections))
HIDDEN(ape_pe_sections_end = .);
ape_pe_sections_end = .;
/* Mach-O */
KEEP(*(.macho))
. = ALIGN(__SIZEOF_POINTER__);
HIDDEN(ape_macho_end = .);
. = ALIGN(. != 0 ? __SIZEOF_POINTER__ : 0);
ape_macho_end = .;
/* APE loader */
KEEP(*(.ape.loader))
. = ALIGN(. != 0 ? CODE_GRANULE : 0);
KEEP(*(.ape.pad.head))
. = ALIGN(SupportsWindows() || SupportsMetal() ? PAGESIZE : 16);
HIDDEN(_ehead = .);
. = ALIGN(. != 0 ? (SupportsWindows() || SupportsMetal() ? CONSTANT(COMMONPAGESIZE) : 16) : 0);
_ehead = .;
} :Head
/*BEGIN: nt addressability guarantee */
@@ -246,12 +271,10 @@ SECTIONS {
/* Code that needs to be addressable in Real Mode */
*(.text.real)
KEEP(*(SORT_BY_NAME(.sort.text.real.*)))
/* Code we want earlier in the binary w/o modifications */
KEEP(*(.ape.loader))
HIDDEN(_ereal = .);
_ereal = .;
/*END: realmode addressability guarantee */
/*BEGIN: morphable code */
. += 1;
. += CODE_GRANULE;
/* Normal Code */
*(.start)
@@ -259,12 +282,9 @@ SECTIONS {
KEEP(*(SORT_BY_NAME(.init.*)))
KEEP(*(.init))
KEEP(*(.initepilogue))
KEEP(*(.pltprologue))
*(.plt)
KEEP(*(.pltepilogue))
KEEP(*(.pltgotprologue))
*(.plt.got)
KEEP(*(.pltgotepilogue))
*(.iplt)
*(.text.startup .text.startup.*)
*(.text.exit .text.exit.*)
*(.text.unlikely .text.*_unlikely .text.unlikely.*)
@@ -280,7 +300,6 @@ SECTIONS {
KEEP(*(.keep.text))
*(.text .stub .text.*)
KEEP(*(SORT_BY_NAME(.sort.text.*)))
*(.subrs)
KEEP(*(.ape.pad.test));
*(.test.unlikely)
@@ -288,17 +307,20 @@ SECTIONS {
/* Privileged code invulnerable to magic */
KEEP(*(.ape.pad.privileged));
. += . > 0 ? 1 : 0;
. = ALIGN(__privileged_end > __privileged_start ? CONSTANT(COMMONPAGESIZE) : 0);
/*END: morphable code */
HIDDEN(__privileged_start = .);
. += . > 0 ? 1 : 0;
__privileged_start = .;
*(.privileged)
HIDDEN(__privileged_end = .);
. += . > 0 ? 1 : 0;
__privileged_end = .;
KEEP(*(.ape.pad.text))
. = ALIGN(. != 0 ? CONSTANT(COMMONPAGESIZE) : 0);
/*END: Read Only Data (only needed for initialization) */
} :Cod
/*BEGIN: Read Only Data */
KEEP(*(.ape.pad.rodata));
.rodata ALIGN(CONSTANT(COMMONPAGESIZE)) : {
KEEP(*(.rodata.pytab.0));
KEEP(*(.rodata.pytab.1));
KEEP(*(.rodata.pytab.2));
@@ -307,115 +329,129 @@ SECTIONS {
*(.ubsan.types)
*(.ubsan.data)
/* Unit Test & Fixture Registry */
/*BEGIN: Read only data that needn't be mapped after initialization */
/* Legal Notices */
#if !defined(IM_FEELING_NAUGHTY) || defined(EMBED_NOTICES)
KEEP(*(.commentprologue))
KEEP(*(.comment))
KEEP(*(.commentepilogue))
#endif
__notices = .;
KEEP(*(.notice))
BYTE(0);
BYTE(10);
BYTE(10);
/*BEGIN: read-only data that's only needed for initialization */
#if SupportsWindows()
/* Windows DLL Import Directory */
KEEP(*(.idata.ro));
KEEP(*(SORT_BY_NAME(.idata.ro.*)))
. = ALIGN(__SIZEOF_POINTER__);
PROVIDE_HIDDEN(__init_array_start = .);
KEEP(*(SORT_BY_INIT_PRIORITY(.init_array.*)
SORT_BY_INIT_PRIORITY(.ctors.*)))
KEEP(*(.ctors))
KEEP(*(.init_array))
KEEP(*(.preinit_array))
PROVIDE_HIDDEN(__init_array_end = .);
. = ALIGN(__SIZEOF_POINTER__);
PROVIDE_HIDDEN(__fini_array_start = .);
KEEP(*(SORT_BY_INIT_PRIORITY(.fini_array.*)
SORT_BY_INIT_PRIORITY(.dtors.*)))
KEEP(*(.dtors))
PROVIDE_HIDDEN(__fini_array_end = .);
#endif
/* Encoded Data Structures w/ Linear Initialization Order */
KEEP(*(.initroprologue))
KEEP(*(SORT_BY_NAME(.initro.*)))
KEEP(*(.initroepilogue))
KEEP(*(SORT_BY_NAME(.sort.rodata.*)))
. = ALIGN(. != 0 ? CONSTANT(COMMONPAGESIZE) : 0); /* don't delete this line :o */
/*END: read-only data that's only needed for initialization */
KEEP(*(.ape.pad.text))
. = ALIGN(PAGESIZE);
HIDDEN(_etext = .);
PROVIDE_HIDDEN(etext = .);
/*END: Read Only Data (only needed for initialization) */
/*END: Read Only Data */
} :Rom
.data ALIGN(PAGESIZE) : {
/*BEGIN: Read/Write Data */
KEEP(*(SORT_BY_NAME(.piro.data.sort.iat.*)))
/*BEGIN: NT FORK COPYING */
KEEP(*(.dataprologue))
*(.data .data.*)
KEEP(*(SORT_BY_NAME(.sort.data.*)))
. += . > 0 ? 1 : 0;
KEEP(*(.gotprologue))
*(.got)
KEEP(*(.gotepilogue))
KEEP(*(.gotpltprologue))
*(.got.plt)
KEEP(*(.gotpltepilogue))
/*BEGIN: Post-Initialization Read-Only */
. = ALIGN(__SIZEOF_POINTER__);
KEEP(*(SORT_BY_NAME(.piro.relo.sort.*)))
PROVIDE_HIDDEN(__relo_end = .);
. = ALIGN(__SIZEOF_POINTER__);
KEEP(*(SORT_BY_NAME(.piro.data.sort.*)))
KEEP(*(.piro.pad.data))
KEEP(*(.dataepilogue))
/*END: NT FORK COPYING */
HIDDEN(_edata = .);
PROVIDE_HIDDEN(edata = .);
KEEP(*(SORT_BY_NAME(.zip.*)))
HIDDEN(_ezip = .);
. = ALIGN(PAGESIZE);
} :Ram
/* initialization image for thread-local storage, this is copied */
/* out to actual TLS areas at runtime, so just make it read-only */
.tdata . : {
_tdata_start = .;
*(SORT_BY_ALIGNMENT(.tdata))
*(SORT_BY_ALIGNMENT(.tdata.*))
_tdata_end = .;
. = ALIGN(PAGESIZE);
} :Tls :Ram
KEEP(*(.ape.pad.rodata))
. = ALIGN(. != 0 ? CONSTANT(COMMONPAGESIZE) : 0);
_etext = .;
PROVIDE(etext = .);
} :Tls :Rom
/*END: Read Only Data */
/*END: file content that's loaded by o/s */
/*BEGIN: bss memory void */
. = ALIGN(CONSTANT(COMMONPAGESIZE));
.tbss . : {
/* this only tells the linker about the layout of uninitialized */
/* TLS data, and does not advance the linker's location counter */
.tbss : {
_tbss_start = .;
*(SORT_BY_ALIGNMENT(.tbss))
*(SORT_BY_ALIGNMENT(.tbss.*))
. = ALIGN(TLS_ALIGNMENT);
KEEP(*(.fstls))
/* the %fs register is based on this location */
_tbss_end = .;
} :Tls
.data . : {
/*BEGIN: Read/Write Data */
#if SupportsWindows()
KEEP(*(SORT_BY_NAME(.piro.data.sort.iat.*)))
#endif
/*BEGIN: NT FORK COPYING */
KEEP(*(.dataprologue))
*(.data .data.*)
*(.gnu_extab)
*(.gcc_except_table .gcc_except_table.*)
*(.exception_ranges*)
*(.PyRuntime) /* for python */
*(.subrs) /* for emacs */
KEEP(*(SORT_BY_NAME(.sort.data.*)))
. += . > 0 ? CODE_GRANULE : 0;
. = ALIGN(. != 0 ? __SIZEOF_POINTER__ : 0);
__got_start = .;
*(.got)
__got_end = .;
*(.got.plt)
. = ALIGN(. != 0 ? __SIZEOF_POINTER__ : 0);
__init_array_start = .;
KEEP(*(.preinit_array))
KEEP(*(SORT_BY_INIT_PRIORITY(.init_array.*)
SORT_BY_INIT_PRIORITY(.ctors.*)))
KEEP(*(.init_array))
KEEP(*(.ctors))
__init_array_end = .;
. = ALIGN(. != 0 ? __SIZEOF_POINTER__ : 0);
__fini_array_start = .;
KEEP(*(SORT_BY_INIT_PRIORITY(.fini_array.*)
SORT_BY_INIT_PRIORITY(.dtors.*)))
KEEP(*(.fini_array))
KEEP(*(.dtors))
__fini_array_end = .;
/*BEGIN: Post-Initialization Read-Only */
. = ALIGN(. != 0 ? __SIZEOF_POINTER__ : 0);
KEEP(*(SORT_BY_NAME(.piro.relo.sort.*)))
. = ALIGN(. != 0 ? __SIZEOF_POINTER__ : 0);
KEEP(*(SORT_BY_NAME(.piro.data.sort.*)))
KEEP(*(.piro.pad.data))
*(.igot.plt)
KEEP(*(.dataepilogue))
. = ALIGN(. != 0 ? CONSTANT(COMMONPAGESIZE) : 0);
/*END: NT FORK COPYING */
_edata = .;
PROVIDE(edata = .);
_ezip = .; /* <-- very deprecated */
} :Ram
. = ALIGN(CONSTANT(COMMONPAGESIZE));
/*END: file content that's loaded by o/s */
/*END: file content */
/*BEGIN: bss memory that's addressable */
.bss ALIGN(64) : {
.bss : {
/*BEGIN: NT FORK COPYING */
KEEP(*(.bssprologue))
KEEP(*(SORT_BY_NAME(.piro.bss.init.*)))
*(.piro.bss)
KEEP(*(SORT_BY_NAME(.piro.bss.sort.*)))
HIDDEN(__piro_end = .);
. += . > 0 ? 1 : 0;
__piro_end = .;
. += . > 0 ? CODE_GRANULE : 0;
/*END: Post-Initialization Read-Only */
/* Statically Allocated Empty Space */
@@ -426,11 +462,15 @@ SECTIONS {
KEEP(*(SORT_BY_NAME(.sort.bss.*)))
KEEP(*(.bssepilogue))
. = ALIGN(. != 0 ? CONSTANT(COMMONPAGESIZE) : 0);
/*END: NT FORK COPYING */
. = ALIGN(PAGESIZE);
HIDDEN(_end = .);
PROVIDE_HIDDEN(end = .);
} :Bss
} :Ram
. = ALIGN(CONSTANT(COMMONPAGESIZE));
_end = .;
PROVIDE(end = .);
/*END: nt addressability guarantee */
/*END: bsd addressability guarantee */
@@ -473,14 +513,34 @@ SECTIONS {
.gnu.attributes 0 : { KEEP(*(.gnu.attributes)) }
.GCC.command.line 0 : { *(.GCC.command.line) }
.zip 0 : {
KEEP(*(.zip.file))
__zip_cdir_start = .;
KEEP(*(.zip.cdir))
__zip_cdir_size = . - __zip_cdir_start;
KEEP(*(.zip.eocd))
}
/DISCARD/ : {
#if !SupportsWindows()
*(.idata.ro);
*(.idata.ro.*)
*(.piro.data.sort.iat.*)
#endif
*(__patchable_function_entries)
*(.note.gnu.property)
*(__mcount_loc)
*(.rela.dyn)
*(.discard)
*(.yoink)
*(.*)
}
}
ape_pe_filealignment = 512;
ape_pe_sectionalignment = 4096;
ape_pe_sizeofheaders = SIZEOF(.head);
ape_pe_sizeofimage = ROUNDUP(_end - __executable_start, ape_pe_sectionalignment);
PFSTUB8(ape_elf_entry, _start);
PFSTUB8(ape_elf_phoff, RVA(ape_phdrs));
PFSTUB8(ape_elf_shoff, 0);
@@ -488,70 +548,63 @@ PFSTUB4(ape_elf_phnum, (ape_phdrs_end - ape_phdrs) / 56);
PFSTUB4(ape_elf_shnum, 0);
PFSTUB4(ape_elf_shstrndx, 0);
HIDDEN(_tls_size = _tbss_end - _tdata_start);
HIDDEN(_tdata_size = _tdata_end - _tdata_start);
HIDDEN(_tbss_size = _tbss_end - _tbss_start);
HIDDEN(_tbss_offset = _tbss_start - _tdata_start);
HIDDEN(_tls_content = (_tdata_end - _tdata_start) + (_tbss_end - _tbss_start));
_tls_size = _tbss_end - _tdata_start;
_tdata_size = _tdata_end - _tdata_start;
_tbss_size = _tbss_end - _tbss_start;
_tbss_offset = _tbss_start - _tdata_start;
_tls_content = (_tdata_end - _tdata_start) + (_tbss_end - _tbss_start);
_tdata_align = ALIGNOF(.tdata);
_tbss_align = ALIGNOF(.tbss);
_tls_align = MAX(TLS_ALIGNMENT, MAX(ALIGNOF(.tdata), ALIGNOF(.tbss)));
HIDDEN(__privileged_addr = ROUNDDOWN(__privileged_start, PAGESIZE));
HIDDEN(__privileged_size = (ROUNDUP(__privileged_end, PAGESIZE) -
ROUNDDOWN(__privileged_start, PAGESIZE)));
ape_cod_offset = 0;
ape_cod_vaddr = ADDR(.head);
ape_cod_paddr = LOADADDR(.head);
ape_cod_filesz = ADDR(.rodata) - ADDR(.head);
ape_cod_memsz = ape_cod_filesz;
ape_cod_align = CONSTANT(COMMONPAGESIZE);
ape_cod_rva = RVA(ape_cod_vaddr);
HIDDEN(ape_rom_offset = 0);
HIDDEN(ape_rom_vaddr = ADDR(.head));
HIDDEN(ape_rom_paddr = LOADADDR(.head));
HIDDEN(ape_rom_filesz = LOADADDR(.data) - ape_rom_paddr);
HIDDEN(ape_rom_memsz = ADDR(.data) - ADDR(.head));
HIDDEN(ape_rom_align = PAGESIZE);
HIDDEN(ape_rom_rva = RVA(ape_rom_vaddr));
ape_rom_vaddr = ADDR(.rodata);
ape_rom_offset = ape_rom_vaddr - __executable_start;
ape_rom_paddr = LOADADDR(.rodata);
ape_rom_filesz = ADDR(.tbss) - ADDR(.rodata);
ape_rom_memsz = ape_rom_filesz;
ape_rom_align = CONSTANT(COMMONPAGESIZE);
ape_rom_rva = RVA(ape_rom_vaddr);
HIDDEN(ape_ram_offset = ape_rom_offset + ape_rom_filesz);
HIDDEN(ape_ram_vaddr = ADDR(.data));
HIDDEN(ape_ram_paddr = LOADADDR(.data));
HIDDEN(ape_ram_filesz = SIZEOF(.data) + SIZEOF(.tdata));
HIDDEN(ape_ram_memsz = ADDR(.bss) + SIZEOF(.bss) - ape_ram_vaddr);
HIDDEN(ape_ram_align = PAGESIZE);
HIDDEN(ape_ram_rva = RVA(ape_ram_vaddr));
ape_ram_vaddr = ADDR(.data);
ape_ram_offset = ape_ram_vaddr - __executable_start;
ape_ram_paddr = LOADADDR(.data);
ape_ram_filesz = ADDR(.bss) - ADDR(.data);
ape_ram_memsz = _end - ADDR(.data);
ape_ram_align = CONSTANT(COMMONPAGESIZE);
ape_ram_rva = RVA(ape_ram_vaddr);
HIDDEN(ape_stack_pf = DEFINED(ape_stack_pf) ? ape_stack_pf : PF_R | PF_W);
HIDDEN(ape_stack_prot = _PF2PROT(ape_stack_pf));
HIDDEN(ape_stack_offset = ape_ram_offset + ape_ram_filesz);
HIDDEN(ape_stack_vaddr = DEFINED(ape_stack_vaddr) ? ape_stack_vaddr : 0x700000000000);
HIDDEN(ape_stack_paddr = ape_ram_paddr + ape_ram_filesz);
HIDDEN(ape_stack_filesz = 0);
HIDDEN(ape_stack_memsz = DEFINED(ape_stack_memsz) ? ape_stack_memsz : STACKSIZE);
HIDDEN(ape_stack_align = 16);
ape_stack_pf = DEFINED(ape_stack_pf) ? ape_stack_pf : PF_R | PF_W;
ape_stack_prot = _PF2PROT(ape_stack_pf);
ape_stack_offset = 0;
ape_stack_vaddr = DEFINED(ape_stack_vaddr) ? ape_stack_vaddr : 0x700000000000;
ape_stack_paddr = ape_ram_paddr + ape_ram_filesz;
ape_stack_filesz = 0;
ape_stack_memsz = DEFINED(ape_stack_memsz) ? ape_stack_memsz : 8 * 1024 * 1024;
ape_stack_align = DEFINED(ape_stack_align) ? MAX(ape_stack_align, 16) : 16;
ape_stack_round = -ape_stack_align;
HIDDEN(ape_note_offset = ape_rom_offset + (ape_note - ape_rom_vaddr));
HIDDEN(ape_note_vaddr = ape_note);
HIDDEN(ape_note_paddr = ape_rom_paddr + ape_note_offset);
HIDDEN(ape_note_filesz = ape_note_end - ape_note);
HIDDEN(ape_note_memsz = ape_note_filesz);
HIDDEN(ape_note_align = __SIZEOF_POINTER__);
ASSERT(ape_stack_align < 1048576, "stack align too big");
ASSERT(!(ape_stack_align & (ape_stack_align - 1)), "stack align must be two power");
HIDDEN(ape_text_offset = ape_rom_offset + LOADADDR(.text) - ape_rom_paddr);
HIDDEN(ape_text_paddr = LOADADDR(.text));
HIDDEN(ape_text_vaddr = ADDR(.text));
HIDDEN(ape_text_filesz = SIZEOF(.text));
HIDDEN(ape_text_memsz = SIZEOF(.text));
HIDDEN(ape_text_align = PAGESIZE);
HIDDEN(ape_text_rva = RVA(ape_text_vaddr));
ape_note_offset = ape_cod_offset + (ape_note - ape_cod_vaddr);
ape_note_filesz = ape_note_end - ape_note;
ape_note_memsz = ape_note_filesz;
HIDDEN(ape_data_offset = ape_ram_offset + LOADADDR(.data) - ape_ram_paddr);
HIDDEN(ape_data_paddr = LOADADDR(.data));
HIDDEN(ape_data_vaddr = ADDR(.data));
HIDDEN(ape_data_filesz = SIZEOF(.data) + SIZEOF(.tdata));
HIDDEN(ape_data_memsz = SIZEOF(.data) + SIZEOF(.tdata));
HIDDEN(ape_data_align = PAGESIZE);
HIDDEN(ape_data_rva = RVA(ape_data_vaddr));
HIDDEN(ape_bss_offset = ape_ram_offset + LOADADDR(.bss) - ape_ram_paddr);
HIDDEN(ape_bss_paddr = LOADADDR(.bss));
HIDDEN(ape_bss_vaddr = ADDR(.bss));
HIDDEN(ape_bss_filesz = 0);
HIDDEN(ape_bss_memsz = SIZEOF(.bss));
HIDDEN(ape_bss_align = PAGESIZE);
ape_text_vaddr = ADDR(.text);
ape_text_offset = ape_text_vaddr - __executable_start;
ape_text_paddr = LOADADDR(.text);
ape_text_filesz = ADDR(.rodata) - ADDR(.text);
ape_text_memsz = ape_text_filesz;
ape_text_align = CONSTANT(COMMONPAGESIZE);
ape_text_rva = RVA(ape_text_vaddr);
/* we roundup here because xnu wants the file load segments page-aligned */
/* but we don't want to add the nop padding to the ape program, so we'll */
@@ -559,27 +612,19 @@ HIDDEN(ape_bss_align = PAGESIZE);
SHSTUB2(ape_loader_dd_skip, DEFINED(ape_loader) ? RVA(ape_loader) / 64 : 0);
SHSTUB2(ape_loader_dd_count,
DEFINED(ape_loader_end)
? ROUNDUP(ape_loader_end - ape_loader, PAGESIZE) / 64
? ROUNDUP(ape_loader_end - ape_loader, CONSTANT(COMMONPAGESIZE)) / 64
: 0);
#if SupportsMetal()
HIDDEN(v_ape_realsectors =
MIN(0x70000 - IMAGE_BASE_REAL,
ROUNDUP(RVA(_tdata_end), 512)) / 512);
HIDDEN(v_ape_realpages = v_ape_realsectors / (4096 / 512));
HIDDEN(v_ape_highsectors =
(ROUNDUP(RVA(_tdata_end), 512) / 512) - v_ape_realsectors);
v_ape_realsectors = MIN(0x70000 - IMAGE_BASE_REAL, ROUNDUP(RVA(_ezip), 512)) / 512;
v_ape_realbytes = v_ape_realsectors * 512;
v_ape_realdwords = v_ape_realsectors * (512 / 4);
v_ape_allsectors = ROUNDUP(RVA(_ezip), 512) / 512;
v_ape_allbytes = v_ape_allsectors * 512;
v_ape_highsectors = MIN(0xffff, v_ape_allsectors - v_ape_realsectors);
TSSDESCSTUB2(_tss, _tss, _tss_end ? _tss_end - _tss - 1 : 0);
#endif
/* ZIP End of Central Directory header */
#define ZIPCONST(NAME, VAL) HIDDEN(NAME = DEFINED(__zip_start) ? VAL : 0);
ZIPCONST(v_zip_cdoffset, __zip_start - IMAGE_BASE_VIRTUAL);
ZIPCONST(v_zip_cdirsize, __zip_end - __zip_start);
ASSERT(v_zip_cdirsize % kZipCdirHdrLinkableSize == 0, "bad zip cdir");
ZIPCONST(v_zip_records, v_zip_cdirsize / kZipCdirHdrLinkableSize);
ZIPCONST(v_zip_commentsize, _ezip - __zip_end - kZipCdirHdrMinSize);
#if SupportsXnu()
/* Generates deterministic ID. */
#define PHI 0x9e3779b9925d4c17
@@ -598,19 +643,8 @@ ZIPCONST(v_zip_commentsize, _ezip - __zip_end - kZipCdirHdrMinSize);
KMH(ape_uuid1, X); \
XORSHIFT(ape_uuid2, X); \
KMH(ape_uuid2, X)
HIDDEN(ape_uuid1 = 88172645463325252);
HIDDEN(ape_uuid2 = 88172645463325252);
CHURN(ape_bss_align);
CHURN(ape_bss_filesz);
CHURN(ape_bss_memsz);
CHURN(ape_bss_offset);
CHURN(ape_bss_paddr);
CHURN(ape_data_filesz);
CHURN(ape_data_memsz);
CHURN(ape_data_offset);
CHURN(ape_data_paddr);
CHURN(ape_data_rva);
CHURN(ape_data_vaddr);
ape_uuid1 = 88172645463325252;
ape_uuid2 = 88172645463325252;
CHURN(ape_elf_entry);
CHURN(ape_elf_phnum);
CHURN(ape_elf_phoff);
@@ -619,13 +653,10 @@ CHURN(ape_elf_shoff);
CHURN(ape_elf_shstrndx);
CHURN(ape_macho_end);
CHURN(ape_note);
CHURN(ape_note_align);
CHURN(ape_note_end);
CHURN(ape_note_filesz);
CHURN(ape_note_memsz);
CHURN(ape_note_offset);
CHURN(ape_note_paddr);
CHURN(ape_note_vaddr);
CHURN(ape_ram_align);
CHURN(ape_ram_filesz);
CHURN(ape_ram_memsz);
@@ -639,7 +670,14 @@ CHURN(ape_rom_memsz);
CHURN(ape_rom_offset);
CHURN(ape_rom_paddr);
CHURN(ape_rom_rva);
CHURN(ape_rom_vaddr);
CHURN(ape_cod_vaddr);
CHURN(ape_cod_align);
CHURN(ape_cod_filesz);
CHURN(ape_cod_memsz);
CHURN(ape_cod_offset);
CHURN(ape_cod_paddr);
CHURN(ape_cod_rva);
CHURN(ape_cod_vaddr);
CHURN(ape_text_align);
CHURN(ape_text_filesz);
CHURN(ape_text_memsz);
@@ -651,7 +689,7 @@ CHURN(ADDR(.bss));
CHURN(_start);
CHURN(ape_phdrs);
#if SupportsMetal()
CHURN(v_ape_realsectors);
CHURN(v_ape_allsectors);
#endif
#if SupportsXnu()
CHURN(ape_macho);
@@ -672,22 +710,22 @@ CHURN(WinMain);
#if SupportsWindows() || SupportsMetal()
#define LINK_WINDOWS (SupportsWindows() && !DEFINED(EfiMain))
PFSTUB4(ape_pe_offset, ape_pe - ape_mz);
HIDDEN(ape_pe_optsz = ape_pe_sections - (ape_pe + 24));
HIDDEN(ape_pe_shnum = (ape_pe_sections_end - ape_pe_sections) / 40);
HIDDEN(ape_pe_base = IMAGE_BASE_VIRTUAL);
HIDDEN(ape_idata = LINK_WINDOWS ? RVA(ape_idata_iat) : 0);
HIDDEN(ape_idata_iatsize = LINK_WINDOWS ? ape_idata_iatend - ape_idata_iat : 0);
HIDDEN(ape_idata = LINK_WINDOWS ? RVA(ape_idata_idt) : 0);
HIDDEN(ape_idata_idtsize = LINK_WINDOWS ? ape_idata_idtend - ape_idata_idt : 0);
HIDDEN(v_ntversion = LINK_WINDOWS ? 6 : 1);
HIDDEN(v_ntdllchar = LINK_WINDOWS ? 288 : 0);
HIDDEN(v_ntsubversion = LINK_WINDOWS ? 6 : 5);
HIDDEN(v_ntsubsystem = (LINK_WINDOWS
? (DEFINED(GetMessage)
? kNtImageSubsystemWindowsGui
: kNtImageSubsystemWindowsCui)
: kNtImageSubsystemEfiApplication));
HIDDEN(ape_pe_entry = LINK_WINDOWS ? WinMain : EfiMain);
ape_pe_optsz = ape_pe_sections - (ape_pe + 24);
ASSERT(ape_pe_optsz % 8 == 0, "SizeOfOptionalHeader must be multiple of 8");
ape_pe_shnum = (ape_pe_sections_end - ape_pe_sections) / 40;
ape_pe_base = IMAGE_BASE_VIRTUAL;
ape_idataz = LINK_WINDOWS ? RVA(ape_idata_iat) : 0;
ape_idata_iatsize = LINK_WINDOWS ? ape_idata_iatend - ape_idata_iat : 0;
ape_idata = LINK_WINDOWS ? RVA(ape_idata_idt) : 0;
ape_idata_idtsize = LINK_WINDOWS ? ape_idata_idtend - ape_idata_idt : 0;
v_ntdllchar = LINK_WINDOWS ? 288 : 0;
v_ntsubversion = LINK_WINDOWS ? 6 : 5;
v_ntsubsystem = (LINK_WINDOWS
? (DEFINED(GetMessage)
? kNtImageSubsystemWindowsGui
: kNtImageSubsystemWindowsCui)
: kNtImageSubsystemEfiApplication);
ape_pe_entry = LINK_WINDOWS ? WinMain : EfiMain;
#endif
#if SupportsXnu()
@@ -705,21 +743,17 @@ ASSERT(((DEFINED(__init_rodata_end) ? __init_rodata_end : 0) %
ASSERT((!DEFINED(ape_grub) ? 1 : RVA(ape_grub) < 8192),
"grub stub needs to be in first 8kb of image");
ASSERT(DEFINED(_start) || DEFINED(_start16),
"please link a _start() or _start16() entrypoint");
ASSERT(!DEFINED(_start16) || REAL(_end) < 65536,
"ape won't support non-tiny real mode programs");
ASSERT(IS2POW(ape_stack_memsz),
"ape_stack_memsz must be a two power");
ASSERT(ape_stack_vaddr % ape_stack_memsz == 0,
"ape_stack_vaddr must have ape_stack_memsz alignment; try using STATIC_STACK_ADDR(0x700000040000 - ape_stack_memsz);");
"ape_stack_vaddr must have ape_stack_memsz alignment; try using STATIC_STACK_ADDR(0x700000040000 & -ape_stack_memsz);");
ASSERT(ALIGNOF(.tdata) <= TLS_ALIGNMENT && ALIGNOF(.tbss) <= TLS_ALIGNMENT,
"_Thread_local _Alignof can't exceed TLS_ALIGNMENT");
ASSERT(DEFINED(main), "main() function not defined");
/* Let's not be like Knight Capital. */
/* NOCROSSREFS_TO(.test .text) */
-188
View File
@@ -1,188 +0,0 @@
#-*-mode:makefile-gmake;indent-tabs-mode:t;tab-width:8;coding:utf-8-*-┐
#───vi: set et ft=make ts=8 tw=8 fenc=utf-8 :vi───────────────────────┘
#
# OVERVIEW
#
# αcτµαlly pδrταblε εxεcµταblε
#
# DESCRIPTION
#
# This file defines the libraries, runtimes, and build rules needed to
# create executables from your Linux workstation that'll run anywhere.
# Loading this package will make certain systemic modifications to the
# build like turning off the System V "Red Zone" optimization, because
# αcτµαlly pδrταblε εxεcµταblεs need to be able to run in kernelspace.
PKGS += APE
APE = o/$(MODE)/ape/ape.o \
o/$(MODE)/ape/ape.lds
APE_NO_MODIFY_SELF = \
o/$(MODE)/ape/ape.lds \
o/$(MODE)/ape/ape-no-modify-self.o
APE_COPY_SELF = \
o/$(MODE)/ape/ape.lds \
o/$(MODE)/ape/ape-copy-self.o
APELINK = \
$(COMPILE) \
-ALINK.ape \
$(LINK) \
$(LINKARGS) \
$(OUTPUT_OPTION)
APE_LOADER_FLAGS = \
-DNDEBUG \
-iquote. \
-Wall \
-Wextra \
-fpie \
-Os \
-ffreestanding \
-mgeneral-regs-only \
-mno-red-zone \
-fno-ident \
-fno-gnu-unique \
-c \
$(OUTPUT_OPTION) \
$<
APE_FILES := $(wildcard ape/*.*)
APE_HDRS = $(filter %.h,$(APE_FILES))
APE_INCS = $(filter %.inc,$(APE_FILES))
APE_SRCS_C = ape/loader.c
APE_SRCS_S = $(filter %.S,$(APE_FILES))
APE_SRCS = $(APE_SRCS_C) $(APE_SRCS_S)
APE_OBJS = $(APE_SRCS_S:%.S=o/$(MODE)/%.o)
APE_CHECKS = $(APE_HDRS:%=o/%.ok)
o/$(MODE)/ape/ape.lds: \
ape/ape.lds \
ape/macros.internal.h \
ape/relocations.h \
libc/intrin/bits.h \
libc/thread/tls.h \
libc/calls/struct/timespec.h \
libc/thread/thread.h \
libc/dce.h \
libc/elf/def.h \
libc/elf/pf2prot.internal.h \
libc/macros.internal.h \
libc/nt/pedef.internal.h \
libc/str/str.h \
libc/zip.h
o/$(MODE)/ape/public/ape.lds: OVERRIDE_CPPFLAGS += -UCOSMO
o/$(MODE)/ape/public/ape.lds: \
ape/public/ape.lds \
ape/ape.lds \
ape/macros.internal.h \
ape/relocations.h \
libc/intrin/bits.h \
libc/thread/tls.h \
libc/calls/struct/timespec.h \
libc/thread/thread.h \
libc/dce.h \
libc/elf/def.h \
libc/elf/pf2prot.internal.h \
libc/macros.internal.h \
libc/nt/pedef.internal.h \
libc/str/str.h \
libc/zip.h
o/ape/idata.inc: \
ape/idata.internal.h \
ape/relocations.h
o/$(MODE)/ape/ape-no-modify-self.o: \
ape/ape.S \
ape/macros.internal.h \
ape/notice.inc \
ape/relocations.h \
libc/dce.h \
libc/elf/def.h \
libc/intrin/asancodes.h \
libc/macho.internal.h \
libc/macros.internal.h \
libc/nexgen32e/uart.internal.h \
libc/calls/metalfile.internal.h \
libc/nexgen32e/vidya.internal.h \
libc/nt/pedef.internal.h \
libc/runtime/e820.internal.h \
libc/runtime/mman.internal.h \
libc/runtime/pc.internal.h \
libc/runtime/symbolic.h \
libc/sysv/consts/prot.h \
o/$(MODE)/ape/ape.elf
@$(COMPILE) \
-AOBJECTIFY.S \
$(OBJECTIFY.S) \
$(OUTPUT_OPTION) \
-DAPE_NO_MODIFY_SELF \
-DAPE_LOADER='"o/$(MODE)/ape/ape.elf"' $<
o/$(MODE)/ape/ape-copy-self.o: \
ape/ape.S \
ape/macros.internal.h \
ape/notice.inc \
ape/relocations.h \
libc/dce.h \
libc/elf/def.h \
libc/intrin/asancodes.h \
libc/macho.internal.h \
libc/macros.internal.h \
libc/nexgen32e/uart.internal.h \
libc/calls/metalfile.internal.h \
libc/nexgen32e/vidya.internal.h \
libc/nt/pedef.internal.h \
libc/runtime/e820.internal.h \
libc/runtime/mman.internal.h \
libc/runtime/pc.internal.h \
libc/runtime/symbolic.h \
libc/sysv/consts/prot.h
@$(COMPILE) \
-AOBJECTIFY.S \
$(OBJECTIFY.S) \
$(OUTPUT_OPTION) \
-DAPE_NO_MODIFY_SELF $<
o/$(MODE)/ape/loader.o: ape/loader.c ape/loader.h
@$(COMPILE) -AOBJECTIFY.c $(CC) -DSUPPORT_VECTOR=0b01111001 -g $(APE_LOADER_FLAGS)
o/$(MODE)/ape/loader-gcc.asm: ape/loader.c
@$(COMPILE) -AOBJECTIFY.c $(CC) -DSUPPORT_VECTOR=0b01111001 -S -g0 $(APE_LOADER_FLAGS)
o/$(MODE)/ape/loader-clang.asm: ape/loader.c
@$(COMPILE) -AOBJECTIFY.c $(CLANG) -DSUPPORT_VECTOR=0b01111001 -S -g0 $(APE_LOADER_FLAGS)
o/$(MODE)/ape/loader-xnu.o: ape/loader.c ape/loader.h
@$(COMPILE) -AOBJECTIFY.c $(CC) -DSUPPORT_VECTOR=0b00001000 -g $(APE_LOADER_FLAGS)
o/$(MODE)/ape/loader-xnu-gcc.asm: ape/loader.c
@$(COMPILE) -AOBJECTIFY.c $(CC) -DSUPPORT_VECTOR=0b00001000 -S -g0 $(APE_LOADER_FLAGS)
o/$(MODE)/ape/loader-xnu-clang.asm: ape/loader.c
@$(COMPILE) -AOBJECTIFY.c $(CLANG) -DSUPPORT_VECTOR=0b00001000 -S -g0 $(APE_LOADER_FLAGS)
o/$(MODE)/ape/ape.elf: o/$(MODE)/ape/ape.elf.dbg
o/$(MODE)/ape/ape.macho: o/$(MODE)/ape/ape.macho.dbg
o/$(MODE)/ape/ape.elf.dbg: \
o/$(MODE)/ape/loader.o \
o/$(MODE)/ape/loader-elf.o \
ape/loader.lds
@$(ELFLINK) -z max-page-size=0x10
o/$(MODE)/ape/ape.macho.dbg: \
o/$(MODE)/ape/loader-xnu.o \
o/$(MODE)/ape/loader-macho.o \
ape/loader-macho.lds
@$(ELFLINK) -z max-page-size=0x10
.PHONY: o/$(MODE)/ape
o/$(MODE)/ape: $(APE_CHECKS) \
o/$(MODE)/ape/ape.o \
o/$(MODE)/ape/ape.lds \
o/$(MODE)/ape/public/ape.lds \
o/$(MODE)/ape/ape.elf \
o/$(MODE)/ape/ape.macho \
o/$(MODE)/ape/ape-copy-self.o \
o/$(MODE)/ape/ape-no-modify-self.o
+91 -39
View File
@@ -1,62 +1,107 @@
#!/bin/sh
PROG=${0##*/}
MODE=${MODE:-$m}
TMPDIR=${TMPDIR:-/tmp}
COSMO=${COSMO:-/opt/cosmo}
COSMOS=${COSMOS:-/opt/cosmos}
if [ ! -f ape/loader.c ]; then
cd "$COSMO" || exit
fi
if [ -x build/bootstrap/make ]; then
MAKE=build/bootstrap/make
else
MAKE=make
fi
if [ "$(id -u)" -eq 0 ]; then
SUDO=
else
elif command -v sudo >/dev/null 2>&1; then
SUDO=sudo
elif command -v doas >/dev/null 2>&1; then
SUDO=doas
else
echo "need root or sudo" >&2
exit
fi
if command -v install >/dev/null 2>&1; then
if [ x"$(uname -s)" = xLinux ]; then
INSTALL="install -o root -g root -m 755"
else
INSTALL="install -o root -g wheel -m 755"
fi
else
INSTALL="cp -f"
fi
echo "Actually Portable Executable (APE) Installer" >&2
echo "Author: Justine Tunney <[email protected]>" >&2
# special installation process for apple silicon
if [ x"$(uname -s)" = xDarwin ] && [ x"$(uname -m)" = xarm64 ]; then
echo "cc -O -o $TMPDIR/ape.$$ ape/ape-m1.c" >&2
cc -O -o "$TMPDIR/ape.$$" ape/ape-m1.c || exit
trap 'rm "$TMPDIR/ape.$$"' EXIT
if [ ! -d /usr/local/bin ]; then
echo "$SUDO mkdir -p /usr/local/bin" >&2
$SUDO mkdir -p /usr/local/bin || exit
fi
echo "$SUDO $INSTALL $TMPDIR/ape.$$ /usr/local/bin/ape" >&2
$SUDO $INSTALL "$TMPDIR/ape.$$" /usr/local/bin/ape || exit
exit
fi
if [ x"$(uname -m)" = xarm64 ] || [ x"$(uname -m)" = xaarch64 ]; then
MODE=aarch64
EXT=elf
BEXT=aarch64
elif [ x"$(uname -m)" = xx86_64 ]; then
MODE=
if [ x"$(uname -s)" = xDarwin ]; then
EXT=macho
else
EXT=elf
fi
BEXT=$EXT
else
echo "unsupported architecture $(uname -m)" >&2
exit
fi
################################################################################
# INSTALL APE LOADER SYSTEMWIDE
if [ -f o/depend ]; then
# mkdeps.com build was successfully run so assume we can build
echo >&2
echo "recompiling ape loader" >&2
echo "running: make -j8 o//ape" >&2
make -j8 o//ape || exit
echo "done" >&2
if [ -f o/$MODE/depend ] && $MAKE -j8 o/$MODE/ape; then
echo "successfully recompiled ape loader" >&2
elif [ -x o/$MODE/ape/ape.$EXT ]; then
echo "using ape loader you compiled earlier" >&2
elif [ -d build/bootstrap ]; then
# if make isn't being used then it's unlikely the user changed the sources
# in that case the prebuilt binaries should be completely up-to-date
echo "using prebuilt ape loader from cosmo repo" >&2
mkdir -p o//ape || exit
cp -af build/bootstrap/ape.elf o//ape/ape.elf || exit
cp -af build/bootstrap/ape.macho o//ape/ape.macho || exit
mkdir -p o/$MODE/ape || exit
cp -af build/bootstrap/ape.$BEXT o/$MODE/ape/ape.$EXT || exit
else
echo "no cosmopolitan libc repository here" >&2
echo "fetching ape loader from justine.lol" >&2
mkdir -p o//ape || exit
echo "fetching ape loader from justine.lol" >&2
mkdir -p o/$MODE/ape || exit
if command -v wget >/dev/null 2>&1; then
wget -qO o//ape/ape.elf https://justine.lol/ape.elf || exit
wget -qO o//ape/ape.macho https://justine.lol/ape.macho || exit
wget -qO o/$MODE/ape/ape.$EXT https://justine.lol/ape.$BEXT || exit
else
curl -Rso o//ape/ape.elf https://justine.lol/ape.elf || exit
curl -Rso o//ape/ape.macho https://justine.lol/ape.macho || exit
curl -Rso o/$MODE/ape/ape.$EXT https://justine.lol/ape.$BEXT || exit
fi
chmod +x o//ape/ape.elf || exit
chmod +x o//ape/ape.macho || exit
chmod +x o/$MODE/ape/ape.$EXT || exit
fi
if [ "$(uname -s)" = "Darwin" ]; then
if ! [ /usr/bin/ape -nt o//ape/ape.macho ]; then
echo >&2
echo "installing o//ape/ape.elf to /usr/bin/ape" >&2
echo "$SUDO mv -f o//ape/ape.elf /usr/bin/ape" >&2
$SUDO cp -f o//ape/ape.macho /usr/bin/ape || exit
echo "done" >&2
fi
else
if ! [ /usr/bin/ape -nt o//ape/ape.elf ]; then
echo >&2
echo "installing o//ape/ape.elf to /usr/bin/ape" >&2
echo "$SUDO mv -f o//ape/ape.elf /usr/bin/ape" >&2
$SUDO cp -f o//ape/ape.elf /usr/bin/ape || exit
echo "done" >&2
fi
if ! [ /usr/bin/ape -nt o/$MODE/ape/ape.$EXT ]; then
echo >&2
echo "installing o/$MODE/ape/ape.$EXT to /usr/bin/ape" >&2
echo "$SUDO $INSTALL o/$MODE/ape/ape.$EXT /usr/bin/ape" >&2
$SUDO $INSTALL o/$MODE/ape/ape.$EXT /usr/bin/ape || exit
echo "done" >&2
fi
################################################################################
@@ -92,13 +137,20 @@ if [ x"$(uname -s)" = xLinux ]; then
echo done >&2
fi
uname_r="$(uname -r)"
if printf '%s\n%s\n' 5.12 "$uname_r" | sort -CV; then
FLAGS=FP
else
FLAGS=F
fi
echo >&2
echo registering APE with binfmt_misc >&2
echo you may need to edit configs to persist across reboot >&2
echo '$SUDO sh -c "echo '"'"':APE:M::MZqFpD::/usr/bin/ape:'"'"' >/proc/sys/fs/binfmt_misc/register"' >&2
$SUDO sh -c "echo ':APE:M::MZqFpD::/usr/bin/ape:' >/proc/sys/fs/binfmt_misc/register" || exit
echo '$SUDO sh -c "echo '"'"':APE-sysv:M::JTqFpD::/usr/bin/ape:'"'"' >/proc/sys/fs/binfmt_misc/register"' >&2
$SUDO sh -c "echo ':APE-sysv:M::JTqFpD::/usr/bin/ape:' >/proc/sys/fs/binfmt_misc/register" || exit
echo '$SUDO sh -c "echo '"'"':APE:M::MZqFpD::/usr/bin/ape:'"$FLAGS'"' >/proc/sys/fs/binfmt_misc/register"' >&2
$SUDO sh -c "echo ':APE:M::MZqFpD::/usr/bin/ape:$FLAGS' >/proc/sys/fs/binfmt_misc/register" || exit
echo '$SUDO sh -c "echo '"'"':APE-jart:M::jartsr::/usr/bin/ape:'"$FLAGS'"' >/proc/sys/fs/binfmt_misc/register"' >&2
$SUDO sh -c "echo ':APE-jart:M::jartsr::/usr/bin/ape:$FLAGS' >/proc/sys/fs/binfmt_misc/register" || exit
echo done >&2
if [ x"$(cat /proc/sys/fs/binfmt_misc/status)" = xdisabled ]; then
+50 -10
View File
@@ -1,28 +1,68 @@
#!/bin/sh
PROG=${0##*/}
MODE=${MODE:-$m}
COSMO=${COSMO:-/opt/cosmo}
COSMOS=${COSMOS:-/opt/cosmos}
if [ ! -f ape/loader.c ]; then
cd "$COSMO" || exit
fi
if [ "$UID" = "0" ]; then
SUDO=
else
elif command -v sudo >/dev/null 2>&1; then
SUDO=sudo
elif command -v doas >/dev/null 2>&1; then
SUDO=doas
else
echo "need root or sudo" >&2
exit
fi
{
echo
echo "APE Uninstaller intends to run"
echo "APE Uninstaller intends to run (in pseudo-shell)"
echo
echo " $SUDO sh -c 'echo -1 >/proc/sys/fs/binfmt_misc/APE'"
echo " $SUDO sh -c 'echo -1 >/proc/sys/fs/binfmt_misc/APE-sysv'"
echo " $SUDO rm -f /usr/bin/ape ~/.ape o/tmp/.ape /tmp/.ape"
echo " sudo echo -1 into /proc/sys/fs/binfmt_misc/APE*"
echo " sudo rm -f /usr/bin/ape ~/.ape /tmp/.ape # etc."
echo
echo "You may then use ape/apeinstall.sh to reinstall it"
echo
} >&2
set -ex
if [ -f /proc/sys/fs/binfmt_misc/APE ]; then
$SUDO sh -c 'echo -1 >/proc/sys/fs/binfmt_misc/APE' || exit
for f in /proc/sys/fs/binfmt_misc/APE*; do
if [ -f $f ]; then
$SUDO sh -c "echo -1 >$f" || exit
fi
done
# system installation
if [ -f /usr/bin/ape ]; then
$SUDO rm -f /usr/bin/ape
fi
if [ -f /proc/sys/fs/binfmt_misc/APE-sysv ]; then
$SUDO sh -c 'echo -1 >/proc/sys/fs/binfmt_misc/APE-sysv' || exit
if [ -f /usr/local/bin/ape ]; then
$SUDO rm -f /usr/local/bin/ape
fi
$SUDO rm -f /usr/bin/ape ~/.ape o/tmp/.ape o/tmp/ape /tmp/.ape /tmp/ape || exit
# legacy installations
rm -f o/tmp/ape /tmp/ape "${TMPDIR:-/tmp}/ape"
# ad-hoc installations
for x in .ape \
.ape-1.1 \
.ape-1.3 \
.ape-1.4 \
.ape-1.5 \
.ape-1.6 \
.ape-1.7 \
.ape-1.8 \
.ape-1.9 \
.ape-1.10; do
rm -f \
~/$x \
/tmp/$x \
o/tmp/$x \
"${TMPDIR:-/tmp}/$x"
done
-10
View File
@@ -1,10 +0,0 @@
#ifndef COSMOPOLITAN_APE_CONFIG_H_
#define COSMOPOLITAN_APE_CONFIG_H_
#if !(__ASSEMBLER__ + __LINKER__ + 0)
#define STATIC_SYMBOL(NAME, VALUE) \
asm(".equ\t" NAME "," VALUE "\n\t" \
".globl\t" NAME)
#endif /* !(__ASSEMBLER__ + __LINKER__ + 0) */
#endif /* COSMOPOLITAN_APE_CONFIG_H_ */
+23 -19
View File
@@ -1,5 +1,5 @@
/*-*- mode:unix-assembly; indent-tabs-mode:t; tab-width:8; coding:utf-8 -*-│
│vi: set et ft=asm ts=8 sw=8 fenc=utf-8 :vi│
│ vi: set noet ft=asm ts=8 sw=8 fenc=utf-8 :vi │
╞══════════════════════════════════════════════════════════════════════════════╡
│ Copyright 2020 Justine Alexandra Roberts Tunney │
│ │
@@ -31,7 +31,8 @@
// @see libc/nt/master.sh
// @see ape/ape.lds
// @see winimp
.macro .imp dll:req fn:req actual:req hint
.macro .imp dll:req fn:req actual:req
#ifdef __x86_64__
.dll "\dll"
.section ".piro.data.sort.iat.2.\dll\().2.\actual","aw",@progbits
.type \fn,@object
@@ -42,25 +43,28 @@
.hidden \fn
.previous
.section ".idata.ro.ilt.\dll\().2.\actual","a",@progbits
".Lidata.ilt.\dll\().\actual":
"idata.ilt.\dll\().\actual":
.quad RVA("\dll\().\actual")
.type ".Lidata.ilt.\dll\().\actual",@object
.size ".Lidata.ilt.\dll\().\actual",.-".Lidata.ilt.\dll\().\actual"
.type "idata.ilt.\dll\().\actual",@object
.size "idata.ilt.\dll\().\actual",.-"idata.ilt.\dll\().\actual"
.previous
.section ".idata.ro.hnt.\dll\().2.\actual","a",@progbits
"\dll\().\actual":
.ifnb \hint # hint i.e. guess function ordinal
.short \hint
.else
.short 0
.endif
.short 0 // hint
.asciz "\actual"
.align 2 # documented requirement
.align 2 // documented requirement
.globl "\dll\().\actual"
.hidden "\dll\().\actual"
.type "\dll\().\actual",@object
.size "\dll\().\actual",.-"\dll\().\actual"
.previous
#else
.section ".data.nt.\actual","aw",@progbits
.globl "\fn"
.balign 8
.weak "\actual"
"\fn": .quad "\actual"
#endif
.endm
// Defines DLL import.
@@ -68,16 +72,16 @@
.macro .dll name:req
.section ".idata.ro.idt.2.\name","aG",@progbits,"\name",comdat
.equ ".Lidata.idt.\name",.
.long RVA("idata.ilt.\name") # ImportLookupTable
.long 0 # TimeDateStamp
.long 0 # ForwarderChain
.long RVA(".Lidata.str.\name") # DllNameRva
.long RVA("idata.iat.\name") # ImportAddressTable
.long RVA("idata.ilt.\name") // ImportLookupTable
.long 0 // TimeDateStamp
.long 0 // ForwarderChain
.long RVA(".Lidata.str.\name") // DllNameRva
.long RVA("idata.iat.\name") // ImportAddressTable
.type ".Lidata.idt.\name",@object
.size ".Lidata.idt.\name",.-".Lidata.idt.\name"
.previous
.section ".idata.ro.ilt.\name\().1","aG",@progbits,"\name",comdat
.align __SIZEOF_POINTER__
.balign __SIZEOF_POINTER__
.type "idata.ilt.\name",@object
"idata.ilt.\name":
.previous/*
@@ -88,12 +92,12 @@
.quad 0
.previous
.section ".idata.ro.hnt.\name\().1","aG",@progbits,"\name",comdat
.align __SIZEOF_POINTER__
.balign __SIZEOF_POINTER__
.type "idata.hnt.\name",@object
.equ "idata.hnt.\name",.
.previous
.section ".piro.data.sort.iat.2.\name\().1","awG",@progbits,"\name",comdat
.align __SIZEOF_POINTER__
.balign __SIZEOF_POINTER__
.type "idata.iat.\name",@object
"idata.iat.\name":
.previous/*
+90
View File
@@ -0,0 +1,90 @@
/*-*- mode:unix-assembly; indent-tabs-mode:t; tab-width:8; coding:utf-8 -*-│
│ vi: set noet ft=asm ts=8 sw=8 fenc=utf-8 :vi │
╞══════════════════════════════════════════════════════════════════════════════╡
│ Copyright 2023 Justine Alexandra Roberts Tunney │
│ │
│ Permission to use, copy, modify, and/or distribute this software for │
│ any purpose with or without fee is hereby granted, provided that the │
│ above copyright notice and this permission notice appear in all copies. │
│ │
│ THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL │
│ WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED │
│ WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE │
│ AUTHOR BE LIABLE FOR ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL │
│ DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR │
│ PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER │
│ TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR │
│ PERFORMANCE OF THIS SOFTWARE. │
╚─────────────────────────────────────────────────────────────────────────────*/
#include "libc/macros.internal.h"
// Calls _start() function of loaded program.
//
// When the program entrypoint is called, all registers shall be
// cleared, with the exception of (1) %rdi will be equal to %rsp
// on FreeBSD and (2) %cl will contain the detected host OS code
//
// We clear all the general registers we can to have some wiggle
// room, to extend the behavior of this loader in the future. We
// don't need to clear the XMM registers because your APE loader
// should be compiled using gcc/clang's -mgeneral-regs-only flag
//
// @param rdi is passed through as-is
// @param rsi is address of entrypoint (becomes zero)
// @param rdx is passed through as-is
// @param rcx is passed through as-is
// @param r8 is stack pointer (becomes zero)
// @noreturn
Launch:
#ifdef __aarch64__
mov x16,x1
mov sp,x4
mov x1,0
mov x4,0
mov x5,0
mov x6,0
mov x7,0
mov x8,0
mov x9,0
mov x10,0
mov x11,0
mov x12,0
mov x13,0
mov x14,0
mov x15,0
mov x17,0
mov x19,0
mov x20,0
mov x21,0
mov x22,0
mov x23,0
mov x24,0
mov x25,0
mov x26,0
mov x27,0
mov x28,0
mov x29,0
mov x30,0
br x16
#else
mov %r8,%rsp
xor %r8d,%r8d
xor %r9d,%r9d
xor %r10d,%r10d
xor %r11d,%r11d
xor %r12d,%r12d
xor %r13d,%r13d
xor %r14d,%r14d
xor %r15d,%r15d
push %rsi
xor %esi,%esi
xor %ebp,%ebp
xor %ebx,%ebx
xor %eax,%eax
ret
#endif
.endfn Launch,globl
-249
View File
@@ -1,249 +0,0 @@
/*-*- mode:unix-assembly; indent-tabs-mode:t; tab-width:8; coding:utf-8 -*-│
│vi: set et ft=asm ts=8 tw=8 fenc=utf-8 :vi│
╞══════════════════════════════════════════════════════════════════════════════╡
│ Copyright 2021 Justine Alexandra Roberts Tunney │
│ │
│ Permission to use, copy, modify, and/or distribute this software for │
│ any purpose with or without fee is hereby granted, provided that the │
│ above copyright notice and this permission notice appear in all copies. │
│ │
│ THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL │
│ WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED │
│ WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE │
│ AUTHOR BE LIABLE FOR ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL │
│ DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR │
│ PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER │
│ TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR │
│ PERFORMANCE OF THIS SOFTWARE. │
╚─────────────────────────────────────────────────────────────────────────────*/
#include "libc/elf/def.h"
#include "libc/sysv/consts/prot.h"
#include "libc/macho.internal.h"
#include "libc/dce.h"
#include "libc/macros.internal.h"
// APE Loader Executable Structure
// Linux, FreeBSD, NetBSD, OpenBSD
.align 8
ehdr: .ascii "\177ELF"
.byte ELFCLASS64
.byte ELFDATA2LSB
.byte 1
.byte ELFOSABI_FREEBSD
.quad 0
.word ET_EXEC # e_type
.word EM_NEXGEN32E # e_machine
.long 1 # e_version
.quad _start # e_entry
.quad phdrs - ehdr # e_phoff
.quad 0 # e_shoff
.long 0 # e_flags
.word 64 # e_ehsize
.word 56 # e_phentsize
.word 4 # e_phnum
.word 0 # e_shentsize
.word 0 # e_shnum
.word 0 # e_shstrndx
.endobj ehdr,globl
.align 8
phdrs: .long PT_LOAD # p_type
.long PF_R|PF_X # p_flags
.quad 0 # p_offset
.quad ehdr # p_vaddr
.quad ehdr # p_paddr
.quad filesz # p_filesz
.quad filesz # p_memsz
.quad PAGESIZE # p_align
.long PT_LOAD # p_type
.long PF_R|PF_W # p_flags
.quad 0 # p_offset
.quad bss # p_vaddr
.quad bss # p_paddr
.quad 0 # p_filesz
.quad bsssize # p_memsz
.quad PAGESIZE # p_align
.long PT_GNU_STACK # p_type
.long PF_R|PF_W # p_flags
.quad 0 # p_offset
.quad 0 # p_vaddr
.quad 0 # p_paddr
.quad 0 # p_filesz
.quad 0 # p_memsz
.quad 16 # p_align
.long PT_NOTE # p_type
.long PF_R # p_flags
.quad note - ehdr # p_offset
.quad note # p_vaddr
.quad note # p_paddr
.quad notesize # p_filesz
.quad notesize # p_memsz
.quad 8 # p_align
.endobj phdrs
note: .long 2f-1f
.long 4f-3f
.long 1
1: .asciz "OpenBSD"
2: .align 4
3: .long 0
4: .long 2f-1f
.long 4f-3f
.long 1
1: .asciz "NetBSD"
2: .align 4
3: .long 901000000
4: .endobj note
notesize = . - note
.align 64,0 # for ape.S dd
.org 0x180 # for ape.S dd
// APE Loader XNU Header
//
// This header is dd'd backwards by the APE shell script when
// running on Mac OS X.
//
// @see ape/ape.S
macho: .long 0xFEEDFACE+1
.long MAC_CPU_NEXGEN32E
.long MAC_CPU_NEXGEN32E_ALL
.long MAC_EXECUTE
.long 5 # number of load commands
.long 60f-10f # size of all load commands
.long MAC_NOUNDEFS # flags
.long 0 # reserved
10: .long MAC_LC_SEGMENT_64
.long 20f-10b # unmaps first page dir
.ascin "__PAGEZERO",16 # consistent with linux
.quad 0,0x200000,0,0 # which forbids mem <2m
.long 0,0,0,0
20: .long MAC_LC_SEGMENT_64
.long 30f-20b
.ascin "__TEXT",16
.quad ehdr # vaddr
.quad 4096 # memsz
.quad 0 # file offset
.quad filesz # file size
.long PROT_EXEC|PROT_READ|PROT_WRITE # maxprot
.long PROT_EXEC|PROT_READ # initprot
.long 1 # segment section count
.long 0 # flags
210: .ascin "__text",16 # section name (.text)
.ascin "__TEXT",16
.quad _start # vaddr
.quad textsz # memsz
.long textoff # offset
.long 6 # align 2**6 = 64
.long 0 # reloc table offset
.long 0 # relocation count
.long MAC_S_ATTR_SOME_INSTRUCTIONS # section type & attributes
.long 0,0,0 # reserved
30: .long MAC_LC_SEGMENT_64
.long 40f-30b
.ascin "__DATA",16
.quad bss # vaddr
.quad bsssize # memsz
.quad 0 # offset
.quad 0 # file size
.long PROT_EXEC|PROT_READ|PROT_WRITE # maxprot
.long PROT_READ|PROT_WRITE # initprot
.long 1 # segment section count
.long 0 # flags
310: .ascin "__bss",16 # section name (.bss)
.ascin "__DATA",16
.quad bss # vaddr
.quad bsssize # memsz
.long 0 # offset
.long 12 # align 2**12 = 4096
.long 0 # reloc table offset
.long 0 # relocation count
.long MAC_S_ZEROFILL # section type & attributes
.long 0,0,0 # reserved
40: .long MAC_LC_UUID
.long 50f-40b
.quad 0x3fb29ee4ac6c87aa # uuid1
.quad 0xdd2c9bb866d9eef8 # uuid2
50: .long MAC_LC_UNIXTHREAD
.long 60f-50b # cmdsize
.long MAC_THREAD_NEXGEN32E # flavaflav
.long (520f-510f)/4 # count
510: .quad 0 # rax
.quad 0 # rbx
.quad 0 # rcx
.quad _HOSTXNU # rdx
.quad 0 # rdi
.quad 0 # rsi
.quad 0 # rbp
.quad 0 # rsp
.quad 0 # r8
.quad 0 # r9
.quad 0 # r10
.quad 0 # r11
.quad 0 # r12
.quad 0 # r13
.quad 0 # r14
.quad 0 # r15
.quad _start # rip
.quad 0 # rflags
.quad 0 # cs
.quad 0 # fs
.quad 0 # gs
520:
60:
.endobj macho
.align 64,0 # for ape.S dd
.org 0x400 # for ape.S dd
// Ape Loader Entrpoint
//
// This is normally called by the operating system. However it may
// be called by the Actually Portable Executables themselves, when
// re-executing a program. Just do this:
//
// memcpy(0x200000, loader)
// lea handoff(%rip),%rcx
// lea argblock(%rip),%rsp
// jmp 0x200400
//
// @see APE_LOADER_ENTRY
// @see ape/loader.h
_start:
// Hack for detecting M1 Rosetta environment.
// https://github.com/jart/cosmopolitan/issues/429#issuecomment-1166704377
cmp $-1,%ebx
jne 0f
cmp $+1,%edx
jne 0f
mov $_HOSTXNU,%dl
xor %ecx,%ecx
0: mov %rsp,%rsi
jmp ApeLoader
.endfn _start,globl
// System Call Entrpoint
//
// This function is used by the APE loader to make system calls.
// We also pass a reference to this function to the APE binary's
// _start() function. It's needed because on OpenBSD, msyscall()
// restricts which pages can issue system calls, and it can only
// be called once. Therefore if we want to be load and re-load a
// binary multiple times without calling the system execve(), we
// need to be able to handover the SYSCALL function. We hardcode
// this to a fixed address, but that shouldn't be used, since we
// would ideally want to move it to a random page in the future.
__syscall_loader:
clc
syscall
jc 1f
ret
1: neg %rax
ret
.endfn __syscall_loader,globl
+57 -100
View File
@@ -1,5 +1,5 @@
/*-*- mode:unix-assembly; indent-tabs-mode:t; tab-width:8; coding:utf-8 -*-│
│vi: set et ft=asm ts=8 tw=8 fenc=utf-8 :vi│
│ vi: set noet ft=asm ts=8 sw=8 fenc=utf-8 :vi │
╞══════════════════════════════════════════════════════════════════════════════╡
│ Copyright 2021 Justine Alexandra Roberts Tunney │
│ │
@@ -21,118 +21,75 @@
#include "libc/dce.h"
#include "libc/macros.internal.h"
// APE Loader Executable Structure for XNU
// Apple Mach-O Executable Headers
// Fixups are applied by objbincopy
// There must exist a MAC_LC_SEGMENT_64 for every PT_LOAD
.align 4096
macho: .long 0xFEEDFACE+1
.section .macho,"a",@progbits
.balign 64
.long 0xFEEDFACE+1
.long MAC_CPU_NEXGEN32E
.long MAC_CPU_NEXGEN32E_ALL
.long MAC_EXECUTE
.long 5 # number of load commands
.long 60f-10f # size of all load commands
.long MAC_NOUNDEFS # flags
.long 0 # reserved
.long 5 // number of load commands
.long 60f-10f // size of all load commands
.long MAC_NOUNDEFS|MAC_SPLIT_SEGS // flags
.long 0 // reserved
10: .long MAC_LC_SEGMENT_64
.long 20f-10b # unmaps first page dir
.ascin "__PAGEZERO",16 # consistent with linux
.quad 0,0x200000,0,0 # which forbids mem <2m
.long 20f-10b // unmaps first page dir
.ascin "__PAGEZERO",16 // consistent with linux
.quad 0,0x200000,0,0 // which forbids mem <2m
.long 0,0,0,0
20: .long MAC_LC_SEGMENT_64
.long 30f-20b
.ascin "__TEXT",16
.quad macho # vaddr
.quad 4096 # memsz
.quad 0 # file offset
.quad filesz # file size
.long PROT_EXEC|PROT_READ|PROT_WRITE # maxprot
.long PROT_EXEC|PROT_READ # initprot
.long 1 # segment section count
.long 0 # flags
210: .ascin "__text",16 # section name (.text)
.ascin "__TEXT",16
.quad _start # vaddr
.quad textsz # memsz
.long textoff # offset
.long 6 # align 2**3 = 64
.long 0 # reloc table offset
.long 0 # relocation count
.long MAC_S_ATTR_SOME_INSTRUCTIONS # section type & attributes
.long 0,0,0 # reserved
.quad 0 // vaddr
.quad 0 // memsz
.quad 0 // file offset
.quad 0 // file size
.long 0 // maxprot
.long 0 // initprot
.long 0 // segment section count
.long 0 // flags
30: .long MAC_LC_SEGMENT_64
.long 40f-30b
.ascin "__DATA",16
.quad bss # vaddr
.quad bsssize # memsz
.quad 0 # offset
.quad 0 # file size
.long PROT_EXEC|PROT_READ|PROT_WRITE # maxprot
.long PROT_READ|PROT_WRITE # initprot
.long 1 # segment section count
.long 0 # flags
310: .ascin "__bss",16 # section name (.bss)
.ascin "__DATA",16
.quad bss # vaddr
.quad bsssize # memsz
.long 0 # offset
.long 12 # align 2**12 = 4096
.long 0 # reloc table offset
.long 0 # relocation count
.long MAC_S_ZEROFILL # section type & attributes
.long 0,0,0 # reserved
.ascin "__RODATA",16
.quad 0 // vaddr
.quad 0 // memsz
.quad 0 // file offset
.quad 0 // file size
.long 0 // maxprot
.long 0 // initprot
.long 0 // segment section count
.long 0 // flags
40: .long MAC_LC_UUID
.long 50f-40b
.quad 0x3fb29ee4ac6c87aa # uuid1
.quad 0xdd2c9bb866d9eef8 # uuid2
.quad 0x4527148ba7a513ef // uuid1
.quad 0x56fa865940665e8f // uuid2
50: .long MAC_LC_UNIXTHREAD
.long 60f-50b # cmdsize
.long MAC_THREAD_NEXGEN32E # flavaflav
.long (520f-510f)/4 # count
510: .quad 0 # rax
.quad 0 # rbx
.quad 0 # rcx
.quad _HOSTXNU # rdx
.quad 0 # rdi
.quad 0 # rsi
.quad 0 # rbp
.quad 0 # rsp
.quad 0 # r8
.quad 0 # r9
.quad 0 # r10
.quad 0 # r11
.quad 0 # r12
.quad 0 # r13
.quad 0 # r14
.quad 0 # r15
.quad _start # rip
.quad 0 # rflags
.quad 0 # cs
.quad 0 # fs
.quad 0 # gs
.long 60f-50b // cmdsize
.long MAC_THREAD_NEXGEN32E // flavaflav
.long (520f-510f)/4 // count
510: .quad 0 // rax
.quad 0 // rbx
.quad 0 // rcx
.quad 0 // rdx
.quad 0 // rdi
.quad 0 // rsi
.quad 0 // rbp
.quad 0 // rsp
.quad 0 // r8
.quad 0 // r9
.quad 0 // r10
.quad 0 // r11
.quad 0 // r12
.quad 0 // r13
.quad 0 // r14
.quad 0 // r15
.quad XnuEntrypoint // rip
.quad 0 // rflags
.quad 0 // cs
.quad 0 // fs
.quad 0 // gs
520:
60:
.endobj macho,globl
.align 64
_start:
// Hack for detecting M1 Rosetta environment.
// https://github.com/jart/cosmopolitan/issues/429#issuecomment-1166704377
cmp $-1,%ebx
jne 0f
cmp $+1,%edx
jne 0f
mov $_HOSTXNU,%dl
xor %ecx,%ecx
0: mov %rsp,%rsi
jmp ApeLoader
.endfn _start,globl
__syscall_loader:
clc
syscall
jc 1f
ret
1: neg %rax
ret
.endfn __syscall_loader,globl
-42
View File
@@ -1,42 +0,0 @@
/*-*- mode: ld-script; indent-tabs-mode: nil; tab-width: 2; coding: utf-8 -*-│
│vi: set et sts=2 tw=2 fenc=utf-8 :vi│
╞══════════════════════════════════════════════════════════════════════════════╡
│ Copyright 2021 Justine Alexandra Roberts Tunney │
│ │
│ Permission to use, copy, modify, and/or distribute this software for │
│ any purpose with or without fee is hereby granted, provided that the │
│ above copyright notice and this permission notice appear in all copies. │
│ │
│ THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL │
│ WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED │
│ WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE │
│ AUTHOR BE LIABLE FOR ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL │
│ DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR │
│ PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER │
│ TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR │
│ PERFORMANCE OF THIS SOFTWARE. │
╚─────────────────────────────────────────────────────────────────────────────*/
ENTRY(_start)
SECTIONS {
. = 0x200000;
.text : {
*(.text)
*(.rodata .rodata.*)
. = ALIGN(4096);
}
filesz = . - macho;
textsz = . - _start;
.bss ALIGN(4096) : {
bss = .;
*(.bss)
. = ALIGN(4096);
}
memsz = . - macho;
/DISCARD/ : {
*(.*)
}
}
bsssize = SIZEOF(.bss);
textoff = _start - macho;
+820 -484
View File
File diff suppressed because it is too large. Load diff
-20
View File
@@ -1,20 +0,0 @@
#ifndef COSMOPOLITAN_APE_LOADER_H_
#define COSMOPOLITAN_APE_LOADER_H_
#define APE_LOADER_BASE 0x200000
#define APE_LOADER_SIZE 0x200000
#define APE_LOADER_ENTRY 0x200400
#define APE_LOADER_BSS (PAGESIZE * 2)
#define APE_LOADER_STACK 0x7f0000000000
#define APE_BLOCK_BASE 0x7e0000000000
#define APE_BLOCK_SIZE 0x000200000000
struct ApeLoader {
int fd;
int os;
char *prog;
char *page;
void *systemcall;
};
#endif /* COSMOPOLITAN_APE_LOADER_H_ */
+71 -19
View File
@@ -1,7 +1,7 @@
/*-*- mode: ld-script; indent-tabs-mode: nil; tab-width: 2; coding: utf-8 -*-│
│vi: set et sts=2 tw=2 fenc=utf-8 :vi│
│ vi: set et sts=2 sw=2 fenc=utf-8 :vi │
╞══════════════════════════════════════════════════════════════════════════════╡
│ Copyright 2021 Justine Alexandra Roberts Tunney │
│ Copyright 2023 Justine Alexandra Roberts Tunney │
│ │
│ Permission to use, copy, modify, and/or distribute this software for │
│ any purpose with or without fee is hereby granted, provided that the │
@@ -16,28 +16,80 @@
│ TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR │
│ PERFORMANCE OF THIS SOFTWARE. │
╚─────────────────────────────────────────────────────────────────────────────*/
ENTRY(_start)
ENTRY(ElfEntrypoint)
PHDRS {
text PT_LOAD FLAGS(1) FILEHDR PHDRS; /* PF_X */
rodata PT_LOAD FLAGS(4); /* PF_R */
stack PT_GNU_STACK FLAGS(6); /* PF_W|PF_R */
note PT_NOTE FLAGS(4); /* PF_R */
}
SECTIONS {
. = 0x200000;
. = SEGMENT_START("text-segment", 0x7f000000);
__executable_start = .;
. += SIZEOF_HEADERS;
.macho : {
KEEP(*(.macho))
} :text
.note : {
KEEP(*(.note))
} :text :note
.text : {
*(.text)
*(.rodata .rodata.*)
. = ALIGN(64);
}
filesz = . - ehdr;
textsz = . - _start;
.bss ALIGN(4096) : {
bss = .;
*(.bss)
. = ALIGN(4096);
}
*(.text .text.* .gnu.linkonce.t.*)
} :text
.rodata ALIGN(CONSTANT(COMMONPAGESIZE)) : {
*(.rodata .rodata.* .gnu.linkonce.r.*)
} :rodata
.stack : {
*(.stack)
} :stack
_end = .;
memsz = _end - ehdr;
.stab 0 : { *(.stab) }
.stabstr 0 : { *(.stabstr) }
.stab.excl 0 : { *(.stab.excl) }
.stab.exclstr 0 : { *(.stab.exclstr) }
.stab.index 0 : { *(.stab.index) }
.stab.indexstr 0 : { *(.stab.indexstr) }
.gnu.build.attributes : { *(.gnu.build.attributes .gnu.build.attributes.*) }
.debug 0 : { *(.debug) }
.line 0 : { *(.line) }
.debug_srcinfo 0 : { *(.debug_srcinfo) }
.debug_sfnames 0 : { *(.debug_sfnames) }
.debug_aranges 0 : { *(.debug_aranges) }
.debug_pubnames 0 : { *(.debug_pubnames) }
.debug_info 0 : { *(.debug_info .gnu.linkonce.wi.*) }
.debug_abbrev 0 : { *(.debug_abbrev) }
.debug_line 0 : { *(.debug_line .debug_line.* .debug_line_end) }
.debug_frame 0 : { *(.debug_frame) }
.debug_str 0 : { *(.debug_str) }
.debug_loc 0 : { *(.debug_loc) }
.debug_macinfo 0 : { *(.debug_macinfo) }
.debug_weaknames 0 : { *(.debug_weaknames) }
.debug_funcnames 0 : { *(.debug_funcnames) }
.debug_typenames 0 : { *(.debug_typenames) }
.debug_varnames 0 : { *(.debug_varnames) }
.debug_pubtypes 0 : { *(.debug_pubtypes) }
.debug_ranges 0 : { *(.debug_ranges) }
.debug_macro 0 : { *(.debug_macro) }
.debug_addr 0 : { *(.debug_addr) }
.debug_loclists 0 : { *(.debug_loclists) }
.debug_rnglists 0 : { *(.debug_rnglists) }
.debug_line_str 0 : { *(.debug_line_str) }
.debug_extra 0 : { *(.debug_line_str) }
.debug_pubtypes 0 : { *(.debug_pubtypes) }
.gnu.attributes 0 : { KEEP(*(.gnu.attributes)) }
/DISCARD/ : {
*(.*)
}
}
bsssize = SIZEOF(.bss);
textoff = _start - ehdr;
+22 -145
View File
@@ -1,5 +1,5 @@
/*-*- mode:unix-assembly; indent-tabs-mode:t; tab-width:8; coding:utf-8 -*-│
│vi: set et ft=asm ts=8 sw=8 fenc=utf-8 :vi│
│ vi: set noet ft=asm ts=8 sw=8 fenc=utf-8 :vi │
╞══════════════════════════════════════════════════════════════════════════════╡
│ Copyright 2020 Justine Alexandra Roberts Tunney │
│ │
@@ -22,119 +22,6 @@
#ifdef __ASSEMBLER__
/* clang-format off */
/**
* @fileoverview Macros relevant to αcτµαlly pδrταblε εxεcµταblε.
*/
// Calls near (i.e. pc+pcrel<64kB) FUNCTION.
// @mode long,legacy,real
// @cost 9 bytes overhead
.macro rlcall function:req
.byte 0x50 # push %[er]ax
.byte 0xb8,0,0 # mov $?,%[e]ax
jmp 911f
.byte 0x58 # pop %[er]ax
.byte 0xe8 # call Jvds
.long \function\()-.-4
jmp 912f
911: .byte 0x58 # pop %[er]ax
.byte 0xe8 # call Jvds
.short \function\()-.-2
912:
.endm
// Loads far (i.e. <1mb) abs constexpr ADDRESS into ES:DI+EDX+RDX.
// @mode long,legacy,real
.macro movesdi address:req
.byte 0xbf # mov $0x????xxxx,%[e]di
.short \address>>4
.byte 0x8e,0xc7 # mov %di,%es
.byte 0xbf # mov $0x????xxxx,%[e]di
.short \address&0xf
jmp 297f
.byte 0xbf # mov $0x????xxxx,%edi
.long \address
297:
.endm
// Loads 16-bit CONSTEXPR into Qw-register w/ optional zero-extend.
// @mode long,legacy,real
.macro bbmov constexpr:req abcd abcd.hi:req abcd.lo:req
.ifnb \abcd
.if (\constexpr)<128 && (\constexpr)>=0
pushpop \constexpr,\abcd
.exitm
.endif
.endif
movb $(\constexpr)>>8&0xff,\abcd.hi
movb $(\constexpr)&0xff,\abcd.lo
.endm
// Compares 16-bit CONSTEXPR with Qw-register.
// @mode long,legacy,real
.macro bbcmp constexpr:req abcd.hi:req abcd.lo:req
cmpb $(\constexpr)>>8&0xff,\abcd.hi
jnz 387f
cmpb $(\constexpr)&0xff,\abcd.lo
387:
.endm
// Adds 16-bit CONSTEXPR to Qw-register.
// @mode long,legacy,real
.macro bbadd constexpr:req abcd.hi:req abcd.lo:req
addb $(\constexpr)&0xff,\abcd.lo
.if (\constexpr) != 0
adcb $(\constexpr)>>8&0xff,\abcd.hi
.endif
.endm
// Subtracts 16-bit CONSTEXPR from Qw-register.
// @mode long,legacy,real
.macro bbsub constexpr:req abcd.hi:req abcd.lo:req
subb $(\constexpr)&0xff,\abcd.lo
.if (\constexpr) != 0
sbbb $(\constexpr)>>8&0xff,\abcd.hi
.endif
.endm
// Ands Qw-register with 16-bit CONSTEXPR.
// @mode long,legacy,real
.macro bband constexpr:req abcd.hi:req abcd.lo:req
.if ((\constexpr)&0xff) != 0xff || ((\constexpr)>>8&0xff) == 0xff
andb $(\constexpr)&0xff,\abcd.lo
.endif
.if ((\constexpr)>>8&0xff) != 0xff
andb $(\constexpr)>>8&0xff,\abcd.hi
.endif
.endm
// Ors Qw-register with 16-bit CONSTEXPR.
// @mode long,legacy,real
.macro bbor constexpr:req abcd.hi:req abcd.lo:req
.if ((\constexpr)&0xff) != 0 || ((\constexpr)>>8&0xff) != 0
orb $(\constexpr)&0xff,\abcd.lo
.endif
.if ((\constexpr)>>8&0xff) != 0
orb $(\constexpr)>>8&0xff,\abcd.hi
.endif
.endm
// Performs ACTION only if in real mode.
// @mode long,legacy,real
.macro rlo clobber:req action:vararg
990: mov $0,\clobber
.if .-990b!=3
.error "bad clobber or assembler mode"
.endif
991: \action
.rept 2-(.-991b)
nop
.endr
.if .-991b!=2
.error "ACTION must be 1-2 bytes"
.endif
.endm
// Initializes real mode stack.
// The most holiest of holy code.
// @mode real
@@ -197,9 +84,10 @@
/* clang-format on */
#elif defined(__LINKER__)
#define BCX_NIBBLE(X) ((((X)&0xf) > 0x9) ? ((X)&0xf) + 0x37 : ((X)&0xf) + 0x30)
#define BCX_OCTET(X) ((BCX_NIBBLE((X) >> 4) << 8) | (BCX_NIBBLE((X) >> 0) << 0))
#define BCX_INT16(X) ((BCX_OCTET((X) >> 8) << 16) | (BCX_OCTET((X) >> 0) << 0))
#define BCX_NIBBLE(X) \
((((X) & 0xf) > 0x9) ? ((X) & 0xf) + 0x37 : ((X) & 0xf) + 0x30)
#define BCX_OCTET(X) ((BCX_NIBBLE((X) >> 4) << 8) | (BCX_NIBBLE((X) >> 0) << 0))
#define BCX_INT16(X) ((BCX_OCTET((X) >> 8) << 16) | (BCX_OCTET((X) >> 0) << 0))
#define BCXSTUB(SYM, X) \
HIDDEN(SYM##_bcx0 = BCX_INT16((X) >> 48)); \
HIDDEN(SYM##_bcx1 = BCX_INT16((X) >> 32)); \
@@ -211,12 +99,12 @@
*
* <p>This allows linker scripts to generate printf commands.
*/
#define BCO_OCTET(X) (((X)&0x7) + 0x30)
#define BCO_OCTET(X) (((X) & 0x7) + 0x30)
#define BCOB_UNIT(X) \
((BCO_OCTET((X) >> 0) << 24) | (BCO_OCTET((X) >> 3) << 16) | \
(BCO_OCTET(((X)&0xff) >> 6) << 8) | 0x5c)
(BCO_OCTET(((X) & 0xff) >> 6) << 8) | 0x5c)
#define PFBYTE(SYM, X, I) HIDDEN(SYM##_bcs##I = BCOB_UNIT((X) >> ((I)*8)))
#define PFBYTE(SYM, X, I) HIDDEN(SYM##_bcs##I = BCOB_UNIT((X) >> ((I) * 8)))
#define PFSTUB2(SYM, X) \
HIDDEN(SYM = (X)); \
PFBYTE(SYM, X, 0); \
@@ -253,23 +141,16 @@
: (X) < 10000000 ? BCD_RIGHT(BCD_SMEAR((X) / 1000)) \
: (X) < 100000000 ? BCD_RIGHT(BCD_SMEAR((X) / 10000)) \
: 0xffffffffffffffff)
#define BCD_RIGHT(X) \
(((X)) < 10000 ? 0x20202020 \
: (X) < 100000 ? 0x20202030 + \
(X) % 10 \
: (X) < 1000000 ? 0x20203030 + \
((X) / 10) % 10 + \
(X) % 10 * 0x100 \
: (X) < 10000000 ? 0x20303030 + \
((X) / 100) % 10 + \
((X) / 10) % 10 * 0x100 + \
(X) % 10 * 0x10000 \
: (X) < 100000000 ? 0x30303030 + \
((X) / 1000) % 10 + \
((X) / 100) % 10 * 0x100 + \
((X) / 10) % 10 * 0x10000 + \
(X) % 10 * 0x1000000 \
: 0xffffffffffffffff)
#define BCD_RIGHT(X) \
(((X)) < 10000 ? 0x20202020 \
: (X) < 100000 ? 0x20202030 + (X) % 10 \
: (X) < 1000000 ? 0x20203030 + ((X) / 10) % 10 + (X) % 10 * 0x100 \
: (X) < 10000000 ? 0x20303030 + ((X) / 100) % 10 + \
((X) / 10) % 10 * 0x100 + (X) % 10 * 0x10000 \
: (X) < 100000000 \
? 0x30303030 + ((X) / 1000) % 10 + ((X) / 100) % 10 * 0x100 + \
((X) / 10) % 10 * 0x10000 + (X) % 10 * 0x1000000 \
: 0xffffffffffffffff)
/**
* Laying out the GDT entries for a TSS for bare metal operation.
@@ -278,15 +159,11 @@
HIDDEN(SYM##_desc_ent0 = TSSDESC_ENT0(BASE, LIM)); \
HIDDEN(SYM##_desc_ent1 = TSSDESC_ENT1(BASE)); \
ASSERT((LIM) >= 0 && (LIM) <= 0xffff, "bare metal TSS is suspiciously fat")
#define TSSDESC_ENT0(BASE, LIM) \
(((LIM) << 0 & 0x000000000000ffff) | \
((BASE) << 16 & 0x000000ffffff0000) | \
0x89 << 40 | \
((LIM) >> 16 << 48 & 0x000f000000000000) | \
0x2 << 52 | \
#define TSSDESC_ENT0(BASE, LIM) \
(((LIM) << 0 & 0x000000000000ffff) | ((BASE) << 16 & 0x000000ffffff0000) | \
0x89 << 40 | ((LIM) >> 16 << 48 & 0x000f000000000000) | 0x2 << 52 | \
((BASE) >> 24 << 56 & 0xff00000000000000))
#define TSSDESC_ENT1(BASE) \
((BASE) >> 32 << 0 & 0x00000000ffffffff)
#define TSSDESC_ENT1(BASE) ((BASE) >> 32 << 0 & 0x00000000ffffffff)
#endif /* __ASSEMBLER__ */
#endif /* APE_MACROS_H_ */
+25 -27
View File
@@ -1,34 +1,32 @@
#ifndef COSMOPOLITAN_APE_SECTIONS_INTERNAL_H_
#define COSMOPOLITAN_APE_SECTIONS_INTERNAL_H_
#if !(__ASSEMBLER__ + __LINKER__ + 0)
COSMOPOLITAN_C_START_
extern unsigned char _base[];
extern unsigned char _ehead[];
extern unsigned char _etext[];
extern unsigned char _edata[];
extern unsigned char _ezip[];
extern unsigned char _end[];
extern unsigned char _ereal[];
extern unsigned char _tdata_start[];
extern unsigned char _tdata_end[];
extern unsigned char _tbss_start[];
extern unsigned char _tbss_end[];
extern unsigned char __privileged_start[];
extern unsigned char __privileged_addr[];
extern unsigned char __privileged_size[];
extern unsigned char __privileged_end[];
extern unsigned char __test_start[];
extern unsigned char __ro[];
extern unsigned char *__relo_start[];
extern unsigned char *__relo_end[];
extern uint8_t __zip_start[];
extern uint8_t __zip_end[];
extern uint8_t __data_start[];
extern uint8_t __data_end[];
extern uint8_t __bss_start[];
extern uint8_t __bss_end[];
extern const char __notices[] __attribute__((__weak__));
extern unsigned char __executable_start[] __attribute__((__weak__));
extern unsigned char __privileged_start[] __attribute__((__weak__));
extern unsigned char _ehead[] __attribute__((__weak__));
extern unsigned char _etext[] __attribute__((__weak__));
extern unsigned char _edata[] __attribute__((__weak__));
extern unsigned char _ezip[] __attribute__((__weak__));
extern unsigned char _end[] __attribute__((__weak__));
extern unsigned char _ereal[] __attribute__((__weak__));
extern unsigned char _tdata_start[] __attribute__((__weak__));
extern unsigned char _tdata_end[] __attribute__((__weak__));
extern unsigned char _tbss_start[] __attribute__((__weak__));
extern unsigned char _tbss_end[] __attribute__((__weak__));
extern unsigned char _tls_align[] __attribute__((__weak__));
extern unsigned char _tdata_align[] __attribute__((__weak__));
extern unsigned char _tbss_align[] __attribute__((__weak__));
extern unsigned char __test_start[] __attribute__((__weak__));
extern unsigned char __ro[] __attribute__((__weak__));
extern unsigned char __data_start[] __attribute__((__weak__));
extern unsigned char __data_end[] __attribute__((__weak__));
extern unsigned char __bss_start[] __attribute__((__weak__));
extern unsigned char __bss_end[] __attribute__((__weak__));
extern unsigned long __got_start[] __attribute__((__weak__));
extern unsigned long __got_end[] __attribute__((__weak__));
extern unsigned char ape_phdrs[] __attribute__((__weak__));
COSMOPOLITAN_C_END_
#endif /* !(__ASSEMBLER__ + __LINKER__ + 0) */
#endif /* COSMOPOLITAN_APE_SECTIONS_INTERNAL_H_ */
+74
View File
@@ -0,0 +1,74 @@
/*-*- mode:unix-assembly; indent-tabs-mode:t; tab-width:8; coding:utf-8 -*-│
│ vi: set noet ft=asm ts=8 sw=8 fenc=utf-8 :vi │
╞══════════════════════════════════════════════════════════════════════════════╡
│ Copyright 2023 Justine Alexandra Roberts Tunney │
│ │
│ Permission to use, copy, modify, and/or distribute this software for │
│ any purpose with or without fee is hereby granted, provided that the │
│ above copyright notice and this permission notice appear in all copies. │
│ │
│ THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL │
│ WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED │
│ WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE │
│ AUTHOR BE LIABLE FOR ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL │
│ DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR │
│ PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER │
│ TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR │
│ PERFORMANCE OF THIS SOFTWARE. │
╚─────────────────────────────────────────────────────────────────────────────*/
#include "libc/dce.h"
#include "ape/ape.h"
#include "libc/macros.internal.h"
#ifdef __aarch64__
_start: mov x1,sp
and sp,x1,#-16
mov x29,0
bl ApeLoader
.endfn _start,globl
#else
XnuEntrypoint:
mov $_HOSTXNU,%dl // xnu's not unix!
ElfEntrypoint:
mov %rsp,%rsi // save real stack
andq $-16,%rsp // force SSE alignment
call ApeLoader
.endfn ElfEntrypoint,globl
.endfn XnuEntrypoint,globl
.section .note,"a",@progbits
.balign 4
openbsd.ident:
.long 2f-1f
.long 4f-3f
.long 1
1: .asciz "OpenBSD"
2: .balign 4
3: .long 0
4: .size openbsd.ident,.-openbsd.ident
.type openbsd.ident,@object
.balign 4
netbsd.ident:
.long 2f-1f
.long 4f-3f
.long 1
1: .asciz "NetBSD"
2: .balign 4
3: .long 901000000
4: .size netbsd.ident,.-netbsd.ident
.type netbsd.ident,@object
.balign 4
ape.ident:
.long 2f-1f
.long 4f-3f
.long 1
1: .asciz "APE"
2: .balign 4
3: .long APE_VERSION_NOTE
4: .size ape.ident,.-ape.ident
.type ape.ident,@object
#endif
+49
View File
@@ -0,0 +1,49 @@
/*-*- mode:unix-assembly; indent-tabs-mode:t; tab-width:8; coding:utf-8 -*-│
│ vi: set noet ft=asm ts=8 sw=8 fenc=utf-8 :vi │
╞══════════════════════════════════════════════════════════════════════════════╡
│ Copyright 2023 Justine Alexandra Roberts Tunney │
│ │
│ Permission to use, copy, modify, and/or distribute this software for │
│ any purpose with or without fee is hereby granted, provided that the │
│ above copyright notice and this permission notice appear in all copies. │
│ │
│ THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL │
│ WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED │
│ WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE │
│ AUTHOR BE LIABLE FOR ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL │
│ DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR │
│ PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER │
│ TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR │
│ PERFORMANCE OF THIS SOFTWARE. │
╚─────────────────────────────────────────────────────────────────────────────*/
#include "libc/macros.internal.h"
// Invokes system call.
//
// This function has eight parameters. The first seven are for
// arguments passed along to the system call. The eight is for
// the magic number that indicates which system call is called
//
// The return value follows the Linux kernel convention, where
// errors are returned as `-errno`. BSD systems are normalized
// to follow this convention automatically.
SystemCall:
#ifdef __aarch64__
mov x8,x7
mov x9,0
adds x9,x9,0
svc 0
bcs 1f
ret
1: neg x0,x0
ret
#else
mov %rcx,%r10
mov 16(%rsp),%eax
clc
syscall
jnc 1f
neg %rax
1: ret
#endif
.endfn SystemCall,globl
BIN
View File
Binary file not shown.
Binary file not shown.
Binary file not shown.
BIN
View File
Binary file not shown.
Binary file not shown.
BIN
View File
Binary file not shown.
BIN
View File
Binary file not shown.
Binary file not shown.
BIN
View File
Binary file not shown.
Binary file not shown.
BIN
View File
Binary file not shown.
Binary file not shown.
BIN
View File
Binary file not shown.
Binary file not shown.
BIN
View File
Binary file not shown.
Binary file not shown.
+58
View File
@@ -0,0 +1,58 @@
--nocompress-debug-sections
--noexecstack
-Wa,--nocompress-debug-sections
-Wa,--noexecstack
-Wa,-msse2avx
-Werror=maybe-uninitialized
-Wno-literal-suffix
-Wno-unused-but-set-variable
-Wunsafe-loop-optimizations
-fbranch-target-load-optimize
-fcx-limited-range
-fdelete-dead-exceptions
-femit-struct-debug-baseonly
-ffp-int-builtin-inexact
-finline-functions-called-once
-fipa-pta
-fivopts
-flimit-function-alignment
-fmerge-constants
-fmodulo-sched
-fmodulo-sched-allow-regmoves
-fno-align-jumps
-fno-align-labels
-fno-align-loops
-fno-code-hoisting
-fno-cx-limited-range
-fno-fp-int-builtin-inexact
-fno-gnu-unique
-fno-inline-functions-called-once
-fno-instrument-functions
-fno-schedule-insns2
-fno-whole-program
-fopt-info-vec
-fopt-info-vec-missed
-freg-struct-return
-freschedule-modulo-scheduled-loops
-frounding-math
-fsched2-use-superblocks
-fschedule-insns
-fschedule-insns2
-fshrink-wrap
-fshrink-wrap-separate
-fsignaling-nans
-fstack-clash-protection
-ftracer
-ftrapv
-ftree-loop-im
-ftree-loop-vectorize
-funsafe-loop-optimizations
-fversion-loops-for-strides
-fwhole-program
-gdescribe-dies
-gstabs
-mcall-ms2sysv-xlogues
-mdispatch-scheduler
-mfpmath=sse+387
-mmitigate-rop
-mno-fentry
BIN
View File
Binary file not shown.
Binary file not shown.
BIN
View File
Binary file not shown.
Binary file not shown.
BIN
View File
Binary file not shown.
Binary file not shown.
BIN
View File
Binary file not shown.
Binary file not shown.
BIN
View File
Binary file not shown.
BIN
View File
Binary file not shown.
Binary file not shown.
BIN
View File
Binary file not shown.
Binary file not shown.
BIN
View File
Binary file not shown.
Binary file not shown.
BIN
View File
Binary file not shown.
BIN
View File
Binary file not shown.
Binary file not shown.
Binary file not shown.
BIN
View File
Binary file not shown.
BIN
View File
Binary file not shown.
Binary file not shown.
+193 -86
View File
@@ -1,21 +1,54 @@
#-*-mode:makefile-gmake;indent-tabs-mode:t;tab-width:8;coding:utf-8-*-┐
#───vi: set et ft=make ts=8 tw=8 fenc=utf-8 :vi───────────────────────┘
#── vi: set et ft=make ts=8 sw=8 fenc=utf-8 :vi ──────────────────────┘
# Default Mode
#
# - `make`
# - Optimized
# - Backtraces
# - Debuggable
# - Syscall tracing
# - Function tracing
# - Reasonably small
#
ifeq ($(MODE),)
CONFIG_CCFLAGS += $(BACKTRACES) $(FTRACE) -O2
ENABLE_FTRACE = 1
CONFIG_OFLAGS ?= -g -ggdb
CONFIG_CCFLAGS += -O2 $(BACKTRACES)
CONFIG_CPPFLAGS += -DSYSDEBUG
TARGET_ARCH ?= -msse3
endif
ifeq ($(MODE), x86_64)
ENABLE_FTRACE = 1
CONFIG_OFLAGS ?= -g -ggdb
CONFIG_CCFLAGS += -O2 $(BACKTRACES)
CONFIG_CPPFLAGS += -DSYSDEBUG
endif
ifeq ($(MODE), aarch64)
ENABLE_FTRACE = 1
CONFIG_OFLAGS ?= -g -ggdb
CONFIG_CCFLAGS += -O2 $(BACKTRACES)
CONFIG_CPPFLAGS += -DSYSDEBUG
endif
# Zero Optimization Mode
#
# - Goes 2x slower
# - Supports --strace
# - Unsupports --ftrace
# - Better GDB debugging
#
ifeq ($(MODE), zero)
ENABLE_FTRACE = 1
CONFIG_OFLAGS ?= -g -ggdb
OVERRIDE_CFLAGS += -O0
OVERRIDE_CXXFLAGS += -O0
CONFIG_CPPFLAGS += -DSYSDEBUG
endif
ifeq ($(MODE), aarch64-zero)
CONFIG_OFLAGS ?= -g -ggdb
OVERRIDE_CFLAGS += -O0 -fdce
OVERRIDE_CXXFLAGS += -O0 -fdce
CONFIG_CPPFLAGS += -DSYSDEBUG
endif
# Fast Build Mode
#
@@ -25,12 +58,11 @@ endif
# - Function tracing
# - Some optimizations
# - Limited Backtraces
# - Compiles 28% faster
#
ifeq ($(MODE),fastbuild)
CONFIG_CCFLAGS += $(BACKTRACES) $(FTRACE) -O
ifeq ($(MODE), fastbuild)
ENABLE_FTRACE = 1
CONFIG_CCFLAGS += $(BACKTRACES) -O
CONFIG_CPPFLAGS += -DSYSDEBUG -DDWARFLESS
CONFIG_OFLAGS += -g0
CONFIG_LDFLAGS += -S
TARGET_ARCH ?= -msse3
endif
@@ -48,12 +80,15 @@ endif
# - GCC 8+ hoists check fails into .text.cold, thus minimizing impact
#
ifeq ($(MODE), opt)
CONFIG_CPPFLAGS += -DNDEBUG -DSYSDEBUG -msse2avx -Wa,-msse2avx
CONFIG_CCFLAGS += $(BACKTRACES) $(FTRACE) -O3 -fmerge-all-constants
ENABLE_FTRACE = 1
CONFIG_OFLAGS ?= -g -ggdb
CONFIG_CPPFLAGS += -DNDEBUG -DSYSDEBUG
CONFIG_CCFLAGS += $(BACKTRACES) -O3 -fmerge-all-constants
TARGET_ARCH ?= -march=native
endif
# Optimized Linux Mode
# The Fastest Mode of All
#
# - `make MODE=optlinux`
# - Turns on red zone
@@ -63,9 +98,10 @@ endif
# - Turns off support for other operating systems
#
ifeq ($(MODE), optlinux)
CONFIG_CPPFLAGS += -DNDEBUG -msse2avx -Wa,-msse2avx -DSUPPORT_VECTOR=1
CONFIG_OFLAGS ?= -g -ggdb
CONFIG_CPPFLAGS += -DNDEBUG -DSYSDEBUG -DSUPPORT_VECTOR=1
CONFIG_CCFLAGS += -O3 -fmerge-all-constants
DEFAULT_COPTS += -mred-zone
CONFIG_COPTS += -mred-zone
TARGET_ARCH ?= -march=native
endif
@@ -77,6 +113,7 @@ endif
# - More optimized
# - Reasonably small
# - Numeric backtraces
# - No DWARF data bloat
# - Toilsome debuggability
# - assert() statements removed
# - DCHECK_xx() statements removed
@@ -84,45 +121,38 @@ endif
# - CHECK_xx() won't leak strings into binary
#
ifeq ($(MODE), rel)
CONFIG_CPPFLAGS += -DNDEBUG
CONFIG_CPPFLAGS += -DNDEBUG -DDWARFLESS
CONFIG_CCFLAGS += $(BACKTRACES) -O2
TARGET_ARCH ?= -msse3
PYFLAGS += -O1
endif
# Asan Mode
#
# Safer binaries good for backend production serving.
#
# - `make MODE=asan`
# - Memory safety
# - Production worthy
# - Backtraces
# - Debuggability
# - Larger binaries
#
ifeq ($(MODE), asan)
CONFIG_CCFLAGS += $(BACKTRACES) -O2 -DSYSDEBUG
CONFIG_COPTS += -fsanitize=address
TARGET_ARCH ?= -msse3
endif
# Debug Mode
#
# - `make MODE=dbg`
# - Backtraces
# - Enables asan
# - Enables ubsan (TODO)
# - Enables ubsan
# - Stack canaries
# - No optimization (TODO)
# - No optimization
# - Enormous binaries
#
ifeq ($(MODE), dbg)
CONFIG_CPPFLAGS += -DMODE_DBG
CONFIG_CCFLAGS += $(BACKTRACES) $(FTRACE) -DSYSDEBUG -O -fno-inline
CONFIG_COPTS += -fsanitize=address -fsanitize=undefined
ENABLE_FTRACE = 1
CONFIG_OFLAGS ?= -g -ggdb
CONFIG_CPPFLAGS += -DMODE_DBG -D__SANITIZE_UNDEFINED__
CONFIG_CCFLAGS += $(BACKTRACES) -DSYSDEBUG -O0 -fno-inline
CONFIG_COPTS += -fsanitize=undefined
TARGET_ARCH ?= -msse3
OVERRIDE_CCFLAGS += -fno-pie
QUOTA ?= -C64 -L300
endif
ifeq ($(MODE), aarch64-dbg)
ENABLE_FTRACE = 1
CONFIG_OFLAGS ?= -g -ggdb
CONFIG_CPPFLAGS += -DMODE_DBG -D__SANITIZE_UNDEFINED__
CONFIG_CCFLAGS += $(BACKTRACES) -DSYSDEBUG -O0 -fno-inline -fdce
CONFIG_COPTS += -fsanitize=undefined
QUOTA ?= -C64 -L300
endif
# System Five Mode
@@ -136,7 +166,9 @@ endif
# - No Windows bloat!
#
ifeq ($(MODE), sysv)
CONFIG_CCFLAGS += $(BACKTRACES) $(FTRACE) -O2
ENABLE_FTRACE = 1
CONFIG_OFLAGS ?= -g -ggdb
CONFIG_CCFLAGS += $(BACKTRACES) -O2
CONFIG_CPPFLAGS += -DSYSDEBUG -DSUPPORT_VECTOR=121
TARGET_ARCH ?= -msse3
endif
@@ -165,20 +197,56 @@ CONFIG_CCFLAGS += \
-fno-align-labels \
-fno-align-loops \
-fschedule-insns2 \
-fomit-frame-pointer \
-momit-leaf-frame-pointer \
-foptimize-sibling-calls \
-DDWARFLESS
CONFIG_OFLAGS += \
-g0
CONFIG_LDFLAGS += \
-S
TARGET_ARCH ?= \
-msse3
PYFLAGS += \
-O2 \
-B
endif
ifeq ($(MODE), x86_64-tiny)
CONFIG_CPPFLAGS += \
-DTINY \
-DNDEBUG \
-DTRUSTWORTHY
CONFIG_CCFLAGS += \
-Os \
-fno-align-functions \
-fno-align-jumps \
-fno-align-labels \
-fno-align-loops \
-fschedule-insns2 \
-momit-leaf-frame-pointer \
-foptimize-sibling-calls \
-DDWARFLESS
TARGET_ARCH ?= \
-msse3
PYFLAGS += \
-O2 \
-B
endif
ifeq ($(MODE), aarch64-tiny)
# TODO(jart): -mcmodel=tiny
CONFIG_CPPFLAGS += \
-DTINY \
-DNDEBUG \
-DTRUSTWORTHY
CONFIG_CCFLAGS += \
-Os \
-fno-align-functions \
-fno-align-jumps \
-fno-align-labels \
-fno-align-loops \
-fschedule-insns2 \
-momit-leaf-frame-pointer \
-foptimize-sibling-calls \
-DDWARFLESS
PYFLAGS += \
-O2 \
-B
endif
# Linux-Only Tiny Mode
#
@@ -200,12 +268,6 @@ CONFIG_CPPFLAGS += \
-DTRUSTWORTHY \
-DSUPPORT_VECTOR=1 \
-DDWARFLESS
DEFAULT_COPTS += \
-mred-zone
CONFIG_OFLAGS += \
-g0
CONFIG_LDFLAGS += \
-S
CONFIG_CCFLAGS += \
-Os \
-fno-align-functions \
@@ -237,12 +299,6 @@ CONFIG_CPPFLAGS += \
-DTRUSTWORTHY \
-DSUPPORT_VECTOR=113 \
-DDWARFLESS
DEFAULT_COPTS += \
-mred-zone
CONFIG_OFLAGS += \
-g0
CONFIG_LDFLAGS += \
-S
CONFIG_CCFLAGS += \
-Os \
-fno-align-functions \
@@ -273,18 +329,12 @@ CONFIG_CPPFLAGS += \
-DTRUSTWORTHY \
-DSUPPORT_VECTOR=121 \
-DDWARFLESS
DEFAULT_COPTS += \
-mred-zone
CONFIG_CCFLAGS += \
-Os \
-fno-align-functions \
-fno-align-jumps \
-fno-align-labels \
-fno-align-loops
CONFIG_OFLAGS += \
-g0
CONFIG_LDFLAGS += \
-S
TARGET_ARCH ?= \
-msse3
endif
@@ -315,40 +365,44 @@ CONFIG_CCFLAGS += \
-fno-align-jumps \
-fno-align-labels \
-fno-align-loops
CONFIG_OFLAGS += \
-g0
CONFIG_LDFLAGS += \
-S
TARGET_ARCH ?= \
-msse3
endif
# GCC11 Mode
# https://justine.lol/compilers/x86_64-linux-musl__x86_64-linux-musl__g++-11.2.0.tar.xz
ifeq ($(MODE), gcc11)
.UNVEIL += rx:/opt/gcc11
CONFIG_CCFLAGS += $(BACKTRACES) $(FTRACE) -DSYSDEBUG -O2
AS = /opt/gcc11/bin/x86_64-linux-musl-as
CC = /opt/gcc11/bin/x86_64-linux-musl-gcc
CXX = /opt/gcc11/bin/x86_64-linux-musl-g++
CXXFILT = /opt/gcc11/bin/x86_64-linux-musl-c++filt
LD = /opt/gcc11/bin/x86_64-linux-musl-ld.bfd
NM = /opt/gcc11/bin/x86_64-linux-musl-nm
GCC = /opt/gcc11/bin/x86_64-linux-musl-gcc
STRIP = /opt/gcc11/bin/x86_64-linux-musl-strip
OBJCOPY = /opt/gcc11/bin/x86_64-linux-musl-objcopy
OBJDUMP = /opt/gcc11/bin/x86_64-linux-musl-objdump
ADDR2LINE = /opt/gcc11/bin/x86_64-linux-musl-addr2line
CONFIG_CCFLAGS += $(BACKTRACES) $(FTRACE) -O2 -Wno-stringop-overread
CONFIG_CFLAGS += -Wno-old-style-definition
CONFIG_CPPFLAGS += -DNDEBUG -DSYSDEBUG
# no x87 instructions mode
#
# export MODE=nox87
# make -j8 toolchain
# cosmocc -o /tmp/hello.com hello.c
#
# lets you shave ~23kb off blink
#
# git clone https://github.com/jart/blink
# cd blink
# ./configure --disable-x87
# make -j8
# o//blink/blink /tmp/hello.com
#
ifeq ($(MODE), nox87)
ENABLE_FTRACE = 1
CONFIG_COPTS += -mlong-double-64
CONFIG_CCFLAGS += $(BACKTRACES) -O2
CONFIG_CPPFLAGS += -DSYSDEBUG -DNOX87
TARGET_ARCH ?= -msse3
endif
# LLVM Mode
#
# We aim to support:
#
# make -j8 m=llvm o/llvm/libc
#
# The rest of the monorepo may not work with llvm.
#
ifeq ($(MODE), llvm)
.STRICT = 0
TARGET_ARCH ?= -msse3
CONFIG_CCFLAGS += $(BACKTRACES) $(FTRACE) -DSYSDEBUG -O2
CONFIG_CCFLAGS += $(BACKTRACES) -DSYSDEBUG -O2
AS = clang
CC = clang
CXX = clang++
@@ -386,12 +440,65 @@ endif
# such as MSVC or XCode. You can run your binary objects through a tool
# like objconv to convert them to COFF or MachO. Then use ANSI mode to
# rollup one header file that'll enable linkage with minimal issues.
ifeq ($(MODE), ansi)
CONFIG_CFLAGS += -std=c11
#CONFIG_CPPFLAGS += -ansi
CONFIG_CXXFLAGS += -std=c++11
TARGET_ARCH ?= -msse3
endif
ifneq ($(ENABLE_FTRACE),)
CONFIG_CPPFLAGS += -DFTRACE
FTRACE_CCFLAGS = -fno-inline-functions-called-once
OVERRIDE_CFLAGS += $(FTRACE_CCFLAGS)
OVERRIDE_CXXFLAGS += $(FTRACE_CCFLAGS)
# function prologue nops for --ftrace
ifeq ($(ARCH), x86_64)
# this flag causes gcc to generate functions like this
#
# nop nop nop nop nop nop nop nop nop
# func:
# nop nop
# ...
#
# which tool/build/fixupobj.c improves at build time like this
#
# nop nop nop nop nop nop nop nop nop
# func:
# xchg %ax,%ax
# ...
#
# which --ftrace morphs at runtime like this
#
# ud2 # 2 bytes
# call ftrace_hook # 5 bytes
# jmp +2 # 2 bytes
# func:
# jmp -7 # 2 bytes
# ...
#
CONFIG_CCFLAGS += -fpatchable-function-entry=18,16
endif
ifeq ($(ARCH), aarch64)
# this flag causes gcc to generate functions like this
#
# nop nop nop nop nop nop
# func:
# nop
# ...
#
# which --ftrace morphs at runtime like this
#
# brk #31337
# stp x29,x30,[sp,#-16]!
# mov x29,sp
# bl ftrace_hook
# ldp x29,x30,[sp],#16
# b +1
# func:
# b -5
# ...
#
CONFIG_CCFLAGS += -fpatchable-function-entry=7,6
endif
endif
+84 -180
View File
@@ -1,5 +1,5 @@
#-*-mode:makefile-gmake;indent-tabs-mode:t;tab-width:8;coding:utf-8-*-┐
#───vi: set et ft=make ts=8 tw=8 fenc=utf-8 :vi───────────────────────┘
#── vi: set et ft=make ts=8 sw=8 fenc=utf-8 :vi ──────────────────────┘
#
# SYNOPSIS
#
@@ -33,95 +33,43 @@
#
# VARIABLES
#
# CCFLAGS gcc frontend flags (.i, .c, .cc, .f, .S, .lds, etc.)
# Our configuration variables, ordered by increasing preference:
#
# CCFLAGS frontend flags (.i, .c, .cc, .f, .S, .lds, etc.)
# OFLAGS objectify flags (precludes -S and -E)
# CPPFLAGS preprocessor flags (.h, .c, .cc, .S, .inc, .lds, etc.)
# TARGET_ARCH microarchitecture flags (e.g. -march=native)
# COPTS c/c++ flags (.c, .cc)
# CFLAGS c flags (.c only)
# CXXFLAGS c++ flags (.cc only)
# COPTS c/c++ flags (.c, .cc)
# LDFLAGS linker flags (don't use -Wl, frontend prefix)
# ASFLAGS assembler flags (don't use -Wa, frontend prefix)
# TARGET_ARCH microarchitecture flags (e.g. -march=native)
LC_ALL = C
SOURCE_DATE_EPOCH = 0
TAGS ?= /usr/bin/ctags # emacs source builds or something breaks it
ARFLAGS = rcsD
ZFLAGS ?=
XARGS ?= xargs -P4 -rs8000
DOT ?= dot
CLANG = clang
FC = gfortran #/opt/cross9f/bin/x86_64-linux-musl-gfortran
TMPDIR = o/tmp
AR = build/bootstrap/ar.com
CP = build/bootstrap/cp.com
RM = build/bootstrap/rm.com -f
ECHO = build/bootstrap/echo.com
TOUCH = build/bootstrap/touch.com
PKG = build/bootstrap/package.com
MKDEPS = build/bootstrap/mkdeps.com
ZIPOBJ = build/bootstrap/zipobj.com
FIXUPOBJ = build/bootstrap/fixupobj.com
MKDIR = build/bootstrap/mkdir.com -p
COMPILE = build/bootstrap/compile.com -V9 -P4096 $(QUOTA)
COMMA := ,
PWD := $(shell build/bootstrap/pwd.com)
IMAGE_BASE_VIRTUAL ?= 0x400000
IGNORE := $(shell $(ECHO) -2 ♥cosmo)
IGNORE := $(shell $(MKDIR) o/tmp)
ifneq ("$(wildcard o/third_party/gcc/bin/x86_64-pc-linux-gnu-*)","")
PREFIX = o/third_party/gcc/bin/x86_64-pc-linux-gnu-
else
IGNORE := $(shell build/bootstrap/unbundle.com)
PREFIX = o/third_party/gcc/bin/x86_64-linux-musl-
endif
AS = $(PREFIX)as
CC = $(PREFIX)gcc
CXX = $(PREFIX)g++
CXXFILT = $(PREFIX)c++filt
LD = $(PREFIX)ld.bfd
NM = $(PREFIX)nm
GCC = $(PREFIX)gcc
STRIP = $(PREFIX)strip
OBJCOPY = $(PREFIX)objcopy
OBJDUMP = $(PREFIX)objdump
ADDR2LINE = $(PWD)/$(PREFIX)addr2line
export ADDR2LINE
export LC_ALL
export MKDIR
export MODE
export SOURCE_DATE_EPOCH
export TMPDIR
#
# For each FOO above, there exists (by increasing preference)
#
# DEFAULT_FOO see build/definitions.mk
# CONFIG_FOO see build/config.mk
# FOO set ~/.cosmo.mk and target-specific
# OVERRIDE_FOO set ~/.cosmo.mk and target-specific (use rarely)
#
ifeq ($(LANDLOCKMAKE_VERSION),)
TMPSAFE = $(TMPDIR)/$(subst /,_,$@).tmp
TMPSAFE = $(join $(TMPDIR)/,$(subst /,_,$@)).tmp
else
TMPSAFE = $(TMPDIR)/
endif
FTRACE = \
-pg
BACKTRACES = \
-fno-schedule-insns2 \
-fno-omit-frame-pointer \
-fno-optimize-sibling-calls \
-mno-omit-leaf-frame-pointer
SANITIZER = \
-fsanitize=address
NO_MAGIC = \
-mno-fentry \
-ffreestanding \
-fno-stack-protector \
-fwrapv \
-fno-sanitize=all
-fno-sanitize=all \
-fpatchable-function-entry=0,0
OLD_CODE = \
-fno-strict-aliasing \
@@ -132,78 +80,108 @@ TRADITIONAL = \
-Wno-return-type \
-Wno-pointer-sign
DEFAULT_CCFLAGS = \
DEFAULT_CCFLAGS += \
-Wall \
-Werror \
-fdebug-prefix-map='$(PWD)'= \
-fno-omit-frame-pointer \
-frecord-gcc-switches
DEFAULT_OFLAGS = \
-g \
-gdwarf-4 \
-gdescribe-dies
DEFAULT_COPTS = \
-mno-red-zone \
-fno-math-errno \
-fno-trapping-math \
-fno-fp-int-builtin-inexact \
DEFAULT_COPTS ?= \
-fno-ident \
-fno-common \
-fno-gnu-unique \
-fstrict-aliasing \
-fstrict-overflow \
-fno-semantic-interposition \
-fno-dwarf2-cfi-asm \
-fno-unwind-tables \
-fno-asynchronous-unwind-tables
ifeq ($(ARCH), x86_64)
# Microsoft says "[a]ny memory below the stack beyond the red zone
# [note: Windows defines the x64 red zone size as 0] is considered
# volatile and may be modified by the operating system at any time."
# https://devblogs.microsoft.com/oldnewthing/20190111-00/?p=100685
DEFAULT_COPTS += \
-mno-red-zone \
-mno-tls-direct-seg-refs
endif
ifeq ($(ARCH), aarch64)
#
# - Apple says in "Writing ARM64 code for Apple platforms" that we're
# not allowed to use the x18 register.
#
# - Cosmopolitan Libc uses x28 for thread-local storage because Apple
# forbids us from using tpidr_el0 too.
#
# - Cosmopolitan currently lacks an implementation of the runtime
# libraries needed by the -moutline-atomics flag
#
DEFAULT_COPTS += \
-ffixed-x18 \
-ffixed-x28 \
-fsigned-char \
-mno-outline-atomics
endif
MATHEMATICAL = \
-O3 \
-fwrapv
DEFAULT_CPPFLAGS = \
-DCOSMO \
DEFAULT_CPPFLAGS += \
-D_COSMO_SOURCE \
-DMODE='"$(MODE)"' \
-DIMAGE_BASE_VIRTUAL=$(IMAGE_BASE_VIRTUAL) \
-Wno-prio-ctor-dtor \
-Wno-unknown-pragmas \
-nostdinc \
-iquote .
-iquote. \
-isystem libc/isystem
DEFAULT_CFLAGS = \
-std=gnu2x
DEFAULT_CXXFLAGS = \
-std=gnu++20 \
-fno-rtti \
-fno-exceptions \
-fuse-cxa-atexit \
-fno-threadsafe-statics \
-Wno-int-in-bool-context \
-Wno-narrowing \
-Wno-literal-suffix
-Wno-literal-suffix \
-isystem third_party/libcxx
DEFAULT_ASFLAGS = \
-W \
-I. \
--noexecstack \
--nocompress-debug-sections
--noexecstack
DEFAULT_LDFLAGS = \
-static \
-nostdlib \
-melf_x86_64 \
-znorelro \
--gc-sections \
-z noexecstack \
--build-id=none \
--no-dynamic-linker \
-zmax-page-size=0x1000 #--cref [email protected]
--no-dynamic-linker
ZIPOBJ_FLAGS = \
-b$(IMAGE_BASE_VIRTUAL)
# # generate linker report files
# DEFAULT_LDFLAGS += --cref [email protected]
PYFLAGS = \
-b$(IMAGE_BASE_VIRTUAL)
ifeq ($(ARCH), aarch64)
DEFAULT_LDFLAGS += \
-zmax-page-size=0x4000 \
-zcommon-page-size=0x4000 \
-znorelro
else
DEFAULT_LDFLAGS += \
-zmax-page-size=0x4000 \
-zcommon-page-size=0x1000
endif
ASONLYFLAGS = \
-c \
-g \
--debug-prefix-map='$(PWD)'=
-g
DEFAULT_LDLIBS =
@@ -277,40 +255,31 @@ LD.libs = \
$(CONFIG_LIBS) \
$(LIBS)
COMPILE.c.flags = $(cc.flags) $(cpp.flags) $(copt.flags) $(c.flags)
COMPILE.cxx.flags = $(cc.flags) $(cpp.flags) $(copt.flags) $(cxx.flags)
COMPILE.f.flags = $(cc.flags) $(copt.flags) $(f.flags)
COMPILE.F.flags = $(cc.flags) $(cpp.flags) $(copt.flags) $(f.flags)
COMPILE.c.flags = $(cc.flags) $(copt.flags) $(cpp.flags) $(c.flags)
COMPILE.cxx.flags = $(cc.flags) $(copt.flags) $(cxx.flags) $(cpp.flags)
COMPILE.i.flags = $(cc.flags) $(copt.flags) $(c.flags)
COMPILE.ii.flags = $(cc.flags) $(copt.flags) $(cxx.flags)
LINK.flags = $(DEFAULT_LDFLAGS) $(CONFIG_LDFLAGS) $(LDFLAGS)
OBJECTIFY.c.flags = $(OBJECTIFY.S.flags) $(copt.flags) $(c.flags)
OBJECTIFY.cxx.flags = $(OBJECTIFY.S.flags) $(copt.flags) $(cxx.flags)
OBJECTIFY.c.flags = $(cc.flags) $(o.flags) $(S.flags) $(cpp.flags) $(copt.flags) $(c.flags)
OBJECTIFY.cxx.flags = $(cc.flags) $(o.flags) $(S.flags) $(cxx.flags) $(cpp.flags) $(copt.flags)
OBJECTIFY.s.flags = $(ASONLYFLAGS) $(s.flags)
OBJECTIFY.S.flags = $(copt.flags) $(cc.flags) $(o.flags) $(cpp.flags) $(S.flags)
OBJECTIFY.f.flags = $(copt.flags) $(cc.flags) $(o.flags) $(copt.flags) $(S.flags) $(f.flags)
OBJECTIFY.F.flags = $(OBJECTIFY.f.flags) $(cpp.flags)
OBJECTIFY.S.flags = $(cc.flags) $(o.flags) $(S.flags) $(cpp.flags)
PREPROCESS.flags = -E $(copt.flags) $(cc.flags) $(cpp.flags)
PREPROCESS.lds.flags = -D__LINKER__ $(filter-out -g%,$(PREPROCESS.flags)) -P -xc
COMPILE.c = $(CC) -S $(COMPILE.c.flags)
COMPILE.cxx = $(CXX) -S $(COMPILE.cxx.flags)
COMPILE.i = $(CC) -S $(COMPILE.i.flags)
COMPILE.f = $(FC) -S $(COMPILE.f.flags)
COMPILE.F = $(FC) -S $(COMPILE.F.flags)
OBJECTIFY.s = $(AS) $(OBJECTIFY.s.flags)
OBJECTIFY.S = $(CC) $(OBJECTIFY.S.flags) -c
OBJECTIFY.f = $(FC) $(OBJECTIFY.f.flags) -c
OBJECTIFY.F = $(FC) $(OBJECTIFY.F.flags) -c
OBJECTIFY.c = $(CC) $(OBJECTIFY.c.flags) -c
OBJECTIFY.cxx = $(CXX) $(OBJECTIFY.cxx.flags) -c
PREPROCESS = $(CC) $(PREPROCESS.flags)
PREPROCESS.lds = $(CC) $(PREPROCESS.lds.flags)
LINK = $(LD) $(LINK.flags)
ELF = o/libc/elf/elf.lds
ELFLINK = $(COMPILE) -ALINK.elf $(LINK) $(LINKARGS) $(OUTPUT_OPTION)
ELFLINK = $(COMPILE) -ALINK.elf $(LINK) $(LINKARGS) $(OUTPUT_OPTION) && $(COMPILE) -AFIXUP.ape -T$@ $(FIXUPOBJ) $@
LINKARGS = $(patsubst %.lds,-T %.lds,$(call uniqr,$(LD.libs) $(filter-out %.pkg,$^)))
LOLSAN = build/lolsan -b $(IMAGE_BASE_VIRTUAL)
# The compiler won't generate %xmm code for sources extensioned .greg.c,
# which is needed for C modules wanting to run at the executive level or
@@ -325,74 +294,9 @@ OBJECTIFY.greg.c = \
-fno-optimize-sibling-calls \
-fno-sanitize=all \
-ffreestanding \
-mno-fentry \
-fwrapv \
-c
OBJECTIFY.ansi.c = $(CC) $(OBJECTIFY.c.flags) -ansi -Wextra -Werror -pedantic-errors -c
OBJECTIFY.c99.c = $(CC) $(OBJECTIFY.c.flags) -std=c99 -Wextra -Werror -pedantic-errors -c
OBJECTIFY.c11.c = $(CC) $(OBJECTIFY.c.flags) -std=c11 -Wextra -Werror -pedantic-errors -c
OBJECTIFY.c2x.c = $(CC) $(OBJECTIFY.c.flags) -std=c2x -Wextra -Werror -pedantic-errors -c
OBJECTIFY.real.c = \
$(GCC) \
-x-no-pg \
$(OBJECTIFY.c.flags) \
-wrapper build/realify.sh \
-D__REAL_MODE__ \
-ffixed-r8 \
-ffixed-r9 \
-ffixed-r10 \
-ffixed-r11 \
-ffixed-r12 \
-ffixed-r13 \
-ffixed-r14 \
-ffixed-r15 \
-mno-red-zone \
-fcall-used-rbx \
-fno-jump-tables \
-fno-shrink-wrap \
-fno-schedule-insns2 \
-flive-range-shrinkage \
-fno-omit-frame-pointer \
-momit-leaf-frame-pointer \
-mpreferred-stack-boundary=3 \
-fno-delete-null-pointer-checks \
-c
OBJECTIFY.ncabi.c = \
$(GCC) \
$(OBJECTIFY.c.flags) \
-mno-sse \
-mfpmath=387 \
-mno-fentry \
-fno-stack-protector \
-fno-instrument-functions \
-fno-optimize-sibling-calls \
-fno-sanitize=all \
-fcall-saved-rcx \
-fcall-saved-rdx \
-fcall-saved-rdi \
-fcall-saved-rsi \
-fcall-saved-r8 \
-fcall-saved-r9 \
-fcall-saved-r10 \
-fcall-saved-r11 \
-c \
-xc
OBJECTIFY.initabi.c = \
$(GCC) \
$(OBJECTIFY.c.flags) \
-mno-fentry \
-fno-stack-protector \
-fno-instrument-functions \
-fno-optimize-sibling-calls \
-fno-sanitize=all \
-fcall-saved-rdi \
-fcall-saved-rsi \
-c
TAGSFLAGS = \
-e \
-a \
+101
View File
@@ -0,0 +1,101 @@
#!/bin/sh
# cosmocc downloader script
# https://justine.lol/cosmo3/#install
# https://github.com/jart/cosmopolitan/blob/master/tool/cosmocc/README.md
# collect arguments
OUTPUT_DIR=${1:?OUTPUT_DIR}
COSMOCC_VERSION=${2:?COSMOCC_VERSION}
COSMOCC_SHA256SUM=${3:?COSMOCC_SHA256SUM}
URL1="https://github.com/jart/cosmopolitan/releases/download/${COSMOCC_VERSION}/cosmocc-${COSMOCC_VERSION}.zip"
URL2="https://cosmo.zip/pub/cosmocc/cosmocc-${COSMOCC_VERSION}.zip"
# helper function
abort() {
printf '%s\n' "download terminated." >&2
exit 1
}
# exit if already downloaded
# we need it because directory timestamps work wierdly
OUTPUT_DIR=${OUTPUT_DIR%/}
if [ -d "${OUTPUT_DIR}" ]; then
exit 0
fi
# find commands we need to securely download cosmocc
if ! UNZIP=$(command -v unzip 2>/dev/null); then
printf '%s\n' "$0: fatal error: you need the unzip command" >&2
printf '%s\n' "please download https://cosmo.zip/pub/cosmos/bin/unzip and put it on the system path" >&2
abort
fi
if command -v sha256sum >/dev/null 2>&1; then
# can use system sha256sum
true
elif command -v shasum >/dev/null 2>&1; then
sha256sum() {
shasum -a 256 "$@"
}
else
if [ ! -f build/sha256sum.c ]; then
printf '%s\n' "$0: fatal error: you need to install sha256sum" >&2
printf '%s\n' "please download https://cosmo.zip/pub/cosmos/bin/sha256sum and put it on the system path" >&2
abort
fi
if ! SHA256SUM=$(command -v "$PWD/o/build/sha256sum" 2>/dev/null); then
if ! CC=$(command -v "$CC" 2>/dev/null); then
if ! CC=$(command -v cc 2>/dev/null); then
if ! CC=$(command -v cosmocc 2>/dev/null); then
printf '%s\n' "$0: fatal error: you need to install either sha256sum, cc, or cosmocc" >&2
printf '%s\n' "please download https://cosmo.zip/pub/cosmos/bin/sha256sum and put it on the system path" >&2
abort
fi
fi
fi
mkdir -p o/build || abort
SHA256SUM="$PWD/o/build/sha256sum"
printf '%s\n' "${CC} -w -O2 -o ${SHA256SUM} build/sha256sum.c" >&2
"${CC}" -w -O2 -o "${SHA256SUM}.$$" build/sha256sum.c || abort
mv -f "${SHA256SUM}.$$" "${SHA256SUM}" || abort
fi
sha256sum() {
"${SHA256SUM}" "$@"
}
fi
if WGET=$(command -v wget 2>/dev/null); then
DOWNLOAD=$WGET
DOWNLOAD_ARGS=-O
elif CURL=$(command -v curl 2>/dev/null); then
DOWNLOAD=$CURL
DOWNLOAD_ARGS=-fLo
else
printf '%s\n' "$0: fatal error: you need to install either wget or curl" >&2
printf '%s\n' "please download https://cosmo.zip/pub/cosmos/bin/wget and put it on the system path" >&2
abort
fi
# create temporary output directory
OLDPWD=$PWD
OUTPUT_TMP="${OUTPUT_DIR}.tmp.$$/"
mkdir -p "${OUTPUT_TMP}" || abort
cd "${OUTPUT_TMP}"
die() {
cd "${OLDPWD}"
rm -rf "${OUTPUT_TMP}"
abort
}
# download cosmocc toolchain
# multiple urls avoids outages and national firewalls
if ! "${DOWNLOAD}" ${DOWNLOAD_ARGS} cosmocc.zip "${URL1}"; then
rm -f cosmocc.zip
"${DOWNLOAD}" ${DOWNLOAD_ARGS} cosmocc.zip "${URL2}" || die
fi
printf '%s\n' "${COSMOCC_SHA256SUM} *cosmocc.zip" >cosmocc.zip.sha256sum
sha256sum -c cosmocc.zip.sha256sum || die
"${UNZIP}" cosmocc.zip || die
rm -f cosmocc.zip cosmocc.zip.sha256sum
# commit output directory
cd "${OLDPWD}" || die
mv "${OUTPUT_TMP}" "${OUTPUT_DIR}" || die
+1 -1
View File
@@ -1,5 +1,5 @@
#-*-mode:makefile-gmake;indent-tabs-mode:t;tab-width:8;coding:utf-8-*-┐
#───vi: set et ft=make ts=8 tw=8 fenc=utf-8 :vi───────────────────────┘
#── vi: set et ft=make ts=8 sw=8 fenc=utf-8 :vi ──────────────────────┘
#
# SYNOPSIS
#
+6 -3
View File
@@ -1,6 +1,6 @@
#!/bin/sh
#-*-mode:sh;indent-tabs-mode:nil;tab-width:2;coding:utf-8-*-┐
#───vi: set net ft=sh ts=2 sts=2 fenc=utf-8 :vi─────────────┘
#── vi: set et ft=sh ts=2 sts=2 fenc=utf-8 :vi ─────────────┘
#
# OVERVIEW
#
@@ -43,6 +43,9 @@
# '(progn
# (add-hook 'c-mode-common-hook 'jart-c-mode-common-hook)))
TAGS="$1"
shift
# ctags doesn't understand atomics, e.g.
# extern char **environ;
set -- --regex-c='/_Atomic(\([^)]*\))/\1/b' "$@"
@@ -52,7 +55,7 @@ set -- --regex-c='/_Atomic(\([^)]*\))/\1/b' "$@"
set -- --regex-c='/^\(\(hidden\|extern\|const\) \)*[_[:alpha:]][_[:alnum:]]*[ *][ *]*\([_[:alpha:]][_[:alnum:]]*[ *][ *]*\)*\([_[:alpha:]][_$[:alnum:]]*\)/\4/b' "$@"
# ctags doesn't understand function prototypes, e.g.
# bool isheap(void *p) dontthrow nocallback;
# bool isheap(void *p) dontthrow dontcallback;
set -- --regex-c='/^[_[:alpha:]][_[:alnum:]]*[ *][ *]*\([_[:alpha:]][_[:alnum:]]*[ *][ *]*\)*\([_[:alpha:]][_$[:alnum:]]*\)(.*/\2/b' "$@"
# ctags doesn't understand function pointers, e.g.
@@ -63,7 +66,7 @@ set -- --regex-c='/^extern [^(]*(\*const \([^)]*\))(/\1/b' "$@"
# struct WorstSoftwareEver;
set -- --regex-c='/^struct.*;$/uehocruehcroue/b' "$@"
exec ${TAGS:-ctags} \
build/run $TAGS \
-e \
--langmap=c:.c.h \
--exclude=libc/nt/struct/imagefileheader.internal.h \
Executable
+29
View File
@@ -0,0 +1,29 @@
#!/bin/sh
COSMO=${COSMO:-/opt/cosmo}
if [ -n "$OBJDUMP" ]; then
exec "$OBJDUMP" "$@"
fi
find_objdump() {
if [ -x .cosmocc/3.3.5/bin/$1-linux-cosmo-objdump ]; then
OBJDUMP=.cosmocc/3.3.5/bin/$1-linux-cosmo-objdump
elif [ -x .cosmocc/3.3.5/bin/$1-linux-musl-objdump ]; then
OBJDUMP=.cosmocc/3.3.5/bin/$1-linux-musl-objdump
elif [ -x "$COSMO/.cosmocc/3.3.5/bin/$1-linux-cosmo-objdump" ]; then
OBJDUMP="$COSMO/.cosmocc/3.3.5/bin/$1-linux-cosmo-objdump"
elif [ -x "$COSMO/.cosmocc/3.3.5/bin/$1-linux-musl-objdump" ]; then
OBJDUMP="$COSMO/.cosmocc/3.3.5/bin/$1-linux-musl-objdump"
else
echo "error: toolchain not found (try running 'cosmocc --update' or 'make' in the cosmo monorepo)" >&2
exit 1
fi
}
if printf '%s\n' "$*" | grep aarch64 >/dev/null 2>&1; then
find_objdump aarch64
exec "$OBJDUMP" "$@"
else
find_objdump x86_64
exec "$OBJDUMP" "$@"
fi
+11 -7
View File
@@ -1,5 +1,5 @@
#-*-mode:makefile-gmake;indent-tabs-mode:t;tab-width:8;coding:utf-8-*-┐
#───vi: set et ft=make ts=8 tw=8 fenc=utf-8 :vi───────────────────────┘
#── vi: set et ft=make ts=8 sw=8 fenc=utf-8 :vi ──────────────────────┘
#
# SYNOPSIS
#
@@ -22,10 +22,14 @@
# - tool/build/runit.c
# - tool/build/runitd.c
.PRECIOUS: o/$(MODE)/%.com.ok
o/$(MODE)/%.com.ok: .PLEDGE = stdio rpath wpath cpath proc fattr inet
o/$(MODE)/%.com.ok: \
o/$(MODE)/tool/build/runit.com \
o/$(MODE)/tool/build/runitd.com \
o/$(MODE)/%.com
.PRECIOUS: o/$(MODE)/%.ok
o/$(MODE)/%.ok: private .PLEDGE = stdio rpath wpath cpath proc fattr inet dns
o/$(MODE)/%.ok: private .UNVEIL += r:/etc/resolv.conf
o/$(MODE)/%.ok: \
o/$(MODE)/tool/build/runit \
o/$(MODE)/tool/build/runitd \
o/$(MODE)/%
@$(COMPILE) -wATEST -tT$@ $^ $(HOSTS)
.PHONY:
o/tiny/tool/build/runit:
+1 -1
View File
@@ -1,5 +1,5 @@
#-*-mode:sed;indent-tabs-mode:t;tab-width:8;coding:utf-8-*-┐
#───vi: et ft=sed ts=8 tw=8 fenc=utf-8 :vi─────────────────┘
#── vi: et ft=sed ts=8 sw=8 fenc=utf-8 :vi ────────────────┘
#
# SYNOPSIS
#
+54 -74
View File
@@ -1,5 +1,5 @@
#-*-mode:makefile-gmake;indent-tabs-mode:t;tab-width:8;coding:utf-8-*-┐
#───vi: set et ft=make ts=8 tw=8 fenc=utf-8 :vi───────────────────────┘
#── vi: set et ft=make ts=8 sw=8 fenc=utf-8 :vi ──────────────────────┘
#
# SYNOPSIS
#
@@ -16,40 +16,15 @@
MAKEFLAGS += --no-builtin-rules
o/%.o: %.s ; @$(COMPILE) -AOBJECTIFY.s $(OBJECTIFY.s) $(OUTPUT_OPTION) $<
o/%.o: o/%.s ; @$(COMPILE) -AOBJECTIFY.s $(OBJECTIFY.s) $(OUTPUT_OPTION) $<
o/%.s: %.S ; @$(COMPILE) -APREPROCESS $(PREPROCESS) $(OUTPUT_OPTION) $<
o/%.s: o/%.S ; @$(COMPILE) -APREPROCESS $(PREPROCESS) $(OUTPUT_OPTION) $<
o/%.o: %.S ; @$(COMPILE) -AOBJECTIFY.S $(OBJECTIFY.S) $(OUTPUT_OPTION) $<
o/%.o: o/%.S ; @$(COMPILE) -AOBJECTIFY.S $(OBJECTIFY.S) $(OUTPUT_OPTION) $<
o/%.lds: %.lds ; @$(COMPILE) -APREPROCESS $(PREPROCESS.lds) $(OUTPUT_OPTION) $<
o/%.inc: %.h ; @$(COMPILE) -APREPROCESS $(PREPROCESS) $(OUTPUT_OPTION) -D__ASSEMBLER__ -P $<
o/%.greg.o: %.greg.c ; @$(COMPILE) -AOBJECTIFY.greg $(OBJECTIFY.greg.c) $(OUTPUT_OPTION) $<
MAKE_ZIPCOPY = $(COMPILE) -AZIPCOPY -wT$@ $(ZIPCOPY) $< $@
ifneq ($(ARCH), aarch64)
MAKE_OBJCOPY = $(COMPILE) -AOBJCOPY -T$@ $(OBJCOPY) -S -O binary $< $@ && $(MAKE_ZIPCOPY)
else
MAKE_OBJCOPY = $(COMPILE) -AOBJCOPY -T$@ $(OBJCOPY) -S $< $@ && $(MAKE_ZIPCOPY)
endif
o/$(MODE)/%: o/$(MODE)/%.dbg ; @$(COMPILE) -AOBJCOPY -T$@ $(OBJCOPY) -S -O binary $< $@
o/$(MODE)/%.o: %.s ; @$(COMPILE) -AOBJECTIFY.s $(OBJECTIFY.s) $(OUTPUT_OPTION) $<
o/$(MODE)/%.o: o/$(MODE)/%.s ; @$(COMPILE) -AOBJECTIFY.s $(OBJECTIFY.s) $(OUTPUT_OPTION) $<
o/$(MODE)/%.o: %.f ; @$(COMPILE) -AOBJECTIFY.f $(OBJECTIFY.f) $(OUTPUT_OPTION) $<
o/$(MODE)/%.o: %.F ; @$(COMPILE) -AOBJECTIFY.F $(OBJECTIFY.F) $(OUTPUT_OPTION) $<
o/$(MODE)/%.h: %.c ; @$(COMPILE) -AAMALGAMATE $(PREPROCESS) $(OUTPUT_OPTION) -fdirectives-only -P $<
o/$(MODE)/%.h: o/$(MODE)/%.c ; @$(COMPILE) -AAMALGAMATE $(PREPROCESS) $(OUTPUT_OPTION) -fdirectives-only -P $<
o/$(MODE)/%.o: %.S ; @$(COMPILE) -AOBJECTIFY.S $(OBJECTIFY.S) $(OUTPUT_OPTION) $<
o/$(MODE)/%.o: o/$(MODE)/%.S ; @$(COMPILE) -AOBJECTIFY.S $(OBJECTIFY.S) $(OUTPUT_OPTION) $<
o/$(MODE)/%.o: %.cc ; @$(COMPILE) -AOBJECTIFY.cxx $(OBJECTIFY.cxx) $(OUTPUT_OPTION) $<
o/$(MODE)/%.lds: %.lds ; @$(COMPILE) -APREPROCESS $(PREPROCESS.lds) $(OUTPUT_OPTION) $<
o/$(MODE)/%.o: %.greg.c ; @$(COMPILE) -AOBJECTIFY.greg $(OBJECTIFY.greg.c) $(OUTPUT_OPTION) $<
o/$(MODE)/%.greg.o: %.greg.c ; @$(COMPILE) -AOBJECTIFY.greg $(OBJECTIFY.greg.c) $(OUTPUT_OPTION) $<
o/$(MODE)/%.initabi.o: %.initabi.c ; @$(COMPILE) -AOBJECTIFY.init $(OBJECTIFY.initabi.c) $(OUTPUT_OPTION) $<
o/$(MODE)/%.ncabi.o: %.ncabi.c ; @$(COMPILE) -AOBJECTIFY.nc $(OBJECTIFY.ncabi.c) $(OUTPUT_OPTION) $<
o/$(MODE)/%.real.o: %.c ; @$(COMPILE) -AOBJECTIFY.real $(OBJECTIFY.real.c) $(OUTPUT_OPTION) $<
o/%.o: %.cc
@$(COMPILE) -AOBJECTIFY.cxx $(OBJECTIFY.cxx) $(OUTPUT_OPTION) $<
@$(COMPILE) -AFIXUPOBJ -wT$@ $(FIXUPOBJ) $@
o/%.o: o/%.cc
@$(COMPILE) -AOBJECTIFY.cxx $(OBJECTIFY.cxx) $(OUTPUT_OPTION) $<
@$(COMPILE) -AFIXUPOBJ -wT$@ $(FIXUPOBJ) $@
o/$(MODE)/%: o/$(MODE)/%.dbg
@$(MAKE_OBJCOPY)
o/$(MODE)/%.o: %.c
@$(COMPILE) -AOBJECTIFY.c $(OBJECTIFY.c) $(OUTPUT_OPTION) $<
@@ -59,10 +34,39 @@ o/$(MODE)/%.o: o/$(MODE)/%.c
@$(COMPILE) -AOBJECTIFY.c $(OBJECTIFY.c) $(OUTPUT_OPTION) $<
@$(COMPILE) -AFIXUPOBJ -wT$@ $(FIXUPOBJ) $@
o/$(MODE)/%.o: o/$(MODE)/%.cc
o/$(MODE)/%.h: %.c
@$(COMPILE) -AAMALGAMATE $(PREPROCESS) $(OUTPUT_OPTION) -fdirectives-only -P $<
o/$(MODE)/%.o: %.cc
@$(COMPILE) -AOBJECTIFY.cxx $(OBJECTIFY.cxx) $(OUTPUT_OPTION) $<
@$(COMPILE) -AFIXUPOBJ -wT$@ $(FIXUPOBJ) $@
o/$(MODE)/%.o: %.cpp
@$(COMPILE) -AOBJECTIFY.cxx $(OBJECTIFY.cxx) $(OUTPUT_OPTION) $<
o/$(MODE)/%.lds: %.lds
@$(COMPILE) -APREPROCESS $(PREPROCESS.lds) $(OUTPUT_OPTION) $<
ifneq ($(ARCH), aarch64)
o/$(MODE)/%.o: %.S
@$(COMPILE) -AOBJECTIFY.S $(OBJECTIFY.S) $(OUTPUT_OPTION) $<
o/$(MODE)/%.o: o/$(MODE)/%.S
@$(COMPILE) -AOBJECTIFY.S $(OBJECTIFY.S) $(OUTPUT_OPTION) $<
else
o/$(MODE)/%.o: %.S libc/empty.s
@$(COMPILE) -AOBJECTIFY.S $(OBJECTIFY.s) $(OUTPUT_OPTION) libc/empty.s
o/$(MODE)/%.o: o/$(MODE)/%.S libc/empty.s
@$(COMPILE) -AOBJECTIFY.S $(OBJECTIFY.s) $(OUTPUT_OPTION) libc/empty.s
endif
o/$(MODE)/%.o: %.greg.c
@$(COMPILE) -AOBJECTIFY.greg $(OBJECTIFY.greg.c) $(OUTPUT_OPTION) $<
@$(COMPILE) -AFIXUPOBJ -wT$@ $(FIXUPOBJ) $@
o/$(MODE)/%.greg.o: %.greg.c
@$(COMPILE) -AOBJECTIFY.greg $(OBJECTIFY.greg.c) $(OUTPUT_OPTION) $<
@$(COMPILE) -AFIXUPOBJ -wT$@ $(FIXUPOBJ) $@
o/%.a:
$(file >$(TMPDIR)/$(subst /,_,$@),$^)
@$(COMPILE) -AARCHIVE -wT$@ $(AR) $(ARFLAGS) $@ @$(TMPDIR)/$(subst /,_,$@)
@@ -75,21 +79,17 @@ o/$(MODE)/%.pkg:
$(file >$(TMPSAFE).args,$(filter %.o,$^))
@$(COMPILE) -APACKAGE -wT$@ $(PKG) $(OUTPUT_OPTION) $(addprefix -d,$(filter %.pkg,$^)) @$(TMPSAFE).args
o/$(MODE)/%.o: %.py o/$(MODE)/third_party/python/pyobj.com
@$(COMPILE) -wAPYOBJ o/$(MODE)/third_party/python/pyobj.com $(PYFLAGS) -o $@ $<
o/$(MODE)/%.o: %.py o/$(MODE)/third_party/python/pyobj
@$(COMPILE) -wAPYOBJ o/$(MODE)/third_party/python/pyobj $(PYFLAGS) -o $@ $<
o/$(MODE)/%.pyc: %.py o/$(MODE)/third_party/python/pycomp.com
@$(COMPILE) -wAPYCOMP o/$(MODE)/third_party/python/pycomp.com $(PYCFLAGS) -o $@ $<
o/$(MODE)/%.pyc: %.py o/$(MODE)/third_party/python/pycomp
@$(COMPILE) -wAPYCOMP o/$(MODE)/third_party/python/pycomp $(PYCFLAGS) -o $@ $<
o/$(MODE)/%.lua: %.lua o/$(MODE)/third_party/lua/luac.com
@$(COMPILE) -wALUAC o/$(MODE)/third_party/lua/luac.com -s -o $@ $<
o/$(MODE)/%.lua: %.lua o/$(MODE)/third_party/lua/luac
@$(COMPILE) -wALUAC o/$(MODE)/third_party/lua/luac -s -o $@ $<
o/$(MODE)/%.lua.runs: %.lua o/$(MODE)/tool/net/redbean.com
@$(COMPILE) -wALUA -tT$@ o/$(MODE)/tool/net/redbean.com $(LUAFLAGS) -i $<
o/$(MODE)/%: o/$(MODE)/%.com o/$(MODE)/tool/build/cp.com o/$(MODE)/tool/build/assimilate.com
@$(COMPILE) -wACP -T$@ o/$(MODE)/tool/build/cp.com $< $@
@$(COMPILE) -wAASSIMILATE -T$@ o/$(MODE)/tool/build/assimilate.com $@
o/$(MODE)/%.lua.runs: %.lua o/$(MODE)/tool/net/redbean
@$(COMPILE) -wALUA -tT$@ o/$(MODE)/tool/net/redbean $(LUAFLAGS) -i $<
################################################################################
# LOCAL UNIT TESTS
@@ -108,22 +108,22 @@ o/$(MODE)/%: o/$(MODE)/%.com o/$(MODE)/tool/build/cp.com o/$(MODE)/tool/build/as
#
# You could then run a command like:
#
# make -j8 o//test/libc/calls/openbsd_test.com.runs
# make -j8 o//test/libc/calls/openbsd_test.runs
#
# You need PROT_EXEC permission since ftrace morphs the binary. It's
# also worth mentioning that the pledge.com command can simulate what
# also worth mentioning that the pledge command can simulate what
# Landlock Make does:
#
# o//tool/build/pledge.com \
# o//tool/build/pledge \
# -v. -p 'stdio rpath wpath cpath tty prot_exec' \
# o//test/libc/calls/openbsd_test.com \
# o//test/libc/calls/openbsd_test \
# ----ftrace
#
# This is useful in the event a test binary should run by itself, but
# fails to run beneath Landlock Make. It's also useful sometimes to
# override the verbosity when running tests:
#
# make V=5 TESTARGS=-b o//test/libc/calls/openbsd_test.com.runs
# make V=5 TESTARGS=-b o//test/libc/calls/openbsd_test.runs
#
# This way, if for some reason a test should fail but calls exit(0),
# then the stdout/stderr output, which would normally be suppressed,
@@ -135,7 +135,7 @@ o/$(MODE)/%.runs: o/$(MODE)/%
################################################################################
# ELF ZIP FILES
#
# zipobj.com lets us do fast incremental linking of compressed data.
# zipobj lets us do fast incremental linking of compressed data.
# it's nice because if we link a hundred binaries that use the time zone
# database, then that database only needs to be DEFLATE'd once.
@@ -183,27 +183,6 @@ o/$(MODE)/%.okk: private .UNSANDBOXED = 1
o/$(MODE)/%.okk: %
@$(COMPILE) -ACHECK.h $(COMPILE.cxx) -xc++ -g0 -o $@ $<
################################################################################
# EXECUTABLE HELPERS
MAKE_OBJCOPY = \
$(COMPILE) -AOBJCOPY -T$@ \
$(OBJCOPY) -S -O binary $< $@
MAKE_SYMTAB_CREATE = \
$(COMPILE) -wASYMTAB \
o/$(MODE)/tool/build/symtab.com \
-o $(TMPSAFE)/.symtab \
$<
MAKE_SYMTAB_ZIP = \
$(COMPILE) -AZIP -T$@ \
o/$(MODE)/third_party/zip/zip.com \
-b$(TMPDIR) \
-9qj \
$@ \
$(TMPSAFE)/.symtab
################################################################################
# EMACS ASSEMBLY GENERATION
@@ -219,6 +198,7 @@ o/$(MODE)/%-clang.asm: private .UNSANDBOXED = 1
o/$(MODE)/%-clang.asm: %.c
@$(COMPILE) -AOBJECTIFY.c $(OBJECTIFY.c) -S -g0 $(OUTPUT_OPTION) $<
# TODO(jart): Make intrinsics support Clang.
o/$(MODE)/%-clang.asm: CC = $(CLANG)
o/$(MODE)/%-clang.asm: private .UNSANDBOXED = 1
o/$(MODE)/%-clang.asm: %.cc
Executable
+8
View File
@@ -0,0 +1,8 @@
#!/bin/sh
UNAMEM=$(uname -m)
UNAMES=$(uname -s)
if [ x"$UNAMES" = x"Darwin" ] && [ x"$UNAMEM" = x"arm64" ]; then
exec ape "$@"
else
exec "$@"
fi
+455
View File
@@ -0,0 +1,455 @@
/*-*- mode:c;indent-tabs-mode:nil;c-basic-offset:2;tab-width:8;coding:utf-8 -*-│
│ vi: set et ft=c ts=2 sts=2 sw=2 fenc=utf-8 :vi │
╞══════════════════════════════════════════════════════════════════════════════╡
│ Copyright 2022 Justine Alexandra Roberts Tunney │
│ │
│ Permission to use, copy, modify, and/or distribute this software for │
│ any purpose with or without fee is hereby granted, provided that the │
│ above copyright notice and this permission notice appear in all copies. │
│ │
│ THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL │
│ WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED │
│ WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE │
│ AUTHOR BE LIABLE FOR ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL │
│ DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR │
│ PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER │
│ TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR │
│ PERFORMANCE OF THIS SOFTWARE. │
╚─────────────────────────────────────────────────────────────────────────────*/
#include <errno.h>
#include <limits.h>
#include <stdarg.h>
#include <stdbool.h>
#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
// this file should not have dependencies, because everything will be
// re-downloaded if the o/tool/sha256sum artifact becomes invalidated
#define PROG "sha256sum"
#define USAGE \
"\
Usage: " PROG " [-?hbctw] [PATH...]\n\
-h help\n\
-c check mode\n\
-b binary mode\n\
-t textual mode\n\
-w warning mode\n"
#define ROTR(a, b) (((a) >> (b)) | ((a) << (32 - (b))))
#define CH(x, y, z) (((x) & (y)) ^ (~(x) & (z)))
#define MAJ(x, y, z) (((x) & (y)) ^ ((x) & (z)) ^ ((y) & (z)))
#define EP0(x) (ROTR(x, 2) ^ ROTR(x, 13) ^ ROTR(x, 22))
#define EP1(x) (ROTR(x, 6) ^ ROTR(x, 11) ^ ROTR(x, 25))
#define SIG0(x) (ROTR(x, 7) ^ ROTR(x, 18) ^ ((x) >> 3))
#define SIG1(x) (ROTR(x, 17) ^ ROTR(x, 19) ^ ((x) >> 10))
struct Sha256Ctx {
uint8_t data[64];
uint32_t datalen;
uint64_t bitlen;
uint32_t state[8];
};
static const uint32_t kSha256Tab[64] = {
0x428a2f98, 0x71374491, 0xb5c0fbcf, 0xe9b5dba5, //
0x3956c25b, 0x59f111f1, 0x923f82a4, 0xab1c5ed5, //
0xd807aa98, 0x12835b01, 0x243185be, 0x550c7dc3, //
0x72be5d74, 0x80deb1fe, 0x9bdc06a7, 0xc19bf174, //
0xe49b69c1, 0xefbe4786, 0x0fc19dc6, 0x240ca1cc, //
0x2de92c6f, 0x4a7484aa, 0x5cb0a9dc, 0x76f988da, //
0x983e5152, 0xa831c66d, 0xb00327c8, 0xbf597fc7, //
0xc6e00bf3, 0xd5a79147, 0x06ca6351, 0x14292967, //
0x27b70a85, 0x2e1b2138, 0x4d2c6dfc, 0x53380d13, //
0x650a7354, 0x766a0abb, 0x81c2c92e, 0x92722c85, //
0xa2bfe8a1, 0xa81a664b, 0xc24b8b70, 0xc76c51a3, //
0xd192e819, 0xd6990624, 0xf40e3585, 0x106aa070, //
0x19a4c116, 0x1e376c08, 0x2748774c, 0x34b0bcb5, //
0x391c0cb3, 0x4ed8aa4a, 0x5b9cca4f, 0x682e6ff3, //
0x748f82ee, 0x78a5636f, 0x84c87814, 0x8cc70208, //
0x90befffa, 0xa4506ceb, 0xbef9a3f7, 0xc67178f2, //
};
static bool g_warn;
static char g_mode;
static bool g_check;
static int g_mismatches;
static void Sha256Transform(uint32_t state[8], const uint8_t data[64]) {
unsigned i;
uint32_t a, b, c, d, e, f, g, h, t1, t2, m[64];
for (i = 0; i < 16; ++i, data += 4) {
m[i] = (uint32_t)data[0] << 24 | data[1] << 16 | data[2] << 8 | data[3];
}
for (; i < 64; ++i) {
m[i] = SIG1(m[i - 2]) + m[i - 7] + SIG0(m[i - 15]) + m[i - 16];
}
a = state[0];
b = state[1];
c = state[2];
d = state[3];
e = state[4];
f = state[5];
g = state[6];
h = state[7];
for (i = 0; i < 64; ++i) {
t1 = h + EP1(e) + CH(e, f, g) + kSha256Tab[i] + m[i];
t2 = EP0(a) + MAJ(a, b, c);
h = g;
g = f;
f = e;
e = d + t1;
d = c;
c = b;
b = a;
a = t1 + t2;
}
state[0] += a;
state[1] += b;
state[2] += c;
state[3] += d;
state[4] += e;
state[5] += f;
state[6] += g;
state[7] += h;
}
static void Sha256Init(struct Sha256Ctx *ctx) {
ctx->datalen = 0;
ctx->bitlen = 0;
ctx->state[0] = 0x6a09e667;
ctx->state[1] = 0xbb67ae85;
ctx->state[2] = 0x3c6ef372;
ctx->state[3] = 0xa54ff53a;
ctx->state[4] = 0x510e527f;
ctx->state[5] = 0x9b05688c;
ctx->state[6] = 0x1f83d9ab;
ctx->state[7] = 0x5be0cd19;
}
static void Sha256Update(struct Sha256Ctx *ctx, const uint8_t *data,
long size) {
long i;
for (i = 0; i < size; ++i) {
ctx->data[ctx->datalen] = data[i];
ctx->datalen++;
if (ctx->datalen == 64) {
Sha256Transform(ctx->state, ctx->data);
ctx->bitlen += 512;
ctx->datalen = 0;
}
}
}
static void Sha256Final(struct Sha256Ctx *ctx, uint8_t *hash) {
long i;
i = ctx->datalen;
ctx->data[i++] = 0x80;
if (ctx->datalen < 56) {
memset(ctx->data + i, 0, 56 - i);
} else {
memset(ctx->data + i, 0, 64 - i);
Sha256Transform(ctx->state, ctx->data);
memset(ctx->data, 0, 56);
}
ctx->bitlen += ctx->datalen * 8;
ctx->data[63] = ctx->bitlen;
ctx->data[62] = ctx->bitlen >> 8;
ctx->data[61] = ctx->bitlen >> 16;
ctx->data[60] = ctx->bitlen >> 24;
ctx->data[59] = ctx->bitlen >> 32;
ctx->data[58] = ctx->bitlen >> 40;
ctx->data[57] = ctx->bitlen >> 48;
ctx->data[56] = ctx->bitlen >> 56;
Sha256Transform(ctx->state, ctx->data);
for (i = 0; i < 4; ++i) {
hash[i] = (ctx->state[0] >> (24 - i * 8)) & 0xff;
hash[i + 4] = (ctx->state[1] >> (24 - i * 8)) & 0xff;
hash[i + 8] = (ctx->state[2] >> (24 - i * 8)) & 0xff;
hash[i + 12] = (ctx->state[3] >> (24 - i * 8)) & 0xff;
hash[i + 16] = (ctx->state[4] >> (24 - i * 8)) & 0xff;
hash[i + 20] = (ctx->state[5] >> (24 - i * 8)) & 0xff;
hash[i + 24] = (ctx->state[6] >> (24 - i * 8)) & 0xff;
hash[i + 28] = (ctx->state[7] >> (24 - i * 8)) & 0xff;
}
}
static char *FormatUint32(char *p, uint32_t x) {
char t;
size_t i, a, b;
i = 0;
do {
p[i++] = x % 10 + '0';
x = x / 10;
} while (x > 0);
p[i] = '\0';
if (i) {
for (a = 0, b = i - 1; a < b; ++a, --b) {
t = p[a];
p[a] = p[b];
p[b] = t;
}
}
return p + i;
}
static char *FormatInt32(char *p, int32_t x) {
if (x < 0)
*p++ = '-', x = -(uint32_t)x;
return FormatUint32(p, x);
}
static size_t StrCat(char *dst, const char *src, size_t dsize) {
size_t m, n = dsize;
const char *p = dst;
const char *q = src;
while (n-- != 0 && *dst != '\0')
dst++;
m = dst - p;
n = dsize - m;
if (n-- == 0) {
return m + strlen(src);
}
while (*src != '\0') {
if (n != 0) {
*dst++ = *src;
n--;
}
src++;
}
*dst = '\0';
return m + (src - q);
}
static void GetOpts(int argc, char *argv[]) {
int opt;
g_mode = ' ';
while ((opt = getopt(argc, argv, "?hbctw")) != -1) {
switch (opt) {
case 'w':
g_warn = true;
break;
case 'c':
g_check = true;
break;
case 't':
g_mode = ' ';
break;
case 'b':
g_mode = '*';
break;
case 'h':
case '?':
(void)write(1, USAGE, sizeof(USAGE) - 1);
exit(0);
default:
(void)write(2, USAGE, sizeof(USAGE) - 1);
exit(64);
}
}
}
static void Write(int fd, const char *s, ...) {
va_list va;
char buf[512];
buf[0] = 0;
va_start(va, s);
do {
StrCat(buf, s, sizeof(buf));
} while ((s = va_arg(va, const char *)));
va_end(va);
(void)write(fd, buf, strlen(buf));
}
static bool IsModeCharacter(char c) {
switch (c) {
case ' ':
case '*':
return true;
default:
return false;
}
}
static bool IsSupportedPath(const char *path) {
size_t i;
for (i = 0;; ++i) {
switch (path[i]) {
case 0:
if (i)
return true;
// fallthrough
case '\r':
case '\n':
case '\\':
Write(2, PROG, ": ", path, ": unsupported path\n", NULL);
return false;
default:
break;
}
}
}
static bool GetDigest(const char *path, FILE *f, uint8_t digest[32]) {
size_t got;
uint8_t buf[512];
struct Sha256Ctx ctx;
Sha256Init(&ctx);
while ((got = fread(buf, 1, sizeof(buf), f))) {
Sha256Update(&ctx, buf, got);
}
if (ferror(f)) {
Write(2, PROG, ": ", path, ": ", strerror(errno), "\n", NULL);
return false;
}
Sha256Final(&ctx, digest);
return true;
}
static char *CopyHex(char *s, const void *p, size_t n) {
const char *d, *e;
for (d = (const char *)p, e = d + n; d < e; ++d) {
*s++ = "0123456789abcdef"[(*d >> 4) & 15];
*s++ = "0123456789abcdef"[(*d >> 0) & 15];
}
*s = 0;
return s;
}
static bool ProduceDigest(const char *path, FILE *f) {
char hexdigest[65];
char mode[2] = {g_mode};
unsigned char digest[32];
if (!IsSupportedPath(path))
return false;
if (!GetDigest(path, f, digest))
return false;
CopyHex(hexdigest, digest, 32);
Write(1, hexdigest, " ", mode, path, "\n", NULL);
return true;
}
static char *Chomp(char *line) {
size_t i;
if (line) {
for (i = strlen(line); i--;) {
if (line[i] == '\r' || line[i] == '\n') {
line[i] = '\0';
} else {
break;
}
}
}
return line;
}
static int HexToInt(int c) {
if ('0' <= c && c <= '9') {
return c - '0';
} else if ('a' <= c && c <= 'f') {
return c - 'a' + 10;
} else if ('A' <= c && c <= 'F') {
return c - 'A' + 10;
} else {
return -1;
}
}
static bool CheckDigests(const char *path, FILE *f) {
FILE *f2;
bool k = true;
int a, b, i, line;
const char *path2, *status;
uint8_t wantdigest[32], gotdigest[32];
char buf[64 + 2 + PATH_MAX + 1 + 1], *p;
for (line = 0; fgets(buf, sizeof(buf), f); ++line) {
if (!*Chomp(buf))
continue;
for (p = buf, i = 0; i < 32; ++i) {
if ((a = HexToInt(*p++ & 255)) == -1)
goto InvalidLine;
if ((b = HexToInt(*p++ & 255)) == -1)
goto InvalidLine;
wantdigest[i] = a << 4 | b;
}
if (*p++ != ' ')
goto InvalidLine;
if (!IsModeCharacter(*p++))
goto InvalidLine;
path2 = p;
if (!*path2)
goto InvalidLine;
if (!IsSupportedPath(path2))
continue;
if ((f2 = fopen(path2, "rb"))) {
if (GetDigest(path2, f2, gotdigest)) {
if (!memcmp(wantdigest, gotdigest, 32)) {
status = "OK";
} else {
status = "FAILED";
++g_mismatches;
k = false;
}
Write(1, path2, ": ", status, "\n", NULL);
} else {
k = false;
}
fclose(f2);
} else {
Write(2, PROG, ": ", path2, ": ", strerror(errno), "\n", NULL);
k = false;
}
continue;
InvalidLine:
if (g_warn) {
char linestr[12];
FormatInt32(linestr, line + 1);
Write(2, PROG, ": ", path, ":", linestr, ": ",
"improperly formatted checksum line", "\n", NULL);
}
}
if (ferror(f)) {
Write(2, PROG, ": ", path, ": ", strerror(errno), "\n", NULL);
k = false;
}
return k;
}
static bool Process(const char *path, FILE *f) {
if (g_check) {
return CheckDigests(path, f);
} else {
return ProduceDigest(path, f);
}
}
int main(int argc, char *argv[]) {
int i;
FILE *f;
bool k = true;
GetOpts(argc, argv);
if (optind == argc) {
f = stdin;
k &= Process("-", f);
} else {
for (i = optind; i < argc; ++i) {
if ((f = fopen(argv[i], "rb"))) {
k &= Process(argv[i], f);
fclose(f);
} else {
Write(2, PROG, ": ", argv[i], ": ", strerror(errno), "\n", NULL);
k = false;
}
}
}
if (g_mismatches) {
char ibuf[12];
FormatInt32(ibuf, g_mismatches);
Write(2, PROG, ": WARNING: ", ibuf, " computed checksum did NOT match\n",
NULL);
}
return !k;
}
+13
View File
@@ -0,0 +1,13 @@
---
BasedOnStyle: Mozilla
IndentWidth: 4
ColumnLimit: 80
---
Language: Cpp
AllowShortFunctionsOnASingleLine: false
AlignTrailingComments: false
AlignEscapedNewlines: DontAlign
AlwaysBreakTemplateDeclarations: true
ConstructorInitializerAllOnOneLineOrOnePerLine: true
FixNamespaceComments: true
---
+48
View File
@@ -0,0 +1,48 @@
#-*-mode:makefile-gmake;indent-tabs-mode:t;tab-width:8;coding:utf-8-*-┐
#── vi: set noet ft=make ts=8 sw=8 fenc=utf-8 :vi ────────────────────┘
PKGS += CTL
CTL_ARTIFACTS += CTL_A
CTL = $(CTL_A_DEPS) $(CTL_A)
CTL_A = o/$(MODE)/ctl/ctl.a
CTL_A_FILES := $(wildcard ctl/*)
CTL_A_HDRS = $(filter %.h,$(CTL_A_FILES))
CTL_A_SRCS = $(filter %.cc,$(CTL_A_FILES))
CTL_A_OBJS = $(CTL_A_SRCS:%.cc=o/$(MODE)/%.o)
CTL_A_CHECKS = \
$(CTL_A).pkg \
$(CTL_A_HDRS:%=o/$(MODE)/%.okk) \
CTL_A_DIRECTDEPS = \
LIBC_INTRIN \
LIBC_MEM \
LIBC_STR \
CTL_A_DEPS := $(call uniq,$(foreach x,$(CTL_A_DIRECTDEPS),$($(x))))
$(CTL_A): ctl/ \
$(CTL_A).pkg \
$(CTL_A_OBJS)
$(CTL_A).pkg: \
$(CTL_A_OBJS) \
$(foreach x,$(CTL_A_DIRECTDEPS),$($(x)_A).pkg)
$(CTL_A_OBJS): private \
OVERRIDE_CXXFLAGS += \
-Wframe-larger-than=4096 \
-Walloca-larger-than=4096 \
-ffunction-sections \
-fdata-sections \
CTL_LIBS = $(foreach x,$(CTL_ARTIFACTS),$($(x)))
CTL_SRCS = $(foreach x,$(CTL_ARTIFACTS),$($(x)_SRCS))
CTL_HDRS = $(foreach x,$(CTL_ARTIFACTS),$($(x)_HDRS))
CTL_CHECKS = $(foreach x,$(CTL_ARTIFACTS),$($(x)_CHECKS))
CTL_OBJS = $(foreach x,$(CTL_ARTIFACTS),$($(x)_OBJS))
$(CTL_OBJS): $(BUILD_FILES) ctl/BUILD.mk
.PHONY: o/$(MODE)/ctl
o/$(MODE)/ctl: $(CTL_CHECKS)
+5015
View File
File diff suppressed because it is too large. Load diff
+64
View File
@@ -0,0 +1,64 @@
// -*-mode:c++;indent-tabs-mode:nil;c-basic-offset:4;tab-width:8;coding:utf-8-*-
// vi: set et ft=cpp ts=4 sts=4 sw=4 fenc=utf-8 :vi
#ifndef CTL_EQUAL_H_
#define CTL_EQUAL_H_
namespace ctl {
template<class InputIt1, class InputIt2>
bool
equal(InputIt1 first1, InputIt1 last1, InputIt2 first2)
{
for (; first1 != last1; ++first1, ++first2) {
if (!(*first1 == *first2)) {
return false;
}
}
return true;
}
// Overload that takes a predicate
template<class InputIt1, class InputIt2, class BinaryPredicate>
bool
equal(InputIt1 first1, InputIt1 last1, InputIt2 first2, BinaryPredicate pred)
{
for (; first1 != last1; ++first1, ++first2) {
if (!pred(*first1, *first2)) {
return false;
}
}
return true;
}
// Overloads that take two ranges (C++14 and later)
template<class InputIt1, class InputIt2>
bool
equal(InputIt1 first1, InputIt1 last1, InputIt2 first2, InputIt2 last2)
{
for (; first1 != last1 && first2 != last2; ++first1, ++first2) {
if (!(*first1 == *first2)) {
return false;
}
}
return first1 == last1 && first2 == last2;
}
template<class InputIt1, class InputIt2, class BinaryPredicate>
bool
equal(InputIt1 first1,
InputIt1 last1,
InputIt2 first2,
InputIt2 last2,
BinaryPredicate pred)
{
for (; first1 != last1 && first2 != last2; ++first1, ++first2) {
if (!pred(*first1, *first2)) {
return false;
}
}
return first1 == last1 && first2 == last2;
}
} // namespace ctl
#endif /* CTL_EQUAL_H_ */
+64
View File
@@ -0,0 +1,64 @@
// -*-mode:c++;indent-tabs-mode:nil;c-basic-offset:4;tab-width:8;coding:utf-8-*-
// vi: set et ft=cpp ts=4 sts=4 sw=4 fenc=utf-8 :vi
#ifndef _LIBCPP_INITIALIZER_LIST
#define _LIBCPP_INITIALIZER_LIST
namespace std {
template<class T>
class initializer_list
{
constexpr initializer_list(const T* b, size_t s) noexcept
: begin_(b), size_(s)
{
}
public:
typedef T value_type;
typedef const T& reference;
typedef const T& const_reference;
typedef size_t size_type;
typedef const T* iterator;
typedef const T* const_iterator;
constexpr initializer_list() noexcept : begin_(nullptr), size_(0)
{
}
constexpr size_t size() const noexcept
{
return size_;
}
constexpr const T* begin() const noexcept
{
return begin_;
}
constexpr const T* end() const noexcept
{
return begin_ + size_;
}
private:
const T* begin_;
size_t size_;
};
template<class T>
inline constexpr const T*
begin(initializer_list<T> i) noexcept
{
return i.begin();
}
template<class T>
inline constexpr const T*
end(initializer_list<T> i) noexcept
{
return i.end();
}
} // namespace std
#endif // _LIBCPP_INITIALIZER_LIST
+38
View File
@@ -0,0 +1,38 @@
// -*-mode:c++;indent-tabs-mode:nil;c-basic-offset:4;tab-width:8;coding:utf-8-*-
// vi: set et ft=cpp ts=4 sts=4 sw=4 fenc=utf-8 :vi
#ifndef CTL_LESS_H_
#define CTL_LESS_H_
#include "utility.h"
namespace ctl {
template<class T = void>
struct less
{
constexpr bool operator()(const T& lhs, const T& rhs) const
{
return lhs < rhs;
}
typedef T first_argument_type;
typedef T second_argument_type;
typedef bool result_type;
};
template<>
struct less<void>
{
template<class T, class U>
constexpr auto operator()(T&& lhs,
U&& rhs) const -> decltype(ctl::forward<T>(lhs) <
ctl::forward<U>(rhs))
{
return ctl::forward<T>(lhs) < ctl::forward<U>(rhs);
}
typedef void is_transparent;
};
} // namespace ctl
#endif /* CTL_LESS_H_ */
+352
View File
@@ -0,0 +1,352 @@
// -*-mode:c++;indent-tabs-mode:nil;c-basic-offset:4;tab-width:8;coding:utf-8-*-
// vi: set et ft=cpp ts=4 sts=4 sw=4 fenc=utf-8 :vi
#ifndef CTL_MAP_H_
#define CTL_MAP_H_
#include "set.h"
namespace ctl {
template<typename Key, typename Value, typename Compare = ctl::less<Key>>
class map
{
class EntryCompare
{
public:
explicit EntryCompare(Compare comp = Compare()) : comp_(comp)
{
}
bool operator()(const ctl::pair<const Key, Value>& lhs,
const ctl::pair<const Key, Value>& rhs) const
{
return comp_(lhs.first, rhs.first);
}
private:
Compare comp_;
};
ctl::set<ctl::pair<const Key, Value>, EntryCompare> data_;
public:
using key_type = Key;
using mapped_type = Value;
using value_type = ctl::pair<const Key, Value>;
using size_type = typename ctl::set<value_type, EntryCompare>::size_type;
using difference_type =
typename ctl::set<value_type, EntryCompare>::difference_type;
using key_compare = Compare;
using value_compare = EntryCompare;
using iterator = typename ctl::set<value_type, EntryCompare>::iterator;
using const_iterator =
typename ctl::set<value_type, EntryCompare>::const_iterator;
using reverse_iterator =
typename ctl::set<value_type, EntryCompare>::reverse_iterator;
using const_reverse_iterator =
typename ctl::set<value_type, EntryCompare>::const_reverse_iterator;
map() : data_(EntryCompare())
{
}
explicit map(const Compare& comp) : data_(EntryCompare(comp))
{
}
map(const map& other) = default;
map(map&& other) noexcept = default;
map(std::initializer_list<value_type> init, const Compare& comp = Compare())
: data_(init, EntryCompare(comp))
{
}
template<typename InputIt>
map(InputIt first, InputIt last, const Compare& comp = Compare())
: data_(first, last, EntryCompare(comp))
{
}
map& operator=(const map& other) = default;
map& operator=(map&& other) noexcept = default;
map& operator=(std::initializer_list<value_type> ilist)
{
data_ = ilist;
return *this;
}
iterator begin() noexcept
{
return data_.begin();
}
const_iterator begin() const noexcept
{
return data_.begin();
}
const_iterator cbegin() const noexcept
{
return data_.cbegin();
}
iterator end() noexcept
{
return data_.end();
}
const_iterator end() const noexcept
{
return data_.end();
}
const_iterator cend() const noexcept
{
return data_.cend();
}
reverse_iterator rbegin() noexcept
{
return data_.rbegin();
}
const_reverse_iterator rbegin() const noexcept
{
return data_.rbegin();
}
const_reverse_iterator crbegin() const noexcept
{
return data_.crbegin();
}
reverse_iterator rend() noexcept
{
return data_.rend();
}
const_reverse_iterator rend() const noexcept
{
return data_.rend();
}
const_reverse_iterator crend() const noexcept
{
return data_.crend();
}
bool empty() const noexcept
{
return data_.empty();
}
size_type size() const noexcept
{
return data_.size();
}
size_type max_size() const noexcept
{
return data_.max_size();
}
Value& operator[](const Key& key)
{
return ((data_.insert(make_pair(key, Value()))).first)->second;
}
Value& operator[](Key&& key)
{
return ((data_.insert(make_pair(ctl::move(key), Value()))).first)
->second;
}
Value& at(const Key& key)
{
auto it = find(key);
if (it == end())
__builtin_trap();
return it->second;
}
const Value& at(const Key& key) const
{
auto it = find(key);
if (it == end())
__builtin_trap();
return it->second;
}
ctl::pair<iterator, bool> insert(const value_type& value)
{
return data_.insert(value);
}
ctl::pair<iterator, bool> insert(value_type&& value)
{
return data_.insert(ctl::move(value));
}
template<typename P>
ctl::pair<iterator, bool> insert(P&& value)
{
return data_.insert(value_type(ctl::forward<P>(value)));
}
iterator insert(const_iterator hint, const value_type& value)
{
return data_.insert(hint, value);
}
iterator insert(const_iterator hint, value_type&& value)
{
return data_.insert(hint, ctl::move(value));
}
template<typename P>
iterator insert(const_iterator hint, P&& value)
{
return data_.insert(hint, value_type(ctl::forward<P>(value)));
}
template<typename InputIt>
void insert(InputIt first, InputIt last)
{
data_.insert(first, last);
}
void insert(std::initializer_list<value_type> ilist)
{
data_.insert(ilist);
}
template<typename... Args>
ctl::pair<iterator, bool> emplace(Args&&... args)
{
return data_.emplace(ctl::forward<Args>(args)...);
}
template<typename... Args>
iterator emplace_hint(const_iterator hint, Args&&... args)
{
return data_.emplace_hint(hint, ctl::forward<Args>(args)...);
}
iterator erase(const_iterator pos)
{
return data_.erase(pos);
}
iterator erase(const_iterator first, const_iterator last)
{
return data_.erase(first, last);
}
size_type erase(const Key& key)
{
return data_.erase(make_pair(key, Value()));
}
void swap(map& other) noexcept
{
data_.swap(other.data_);
}
void clear() noexcept
{
data_.clear();
}
iterator find(const Key& key)
{
return data_.find(make_pair(key, Value()));
}
const_iterator find(const Key& key) const
{
return data_.find(make_pair(key, Value()));
}
size_type count(const Key& key) const
{
return data_.count(make_pair(key, Value()));
}
iterator lower_bound(const Key& key)
{
return data_.lower_bound(make_pair(key, Value()));
}
const_iterator lower_bound(const Key& key) const
{
return data_.lower_bound(make_pair(key, Value()));
}
iterator upper_bound(const Key& key)
{
return data_.upper_bound(make_pair(key, Value()));
}
const_iterator upper_bound(const Key& key) const
{
return data_.upper_bound(make_pair(key, Value()));
}
ctl::pair<iterator, iterator> equal_range(const Key& key)
{
return data_.equal_range(make_pair(key, Value()));
}
ctl::pair<const_iterator, const_iterator> equal_range(const Key& key) const
{
return data_.equal_range(make_pair(key, Value()));
}
key_compare key_comp() const
{
return data_.value_comp().comp;
}
value_compare value_comp() const
{
return data_.value_comp();
}
friend bool operator==(const map& lhs, const map& rhs)
{
return lhs.data_ == rhs.data_;
}
friend bool operator!=(const map& lhs, const map& rhs)
{
return !(lhs == rhs);
}
friend bool operator<(const map& lhs, const map& rhs)
{
return lhs.data_ < rhs.data_;
}
friend bool operator<=(const map& lhs, const map& rhs)
{
return !(rhs < lhs);
}
friend bool operator>(const map& lhs, const map& rhs)
{
return rhs < lhs;
}
friend bool operator>=(const map& lhs, const map& rhs)
{
return !(lhs < rhs);
}
friend void swap(map& lhs, map& rhs) noexcept
{
lhs.swap(rhs);
}
};
} // namespace ctl
#endif // CTL_MAP_H_
Loaded 100 of 25560 files, more files were not shown because too many files have changed in this diff. Show more