This header first appeared in 4.4BSD and is provided by a number of C
libraries including Newlib. Several of our embedded projects use this
header and so to make LLVM libc a drop-in replacement, we need to
provide it as well.
For the initial commit, we only implement singly linked variants (SLIST
and STAILQ). The doubly linked variants (LIST, TAILQ and CIRCLEQ) can be
implemented in the future as needed.
In particular, we have internal customers that would like to use nanf
and
scalbnf.
The differences between various entrypoint files can be checked via:
$ comm -3 <(grep libc\.src path/to/entrypoints.txt | sort) \
<(grep libc\.src path/to/other/entrypoints.txt | sort)
* separate initialization routines into _start and do_start for all
architectures.
* lift do_start as a separate object library to avoid code duplication.
* (addtionally) address the problem of building hermetic libc with
-fstack-pointer-*
The `crt1.o` is now a merged result of three components:
```
___
|___ x86_64
| |_______ start.cpp.o <- _start (loads process initial stack and aligns stack pointer)
| |_______ tls.cpp.o <- init_tls, cleanup_tls, set_thread_pointer (TLS related routines)
|___ do_start.cpp.o <- do_start (sets up global variables and invokes the main function)
```
`__stack_chk_fail` uses `write_to_stderr` and `abort` but these
currently aren't included in baremetal targets resulting in a CMake
build error:
```
CMake Error at /llvm-project/libc/cmake/modules/LLVMLibCObjectRules.cmake:693 (target_link_libraries):
Target "libc.src.stdlib.abort" of type UTILITY may not be linked into
another target. One may link only to INTERFACE, OBJECT, STATIC or SHARED
libraries, or to executables with the ENABLE_EXPORTS property set.
Call Stack (most recent call first):
/llvm-project/libc/cmake/modules/LLVMLibCObjectRules.cmake:811 (create_entrypoint_object)
/llvm-project/libc/cmake/modules/LLVMLibCObjectRules.cmake:891 (expand_flags_for_entrypoint_object)
/llvm-project/libc/src/compiler/generic/CMakeLists.txt:1 (add_entrypoint_object)
CMake Error at /llvm-project/libc/cmake/modules/LLVMLibCLibraryRules.cmake:5 (get_target_property):
get_target_property() called with non-existent target
"libc.src.__support.OSUtil.osutil".
Call Stack (most recent call first):
/llvm-project/libc/cmake/modules/LLVMLibCLibraryRules.cmake:36 (collect_object_file_deps)
/llvm-project/libc/cmake/modules/LLVMLibCLibraryRules.cmake:36 (collect_object_file_deps)
/llvm-project/libc/cmake/modules/LLVMLibCLibraryRules.cmake:85 (collect_object_file_deps)
/llvm-project/libc/lib/CMakeLists.txt:26 (add_entrypoint_library)
```
To address these errors, we need to include `abort` in the list of
baremetal entrypoints. We also need to provide `write_to_stderr`
baremetal implementation, but this is challenging since there is no
uniform way to print a message on these platforms (sometimes there may
not be any way to print a message). We instead defer to
`__libc_log_write` which can be provided by the underlying platform. We
use a similar approach for `quick_exit`, defering to
`__libc_quick_exit`.
__stack_chk_fail should be provided by libc.a, not startup files.
Add __stack_chk_fail to existing linux and arm entrypoints. On Windows
(when
not targeting MinGW), it seems that the corresponding function
identifier is
__security_check_cookie, so no entrypoint is added for Windows.
Baremetal
targets also ought to be compileable with `-fstack-protector*`
There is no common header for this prototype, since calls to
__stack_chk_fail
are meant to be inserted by the compiler upon function return when
compiled
`-fstack-protector*`.
This patch lifts aux vector related definitions to app.h. Because
startup's refactoring is in progress, this patch still contains
duplicated changes. This problem will be addressed very soon in an
incoming patch.
Implement `prctl` as specified in
https://man7.org/linux/man-pages/man2/prctl.2.html.
This patch also includes test cases covering two simple use cases:
1. `PR_GET_NAME/PR_SET_NAME`: where userspace data is passed via arg2.
2. `PR_GET_THP_DISABLE`: where return value is passed via syscal retval.
The test cases of mincore require getting correct page size from OS. As
`sysconf` is not functioning correctly, these patches are implemented in
a somewhat confusing way. We revert such patches and will reintroduce
mincore after we correct sysconf.
This reverts 54878b8, 985c0d1 and 418a3a4.
This patch implements `hcreate(_r)/hsearch(_r)/hdestroy(_r)` as
specified in https://man7.org/linux/man-pages/man3/hsearch.3.html.
Notice that `neon/asimd` extension is not yet added in this patch.
- The implementation is largely simplified from rust's
[`hashbrown`](https://github.com/rust-lang/hashbrown/blob/master/src/raw/mod.rs)
as we only consider fix-sized insertion-only hashtables. Technical
details are provided in code comments.
- This patch also contains a portable string hash function, which is
derived from [`aHash`](https://github.com/tkaitchuck/aHash)'s fallback
routine. Not using any SIMD acceleration, it has a good enough quality
(passing all SMHasher tests) and is not too bad in speed.
- Some general functionalities are added, such as `memory_size`,
`offset_to`(alignment), `next_power_of_two`, `is_power_of_two`.
`ctz/clz` are extended to support shorter integers.
Implements the `nexttoward`, `nexttowardf` and `nexttowardl` functions.
Also, raise excepts required by the standard in `nextafter` functions.
cc: @lntue
Currently the only way to add or remove entrypoints is to modify the
entrypoints.txt file for the current target. This isn't ideal since
a user would have to carry a diff for this file when updating their
checkout. This patch adds a basic mechanism to allow the user to remove
entrypoints without modifying the repository.
Software prefetching helps recover performance when hardware prefetching
is disabled. The 'LIBC_COPT_MEMSET_X86_USE_SOFTWARE_PREFETCHING' compile
time option allows users to use this patch.
We compute `pow(x, y)` using the formula
```
pow(x, y) = x^y = 2^(y * log2(x))
```
We follow similar steps as in `log2f(x)` and `exp2f(x)`, by breaking
down into `hi + mid + lo` parts, in which `hi` parts are computed using
the exponent field directly, `mid` parts will use look-up tables, and
`lo` parts are approximated by polynomials.
We add some speedup for common use-cases:
```
pow(2, y) = exp2(y)
pow(10, y) = exp10(y)
pow(x, 2) = x * x
pow(x, 1/2) = sqrt(x)
pow(x, -1/2) = rsqrt(x) - to be added
```
Summary:
We previously had to disable these string functions because they were
not compatible with the definitions coming from the GNU / host
environment. The GPU, when exporting its declarations, has a very
difficult requirement that it be compatible with the host environment as
both sides of the compilation need to agree on definitions and what's
present.
This patch more or less gives up an just copies the definitions as
expected by `glibc` if they are provided that way, otherwise we fall
back to the accepted way. This is the alternative solution to an
existing PR which instead disable's GCC's handling.
Summary:
This patch partially implements the `rand` function on the GPU. This is
partial because the GPU currently doesn't support thread local storage
or static initializers. To implement this on the GPU. I use 1/8th of the
local / shared memory quota to treak the shared memory as thread local
storage. This is done by simply allocating enough storage for each
thread in the block and indexing into this based off of the thread id.
The downside to this is that it does not initialize `srand` correctly to
be `1` as the standard says, it is also wasteful. In the future we
should figure out a way to support TLS on the GPU so that this can be
completely common and less resource intensive.
This patch populates the GPU version of `libm` with missing vendor entrypoints. The vendor math entrypoints are disabled by default but can be enabled with the CMake option `LIBC_GPU_VENDOR_MATH=ON`.
Summary:
This function follows closely with the pattern of all the other
functions. That is, making a new opcode and forwarding the call to the
host. However, this also required modifying the test somewhat. It seems
that not all `libc` implementations follow the same error rules as are
tested here, and it is not explicit in the standard, so we simply
disable these EOF checks when targeting the GPU.
Implementing expm1 function for double precision based on exp function
algorithm:
- Reduced x = log2(e) * (hi + mid1 + mid2) + lo, where:
* hi is an integer
* mid1 * 2^-6 is an integer
* mid2 * 2^-12 is an integer
* |lo| < 2^-13 + 2^-30
- Then exp(x) - 1 = 2^hi * 2^mid1 * 2^mid2 * exp(lo) - 1 ~ 2^hi *
(2^mid1 * 2^mid2 * (1 + lo * P(lo)) - 2^(-hi) )
- We evaluate fast pass with P(lo) is a degree-3 Taylor polynomial of
(e^lo - 1) / lo in double precision
- If the Ziv accuracy test fails, we use degree-6 Taylor polynomial of
(e^lo - 1) / lo in double double precision
- If the Ziv accuracy test still fails, we re-evaluate everything in
128-bit precision.
Summary:
These wrapper headers need to work around things in the standard
headers. The existing workarounds didn't correctly handle the macros for
`iscascii` and `toascii`. Additionally, `memrchr` can't be used because
it has a different declaration for C++ mode. Fix this so it can be
compiled.
This patch enables the compilation of libc for rv32 by unifying the
current rv64 and rv32 implementation into a single rv implementation.
We updated the cmake file to match the new riscv32 arch and force
LIBC_TARGET_ARCHITECTURE to be "riscv" whenever we find "riscv32" or
"riscv64". This is required as LIBC_TARGET_ARCHITECTURE is used in the
path for several platform specific implementations.
Reviewed By: michaelrj
Differential Revision: https://reviews.llvm.org/D148797
Summary:
This patch adds the necessary entrypoints to handle the `fseek`,
`fflush`, and `ftell` functions. These are all very straightfoward, we
simply make RPC calls to the associated function on the other end.
Implementing it this way allows us to more or less borrow the state of
the stream from the server as we intentionally maintain no internal
state on the GPU device. However, this does not implement the `errno`
functinality so that must be ignored.
In a previous patch, the printf writer was rewritten to use a single
writer class with a buffer and a callback hook. This patch refactors
scanf's reader to match conceptually.
Summary:
This patch removes the `rpc_reset` function. This was previously used to
initialize the RPC client on the device by setting up the pointers to
communicate with the server. The purpose of this was to make it easier
to initialize the device for testing. However, this prevented us from
enforcing an invariant that the buffers are all read-only from the
client side.
The expected way to initialize the server is now to copy it from the
host runtime. This will allow us to maintain that the RPC client is in
the constant address space on the GPU, potentially through inference,
and improving caching behaviour.
Summary:
This patch copies a config file for the GPU similar to the
baremetal/embedded implementation. This will configure the
implementations of functions like `sprintf` and `snprintf` to be
compiled into more simple versions that can be run on the GPU. These
functions cannot be enabled yet as Vararg support hasn't landed, but it
will be used then.
Summary:
This patch implements the `fgets`, `getc`, `fgetc`, and `getchar`
functions on the GPU. Their implementations are straightforward enough.
One thing worth noting is that the implementation of `fgets` will be
extremely slow due to the high latency to read a single char. A faster
solution would be to make a new RPC call to call `fgets` (due to the
special rule that newline or null breaks the stream). But this is left
out because performance isn't the primary concern here.
This patch changes the default types of argc/argv so it's no longer a
uint64_t in all systems, instead, it's now a uintptr_t, which fixes
crashes in 32-bit systems that expect 32-bit types. This patch also adds
two uintptr_t types (EnvironType and AuxEntryType) for the same reason.
The patch also adds a PgrHdrTableType type behind an ifdef that's
Elf64_Phdr in 64-bit systems and Elf32_Phdr in 32-bit systems.
This patch adds the long double table option for printf into the new
configuration scheme. This allows it to be set for most targets but
unset for baremetal.
Summary:
This patch simply adds the necessary config to enable qsort and bsearch
on the GPU. It is *highly* unlikely that anyone will use these, as they
are single threaded, but we may as well support all entrypoints that we
can.
Summary:
This patch implements fwrite, putc, putchar, and fputc on the GPU. These
are very straightforward, the main difference for the GPU implementation
is that we are currently ignoring `errno`. This patch also introduces a
minimal smoke test for `putc` that is an exact copy of the `puts` test
except we print the string char by char. This also modifies the `fopen`
test to use `fwrite` to mirror its use of `fread` so that it is tested
as well.
Few printf config options have been setup using this new config system
along with their baremetal overrides. A follow up patch will add generation
of doc/config.rst, which will contain the full list of libc config options
and short description explaining how they affect the libc.
Reviewed By: gchatelet
Differential Revision: https://reviews.llvm.org/D159158
This patch adds the necessary support to provide `assert` functionality
through the GPU `libc` implementation. This implementation creates a
special-case GPU implementation rather than relying on the common
version. This is because the GPU has special considerings for printing.
The assertion is printed out in chunks with `write_to_stderr`, however
when combined with the GPU execution model this causes 32+ threads to
all execute in-lock step. Meaning that we'll get a horribly fragmented
message. Furthermore, potentially thousands of threads could hit the
assertion at once and try to print even if we had it all in one
`printf`.
This is solved by having a one-time lock that each thread group / wave /
warp will attempt to claim. We only let one thread group pass through
while the others simply stop executing. Finally only the first thread in
that group will do the printing until we finally abort execution.
Reviewed By: sivachandra
Differential Revision: https://reviews.llvm.org/D159296
This function implements the `abort` function on the GPU. The
implementation here closely mirros the `exit` call where we first
synchornize with the RPC server to make sure it's listening and then we
exit on the GPU.
I was unsure if this should be a simple `__builtin_assert` on the GPU. I
elected to go with an RPC approach to make this a more "true" `abort`
call. That is, it should invoke some signal handlers and exit with the
proper code according to the implemented C library on the server.
Reviewed By: jdoerfert
Differential Revision: https://reviews.llvm.org/D159210
