darlinghq/darling-docs
1
0
Fork 0
mirror of https://github.com/darlinghq/darling-docs.git synced 2024-11-26 22:00:34 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

Import from dokuwiki

Browse Source
This commit is contained in:
Lubos Dolezel 2020-04-03 22:13:14 +02:00
commit c2e8d9b7f4
30 changed files with 1826 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
.gitignore vendored Normal file
View File

@ -0,0 +1 @@
book

6
book.toml Normal file
View File

@ -0,0 +1,6 @@
[book]
authors = ["Lubos Dolezel"]
language = "en"
multilingual = false
src = "src"
title = "Darling"

29
src/SUMMARY.md Normal file
View File

@ -0,0 +1,29 @@
# Darling
- [Build Instructions](build_instructions.md)
- [Uninstall](uninstall.md)
- [Darling Shell](darling_shell.md)
- [Darling Prefix](darling_prefix.md)
- [Installing Software](installing_software.md)
- [What to Try](what_to_try.md)
- [Documentation](documentation/README.md)
- [Basics](documentation/BASICS.md)
- [Loader](documentation/loader.md)
- [System Call Emulation](documentation/system_call_emulation.md)
- [Containerization](documentation/containerization.md)
- [Threading](documentation/THREADING.md)
- [Thread Implementation](documentation/thread_implementation.md)
- [Thread Local Storage](documentation/thread_local_storage.md)
- [MacOS Specifics](documentation/SPECIFICS.md)
- [Mach Ports](documentation/mach_ports.md)
- [Commpage](documentation/commpage.md)
- [Distributed Objects](documentation/distributed_objects.md)
- [Other](documentation/OTHER.md)
- [Calling Host System APIs](documentation/calling_host_system_apis.md)
- [Contributing](contributing.md)
- [High Priority Stuff](high_priority_stuff.md)
- [Google Summer of Code](google_summer_of_code.md)
- [Generating Stubs](generating_stubs.md)
- [Debugging](debugging.md)

178
src/build_instructions.md Normal file
View File

@ -0,0 +1,178 @@
# Build Instructions
You must be running a 64-bit x86 Linux distribution. Darling cannot be used on a 32-bit x86 system, not even to run 32-bit applications.
# Dependencies
It is recommended that you use at least Clang 3.8. You can force a specific version of Clang (if it is installed on your system) by editing ''Toolchain.cmake''.
Linux 4.9 and higher is required.
## Debian 9
sudo apt install cmake clang-4.0 bison flex xz-utils libfuse-dev libudev-dev pkg-config libc6-dev-i386 linux-headers-amd64 libcap2-bin git libcairo2-dev libgl1-mesa-dev libtiff5-dev libfreetype6-dev libxml2-dev libegl1-mesa-dev libfontconfig1-dev libbsd-dev libxrandr-dev libxcursor-dev libgif-dev libpulse-dev libavformat-dev libavcodec-dev libavresample-dev
## Debian 10
sudo apt install cmake clang-6.0 bison flex xz-utils libfuse-dev libudev-dev pkg-config libc6-dev-i386 linux-headers-amd64 libcap2-bin git libcairo2-dev libgl1-mesa-dev libtiff5-dev libfreetype6-dev libxml2-dev libegl1-mesa-dev libfontconfig1-dev libbsd-dev libxrandr-dev libxcursor-dev libgif-dev libpulse-dev libavformat-dev libavcodec-dev libavresample-dev
## Debian Testing
sudo apt install cmake clang-7 bison flex xz-utils libfuse-dev libudev-dev pkg-config libc6-dev-i386 linux-headers-amd64 libcap2-bin git libcairo2-dev libgl1-mesa-dev libtiff5-dev libfreetype6-dev libxml2-dev libegl1-mesa-dev libfontconfig1-dev libbsd-dev libxrandr-dev libxcursor-dev libgif-dev libpulse-dev libavformat-dev libavcodec-dev libavresample-dev
## Ubuntu 18.04.3
sudo apt install cmake clang bison flex libfuse-dev libudev-dev pkg-config libc6-dev-i386 linux-headers-generic gcc-multilib libcairo2-dev libgl1-mesa-dev libtiff5-dev libfreetype6-dev git libelf-dev libxml2-dev libegl1-mesa-dev libfontconfig1-dev libbsd-dev libxrandr-dev libxcursor-dev libgif-dev libavutil-dev libpulse-dev libavformat-dev libavcodec-dev libavresample-dev
## Arch Linux & Manjaro
sudo pacman -S --needed make cmake clang flex bison icu fuse linux-headers gcc-multilib lib32-gcc-libs pkg-config fontconfig cairo libtiff python2 mesa llvm
Make sure you install the headers package that matches your kernel version. The kernel version can be checked with ''uname -r''.
$ uname -r
5.4.6-2-MANJARO
Then you should have ''linux54-headers'' installed. You will typically be prompted but may have to install this manually.
## Fedora and CentOS
[RPMFusion](https///rpmfusion.org/RPM%20Fusion) is required for FFmpeg.
sudo dnf install make cmake clang bison flex python2 glibc-devel.i686 fuse-devel systemd-devel kernel-devel elfutils-libelf-devel cairo-devel freetype-devel.{x86_64,i686} libjpeg-turbo-devel.{x86_64,i686} libtiff-devel.{x86_64,i686} fontconfig-devel.{x86_64,i686} libglvnd-devel.{x86_64,i686} mesa-libGL-devel.{x86_64,i686} mesa-libEGL-devel.{x86_64,i686} libxml2-devel libbsd-devel git libXcursor-devel giflib-devel ffmpeg-devel pulseaudio-libs-devel
# Fetch the Sources
Darling makes extensive use of Git submodules, therefore you cannot use a plain ''git clone''. Make a clone like this:
git clone --recursive https://github.com/darlinghq/darling.git
# Updating sources
If you have already cloned Darling and would like to get the latest changes, do this in the source root:
git pull
git submodule init
git submodule update
# Build
The build system of Darling is CMake. Makefiles are generated by CMake by default.
## Building and Installing
Now let's build Darling:
# Move into the cloned sources
cd darling
# Make a build directory
mkdir build && cd build
# Configure the build
cmake ..
# Build and install Darling
make
sudo make install
Darling also requires a kernel module named ''darling-mach'':
make lkm
sudo make lkm_install
If module installation produces warnings such as ''SSL error:02001002:system library:fopen:No such file or directory: bss_file.c:175'', then these can be usually ignored, unless you configured your system to enforce secure boot.
The kernel module is an experimental piece of code; it's likely to have many bugs and vulnerabilities. Be prepared for kernel hangups and crashes, and run Darling on a virtual machine if possible.
## Build Options
You will notice that it takes a long time to build Darling. Darling contains the software layer equivalent to an entire operating system, which means a large amount of code. You can optionally disable some large and less vital parts of the build in order to get faster builds.
To do this, use the ''-DFULL_BUILD=OFF'' option when configuring Darling through CMake.
You may encounter some things to be missing, such as JavaScriptCore. Before creating an issue about a certain library or framework missing from Darling, verify that you are doing a full build by not using this option or setting it to ''ON''.
Another way to speed up the build is to run ''make'' with multiple jobs. For this, run ''make -j8'' instead, where 8 is a number of current jobs to run of your choosing. In general, avoid running more jobs than twice the amount CPU cores of your machine.
If you run lldb and encounter messages indicating a lack of debug symbols, make sure you are doing a debug build. To do this, use the ''-DCMAKE_BUILD_TYPE=Debug''.
## Known Issues
### BackBox
If your distribution is Backbox and you run into build issues try the following commands:
sudo update-alternatives --install /usr/bin/clang clang /usr/bin/clang-4.0 400
sudo update-alternatives --install /usr/bin/clang++ clang++ /usr/bin/clang++-4.0 400
### SELinux
On SELinux you may see the following error when starting darling:
`Cannot open mnt namespace file: No such file or directory`
To work around this try this command: ''setsebool -P mmap_low_allowed 1''.
### Secure Boot
If Secure Boot is enabled you may see:
modprobe: ERROR: could not insert 'darling_mach': Operation not permitted
Failed to load the kernel module
Use the following commands to generate a key and self-sign the kernel module:
# Generate Key
openssl req -new -x509 -newkey rsa:2048 -keyout MOK.priv -outform DER -out MOK.der -nodes -days 36500 -subj "/CN=Darling LKM/"
# Enroll Key
sudo mokutil --import MOK.der
# Sign Module
sudo kmodsign sha512 MOK.priv MOK.der /lib/modules/$(uname -r)/extra/darling-mach.ko
# Reboot System and Enroll Key
### No rule to make target 'modules'
This error can occur for a number of reasons. The most common is that your currently running kernel is no longer installed, which occurs after an upgrade. Before trying other steps reboot your system in order to test against this.
Another cause is that the kernel headers may not be installed. Distributions such as Ubuntu will install the correct headers automatically, but Arch/Manjaro may require you to install the appropriate headers manually. See the "Arch Linux & Manjaro" section earlier on this page for instructions on how to install the appropriate Linux headers.
make -C /lib/modules/5.4.2-1-MANJARO/build M=/home/xeab/Downloads/darling/src/lkm modules
make[5]: Entering directory '/usr/lib/modules/5.4.2-1-MANJARO/build'
make[5]: *** No rule to make target 'modules'. Stop.
# See also
[Uninstall](Uninstall)

38
src/contributing.md Normal file
View File

@ -0,0 +1,38 @@
# Contributing
If you are familiar with how GitHub Pull Requests work, you should feel right at home contributing to Darling.
# Fork the repository
Locate the repository that you made changes in on GitHub. The following command can help.
user@machine:src/external/less$ git remote get-url origin
https://github.com/darlinghq/darling-less.git
If it is an https scheme then you can paste the URL directly into your browser.
Once at the page for the repository you made changes to, click the fork button. GitHub will then take you to the page for your fork of that repository. The last step here is to copy the URL for your fork. Use the Clone or download button to copy it.
# Commit and push your changes
Create and check out a branch describing your changes. In this example, we will use reinvent-wheel. Next, add your fork as a remote.
git remote add myfork git@github.com:ahyattdev/darling-less.git
After this, push your commits to your fork.
git push -u myfork reinvent-wheel
The -u myfork part is only necessary when a branch has never been pushed to your fork before.
# Submit a pull request
On the GitHub page of your fork, select the branch you just pushed from the branch dropdown menu (you may need to reload). Click the New pull request button. Give it a useful title and descriptive comment. After this, you can click create.
After this, your changes will be reviewed by a Darling Project member.

18
src/darling_prefix.md Normal file
View File

@ -0,0 +1,18 @@
# Darling Prefix
Darling prefix is a [container](documentation/containerization) overlayed on top of a base macOS-like root file system located in ''$installation_prefix/libexec/darling''. The default location is ''~/.darling'' and this can be controlled with the ''DPREFIX'' environment variable, very much like ''WINEPREFIX'' under Wine.
The container uses overlayfs along with a user mount namespace to provide a different idea of ''/'' for macOS applications.
When you run an executable inside the prefix for the first time (after boot), an init-like process representing the container is started. This init process keeps the root file system mounted. In future, Apple's ''launchd'' will be used for this purpose.
## Updating the Prefix
Unlike Wine, Darling doesn't need to update the prefix whenever the Darling installation is updated. There is one caveat, though: since overlayfs caches the contents of underlying file system(s), you may need to terminate the container to see Darling's updated files:
darling shutdown
Note that this will terminate all processes running in the container.

45
src/darling_shell.md Normal file
View File

@ -0,0 +1,45 @@
# Darling Shell
We plan to implement a nice and user-friendly GUI for Darling, but for now the primary way to use Darling and interact with it is via the Darling Shell.
## Basic Usage
To get a shell inside the [container](documentation/containerization), just run ''darling shell'' as a regular user. Behind the scenes, this command will start the container or connect to an already-running one and spawn a shell inside. It will also automatically load the kernel module and initialize the [prefix](Darling Prefix) contents if needed.
Inside, you'll find an emulated macOS-like environment. macOS is Unix-like, so most familiar commands will work. For example, it may be interesting to run ''ls -l /'', ''uname'' and ''sw_vers'' to explore the emulated system. Darling bundles many of the command-line tools macOS ships -- of the same ancient versions. The shell itself is Bash version 3.2.
The filesystem layout inside the container is similar to that of macOS, including the top-level ''/Applications'', ''/Users'' and ''/System'' directories. The original Linux filesystem is visible as a separate partition that's mounted on ''/Volumes/SystemRoot''. When running macOS programs under Darling, you'll likely want them to access files in you home folder; to make this very convenient, ''/Users'' and ''/home'' inside the container are both symlinked to ''/Volumes/SystemRoot/home''.
## Running Linux Binaries
You can run normal Linux binaries inside the container, too. They won't make use of Darling's libraries and [system call emulation](documentation/system_call_emulation), but they will still see the macOS-like environment:
$ darling shell
Darling [~]$ uname
Darwin
Darling [~]$ /Volumes/SystemRoot/bin/uname
Linux
Darling [~]$ /Volumes/SystemRoot/bin/ls /System
Library
## Setting up the Environment
Darling Shell is compiled not to read ''~/.bashrc'' at startup. Instead, Darling Shell reads ''~/.dshellrc'', which allows you to configure ''PATH'' in a way that makes sense inside your prefix, for example.
If you want to alter your environment for a specific prefix only, edit ''/etc/bashrc'' inside the prefix.
## Becoming Root
Should you encounter an application that bails out because you are not root (typically because it needs write access outside your home directory), you can use the fake ''sudo'' command. It is fake, because it only makes [getuid](https///developer.apple.com/library/mac/documentation/Darwin/Reference/ManPages/man2/getuid.2.html) and [geteuid](https///developer.apple.com/library/mac/documentation/Darwin/Reference/ManPages/man2/geteuid.2.html) system calls return 0, but grants you no extra privileges.
## Examples
* ''darling shell'': Opens a Bash prompt.
* ''darling shell /usr/local/bin/someapp arg'': Execute ''/usr/local/bin/someapp'' with an argument. Note that the path is evaluated inside the [Darling Prefix](Darling Prefix). The command is started through the shell (uses ''sh -c'').
* ''darling ~/.darling/usr/local/bin/someapp arg'': Equivalent of the previous example (which doesn't make use of the shell), assuming that the prefix is ''~/.darling''.

103
src/debugging.md Normal file
View File

@ -0,0 +1,103 @@
# Debugging
We provide a [build of LLDB](https///osdn.net/projects/darling/storage/apps/lldb.tar.bz2) that is known to work under Darling. It is built from vanilla sources, i.e. without any modifications.
That doesn't mean it works *reliably*. Fully supporting a debugger is a complex task, so LLDB is known to be buggy under Darling.
## Troubleshooting LLDB
If you want to troubleshoot a problem with how LLDB runs under Darling, you need to make ''debugserver'' produce a log file.
If running ''debugserver'' separately, add ''-l targetfile''. If using LLDB directly (which spawns ''debugserver'' as a subprocess automatically), pass an additional environment variable to LLDB:
`LLDB_DEBUGSERVER_LOG_FILE=somelogfile.txt`
## External DebugServer
If you're having trouble using LLDB normally, you may get luckier by running the ''debugserver'' separately.
In one terminal, start the ''debugserver'':
`./debugserver 127.0.0.1:12345 /bin/bash`
In another terminal, connect to the server in LLDB:
$ ./lldb
(lldb) platform select remote-macosx
Platform: remote-macosx
Connected: no
(lldb) process connect connect://127.0.0.1:12345
Please note that environment variables may be missing by default, if used like this.
## How it Works
Debugging support in Darling makes use of what we call "cooperative debugging". It means the code in the debuggee is aware it's being debugged and actively assists the process. In Darling, this role is taken on mainly by [sigexc.c](https///github.com/darlinghq/darling/blob/master/src/kernel/emulation/linux/signal/sigexc.c) in ''libsystem_kernel.dylib'', so no application modifications are necessary.
MacOS debuggers use a combination of BSD-like and Mach APIs to control and inspect the debuggee.
To emulate the macOS behavior, Darling makes use of POSIX real-time signals to invoke actions in the cooperative debugging code.
| Operation | macOS | Linux | Darling implementation |
| --------- | ----- | ----- | ---------------------- |
| Attach to debuggee | ''ptrace(PT_ATTACHEXC)'' \\ Causes the kernel to redirect all signals (aka exceptions) to the Mach "exception port" of the process. Only debuggee termination is notified via ''wait()''. | ''ptrace(PTRACE_ATTACH)'' \\ Signals sent to the debuggee and the debuggee termination event are received in the debugger via ''wait()''. | Notify the LKM that we will be tracing the process. Send a RT signal to the debuggee to notify it of the situation. The debuggee sets up handlers to handle all signals and forward them to the exception port. |
| Examine registers | ''thread_get_state(X86_THREAD_STATE)'' | ''ptrace(PTRACE_GETREGS)'' | Upon receiving a signal, the debuggee reads its own register state and passes it to the kernel via ''thread_set_state()''. |
| Pausing the debuggee | ''kill(SIGSTOP)'' | ''kill(SIGSTOP)'' or ''ptrace(PTRACE_INTERRUPT)'' | Send a RT signal to the debuggee that it should act as if SIGSTOP were sent to the process. We cannot send a real SIGSTOP, because then the debuggee couldn't provide/update register state to the debugger etc. |
| Change signal delivery | ''ptrace(PT_THUPDATE)'' | ''ptrace(PTRACE_rest)'' | Send a RT signal to the debuggee to inform it what it should do with the signal (ignore, pass it to the application etc.) |
| Set memory watchpoints | ''thread_set_state(X86_DEBUG_STATE)'' | ''ptrace(PTRACE_POKEUSER)'' | Implement the effects of ''PTRACE_POKEUSER'' in the LKM. |
## Built-in debugging utilities
### malloc
**libmalloc** (the default library that implements ''malloc()'', ''free()'' and friends) supports many debug features that can be turned on via environment, among them:
* ''MallocScribble'' (fill freed memory with 0x55)
* ''MallocPreScribble'' (fill allocated memory with 0xaa)
* ''MallocGuardEdges'' (guard edges of large allocations)
When this is not enough, you can use **libgmalloc**, which is (for the most part) a drop-in replacement for libmalloc. This is how you use it:
`$ DYLD_INSERT_LIBRARIES=/usr/lib/libgmalloc.dylib DYLD_FORCE_FLAT_NAMESPACE=1 ./test`
libgmalloc catches memory issues such as use-after-free and buffer overflows, and attempts to do this the moment they occur, rather than wait for them to mess up some internal state and manifest elsewhere. It does that by placing each allocation on its own memory page(s), and adding a guard page next to it (like ''MallocGuardEdges'' on steroids) -- by default, after the allocated buffer (to catch buffer overruns), or with ''MALLOC_PROTECT_BEFORE'', before it (to catch buffer underruns). You're likely want to try it both ways, with and without ''MALLOC_PROTECT_BEFORE''. Another useful option is ''MALLOC_FILL_SPACE'' (similar to ''MallocPreScribble'' from above). See [libgmalloc(3)](https///www.manpagez.com/man/3/libgmalloc/) for more details.
### Objective-C and Cocoa
In Objective-C land, you can set ''NSZombieEnabled=YES'' in order to detect use-after-free of Objective-C objects. This replaces ''-[NSObject dealloc]'' with a different implementation that does not deallocate the memory, but instead turns the object into a "zombie". Upon receiving any message (which indicates a use-after-free), the zombie will log the message and abort the process.
Another useful option is ''NSObjCMessageLoggingEnabled=YES'', which will instruct the Objective-C runtime to log all the sent messages to a file in ''/tmp''.
In AppKit, you can set the ''NSShowAllViews'' default (e.g. with ''-NSShowAllViews 1'' on the command line) to cause it to draw a colorful border around each view.
You can find more tips (not all of which work under Darling) at https://developer.apple.com/library/archive/technotes/tn2124/_index.html
### xtrace
You can use **xtrace** to trace Darwin syscalls a program makes, a lot like using ''strace'' on Linux:
$ xtrace vm_stat
[139] fstat64(1, 0x7fffffdfe340) -> 0
[139] host_self_trap() -> port right 2563
[139] mach_msg_trap(0x7fffffdfdfc0, MACH_SEND_MSG|MACH_RCV_MSG, 40, 1072, port 1543, 0, port 0)
[139] {remote = copy send 2563, local = make send-once 1543, id = 219}, 16 bytes of inline data
[139] mach_host::host_statistics64(copy send 2563, 4, 38)
[139] mach_msg_trap() -> KERN_SUCCESS
[139] {local = move send-once 1543, id = 319}, 168 bytes of inline data
[139] mach_host::host_statistics64() -> [75212, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1024, 4159955152, 720014745, 503911965, 0, 4160737656, 2563, 4292863152, 4160733284, 4160733071, 0], 38
[139] write_nocancel(1, 0x7fd456800600, 58)Mach Virtual Memory Statistics: (page size of 4096 bytes)
-> 58
[139] write_nocancel(1, 0x7fd456800600, 49)Pages free: 75212.
-> 49
...
''xtrace'' can trace both BSD syscalls and Mach traps. For ''mach_msg()'' in particular, ''xtrace'' additionally displays the message being sent or received, as well as tries to decode it as a MIG routine call or reply.
Note that ''xtrace'' only traces emulated Darwin syscalls, so any native Linux syscalls made (usually by native ELF libraries) will not be displayed, which means information and open file descriptors may appear to come from nowhere in those cases.

1
src/documentation/BASICS.md Normal file
View File

@ -0,0 +1 @@
# Basics

1
src/documentation/OTHER.md Normal file
View File

@ -0,0 +1 @@
# Other

0
src/documentation/README.md Normal file
View File

1
src/documentation/SPECIFICS.md Normal file
View File

@ -0,0 +1 @@
# MacOS Specifics

1
src/documentation/THREADING.md Normal file
View File

@ -0,0 +1 @@
# Threading

74
src/documentation/calling_host_system_apis.md Normal file
View File

@ -0,0 +1,74 @@
# Calling Host System APIs
This article describes how Darling enables code compiled into Mach-O files to interact with API exported from ELF files on the host system. This is needed, for example, to access ALSA audio APIs.
Apple's dynamic loader (dyld) cannot load ELF files and extending it in this direction would be a rather extensive endeavor, also because Apple's linker (ld64) would need to be extended at the same time. This means some kind of bridge between the host platform's ELF loader and the "Mach-O world" has to be set up.
## The ELF Bridge
The set of information provided to [dyld](documentation/loader#dyld) by [mldr](documentation/loader#mldr) whenever a Mach-O executable is to be launched is precisely defined. You probably know the typical C main function signature:
int main(int argc, const char** argv)
If you have fiddled with this stuff before, you probably know there is also an extra ''const char%%**%% envp'' parameter containing all of the environment variables. Well, on Apple's systems, ''envp'' is followed by ''const char%%**%% apple'' containing miscelaneous extra pieces of information, e.g. a full path to the executable being run or possibly a block of random bytes as a seed for libc.
So this gives us the following signature:
int main(int argc, const char** argv, const char** envp, const char** apple)
These arguments are passed to ''main'' by dyld, which receives them on the stack from ''mldr'' (or from the kernel on a real macOS). By the way, these arguments are also passed to all initializers marked with ''%%__attribute__((constructor))%%''.
The ''apple'' parameter is ideal for passing additional information without interfering with the environment undesirably. mldr [declares](https///github.com/darlinghq/darling/blob/master/src/startup/elfcalls.h) ''struct elf_calls'' with a set of function pointers, instantiates it and fills it out. An address of this structure is then added as a string formatted as ''elf_calls=%p'' into ''apple''.
The address of ''struct elf_calls'' is later parsed inside libsystem_kernel an exported as a symbol named ''_elfcalls''.
## Wrappers
To enable easy linking, a concept of ELF wrappers was introduced, along with a tool named ''wrapgen''. ''wrapgen'' parses ELF libraries, extracts the SONAME (name of library to be loaded) and a list of visible symbols exported from the library.
Now that we have the symbols, a neat trick is used to make them available to Mach-O applications. The Mach-O format supports so called *symbol resolvers*, which are functions that return the address of the symbol they represent. ''dyld'' calls them and provides the result as symbol address to whoever needs the symbol.
Therefore, ''wrapgen'' produces C files such as this:
#include `<elfcalls.h>`
extern struct elf_calls* _elfcalls;
static void* lib_handle;
__attribute__((constructor)) static void initializer() {
lib_handle = _elfcalls->dlopen_fatal("libasound.so.2");
}
__attribute__((destructor)) static void destructor() {
_elfcalls->dlclose_fatal(lib_handle);
}
void* snd_pcm_open() {
__asm__(".symbol_resolver _snd_pcm_open");
return _elfcalls->dlsym_fatal(lib_handle, "snd_pcm_open");
}
The C file is then compiled into a Mach-O library, which transparently wraps an ELF library on the host system.
## CMake Integration
To make things very easy, there is a CMake function that automatically takes care of wrapping a host system's library.
Example:
include(wrap_elf)
include(darling_exe)
wrap_elf(asound libasound.so)
add_darling_executable(pcm_min pcm_min.c)
target_link_libraries(pcm_min system asound)
The ''wrap_elf()'' call creates a Mach-O library of given name, wrapping an ELF library of given name, and installs it into ''/usr/lib/native'' inside the prefix.

27
src/documentation/commpage.md Normal file
View File

@ -0,0 +1,27 @@
# Commpage
Commpage is a special memory structure that is always located at the same address in all processes. On macOS, this mapping is provided by the kernel. In case of Darling, this functionality is supplemented by ''mldr''.
## Purpose
* CPU information (number of cores, CPU capabilities etc.)
* Current precise date and time (this information is not filled in by Darling, causing a fall back to a system call).
* PFZ - preemption-free zone. Contains a piece of code which when run prevents the process from being preempted. This is used for a lock-free implementation of certain primitives (e.g. OSAtomicFifoEnqueue). (Not available under Darling.)
It is somewhat related to [vDSO](https///en.wikipedia.org/wiki/VDSO) on Linux, except that vDSO behaves like a real library, while commpage is just a chunk of data.
The commpage is not documented anywhere, meaning it's not an API intended to be used by 3rd party software. It is however used in source code provided on [opensource.apple.com](http://opensource.apple.com). Darling [provides](https///github.com/darlinghq/darling/blob/master/src/startup/commpage.c) a commpage for compatibility reasons.
## Location
The address differs between 32-bit and 64-bit systems.
* 32-bit systems: ''0xffff0000''
* 64-bit systems: ''0x7fffffe00000''
Note that the 32-bit address is outside of permissible user space area on 32-bit Linux kernels. This is why Darling runs only under 64-bit kernels, which don't have this limitation.

24
src/documentation/containerization.md Normal file
View File

@ -0,0 +1,24 @@
# Containerization
Darling supports use of multiple prefixes (virtual root directories), very much like Wine. Unlike Wine, Darling makes use of Linux's support for various user-controlled namespaces. This makes Darling's prefixes behave a lot more like Docker/LXC containers.
## Implementation
The implementation fully resides in the ''darling'' [binary](https///github.com/darlinghq/darling/blob/master/src/startup/darling.c), which performs several tasks:
* Create a new mount namespace. Mounts created inside the namespace are automatically destroyed when the container is stopped.
* Set up an overlayfs mount, which overlays Darling's readonly root tree (which is installed e.g. in ''/usr/local/libexec/darling'') with the prefix's path. This means the prefix gets updated prefix contents for free (unlike in Wine), but the user can still manipulate prefix contents.
* Use ''pivot_root()'' to change the root directory. The original root is accessible via ''/Volumes/SystemRoot'' (from inside the container).
* Set up a new PID namespace. A virtual "init" process is started, which reaps zombie processes. The init process is also used for joining the namespace. (In future, launchd should be used here.)
More namespaces (e.g. UID or network) will be considered in future.
## Caveats
* When you make changes to Darling's installation directory (e.g. ''/usr/local/libexec/darling''), you must stop running containers (via ''darling shutdown'') so that the changes take effect.

25
src/documentation/distributed_objects.md Normal file
View File

@ -0,0 +1,25 @@
# Distributed Objects
Here's how the Distributed Objects is structured internally:
* **''NSPort''** is a type that abstracts away a *port* -- something that can receive and send messages (''NSPortMessage''). The few commonly used concrete port classes are ''NSMachPort'' (Mach port), ''NSSocketPort'' (network socket, possibly talking to another host), and ''NSMessagePort'' (Unix domain socket). With some luck, it's possible to use a custom port class, too. ''NSPort'' is really supposed to be a Mach port (and that's what ''[NSPort port]'' creates), while other port types have kind of been retrofitted on top of the existing Mach port semantics. ''NSPort'' itself conforms to ''NSCoding'', so you can "send" a port over another port (it does not support coders other than ''NSPortCoder'').
* **''NSPortMessage''** roughly describes a Mach message. It has "send" and "receive" ports, a msgid, and an array of *components*. Individual components can be either data (''NSData'') or a port (''NSPort''), corresponding to ''MACH_MSG_OOL_DESCRIPTOR'' and ''MACH_MSG_PORT_DESCRIPTOR''. Passing a port will only work with ports of the same type as the port you're sending this message through.
* **''NSPortNameServer''** abstracts away a name server that you can use to map (string) names to port. You can register your port for a name and lookup other ports by name. ''NSMachBootstrapServer'' implements this interface on top of the Mach bootstrap server (**launchd** on Darwin).
* **''NSPortCoder''** is an ''NSCoder'' that essentially serializes and deserializes data (and ports) to and from port messages, using the same ''NSCoding'' infrastructure a lot of types already implement. Unlike other coders (read: archivers), it supports encoding and decoding ports, though this is mostly useless as DO itself doesn't make use of this. ''NSPortCoder'' doesn't support keyed coding. It also sends ''replacementObjectForPortCoder:'' (instead of the usual ''replacementObjectForPortCoder:'') to objects being encoded.
* **''NSDistantObject''** is a proxy (''NSProxy'') that stands in for a remote object and forwards any messages over to the remote object. The same ''NSDistantObject'' type is returned by the default ''NSObject'' implementation of ''replacementObjectForPortCoder:'', and this instance is what gets serialized and sent over the connection (and deserialized to a ''NSDistantObject'' on the other end). ''NSDistantObject'' really wants to be serialized or deserialized as a part of an ''NSConnection''.
* **''NSConnection''** represents a connection (described by a pair of ports). ''NSConnection'' stores local and remote object maps, to intern created proxies (''NSDistantObject''s) when receiving or sending objects. ''NSConnection'' actually implements forwarding method invocations, and also serves as a port's delegate, handling received messages.
Notably, ''NSPort'', ''NSPortMessage'', ''NSPortNameServer'', and ''NSPortCoder'' do not "know" they're being used for DO/RPC, and can be used for regular communication directly, with the caveat that ''NSPortCoder'' will attempt to serialize ''NSDistantObject''s instead of most objects, and that will fail unless there is an actual ''NSConnection''.
# Resources
* https://developer.apple.com/library/archive/documentation/Cocoa/Conceptual/DistrObjects/DistrObjects.html
* GNUstep implementation

34
src/documentation/loader.md Normal file
View File

@ -0,0 +1,34 @@
# Loader
Multiple loaders are involved in loading macOS applications on Linux. This is due to the fact that macOS applications use [Mach-O](https///en.wikipedia.org/wiki/Mach-O) as the file format of applications, dynamic libraries and so on, whereas Linux uses the [ELF](https///en.wikipedia.org/wiki/Executable_and_Linkable_Format) format.
## mldr
[mldr](https///github.com/darlinghq/darling/blob/master/src/startup/mldr.c) is the first loader. It's an ordinary Linux executable written in C. Its main purpose is to map the Mach-O executable to be started into the memory, find out what dynamic linker is required to fully load it (''/usr/lib/dyld'' is used on macOS exclusively), load the respective dynamic linker and hand control over to it.
mldr's secondary responsibility is to provide a gateway to the Linux and ELF world. This involves aiding in [thread management](documentation/thread_implementation) or loading required native libraries.
### 32 in 64
As an experimental technology, ''mldr'' is now executing 32-bit macOS applications in 64-bit Linux processes. This is not such a daunting task as it may seem (some general ideas are laid out in [this blog article](http://blog.dolezel.info/2017/02/running-32-bit-code-in-64-bit-linux.html)).
This approach has a few advantages. It eliminates the need for having all of Darling's dependencies also in a 32-bit version. This becomes especially pronounced when building Darling on Debian-based systems:
* Darling builds 64-bit and 32-bit code in one go (so that it can produce fat Mach-O binaries).
* On Debian, you need separate -dev packages for 64-bit and 32-bit build.
* These packages cannot be installed at the same time (they conflict).
There are other benefits as well. For instance, it enables us to map the [commpage](documentation/commpage) at the address where it exists on macOS in 32-bit processes, which would not be possible in 32-bit Linux processes, where the commpage address belongs in the kernel-only memory range (upper 1 GB). This problem is non-existent in 64-bit Linux processes.
There are disadvantages as well. Interfacing with native Linux libraries requires some degree of translation to be performed.
## dyld
Dyld is Apple's dynamic linker. It examines what libraries are needed by the macOS application, loads them and performs other necessary tasks. After it is done, it jumps to application's entry point.
Dyld is special in the sense that it as the only "executable" on macOS, it does not (cannot) link to any libraries. As a consequence of this, it has to be statically linked to a subset of libSystem (counterpart of glibc on Linux).

540
src/documentation/mach_ports.md Normal file
View File

@ -0,0 +1,540 @@
# Mach ports
Mach ports are the IPC primitives under [Mach](Mach). They are conceptually similar to Unix pipes, sockets or message queues: using ports, tasks (and the kernel) can send each other messages.
Drawing the analogy with pipes further,
* A **port** is like a pipe. It is conceptually a message queue maintained by the kernel -- this is where it differs from a pipe, which is an uninterpreted stream of raw bytes.
* Tasks and the kernel itself can enqueue and dequeue messages to/from a port via a **port right** to that port. A port right is a handle to a port that allows either sending (enqueuing) or receiving (dequeuing) messages, a lot like a file descriptor connected to either the read or the write end of a pipe. There are the following kinds of port rights:
* **Receive right**, which allows receiving messages sent to the port. Mach ports are MPSC (multiple-producer, single-consumer) queues, which means that there may only ever be one receive right for each port in the whole system (unlike with pipes, where multiple processes can all hold file descriptors to the read end of one pipe).
* **Send right**, which allows sending messages to the port.
* **Send-once right**, which allows sending one message to the port and then disappears.
* **Port set right**, which denotes a *port set* rather than a single port. Dequeuing a message from a port set dequeues a message from one of the ports it contains. Port sets can be used to listen on several ports simultaneously, a lot like ''select''/''poll''/''epoll''/''kqueue'' in Unix.
* **Dead name**, which is not an actual port right, but merely a placeholder. When a port is destroyed, all existing port rights to the port turn into dead names.
* A **port right name** is a specific integer value a task uses to refer to a port right it holds, a lot like a file descriptor that a process uses to refer to an open file. Sending a port right name, the integer, to another task does *not* allow it to use the name to access the port right, because the name is only meaningful in the context of the *port right namespace* of the original task.
The above compares both port rights and port right names to file descriptors, because Unix doesn't differentiate between the *handle to an open file* and the *small integer value* aspects of file descriptors. Mach does, but even when talking about Mach, it's common to say *a port* or *a port right* actually meaning *a port right name* that denotes the right to the port. In particular, the ''mach_port_t'' C type (aka ''int'') is actually the type of port right names, not ports themselves (which are implemented in the kernel and have the real type of ''struct ipc_port'').
## Low-level API
It's possible to use Mach ports directly by using the ''mach_msg()'' syscall (Mach trap; actually, ''mach_msg()'' is a user-space wrapper over the ''mach_msg_overwrite_trap'' trap) and various ''mach_*'' functions provided by the kernel (which are in fact MIG routines, see below).
Here's an example of sending a Mach message between processes:
`<code c sender.c>`
#include `<stdio.h>`
#include `<mach/mach.h>`
#include `<servers/bootstrap.h>`
int main() {
// Lookup the receiver port using the bootstrap server.
mach_port_t port;
kern_return_t kr = bootstrap_look_up(bootstrap_port, "org.darlinghq.example", &port);
if (kr != KERN_SUCCESS) {
printf("bootstrap_look_up() failed with code 0x%x\n", kr);
return 1;
}
printf("bootstrap_look_up() returned port right name %d\n", port);
// Construct our message.
struct {
mach_msg_header_t header;
char some_text[10];
int some_number;
} message;
message.header.msgh_bits = MACH_MSGH_BITS(MACH_MSG_TYPE_COPY_SEND, 0);
message.header.msgh_remote_port = port;
message.header.msgh_local_port = MACH_PORT_NULL;
strncpy(message.some_text, "Hello", sizeof(message.some_text));
message.some_number = 35;
// Send the message.
kr = mach_msg(
&message.header, // Same as (mach_msg_header_t *) &message.
MACH_SEND_MSG, // Options. We're sending a message.
sizeof(message), // Size of the message being sent.
0, // Size of the buffer for receiving.
MACH_PORT_NULL, // A port to receive a message on, if receiving.
MACH_MSG_TIMEOUT_NONE,
MACH_PORT_NULL // Port for the kernel to send notifications about this message to.
);
if (kr != KERN_SUCCESS) {
printf("mach_msg() failed with code 0x%x\n", kr);
return 1;
}
printf("Sent a message\n");
}
`</code>`
`<code c receiver.c>`
#include `<stdio.h>`
#include `<mach/mach.h>`
#include `<servers/bootstrap.h>`
int main() {
// Create a new port.
mach_port_t port;
kern_return_t kr = mach_port_allocate(mach_task_self(), MACH_PORT_RIGHT_RECEIVE, &port);
if (kr != KERN_SUCCESS) {
printf("mach_port_allocate() failed with code 0x%x\n", kr);
return 1;
}
printf("mach_port_allocate() created port right name %d\n", port);
// Give us a send right to this port, in addition to the receive right.
kr = mach_port_insert_right(mach_task_self(), port, port, MACH_MSG_TYPE_MAKE_SEND);
if (kr != KERN_SUCCESS) {
printf("mach_port_insert_right() failed with code 0x%x\n", kr);
return 1;
}
printf("mach_port_insert_right() inserted a send right\n");
// Send the send right to the bootstrap server, so that it can be looked up by other processes.
kr = bootstrap_register(bootstrap_port, "org.darlinghq.example", port);
if (kr != KERN_SUCCESS) {
printf("bootstrap_register() failed with code 0x%x\n", kr);
return 1;
}
printf("bootstrap_register()'ed our port\n");
// Wait for a message.
struct {
mach_msg_header_t header;
char some_text[10];
int some_number;
mach_msg_trailer_t trailer;
} message;
kr = mach_msg(
&message.header, // Same as (mach_msg_header_t *) &message.
MACH_RCV_MSG, // Options. We're receiving a message.
0, // Size of the message being sent, if sending.
sizeof(message), // Size of the buffer for receiving.
port, // The port to receive a message on.
MACH_MSG_TIMEOUT_NONE,
MACH_PORT_NULL // Port for the kernel to send notifications about this message to.
);
if (kr != KERN_SUCCESS) {
printf("mach_msg() failed with code 0x%x\n", kr);
return 1;
}
printf("Got a message\n");
message.some_text[9] = 0;
printf("Text: %s, number: %d\n", message.some_text, message.some_number);
}
`</code>`
Darling [/tmp]$ ./receiver
mach_port_allocate() created port right name 2563
mach_port_insert_right() inserted a send right
bootstrap_register()'ed our port
# in another terminal:
Darling [/tmp]$ ./sender
bootstrap_look_up() returned port right name 2563
Sent a message
# back in the first terminal:
Got a message
Text: Hello, number: 35
As you can see, in this case the kernel decided to use the same number (2563) for port right names in both processes. This cannot be relied upon; in general, the kernel is free to pick any unused names for new ports.
## Ports as capabilities
In addition to custom "inline" data as shown above, it's possible for a message to contain:
* **Out-of-line data** that will be copied to a new virtual memory page in the receiving task. If possible (i.e. if the out-of-line data is page-aligned in the sending task), Mach will use copy-on-write techniques to pass the data to the receiving task without actually copying it.
* **Port rights** that will be sent to the receiving task. This way, it's possible for a task to transfer its port rights to another task. This is how ''bootstrap_register()'' and ''bootstrap_look_up()'' work in the above example.
The only way for a task to get a port right for a port is to either create that port, or have some other task (or the kernel) send it the right. In this way, Mach ports are *capabilities* (as in [capability-based security](https///en.wikipedia.org/wiki/Capability-based_security)).
Among other things, that means that for Mach programs (including the kernel itself) which allow other tasks to ask it to perform operations by sending messages to a port the program listens on, it's idiomatic not to explicitly check if the sender has any specific "permissions" to perform this task; just having a send right to the port is considered enough of a permission. For example, you can manipulate any task (with calls like ''vm_write()'' and ''thread_create()'') as long as you can get its *task port*; it doesn't matter if you're *root* or not and so on (and indeed, UIDs and other Unix security measures don't exist on the Mach level).
## Where to get ports
It would make a lot of sense to make Mach port inheritable across ''fork()'' and ''exec()'' like file descriptors are (and that is the way it work on Hurd), but on Darwin, they're not. A task starts with a fresh port right namespace after either ''fork()'' or ''exec()'', except for a few **special ports** that *are* inherited:
* **Task port** (aka kernel port), a send right to a port whose receive right is held by the kernel. This port allows to manipulate the task it refers to, including reading and writing its virtual memory, creating and otherwise manipulating its threads, and terminating (killing) the task (see [task.defs](https///github.com/darlinghq/darling/blob/master/platform-include/mach/task.defs), [mach_vm.defs](https///github.com/darlinghq/darling/blob/master/platform-include/mach/mach_vm.defs) and [vm_map.defs](https///github.com/darlinghq/darling/blob/master/platform-include/mach/vm_map.defs)). Call ''mach_task_self()'' to get the name for this port for the caller task. This port is only inherited across ''exec()''; a new task created with ''fork()'' gets a new task port (as a special case, a task also gets a new task port after ''exec()''ing a suid binary). The Mach ''task_create()'' function that allows creating new tasks returns the task port of the new task to the caller; but it's unavailable (always returns ''KERN_FAILURE'') on Darwin, so the only way to spawn a task and get its port is to perform the ["port swap dance"](https///robert.sesek.com/2014/1/changes_to_xnu_mach_ipc.html) while doing a ''fork()''.
* **Host port**, a send right to another port whose receive right is held by the kernel. The host port allows getting information about the kernel and the host machine, such as the OS (kernel) version, number of processors and memory usage statistics (see [mach_host.defs](https///github.com/darlinghq/darling/blob/master/platform-include/mach/mach_host.defs)). Get it using ''mach_host_self()''. There also exists a "privileged host control port" (''host_priv_t'') that allows privileged tasks (aka processes running as root) to *control* the host (see [host_priv.defs](https///github.com/darlinghq/darling/blob/master/platform-include/mach/host_priv.defs)). The official way to get it is by calling ''host_get_host_priv_port()'' passing the "regular" host port; in reality it returns either the same port name (if the task is privileged) or ''MACH_PORT_NULL'' (if it's not).
* **Task name port**, an Apple extension, an unprivileged version of the *task port*. It references the task, but does not allow controlling it. The only thing that seems to be available through it is ''task_info()''.
* **Bootstrap port**, a send right to the *bootstrap server* (on Darwin, this is launchd). The bootstrap server serves as a *server registry*, allowing other servers to export their ports under well-known reverse-DNS names (such as ''com.apple.system.notification_center''), and other tasks to look up those ports by these names. The bootstrap server is the primary way to "connect" to another task, comparable to D-Bus under Linux. The bootstrap port for the current task is available in the ''bootstrap_port'' global variable. On Hurd, the filesystem serves as the server registry, and they use the bootstrap port for passing context to *translators* instead.
* **Seatbelt port**, **task access port**, **debug control port**, which are yet to be documented.
In addition to using the ''*_self()'' traps and other methods mentioned above, you can get all these ports by calling ''task_get_special_port()'', passing in the task port (for the caller task or any other task) and an identifier of the port (''TASK_BOOTSTRAP_PORT'' and so on). There also exist wrapper macros (''task_get_bootstrap_port()'' and so on) that pass the right identifier automatically.
There also exists ''task_set_special_port()'' (and the wrapper macros) that allows you to *change* the special ports for a given task to any send right you provide. ''mach_task_self()'' and all the other APIs discussed above will, in fact, return these replaced ports rather than the *real* ports for the task, host and so on. This is a powerful mechanism that can be used, for example, to disallow task to manipulate itself, log and forward any messages it sends and receives (Hurd's ''rpctrace''), or make it believe it's running on a different version of the kernel or on another host. This is also how tasks get the bootstrap port: the first task (launchd) starts with a null bootstrap port, allocates a port and sets it as the bootstrap port for the tasks it spawns.
There also are two special sets of ports that tasks inherit:
* **Registered ports**, up to 3 of them. They can be registered using ''mach_ports_register()'' and later looked up using ''mach_ports_lookup()''. Apple's XPC uses these to pass down its "XPC bootstrap port".
* **Exception ports**, which are the ports where the kernel should *send* info about *exceptions* happening during the task execution (for example, Unix's Segmentation Fault corresponds to EXC_BAD_ACCESS). This way, a task can handle its own exceptions or let another task handle its exceptions. This is how the Crash Reporter.app works on macOS, and this is also what LLDB uses. Under Darling, the Linux kernel delivers these kinds of events as Unix signals to the process that they happen in, then the process converts the received signals to Mach exceptions and sends them to the correct exception port (see [sigexc.c](https///github.com/darlinghq/darling/blob/master/src/kernel/emulation/linux/signal/sigexc.c)).
As a Darwin extension, there are ''pid_for_task()'', ''task_for_pid()'', and ''task_name_for_pid()'' syscalls that allow converting between Mach task ports and Unix PIDs. Since they essentially circumvent the capability model (PIDs are just integers in the global namespace), they are crippled on iOS/macOS with UID checks, entitlements, SIP, etc. limiting their use. On Darling, they are unrestricted.
Similarly to task ports, threads have their corresponding **thread ports**, obtainable with ''mach_thread_self()'' and interposable with ''thread_set_special_port()''.
## Higher-level APIs
Most programs don't construct Mach messages manually and don't call ''mach_msg()'' directly. Instead, they use higher-level APIs or tools that wrap Mach messaging.
### MIG
MIG stands for **Mach Interface Generator**. It takes interface definitions like this example:
`<code c window.defs>`
subsystem window 35000;
#include `<mach/std_types.defs>`
routine create_window(
server: mach_port_t;
out window: mach_port_t);
routine window_set_frame(
window: mach_port_t;
x: int;
y: int;
width: int;
height: int);
`</code>`
And generates the C boilerplate for serializing and deserializing the calls and sending/handling the Mach messages. Here's a (much simplified) client code that it generates:
`<code c windowUser.c>`
/* Routine create_window */
kern_return_t create_window(mach_port_t server, mach_port_t *window) {
typedef struct {
mach_msg_header_t Head;
} Request;
typedef struct {
mach_msg_header_t Head;
/* start of the kernel processed data */
mach_msg_body_t msgh_body;
mach_msg_port_descriptor_t window;
/* end of the kernel processed data */
mach_msg_trailer_t trailer;
} Reply;
union {
Request In;
Reply Out;
} Mess;
Request *InP = &Mess.In;
Reply *Out0P = &Mess.Out;
mach_msg_return_t msg_result;
InP->Head.msgh_bits = MACH_MSGH_BITS(MACH_MSG_TYPE_COPY_SEND, MACH_MSG_TYPE_MAKE_SEND_ONCE);
/* msgh_size passed as argument */
InP->Head.msgh_request_port = server;
InP->Head.msgh_reply_port = mig_get_reply_port();
InP->Head.msgh_id = 35000;
InP->Head.msgh_reserved = 0;
msg_result = mach_msg(
&InP->Head,
MACH_SEND_MSG|MACH_RCV_MSG|MACH_MSG_OPTION_NONE,
(mach_msg_size_t) sizeof(Request),
(mach_msg_size_t) sizeof(Reply),
InP->Head.msgh_reply_port,
MACH_MSG_TIMEOUT_NONE,
MACH_PORT_NULL
);
if (msg_result != MACH_MSG_SUCCESS) {
return msg_result;
}
*window = Out0P->window.name;
return KERN_SUCCESS;
}
/* Routine window_set_frame */
kern_return_t window_set_frame(mach_port_t window, int x, int y, int width, int height) {
typedef struct {
mach_msg_header_t Head;
int x;
int y;
int width;
int height;
} Request;
typedef struct {
mach_msg_header_t Head;
mach_msg_trailer_t trailer;
} Reply;
union {
Request In;
Reply Out;
} Mess;
Request *InP = &Mess.In;
Reply *Out0P = &Mess.Out;
mach_msg_return_t msg_result;
InP->x = x;
InP->y = y;
InP->width = width;
InP->height = height;
InP->Head.msgh_bits = MACH_MSGH_BITS(MACH_MSG_TYPE_COPY_SEND, MACH_MSG_TYPE_MAKE_SEND_ONCE);
/* msgh_size passed as argument */
InP->Head.msgh_request_port = window;
InP->Head.msgh_reply_port = mig_get_reply_port();
InP->Head.msgh_id = 35001;
InP->Head.msgh_reserved = 0;
msg_result = mach_msg(
&InP->Head,
MACH_SEND_MSG|MACH_RCV_MSG|MACH_MSG_OPTION_NONE,
(mach_msg_size_t) sizeof(Request),
(mach_msg_size_t) sizeof(Reply),
InP->Head.msgh_reply_port,
MACH_MSG_TIMEOUT_NONE,
MACH_PORT_NULL
);
if (msg_result != MACH_MSG_SUCCESS) {
return msg_result;
}
return KERN_SUCCESS;
}
`</code>`
To get the reply from the server, MIG includes a port right (called the *reply port*) in the message, and then performs a send on the server port and a receive on the reply port with a single ''mach_msg()'' call. The client keeps the receive right for the reply port, while the server gets sent a send-once right. This way, event though MIG reuses a single (per-thread) reply port for all the servers it talks to, servers can't impersonate each other.
And the corresponding server code, also simplified:
`<code c windowServer.c>`
/* Routine create_window */
extern kern_return_t create_window(mach_port_t server, mach_port_t *window);
/* Routine create_window */
mig_internal novalue _Xcreate_window(mach_msg_header_t *InHeadP, mach_msg_header_t *OutHeadP) {
typedef struct {
mach_msg_header_t Head;
mach_msg_trailer_t trailer;
} Request;
typedef struct {
mach_msg_header_t Head;
/* start of the kernel processed data */
mach_msg_body_t msgh_body;
mach_msg_port_descriptor_t window;
/* end of the kernel processed data */
} Reply;
Request *In0P = (Request *) InHeadP;
Reply *OutP = (Reply *) OutHeadP;
kern_return_t RetCode;
OutP->window.disposition = MACH_MSG_TYPE_COPY_SEND;
OutP->window.pad1 = 0;
OutP->window.pad2 = 0;
OutP->window.type = MACH_MSG_PORT_DESCRIPTOR;
RetCode = create_window(In0P->Head.msgh_request_port, &OutP->window.name);
if (RetCode != KERN_SUCCESS) {
MIG_RETURN_ERROR(OutP, RetCode);
}
OutP->Head.msgh_bits |= MACH_MSGH_BITS_COMPLEX;
OutP->Head.msgh_size = (mach_msg_size_t) (sizeof(Reply));
OutP->msgh_body.msgh_descriptor_count = 1;
}
/* Routine window_set_frame */
extern kern_return_t window_set_frame(mach_port_t window, int x, int y, int width, int height);
/* Routine window_set_frame */
mig_internal novalue _Xwindow_set_frame(mach_msg_header_t *InHeadP, mach_msg_header_t *OutHeadP) {
typedef struct {
mach_msg_header_t Head;
int x;
int y;
int width;
int height;
mach_msg_trailer_t trailer;
} Request;
typedef struct {
mach_msg_header_t Head;
} Reply;
Request *In0P = (Request *) InHeadP;
Reply *OutP = (Reply *) OutHeadP;
window_set_frame(In0P->Head.msgh_request_port, In0P->x, In0P->y, In0P->width, In0P->height);
}
/* Description of this subsystem, for use in direct RPC */
const struct window_subsystem {
mach_msg_id_t start; /* Min routine number */
mach_msg_id_t end; /* Max routine number + 1 */
unsigned int maxsize; /* Max msg size */
struct routine_descriptor /* Array of routine descriptors */
routine[2];
} window_subsystem = {
35000,
35002,
(mach_msg_size_t) sizeof(union __ReplyUnion__window_subsystem),
{
_Xcreate_window,
_Xwindow_set_frame
}
};
boolean_t window_server(mach_msg_header_t *InHeadP, mach_msg_header_t *OutHeadP) {
mig_routine_t routine;
OutHeadP->msgh_bits = MACH_MSGH_BITS(MACH_MSGH_BITS_REPLY(InHeadP->msgh_bits), 0);
OutHeadP->msgh_remote_port = InHeadP->msgh_reply_port;
/* Minimal size: routine() will update it if different */
OutHeadP->msgh_size = (mach_msg_size_t) sizeof(mig_reply_error_t);
OutHeadP->msgh_local_port = MACH_PORT_NULL;
OutHeadP->msgh_id = InHeadP->msgh_id + 100;
OutHeadP->msgh_reserved = 0;
if ((InHeadP->msgh_id > 35001) || (InHeadP->msgh_id < 35000)) {
return FALSE;
}
routine = window_subsystem.routine[InHeadP->msgh_id - 35000];
(*routine) (InHeadP, OutHeadP);
return TRUE;
}
`</code>`
Client-side usage looks just like invoking the routines:
:::c
mach_port_t server_port = ...;
mach_port_t window;
kern_return_t kr;
kr = create_window(server_port, &window);
kr = window_set_frame(window, 50, 55, 200, 100);
And on the server side, you implement the corresponding functions and then call ''mach_msg_server()'':
:::c
kern_return_t create_window(mach_port_t server, mach_port_t *window) {
// ...
}
kern_return_t window_set_frame(mach_port_t window, int x, int y, int width, int height) {
// ...
}
int main() {
mach_port_t server_port = ...;
mach_msg_server(window_server, window_subsystem.maxsize, server_port, 0);
}
MIG supports a bunch of useful options and features. It's extensively used in Mach for RPC (remote procedure calls), including for communicating between the kernel and the userspace. Other than a few direct Mach traps such as ''msg_send()'' itself, Mach kernel API functions (such as the ones for task and port manipulation) are in fact MIG routines.
### Distributed objects
Distributed objects is a *dynamic* RPC platform, as opposed to MIG, which is fundamentally based on static code generation. Distributed objects allows you to transparently use [Objective-C](documentation/objective-c) objects from other processes as if they were regular Objective-C objects.
`<code objc server.m>`
NSConnection *connection = [NSConnection defaultConnection];
[connection setRootObject: myAwesomeObject];
[connection registerName: @"org.darlinghq.example"];
[[NSRunLoop currentRunLoop] run];
`</code>`
`<code objc client.m>`
NSConnection *connection =
[NSConnection connectionWithRegisteredName: @"org.darlinghq.example"
host: nil];
MyAwesomeObject *proxy = [[connection rootProxy] retain];
[proxy someMethod];
`</code>`
There is no need to statically generate any code for this, it all works at runtime through the magic of objc message forwarding infrastructure. Methods you call may return other objects (or take them as arguments), and the distributed objects support code will automatically either send a copy of the object to the remote process (using ''NSCoding'') or proxy methods called on those objects as well.
Note: Distributed objects don't yet work on Darling.
### XPC
XPC is a newer IPC framework from Apple, tightly integrated with launchd. Its lower-level C API allows processes to exchange [plist](documentation/plist)-like data via Mach messages. Higher-level Objective-C API (''NSXPC*'') exports a proxying interface similar to Distributed objects. Unlike Distributed objects, it's asynchronous, doesn't try to hide the possibility of connection errors, and only allows passing whitelisted types (to prevent certain kinds of attacks).
Apple's XPC is not open source. On Darling, the low-level XPC implementation ([libxpc](https///github.com/darlinghq/darling-libxpc)) is based on NextBSD libxpc. The high-level Cocoa APIs are not yet implemented.
## Useful resources
Note that Apple's version of Mach as used in XNU/Darwin is subtly different than both OSF Mach and GNU Mach.
* [Inter Process Communication - The GNU Mach Reference Manual](https///www.gnu.org/software/hurd/gnumach-doc/Inter-Process-Communication.html)
* [Mach Kernel Interface Reference Manual](http://web.mit.edu/darwin/src/modules/xnu/osfmk/man/)
* [Changes to XNU Mach IPC](https///robert.sesek.com/2014/1/changes_to_xnu_mach_ipc.html)
* [Debugging Mach Ports](https///robert.sesek.com/2012/1/debugging_mach_ports.html)
* [Some Fun with Mach Ports](http://www.foldr.org/~michaelw/log/computers/macosx/task-info-fun-with-mach)
* [Reaching the MACH layer](http://blog.wuntee.sexy/reaching-the-mach-layer)
* [Interprocess communication on iOS with Mach messages](http://ddeville.me/2015/02/interprocess-communication-on-ios-with-mach-messages)
* [Mach Message and Bootstrap Server on OS X](http://przhu.github.io/using%20mac/2012/08/25/mach-message-and-bootstrap-server-on-os-x/)
* [Friday Q&A 2013-01-11: Mach Exception Handlers](https///www.mikeash.com/pyblog/friday-qa-2013-01-11-mach-exception-handlers.html)
* [Mach 3 Server Writer's Guide](http://shakthimaan.com/downloads/hurd/server_writer.pdf)
* [Revisiting Apple IPC: (1) Distributed Objects](https///googleprojectzero.blogspot.com/2015/09/revisiting-apple-ipc-1-distributed_28.html)
* [About Distributed Objects](https///developer.apple.com/library/archive/documentation/Cocoa/Conceptual/DistrObjects/Concepts/AboutDistributedObjects.html)
* [GNUstep Distributed Objects](http://www.gnustep.it/nicola/Tutorials/DistributedObjects/)
* [Objective-C GNUstep Base Programming Manual: 7. Distributed Objects](http://www.gnustep.org/resources/documentation/Developer/Base/ProgrammingManual/manual_7.html)
* [Creating XPC Services](https///developer.apple.com/library/archive/documentation/MacOSX/Conceptual/BPSystemStartup/Chapters/CreatingXPCServices.html)
* [Auditing and Exploiting Apple IPC](https///thecyberwire.com/events/docs/IanBeer_JSS_Slides.pdf)

47
src/documentation/start.md Normal file
View File

@ -0,0 +1,47 @@
# Documentation
Documentation of Darling to help new developers understand how Darling (and macOS) work.
## Basics
* [documentation:Loader](documentation/Loader)
* [documentation:System Call Emulation](documentation/System Call Emulation)
* [documentation:Containerization](documentation/Containerization)
## Threading
* [documentation:Thread Implementation](documentation/Thread Implementation)
* [documentation:Thread Local Storage](documentation/Thread Local Storage)
## MacOS Specifics
* [documentation:Mach Ports](documentation/Mach Ports)
* [documentation:Commpage](documentation/Commpage)
* [documentation:Objective-C](documentation/Objective-C)
* [documentation:Distributed Objects](documentation/Distributed Objects)
## File Formats
* [documentation:DMG](documentation/DMG)
* [documentation:PKG](documentation/PKG)
* [documentation:XAR](documentation/XAR)
* [documentation:XIP](documentation/XIP)
## Other
* [documentation:Calling Host System APIs](documentation/Calling Host System APIs)

43
src/documentation/system_call_emulation.md Normal file
View File

@ -0,0 +1,43 @@
# System Call Emulation
System calls are the most critical interface of all user space software. They are the means of invoking kernel's functionality. Without them, software would not be able to communicate with the user, access the file system or establish network connections.
Every kernel provides a very specific set of system calls, even if there is a similar set of libc APIs available on different systems. For example, the system call set greatly differs between Linux and FreeBSD.
MacOS uses a kernel named XNU. XNU's system calls differ greatly from those of Linux, which is why XNU system call emulation is at the heart of Darling.
## XNU System Calls
Unlike other kernels, XNU has three distinct system call tables:
1. **BSD system calls**, invoked via sysenter/syscall. These calls frequently have a direct counterpart on Linux, which makes them easiest to implement.
2. **Mach system calls**, which use negative system call numbers. These are non-existent on Linux and are mostly implemented in the darling-mach Linux kernel module. These calls are built around [Mach Ports](documentation/mach_ports).
3. **Machine dependent system calls**, invoked via ''int 0x82''. There is only a few of them and they are mainly related to [Thread Local Storage](documentation/thread_local_storage).
## Introduction
Darling emulates system calls by providing a modified ''/usr/lib/system/libsystem_kernel.dylib'' library, the source code of which is located in ''src/kernel''. Even though some parts of the emulation are located in Darling's kernel module (located in ''src/lkm''), Darling's emulation is user space based.
This is why ''libsystem_kernel.dylib'' (and also ''dyld'', which contains a static build of this library) can never be copied from macOS to Darling.
Emulating XNU system calls directly in Linux would have a few benefits, but isn't really workable. Unlike BSD kernels, Linux has no support for foreign system call emulation and having such an extensive and intrusive patchset merged into Linux would be too difficult. Requiring Darling's users to patch their kernels is out of question as well.
### Disadvantages of this approach
* Unability to run a full copy of macOS under Darling (notwithstanding the legality of such endeavor), at least the files mentioned above need to be different.
* Unability to run software making direct system calls. This includes Go applications and some UPX packed executables. Note that [Apple provides no support](https///developer.apple.com/library/content/qa/qa1118/_index.html) for making direct system calls (which is effectively very similar to distributing statically linked executables described in the article) and frequently changes the system call table, hence such software is bound to break over time.
### Advantages of this approach
* Significantly easier development.
* No need to worry about having patches merged into Linux.
* It is easier for users to have newest code available, it is not needed to run the latest kernel to have the most up to date system call emulation.
## Implementation
TODO

19
src/documentation/thread_implementation.md Normal file
View File

@ -0,0 +1,19 @@
# Thread Implementation
## Rationale
Darling uses Apple's original libpthread library. MacOS applications running under Darling therefore use the same exact threading library as on macOS.
Apple's libpthread manages threads through a collection of ''bsdthread*'' system calls, which are [implemented by Darling](https///github.com/darlinghq/darling/tree/master/src/kernel/emulation/linux/bsdthread). This way Apple's libpthread could operate absolutely independently on Linux.
However, there is a huge catch. Darling could set up threads on its own and everything would be working fine, but only unless no calls to native Linux libraries (e.g. PulseAudio) would be made. The problem would become obvious as soon as given Linux library would make any thread-related operations on its own - they would crash immediately. This includes many ''pthread_*'' calls, but Thread-Local Storage access as well.
## Wrapping Native Libpthread
In Darling, ''bsdthread*'' system calls call back into the [loader](documentation/Loader), which [uses](https///github.com/darlinghq/darling/blob/master/src/startup/threads.c) native libpthread to start a thread. Once native libpthread sets up the thread, the control is handed over to Apple's libpthread.
This carries its own share of complications. Apple's libpthread needs control over the stack, hence we cannot use the stack provided and managed by native libpthread. Furthermore, someone needs to call ''pthread_join'' on the native Linux pthread object in order to free all associated resources.
This is why there is a special "reaper" thread fired up once the macOS application starts using threads. Note that starting additional threads not directly initiated by the application - such as the reaper thread - is of no concern on Darling. The use of Grand Central Dispatch (libdispatch) involves possible creation of many unsolicited threads, so applications are not expected to treat additional threads as a sign of "cracking", like some Windows apps may do.

54
src/documentation/thread_local_storage.md Normal file
View File

@ -0,0 +1,54 @@
# Thread-Local Storage
Thread-Local Storage (TLS) is a critical feature enabling store and retrieval of per-thread data.
In user applications (written in C/C++), TLS variables are typically accessed via ''pthread_getspecific()'' and ''pthread_setspecific()'' or via special attributes ''__thread'' or ''thread_local'' (since C++11). However, TLS is no less important even in applications that do not make use of this facility explicitly.
This is because TLS is a much needed functionality used by the threading library libpthread (otherwise ''pthread_self()'' would not work) and also in libc itself (''errno'' is typically a per-thread value to avoid races).
On 32/64-bit x86, old [segment registers](https///en.wikipedia.org/wiki/X86_memory_segmentation) from the 16-bit times have found new use thanks to TLS.
## TLS Setup
1. When a new thread is being set up, a block of memory is allocated by libpthread. This block of memory is usually what ''pthread_self()'' returns.
2. libpthread asks the kernel to set up a new [GDT](https///en.wikibooks.org/wiki/X86_Assembly/Global_Descriptor_Table) entry that refers to this memory block.
3. The entry number is then set into the respective segment register.
### Linux Specific
On Linux, there are two different system calls used to create GDT entries.
* ''set_thread_area()'' is available on both x86 and x86-64.
* ''arch_prctl()'' is only available on x86-64, but unlike ''set_thread_area()'' it also supports 64-bit base addresses.
### macOS Specific
A machine-dependent ("machdep") system call ''thread_fast_set_cthread_self()'' is used to create a new segment. Darling translates this call to one of the above Linux system calls.
## TLS Usage
The concept of memory segments is very simple. You can imagine that the segment refers to a certain area of your process' virtual memory. Any memory accesses you make via the segment register are relative to start of that area and are limited by area's size.
Imagine that you have set up FS to point to an integer array. Now you can set an element whose index is in EAX to value 42:
movl $42, %fs:(%eax, 4)
## Registers
While x86 offers a total of six segment registers, only two of them (FS and GS) can be used for TLS (the others being CS, DS, ES and SS). This effectively limits the number of facilities managing TLS areas in a single process to two. One of them is typically used by the platform's native libc, whilst the other one is typically used by foreign libc's (courtesy of Darling and Wine).
The following table explains which registers are used by whom:
| System | TLS register on i386 | TLS register on x86-64 |
| ------ | -------------------- | ---------------------- |
| Linux libc | GS | FS |
| Apple's libSystem | GS | GS |
| Apple's libSystem (Darling) | FS | GS |
This is also why Wine for macOS can never run under Darling: it would always overwrite either the register used by Linux libc or Darling's libSystem.

59
src/generating_stubs.md Normal file
View File

@ -0,0 +1,59 @@
# Generating Stubs
Darling has a stub generator that is capable of generating stubs for C and Objective-C frameworks and shared libraries. A computer running macOS is required to run the stub generator.
# Preparations
You don't need to do this step if you already have a ''bin'' folder in your home directory, with the ''PATH'' variable pointing to it. If not, copy/paste the following commands into Terminal.
Create the ''bin'' folder if it doesn't exist.
`mkdir ~/bin`
If you ''PATH'' variable does not include the ''bin'' folder, you will need to add it.
`<code>`#If your shell is bash
echo "export PATH=\"~/bin:\$PATH\"" >> ~/.bash_profile && source ~/.bash_profile
#If your shell is zsh
echo "export PATH=\"\$HOME/bin:\$PATH\"" >> ~/.zshenv && source ~/.zshenv`</code>`
## Getting the stub generator
Copy/paste the following command into Terminal. It will download both ''darling-stub-gen'' and ''class-dump'' and place it in the ''bin'' folder
`curl https://raw.githubusercontent.com/darlinghq/darling/master/tools/darling-stub-gen -o ~/bin/darling-stub-gen && chmod +x ~/bin/darling-stub-gen && curl https://github.com/darlinghq/class-dump/releases/download/mojave/class-dump -L -o ~/bin/class-dump && chmod +x ~/bin/class-dump`
## Using the stub generator
To run the stub generator, structure your arguments like this:
`darling-stub-gen /System/Library/Frameworks/DVDPlayback.framework/DVDPlayback DVDPlayback`
The process is identical for dynamic libraries.
The above command will create a folder that can be placed in the ''src/frameworks/'' directory of Darling's source tree. It is generated from the DVDPlayback framework. Note that the first argument points to the actual binary of the framework, not the root directory of the framework.
## Applying the stubs to Darling
Once you have generated the stub folder for the framework, copy that folder into Darling's source tree under ''src/frameworks/''.
Then traverse to the ''src/frameworks/include/'' directory (also located inside Darling's source tree) and create a soft symbolic link. The link should point to the folder inside the include directory (ex: ''MyNewFolder/include/MyNewFolder'' )
Below is an example of creating a soft symbolic link using ''ln''. Don't forget to ''cd'' into ''src/frameworks/include/'' first.
`ln -s ../MyNewFolder/include/MyNewFolder MyNewFolder`
Finally, you will need to add the folder to the build. In ''src/frameworks/CMakeLists.txt'', add the following line: ''add_subdirectory(MyNewFolder)''. Make sure you put it in alphabetical order.
Run a build and make sure your new code compiles. After that completes, you are ready to submit a pull request.
See [Contributing](contributing) for how to submit a pull request. [This commit](https///github.com/darlinghq/darling/commit/92233d4e5ca613658345910d1acf4b3b7620a4f6) is an example of a stub for a framework that was added to Darling using the process described in this article. Most notable is what it does to /src/CMakeLists.txt.
# Known issues
* The stub generator does not currently generate symbols for constants. Those must be manually added if a program needs them.
* Generating stubs for platforms outside of x86 (macOS, iOS Simulator) is not supported
* (**TODO**: Figure out how to generate stubs from a ''dyld_shared_cache'' file)

12
src/google_summer_of_code.md Normal file
View File

@ -0,0 +1,12 @@
# Google Summer of Code
## 2019 Tasks
* Implement NSUserNotification over libnotify or directly over the D-Bus API.
* Implement launch services & open(1).
* Fix all warnings in CoreFoundation, Foundation, AppKit, and CoreGraphics.
* Get TextEdit to compile and fully function.

42
src/high_priority_stuff.md Normal file
View File

@ -0,0 +1,42 @@
# High Priority Stuff
The intention of this page is to serve as a location for pointing developers to areas where work is most needed.
### Work to Be Done
* Reimplement [CoreCrypto](https///github.com/darlinghq/darling-corecrypto). CoreCrypto's source code is publicly available, but its license prevents us from using it for Darling. Luckily, it's not that difficult! Some work has already been done. Having CoreCrypto will also enable us to build a recent version of [CommonCrypto](https///github.com/darlinghq/darling-commoncrypto).
## AppKit.framework
More details to be added.
## libxpc
* Implement missing APIs - many things in xpc_dictionary are missing
* xpc_main should access the calling application's bundle and automatically listen as the defined ''XPCService''.
* This is a little tricky, because libxpc may not use CoreFoundation to parse the ''Info.plist'' (at least not directly). It should use ''_NSGetExecutablePath()'' and ''xpc_create_from_plist()''.
## Foundation.framework
* Implement NSXPC classes that wrap libxpc.
## CoreAudio.framework
* Implement CoreAudio.framework as the core place for platform audio abstraction (PulseAudio, ALSA, ...).
* Modify existing AudioUnit code to use CoreAudio instead of accessing PA/ALSA directly.
* Implement AUGraph and AudioQueue utility APIs.
## CoreServices.framework
* Implement LaunchServices APIs for applications and file type mappings, backed by a database.
* Implement UTI (Uniform Type Identifiers) API, also backed by a database.

51
src/installing_software.md Normal file
View File

@ -0,0 +1,51 @@
# Installing Software
There are multiple ways to install software on macOS, and our aim is to make all of them work on Darling as well. However there currently are a few limitations, mainly the lack of GUI.
### You might not even need to install it
[Unlike Wine](https///wiki.winehq.org/FAQ#I_have_lots_of_applications_already_installed_in_Windows._How_do_I_run_them_in_Wine.3F), Darling can run software that's installed on an **existing macOS installation** on the same computer. This is possible thanks to the way application bundles (''.app''-s) work on macOS and Darling.
To use an app that's already installed, you just need to locate the existing installation (e.g. ''/Volumes/SystemRoot/run/media/username/Macintosh HD/Applications/SomeApp.app'') and run the app from there.
### DMG files
Many apps for macOS are distributed as ''.dmg'' (disk image) files that contain the ''.app'' bundle inside. Under macOS, you would click the DMG to *mount* it and then drag the ''.app'' to your ''Applications'' folder to copy it there.
Under Darling, use ''hdiutil attach SomeApp.dmg'' to mount the DMG (the same command works on macOS too), and then copy the ''.app'' using ''cp'':
Darling [~]$ hdiutil attach Downloads/SomeApp.dmg
/Volumes/SomeApp
Darling [~]$ cp -r /Volumes/SomeApp/SomeApp.app /Applications/
### Archives
Some apps are distributed as archives instead of disk images. To install such an app, unpack the archive using the appropriate CLI tools and copy the ''.app'' to ''/Applications''.
### Mac App Store
Many apps are only available via Apple's Mac App Store. To install such an application in Darling, download the app from a real App Store (possibly running on another computer) and copy the ''.app'' over to Darling.
### PKG files
Many apps use ''.pkg'', the native package format of macOS, as their distribution format. It's not enough to simply copy the contents of a package somewhere, they are really meant to be *installed* and can run arbitrary scripts during installation.
Under macOS, you would use the graphical Installer.app or the command-line ''installer'' utility to install this kind of packages. You can do the latter under Darling as well:
Darling [~]$ installer -pkg mc-4.8.7-0.pkg -target /
Unlike macOS, Darling also has the ''uninstaller'' command, which lets you easily uninstall packages.
### Package managers
There are many third-party package managers for macOS, the most popular one being [Homebrew](https///brew.sh/). Ultimately, we want to make it possible to use all the existing package managers with Darling, however, some may not work well right now.
We have found that the [Rudix Package Manager](http://rudix.org/) works well. Follow instructions on their website to install Rudix itself; you can then use it to install many common Unix utilities, such as ''wget'' and ''mc'':
Darling [~]$ sudo rudix install wget mc

12
src/roadmap.md Normal file
View File

@ -0,0 +1,12 @@
# Roadmap
## Low-Level
* Have ELF (native Linux) code run inside ''/Volumes/SystemRoot''.
* Extend the kernel module to produce Mach-O core dumps.
* Enable macOS Homebrew to run (current hurdle: [missing memberd](https///github.com/darlinghq/darling/issues/353)).
* Fix the [bug](https///github.com/darlinghq/darling/issues/368) with a single Mach port in multiple port sets.

11
src/uninstall.md Normal file
View File

@ -0,0 +1,11 @@
# Uninstall
This page is only for if Darling was build and installed manually as instructed on [Build Instructions](Build Instructions). If you installed Darling through a package manager please remove the related packages using that package manager.
## Uninstall commands
The following commands will completely remove Darling. Replace the source root with the path to your local copy of the Darling source code.
cd darling-source-root && tools/uninstall

330
src/what_to_try.md Normal file
View File

@ -0,0 +1,330 @@
# What to Try
Here are some things you may want to try after installing Darling.
## Print "Hello World"
See if Darling can print the famous greeting:
Darling [~]$ echo Hello World
Hello World
It works!
## Run uname
''uname'' is a standard Unix command to get the name (and optionally the version) of the core OS. On Linux distributions, it prints "Linux":
$ uname
Linux
But Darling emulates a complete Darwin environment, so running ''uname'' results in "Darwin":
Darling [~]$ uname
Darwin
## Run sw_vers
''sw_vers'' (for "software version") is a Darwin command that prints the user-facing name, version and code name (such as "El Capitan") of the OS:
Darling [~]$ sw_vers
ProductName: Mac OS X
ProductVersion: 10.12
BuildVersion: Darling
## Explore the file system
Explore the file system Darling presents to Darwin programs, e.g.:
Darling [~]$ ls -l /
`<...>`
Darling [~]$ ls /System/Library
`<...>`
Darling [~]$ ls /usr/lib
`<...>`
Darling [~]$ ls -l /Volumes
`<...>`
## Inspect the Mach-O binaries
Darling ships with tools like ''nm'' and ''otool'' that let you inspect Mach-O binaries, ones that make up Darling and any third-party ones:
Darling [~]$ nm /usr/lib/libobjc.A.dylib
`<...>`
Darling [~]$ otool -l /bin/bash
`<...>`
## Explore process memory layout
While Darling emulates a complete Darwin system, it's still powered by Linux underneath. Sometimes,
this may prove useful. For example, you can use Linux's ''/proc'' pseudo-filesystem to explore the
running processes. Let's use ''cat'' to explore its own process memory layout:
Darling [~]$ cat /proc/self/maps
`<...>`
## Check out the mounts
Darling runs in a mount namespace that's separate from the host. You can use host's native ''mount'' tool to inspect it:
Darling [~]$ /Volumes/SystemRoot/usr/bin/mount | column -t
/Volumes/SystemRoot/dev/sda3 on /Volumes/SystemRoot type ext4 (rw,relatime,seclabel)
overlay on / type overlay (rw,relatime,seclabel,lowerdir=/usr/local/libexec/darling,upperdir=/home/user/.darling,workdir=/home/user/.darling.workdir)
proc on /proc type proc (rw,relatime)
`<...>`
Notice that not only can you simply run a native ELF executable installed on the host, you can also pipe its output directly into a Darwin command (like ''column'' in this case).
Alternatively, you can read the same info from the ''/proc'' pseudo-filesystem:
Darling [~]$ column -t /proc/self/mounts
`<...>`
## List running processes
Darling emulates the BSD ''sysctl''s that are needed for ''ps'' to work:
Darling [~]$ ps aux
USER PID %CPU %MEM VSZ RSS TT STAT STARTED TIME COMMAND
user 32 0.0 0.4 4229972 13016 ?? ? 1Jan70 0:00.05 ps aux
user 5 0.0 0.5 4239500 15536 ?? ? 1Jan70 0:00.22 /bin/launchctl bootstrap -S System
user 6 0.0 0.4 4229916 11504 ?? ? 1Jan70 0:00.09 /usr/libexec/shellspawn
user 7 0.0 0.6 4565228 17308 ?? ? 1Jan70 0:00.14 /usr/sbin/syslogd
user 8 0.0 0.6 4407876 18936 ?? ? 1Jan70 0:00.15 /usr/sbin/notifyd
user 29 0.0 0.2 4229948 7584 ?? ?N 1Jan70 0:00.03 /usr/libexec/shellspawn
user 30 0.0 0.5 4231736 14268 ?? ? 1Jan70 0:00.11 /bin/bash
user 1 0.0 0.5 4256056 15484 ?? ? 1Jan70 0:00.25 launchd
## Read the manual
Darling ships with many ''man'' pages you can read:
Darling [~]$ man dyld
## Run a script
Like Darwin, Darling ships with a build of Python, Ruby and Perl. You can try running a script or exploring them interactively.
Darling [~]$ python
Python 2.7.10 (default, Sep 8 2018, 13:32:07)
[GCC 4.2.1 Compatible Clang 6.0.1 (tags/RELEASE_601/final)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.platform
'darwin'
## Trace a process
Use our ''xtrace'' tool to trace the emulated Darwin syscalls a process makes:
Darling [~]$ xtrace arch
`<...>`
[223] mach_timebase_info_trap (...)
[223] mach_timebase_info_trap () -> KERN_SUCCESS
[223] issetugid (...)
[223] issetugid () -> 0
[223] host_self_trap ()
[223] host_self_trap () -> port right 2563
[223] mach_msg_trap (...)
[223] mach_msg_trap () -> KERN_SUCCESS
[223] _kernelrpc_mach_port_deallocate_trap (task=2563, name=-6)
[223] _kernelrpc_mach_port_deallocate_trap () -> KERN_SUCCESS
[223] ioctl (...)
[223] ioctl () -> 0
[223] fstat64 (...)
[223] fstat64 () -> 0
[223] ioctl (...)
[223] ioctl () -> 0
[223] write_nocancel (...)
i386
[223] write_nocancel () -> 5
[223] exit (...)
## Control running services
Use ''launchctl'' tool to control ''launchd'':
Darling [~]$ launchctl list
PID Status Label
55 - 0x7ffb0dc015f0.anonymous.launchctl
8 - 0x7ffb0dc04ef0.anonymous.shellspawn
9 - 0x7ffb0dc04c70.anonymous.bash
6 - org.darlinghq.shellspawn
10 - com.apple.notifyd
- 0 com.apple.periodic-daily
- 0 com.apple.periodic-monthly
- 0 com.apple.newsyslog
- 0 com.apple.periodic-weekly
7 - com.apple.syslogd
- 0 com.apple.var-db-dslocal-backup
- -11 com.apple.aslmanager
- 0 com.apple.launchctl.System
Read ''man launchctl'' for more information of other commands ''launchctl'' has.
## Fetch a webpage
See if networking works as it should:
Darling [~]$ curl http://darlinghq.org
`<!DOCTYPE html>`
`<html lang="en-US">`
`<head>`
`<...>`
## Try using sudo
Just like [ the real Mac OS X may](https///www.macrumors.com/2017/11/28/macos-high-sierra-bug-admin-access/ ), Darling allows you to get root priveleges without having to enter any password, except in our case it's a feature:
Darling [~]$ whoami
user
Darling [~]$ sudo whoami
root
Of course, our ''sudo'' command only gives the program the *impression* it's running as root; in reality, it still runs with privileges of your user. Some programs explicitly check that they're running as root, so you can use our ''sudo'' to convince them to run.
## Use a package manager
Download and install the Rudix Package Manager:
Note: Not currently working due to lack of TLS support.
Darling [~]$ curl -s https://raw.githubusercontent.com/rudix-mac/rpm/2015.10.20/rudix.py | sudo python - install rudix
Now you can install arbitrary packages using the ''rudix'' command:
Darling [~]$ sudo rudix install wget mc
## Try running Midnight Commander
If you've installed Midnight Commander (''mc'' package in Rudix), launch it to see if it runs smoothly:
Darling [~]$ mc
## Manually install a package
You can also try installing a ''.pkg'' file manually using the ''installer'' command:
Darling [~]$ installer -pkg mc-4.8.7-0.pkg -target /
Unlike macOS, Darling also ships with an ''uninstaller'' command which you can use to easily uninstall packages.
## Attach disk images
Darling ships with an implementation of ''hdiutil'', a tool that allows you to attach and detach DMG disk images:
Darling [~]$ hdiutil attach Downloads/SomeApp.dmg
/Volumes/SomeApp
Darling [~]$ ls /Volumes/SomeApp
`<...>`
Darling [~]$ cp -r /Volumes/SomeApp/SomeApp.app /Applications/
Darling [~]$ hdiutil detach /Volumes/SomeApp
## Run neofetch
Get the ''neofetch.sh'' script from [ its homepage](https///github.com/dylanaraps/neofetch ) and run it:
Darling [~]$ bash neofetch.sh
'c. user@User-VirtualBox
,xNMM. ------------------------
.OMMMMo OS: macOS Sierra 10.12 Darling x86_64
OMMM0, Kernel: 16.0.0
.;loddo:' loolloddol;. Uptime: 3 hours, 20 mins
cKMMMMMMMMMMNWMMMMMMMMMM0: Shell: bash 3.2.57
.KMMMMMMMMMMMMMMMMMMMMMMMWd. DE: Aqua
XMMMMMMMMMMMMMMMMMMMMMMMX. WM: Quartz Compositor
;MMMMMMMMMMMMMMMMMMMMMMMM: Terminal: /dev/pts/0
:MMMMMMMMMMMMMMMMMMMMMMMM: CPU: GenuineIntel
.MMMMMMMMMMMMMMMMMMMMMMMMX. Memory: 0MiB / 983MiB
kMMMMMMMMMMMMMMMMMMMMMMMMWd.
.XMMMMMMMMMMMMMMMMMMMMMMMMMMk
.XMMMMMMMMMMMMMMMMMMMMMMMMK.
kMMMMMMMMMMMMMMMMMMMMMMd
;KMMMMMMMWXXWMMMMMMMk.
.cooc,. .,coo:.
''neofetch'' exercises a lot of the Darwin internals, including BSD ''sysctl'' calls, Mach IPC and host info API, and the "defaults" subsystem (accessed by the ''defaults'' tool, which is implemented in Objective-C on top of Foundation's ''NSUserDefaults'').
## Compile and run a program
If you have Xcode installed, you can use the SDK it provides to compile and run a program. First, select an SDK using our implementation of the ''xcode-select'' tool:
Darling [~]$ xcode-select --switch /Applications/Xcode.app
Now, build a "Hello World" C program using the Clang compiler:
Darling [~]$ cat > helloworld.c
#include `<stdio.h>`
int main() {
puts("Hello World!");
}
Darling [~]$ clang helloworld.c -o helloworld
And run it:
Darling [~]$ ./helloworld
Hello world!
The whole compiler stack works, how cool is that! Now, let's try Swift:
Darling [~]$ cat > hi.swift
print("Hi!")
Darling [~]$ swiftc hi.swift
Darling [~]$ ./hi
Hi!