Ports aren't guaranteed to compile with `-fsigned-char`, resulting in
them using a different incompatible ABI when using serenity libraries.
I think we should rather follow the target architecture ABI than
assuming that `char` is signed.
Building without `-fsigned-char` only required a couple of small changes
to prevent -Wtype-limits warnings.