If invalid UTF-8 is encountered, return the standard Unicode replacement character 0xFFFD. When incrementing or decrementing, make a best effort to follow the code's instructions. It might produce one or more � characters, but that's better than an Error and a hard crash.
Also move character constants to standardized locations.
This fixes various issues I found while trying to compile FSO in a cross
compilation setup using MinGW. This is a prerequisite for merging Tea-CI
support.
When Unicode_text_mode was false, unicode::num_codepoints() would return (the size_t equivalent of) a negative number, because it was checking the distance from end to start instead of the distance from start to end.
This allows to set string variables to localized values which will make
translating missions much easier.
This resolves one of the points mentioned in #1522.
This adds proper Unicode support by allowing the parse system to read
UTF-8 encoded files. The new mode is controlled by a mod option which
allows older mods to continue using the legacy text encoding.
That option changes the behavior of several systems in FSO. It disallows
the usage of Volition fonts since they don't support Unicode encoding
and changes a few usages of special characters to their unicode
counterparts of generic rendering solutions.
This fixes#1415.
Goober5000