Different platforms have different levels of support of encodings and
often have slight variations. We already have tables for most encoding
with only CJK missing. Full transcoding inclusion allows us to get reliable
encoding results independently of platform. The biggest con is the need for
external tables encoding.dat.
It removes a duplicate table for korean in graphics/korfont.cpp
The boundary conditions were reversed in Box.Inbox when compared with
Box.Overlaps. Since InBox is not being used and Overlaps is, I went with the
definitions from Overlaps. Also configured unit tests to exercise this
behavior.
After reading directly from the parent stream, the buffer needs to be
filled with the end of the read data so seeking back in the stream into
the buffered area and reading returns data consistent with the parent
stream.
Fixes bug #11342 (PSP port only).
This can be handled differently by each conversion method.
The "Šáleček" could be transliterated as "Salecek" or as
"S'alecek" or maybe even differently.
Symbols for codes shorter than the prefix table index width are stored
in the table. All the entries in the table with an index starting with
the code are set to the symbol value. That way, when decoding it is
possible to get the number of bits corresponding to the table width from
the bitstream and directly find the symbol value. Longer code still need
to be searched for in the codes list.
* Fixed peekBits not to seek the underlying stream. Seeking can be slow
when the stream is a file.
* Changed multi-bit operations to work on multiple bits at once rather
than iterating over single-bit operations.
This is an almost direct port of a patch for xoreos provided by DrMcCoy.
Symbols for codes shorter than the prefix table index width are stored
in the table. All the entries in the table with an index starting with
the code are set to the symbol value. That way, when decoding it is
possible to get the number of bits corresponding to the table width from
the bitstream and directly find the symbol value. Longer code still need
to be searched for in the codes list.
* Fixed peekBits not to seek the underlying stream. Seeking can be slow
when the stream is a file.
* Changed multi-bit operations to work on multiple bits at once rather
than iterating over single-bit operations.
This is an almost direct port of a patch for xoreos provided by DrMcCoy.
If we want to properly test the computation is in double precision,
we should be using a much smaller error than the one used for the
single precision tests.
Since the math is doing subtraction on numbers around 360 and single
precision has about ~7 decimal digits of accuracy we can only
compare numbers to 360.0001 to be safe i'll use 1e-3.
This should fix debian build.
Its seems the precision that debian is using is rounding the
error delta to zero so that the actual error is not less than
this.
matchString patterns couldn't be used to find files with the # character
as it was only treated as a digit wildcard. SCI expected that to work as
it looks for files that start with the # character.
This was being patched out downstream in Debian. A solution that
works for everybody is to just not use a zero-length string when
testing formatting with no conversion specifications.
Custom deleters of ScopedPtr are not currently fully conforming to
C++11's support for custom deleters in std::unique_ptr for the
sake of simplicity of implementation. Unlike in the standard
library, plain functions and lvalue references are not supported,
nor may custom deleters be passed to the constructor at runtime.
This can be improved in the future, if necessary, by doing what
standard library implementations usually do and creating a Pair
class that uses the Empty Base Optimization idiom to avoid extra
storage overhead of the deleter instance when it is not needed, as
in typical standard library implementations, plus some additional
type traits to support the necessary metaprogramming for the
different type overloads.
Although the previous count-constructor would never make a copy of
a member at runtime, Array<T>::reserve *may* copy-construct, so
the compiler would forbid creation of arrays of NonCopyable objects
even when the array was created only once and then never resized
(and thus never actually tried to perform a copy-construction).
This matches the C++11 std::vector method of the same name, and
replaces usage of taking the address of the first element of an
array by &array[0] or &array.front() or &*array.begin(). The data
method is better than these usages because it can be used even
when the array is empty.
These are additions to match C++11 std::vector common init
patterns, to make Common::Array cover more common use cases where
C-style arrays are currently used (and should not be).
Until C++11 (which introduces the z and t length modifiers), there
is no consistent way to print size_t and ptrdiff_t types using
printf formatting across 32-bit, LLP64, and LP64 architectures
without using a cumbersome macro to select the appropriate length
modifier for the target architecture. Since ScummVM engines
currently need to support 32-bit targets, there is no reason at
the moment to support any larger memory sizes in Span anyway.
Span error output is also updated in this commit to reflect that
index values are unsigned.
The recent changes were made in attempt to fix the test building
for NDS target, but since we're not building tests there, these
changes are reverted for the sake of cleaner code.
Implicitly generated constructors can be used instead of explicit
constructors, which reduces the amount of necessary boilerplate.
Long lists of identical typedefs to the superclass are now defined
using a macro.
data() const now returns a pointer to data that matches the
value_type of the data, instead of forcing the data to be const.
This better matches the intent of the Span class, which provides
a view into data, rather than being a container that holds data.
Span is roughly modelled on the GSL span<T> type, and is intended
to replace direct access to raw pointers -- especially pointers
that are passed to functions along with a separate size
parameter. It provides low-cost bounds-checked reads and writes,
as well as convenience functions for reading common values
(integers of varying endianness, strings, etc.). While similar to
MemoryReadStream in purpose, Span is superior in cases where
memory is writable, where memory is accessed randomly rather than
sequentially, or where any invalid access should be treated as an
unrecoverable error. It should also be more efficient than a
MemoryReadStream because it is implemented using CRTP, so there is
no runtime overhead from dynamic dispatch.
NamedSpan is an extension of Span which provides enhanced
debugging information when out-of-bounds memory accesses occur.
It allows programmers to name the memory span at construction time,
and it also tracks the offsets of subspans so that the absolute
byte offset of the original memory can be provided in the error
message if an out-of-bounds access occurs.
SpanOwner is similar to ScopedPtr but has awareness of the design
of Span objects, so allows the memory pointed to by the Span object
inside the SpanOwner to be freed when the SpanOwner is freed
without requiring holding a separate pointer to the start of
memory. It also provides some copy semantics, so unlike a ScopedPtr,
SpanOwners can be held by objects in movable containers like
Common::Array -- but note that because there are no move semantics
in C++98, this means that a new, complete memory copy of the
pointed-to data will be created, rather than just a new Span
pointing to the same block of memory, when a container holding a
SpanOwner expands.
This API is intended for use in cases where C strings come
from untrusted sources like game files, where malformed data
missing the null terminator would cause strlen to read out of
bounds.
I put scare quotes around "correctly" because I can't swear this is the
intended behaviour of the original interpreter.
I don't think accessing filenames that end with / in the .DCPs is even
defined behaviour, so this is a best guess.
COMMON: Add replacement to common/algorithm.h
COMMON: Intermediate commit to show doubts.
COMMON: Basic String::replace() methods implemented.
COMMON: Fix typo in the algorithm.h documentation.
COMMON: Fix documentation of String::replace()
COMMON: Fix formatting issues in method signatures.
COMMON: Add assert and reformat loops in str and algorithm.
COMMON: Fix typo in comment.
COMMON: Fix style in string test cases.
COMMON: Add Doxygen documentation to algorithm and String.
COMMON: Add Doxygen documentation to algorithm and String.
COMMON: Add Doxygen documentation to algorithm.
COMMON: Fix style in algorithm comments.
COMMON: Add Doxygen comments to String.
COMMON: Add Doxygen comments to algorithm test function.
COMMON: Add String support for substring replace.
COMMON: Fix string replace to comply with STL
COMMON: Fix documentation on string replace
COMMON: Fix style in string replace
COMMON: Fix unwanted reference problem in String::replace().
COMMON: Fix indentation in comments for replace
COMMON: Fix indentation in replace
COMMON: Fix comments in String::replace to match implementation.
COMMON: Remove assert to allow for not-null-terminated character arrays
COMMON: Add new test for String::replace
COMMON: Fix broken comments on String::replace
COMMON: Fix sharing bug on ensureCapacity
COMMON: Remove superfluous call to makeUnique()
This specific test checked whether the pointer for two static strings have a
different address. Since the specific strings checked had the same "value"
string pooling optimizations will result in them to have the same address and
thus make this test fail. Furthermore, the test seemed completely pointless,
I simply dropped it now.
A few tests for the Huffman decoder.
The encoding is the example from Wikipedia.
This could be improved by someone more knowledgeable by
generating one at runtime or using multiple encodings
which would each contain one edge case.
find -name '*.h' -or -name '*.cpp' | xargs sed -r -i 's@\(([A-Za-z0-9]+)\*\)@(\1 *)@g'
This seems to have caught some params as well which is not undesirable IMO.
It also caught some strings containing this which is undesirable so I
excluded them manually. (engines/sci/engine/kernel_tables.h)
This drops the (unused) ability to play based on a list of input blocks. This
was formely only used by the NDS specific VOC streaming code, which has been
removed in 9fa9f68789.
Currently there is no iterator returned from this method, to have some
similarity to associative containers of the STL.
I also "added" one unit test for this method, which is basically just
a copy of the HashMap::erase(const Key &) test with the required adaptions.
svn-id: r55661
* names now comply to our naming conventions
* the function computeStreamMD5AsString which computes the MD5
as a hex string now returns it as a Common::String
* add doxygen comments
svn-id: r54121
On my OS X machine, -Wglobal-constructors is always added by configure.
But the test runner makes heavy use of global constructors.
This change allows me to run "make test" even when using -Werror.
svn-id: r54083
This is a first step towards getting rid of all uses of regular printf,
fprintf, vprintf, vfprintf, puts, fputs, etc. in our codebase.
The name format() reflects the purpose of the function, and parallels
String.format() in Java, boost::format, and others.
svn-id: r54004
In SCI01 and up, each typed word may be interpreted as multiple
class,group pairs. This patch adds support to the vocabulary and
parser. It uses the matcher support added in r52985.
This fixes parser issues in German LSL3, but needs testing.
svn-id: r52989
Prior to this change lastPathComponent would not create a correct result,
when the input of lastPathComponent did not contain a single separator.
I also added a test case for this in our unit tests.
svn-id: r52123
* Fix Common::gcd to work with negative input
* This fixes a bug in Common::Rational's multiplication code
* Add some more basic unit tests (including one which checks for
the now fixed multiplication bug)
* cleanup
svn-id: r49064
This removes the dependency on array.h from str.h.
Also, begun migration from the confusing type name "StringList" to
the more appropriate StringArray.
svn-id: r48282
