Files
mono/mcs/class/corlib/Test/System.Globalization/ChangeLog
T
Damien Diederen 4857d28d99 [480178] Fix handling of surrogate characters.
Note: This revision was originally supplied in six parts. Notes
accompanying each of the parts are reproduced below.

[PATCH v5 1/1] System.Char icall: New Unicode category tables compatible with MS .NET

These new category tables, generated using create-category-table(.cs)
available in the mcs sources, make the Mono runtime understand the
same Unicode category codes as recent versions of Microsoft's runtime.

CategoryData_v2 matches versions v2.0.50727 and v3.5.21022 of .NET.

CategoryData_v4 matches version v4.0.30319 of .NET.  The v4 tables can
be disabled by defining the DISABLE_NET_4_0 preprocessor symbol.

Similarly, the astral portions of both tables can be disabled by
defining the DISABLE_ASTRAL preprocessor symbol.

CAUTION: Mcs has to be updated with the corresponding patches, or Mono
will suffer a low-level (internal call) crash when initializing
System.Char.

[PATCH v5 1/5] create-category-table: Utility to generate reasonably-packed Unicode tables

This program generates (partially) bi-level tables encoding the
contents of the Unicode character category database.

Mono embeds a linear table with category codes for the Unicode
BMP (first 65536 codepoints), and lacks information about characters
in the astral planes--leading to requests such as bug 480178.
Extending the linear table to cover the full codespace is not an ideal
solution, as that would expand the embedded "blob" by a factor of 17.

The new tables generated by this program can be used to support the
full range of characters.  An additional level of indirection used for
characters outside the U+0000..U+FFFF range enables "page" sharing, so
that the total amount of embedded data only grows by 13.5kB.

Cf. in-file comments for usage instructions.

[PATCH v5 2/5] System.Char: Handle astral planes in (string,int) overload

If the string element at index starts a surrogate pair, we decode the
full codepoint and "query" the higher planes of the database.

This fixes bug 480178.

CAUTION: This commit depends on the following runtime change:

  System.Char icall: New Unicode category tables compatible with
  MS .NET.

Without it, Mono will suffer a low-level (internal call) crash when
initializing System.Char.

COMPATIBILITY

The updated Mono runtime as been verified to produce the same results
as Microsoft's; here are the MD5 sums of their Unicode category
database dumps (generated via create-category-table --dump, compiled
and executed under the relevant runtime):

    eba45e00acdc82f9a08873465110aef4  v2.0.50727.dump
    eba45e00acdc82f9a08873465110aef4  v3.5.21022.dump
    56fd5c828fbb9083693835680667fd2c  v4.0.30319.dump

    eba45e00acdc82f9a08873465110aef4  gmcs.dump
    56fd5c828fbb9083693835680667fd2c  dmcs.dump

These results will naturally differ if support for .NET v4 or astral
planes is disabled when building the runtime.  Note, also, that these
are different from the results produced by Mono prior to the
introduction of these changes, even for the BMP.

Other versions of the internal database can be easily generated using
create-category-table(.cs), but this currently requires a rebuild.

PERFORMANCE

Direct array indexing is mandatory for code points in the
U+0000..U+FFFF range; as pointed out by Andreas Nahr, performing
bi-level lookups in the Char.Is*(char) predicates cause the JIT to
stop inlining them and results in an important performance drop.

The simple data access pattern used for higher planes, suggested by
Paolo Molaro, is fairly efficient but currenly only used by this
non-optimized method.

[PATCH v5 3/5] Globalization.IdnMappingTest: U+0221 is not unassigned anymore, use U+18B0

U+0221 is LATIN SMALL LETTER D WITH CURL in Unicode 4.0, and is recognized as a
valid letter in recent versions of the .NET framework.  This causes
GetAsciiInvalid to (unexpectedly) succeed, and the test to fail.

Use U+18B0 instead, which is unassigned as of Unicode 5.2 and .NET 3.5 SP1.

[PATCH v5 4/5] CharCategoryTest: Update category dumps to match MS .NET

The category codes in newer versions of Microsoft's framework are not
compatible with the ones encoded in this class, causing tests to fail.
Fix that by using new sets of dumps generated under v3.5.21022 and
v4.0.30319.

Also, drop CharTest.IsWhiteSpace_Complete as the same checks are
performed by CharCategoryTest.IsWhiteSpace.

While we are at it, improve the generator so that:

  - common code is shared between the generator and the test;

  - a number of test cases can be generated at once via
    command-line arguments;

  - the dumps are word-wrapped, enhancing legibility and avoiding
    line length limitations in some compilers.

[PATCH v5 5/5] CharTest: Add tests for astral plane characters/surrogate pairs

  - Show that Char.GetUnicodeCategory(string,int) results in a
    lookup into the higher planes of the Unicode database when invoked
    on a valid surrogate pair, and returns the "normal" category code
    otherwise.

  - Perform a few sanity checks on the results of ConvertFromUtf32
    and GetUnicodeCategory when invoked on codepoints from the
    higher planes.
(cherry picked from commit 70dff8b4272dba83bed23e6502499646526efd59)
2011-07-25 12:43:03 -07:00

393 lines
12 KiB
Plaintext

2010-05-14 Damien Diederen <[email protected]>
* IdnMappingTest.cs: U+0221 is not unassigned anymore, use U+18B0
U+0221 is LATIN SMALL LETTER D WITH CURL in Unicode 4.0,
and is recognized as a valid letter in recent versions of the
.NET framework. This causes GetAsciiInvalid to (unexpectedly)
succeed, and the test to fail.
Use U+18B0 instead, which is unassigned as of Unicode 5.2 and
.NET 3.5 SP1.
2009-12-09 Jb Evain <[email protected]>
* StringInfoTest.cs: test the default constructor.
2009-10-29 Sebastien Pouliot <[email protected]>
* CompareInfoTest.cs: Add test cases to validate parameters
2009-06-25 Zoltan Varga <[email protected]>
* *.cs: Convert all tests to new-style nunit classes/methods.
2009-05-27 Zoltan Varga <[email protected]>
* CultureInfoTest.cs: Fix the net 1.1 build.
2009-05-21 Atsushi Enomoto <[email protected]>
* CultureInfoTest.cs : test zh-Hant.
2009-01-07 Atsushi Enomoto <[email protected]>
* CompareInfoTest.cs : added test for #462957.
2008-08-14 Atsushi Enomoto <[email protected]>
* CultureInfoTest.cs : added test for bug #402128.
2008-04-02 Atsushi Enomoto <[email protected]>
* SortKeyTest.cs : new test, for bug #376171.
2007-11-03 Atsushi Enomoto <[email protected]>
* DateTimeFormatInfoTest.cs : enabled test for bug #332553.
2007-10-30 Atsushi Enomoto <[email protected]>
* DateTimeFormatInfoTest.cs : added test for bug #335298.
2007-10-10 Gert Driesen <[email protected]>
* DateTimeFormatInfoTest.cs: Added test for bug #332553.
2007-07-17 Atsushi Enomoto <[email protected]>
* CalendarTest.cs : true test for bug #81783 (overflown month in
AddMonths().)
2007-07-10 Atsushi Enomoto <[email protected]>
* StringInfoTest.cs : new test.
2007-07-10 Atsushi Enomoto <[email protected]>
* CalendarTest.cs : enabled ToFourDigitYear() test from 5 years ago.
and added some more ToFourDigitYear() tests.
2007-07-09 Atsushi Enomoto <[email protected]>
* CalendarTest.cs : added GetWeekOfYear() test.
2007-07-05 Gert Driesen <[email protected]>
* CultureInfoTest.cs: Fixed UseUserOverride tests to pass on systems
where invariant culture is default locale.
2007-07-03 Gert Driesen <[email protected]>
* CultureInfoTest.cs: Added extra info in order to troubleshoot
buildbot failure on redhat9-i386.
2007-07-03 Atsushi Enomoto <[email protected]>
* CalendarTest.cs, EastAsianLunisolarCalendarTest.cs :
more tests needed mark as NotWorking.
2007-07-02 Gert Driesen <[email protected]>
* EastAsianLunisolarCalenderTest.cs: Fixed build.
2007-07-02 Atsushi Enomoto <[email protected]>
* EastAsianLunisolarCalendarTest.cs : new test, totally not-working.
* CalendarTest.cs : added couple of 2.0 tests. Some are not working.
2007-06-24 Gert Driesen <[email protected]>
* CultureInfoTest.cs: Added exception tests when invoking NumberFormat
or DateTimeFormat on a neutral culture. Added tests for bug #81930.
No longer derive from deprecated Assertion class. Added tests for
GetCultureInfo (2.0).
2007-06-01 Atsushi Enomoto <[email protected]>
* CalendarTest.cs : test for bug #81783 and AddYears() for leap year
(similar issue).
2007-05-31 Rolf Bjarne Kvinge <[email protected]>
* DateTimeFormatInfo.cs: Added EraName.
2007-05-25 Atsushi Enomoto <[email protected]>
* IdnMappingTest.cs : new test.
2007-05-15 Atsushi Enomoto <[email protected]>
* RegionInfoTest.cs : new tests.
2007-03-06 Atsushi Enomoto <[email protected]>
* CompareInfoTest.cs : added test for bug #79714.
2007-02-15 Atsushi Enomoto <[email protected]>
* CompareInfoTest.cs : added test for bug #80865.
2007-01-25 Atsushi Enomoto <[email protected]>
* CompareInfoTest.cs : added test for bug #80612.
2007-01-22 Atsushi Enomoto <[email protected]>
* CompareInfoTest.cs : Added Compare() tests for hyphens.
Removed NotWorking from previous new test (its existence wasn't
even recognized until today).
2007-01-11 Gert Driesen <[email protected]>
* CompareInfoTest.cs: Added test for checking valid CompareOptions for
GetSortKey.
2006-12-25 Atsushi Enomoto <[email protected]>
* CompareInfoTest.cs : added tests for OrdinalIgnoreCase versions
of IndexOf() and LastIndexOf(). Added Ordinal LastIndexOf() tests.
2006-11-27 Atsushi Enomoto <[email protected]>
* CompareInfoTest.cs : why there was "return" before OrdinalIgnoreCase
tests ...
2006-06-08 Atsushi Enomoto <[email protected]>
* DateTimeFormatInfoTest.cs : added test for bug #78569.
2006-04-25 Atsushi Enomoto <[email protected]>
* CompareInfoTest.cs : seems like Microsoft had introduced several
bugs in CompareInfo, so marked many of them as NotDotNet.
2006-04-10 Atsushi Enomoto <[email protected]>
* CultureInfoTest.cs : added test for bug #69652.
2006-03-21 Atsushi Enomoto <[email protected]>
* CalendarTest.cs : added test for bug #76252.
2006-01-25 Atsushi Enomoto <[email protected]>
* CultureInfoTest.cs : added test for bug #77347.
2005-12-08 Sebastien Pouliot <[email protected]>
* DaylightTimeTest.cs: New. Serialization interop test cases.
2005-12-06 Sebastien Pouliot <[email protected]>
* TextInfoTest.cs: Added new tests for serialization and for new 2.0
features.
2005-12-05 Atsushi Enomoto <[email protected]>
* CompareInfoTest.cs : split the last case in NullCharacter() out
and marked it as NotDotNet (.NET is inconsistent between Compare
and IndexOf).
2005-11-14 Atsushi Enomoto <[email protected]>
* CompareInfoTest.cs : added NullCharacter() for bug #76702.
2005-11-14 Atsushi Enomoto <[email protected]>
* CompareInfoTest.cs : added test for Compare() w/ OrdinalIgnoreCase.
2005-11-14 Atsushi Enomoto <[email protected]>
* CompareInfoTest.cs : added more tests for some specific cultures
(say, Japanese) that have replacement contraction in ASCII range
and caused incorrect results for LastIndexOf() and all dependent
members.
2005-10-14 Ben Maurer <[email protected]>
* DateTimeFormatInfoTest.cs: New test to make sure we do not
return the same array from GetAllDateTimePatterns.
2005-09-27 Atsushi Enomoto <[email protected]>
* CompareInfoTest.cs : added testcase for bug #76243.
2005-09-20 Atsushi Enomoto <[email protected]>
* CompareInfoTest.cs : added empty string cases for IsPrefix/IsSuffix.
2005-09-09 Atsushi Enomoto <[email protected]>
* CompareInfoTest.cs : added another test for IsSuffix() that I found.
2005-08-14 Gert Driesen <[email protected]>
* TextInfoTest.cs: Added ListSeparator test.
2005-08-11 Atsushi Enomoto <[email protected]>
* CultureInfoTest.cs : added OptionalCalendars().
* CompareInfoTest.cs : added possible bug test for checkedFlags
optimization.
2005-07-15 Atsushi Enomoto <[email protected]>
* CompareInfoTest.cs : Some GetSortKey That tests got working.
2005-07-15 Atsushi Enomoto <[email protected]>
* CompareInfoTest.cs : Uncommented tests that got working. Added two
cornercase notworking LastIndexOf() tests.
2005-07-14 Atsushi Enomoto <[email protected]>
* CompareInfoTest.cs : IsSuffix() extender tests got working.
2005-07-14 Atsushi Enomoto <[email protected]>
* CompareInfoTest.cs : added tests for IndexOf() and LastIndexOf()
with ignorable source character. Added some non-working tests.
2005-07-14 Atsushi Enomoto <[email protected]>
* CompareInfoTest.cs : added tests for IndexOf() and LastIndexOf()
that takes an expansion for target char.
2005-07-14 Atsushi Enomoto <[email protected]>
* CompareInfoTest.cs : removed some FIXMEs. Added more extender tests.
2005-07-14 Atsushi Enomoto <[email protected]>
* CompareInfoTest.cs : some special weight tests got working.
Commented on some FIXME tests why they are so.
2005-07-14 Atsushi Enomoto <[email protected]>
* CompareInfoTest.cs : Added extender tests for GetSortKey(),
Compare(), IsPrefix(), IndexOf() and LastIndexOf().
2005-07-13 Atsushi Enomoto <[email protected]>
* CompareInfoTest.cs : Added Compare() test that contains expansion.
Fixed CompareSpecialWeight() which was not testing what I intended.
Added IsPrefix() tests with special weight (copying Compare() tests)
2005-07-13 Atsushi Enomoto <[email protected]>
* CompareInfoTest.cs : added CompareSpecialWeight() and moved special
weight comparison there.
2005-07-13 Atsushi Enomoto <[email protected]>
* CompareInfoTest.cs : added GetSortKey() testcase that tests a
sequence of nonprimary character followed by an extender.
2005-07-13 Atsushi Enomoto <[email protected]>
* CompareInfoTest.cs : added more culture-sensitive "empty" test.
2005-07-13 Atsushi Enomoto <[email protected]>
* CompareInfoTest.cs : Test for IndexOf() and LastIndexOf() with
"empty" target (in culture-sensitive context).
2005-07-13 Atsushi Enomoto <[email protected]>
* CompareInfoTest.cs : added ignorable character in target for
IsPrefix/IsSuffix/IndexOf/LastIndexOf methods.
2005-07-13 Atsushi Enomoto <[email protected]>
* CompareInfoTest.cs : added more special weight testcases.
2005-07-13 Atsushi Enomoto <[email protected]>
* CompareInfoTest.cs : added Czech and Hungarian sorting tests.
2005-07-13 Atsushi Enomoto <[email protected]>
* CompareInfoTest.cs : added more diacritical tests.
2005-07-13 Atsushi Enomoto <[email protected]>
* CompareInfoTest.cs : added test for CJK mapping customization.
2005-07-12 Atsushi Enomoto <[email protected]>
* CompareInfoTest.cs : added GetSortKey() test for Thai.
2005-07-12 Atsushi Enomoto <[email protected]>
* CompareInfoTest.cs : added special weight GetSortKey() test.
2005-07-12 Atsushi Enomoto <[email protected]>
* CompareInfoTest.cs : added several GetSortKey() tests.
2005-07-12 Atsushi Enomoto <[email protected]>
* CompareInfoTest.cs : added shift weight testcases. Commented one
French sort thingy out.
2005-07-12 Atsushi Enomoto <[email protected]>
* CompareInfoTest.cs : added another French testcase.
2005-07-12 Atsushi Enomoto <[email protected]>
* CompareInfoTest.cs : added LastIndexOf() contraction cases and more.
2005-07-12 Atsushi Enomoto <[email protected]>
* CompareInfoTest.cs : added tests for IsSuffix(),
IndexOf(string,string) and LastIndexOf(string,string).
2005-07-12 Atsushi Enomoto <[email protected]>
* CompareInfoTest.cs : added tests for IsPrefix() and optionally
Compare() (which is currently used inside IsPrefix() which is
however wrong when it comes to managed collation).
2005-07-12 Atsushi Enomoto <[email protected]>
* CompareInfoTest.cs : added tests for IndexOf() and LastIndexOf().
Fixed tests for Compare() - it should use overloaded ones of
the tested library (do not call overloaded tests themselves).
2005-07-12 Atsushi Enomoto <[email protected]>
* CompareInfoTest.cs : added a few managed collation tests for
GetSortKey() and Compare() (they are ignored when managed collation
is off).
2005-06-27 Gonzalo Paniagua Javier <[email protected]>
* GlobalizationAndDomains.cs: more tests for bugz.
2005-06-07 Gonzalo Paniagua Javier <[email protected]>
* CultureInfoTest.cs: added test that tries setting the culture to a
neutral culture that is not the InvariantCulture.
2005-05-06 Gonzalo Paniagua Javier <[email protected]>
* CultureInfoTest.cs: added new test.
2005-04-05 Atsushi Enomoto <[email protected]>
* TextInfoTest.cs : added further ToTitleCase() tests.
2004-10-27 Ben Maurer <[email protected]>
* TextInfoTest.cs: Fix up migster's type
2003-12-19 Dick Porter <[email protected]>
* CompareInfoTest.cs: Added some Compare tests
2002-12-21 Nick Drochak <[email protected]>
* all: make tests build and run under nunit2