Nathan Froyd 8d01bc5e88 store smaller slices in unicode data tables ...

Rust's default slices are convenient, but for tables like:

  const f: &'static [(char, &'static [char])]

they take up far too much space.  An element of the above array consumes
24 bytes on 64-bit platforms, and unicode-normalization contains about
6000 such array elements.

A better approach is to manually store a smaller slice type:

  struct Slice {
    offset: u16,
    length: u16,
  }

  const f: &'static [(char, Slice)]

and store the actual character data in a separate array on the side.
The `Slice` structures then point in to this separate array, but at a
much smaller space cost: elements of the modified `f` take up only 8
bytes on 64-bit platforms, which implies a space savings of ~96K on
64-bit platforms.  On some systems, this strategy also eliminates the
necessity of run-time relocations, which can be a further, significant
savings in binary size and runtime cost.

This change is strictly local to the library; it does not affect the
public API.

2017-03-27 16:58:31 -04:00

..

unicode_gen_normtests.py

add Unicode norms test suite; bump to ver 0.0.3

2015-04-15 15:40:06 -04:00

unicode.py

store smaller slices in unicode data tables

2017-03-27 16:58:31 -04:00