Ting-Yu Lin 57b867e7ff Bug 1728708 Part 4 - Simplify WordBreaker::Next() and make it recognize the end of text a word break opportunity. r=jfkthame
A UAX29 compatible word breaker (like ICU4C) treat the end of text as a
word break opportunity (rule WB2 [1]), but currently lwbrk word breaker
doesn't.

The motivation of this patch is to make `WordBreaker::Next()` closer to
a UAX29 compatible one (at least for English text), and see if the
callers need to change. This should make the future integration of ICU4X
segmenter easier.

The only caller of WordBreaker::Next() is ClusterIterator's constructor.
This patch shouldn't change its behavior because we've already manually
assigned a word break point at the end of the line when `aContext` is
empty and `aDirection` is -1. This patch generalizes it to all
conditions.

Also, update TestPrintWordWithBreak() so that the result string makes
more sense.

[1] https://www.unicode.org/reports/tr29/#WB2

Differential Revision: https://phabricator.services.mozilla.com/D124304
2021-09-08 04:19:38 +00:00

An explanation of the Firefox Source Code Directory Structure and links to
project pages with documentation can be found at:

    https://firefox-source-docs.mozilla.org/contributing/directory_structure.html

For information on how to build Firefox from the source code and create the patch see:

    https://firefox-source-docs.mozilla.org/contributing/contribution_quickref.html

If you have a question about developing Firefox, and can't find the solution
on https://firefox-source-docs.mozilla.org/, you can try asking your question on Matrix at chat.mozilla.org in `Introduction` (https://chat.mozilla.org/#/room/#introduction:mozilla.org) channel.


Nightly development builds can be downloaded from:

    https://archive.mozilla.org/pub/firefox/nightly/latest-mozilla-central/
            - or -
    https://www.mozilla.org/firefox/channel/desktop/#nightly

Keep in mind that nightly builds, which are used by Firefox developers for
testing, may be buggy.
Description
Read-only Git mirror of the Mercurial gecko repositories at https://hg.mozilla.org. How to contribute: https://firefox-source-docs.mozilla.org/contributing/contribution_quickref.html
Readme 6.1 GiB
Languages
JavaScript 32.3%
C++ 25.4%
HTML 21%
C 10.8%
Python 2.8%
Other 7.2%