Commit Graph

5822 Commits

Author SHA1 Message Date
Makoto Kato
a35ac330ae Bug 1868324 - Add more explicit keyword for ICU4X C++ FFI. r=TYLin
Tier-2 job for static-analysis might be failed due to no explicit keyword.

Differential Revision: https://phabricator.services.mozilla.com/D195531
2023-12-06 09:38:28 +00:00
Ting-Yu Lin
7817dcc4fb Bug 1868454 - Check in icu_capi/Cargo.toml.orig. r=m_kato DONTBUILD
This file shows up after running `update-icu4x.sh`. It is part of the downloaded
`icu_capi` crate. We should check it in for completeness even if it is not used.

Differential Revision: https://phabricator.services.mozilla.com/D195591
2023-12-06 05:06:46 +00:00
Sylvestre Ledru
ef35fa7374 Bug 1519636 - Reformat recent changes to the Google coding style r=emilio,media-playback-reviewers,padenot
clang-format version 17.0.6 (taskcluster-QueHFUviQeyM8Y7r_d7MQQ)

# ignore-this-changeset

Differential Revision: https://phabricator.services.mozilla.com/D195397
2023-12-04 10:09:36 +00:00
Makoto Kato
f099ad2320 Bug 1854031 - Part 8. Update ICU4X document. r=TYLin
Depends on D193883

Differential Revision: https://phabricator.services.mozilla.com/D193886
2023-12-04 09:02:40 +00:00
Makoto Kato
509ad0c2c1 Bug 1854031 - Part 7. Gecko changes for ICU4X 1.4. r=TYLin
Depends on D193882

Differential Revision: https://phabricator.services.mozilla.com/D193883
2023-12-04 09:02:40 +00:00
Makoto Kato
652974aeed Bug 1854031 - Part 3. Import icu_capi and icu_segmenter_data crate in tree. r=TYLin
By update-icu4x.sh script.

Depends on D192900

Differential Revision: https://phabricator.services.mozilla.com/D193880
2023-12-04 09:02:28 +00:00
Makoto Kato
5a8f82efb4 Bug 1854031 - Part 1. Update update-icu4x.sh script to import icu_capi to local and for datagen change. r=TYLin
Although since `icu_capi` uses weak dependency syntax, cargo vendor doesn't
recognize it. So this command will copy unnecessary crates. To avoid it, I
would like to use modified version of icu_capi.

And this has another issue. `icu_capi`'s C++ headers isn't compatible with
clang [*1]. So we need the workaround for it.

ICU4X 1.3 has another change for data provider with `icu_capi`.

From ICU4X 1.3, there are new `icu_*_data` crates to custom data file, instead
of `icu_testdata`. So we have to add each data crate if using `icu_capi`.

*1 https://github.com/llvm/llvm-project/issues/70162

Differential Revision: https://phabricator.services.mozilla.com/D192902
2023-12-04 09:02:27 +00:00
Sylvestre Ledru
d6cea3eede Bug 1519636 - Remove nbsp in the doc r=firefox-source-docs-reviewers,geckoview-reviewers,ahal,m_kato
Differential Revision: https://phabricator.services.mozilla.com/D194782
2023-12-03 11:01:53 +00:00
Joel Maher
9d203b1d46 Bug 1859893 - batch 15 - migrate xpcshell.ini -> .toml. r=aryx,geckoview-reviewers,extension-reviewers,valentin,cookie-reviewers,pbz,owlish,robwu
Differential Revision: https://phabricator.services.mozilla.com/D194384
2023-11-24 14:41:35 +00:00
Eemeli Aro
8650501442 Bug 1854412 - Drop broken legacy links in l10n tutorial. r=bolsson
Differential Revision: https://phabricator.services.mozilla.com/D193530
2023-11-22 21:59:11 +00:00
Jonathan Kew
2021cc63fc Bug 1865482 - Make CSS :lang() matching more correctly follow the BCP4647 Extended Filtering algorithm. r=layout-reviewers,emilio
The matching behavior implemented in bug 1857742 did not quite follow the spec,
particularly with regard to language *ranges* (as used in the :lang() pseudo)
that are not themselves valid language *tags*.

This updates the LangTagCompare function to more correctly follow the BCP4647
"Extended Filtering" algorithm, and adjusts the relevant WPT tests (originally
from bug 1857742) to reflect the corrected behavior.

Differential Revision: https://phabricator.services.mozilla.com/D194054
2023-11-20 13:44:06 +00:00
Jonathan Kew
20b8ef07ee Bug 1857742 - patch 2 - Use oxilangtag rather than unic_langid to parse lang tags for nsStyleUtil::LangTagCompare. r=layout-reviewers,dholbert,TYLin
Differential Revision: https://phabricator.services.mozilla.com/D193892
2023-11-18 10:36:01 +00:00
Noemi Erli
af7888da9c Backed out 18 changesets (bug 1863960, bug 1859752) as requested in Bug 1864612 CLOSED TREE
Backed out changeset 814145392ecb (bug 1863960)
Backed out changeset f3564a69fd3f (bug 1863960)
Backed out changeset 35ff07f4726a (bug 1859752)
Backed out changeset b5bf398e3069 (bug 1859752)
Backed out changeset da94a164d4ea (bug 1859752)
Backed out changeset c5eca26f8dff (bug 1859752)
Backed out changeset a2c6ae62fbf7 (bug 1859752)
Backed out changeset 33d0d72d1abe (bug 1859752)
Backed out changeset a42248554021 (bug 1859752)
Backed out changeset 0e7eefda2425 (bug 1859752)
Backed out changeset 1e39572cb1c7 (bug 1859752)
Backed out changeset cbde31ce55a8 (bug 1859752)
Backed out changeset 152860574068 (bug 1859752)
Backed out changeset 16b9813a9c61 (bug 1859752)
Backed out changeset e4495cec8e4b (bug 1859752)
Backed out changeset 1e22b97c98ba (bug 1859752)
Backed out changeset b191be0ec825 (bug 1859752)
Backed out changeset 8b46595de6b5 (bug 1859752)
2023-11-14 17:07:57 +02:00
Jonathan Kew
d523a6de1b Bug 1861254 - Safely bail out of LocaleAtomFromPath if passed a bad hyphenation-file path. r=layout-reviewers,emilio
Differential Revision: https://phabricator.services.mozilla.com/D192934
2023-11-08 10:34:42 +00:00
André Bargull
12f99db68f Bug 1859752 - Part 14: Update expected test results for non-SpiderMonkey code. r=platform-i18n-reviewers,dminor
Update test results for CLDR 44.

Depends on D192738

Differential Revision: https://phabricator.services.mozilla.com/D192739
2023-11-07 08:45:51 +00:00
André Bargull
43d5add4d4 Bug 1859752 - Part 11: Add workarounds for ICU bugs. r=platform-i18n-reviewers,dminor
The fix for <https://unicode-org.atlassian.net/browse/ICU-22547> only handled four
character long language subtags, but language subtags with more than four
characters are still handled incorrectly.

Depends on D192735

Differential Revision: https://phabricator.services.mozilla.com/D192736
2023-11-07 08:45:50 +00:00
André Bargull
3967539454 Bug 1859752 - Part 10: Adjust currency display name fallback handling. r=platform-i18n-reviewers,dminor
The currency display name for multiple locales is now only defined in the "root" locale,
which triggers the `U_USING_DEFAULT_WARNING` case, so we have to change our detection
when no localisation was found.

Depends on D192734

Differential Revision: https://phabricator.services.mozilla.com/D192735
2023-11-07 08:45:50 +00:00
André Bargull
a8fbed0dc7 Bug 1859752 - Part 9: Add new line break classes. r=platform-i18n-reviewers,dminor
Use `CLASS_CHARACTER` because that matches the previous character class for most
characters which are now part of the new character classes.

Depends on D192733

Differential Revision: https://phabricator.services.mozilla.com/D192734
2023-11-07 08:45:49 +00:00
André Bargull
f8ab1fbae7 Bug 1859752 - Part 8: Update Gecko to Unicode 15.1. r=platform-i18n-reviewers,dminor
Update by running `genUnicodePropertyData.pl`, `genSpecialCasingData.pl`, and `cldr-quotes.pl`.

Depends on D192732

Differential Revision: https://phabricator.services.mozilla.com/D192733
2023-11-07 08:45:49 +00:00
André Bargull
8c15f17b8a Bug 1859752 - Part 4: Update language tag mappings. r=platform-i18n-reviewers,dminor
Update language tag mappings by running `./make_intl_data.py langtags`.

Differential Revision: https://phabricator.services.mozilla.com/D192729
2023-11-07 08:45:47 +00:00
André Bargull
ecfc3b211d Bug 1859752 - Part 3: Update in-tree ICU to 74. r=platform-i18n-reviewers,dminor
Differential Revision: https://phabricator.services.mozilla.com/D192728
2023-11-07 08:45:28 +00:00
André Bargull
564c2c5d9a Bug 1859752 - Part 2: Update ICU patch files. r=platform-i18n-reviewers,dminor
Update patch file to apply cleanly on ICU 74.

Depends on D192708

Differential Revision: https://phabricator.services.mozilla.com/D192709
2023-11-07 08:45:28 +00:00
André Bargull
70afc78a1f Bug 1859752 - Part 1: Add or remove ICU patch files. r=platform-i18n-reviewers,dminor
Remove files for patches which have been integrated into upstream and add
new patch files to fix regressions in ICU 74.

Differential Revision: https://phabricator.services.mozilla.com/D192708
2023-11-07 08:45:27 +00:00
André Bargull
448ca7b4bb Bug 1856291 - Part 1: Increase limit for minimum time zone offset. r=dminor
Add a workaround for <https://unicode-org.atlassian.net/browse/ICU-22526>.

Differential Revision: https://phabricator.services.mozilla.com/D189744
2023-11-05 12:27:07 +00:00
André Bargull
39f07f7d01 Bug 1856290: Handle time zone offset skeletons in DateIntervalFormat. r=dminor
Add various workarounds for ICU-20548 to improve the output of the "timeZoneName"
option when formatting date-time ranges.

Changes to "intl/icu/source/i18n/dtitvinf.cpp":

The existing ICU code was already changing `LOW_Z` to `LOW_V` when searching
for the best-fit pattern, but it was missing support for `CAP_O`. (The other
time zone name skeleton characters aren't currently supported in ECMA-402, so
I didn't handle them here.)


Changes to "intl/icu/source/i18n/dtitvfmt.cpp":

In `DateIntervalFormat::getDateTimeSkeleton()`, handle `CAP_O` similar to the
existing code for `LOW_Z` and `LOW_V`. And also keep the original number of
skeleton characters in `normalizedTimeSkeleton`.

In `DateIntervalFormat::adjustFieldWidth()`, copy the field width information
for `LOW_V` into `LOW_Z` resp. `CAP_O` because `LOW_V` is replaced in the
resolved pattern with `LOW_Z` resp. `CAP_O`.

Differential Revision: https://phabricator.services.mozilla.com/D189743
2023-11-05 12:27:06 +00:00
André Bargull
16abe6be85 Bug 1855749 - Part 3: Correctly handle ignorePunctuation option. r=dminor
The default value for `GetOption` in `InitializeCollator` is changed from `false`
to `undefined` to avoid having to `intl_isIgnorePunctuation` in the constructor
function.

The corresponding spec PR is <https://github.com/tc39/ecma402/pull/833>, but
our behaviour is already not strictly spec-compliant as described in
<https://github.com/tc39/ecma402/issues/832>, so it seems reasonable to simply
implement the spec PR ahead of time. (We don't want to "fix" our implementation
to strictly follow the spec without the PR, because that could be web-incompatible
resp. at least disruptive for Thai users, which currently already get the expected
behaviour where punctuation characters are ignored in Thai.)

Differential Revision: https://phabricator.services.mozilla.com/D189545
2023-11-05 12:27:05 +00:00
André Bargull
76217ccc30 Bug 1855749 - Part 1: Add intl::Collator::GetIgnorePunctuation(). r=dminor
Add `Collator::GetIgnorePunctuation()` similar to the existing
`Collator::GetCaseFirst()` function.

Drive-by change:
- Remove unnecessary macro with an inline function.

Differential Revision: https://phabricator.services.mozilla.com/D189543
2023-11-05 12:27:04 +00:00
André Bargull
113bfe78cc Bug 1855748: Handle related year and year names when computing the resolved components. r=dminor
Map both "r" (related Gregorian year) and "U" (cyclic year name) to a resolved numeric year.

From <https://github.com/tc39/ecma402/issues/816#issuecomment-1667481214>:
> Per 11.2.3 Internal slots, "{relatedYear}" can only appear in a pattern string
> when the format record has a [[year]] field. And if a [[year]] field is present,
> CreateDateTimeFormat will set the Intl.DateTimeFormat's [[Year]] field to the
> format records [[year]] value. Intl.DateTimeFormat.prototype.resolvedOptions
> should then return the value of the [[Year]] internal slot.

Fixes <https://github.com/tc39/ecma402/issues/816>.

Differential Revision: https://phabricator.services.mozilla.com/D189542
2023-11-05 12:27:04 +00:00
André Bargull
c9f06ab4c4 Bug 1855747: Raise fractional digits limit to 100. r=dminor
Implements the changes from <https://github.com/tc39/ecma402/pull/786>.

Drive-by change:
Enable another test which no longer fails.

Differential Revision: https://phabricator.services.mozilla.com/D189541
2023-11-05 12:27:03 +00:00
Makoto Kato
37720e005a Bug 1719537 - Add LineBreaker API benchmarks with legacy / new segmenter. r=TYLin
Differential Revision: https://phabricator.services.mozilla.com/D192547
2023-11-04 04:46:52 +00:00
Tawah Peggy
533c58ddf0 Bug 1861622 - Simplify the code by removing some 'else' r=jfkthame DONTBUILD
Differential Revision: https://phabricator.services.mozilla.com/D192343
2023-11-01 08:43:57 +00:00
André Bargull
a807279eb6 Bug 1855986: Don't attempt to use in-tree double-conversion for ICU standalone builds. r=sergesanspaille,glandium
Updates the ICU patch "double-conversion.diff" to only use the in-tree copy of
double-conversion when building Firefox/SpiderMonkey. Standalone ICU builds
instead use the ICU copy of double-conversion. Standalone ICU builds happen
when running "intl/icu_sources_data.py" and when converting the in-tree
little-endian ICU data file to the big-endian format.

`JS_HAS_INTL_API` is used to detect how ICU is built: When building ICU as part
of Firefox/SpiderMonkey, `JS_HAS_INTL_API` is guaranteed to be set, whereas in
standalone ICU builds `JS_HAS_INTL_API` isn't defined.

After applying the updated "double-conversion.diff" patch, ICU was reimported
to restore the previously deleted double-conversion sources files from ICU.

Depends on D190791

Differential Revision: https://phabricator.services.mozilla.com/D190792
2023-10-20 15:21:43 +00:00
André Bargull
c35177d9d3 Bug 1855981: Add missing patch file entry to update-icu.sh. r=hsivonen
Differential Revision: https://phabricator.services.mozilla.com/D190791
2023-10-20 15:21:42 +00:00
Emilio Cobos Álvarez
4eb3dda25e Bug 1859427 - Cache number parsers and formatters per language. r=smaug
This makes number localization cheaper / halves the time in the
microbenchmark.

Change the content-language handling to use atoms. This exposes some
interesting inconsistencies but I tried not to change behavior there.

Differential Revision: https://phabricator.services.mozilla.com/D191174
2023-10-17 11:01:13 +00:00
Sylvestre Ledru
d0f6c7fc66 Bug 1856795 - Remove redundant member init r=emilio
Done with:
./mach static-analysis check --checks="-*, readability-redundant-member-init" --fix .

https://clang.llvm.org/extra/clang-tidy/checks/readability/redundant-member-init.html

Differential Revision: https://phabricator.services.mozilla.com/D190002
2023-10-15 15:29:02 +00:00
Makoto Kato
66ef209e21 Bug 1856428 - MacOS Sonoma doesn't return correct timezone. r=anba
Import from https://unicode-org.atlassian.net/browse/ICU-22541
(https://github.com/unicode-org/icu/pull/2669).

Differential Revision: https://phabricator.services.mozilla.com/D191013
2023-10-15 10:43:10 +00:00
Cristina Horotan
6a90f5eded Backed out 2 changesets (bug 1856795) for causing build bustage at BasicEvents.h CLOSED TREE
Backed out changeset 1d98b028923a (bug 1856795)
Backed out changeset eae2ac93e17c (bug 1856795)
2023-10-14 21:42:14 +03:00
Sylvestre Ledru
cc8a0ee742 Bug 1856795 - Remove redundant member init r=emilio
Done with:
./mach static-analysis check --checks="-*, readability-redundant-member-init" --fix .

https://clang.llvm.org/extra/clang-tidy/checks/readability/redundant-member-init.html

Differential Revision: https://phabricator.services.mozilla.com/D190002
2023-10-14 17:34:26 +00:00
Makoto Kato
64a876087b Bug 1858068 - Cache ICU4XGraphemeClusterSegmenter. r=TYLin
Since `CountGraphemeClusters` is called from PresShell::DoFlushLayout when
using counter style. To optimize this, I would like to cache
`ICU4XGraphemeClusterSegmenter`.

Differential Revision: https://phabricator.services.mozilla.com/D190750
2023-10-13 12:49:08 +00:00
Natalia Csoregi
ec862c541a Backed out changeset 2d9717f5ca10 (bug 1858068) for causing bustage on Segmenter.cpp. CLOSED TREE 2023-10-13 12:15:55 +03:00
Makoto Kato
1c18dce421 Bug 1858068 - Cache ICU4XGraphemeClusterSegmenter. r=TYLin
Since `CountGraphemeClusters` is called from PresShell::DoFlushLayout when
using counter style. To optimize this, I would like to cache
`ICU4XGraphemeClusterSegmenter`.

Differential Revision: https://phabricator.services.mozilla.com/D190750
2023-10-13 08:41:42 +00:00
Tom Marble
0cb91fe33e Bug 1855300 - convert .ini manifests to .toml: batch 11 remaining **/mochitest.ini r=jmaher,geckoview-reviewers,extension-reviewers,valentin,credential-management-reviewers,cookie-reviewers,sgalich,m_kato,robwu
Differential Revision: https://phabricator.services.mozilla.com/D190515
2023-10-13 02:32:24 +00:00
Makoto Kato
15690ec8ce Bug 1848282 - Part 2. FindWord adds an option whether segment is punctuation. r=TYLin
This fix adds stop at punctuation mode for FindWord for a11y.

Also, IS_ASCII is conflict with nsUnicharUtils.h's, so I rename it.

Differential Revision: https://phabricator.services.mozilla.com/D188785
2023-10-03 10:43:17 +00:00
Makoto Kato
9dde11abbe Bug 1848282 - Part 1. Move IsPunctuation to nsUnicharUtils. r=TYLin
Remove duplication code.

Differential Revision: https://phabricator.services.mozilla.com/D188784
2023-10-03 10:43:17 +00:00
Sylvestre Ledru
4c29c24344 Bug 1856530 - run file-whitespace on toml files r=linter-reviewers,Standard8
Differential Revision: https://phabricator.services.mozilla.com/D189896
2023-10-03 09:22:18 +00:00
Jan Varga
c458754592 Bug 1855134 - Move existing MOZ_TRY macros to a dedicated header file; r=glandium
Differential Revision: https://phabricator.services.mozilla.com/D189226
2023-09-28 20:46:07 +00:00
Natalia Csoregi
a03097876a Backed out 3 changesets (bug 1848282) for causing gv-junit failures. CLOSED TREE
Backed out changeset d1fc71cc3817 (bug 1848282)
Backed out changeset 3446c1367bfd (bug 1848282)
Backed out changeset c800ae9f7943 (bug 1848282)
2023-09-28 08:03:34 +03:00
Makoto Kato
c88a7388e1 Bug 1848282 - Part 2. FindWord adds an option whether segment is punctuation. r=TYLin
This fix adds stop at punctuation mode for FindWord for a11y.

Also, IS_ASCII is conflict with nsUnicharUtils.h's, so I rename it.

Differential Revision: https://phabricator.services.mozilla.com/D188785
2023-09-28 04:11:05 +00:00
Makoto Kato
750893b128 Bug 1848282 - Part 1. Move IsPunctuation to nsUnicharUtils. r=TYLin
Remove duplication code.

Differential Revision: https://phabricator.services.mozilla.com/D188784
2023-09-28 04:11:05 +00:00
Bryan Olsson
fd6a7d6aea Bug 1854018 - How to Test Migration Recipes documentation update, r=flod DONTBUILD
Differential Revision: https://phabricator.services.mozilla.com/D188641
2023-09-22 06:22:30 +00:00