Commit Graph

5812 Commits

Author SHA1 Message Date
Jonathan Kew
2021cc63fc Bug 1865482 - Make CSS :lang() matching more correctly follow the BCP4647 Extended Filtering algorithm. r=layout-reviewers,emilio
The matching behavior implemented in bug 1857742 did not quite follow the spec,
particularly with regard to language *ranges* (as used in the :lang() pseudo)
that are not themselves valid language *tags*.

This updates the LangTagCompare function to more correctly follow the BCP4647
"Extended Filtering" algorithm, and adjusts the relevant WPT tests (originally
from bug 1857742) to reflect the corrected behavior.

Differential Revision: https://phabricator.services.mozilla.com/D194054
2023-11-20 13:44:06 +00:00
Jonathan Kew
20b8ef07ee Bug 1857742 - patch 2 - Use oxilangtag rather than unic_langid to parse lang tags for nsStyleUtil::LangTagCompare. r=layout-reviewers,dholbert,TYLin
Differential Revision: https://phabricator.services.mozilla.com/D193892
2023-11-18 10:36:01 +00:00
Noemi Erli
af7888da9c Backed out 18 changesets (bug 1863960, bug 1859752) as requested in Bug 1864612 CLOSED TREE
Backed out changeset 814145392ecb (bug 1863960)
Backed out changeset f3564a69fd3f (bug 1863960)
Backed out changeset 35ff07f4726a (bug 1859752)
Backed out changeset b5bf398e3069 (bug 1859752)
Backed out changeset da94a164d4ea (bug 1859752)
Backed out changeset c5eca26f8dff (bug 1859752)
Backed out changeset a2c6ae62fbf7 (bug 1859752)
Backed out changeset 33d0d72d1abe (bug 1859752)
Backed out changeset a42248554021 (bug 1859752)
Backed out changeset 0e7eefda2425 (bug 1859752)
Backed out changeset 1e39572cb1c7 (bug 1859752)
Backed out changeset cbde31ce55a8 (bug 1859752)
Backed out changeset 152860574068 (bug 1859752)
Backed out changeset 16b9813a9c61 (bug 1859752)
Backed out changeset e4495cec8e4b (bug 1859752)
Backed out changeset 1e22b97c98ba (bug 1859752)
Backed out changeset b191be0ec825 (bug 1859752)
Backed out changeset 8b46595de6b5 (bug 1859752)
2023-11-14 17:07:57 +02:00
Jonathan Kew
d523a6de1b Bug 1861254 - Safely bail out of LocaleAtomFromPath if passed a bad hyphenation-file path. r=layout-reviewers,emilio
Differential Revision: https://phabricator.services.mozilla.com/D192934
2023-11-08 10:34:42 +00:00
André Bargull
12f99db68f Bug 1859752 - Part 14: Update expected test results for non-SpiderMonkey code. r=platform-i18n-reviewers,dminor
Update test results for CLDR 44.

Depends on D192738

Differential Revision: https://phabricator.services.mozilla.com/D192739
2023-11-07 08:45:51 +00:00
André Bargull
43d5add4d4 Bug 1859752 - Part 11: Add workarounds for ICU bugs. r=platform-i18n-reviewers,dminor
The fix for <https://unicode-org.atlassian.net/browse/ICU-22547> only handled four
character long language subtags, but language subtags with more than four
characters are still handled incorrectly.

Depends on D192735

Differential Revision: https://phabricator.services.mozilla.com/D192736
2023-11-07 08:45:50 +00:00
André Bargull
3967539454 Bug 1859752 - Part 10: Adjust currency display name fallback handling. r=platform-i18n-reviewers,dminor
The currency display name for multiple locales is now only defined in the "root" locale,
which triggers the `U_USING_DEFAULT_WARNING` case, so we have to change our detection
when no localisation was found.

Depends on D192734

Differential Revision: https://phabricator.services.mozilla.com/D192735
2023-11-07 08:45:50 +00:00
André Bargull
a8fbed0dc7 Bug 1859752 - Part 9: Add new line break classes. r=platform-i18n-reviewers,dminor
Use `CLASS_CHARACTER` because that matches the previous character class for most
characters which are now part of the new character classes.

Depends on D192733

Differential Revision: https://phabricator.services.mozilla.com/D192734
2023-11-07 08:45:49 +00:00
André Bargull
f8ab1fbae7 Bug 1859752 - Part 8: Update Gecko to Unicode 15.1. r=platform-i18n-reviewers,dminor
Update by running `genUnicodePropertyData.pl`, `genSpecialCasingData.pl`, and `cldr-quotes.pl`.

Depends on D192732

Differential Revision: https://phabricator.services.mozilla.com/D192733
2023-11-07 08:45:49 +00:00
André Bargull
8c15f17b8a Bug 1859752 - Part 4: Update language tag mappings. r=platform-i18n-reviewers,dminor
Update language tag mappings by running `./make_intl_data.py langtags`.

Differential Revision: https://phabricator.services.mozilla.com/D192729
2023-11-07 08:45:47 +00:00
André Bargull
ecfc3b211d Bug 1859752 - Part 3: Update in-tree ICU to 74. r=platform-i18n-reviewers,dminor
Differential Revision: https://phabricator.services.mozilla.com/D192728
2023-11-07 08:45:28 +00:00
André Bargull
564c2c5d9a Bug 1859752 - Part 2: Update ICU patch files. r=platform-i18n-reviewers,dminor
Update patch file to apply cleanly on ICU 74.

Depends on D192708

Differential Revision: https://phabricator.services.mozilla.com/D192709
2023-11-07 08:45:28 +00:00
André Bargull
70afc78a1f Bug 1859752 - Part 1: Add or remove ICU patch files. r=platform-i18n-reviewers,dminor
Remove files for patches which have been integrated into upstream and add
new patch files to fix regressions in ICU 74.

Differential Revision: https://phabricator.services.mozilla.com/D192708
2023-11-07 08:45:27 +00:00
André Bargull
448ca7b4bb Bug 1856291 - Part 1: Increase limit for minimum time zone offset. r=dminor
Add a workaround for <https://unicode-org.atlassian.net/browse/ICU-22526>.

Differential Revision: https://phabricator.services.mozilla.com/D189744
2023-11-05 12:27:07 +00:00
André Bargull
39f07f7d01 Bug 1856290: Handle time zone offset skeletons in DateIntervalFormat. r=dminor
Add various workarounds for ICU-20548 to improve the output of the "timeZoneName"
option when formatting date-time ranges.

Changes to "intl/icu/source/i18n/dtitvinf.cpp":

The existing ICU code was already changing `LOW_Z` to `LOW_V` when searching
for the best-fit pattern, but it was missing support for `CAP_O`. (The other
time zone name skeleton characters aren't currently supported in ECMA-402, so
I didn't handle them here.)


Changes to "intl/icu/source/i18n/dtitvfmt.cpp":

In `DateIntervalFormat::getDateTimeSkeleton()`, handle `CAP_O` similar to the
existing code for `LOW_Z` and `LOW_V`. And also keep the original number of
skeleton characters in `normalizedTimeSkeleton`.

In `DateIntervalFormat::adjustFieldWidth()`, copy the field width information
for `LOW_V` into `LOW_Z` resp. `CAP_O` because `LOW_V` is replaced in the
resolved pattern with `LOW_Z` resp. `CAP_O`.

Differential Revision: https://phabricator.services.mozilla.com/D189743
2023-11-05 12:27:06 +00:00
André Bargull
16abe6be85 Bug 1855749 - Part 3: Correctly handle ignorePunctuation option. r=dminor
The default value for `GetOption` in `InitializeCollator` is changed from `false`
to `undefined` to avoid having to `intl_isIgnorePunctuation` in the constructor
function.

The corresponding spec PR is <https://github.com/tc39/ecma402/pull/833>, but
our behaviour is already not strictly spec-compliant as described in
<https://github.com/tc39/ecma402/issues/832>, so it seems reasonable to simply
implement the spec PR ahead of time. (We don't want to "fix" our implementation
to strictly follow the spec without the PR, because that could be web-incompatible
resp. at least disruptive for Thai users, which currently already get the expected
behaviour where punctuation characters are ignored in Thai.)

Differential Revision: https://phabricator.services.mozilla.com/D189545
2023-11-05 12:27:05 +00:00
André Bargull
76217ccc30 Bug 1855749 - Part 1: Add intl::Collator::GetIgnorePunctuation(). r=dminor
Add `Collator::GetIgnorePunctuation()` similar to the existing
`Collator::GetCaseFirst()` function.

Drive-by change:
- Remove unnecessary macro with an inline function.

Differential Revision: https://phabricator.services.mozilla.com/D189543
2023-11-05 12:27:04 +00:00
André Bargull
113bfe78cc Bug 1855748: Handle related year and year names when computing the resolved components. r=dminor
Map both "r" (related Gregorian year) and "U" (cyclic year name) to a resolved numeric year.

From <https://github.com/tc39/ecma402/issues/816#issuecomment-1667481214>:
> Per 11.2.3 Internal slots, "{relatedYear}" can only appear in a pattern string
> when the format record has a [[year]] field. And if a [[year]] field is present,
> CreateDateTimeFormat will set the Intl.DateTimeFormat's [[Year]] field to the
> format records [[year]] value. Intl.DateTimeFormat.prototype.resolvedOptions
> should then return the value of the [[Year]] internal slot.

Fixes <https://github.com/tc39/ecma402/issues/816>.

Differential Revision: https://phabricator.services.mozilla.com/D189542
2023-11-05 12:27:04 +00:00
André Bargull
c9f06ab4c4 Bug 1855747: Raise fractional digits limit to 100. r=dminor
Implements the changes from <https://github.com/tc39/ecma402/pull/786>.

Drive-by change:
Enable another test which no longer fails.

Differential Revision: https://phabricator.services.mozilla.com/D189541
2023-11-05 12:27:03 +00:00
Makoto Kato
37720e005a Bug 1719537 - Add LineBreaker API benchmarks with legacy / new segmenter. r=TYLin
Differential Revision: https://phabricator.services.mozilla.com/D192547
2023-11-04 04:46:52 +00:00
Tawah Peggy
533c58ddf0 Bug 1861622 - Simplify the code by removing some 'else' r=jfkthame DONTBUILD
Differential Revision: https://phabricator.services.mozilla.com/D192343
2023-11-01 08:43:57 +00:00
André Bargull
a807279eb6 Bug 1855986: Don't attempt to use in-tree double-conversion for ICU standalone builds. r=sergesanspaille,glandium
Updates the ICU patch "double-conversion.diff" to only use the in-tree copy of
double-conversion when building Firefox/SpiderMonkey. Standalone ICU builds
instead use the ICU copy of double-conversion. Standalone ICU builds happen
when running "intl/icu_sources_data.py" and when converting the in-tree
little-endian ICU data file to the big-endian format.

`JS_HAS_INTL_API` is used to detect how ICU is built: When building ICU as part
of Firefox/SpiderMonkey, `JS_HAS_INTL_API` is guaranteed to be set, whereas in
standalone ICU builds `JS_HAS_INTL_API` isn't defined.

After applying the updated "double-conversion.diff" patch, ICU was reimported
to restore the previously deleted double-conversion sources files from ICU.

Depends on D190791

Differential Revision: https://phabricator.services.mozilla.com/D190792
2023-10-20 15:21:43 +00:00
André Bargull
c35177d9d3 Bug 1855981: Add missing patch file entry to update-icu.sh. r=hsivonen
Differential Revision: https://phabricator.services.mozilla.com/D190791
2023-10-20 15:21:42 +00:00
Emilio Cobos Álvarez
4eb3dda25e Bug 1859427 - Cache number parsers and formatters per language. r=smaug
This makes number localization cheaper / halves the time in the
microbenchmark.

Change the content-language handling to use atoms. This exposes some
interesting inconsistencies but I tried not to change behavior there.

Differential Revision: https://phabricator.services.mozilla.com/D191174
2023-10-17 11:01:13 +00:00
Sylvestre Ledru
d0f6c7fc66 Bug 1856795 - Remove redundant member init r=emilio
Done with:
./mach static-analysis check --checks="-*, readability-redundant-member-init" --fix .

https://clang.llvm.org/extra/clang-tidy/checks/readability/redundant-member-init.html

Differential Revision: https://phabricator.services.mozilla.com/D190002
2023-10-15 15:29:02 +00:00
Makoto Kato
66ef209e21 Bug 1856428 - MacOS Sonoma doesn't return correct timezone. r=anba
Import from https://unicode-org.atlassian.net/browse/ICU-22541
(https://github.com/unicode-org/icu/pull/2669).

Differential Revision: https://phabricator.services.mozilla.com/D191013
2023-10-15 10:43:10 +00:00
Cristina Horotan
6a90f5eded Backed out 2 changesets (bug 1856795) for causing build bustage at BasicEvents.h CLOSED TREE
Backed out changeset 1d98b028923a (bug 1856795)
Backed out changeset eae2ac93e17c (bug 1856795)
2023-10-14 21:42:14 +03:00
Sylvestre Ledru
cc8a0ee742 Bug 1856795 - Remove redundant member init r=emilio
Done with:
./mach static-analysis check --checks="-*, readability-redundant-member-init" --fix .

https://clang.llvm.org/extra/clang-tidy/checks/readability/redundant-member-init.html

Differential Revision: https://phabricator.services.mozilla.com/D190002
2023-10-14 17:34:26 +00:00
Makoto Kato
64a876087b Bug 1858068 - Cache ICU4XGraphemeClusterSegmenter. r=TYLin
Since `CountGraphemeClusters` is called from PresShell::DoFlushLayout when
using counter style. To optimize this, I would like to cache
`ICU4XGraphemeClusterSegmenter`.

Differential Revision: https://phabricator.services.mozilla.com/D190750
2023-10-13 12:49:08 +00:00
Natalia Csoregi
ec862c541a Backed out changeset 2d9717f5ca10 (bug 1858068) for causing bustage on Segmenter.cpp. CLOSED TREE 2023-10-13 12:15:55 +03:00
Makoto Kato
1c18dce421 Bug 1858068 - Cache ICU4XGraphemeClusterSegmenter. r=TYLin
Since `CountGraphemeClusters` is called from PresShell::DoFlushLayout when
using counter style. To optimize this, I would like to cache
`ICU4XGraphemeClusterSegmenter`.

Differential Revision: https://phabricator.services.mozilla.com/D190750
2023-10-13 08:41:42 +00:00
Tom Marble
0cb91fe33e Bug 1855300 - convert .ini manifests to .toml: batch 11 remaining **/mochitest.ini r=jmaher,geckoview-reviewers,extension-reviewers,valentin,credential-management-reviewers,cookie-reviewers,sgalich,m_kato,robwu
Differential Revision: https://phabricator.services.mozilla.com/D190515
2023-10-13 02:32:24 +00:00
Makoto Kato
15690ec8ce Bug 1848282 - Part 2. FindWord adds an option whether segment is punctuation. r=TYLin
This fix adds stop at punctuation mode for FindWord for a11y.

Also, IS_ASCII is conflict with nsUnicharUtils.h's, so I rename it.

Differential Revision: https://phabricator.services.mozilla.com/D188785
2023-10-03 10:43:17 +00:00
Makoto Kato
9dde11abbe Bug 1848282 - Part 1. Move IsPunctuation to nsUnicharUtils. r=TYLin
Remove duplication code.

Differential Revision: https://phabricator.services.mozilla.com/D188784
2023-10-03 10:43:17 +00:00
Sylvestre Ledru
4c29c24344 Bug 1856530 - run file-whitespace on toml files r=linter-reviewers,Standard8
Differential Revision: https://phabricator.services.mozilla.com/D189896
2023-10-03 09:22:18 +00:00
Jan Varga
c458754592 Bug 1855134 - Move existing MOZ_TRY macros to a dedicated header file; r=glandium
Differential Revision: https://phabricator.services.mozilla.com/D189226
2023-09-28 20:46:07 +00:00
Natalia Csoregi
a03097876a Backed out 3 changesets (bug 1848282) for causing gv-junit failures. CLOSED TREE
Backed out changeset d1fc71cc3817 (bug 1848282)
Backed out changeset 3446c1367bfd (bug 1848282)
Backed out changeset c800ae9f7943 (bug 1848282)
2023-09-28 08:03:34 +03:00
Makoto Kato
c88a7388e1 Bug 1848282 - Part 2. FindWord adds an option whether segment is punctuation. r=TYLin
This fix adds stop at punctuation mode for FindWord for a11y.

Also, IS_ASCII is conflict with nsUnicharUtils.h's, so I rename it.

Differential Revision: https://phabricator.services.mozilla.com/D188785
2023-09-28 04:11:05 +00:00
Makoto Kato
750893b128 Bug 1848282 - Part 1. Move IsPunctuation to nsUnicharUtils. r=TYLin
Remove duplication code.

Differential Revision: https://phabricator.services.mozilla.com/D188784
2023-09-28 04:11:05 +00:00
Bryan Olsson
fd6a7d6aea Bug 1854018 - How to Test Migration Recipes documentation update, r=flod DONTBUILD
Differential Revision: https://phabricator.services.mozilla.com/D188641
2023-09-22 06:22:30 +00:00
Chris Peterson
cc722739c5 Bug 1853612 - Specify argument type for diplomat::span alias template to fix C++20 compilation error. r=m_kato
When I try to compile mozilla-central as C++20, clang reports the following compilation error in code added in bug 1851105:

intl/lwbrk/WordBreaker.cpp:123:9: error: alias template 'span' requires template arguments; argument deduction only allowed for class templates
        diplomat::span((const uint16_t*)aText.BeginReading(), aText.Length()));
        ^
third_party/rust/icu_capi/cpp/include/diplomat_runtime.hpp:131:19: note: template is declared here
template<class T> using span = std::span<T>;
                  ^

Differential Revision: https://phabricator.services.mozilla.com/D188428
2023-09-20 00:07:24 +00:00
serge-sans-paille
ee24dfd589 Bug 1852210 - Remove duplicate double_conversion sources r=dminor
The original sources were compiled... three times.
We got rid of the chromium duplicate in Bug 1850328 and we remove the
last duplicate with this patch.

Differential Revision: https://phabricator.services.mozilla.com/D187753
2023-09-15 08:27:43 +00:00
Cosmin Sabou
77fe6109bb Backed out changeset 62742440d009 (bug 1852210) for causing SM bustages on number_decimalquantity.cpp. CLOSED TREE 2023-09-15 10:38:19 +03:00
serge-sans-paille
3030c5ccde Bug 1852210 - Remove duplicate double_conversion sources r=dminor
The original sources were compiled... three times.
We got rid of the chromium duplicate in Bug 1850328 and we remove the
last duplicate with this patch.

Differential Revision: https://phabricator.services.mozilla.com/D187753
2023-09-15 06:57:17 +00:00
Tom Marble
9e40a719f2 Bug 1850545 - convert .ini manifests to .toml: batch 3 chrome.ini (bis) r=jmaher,geckoview-reviewers,extension-reviewers,credential-management-reviewers,devtools-reviewers,fxview-reviewers,places-reviewers,profiler-reviewers,m_kato,mak,mtigley
Differential Revision: https://phabricator.services.mozilla.com/D187597
2023-09-13 18:28:32 +00:00
Mark Banner
07a6921e17 Bug 1851797 - Remove nsIScriptableUnicodeConverter convertToByteArray and convertToInputStream. r=jfkthame,perftest-reviewers,application-update-reviewers,AlexandruIonescu,bytesized,hsivonen
Differential Revision: https://phabricator.services.mozilla.com/D187548
2023-09-13 08:38:19 +00:00
Makoto Kato
1e6318b7bb Bug 1851323 - Apply the latest line break rules. r=TYLin
Since some issues such as bug 1851323 are fixed by ICU4X 1.3's data rule file,
I would like to apply it to get feedback in Gecko.

Differential Revision: https://phabricator.services.mozilla.com/D187980
2023-09-13 05:11:41 +00:00
Bryan Olsson
3432a23987 Bug 1852843 - Add Santali (sat) to language names in mozIntl and language pack metadata, r=fluent-reviewers,flod
Differential Revision: https://phabricator.services.mozilla.com/D188062
2023-09-13 05:08:11 +00:00
Makoto Kato
ef2a0ed90f Bug 1851105 - Use nsAString as parameter for WordBreaker::FindWord. r=TYLin
It is better to use nsAString rather than using pointer directly.

Differential Revision: https://phabricator.services.mozilla.com/D187256
2023-09-08 04:35:38 +00:00
Markus Stange
fa8bfb6bc7 Bug 1851263 - Rename nsIGlobalObject::AsInnerWindow to GetAsInnerWindow because it can return null. r=smaug
This is also the name that was originally suggested in https://bugzilla.mozilla.org/show_bug.cgi?id=218415#c53 .

Differential Revision: https://phabricator.services.mozilla.com/D187340
2023-09-04 04:06:30 +00:00