Add additional logic to our language negotation to do apply likelySubtags when a direct match is not available.
Currently, if the user specifies the locale with region, and we do not have a direct for that region, we pick all locales for the same language and other regions in no order.
The example of where it returns suboptimal results:
1) Requested locale "en-CA"
2) Available locales ["en-ZA", "en-GB", "en-US"]
3) Negotiated locales ["en-ZA", "en-GB", "en-US"]
This would not happen, if the user requested a generic "de", "en" etc.:
1) Requested locale "en"
2) Available locales ["en-ZA", "en-GB", "en-US"]
3) Negotiated locales ["en-US", "en-ZA", "en-GB"]
because after not finding a direct match, we would use likelySubtags to extend "en" to "en-Latn-US" and then find the priority match in "en-US".
This patch extends this logic to "en-US" or "de-LU" by adding a step which strips the region tag and then applies likelySubtag on the result.
This means that in absence of direct match the following fallbacks would happen:
"de-LU" -> "de-DE"
"es-CL" -> "es-ES"
"en-CA" -> "en-US"
This does not affect languages that use multiple scripts, so ar, sr and zh are not affected.
MozReview-Commit-ID: BR1WrgXSf6a
--HG--
extra : rebase_source : abc205c4f993680ab0cd0c8b8c016543d5462d01
By the time the LocaleService destructor is called, by definition sInstance
has been cleared and its preference observers have been removed, since both
are strong references. In practice, this doesn't seem to cause trouble, but it
does lead to worrying warnings at shutdown in debug builds, which are annoying
red herrings when debugging other issues.
MozReview-Commit-ID: IalOigr2GWN
--HG--
extra : rebase_source : f86a8e81b91c639dedaca85e5bc3b53d54e01645
extra : histedit_source : 643b76de0d29ca4890ac4dd4175259989a63688e
Until Firefox 54, font fallback uses user locale. Example, OS is Windows 10 JA with JA UI, Firefox uses JA font for fallback even if Firefox is US version.
But since we changed some locale code on Firefox 55 development cycle, there were some regressions such as bug 1346674. So we changed fallback locale to system locale (instead of Firefox UX locale) for font selection by bug 1348259.
It isn't good for old compatibility, so we should use user locale for font fallback like Firefox 54. Now we can get user locale by OSPreferences::GetRegionalPrefsLocales, use it for font fallback
MozReview-Commit-ID: 7qwDDeU1ZPt
--HG--
extra : rebase_source : 3a307dd239bea516e117961a944b714bc43a993c
This should be the only callee affected, given
gfxPlatformFontList::GetFontPrefLangFor already uses case-insensitive string
comparison.
This makes it also consistent with the rest of the functions in the file, which
lowercase their inputs as well.
MozReview-Commit-ID: 4fNUBdpayHL
--HG--
extra : rebase_source : d669f5da2a1ed06f6348fc6b86eb379b514ddcf3
Original issue is that Microsoft Dynamics CRM uses invalid lang attribute in <xsl:sort>.
<xsl:sort order="descending"
select="@displayname[$sortColumnName='displayname'] |
@name[$sortColumnName='name'] |
exslt:node-set($FriendlyTypeNames)/types/type[@xmlName=current()/@datatype and $sortColumnName='datatype']"
lang="$languageName"/>
Our XSLT implementation detects "$languageName" as locale name, then use it to nsICollation.
Until Gecko 54 for Windows, even if using invalid locale name for nsICollation, it uses platform locale as fallback. But from 55, we use same implementation as macOS's to use ICU. So this issue occurs. ICU implementation doesn't use fallback locale if it is invalid.
We should use fallback locale if locale is invalid. Most code for fallback locale such as FallbackEncoding uses application locale, so use it.
MozReview-Commit-ID: EKYkZG7Hnz0
--HG--
extra : rebase_source : fec89c67317d7df041f3b237122fb7e20e32fe1b
HasRTLChars() appears in profile of attachment 8848015 after landing the patches
for bug 1391538 because the landing made text in <input> or <textarea> always
treated as non-single byte characters. Therefore, HasRTLChars() is now called
by nsTextFragment::SetTo() a lot in editors.
HasRTLChar() checks if it's in an RTL character for each character until it
becomes true. However, if character is less than the minimum RTL character,
U+0590, it works faster at least for Western languages.
MozReview-Commit-ID: 4YecxQWUcmK
--HG--
extra : rebase_source : 172be670795f0e2155d4bd5044ee37f652367734
These are all easy cases where an nsXPIDLCString local variable is set via
getter_Copies() and then is only used in ways that nsCStrings can also be used
(i.e. no null checks or implicit conversions to |char*|).
In every case the patch trivially replaces the nsXPIDLCString with an
nsCString. (Also, there are a couple of unused nsXPIDLCString variables that
the patch simply removes.)
This patch introduces a new registry for localization resources to replace
ChromeRegistry. It uses asynchronous I/O and iterators to generate
permutations of possible sources of language resources for use in the new
Localization API.
In the initial form the API handles packages resources and has API for
interacting with the AddonsManager which will report language packs.
MozReview-Commit-ID: LfERDYMROh
--HG--
extra : rebase_source : 68a664c2c59a82b4dfcae66542c315a00ddae565
This patch lands the core of the new localization API:
* Parser for the Fluent syntax called "FTL"
* Resolver for the Fluent logic
* MessageContext class which is a central class for storing and operating
on Fluent messages.
MozReview-Commit-ID: E7nKGsLOCJe
--HG--
extra : rebase_source : 893178263744ef6cf6e64c135d28e10a53b61a19
This patch introduces a new registry for localization resources to replace
ChromeRegistry. It uses asynchronous I/O and iterators to generate
permutations of possible sources of language resources for use in the new
Localization API.
In the initial form the API handles packages resources and has API for
interacting with the AddonsManager which will report language packs.
MozReview-Commit-ID: LfERDYMROh
--HG--
extra : rebase_source : a6e5b790142e5afb1ce750478190e5ad87012f8d
This patch lands the core of the new localization API:
* Parser for the Fluent syntax called "FTL"
* Resolver for the Fluent logic
* MessageContext class which is a central class for storing and operating
on Fluent messages.
MozReview-Commit-ID: E7nKGsLOCJe
--HG--
extra : rebase_source : 9bb4c7b155c8b9bd8923534b69f643a6e85bb8af
This removes about 2/3 of the occurrences of nsXPIDLString in the tree. The
places where nsXPIDLStrings are null-checked are replaced with |rv| checks.
The patch also removes a couple of unused declarations from
nsIStringBundle.idl.
Note that nsStringBundle::GetStringFromNameHelper() was merged into
GetStringFromName(), because they both would have had the same signature.
--HG--
extra : rebase_source : ac40bc31c2a4997f2db0bd5069cc008757a2df6d
Introduce a separate API for retrieving locale set from the host environment
used for regional preferences localization.
MozReview-Commit-ID: 3597QstZjS3
--HG--
extra : rebase_source : 2ac30ba2f2c9bd1ed308cfe7be88f6add4f0f5ae
This patch moves the destructor after the constructor, puts the
NS_IMPL_ISUPPORTS line in a more sensible spot, and moves the Get*() functions
before the Format*() functions in order to match the order in
nsIStringBundle.idl.
--HG--
extra : rebase_source : 54250432bd084acd628ab1834bd344687a2f961d
GetStringFromName() enters mReentrantMonitor(), and the other Get*() functions
go through GetStringFromName(). But none of the Format*() functions enter the
monitor.
This patch moves some repeated code from GetStringFromName() and
FormatStringFromName() into GetStringFromNameHelper(), including the monitor
entry. All the Get*() and Format*() functions now use
GetStringFromNameHelper(), which means they now all enter the monitor.
--HG--
extra : rebase_source : 5d4c1931fb374d7aa10d6851f5ad8c0c774ffa79
Similar to the previous patch, this patch changes GetStringFromID() to call
GetStringFromName(), but this time in nsExtensibleStringBundle.
--HG--
extra : rebase_source : 38b12f5f0010396b2606969fb0c7baef96debe21
GetStringFromIDHelper() is very similar to GetSringFromNameHelper(). This patch
removes the former, and changes GetStringFromID() to call GetStringFromName(),
similar to how FormatStringFromID() calls FormatStringFromName().
--HG--
extra : rebase_source : ca1d6abffde106d5409b71c8c181fd604fc974ce
nsStringBundle has two public methods GetStringFrom{ID,Name}() and two
very similar helper methods with the same name. It's easy to get confused
between the two -- indeed, it may be the reason for the data race that this bug
is about -- so this patch renames the helper functions by adding a "Helper"
suffix.
--HG--
extra : rebase_source : 75a15e91c50c4b50b2984377980785b64fcf8852
Most of the names passed to nsIStringBundle::{Get,Format}StringFromUTF8Name
have one of the two following forms:
- a 16-bit C string literal, which is then converted to an 8-bit string in
order for the lookup to occur;
- an 8-bit C string literal converted to a 16-bit string, which is then
converted back to an 8-bit string in order for the lookup to occur.
This patch introduces and uses alternative methods that can take an 8-bit C
string literal, which requires changing some signatures in other methods and
functions. It replaces all C++ uses of the old methods.
The patch also changes the existing {Get,Format}StringFromName() methods so
they take an AUTF8String argument for the name instead of a wstring, because
that's nicer for JS code.
Even though there is a method for C++ code and a different one for JS code,
|binaryname| is used so that the existing method names can be used for the
common case in both languages.
The change reduces the number of NS_ConvertUTF8toUTF16 and
NS_ConvertUTF16toUTF8 conversions while running Speedometer v2 from ~270,000 to
~160,000. (Most of these conversions involved the string
"deprecatedReferrerDirective" in nsCSPParser.cpp.)
--HG--
extra : rebase_source : 3bee57a501035f76a81230d95186f8c3f460ff8e
In scenarios where users use OS in one locale set and the App in another,
users should be able to chose which locale set to follow for regional
preferences. This method should be used over getAppLocales for all cases
where the locales are used to format regional preferences related
items like calendars, dates, units etc.
MozReview-Commit-ID: OOBYIZCKXE
--HG--
extra : rebase_source : be1b96728ff85b6c3e96a2a08db737c3e247cbbc
It's silly to use prmem.h within Firefox code given that in our configuration
its functions are just wrappers for malloc() et al. (Indeed, in some places we
mix PR_Malloc() with free(), or malloc() with PR_Free().)
This patch removes all uses, except for the places where we need to use
PR_Free() to free something allocated by another NSPR function; in those cases
I've added a comment explaining which function did the allocation.
--HG--
extra : rebase_source : 0f781bca68b5bf3c4c191e09e277dfc8becffa09
All the instances are converted as follows.
- nsSubstring --> nsAString
- nsCSubstring --> nsACString
--HG--
extra : rebase_source : cfd2238c52e3cb4d13e3bd5ddb80ba6584ab6d91
All the instances are converted as follows.
- nsAFlatString --> nsString
- nsAFlatCString --> nsCString
--HG--
extra : rebase_source : b37350642c58a85a08363df2e7c610873faa6e41
Move the class definition out of an unnamed namespace because it is in a header files included in multiple source files. clang treats the header file's class declaration in an unnamed namespace as distinct in each compilation unit and emits -Wunused-member-function warnings because the member function release() is used in some compilation units but not others.
intl/unicharutil/util/ICUUtils.h:25:17 [-Wunused-member-function] unused member function 'empty'
intl/unicharutil/util/ICUUtils.h:26:17 [-Wunused-member-function] unused member function 'release'
MozReview-Commit-ID: 7AgzzzRgni4
--HG--
extra : source : aeed73b6225ba4834a25de0ec9e23592de25c871
extra : intermediate-source : d0788a9eba9621e04f6be30771acac5707d4a7e1
Code search indicates that it is used neither internally nor in any
addon. It should be safe to remove.
MozReview-Commit-ID: K3dBoydIwGk
--HG--
extra : rebase_source : 984aee342b0006af94eec1cd7fce123bccea4bd7
Code search indicates that this is not used internally, and the only
addon uses this is Breadcrumbs, which hasn't been updated since 2009,
thus it is already not compatible with current Firefox.
MozReview-Commit-ID: 1lUs7ceaaE1
--HG--
extra : rebase_source : 87060c33291ba415c6c70767c132c4a5b74d29fb
Since our platform support is Windows 7+, we can replace nsWin32Locale::GetXPLocale with LCIDToLocaleName.
LCIDToLocaleName has LOCALE_SYSTEM_DEFAULT and LOCALE_USER_DEFAULT, so we should it simply.
MozReview-Commit-ID: 7JhhXayIK7S
--HG--
extra : rebase_source : d6579e94b8471c73d03c1ac08cdb4b8ab072bed5
LocaleService serves two main functions. It is a central place for all code in the
engine to learn about locales, but it also does the language negotiation and selection.
The former is relevant in all processes, but the latter should only be performed
by the "main" process. In case of current Desktop Firefox, the parent process
is the one performing all the language negotiation, and content processes should
operate in the "client" mode.
In Fennec, there's a Java app on top of Gecko which should work as a "server"
and then all processes, including parent process of Gecko is merely a "client" for that.
This refactor finalizes this duality making it easily configurable to define in
which mode a given LocaleService operates.
The server-client model allows all clients to stay in sync with the server,
but operate transparently for all callers just returning the right values.
In order to initialize LocaleService in the client mode in child process with the
right locales I'm adding the list of app locales to the XPCOMInitData,
and then fire LocaleService::SetAppLocales in the child process initialization.
In order to keep the list up to date, I'm adding intl:app-locales-changed to
the list of observed topics, and when triggered, I send the updated list
to the child process, which updates LocaleService::SetAppLocales with the new
list.
MozReview-Commit-ID: K9X6berF3IO
--HG--
extra : rebase_source : ca5e502d064023fddfd63fe6fe5eccefce8dee52
clang's -Wcomma warning warns about suspicious use of the comma operator such as between two statements.
intl/uconv/util/nsUCSupport.cpp:552:34 [-Wcomma] possible misuse of comma operator here
MozReview-Commit-ID: Emk05qDJHbm
--HG--
extra : rebase_source : cfc7b16894f584f1e444e3296ab15c8d46f09698
extra : source : d61747509865b526772da48f94f9e1ba0548e6d3
StringBundle caches bundles, so when language chain changes we should
flush the cache to enable new strings to be loaded.
This also affects localized prefs like intl.accept_languages.
Then in HttpHandler we have to mark the value as dirty so that next
time it's called it actually recalculates using flushed string bundle
with the new locale.
MozReview-Commit-ID: DKWEDUli4yH
--HG--
extra : rebase_source : 75ecc4204deca066d7492d1494492a91685f36be
I'm adding a helper function mozILocaleService::GetRequestedLocale to simplify
most of the callsites that are looking for the first of the requested locales.
In most cases, I'm just matching the behavior of the code with reusing
LocaleService API instead of direct manipulation on the prefs.
That includes how I handle error case scenarios.
In case of sdk/l10n/locale.js I am reusing LocaleService heuristics over
the custom one from the file since the ones in LocaleService are just
more correct and unified accross the whole platform.
In case of FallbackEncoding I have to turn it into a nsIObserver to listen
to intl:requested-locales-changed.
MozReview-Commit-ID: 7rOr2CovLK
--HG--
extra : rebase_source : 883a91b249b6953b7872bfb9a8851e8be7257c7b
I'm adding a helper function mozILocaleService::GetRequestedLocale to simplify
most of the callsites that are looking for the first of the requested locales.
In most cases, I'm just matching the behavior of the code with reusing
LocaleService API instead of direct manipulation on the prefs.
That includes how I handle error case scenarios.
In case of sdk/l10n/locale.js I am reusing LocaleService heuristics over
the custom one from the file since the ones in LocaleService are just
more correct and unified accross the whole platform.
In case of FallbackEncoding I have to turn it into a nsIObserver to listen
to intl:requested-locales-changed.
MozReview-Commit-ID: 7rOr2CovLK
--HG--
extra : rebase_source : 2f166cf1746f389a035f7cf557edcadeacb10fa0
CLOSED TREE
Backed out changeset 1cce9249b4a2 (bug 1351200)
Backed out changeset 792fb55377f5 (bug 1351200)
Backed out changeset 767bee78574c (bug 1351200)
After landing bug bug 943283, no one uses nsIPlatformCharset::GetDefaultCharsetForLocale. Also, nsIPlatformCharset isn't scriptable interface, so it is unnecessary now.
MozReview-Commit-ID: Giy68AS9yK2
--HG--
extra : rebase_source : a0ae86c5472e1ed74b41c2f6af5cdace353b335d
This patch fixes a regression that we hit on Fennec when we handed over control
over language negotiation to LocaleService in bug 1347306.
The problem is that we currently do not have good Android bindings for
OSPreferences::GetSystemLocale and in result we could not negotiate
languages between user-selected OS locales and available locales.
The temporary solution implemented here binds android OS preferences to
`intl.locale.os` pref which is set on Locale:OS event in browser.js.
In LocaleService we add a listener on the pref to renegotiate languages
when OS locale changes.
MozReview-Commit-ID: KTqjYSzLNNO
--HG--
extra : rebase_source : 61b692e81ce67cdb15a7962e41ebe78164dbe1aa
Beta and release channel of Android still turn off ICU. So we should add UTF-8 version of nsICollation for fallback.
Some people want to keep --enale-intl-api=no for faster build on developer environment, so even if non-Android, it will be required.
MozReview-Commit-ID: 3RcjqoVip1W
--HG--
rename : intl/locale/unix/nsCollationUnix.cpp => intl/locale/nsCollationAndroid.cpp
extra : rebase_source : 6844f32efe8c37b020f30bc745d4d4d8e95c4158
nsICollation for macOS version uses ICU, so we should use it on all platform since nightly and aurora can use ICU on all platform.
MozReview-Commit-ID: 4nTzDCjOQXJ
--HG--
rename : intl/locale/tests/unit/test_collation_mac_icu.js => intl/locale/tests/unit/test_collation.js
extra : rebase_source : 3b47f9a521e79be929bb4c39d0c68b9438075e4c
nsCollation.cpp implements nsCollationFactory and nsCollation utils classes.
To remove nsIPlatfromCharset dependency, I would like to remove nsCollaction utils class. So at first, I would like to rename it to nsCollactionFactory.cpp. (then, nsICollation implementation will be nsCollation.cpp)
MozReview-Commit-ID: FaG3vfzi9cl
--HG--
rename : intl/locale/nsCollation.cpp => intl/locale/nsCollationFactory.cpp
rename : intl/locale/nsCollation.h => intl/locale/nsCollationFactory.h
extra : rebase_source : 25eebf52634310891d1bbcc974635b5cc4b1c402
After landing bug 1301640, 1st parameter of nsIScriptableDateFormat.FormatDateTime() is ignored now. So it is unnecessary to create nsILocaleService from locale parameter.
MozReview-Commit-ID: GlqC0D9shdy
In cases, where the caller is looking for the locale to be used for JS Intl API,
we can now replace it with `undefined` which causes JS Intl API to use the default
locale which since bug 1346674 is resolved to the app locale.
This allows us to remove a lot of calls for the app locale.
The remaining ones are split between `AsBCP47` and `AsLangTag`.
Here, the `AsLangTag` is used, as described in the API docs, for cases where
the language string is used for localization purposes, such as language negotaition
matching to our language resources etc.
`AsBCP47` is used when the returned value is handed over to ICU API.
MozReview-Commit-ID: DzmFEUvMq3N
--HG--
extra : rebase_source : 513ed31d995864939aa893e73c81ffdf591a6617
In cases, where the caller is looking for the locale to be used for JS Intl API,
we can now replace it with `undefined` which causes JS Intl API to use the default
locale which since bug 1346674 is resolved to the app locale.
This allows us to remove a lot of calls for the app locale.
The remaining ones are split between `AsBCP47` and `AsLangTag`.
Here, the `AsLangTag` is used, as described in the API docs, for cases where
the language string is used for localization purposes, such as language negotaition
matching to our language resources etc.
`AsBCP47` is used when the returned value is handed over to ICU API.
MozReview-Commit-ID: DzmFEUvMq3N
--HG--
extra : rebase_source : 13fa4c397ba4c79303a2cd76684b5b8c4bd17331
Sicne some tests doesn't work with --with-intl-api=no, we should skip it on Android
MozReview-Commit-ID: IjlH3aQdiqb
--HG--
extra : rebase_source : ac6d91bf291f6268ea29bcb29180b2f5197e012c
OSPreferences_unix.cpp is used on Android only. So we should rename it to OSPreferences_android.cpp
Also, we should create android directory into intl/locale, then move OSPreferences_android.cpp and nsAndroidCharset.cpp into it.
MozReview-Commit-ID: 2CUbV1Yc2fu
--HG--
rename : intl/locale/unix/OSPreferences_unix.cpp => intl/locale/android/OSPreferences_android.cpp
rename : intl/locale/unix/nsAndroidCharset.cpp => intl/locale/android/nsAndroidCharset.cpp
extra : rebase_source : f677428cc899c767e784b9f2c793b84e3fd95c11
Since we have to keep --with-intl-api=no setting, we allow OSPrefrences API wiout ENABLE_INTL_API.
MozReview-Commit-ID: 8sQHW6iq9Ss
--HG--
extra : rebase_source : 94e2efbeeb70f3edc519ffc9bfeaf7416ea1dc0e
I've moved the body of the APIs into the few remaining callers.
MozReview-Commit-ID: 9ALoSmQHkGM
--HG--
extra : rebase_source : e0e747dec23424446b18d53ab65e4fa262e16554
Also removed unused format selectors and removed option to pass in a locale. Changed all callers to use new version.
MozReview-Commit-ID: Gq0b1c3C1eO
--HG--
rename : intl/locale/unix/nsDateTimeFormatUnix.cpp => intl/locale/DateTimeFormatUnix.cpp
extra : source : 7e747433d1c9fae8a0acb4090f61335fcc165e4c
Also removed unused format selectors and removed option to pass in a locale. Changed all callers to use new version.
MozReview-Commit-ID: Gq0b1c3C1eO
--HG--
rename : intl/locale/unix/nsDateTimeFormatUnix.cpp => intl/locale/DateTimeFormatUnix.cpp
extra : rebase_source : 54d8cfd01edce5f0a78c8b5c6eb89a245c12af01
This commit does the following.
- Converts the test to a gtest, including renaming the directory.
- Factors out the duplicated checking code into a separate Check() function.
- Avoids printing any output unless a failure occurs. (The gtest harness prints
out minimal output, like the name of the current test.)
- Removes SampleWordBreakUsage(), which prints some output but doesn't actually
test anything, and so isn't useful.
--HG--
rename : intl/lwbrk/tests/TestLineBreak.cpp => intl/lwbrk/gtest/TestLineBreak.cpp
rename : intl/lwbrk/tests/moz.build => intl/lwbrk/gtest/moz.build
extra : rebase_source : e673299e237a1a8a3b0a33cfc0a0024ab7dee271
