Add navigator.locales for user preferences

[littledan]

There are already two languages for specifying options to a locale:

• A simple locale tag (e.g., "en-US") plus an object with further options ({ tz: "Asia/Jerusalem" })
• A BCP 47 string representing the combination of the two (e.g., "en-US-u-tz-jerusalm")

Each has some advantages:

• The options object is easier for programmers to read and write, and easier to manipulate one field of.
• The BCP47 tag is easier to pass over the network, and it can passed around as a single object for all the possible locale information (TODO: confirm that everything we represent in options objects has a BCP47 subtag)

An idea here: resolvedOptions() shouldn't return just a plain old object, but an instance of a special class whose toString() method returns the BCP47 locale for those options! This way, we get the best of all worlds. The output of resolvedOptions() can also then be used as the first (and only) argument to constructors like Intl.DateTimeFormat.

These Intl.Locale objects can be used generally as the way of representing locale information within ECMAScript. HTML could include an Array of relevant complex Locales (not just simple locales like "en-US") in navigator.locales, which would be usable either as a string or to get these higher-level properties; detailed user preferences could be reflected in the Locale which would be difficult to reflect in the simple navigator.language. Locales would still have own properties for things that resolvedOptions() currently includes as own properties, and would therefore probably be web-compatible as an upgrade. We probably couldn't just replace navigator.language with a Locale for web compatibility reasons, but there's no reason that the string first argument to formatters can't be more flexible (if it isn't already) by continuing to call ToString() and then parsing the BCP47.

(To clarify, this isn't my idea but was something a group of us came up with yesterday.)

@domenic @ericf @caridy Thoughts?

[rxaviers]

The idea of having the Locale API to transform nice verbose objects like {tz: "Asia/Jerusalem"} into respective BCP 47 "-u-tz-jerusalm" sounds interesting to me. This could be also used for any other Unicode extensions like numbering system (-u-nu), currency (u-cu), etc. Having said that, I have no comments (for now) about the rest of your proposal: on how this could be exposed / used by formatters.

[zbraniecki]

Three settings that would be interesting for us to see how they fit are:

• hour12/hour24
• first day of the week
• calendar

If I understand the proposal correct:

var f = Intl.DateTimeFormat(['fr'], {hour:'2-digit', minute: '2-digit'});
console.log(f.format(0)); // 23:41

var f = Intl.DateTimeFormat(['fr-u-hc-h12'], {hour:'2-digit', minute: '2-digit'});
console.log(f.format(0)); // 11:41 PM

var f = Intl.DateTimeFormat(['fr-u-hc-h24'], {hour:'2-digit', minute: '2-digit'});
console.log(f.format(0)); // 23:41

Then, we'd expect environment to build things like navigator.locales based on navigator.languages with those settings added to locale tags.

[rxaviers]

I've re-read the issue description and just noticed my comment was completely off.

Please, could you clarify your initial statement? I got confused when you say there are already two languagues for specifying options to a locale. Is this related to formatters? Because, the current specification doesn't actually allow such choice. For example, Intl.DateTimeFormat takes timeZone as an option (12.1.1), but it ignores the timezone info from the locale -u-tz (12.2.3). Similarly, Intl.NumberFormat takes currency as an option (11.1.1), but it ignores the currency info from the locale -u-cu (11.2.3). The opposite also happens, e.g., Intl.DateTimeFormat takes calendar and numbering system respectively from the locale data -u-ca and -u-nu, but there's no corresponding key-value pair option to modify the formatter output. Or are those two languages already available internally in some locale-related-operation?

Are you suggesting a change in the internals only, or does it have implications on how to use the formatters too? For example, does your suggestion imply that info like calendar, timezone, currency could now be passed either via locale or key-value pair options? Would formatters accept either a locale string or a locale instance?

Thanks

[littledan]

OK, good catch. My suggestion would be to change that part of the spec to actually not ignore things like -u-cu. We might also want to add a calendar: field to the options object.

Faithfully representing and using all data in both of them is a project which would take a bunch of work, and could be done incrementally. I don't think we need to do it all at once--we can implement toString() even before it's all done. Would there be downsides to gradually evolving the spec in this direction?

The strawman suggestion was that formatters would end up accepting a locale instance where they currently accept a locale string simply by calling ToString() on the locale instance, and then end up parsing that into the options again. But there are other ways we could phrase the spec, like by checking whether it is an instanceof Intl.Locale or has a locale property or something like that.

[domenic]

I really like this proposal, and would be happy to add navigator.locales to HTML (assuming implementer interest) to support it. (Or maybe the only thing we need is navigator.userLocale? No user has a priority list of locale options, after all, whereas they do for languages.)

I'd like someone here to work out an initial proposal for a few properties that would be useful to include in such an object, along with their corresponding serialization behavior.

[littledan]

We discussed this idea some more this morning. Sounds like one piece that's needed is a Locale class that has methods to get various pieces of data out, rather than an object that's already populated with fields for everything.

It seems to me like having an array of locales is the most general thing, perhaps denormalized but it seems more future-proof.

[caridy]

side note: A request header with the locales with extensions should also be added, e.g.: Accept-Locales: en-u-tz-jerusalm-US, en-u-tz-jerusalm;q=0.8, es-u-tz-jerusalm;q=0.6, so isomorphic apps can do the right thing. The concern here is probably finger printing, but we can probably add that request header only over https connections. Just food for thoughts!

[domenic]

In general browsers are loathe to add new headers (extra bytes) to every request. Also fingerprinting is just as much a concern for HTTPS sites as for insecure ones so there's no reason to restrict. Finally I think q factors are out of vogue these days?

I am not too knowledgeable on the HTTP side of things though, so it would be good to get an implementer comment from someone who is. I might just be wrong about much of the above. Maybe @mnot could lend his wisdom on how much of a good idea or not such a header is.

[mnot]

I think the fingerprinting question is the biggest potential blocker; sending it in a header without any server action changes it from active fingerprinting (i.e., you can tell if a script probes this info and sends it back to the server) vs. passive (you have no idea if they're using it, because you send it on every request).

Assuming that doesn't kill it, a few things to keep in mind:

• This is effectively yet another axis for content negotiation, so when the resopnse depended on Accept-Locale, you'd need to list it in Vary. That will affect cache hit rates in CDNs and reverse proxies, but not in browsers (assuming the browser sticks with one locale).
• The extra bits are a concern for HTTP/1.1, less so for HTTP/2 thanks to header compression.
• There have been requests for timezone information in request headers in the past, so there is some amount of demand for this; not really sure how much, though.
• The absolutely easiest way to do this would be to just add the locales to the Accept-Language request header field.
• Yeah, q-values kinda suck, but we haven't quite abandoned them yet. See the text at the link above.

[caridy]

@mnot if the q-values is an option, and the absolutely easiest way to do this, that works for me. I just want to make sure that if we introduce navigator.locales, users can, somehow, get those values on the server side as well.

[zbraniecki]

Kickstarted a bug to add navigator.locales internally in Gecko - https://bugzilla.mozilla.org/show_bug.cgi?id=1303579

@domenic - do we have enough consensus here to push it to DOM? If so, would you be willing to start the conversation there?

[zbraniecki]

OK, good catch. My suggestion would be to change that part of the spec to actually not ignore things like -u-cu.

I'd be happy to champion this change in ECMA402 spec.

[littledan]

Before we make a change like this with respect to the DOM/HTTP headers, I'd like feedback from the W3C TAG on fingerprinting-- @slightyoff would we need a user permission to expose more data that would give more information about a user agent's configuration that could lead to fingerprinting?

[slightlyoff]

Hey all,

So the TAG considered this pretty deeply last year. There are many bits of data that browsers already leak and, in general, it's very very difficult to design around adding new ones. Therefore, we don't. Instead, we want to avoid intentional and unclearable supercookies.

The primary question from my perspective is: how much of this data is already available in other observable ways in the browser? Is there anything about the spec that would prevent this from being user-settable or user-resettable (it doesn't seem so at first glance)?

Thanks

[domenic]

@zbraniecki so this would be navigator.locales which returns these Locale objects? Where are the Locale objects specced?

Do we have implementer commitment from 2+ implementers? Do you have Gecko buy-in? Does @littledan have Blink buy-in?

[zbraniecki]

The primary question from my perspective is: how much of this data is already available in other observable ways in the browser?

So, this data is not available or observable at the moment. In fact that's the very problem I'm trying to solve.
At the moment, user may specify, for example, which type of time format she wants to use (hour12 or hour24), but from within the web app there is no way to know that and adapt UI for that.

My particular use case for the feature proposed by @littledan is to get this information. We can either expose it via API (like navigator.locales) and allow it to be used for fingerprinting, or make Intl API implicitly get the preferences from the OS and then fingerprinting is still possible, but takes a bit more time.
Since we can't prevent fingerprinting, I believe we should just expose the data via API.

so this would be navigator.locales which returns these Locale objects? Where are the Locale objects specced?

I wanted to start by exposing locale strings counterparts of language tags on navigator.languages.

Since we will definitely want to keep allowing for the first argument to Intl APIs to be a string, we can decouple the idea to create a special object Intl.Locale and work on this proposal separately.

In other words, I think that there are four ideas here:

• Make Intl APIs accept extension tags in locale codes and apply them onto options
• Create navigator.locales and onlocaleschange event coutnerparts to navigator.languages that present the same list of languages but with extension tags from the host environment
• Create a new Intl.Locale class and turn values returned by resolvedOptions() to return an object of this class. It's toString returns a locale code so the object can be passed directly to Intl API as a locale tag.
• Add missing options to the options object - calendar, numeric system, first day of the week etc.

I'd like to initially work on the first and second since they, when coupled, give us ability to create date/time UI elements that match users preferences defined in the host OS.

[zbraniecki]

Uhh, I just realized that we need the Intl.Locale for the navigator.locales... :)

Soo, I can champion two pieces:

• Intl.Locale object which has modified toString which generates canonical locale string
• Modifying current Intl APIs to accept extension tags

And I need help to get DOM piece:

• navigator.locales and onlocaleschange which return a list of language tags with extension tags from the host OS.

Does that sound good?

[zbraniecki]

@littledan, @caridy , @ericf, @rxaviers, @domenic ?

[littledan]

That document says,

Believes that, because combatting fingerprinting is difficult, new Web specifications should take reasonable measures to avoid adding unneeded fingerprinting surface area. However, added surface area should not be a primary factor in determining whether to add a new feature.

We can be sure to document the new fingerprinting surface area in the spec. Users can reset their user agent's preferences (whether it's in the OS or the browser) to system default to avoid the additional fingerprinting surface. Would that be sufficient, @slightlyoff ?

[sffc]

Let's focus this particular issue on navigator.locales + onlocaleschange.

• The main Intl.Locale issue is Intl.Locale #106
• The part about making Intl APIs accept extension tags should be handled in Complete the extension keys + options + resolvedOptions set #105

[sffc]

Blocked pending conclusion on #416

[littledan]

See other discussion in whatwg/html#3046

[sffc]

@aphillips said in tc39/proposal-intl-locale#84:

So this morning I was doing a code review of some Typescript code (implementing message format) and stumbled over a developer using new Intl.NumberFormat().resolvedOptions().locale to find out the default locale. This apparently is still the "canonical" way to find out what the "default locale" is in general. The problem is that this is a potentially expensive operation. The MDN documentation refers to "the default locale", but there is no way to find it out without building some formatter (or collator etc.) object. This seems undesirable.

Shouldn't there be a generic Locale interface to find out what the default locale is (for developers such as my team's who are writing "locale aware" functions)? It wouldn't prevent specific formatters or functions from using LC_xxx-style separate values. Or am I missing something?

[justingrant]

More generally, the same concern above applies to user timezone and user calendar. Requiring the creation of a (very expensive) new Intl.DateTimeFormat() call is expensive for developers who need to get the user timezone. The problem is that this expensiveness will lead developers to call the API once and cache the result, which will make the app brittle in the face of users who dynamically change locales, timezones, or calendars. Timezones in particular change a lot for users who travel on airplanes. Is there already a general issue capturing the suggestion for a cheaper and more ergonomic way to get all of these user preferences (timezone, locale, calendar) ?

[ryzokuken]

@justingrant I have a private (for now) doc where I am researching and exploring these. Feel free to DM me about this problem or ask for an invite.