Intl.Collator('generic') fails

[kunalspathak]

Intl.Collator('generic');

gives following error.

RangeError: Locale 'generic' is not well-formed
   at Global code (e:\temp\ic.js:1:1)

This is happening because Windows.globlalization.dll here returns false and Intl.js here throws. I tried debugging and it is the code inside Bcp47Langs.dll that returns it false. May be because it treats 'generic' as special tag and so call it as not well-formed. I didn't notice anything special about 'generic' in spec. Also this works in Edge, so might worth investigating why it doesn't work in ch.exe. This breaks node-chakracore while running eslint rules.

[dilijev]

@kunalspathak It looks like it doesn't work in Edge 15063. When did you last observe it working in Edge?

[dilijev]

> es -its -h d8,sm,ch-1.2.3,ch-1.3.2,ch-1.4.3,ch-dev,node,node-ch -e "Intl.Collator('generic')"
## Source
print(Intl.Collator('generic'))

┌──────────┬─────────────────────────────────────────────────┐
│ ch-1.2.3 │                                                 │
│ ch-1.3.2 │ RangeError: Locale 'generic' is not well-formed │
│ ch-1.4.3 │                                                 │
│ ch-dev   │                                                 │
│ node-ch  │                                                 │
├──────────┼─────────────────────────────────────────────────┤
│ node     │ Collator {}                                     │
├──────────┼─────────────────────────────────────────────────┤
│ d8       │ [object Object]                                 │
│ sm       │                                                 │
└──────────┴─────────────────────────────────────────────────┘

[dilijev]

• https://tc39.github.io/ecma402/#sec-intl.collator (this anchor has a bug (Anchor for 10.1.2 Intl.Collator is the same as 8.1.1 Intl.Collator tc39/ecma402#145); see spec section 10.1.2)
• https://tc39.github.io/ecma402/#sec-initializecollator
• https://tc39.github.io/ecma402/#sec-canonicalizelocalelist (which actually produces the error in our case)
• https://tc39.github.io/ecma402/#sec-isstructurallyvalidlanguagetag (which returns false, causing the error)

'generic' as a language tag is gramatically valid and "potentially" valid (if "registered") see the RFC 5646 Section 2.1 grammar for language tags, specifically this production:

 language      = ...
               ...
               / 5*8ALPHA            ; or registered language subtag

It appears that WinGlob believes this is not a valid language tag. This could be because "generic" is not a registered language subtag under WinGlob. (It's also possible that WinGlob does not support this production of the language tag grammar.)

Since IMO Intl errors should not be gated on the contents of the Intl database (consistent across all engines regardless of database content), we should simply recognize the grammar of possibly valid tags for the purposes of determining errors (which means supporting the grammar production outside of the i18n database we use). If the WinGlob database does not support this language tag, then the resulting locale set from e.g. Intl.Collator.supportedLocalesOf('generic') would be empty, but that is still "correct" behavior. From Intl.getCanonicalLocales, the tag should be canonicalized even if it is not supported.

/cc @bterlson I think the spec text is fairly clear here, but maybe there's potential for confusion on the conditional ("registered") above. What do you think?

This issue can now be observed more directly via use of the Intl.getCanonicalLocales API.

> es -its -h sm,d8,node,ch-intl* -e "Intl.getCanonicalLocales('generic')[0]"
## Source
print(Intl.getCanonicalLocales('generic')[0])

┌─────────────┬─────────────────────────────────────────────────┐
│ d8          │ generic                                         │
│ sm          │                                                 │
│ node        │                                                 │
├─────────────┼─────────────────────────────────────────────────┤
│ ch-intl-gcl │                                                 │
│             │ RangeError: Locale 'generic' is not well-formed │
└─────────────┴─────────────────────────────────────────────────┘

> es -its -h sm,d8,node,ch-intl*,ch-dev -e "Intl.getCanonicalLocales(['generic-us'])[0]"
## Source
print(Intl.getCanonicalLocales(['generic-us']))

┌─────────────┬────────────────────────────────────────────────────┐
│ d8          │ generic-US                                         │
│ sm          │                                                    │
│ node        │                                                    │
├─────────────┼────────────────────────────────────────────────────┤
│ ch-intl-gcl │                                                    │
│             │ RangeError: Locale 'generic-us' is not well-formed │
└─────────────┴────────────────────────────────────────────────────┘

> es -its -h sm,d8,node,ch-intl* -e "Intl.getCanonicalLocales('generic-latn-us')[0]"
## Source
print(Intl.getCanonicalLocales('generic-latn-us')[0])

┌─────────────┬─────────────────────────────────────────────────────────┐
│ d8          │ generic-Latn-US                                         │
│ sm          │                                                         │
│ node        │                                                         │
├─────────────┼─────────────────────────────────────────────────────────┤
│ ch-intl-gcl │                                                         │
│             │ RangeError: Locale 'generic-latn-us' is not well-formed │
└─────────────┴─────────────────────────────────────────────────────────┘

[dilijev]

Tracking as part of #3644