IANA timezone db reference in the spec : should backzone be taken into account?

[jungshik]

The current spec keeps referring to 'Zone and Link names', but that's not sufficient and leads to a divergence between implementations.

The main question is whether or not to take into account 'backzone' file in the IANA timezone database.

The ECMAScript 2018 Internationalization API Specification identifies time zones using the Zone and Link names of the IANA Time Zone Database. Their canonical form is the corresponding Zone name in the casing used in the IANA Time Zone Database.

All registered Zone and Link names are allowed. Implementations must recognize all such names

Firefox uses zone and link names in 'backzone' file of the IANA tz db, but some links in 'backzone' file contradicts what's in other files.

backward file has the following:

Link    Asia/Shanghai           Asia/Chongqing
Link    Asia/Shanghai           Asia/Chungking

backzone file has the following:

Link Asia/Chongqing Asia/Chungking

Note that backzone file has the following comment at the top:

# This file contains data outside the normal scope of the tz database,
# in that its zones do not differ from normal tz zones after 1970.
# Links in this file point to zones in this file, superseding links in
# the file 'backward'.

Because Firefox takes into account 'backzone' file, 'Asia/Chungking' is canonicalized to 'Asia/Chongqing' instead of 'Asia/Shanghai'.

ICU/CLDR (as used by v8) ignores 'backzone' file and both 'Asia/Chungking' and 'Asia/Chongqing' are canonicalized to 'Asia/Shaghai' per 'backward' file.

CLDR/ICU, however, do not canonicalize 'Asia/Phnom_Penh' and 'Asia/Vientiane' to 'Asia/Bangkok' despite the following in 'asia' file:

Link Asia/Bangkok Asia/Phnom_Penh       # Cambodia
Link Asia/Bangkok Asia/Vientiane        # Laos

That's because IANA timezone DB relegated the two zone names to links rather recently (2014-2015) and CLDR/ICU do not want to destabilize the tz ID space. So, they kept them as canonical zone IDs.

[anba]

We've went with using the 'backzone' ids in Firefox to avoid the risk to make users in the affected time zones upset. For example canonicalizing 'Europe/Ljubljana', 'Europe/Podgorica', 'Europe/Sarajevo', 'Europe/Skopje', and 'Europe/Zagreb' to 'Europe/Belgrade' (which would be the case when not applying 'backzone') may have negative cultural/political effects.

Relevant comments in the Firefox bug tracker: https://bugzilla.mozilla.org/show_bug.cgi?id=1303091#c3, https://bugzilla.mozilla.org/show_bug.cgi?id=1303091#c9, https://bugzilla.mozilla.org/show_bug.cgi?id=1303091#c11

Unfortunately just using CLDR instead of IANA data can also lead to wrong canonicalizations, cf. https://unicode-org.atlassian.net/browse/ICU-12044 and http://unicode.org/cldr/trac/ticket/9892. In the CLDR bug, jungshik has also given an example where CLDR didn't update the mapping despite being outdated since 1993.

And the related (unresolved) bugs.ecmascript.org bug which also mentions the complications between selecting which tz links are safe to apply and which ones are more contentious: https://tc39.github.io/archives/bugzilla/1892/

And more related threads from the tzdata mailing list (mostly from 2013-2014 when many zones were moved to the backzone file): http://mm.icann.org/pipermail/tz/2014-July/021170.html, https://mm.icann.org/pipermail/tz/2013-September/019821.html, https://mm.icann.org/pipermail/tz/2014-November/021888.html.

[jungshik]

'Europe/Ljubljana', 'Europe/Podgorica', 'Europe/Sarajevo', 'Europe/Skopje', and 'Europe/Zagreb' to 'Europe/Belgrade'

Thanks a lot for alerting me about those entries and references to TZ mailing list threads on the topic. A similar sentiment may exist about canonicalizing Asia/Phnom_Penh and Asia/Vientiane to Asia/Bangkok.

Unfortunately just using CLDR instead of IANA data can also lead to wrong canonicalizations,

Yup, you're right. I'm aware of the issue because CLDR sticks to pretty old IDs that had been deprecated well before CLDR project started. (Calcutta vs Kolkata, Saigon vs Ho_Chi_Minh, Katmandu vs Kathmadu and many others).

@yumaoka

[ryzokuken]

@sffc I'm unsure what needs to be done here. Could you tell me what the web reality is? IIUC, Firefox now uses ICU too, but did ICU ever end up taking this into account and start using the backzone file?

[sffc]

@anba What is Firefox doing these days? Is it still necessary to put in the exception to allow backzone to be used for time zone names?

[sffc]

Is there a snippet of code that can reproduce the Firefox/Chrome discrepancy? It appears that Asia/Chongqing and Asia/Shanghai are equivalent in modern times, but may have differed at some time in the past, perhaps before China decided to unify under one time zone.

I wrote the following code in my best attempt to reproduce the difference, but was unsuccessful in finding a difference:

new Date(1945, 0, 1).toLocaleString("en", { timeZone: "Asia/Chongqing", timeZoneName: "long" })
// "1/1/1945, 2:00:00 PM GMT+09:00"
new Date(1945, 0, 1).toLocaleString("en", { timeZone: "Asia/Shanghai", timeZoneName: "long" })
// "1/1/1945, 2:00:00 PM GMT+09:00"

[jungshik]

IIUC, Firefox now uses ICU too, but did ICU ever end up taking this into account and start using the backzone file?

I think Firefox still uses a rather large override map (to take care of cases mentioned in this issue) on top of ICU. Firefox already used ICU when this issue was filed, btw. :-)

CLDR has a policy on the ID stability and it's a bit hard to change that, I'm afraid. Given this, I was thinking of what Firefox does in v8 to handle 'Saigon => Ho_Chi_Minh', 'Calcutta => Kolkata', etc, but held it off because I wanted it to be resolved at the CLDR so that v8 does not need a local override map [1]. My (dim) hope for a possible CLDR change was based on my 'findings' that turned out to be false. See below.

As for using 'backzone' (this issue), it's related but a bit different.

Unfortunately just using CLDR instead of IANA data can also lead to wrong canonicalizations, cf. https://unicode-org.atlassian.net/browse/ICU-12044 and http://unicode.org/cldr/trac/ticket/9892. In the CLDR bug, jungshik has also given an example where CLDR didn't update the mapping despite being outdated since 1993.

And, unfortunately, my claim turned out to be false. I thought 'Asia/Calcutta' had been changed to 'Asia/Kolkata' well before the CLDR project started. In https://unicode-org.atlassian.net/browse/CLDR-9892, @yumaoka dug up the historic IANA timezone files and found that as lately as 2008 (well after the CLDR project started) had 'Asia/Calcutta' instead of 'Asia/Kolkata'. He suspected that the same was true of 'Saigon vs Ho_Chi_Minh' and 'Katmandu vs Kathmandu'.

[1] To make things complicated, there's a possibility that the override map needs to be duplicated for Chrome OS, which was yet another reason I wanted it to be resolved in CLDR. An alternative of changing the ICU data locally for Chromium was not desirable, either because that'd make the TZ db update process more complicated (although it may not be that bad).

[jungshik]

The repro step is as following:

new Intl.DateTimeFormat("en", {timeZone:"Asia/Chongqing"}).resolvedOptions().timeZone

"Asia/Chongqing" : Firefox
"Asia/Shanghai" : Chrome

[jungshik]

Without underlying zoneinfo files supporting the historical difference between Asia/Chongqing and Asia/Shanghai, I think it's all but pointless to treat them as separate zones.

Below is what Firefox does with my computer timezone set to America/Los_Angeles. Note that Asia/Chongqing and Asia/Shanghai had different local mean time (they have different longitudes), but the result is the same. The same holds for Asia/Bangkok vs Asia/Phnom_Penh.

new Date(1850,0,1).getTimezoneOffset()
472.96666666666664.   # In 1850, LMT was used everywhere including America/Los_Angeles 
new Date(1850,0,1).toLocaleString("en")
"1/1/1850, 12:00:00 AM"
new Date(1850,0,1).toLocaleString("en", {timeZone: "UTC"})
"1/1/1850, 7:52:58 AM"


new Date(1850,0,1).toLocaleString("en", {timeZone: "Asia/Shanghai"})
"1/1/1850, 3:58:41 PM"
new Date(1850,0,1).toLocaleString("en", {timeZone: "Asia/Chongqing"})
"1/1/1850, 3:58:41 PM"

new Date(1850,0,1).toLocaleString("en", {timeZone: "Asia/Phnom_Penh"})
"1/1/1850, 2:35:02 PM"
new Date(1850,0,1).toLocaleString("en", {timeZone: "Asia/Bangkok"})
"1/1/1850, 2:35:02 PM"