Unexpected Incompatibilities with Java 11
Java 11 introduces some backwards incompatible changes. One that surprised me were the ramifications of upgrading to CLDR 33. The Common Locale Data Repository is the de-facto standard for locale data. It is included starting with Java 8 and enabled by default in Java 9 (see JEP 252).
So how can the upgrade to CLDR 33 affect you? Consider the following test, it passes on Java 8
but fails on Java 11 with
expected: <1'234'567.80> but was: <1’234’567.80>The number is now formatted differently. Instead of an ASCII apostrophe (\u0027) a “Right Single Quotation Mark” (\u2019) is used as a grouping separator for the Swiss German (de_CH) locale. For a full discussion why this was changed see “Why is the right single quote (U+2019), and not the semantically distinct apostrophe (U+0027), the preferred apostrophe character in Unicode?”. It is important to note that the OpenJDK project has made it clear that they do not consider this to be a bug.
This also affects the #parse(String)method as the apostrophe can no longer be parsed. As a consequence "1'234'567.80" will be parsed as 1 with Java 11 and the locale de_CH.
Does this affect you?
Whether you are affected by this change depends on where and how you format numbers and currencies. You will have to review all your input and output interfaces where you use grouping separators for numbers and check whether they changed.
If you use grouping separators only for display purposes, eg. on web sites, you likely will be fine. However there may still be a small impact, for example Excel 2010 on Windows 7 may have trouble while Excel 2016 on Windows 10 should be able to handle it fine.
If you use grouping separators in interfaces or reports with defined formats then you may be impacted more severely as surrounding systems may expect or send data formatted in a certain way. Also you will no longer be able to use ASCII or ISO-8859–1 as an encoding as you now have a code point with a value higher than 255.
How can you get back the old behavior?
If you determined that you are affected by the new behavior and that you need the old behavior back you can explicitly set the grouping separator by calling the DecimalFormatSymbols#setGroupingSeparator(char) method.
This gives you the same behavior in Java 8 and Java 11 but it is only applicable when you are in full control of the Java code. If the formatting is done indirectly by a different layer and you are only in control of the pattern and the locale then a different solution has to be found.
A more drastic way is to set the set the system property java.locale.providers to a value with COMPAT ahead of CLDR or implement a custom LocaleServiceProvider. Again see JEP 252, the Internationalization Enhancements chapter in the Java SE 11 Internationalization Guide and the LocaleServiceProvider class comment for more information.
Take Aways
The take aways from this story are the same as for many other stories.
- It is critical to have a test harness with a good coverage.
- As a software engineer ignoring a topic like Unicode (or time zones or encodings or licenses or …) does not make it make it go away. Much like ignoring security this will only lead to harder to fix issues further down the road.
