[PARQUET-430] Change to use Locale parameterized version of String.toUpperCase()/toLowerCase

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 1.8.0, 1.8.1
• Fix Version/s: 1.9.0, 1.8.2
• Component/s: parquet-mr
• Assignee: Liwei Lin(Inactive)
• Reporter: Liwei Lin(Inactive)
• Created: 17/Jan/16 09:03
• Updated: 23/Jun/24 03:27
• Resolved: 16/Feb/16 00:36

Description

A String is being converted to upper or lowercase, using the platform's default encoding. This may result in improper conversions when used with international characters.

For instance, "TITLE".toLowerCase() in a Turkish locale returns "tıtle", where 'ı' – without a dot – is the LATIN SMALL LETTER DOTLESS I character. To obtain correct results for locale insensitive strings, we'd better use toLowerCase(Locale.ENGLISH).

For more information on this, please see:

• http://stackoverflow.com/questions/11063102/using-locales-with-javas-tolowercase-and-touppercase
• http://lotusnotus.com/lotusnotus_en.nsf/dx/dotless-i-tolowercase-and-touppercase-functions-use-responsibly.htm
• http://java.sys-con.com/node/46241

This ticket proposes to change our use of String.toUpperCase()/toLowerCase() to String.toUpperCase(Locale.ENGLISH)/toLowerCase(Locale.ENGLISH)