[AVRO-1493] Avoid the "Turkish Locale Problem"

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.7.6
• Fix Version/s: 1.8.1
• Component/s: java

Environment

Hadoop trunk build error on mac-os with turkish locale.

• Assignee: Kevin Schultz
• Reporter: Serkan Taş
• Created: 06/Apr/14 03:49
• Updated: 07/Aug/19 12:58
• Resolved: 22/Feb/16 02:13

Description

Locale dependent String.toUpperCase(), String.toLowerCase() causes unexpected behavior if the the locale is Turkish
Not sure about String.equalsIgnoreCase(..).

Here is the error :

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:2.5.1:testCompile (default-testCompile) on project hadoop-common: Compilation failure
[ERROR] /Users/serkan/programlar/dev/hadooptest/hadoop-trunk/hadoop-common-project/hadoop-common/target/generated-test-sources/java/org/apache/hadoop/io/serializer/avro/AvroRecord.java:[10,244] unmappable character for encoding UTF-8
[ERROR] -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:2.5.1:testCompile (default-testCompile) on project hadoop-common: Compilation failure
/Users/serkan/programlar/dev/hadooptest/hadoop-trunk/hadoop-common-project/hadoop-common/target/generated-test-sources/java/org/apache/hadoop/io/serializer/avro/AvroRecord.java:[10,244] unmappable character for encoding UTF-8

I f i check the code i discovered the reason for error :

public static final org.apache.avro.Schema SCHEMA$ = new org.apache.avro.Schema.Parser().parse("{\"type\":\"record\",\"name\":\"AvroRecord\",\"namespace\":\"org.apache.hadoop.io.serializer.avro\",\"fields\":[

{\"name\":\"intField\",\"type\":\"Ýnt\"}

]}");

For the code generated from schema, locale dependent capitalization of letter "i" turns in to "Ý" should be the same for "I" to "ı".

Same bug exist in OPENEJB-1071, OAK-260, IBATIS-218.

Issue Links

• relates to: AVRO-2500 Avro library for .NET generates invalid schema json in some cultures
• links to: GitHub Pull Request #69