[IMPALA-787] HdfsStorageDescriptor should handle escaped delimiter values

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: Impala 1.2.3
• Fix Version/s: Impala 1.3
• Assignee: Lenni Kuff
• Reporter: Skye Wanderman-Milne
• Created: 06/Feb/14 23:27
• Updated: 13/Mar/14 18:11
• Resolved: 13/Mar/14 18:11

Description

Currently the delimiter string must be the literal ASCII character, e.g., 'þ'. Hive supports writing the escaped numeric ASCII value, e.g., '\254' or '-2'. Impala will throw the following exception:

com.cloudera.impala.catalog.HdfsStorageDescriptor$InvalidStorageDescriptorException: Invalid delimiter: field.delim has invalid length (-2). Already found: {line.delim=
}
        at com.cloudera.impala.catalog.HdfsStorageDescriptor.extractDelimiters(HdfsStorageDescriptor.java:104)
        at com.cloudera.impala.catalog.HdfsStorageDescriptor.fromStorageDescriptor(HdfsStorageDescriptor.java:166)
        at com.cloudera.impala.catalog.HdfsTable.addDefaultPartition(HdfsTable.java:635)
        at com.cloudera.impala.catalog.HdfsTable.loadPartitions(HdfsTable.java:442)
        at com.cloudera.impala.catalog.HdfsTable.load(HdfsTable.java:752)

To avoid this exception, use the literal character rather than the escaped value.