[HIVE-9896] \N un-recognized in AVRO format Hive tables

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 0.13.0
• Component/s: Database/Schema, File Formats, Hive

Environment

CDH5.2.1, RHEL6.5, Java 7

• Assignee: Unassigned
• Reporter: Madhan Sundararajan Devaki
• Created: 09/Mar/15 11:55
• Updated: 16/Mar/15 10:26

Description

We Sqooped (1.4.5) data from many RDBMS into HDFS in text format with options --null-non-string '\N' --null-string '\N'.
When we load these into Hive tables in text format the \N is properly recognized as NULL and we are able to use SQL clauses such as IS NULL and IS NOT NULL against columns.
However, when we convert the text files into AVRO (1.7.6) with SNAPPY compression through Pig (0.12) and try to query using the above SQL clauses, the query does not return results as expected.
Further, we have to use column_name = '\N' or column_name <> '\N' as a workaround.