[HIVE-136] SerDe should escape some special characters

• Type: Bug
• Status: Closed
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 0.3.0
• Fix Version/s: 0.4.0
• Component/s: Serializers/Deserializers
• Hadoop Flags: Reviewed
• Assignee: Zheng Shao
• Reporter: Zheng Shao
• Created: 08/Dec/08 21:36
• Updated: 17/Dec/11 00:07
• Resolved: 02/Jul/09 02:03

Description

MetadataTypedColumnsetSerDe and DynamicSerDe should escape some special characters like '\n' or the column/item/key separator.
Otherwise the data will look corrupted.

We plan to deprecate MetadataTypedColumnsetSerDe and DynamicSerDe for the simple delimited format, and use LazySimpleSerDe instead.
So LazySimpleSerDe needs to have the capability of escaping and unescaping.

Issue Links

• is blocked by: HIVE-337 LazySimpleSerDe should support multi-level nested array, map, struct types
• is related to: HIVE-1898 The ESCAPED BY clause does not seem to pick up newlines in columns and the line terminator cannot be changed
• is related to: HIVE-270 Add a lazy-deserialized SerDe for space and cpu efficient serialization of rows with primitive types