[HIVE-2917] Add support for various charsets in LazySimpleSerDe

• Type: New Feature
• Status: Patch Available
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 0.9.0
• Component/s: CLI, Serializers/Deserializers
• Assignee: Unassigned
• Reporter: Kai Zhang
• Created: 01/Apr/12 10:41
• Updated: 03/May/12 01:53

Description

Currently hive can only serialize/deserialize data encoded in utf-8.

It would be useful to specify the data's charset when creating the table.

The idea is to add a new keyword CHARSET to set charset at table level.
For example:
CREATE TABLE tbl1 (col1 STRING) ROW FORMAT CHARET "GBK" DELIMITED FIELDS TERMINATED BY '\t';

Another place to use CHARSET is in TRANSFORM clause.
For example:
SELECT TRANSFORM(col1, col2) ROW FORMAT CHARSET 'gbk'
USING 'some_script'
AS (col3, col4) ROW FORMAT CHARSET 'utf-8';