[HIVE-7142] Hive multi serialization encoding support

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.14.0
• Component/s: Serializers/Deserializers
• Assignee: Chengxiang Li
• Reporter: Chengxiang Li
• Created: 29/May/14 03:05
• Updated: 16/Mar/17 18:58
• Resolved: 14/Aug/14 03:45

Description

Currently Hive only support serialize data into UTF-8 charset bytes or deserialize from UTF-8 bytes, real world users may want to load different kinds of encoded data into hive directly. This jira is dedicated to support serialize/deserialize all kinds of encoded data in SerDe layer.

For user, only need to configure serialization encoding on table level by set serialization encoding through serde parameter, for example:

CREATE TABLE person(id INT, name STRING, desc STRING)ROW FORMAT SERDE 'org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe' WITH SERDEPROPERTIES("serialization.encoding"='GBK');

or

ALTER TABLE person SET SERDEPROPERTIES ('serialization.encoding'='GBK'); 

LIMITATIONS: Only LazySimpleSerDe support "serialization.encoding" property in this patch.

Issue Links

• is related to: HIVE-15826 Add 'serialization.encoding' To All SerDes
• links to: https://reviews.apache.org/r/24377/