[HIVE-3211] Enhance hbase column mapping strategy to support complicated structures

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 0.9.0
• Component/s: HBase Handler
• Assignee: Swarnim Kulkarni
• Reporter: Swarnim Kulkarni
• Created: 28/Jun/12 15:57
• Updated: 07/Jan/14 16:49

Description

With the current Hive-HBase integration, we need to specify the "hbase.columns.mapping" property that maps the hbase columns to the columns in hive. So for example,

hive> CREATE EXTERNAL TABLE complex(
key string, 
s struct<col1 : int, col2 : int>)
STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
WITH SERDEPROPERTIES (
"hbase.columns.mapping" = ":key,cf:a"
)TBLPROPERTIES ("hbase.table.name" = "TEST_TABLE");;

The struct definition in the above query can quickly get very ugly if we are dealing with a very complicated structures stored in hbase columns. We should probably enhance the current columns mapping strategy to be able to provide a custom serializer and let it detect the structure by itself.

Issue Links

• is related to: HBASE-10091 Exposing HBase DataTypes to non-Java interfaces