[IMPALA-579] Insert into a binary encoded hbase table can produce incorrect results.

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: Impala 1.2
• Fix Version/s: Impala 1.2
• Assignee: Alexander Behm
• Reporter: Ishaan Joshi
• Created: 05/Sep/13 00:39
• Updated: 21/Sep/13 01:04
• Resolved: 21/Sep/13 01:04

Description

We insert into a binary encoded hbase table from an existing text table.

hbase table definition in hive:

CREATE EXTERNAL TABLE IF NOT EXISTS hbase_table (
  id int,
  bool_col boolean,
  tinyint_col tinyint,
  smallint_col smallint,
  int_col int,
  bigint_col bigint,
  float_col float,
  double_col double,
  date_string_col string,
  string_col string,
  timestamp_col timestamp,
  year int,
  month int,
  day int)
STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
WITH SERDEPROPERTIES (
  "hbase.columns.mapping" =
  ":key#b,d:bool_col#b,d:tinyint_col#b,d:smallint_col#b,d:int_col#b,d:bigint_col#b,d:float_col#b,d:double_col#b,d:date_string_col,d:string_col,d:timestamp_col,d:year#b,d:month#b,d:day#b"
)

The hive schema matches the hbase external table schema, with the last three columns being partitions.

When an insert into is issued into the hbase table, some of the values in the date_string_col are occasionally garbled, leading to a failure.

insert into table hbase_table
select id, bigint_col, bool_col, date_string_col, day, double_col, float_col

int_col, month, smallint_col, string_col, timestamp_col, tinyint_col, year from hive_table;