[HIVE-15103] Error when inserting into hive table with HBase storage handler

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved

Environment

HDP 2.4.2 on centos 6.8 x64

• Assignee: Unassigned
• Reporter: Edward Chen
• Created: 01/Nov/16 07:41
• Updated: 11/Nov/16 20:36

Description

Exceptions are returned when executing the following simple insert statement in hive on tez.

insert into table accounts  select * from temp_account;

where table accounts is a hive external table, stored in hbase:

create external table accounts (key:string, value:string)
stored by 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
with serdeproperties ("hbase.columns.mapping" = ":key,0:val")
tblproperties ("hbase.table.name" = "accounts_hbase", "hbase.mapred.output.outputtable" = "accounts_hbase");

The SQL could be executed when hive.execution.engine=mr, however, when set hive.execution.engine=tez, we got following error:

Status: Failed
Vertex failed, vertexName=Map 1, vertexId=vertex_1475059990829_1927_1_00, diagnostics=[Task failed, taskId=task_1475059990829_1927_1_00_000099, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:java.lang.RuntimeException: java.lang.RuntimeException: Map operator initialization failed
        at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:173)
        at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:139)
        at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:344)
        at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:181)
        at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:172)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:422)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1709)
        at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:172)
        at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:168)
        at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: Map operator initialization failed
        at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:265)
        at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:149)
        ... 14 more
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.IllegalArgumentException: Must specify table name
        at org.apache.hadoop.hive.ql.exec.FileSinkOperator.createHiveOutputFormat(FileSinkOperator.java:1139)
        at org.apache.hadoop.hive.ql.exec.FileSinkOperator.initializeOp(FileSinkOperator.java:346)
        at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:363)
        at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:482)
        at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:439)
        at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376)
        at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:482)
        at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:439)
        at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376)
        at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:482)
        at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:439)
        at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376)
        at org.apache.hadoop.hive.ql.exec.MapOperator.initializeMapOperator(MapOperator.java:486)
        at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:234)
        ... 15 more
Caused by: java.lang.IllegalArgumentException: Must specify table name
        at org.apache.hadoop.hbase.mapreduce.TableOutputFormat.setConf(TableOutputFormat.java:195)
        at org.apache.hive.common.util.ReflectionUtil.setConf(ReflectionUtil.java:101)
        at org.apache.hive.common.util.ReflectionUtil.newInstance(ReflectionUtil.java:87)
        at org.apache.hadoop.hive.ql.io.HiveFileFormatUtils.getHiveOutputFormat(HiveFileFormatUtils.java:277)
        at org.apache.hadoop.hive.ql.io.HiveFileFormatUtils.getHiveOutputFormat(HiveFileFormatUtils.java:267)
        at org.apache.hadoop.hive.ql.exec.FileSinkOperator.createHiveOutputFormat(FileSinkOperator.java:1137)
        ... 28 more