[HIVE-21068] hive table join hbase table return code 3

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 1.1.0
• Component/s: HBase Handler

Environment

hive version 1.1.0-cdh5.7.2 by Apache Hive

hbase version 1.2.0-cdh5.7.2

Hadoop version  2.6.0-cdh5.7.2

OS : centOs6.5

 

• Assignee: Unassigned
• Reporter: Baozhu Zhao
• Created: 26/Dec/18 07:40
• Updated: 21/May/19 01:52

Description

 desc formatted  tmp.tmp_promote_c_rank ;

col_name	data_type	comment
1	# col_name	data_type	comment
2		NULL	NULL
3	uid	bigint
4	km_id	bigint
5		NULL	NULL
6	# Detailed Table Information	NULL	NULL
7	Database:	tmp	NULL
8	Owner:	hdfs	NULL
9	CreateTime:	Wed Dec 26 12:39:13 CST 2018	NULL
10	LastAccessTime:	UNKNOWN	NULL
11	Protect Mode:	None	NULL
12	Retention:	0	NULL
13	Location:	hdfs://nameservice1/user/hive/warehouse/tmp.db/tmp_promote_c_rank	NULL
14	Table Type:	MANAGED_TABLE	NULL
15	Table Parameters:	NULL	NULL
16		COLUMN_STATS_ACCURATE	true
17		numFiles	1
18		numRows	7102588
19		rawDataSize	68906508
20		totalSize	76009096
21		transient_lastDdlTime	1545799153
22		NULL	NULL
23	# Storage Information	NULL	NULL
24	SerDe Library:	org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe	NULL
25	InputFormat:	org.apache.hadoop.mapred.TextInputFormat	NULL
26	OutputFormat:	org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat	NULL
27	Compressed:	No	NULL
28	Num Buckets:	-1	NULL
29	Bucket Columns:	[]	NULL
30	Sort Columns:	[]	NULL
31	Storage Desc Params:	NULL	NULL
32		serialization.format	1

 

desc formatted  ewt_rpt.rpt_ewt_hb_winter_result_7_f_1d;

col_name	data_type	comment
	col_name	data_type	comment
1	# col_name	data_type	comment
2		NULL	NULL
3	key	string	from deserializer
4	uid	bigint	from deserializer
5	km_name	string	from deserializer
6	km_id	bigint	from deserializer
7	rank_id	bigint	from deserializer
8	knowledge_id	bigint	from deserializer
9	knowledge_name	string	from deserializer
10	lesson_list	string	from deserializer
11	question_list	string	from deserializer
12	row_number	bigint	from deserializer
13		NULL	NULL
14	# Detailed Table Information	NULL	NULL
15	Database:	ewt_rpt	NULL
16	Owner:	hdfs	NULL
17	CreateTime:	Fri Dec 21 17:18:07 CST 2018	NULL
18	LastAccessTime:	UNKNOWN	NULL
19	Protect Mode:	None	NULL
20	Retention:	0	NULL
21	Location:	hdfs://nameservice1/user/hive/warehouse/ewt_rpt.db/rpt_ewt_hb_winter_result_7_f_1d	NULL
22	Table Type:	EXTERNAL_TABLE	NULL
23	Table Parameters:	NULL	NULL
24		COLUMN_STATS_ACCURATE	false
25		EXTERNAL	TRUE
26		hbase.table.name	rpt_ewt_hb_winter_result_7_f_1d
27		numFiles	0
28		numRows	-1
29		rawDataSize	-1
30		storage_handler	org.apache.hadoop.hive.hbase.HBaseStorageHandler
31		totalSize	0
32		transient_lastDdlTime	1545383887
33		NULL	NULL
34	# Storage Information	NULL	NULL
35	SerDe Library:	org.apache.hadoop.hive.hbase.HBaseSerDe	NULL
36	InputFormat:	null	NULL
37	OutputFormat:	null	NULL
38	Compressed:	No	NULL
39	Num Buckets:	-1	NULL
40	Bucket Columns:	[]	NULL
41	Sort Columns:	[]	NULL
42	Storage Desc Params:	NULL	NULL
43		hbase.columns.mapping	:key,val1:uid,val1:km_name,val1:km_id,val1:rank_id,val1:knowledge_id,val1:knowledge_name,val1:lesson_list,val1:question_list,val1:row_number
44		serialization.format	1

 

• run sql:

select a.*
from ewt_rpt.rpt_ewt_hb_winter_result_7_f_1d a inner join
tmp.tmp_promote_c_rank b
on a.uid=b.uid
and a.km_id=b.km_id;

 

• run log :

INFO : <PERFLOG method=compile from=org.apache.hadoop.hive.ql.Driver>
INFO : Compiling command(queryId=hive_20181226153232_0781540c-6e60-4688-8ed5-a7044a86af8d): select a.*
from ewt_rpt.rpt_ewt_hb_winter_result_7_f_1d a inner join
tmp.tmp_promote_c_rank b
on a.uid=b.uid
and a.km_id=b.km_id
INFO : Semantic Analysis Completed
INFO : Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:a.key, type:string, comment:null), FieldSchema(name:a.uid, type:bigint, comment:null), FieldSchema(name
:a.km_name, type:string, comment:null), FieldSchema(name:a.km_id, type:bigint, comment:null), FieldSchema(name:a.rank_id, type:bigint, comment:null), FieldSchema(name:a.knowledge_id, type:bigint, comment:null), FieldSchema(name:a.knowledge_name, type:string, comment:null), FieldSchema(name:a.lesson_list, type:string, comment:null), FieldSchema(name:a.question_list, type:string, comment:null), FieldSchema(name:a.row_number, type:bigint, comment:null)], properties:null)INFO : Completed compiling command(queryId=hive_20181226153232_0781540c-6e60-4688-8ed5-a7044a86af8d); Time taken: 0.179 seconds
INFO : Concurrency mode is disabled, not creating a lock manager
INFO : Executing command(queryId=hive_20181226153232_0781540c-6e60-4688-8ed5-a7044a86af8d): select a.*
from ewt_rpt.rpt_ewt_hb_winter_result_7_f_1d a inner join
tmp.tmp_promote_c_rank b
on a.uid=b.uid
and a.km_id=b.km_id
INFO : Query ID = hive_20181226153232_0781540c-6e60-4688-8ed5-a7044a86af8d
INFO : Total jobs = 1
INFO : Starting task [Stage-4:MAPREDLOCAL] in serial mode
ERROR : Obtaining error information
ERROR :
Task failed!
Task ID:
Stage-4

Logs:

ERROR : /var/log/hive/hadoop-cmf-hive-HIVESERVER2-hadoop01.log.out
ERROR : FAILED: Execution Error, return code 3 from org.apache.hadoop.hive.ql.exec.mr.MapredLocalTask
INFO : Completed executing command(queryId=hive_20181226153232_0781540c-6e60-4688-8ed5-a7044a86af8d); Time taken: 165.078 seconds
Error: Error while processing statement: FAILED: Execution Error, return code 3 from org.apache.hadoop.hive.ql.exec.mr.MapredLocalTask (state=08S01,code=3)

Issue Links

• is related to: HIVE-21748 HBase Operations Can Fail When Using MAPREDLOCAL