[HIVE-22198] Execute unoin-all with childs Join in parallel

• Type: Bug
• Status: Patch Available
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 3.1.0
• Assignee: GuangMing Lu
• Reporter: GuangMing Lu
• Created: 12/Sep/19 13:26
• Updated: 08/Oct/19 07:31

Description

set parallel is true, set skewjoin is false, set auto convert join is false. run a unoin all, There is nothing error message, but some result data is missing, details check attatchment test-parallel.sql

create table tab1(tid int, com string) row format delimited fields terminated by '\t' stored as textfile;
create table tab2(tid int, com string) row format delimited fields terminated by '\t' stored as textfile;
create table tab3(tid int, com string) row format delimited fields terminated by '\t' stored as textfile;
create table tab4(tid int, com string) row format delimited fields terminated by '\t' stored as textfile;

insert into tab1 values(1,'abc'),(2,'bcd'),(3,'cde');
insert into tab2 values(1,'abc'),(2,'bcd'),(3,'cde');
insert into tab3 values(1,'abc'),(2,'bcd'),(3,'cde');
insert into tab4 values(1,'abc'),(2,'bcd'),(3,'cde');

set hive.auto.convert.join=false;
set hive.optimize.skewjoin=true;
set hive.exec.parallel=true;

SELECT sum(1) as a
FROM tab1 t1
INNER JOIN tab2 t2
ON t1.com = t2.com
UNION ALL
SELECT sum(1) as a
FROM tab3 t3
INNER JOIN tab4 t4
ON t3.com = t4.com;

create table test_parallel stored as orcfile as
SELECT sum(1) as a
FROM tab1 t1
INNER JOIN tab2 t2
ON t1.com = t2.com
UNION ALL
SELECT sum(1) as a
FROM tab3 t3
INNER JOIN tab4 t4
ON t3.com = t4.com;

select * from test_parallel;

The result data should be two, but only one.

Issue Links

• is related to: HIVE-20456 Query fails with FNFException using MR with skewjoin enabled and auto convert join disabled
• is related to: HIVE-14129 Execute move tasks in parallel