[HIVE-10062] HiveOnTez: Union followed by Multi-GB followed by Multi-insert loses data

• Type: Bug
• Status: Closed
• Priority: Critical
• Resolution: Fixed
• Fix Version/s: 1.2.0
• Assignee: Pengcheng Xiong Assign to me
• Reporter: Pengcheng Xiong
• Created: 23/Mar/15 21:04
• Updated: 10/Aug/15 22:52
• Resolved: 23/Apr/15 20:56

Description

In q.test environment with src table, execute the following query:

CREATE TABLE DEST1(key STRING, value STRING) STORED AS TEXTFILE;

CREATE TABLE DEST2(key STRING, val1 STRING, val2 STRING) STORED AS TEXTFILE;

FROM (select 'tst1' as key, cast(count(1) as string) as value from src s1
                         UNION all 
      select s2.key as key, s2.value as value from src s2) unionsrc
INSERT OVERWRITE TABLE DEST1 SELECT unionsrc.key, COUNT(DISTINCT SUBSTR(unionsrc.value,5)) GROUP BY unionsrc.key
INSERT OVERWRITE TABLE DEST2 SELECT unionsrc.key, unionsrc.value, COUNT(DISTINCT SUBSTR(unionsrc.value,5)) 
GROUP BY unionsrc.key, unionsrc.value;

select * from DEST1;
select * from DEST2;

DEST1 and DEST2 should both have 310 rows. However, DEST2 only has 1 row "tst1 500 1"

Issue Links

• links to: RB