[HIVE-22501] Stats reported multiple times during MR execution for UNION queries

• Type: Bug
• Status: Patch Available
• Priority: Major
• Resolution: Unresolved
• Labels: mapreduce
• Assignee: Daniel Voros
• Reporter: Daniel Voros
• Created: 15/Nov/19 14:22
• Updated: 16/Nov/19 01:25

Description

Take the following example:

set hive.execution.engine=mr;

create table tb(id string) stored as orc;
insert into tb values('1');
create table tb2 like tb stored as orc;

insert into tb2 select * from tb union all select * from tb;

Last insert results in 2 records in the table, but TOTAL_TABLE_ROWS_WRITTEN statistic (and number of affected rows on the consolse) is 4.

We seem to traverse the operator graph multiple times starting from every TS operator and increment the counters every time we hit the FS operator. UNION-ing the table 3 times results in 9 TOTAL_TABLE_ROWS_WRITTEN.