[HIVE-25918] Invalid stats after multi inserting into the same partition

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Component/s: Statistics
• Labels: pull-request-available
• Target Version/s: 4.0.0
• Assignee: Krisztian Kasa
• Reporter: Krisztian Kasa
• Created: 01/Feb/22 16:44
• Updated: 22/Feb/22 19:05
• Resolved: 09/Feb/22 16:51

Description

create table source(p int, key int,value string);
insert into source(p, key, value) values (101,42,'string42');

create table stats_part(key int,value string) partitioned by (p int);

from source
insert into stats_part select key, value, p
insert into stats_part select key, value, p;

select count(*) from stats_part;

In this case StatsOptimizer helps serving this query because the result should be rowNum of the partition p=101. The result is

1

however it shloud be

2

because both insert branches inserts 1-1 records.

Issue Links

• is caused by: HIVE-15250 Reuse partitions info generated in MoveTask to its subscribers (StatsTask)
• relates to: HIVE-23949 Introduce caching layer in HS2 to accelerate query compilation
• links to: GitHub Pull Request #2991
• links to: GitHub Pull Request #3020
• links to: GitHub Pull Request #3030