[IMPALA-8205] Illegal statistics for numFalse and numTrue

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: Impala 4.0.0
• Component/s: Catalog
• Labels: impala numFalse numTrue statistics
• Epic Color: ghx-label-2
• Assignee: wuchang
• Reporter: wuchang
• Created: 15/Feb/19 09:43
• Updated: 27/Mar/23 23:01
• Resolved: 22/Dec/20 21:04

Description

When impala compute statistics, it set numFalse = -1 and numTrue = 1 when the statistic is missing;

-1 for numFalse will corrupt some query engine like Presto and there already exists some PR report and hotfix it : presto-11859

1 for numTrue is also unreasonable because we are not sure whether it indicates the real numTrue statistics or a missing statistics;

Also, previously , the nullCount also use -1 to indicate its absence which also caused problem for Presto. Presto has to add a hotfix for it(presto-11549) . But it is a fortunate that impala has fixed this bug;

It is necessary to set to null when these statistics are absent instead of -1 and 1.

Issue Links

• is related to: IMPALA-11953 num_trues and num_falses in TIntermediateColumnStats should be optional
• relates to: IMPALA-7659 Collect count of nulls when collecting stats

Sub-Tasks

• Don't set numTrue = 1: Resolved