[IMPALA-11668] Investigate whether Impala's frontend could still use the statistics for an empty table with greater-than-zero file size

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: Frontend
• Epic Color: ghx-label-10
• Assignee: Fang-Yu Rao
• Reporter: Fang-Yu Rao
• Created: 17/Oct/22 19:08
• Updated: 17/Oct/22 19:13

Description

In IMPALA-11666, it has been shown that it's possible to create a transactional ORC table with 0 row but greater-than-zero file size even though there is no corrupt statistics.

Impala's frontend in this case would compute an estimated number of rows instead of using the accurate number of rows (i.e., 0) in HdfsScanNode()#getStatsNumRows(). As a result, the returned number of rows is not accurate.

It would be good if Impala's frontend could determine whether or not there is indeed corrupt statistics associated with a table when its number of row is 0 and its associated file size is greater than 0.