[IMPALA-8409] STRINGs without stats have too low row-size in explain plan

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: Impala 3.2.0
• Fix Version/s: Impala 3.3.0
• Component/s: Frontend
• Labels: explain statistics
• Epic Color: ghx-label-6
• Assignee: Csaba Ringhofer
• Reporter: Csaba Ringhofer
• Created: 12/Apr/19 12:04
• Updated: 08/Jan/21 03:55
• Resolved: 03/Jun/20 09:51

Description

STRING columns without avg_size statistic are calculated into the row-size as 11 bytes, while they take 12 bytes in the tuple (+ more somewhere in the memory if they are not empty). The issue is caused by adding -1 (meaning unknown) to the 12 byte slot size.

I think that this doesn't cause problems, as the estimation is probably way off without statistics anyway, but row-size >= tuple size seems like a meaningful invariant that we shouldn't break.

Reproduce:

create table test_row_size (s string);
explain select * from test_row_size; 
Result:
...
WARNING: The following tables are missing relevant table and/or column statistics.
default.test_row_size
...
00:SCAN HDFS [default.test_row_size]
   partitions=1/1 files=0 size=0B
   row-size=11B cardinality=0

Issue Links

• breaks: IMPALA-8849 IllegalStateException in HashJoinNode because of missing memory estimate
• is related to: IMPALA-8849 IllegalStateException in HashJoinNode because of missing memory estimate