[IMPALA-8458] Can't set numNull/maxSize/avgSize column stats with local catalog without also setting NDV

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: Impala 3.3.0
• Fix Version/s: Impala 3.3.0
• Component/s: Catalog
• Target Version: Impala 3.3.0
• Epic Color: ghx-label-1
• Assignee: Todd Lipcon
• Reporter: Tim Armstrong
• Created: 26/Apr/19 00:03
• Updated: 23/May/19 18:24
• Resolved: 22/May/19 23:02

Description

Repro:

[tarmstrong-box2.ca.cloudera.com:21000] default> create table test_stats2(s string);
+-------------------------+
| summary                 |
+-------------------------+
| Table has been created. |
+-------------------------+
Fetched 1 row(s) in 0.36s
[tarmstrong-box2.ca.cloudera.com:21000] default> show column stats test_stats2;
+--------+--------+------------------+--------+----------+----------+
| Column | Type   | #Distinct Values | #Nulls | Max Size | Avg Size |
+--------+--------+------------------+--------+----------+----------+
| s      | STRING | -1               | -1     | -1       | -1       |
+--------+--------+------------------+--------+----------+----------+
Fetched 1 row(s) in 0.02s
[tarmstrong-box2.ca.cloudera.com:21000] default> alter table test_stats2 set column stats s('avgSize'='1234');
+-----------------------------------------+
| summary                                 |
+-----------------------------------------+
| Updated 0 partition(s) and 1 column(s). |
+-----------------------------------------+
Fetched 1 row(s) in 0.14s
[tarmstrong-box2.ca.cloudera.com:21000] default> show column stats test_stats2;
+--------+--------+------------------+--------+----------+----------+
| Column | Type   | #Distinct Values | #Nulls | Max Size | Avg Size |
+--------+--------+------------------+--------+----------+----------+
| s      | STRING | -1               | -1     | -1       | -1       |
+--------+--------+------------------+--------+----------+----------+
Fetched 1 row(s) in 0.02s
[tarmstrong-box2.ca.cloudera.com:21000] default> alter table test_stats2 set column stats s('maxSize'='1234');
+-----------------------------------------+
| summary                                 |
+-----------------------------------------+
| Updated 0 partition(s) and 1 column(s). |
+-----------------------------------------+
Fetched 1 row(s) in 0.10s
[tarmstrong-box2.ca.cloudera.com:21000] default> show column stats test_stats2;
+--------+--------+------------------+--------+----------+----------+
| Column | Type   | #Distinct Values | #Nulls | Max Size | Avg Size |
+--------+--------+------------------+--------+----------+----------+
| s      | STRING | -1               | -1     | -1       | -1       |
+--------+--------+------------------+--------+----------+----------+
Fetched 1 row(s) in 0.02s
[tarmstrong-box2.ca.cloudera.com:21000] default> invalidate metadata test_stats2;
Fetched 0 row(s) in 0.03s
[tarmstrong-box2.ca.cloudera.com:21000] default> show column stats test_stats2;
Query: show column stats test_stats2
+--------+--------+------------------+--------+----------+----------+
| Column | Type   | #Distinct Values | #Nulls | Max Size | Avg Size |
+--------+--------+------------------+--------+----------+----------+
| s      | STRING | -1               | -1     | -1       | -1       |
+--------+--------+------------------+--------+----------+----------+
Fetched 1 row(s) in 0.07s

I expected that the updates would take effect. Weirdly it doesn't happen for NDV and NULLS:

[tarmstrong-box2.ca.cloudera.com:21000] default> alter table test_stats2 set column stats s('numDVs'='1234','numNulls'='12345');
Query: alter table test_stats2 set column stats s('numDVs'='1234','numNulls'='12345')
+-----------------------------------------+
| summary                                 |
+-----------------------------------------+
| Updated 0 partition(s) and 1 column(s). |
+-----------------------------------------+
Fetched 1 row(s) in 0.12s
[tarmstrong-box2.ca.cloudera.com:21000] default> show column stats test_stats2;
Query: show column stats test_stats2
+--------+--------+------------------+--------+----------+----------+
| Column | Type   | #Distinct Values | #Nulls | Max Size | Avg Size |
+--------+--------+------------------+--------+----------+----------+
| s      | STRING | 1234             | 12345  | -1       | -1       |
+--------+--------+------------------+--------+----------+----------+
Fetched 1 row(s) in 0.02s