[IMPALA-7225] Refresh on single partition resets partition's row count to -1

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: Impala 2.10.0, Impala 2.12.0
• Fix Version/s: Impala 3.1.0
• Component/s: Catalog
• Target Version: Impala 3.1.0
• Epic Color: ghx-label-7
• Assignee: Bharath Vissapragada
• Reporter: Mala Chikka Kempanna
• Created: 29/Jun/18 19:57
• Updated: 27/Jul/18 05:01
• Resolved: 27/Jul/18 00:27

Description

Doing refresh on single partition resets it's row count to -1

 

[host-2.x.y.z:21000] > show partitions web_logs_new;
Query: show partitions web_logs_new
+------------+-------+--------+----------+--------------+-------------------+--------+-------------------+---------------------------------------------------------------------------------------------------------+
| date_col | #Rows | #Files | Size | Bytes Cached | Cache Replication | Format | Incremental stats | Location |
+------------+-------+--------+----------+--------------+-------------------+--------+-------------------+---------------------------------------------------------------------------------------------------------+
| 2015-11-18 | -1 | 1 | 112.15KB | NOT CACHED | NOT CACHED | TEXT | false | hdfs://nightly513-unsecure-1.gce.cloudera.com:8020/user/hive/warehouse/web_logs_new/date_col=2015-11-18 |
| 2015-11-19 | -1 | 1 | 98.83KB | NOT CACHED | NOT CACHED | TEXT | false | hdfs://nightly513-unsecure-1.gce.cloudera.com:8020/user/hive/warehouse/web_logs_new/date_col=2015-11-19 |
| 2015-11-20 | -1 | 1 | 101.57KB | NOT CACHED | NOT CACHED | TEXT | false | hdfs://nightly513-unsecure-1.gce.cloudera.com:8020/user/hive/warehouse/web_logs_new/date_col=2015-11-20 |
| 2015-11-21 | -1 | 1 | 82.99KB | NOT CACHED | NOT CACHED | TEXT | false | hdfs://nightly513-unsecure-1.gce.cloudera.com:8020/user/hive/warehouse/web_logs_new/date_col=2015-11-21 |
| Total | -1 | 4 | 395.54KB | 0B | | | | |
+------------+-------+--------+----------+--------------+-------------------+--------+-------------------+---------------------------------------------------------------------------------------------------------+
Fetched 5 row(s) in 0.01s
[host-2.x.y.z:21000] > compute stats web_logs_new;
Query: compute stats web_logs_new
+------------------------------------------+
| summary |
+------------------------------------------+
| Updated 4 partition(s) and 28 column(s). |
+------------------------------------------+
Fetched 1 row(s) in 1.31s
[nightly513-unsecure-2.gce.cloudera.com:21000] > show partitions web_logs_new;
Query: show partitions web_logs_new
+------------+-------+--------+----------+--------------+-------------------+--------+-------------------+---------------------------------------------------------------------------------------------------------+
| date_col | #Rows | #Files | Size | Bytes Cached | Cache Replication | Format | Incremental stats | Location |
+------------+-------+--------+----------+--------------+-------------------+--------+-------------------+---------------------------------------------------------------------------------------------------------+
| 2015-11-18 | 250 | 1 | 112.15KB | NOT CACHED | NOT CACHED | TEXT | false | hdfs://nightly513-unsecure-1.gce.cloudera.com:8020/user/hive/warehouse/web_logs_new/date_col=2015-11-18 |
| 2015-11-19 | 250 | 1 | 98.83KB | NOT CACHED | NOT CACHED | TEXT | false | hdfs://nightly513-unsecure-1.gce.cloudera.com:8020/user/hive/warehouse/web_logs_new/date_col=2015-11-19 |
| 2015-11-20 | 250 | 1 | 101.57KB | NOT CACHED | NOT CACHED | TEXT | false | hdfs://nightly513-unsecure-1.gce.cloudera.com:8020/user/hive/warehouse/web_logs_new/date_col=2015-11-20 |
| 2015-11-21 | 250 | 1 | 82.99KB | NOT CACHED | NOT CACHED | TEXT | false | hdfs://nightly513-unsecure-1.gce.cloudera.com:8020/user/hive/warehouse/web_logs_new/date_col=2015-11-21 |
| Total | 1000 | 4 | 395.54KB | 0B | | | | |
+------------+-------+--------+----------+--------------+-------------------+--------+-------------------+---------------------------------------------------------------------------------------------------------+
Fetched 5 row(s) in 0.01s


[host-2.x.y.z:21000] > refresh web_logs_new partition(date_col='2015-11-18');
Query: refresh web_logs_new partition(date_col='2015-11-18')
Query submitted at: 2018-06-29 12:53:32 (Coordinator: http://nightly513-unsecure-2.gce.cloudera.com:25000)
Query progress can be monitored at: http://nightly513-unsecure-2.gce.cloudera.com:25000/query_plan?query_id=7146dedb62cb6503:bc403a8500000000
Fetched 0 row(s) in 0.06s

[host-2.x.y.z:21000] > show partitions web_logs_new;
Query: show partitions web_logs_new
+------------+-------+--------+----------+--------------+-------------------+--------+-------------------+---------------------------------------------------------------------------------------------------------+
| date_col | #Rows | #Files | Size | Bytes Cached | Cache Replication | Format | Incremental stats | Location |
+------------+-------+--------+----------+--------------+-------------------+--------+-------------------+---------------------------------------------------------------------------------------------------------+
| 2015-11-18 | -1 | 1 | 112.15KB | NOT CACHED | NOT CACHED | TEXT | false | hdfs://nightly513-unsecure-1.gce.cloudera.com:8020/user/hive/warehouse/web_logs_new/date_col=2015-11-18 |
| 2015-11-19 | 250 | 1 | 98.83KB | NOT CACHED | NOT CACHED | TEXT | false | hdfs://nightly513-unsecure-1.gce.cloudera.com:8020/user/hive/warehouse/web_logs_new/date_col=2015-11-19 |
| 2015-11-20 | 250 | 1 | 101.57KB | NOT CACHED | NOT CACHED | TEXT | false | hdfs://nightly513-unsecure-1.gce.cloudera.com:8020/user/hive/warehouse/web_logs_new/date_col=2015-11-20 |
| 2015-11-21 | 250 | 1 | 82.99KB | NOT CACHED | NOT CACHED | TEXT | false | hdfs://nightly513-unsecure-1.gce.cloudera.com:8020/user/hive/warehouse/web_logs_new/date_col=2015-11-21 |
| Total | 1000 | 4 | 395.54KB | 0B | | | | |
+------------+-------+--------+----------+--------------+-------------------+--------+-------------------+---------------------------------------------------------------------------------------------------------+
Fetched 5 row(s) in 0.01s