Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-7225

Refresh on single partition resets partition's row count to -1

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: Impala 2.10.0, Impala 2.12.0
    • Fix Version/s: Impala 3.1.0
    • Component/s: Catalog
    • Labels:
      None

      Description

      Doing refresh on single partition resets it's row count to -1

       

      [host-2.x.y.z:21000] > show partitions web_logs_new;
      Query: show partitions web_logs_new
      +------------+-------+--------+----------+--------------+-------------------+--------+-------------------+---------------------------------------------------------------------------------------------------------+
      | date_col | #Rows | #Files | Size | Bytes Cached | Cache Replication | Format | Incremental stats | Location |
      +------------+-------+--------+----------+--------------+-------------------+--------+-------------------+---------------------------------------------------------------------------------------------------------+
      | 2015-11-18 | -1 | 1 | 112.15KB | NOT CACHED | NOT CACHED | TEXT | false | hdfs://nightly513-unsecure-1.gce.cloudera.com:8020/user/hive/warehouse/web_logs_new/date_col=2015-11-18 |
      | 2015-11-19 | -1 | 1 | 98.83KB | NOT CACHED | NOT CACHED | TEXT | false | hdfs://nightly513-unsecure-1.gce.cloudera.com:8020/user/hive/warehouse/web_logs_new/date_col=2015-11-19 |
      | 2015-11-20 | -1 | 1 | 101.57KB | NOT CACHED | NOT CACHED | TEXT | false | hdfs://nightly513-unsecure-1.gce.cloudera.com:8020/user/hive/warehouse/web_logs_new/date_col=2015-11-20 |
      | 2015-11-21 | -1 | 1 | 82.99KB | NOT CACHED | NOT CACHED | TEXT | false | hdfs://nightly513-unsecure-1.gce.cloudera.com:8020/user/hive/warehouse/web_logs_new/date_col=2015-11-21 |
      | Total | -1 | 4 | 395.54KB | 0B | | | | |
      +------------+-------+--------+----------+--------------+-------------------+--------+-------------------+---------------------------------------------------------------------------------------------------------+
      Fetched 5 row(s) in 0.01s
      [host-2.x.y.z:21000] > compute stats web_logs_new;
      Query: compute stats web_logs_new
      +------------------------------------------+
      | summary |
      +------------------------------------------+
      | Updated 4 partition(s) and 28 column(s). |
      +------------------------------------------+
      Fetched 1 row(s) in 1.31s
      [nightly513-unsecure-2.gce.cloudera.com:21000] > show partitions web_logs_new;
      Query: show partitions web_logs_new
      +------------+-------+--------+----------+--------------+-------------------+--------+-------------------+---------------------------------------------------------------------------------------------------------+
      | date_col | #Rows | #Files | Size | Bytes Cached | Cache Replication | Format | Incremental stats | Location |
      +------------+-------+--------+----------+--------------+-------------------+--------+-------------------+---------------------------------------------------------------------------------------------------------+
      | 2015-11-18 | 250 | 1 | 112.15KB | NOT CACHED | NOT CACHED | TEXT | false | hdfs://nightly513-unsecure-1.gce.cloudera.com:8020/user/hive/warehouse/web_logs_new/date_col=2015-11-18 |
      | 2015-11-19 | 250 | 1 | 98.83KB | NOT CACHED | NOT CACHED | TEXT | false | hdfs://nightly513-unsecure-1.gce.cloudera.com:8020/user/hive/warehouse/web_logs_new/date_col=2015-11-19 |
      | 2015-11-20 | 250 | 1 | 101.57KB | NOT CACHED | NOT CACHED | TEXT | false | hdfs://nightly513-unsecure-1.gce.cloudera.com:8020/user/hive/warehouse/web_logs_new/date_col=2015-11-20 |
      | 2015-11-21 | 250 | 1 | 82.99KB | NOT CACHED | NOT CACHED | TEXT | false | hdfs://nightly513-unsecure-1.gce.cloudera.com:8020/user/hive/warehouse/web_logs_new/date_col=2015-11-21 |
      | Total | 1000 | 4 | 395.54KB | 0B | | | | |
      +------------+-------+--------+----------+--------------+-------------------+--------+-------------------+---------------------------------------------------------------------------------------------------------+
      Fetched 5 row(s) in 0.01s
      
      
      [host-2.x.y.z:21000] > refresh web_logs_new partition(date_col='2015-11-18');
      Query: refresh web_logs_new partition(date_col='2015-11-18')
      Query submitted at: 2018-06-29 12:53:32 (Coordinator: http://nightly513-unsecure-2.gce.cloudera.com:25000)
      Query progress can be monitored at: http://nightly513-unsecure-2.gce.cloudera.com:25000/query_plan?query_id=7146dedb62cb6503:bc403a8500000000
      Fetched 0 row(s) in 0.06s
      
      [host-2.x.y.z:21000] > show partitions web_logs_new;
      Query: show partitions web_logs_new
      +------------+-------+--------+----------+--------------+-------------------+--------+-------------------+---------------------------------------------------------------------------------------------------------+
      | date_col | #Rows | #Files | Size | Bytes Cached | Cache Replication | Format | Incremental stats | Location |
      +------------+-------+--------+----------+--------------+-------------------+--------+-------------------+---------------------------------------------------------------------------------------------------------+
      | 2015-11-18 | -1 | 1 | 112.15KB | NOT CACHED | NOT CACHED | TEXT | false | hdfs://nightly513-unsecure-1.gce.cloudera.com:8020/user/hive/warehouse/web_logs_new/date_col=2015-11-18 |
      | 2015-11-19 | 250 | 1 | 98.83KB | NOT CACHED | NOT CACHED | TEXT | false | hdfs://nightly513-unsecure-1.gce.cloudera.com:8020/user/hive/warehouse/web_logs_new/date_col=2015-11-19 |
      | 2015-11-20 | 250 | 1 | 101.57KB | NOT CACHED | NOT CACHED | TEXT | false | hdfs://nightly513-unsecure-1.gce.cloudera.com:8020/user/hive/warehouse/web_logs_new/date_col=2015-11-20 |
      | 2015-11-21 | 250 | 1 | 82.99KB | NOT CACHED | NOT CACHED | TEXT | false | hdfs://nightly513-unsecure-1.gce.cloudera.com:8020/user/hive/warehouse/web_logs_new/date_col=2015-11-21 |
      | Total | 1000 | 4 | 395.54KB | 0B | | | | |
      +------------+-------+--------+----------+--------------+-------------------+--------+-------------------+---------------------------------------------------------------------------------------------------------+
      Fetched 5 row(s) in 0.01s
      
       
      
      

        Attachments

          Activity

            People

            • Assignee:
              bharathv Bharath Vissapragada
              Reporter:
              mkempanna Mala Chikka Kempanna
            • Votes:
              1 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: