Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-13200

Aggregation functions returning empty rows on partitioned columns

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.0.0, 2.0.0
    • Fix Version/s: 1.3.0, 2.0.1, 2.1.0
    • Component/s: Physical Optimizer
    • Labels:
      None

      Description

      Running aggregation functions like MAX, MIN, DISTINCT against partitioned columns will return empty rows if table has property: 'skip.header.line.count'='1'
      Reproduce:

      DROP TABLE IF EXISTS test;
      
      CREATE TABLE test (a int) 
      PARTITIONED BY (b int) 
      ROW FORMAT DELIMITED FIELDS TERMINATED BY '|' 
      TBLPROPERTIES('skip.header.line.count'='1');
      
      INSERT OVERWRITE TABLE test PARTITION (b = 1) VALUES (1), (2), (3), (4);
      INSERT OVERWRITE TABLE test PARTITION (b = 2) VALUES (1), (2), (3), (4);
      
      SELECT * FROM test;
      
      SELECT DISTINCT b FROM test;
      SELECT MAX(b) FROM test;
      SELECT DISTINCT a FROM test;
      

      The output:

      0: jdbc:hive2://localhost:10000/default> SELECT * FROM test;
      +---------+---------+--+
      | test.a  | test.b  |
      +---------+---------+--+
      | 2       | 1       |
      | 3       | 1       |
      | 4       | 1       |
      | 2       | 2       |
      | 3       | 2       |
      | 4       | 2       |
      +---------+---------+--+
      6 rows selected (0.631 seconds)
      
      0: jdbc:hive2://localhost:10000/default> SELECT DISTINCT b FROM test;
      +----+--+
      | b  |
      +----+--+
      +----+--+
      No rows selected (47.229 seconds)
      
      0: jdbc:hive2://localhost:10000/default> SELECT MAX(b) FROM test;
      +-------+--+
      |  _c0  |
      +-------+--+
      | NULL  |
      +-------+--+
      1 row selected (49.508 seconds)
      
      0: jdbc:hive2://localhost:10000/default> SELECT DISTINCT a FROM test;
      +----+--+
      | a  |
      +----+--+
      | 2  |
      | 3  |
      | 4  |
      +----+--+
      3 rows selected (46.859 seconds)
      

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                ychena Yongzhi Chen
                Reporter:
                ychena Yongzhi Chen
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: