Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-13200

Aggregation functions returning empty rows on partitioned columns

Log workAgile BoardRank to TopRank to BottomVotersWatch issueWatchersCreate sub-taskConvert to sub-taskMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.0.0, 2.0.0
    • Fix Version/s: 1.3.0, 2.0.1, 2.1.0
    • Component/s: Physical Optimizer
    • Labels:
      None

      Description

      Running aggregation functions like MAX, MIN, DISTINCT against partitioned columns will return empty rows if table has property: 'skip.header.line.count'='1'
      Reproduce:

      DROP TABLE IF EXISTS test;
      
      CREATE TABLE test (a int) 
      PARTITIONED BY (b int) 
      ROW FORMAT DELIMITED FIELDS TERMINATED BY '|' 
      TBLPROPERTIES('skip.header.line.count'='1');
      
      INSERT OVERWRITE TABLE test PARTITION (b = 1) VALUES (1), (2), (3), (4);
      INSERT OVERWRITE TABLE test PARTITION (b = 2) VALUES (1), (2), (3), (4);
      
      SELECT * FROM test;
      
      SELECT DISTINCT b FROM test;
      SELECT MAX(b) FROM test;
      SELECT DISTINCT a FROM test;
      

      The output:

      0: jdbc:hive2://localhost:10000/default> SELECT * FROM test;
      +---------+---------+--+
      | test.a  | test.b  |
      +---------+---------+--+
      | 2       | 1       |
      | 3       | 1       |
      | 4       | 1       |
      | 2       | 2       |
      | 3       | 2       |
      | 4       | 2       |
      +---------+---------+--+
      6 rows selected (0.631 seconds)
      
      0: jdbc:hive2://localhost:10000/default> SELECT DISTINCT b FROM test;
      +----+--+
      | b  |
      +----+--+
      +----+--+
      No rows selected (47.229 seconds)
      
      0: jdbc:hive2://localhost:10000/default> SELECT MAX(b) FROM test;
      +-------+--+
      |  _c0  |
      +-------+--+
      | NULL  |
      +-------+--+
      1 row selected (49.508 seconds)
      
      0: jdbc:hive2://localhost:10000/default> SELECT DISTINCT a FROM test;
      +----+--+
      | a  |
      +----+--+
      | 2  |
      | 3  |
      | 4  |
      +----+--+
      3 rows selected (46.859 seconds)
      

        Attachments

        1. HIVE-13200.1.patch
          14 kB
          Yongzhi Chen

        Issue Links

          Activity

          $i18n.getText('security.level.explanation', $currentSelection) Viewable by All Users
          Cancel

            People

            • Assignee:
              ychena Yongzhi Chen Assign to me
              Reporter:
              ychena Yongzhi Chen

              Dates

              • Created:
                Updated:
                Resolved:

                Issue deployment