Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-12788

Setting hive.optimize.union.remove to TRUE will break UNION ALL with aggregate functions

Log workAgile BoardRank to TopRank to BottomBulk Copy AttachmentsBulk Move AttachmentsVotersWatch issueWatchersCreate sub-taskConvert to sub-taskMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 1.1.1
    • 2.0.0
    • Hive
    • None

    Description

      See the test case below:

      0: jdbc:hive2://localhost:10000/default> create table test (a int);
      
      0: jdbc:hive2://localhost:10000/default> insert overwrite table test values (1);
      
      0: jdbc:hive2://localhost:10000/default> set hive.optimize.union.remove=true;
      No rows affected (0.01 seconds)
      
      0: jdbc:hive2://localhost:10000/default> set hive.mapred.supports.subdirectories=true;
      No rows affected (0.007 seconds)
      
      0: jdbc:hive2://localhost:10000/default> SELECT COUNT(1) FROM test UNION ALL SELECT COUNT(1) FROM test;
      +----------+--+
      | _u1._c0  |
      +----------+--+
      +----------+--+
      

      UNION ALL without COUNT function will work as expected:

      0: jdbc:hive2://localhost:10000/default> select * from test UNION ALL SELECT * FROM test;
      +--------+--+
      | _u1.a  |
      +--------+--+
      | 1      |
      | 1      |
      +--------+--+
      

      Run the same query without setting hive.mapred.supports.subdirectories and hive.optimize.union.remove to true will give correct result:

      0: jdbc:hive2://localhost:10000/default> set hive.optimize.union.remove;
      +-----------------------------------+--+
      |                set                |
      +-----------------------------------+--+
      | hive.optimize.union.remove=false  |
      +-----------------------------------+--+
      
      0: jdbc:hive2://localhost:10000/default> SELECT COUNT(1) FROM test UNION ALL SELECT COUNT(1) FROM test;
      +----------+--+
      | _u1._c0  |
      +----------+--+
      | 1        |
      | 1        |
      +----------+--+
      

      Attachments

        1. HIVE-12788-branch-1.patch
          47 kB
          Chaoyu Tang
        2. HIVE-12788.1.patch
          47 kB
          Chaoyu Tang
        3. HIVE-12788.patch
          46 kB
          Chaoyu Tang

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            ctang Chaoyu Tang Assign to me
            ericlin Eric Lin
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment