Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-12788

Setting hive.optimize.union.remove to TRUE will break UNION ALL with aggregate functions

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.1.1
    • Fix Version/s: 2.0.0
    • Component/s: Hive
    • Labels:
      None

      Description

      See the test case below:

      0: jdbc:hive2://localhost:10000/default> create table test (a int);
      
      0: jdbc:hive2://localhost:10000/default> insert overwrite table test values (1);
      
      0: jdbc:hive2://localhost:10000/default> set hive.optimize.union.remove=true;
      No rows affected (0.01 seconds)
      
      0: jdbc:hive2://localhost:10000/default> set hive.mapred.supports.subdirectories=true;
      No rows affected (0.007 seconds)
      
      0: jdbc:hive2://localhost:10000/default> SELECT COUNT(1) FROM test UNION ALL SELECT COUNT(1) FROM test;
      +----------+--+
      | _u1._c0  |
      +----------+--+
      +----------+--+
      

      UNION ALL without COUNT function will work as expected:

      0: jdbc:hive2://localhost:10000/default> select * from test UNION ALL SELECT * FROM test;
      +--------+--+
      | _u1.a  |
      +--------+--+
      | 1      |
      | 1      |
      +--------+--+
      

      Run the same query without setting hive.mapred.supports.subdirectories and hive.optimize.union.remove to true will give correct result:

      0: jdbc:hive2://localhost:10000/default> set hive.optimize.union.remove;
      +-----------------------------------+--+
      |                set                |
      +-----------------------------------+--+
      | hive.optimize.union.remove=false  |
      +-----------------------------------+--+
      
      0: jdbc:hive2://localhost:10000/default> SELECT COUNT(1) FROM test UNION ALL SELECT COUNT(1) FROM test;
      +----------+--+
      | _u1._c0  |
      +----------+--+
      | 1        |
      | 1        |
      +----------+--+
      

        Attachments

        1. HIVE-12788-branch-1.patch
          47 kB
          Chaoyu Tang
        2. HIVE-12788.patch
          46 kB
          Chaoyu Tang
        3. HIVE-12788.1.patch
          47 kB
          Chaoyu Tang

          Issue Links

            Activity

              People

              • Assignee:
                ctang Chaoyu Tang
                Reporter:
                ericlin Eric Lin
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: