Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-12788

Setting hive.optimize.union.remove to TRUE will break UNION ALL with aggregate functions

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 1.1.1
    • 2.0.0
    • Hive
    • None

    Description

      See the test case below:

      0: jdbc:hive2://localhost:10000/default> create table test (a int);
      
      0: jdbc:hive2://localhost:10000/default> insert overwrite table test values (1);
      
      0: jdbc:hive2://localhost:10000/default> set hive.optimize.union.remove=true;
      No rows affected (0.01 seconds)
      
      0: jdbc:hive2://localhost:10000/default> set hive.mapred.supports.subdirectories=true;
      No rows affected (0.007 seconds)
      
      0: jdbc:hive2://localhost:10000/default> SELECT COUNT(1) FROM test UNION ALL SELECT COUNT(1) FROM test;
      +----------+--+
      | _u1._c0  |
      +----------+--+
      +----------+--+
      

      UNION ALL without COUNT function will work as expected:

      0: jdbc:hive2://localhost:10000/default> select * from test UNION ALL SELECT * FROM test;
      +--------+--+
      | _u1.a  |
      +--------+--+
      | 1      |
      | 1      |
      +--------+--+
      

      Run the same query without setting hive.mapred.supports.subdirectories and hive.optimize.union.remove to true will give correct result:

      0: jdbc:hive2://localhost:10000/default> set hive.optimize.union.remove;
      +-----------------------------------+--+
      |                set                |
      +-----------------------------------+--+
      | hive.optimize.union.remove=false  |
      +-----------------------------------+--+
      
      0: jdbc:hive2://localhost:10000/default> SELECT COUNT(1) FROM test UNION ALL SELECT COUNT(1) FROM test;
      +----------+--+
      | _u1._c0  |
      +----------+--+
      | 1        |
      | 1        |
      +----------+--+
      

      Attachments

        1. HIVE-12788.1.patch
          47 kB
          Chaoyu Tang
        2. HIVE-12788.patch
          46 kB
          Chaoyu Tang
        3. HIVE-12788-branch-1.patch
          47 kB
          Chaoyu Tang

        Issue Links

          Activity

            People

              ctang Chaoyu Tang
              ericlin Eric Lin
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: