Hive
  1. Hive
  2. HIVE-6998

Select query can only support maximum 128 distinct expressions

    Details

      Description

      Select query can only support maximum 128 distinct expressions. Otherwise, you will be thrown ArrayIndexOutOfBoundsException. For a query like:
      select count(distinct c1), count(distinct c2), count(distinct c3), count(distinct c4), count(distinct c5), count(distinct c6), ...., count(distinct c128), count(distinct c129) from tbl_129columns;

      you will get error like:

      java.lang.Exception: java.lang.RuntimeException: Hive Runtime Error while closing operators
              at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:354)
      Caused by: java.lang.RuntimeException: Hive Runtime Error while closing operators
              at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.close(ExecMapper.java:260)
              at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:57)
              at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:430)
              at org.apache.hadoop.mapred.MapTask.run(MapTask.java:366)
              at org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:223)
              at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
              at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
              at java.util.concurrent.FutureTask.run(FutureTask.java:138)
              at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
              at java.lang.Thread.run(Thread.java:695)
      Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: org.apache.hadoop.hive.ql.metadata.HiveException: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.ArrayIndexOutOfBoundsException: -128
              at org.apache.hadoop.hive.ql.exec.GroupByOperator.closeOp(GroupByOperator.java:1141)
              at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:579)
              at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:591)
              at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:591)
              at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:591)
              at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.close(ExecMapper.java:227)
              ... 10 more
      Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.ArrayIndexOutOfBoundsException: -128
              at org.apache.hadoop.hive.ql.exec.GroupByOperator.flush(GroupByOperator.java:1099)
              at org.apache.hadoop.hive.ql.exec.GroupByOperator.closeOp(GroupByOperator.java:1138)
              ... 15 more
      Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.ArrayIndexOutOfBoundsException: -128
              at org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.processOp(ReduceSinkOperator.java:327)
              at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:796)
              at org.apache.hadoop.hive.ql.exec.GroupByOperator.forward(GroupByOperator.java:1064)
              at org.apache.hadoop.hive.ql.exec.GroupByOperator.flush(GroupByOperator.java:1082)
              ... 16 more
      Caused by: java.lang.ArrayIndexOutOfBoundsException: -128
              at java.util.ArrayList.get(ArrayList.java:324)
              at org.apache.hadoop.hive.serde2.binarysortable.BinarySortableSerDe.serialize(BinarySortableSerDe.java:838)
              at org.apache.hadoop.hive.serde2.binarysortable.BinarySortableSerDe.serialize(BinarySortableSerDe.java:600)
              at org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.toHiveKey(ReduceSinkOperator.java:401)
              at org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.processOp(ReduceSinkOperator.java:320)
              ... 19 more
      

        Activity

        No work has yet been logged on this issue.

          People

          • Assignee:
            Unassigned
            Reporter:
            Chaoyu Tang
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:

              Development