Details
-
Bug
-
Status: Closed
-
Major
-
Resolution: Fixed
-
None
-
None
Description
Consider the following query:
SELECT to_ordered_list(null, null)
Here, even though to_ordered_list() automatically ignores NULL key and/or value, the query throws NPE because of NULL queue handler access in drainQueue():
org.apache.hadoop.hive.ql.metadata.HiveException: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.NullPointerException at org.apache.hadoop.hive.ql.exec.GroupByOperator.closeOp(GroupByOperator.java:1102) at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:683) at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:697) at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:697) at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:697) at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.close(ExecMapper.java:189) at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:61) at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:453) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343) at org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:270) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.NullPointerException at org.apache.hadoop.hive.ql.exec.GroupByOperator.flush(GroupByOperator.java:1060) at org.apache.hadoop.hive.ql.exec.GroupByOperator.closeOp(GroupByOperator.java:1099) ... 14 more Caused by: java.lang.NullPointerException at hivemall.tools.list.UDAFToOrderedList$UDAFToOrderedListEvaluator$QueueAggregationBuffer.drainQueue(UDAFToOrderedList.java:411) at hivemall.tools.list.UDAFToOrderedList$UDAFToOrderedListEvaluator.terminatePartial(UDAFToOrderedList.java:313) at org.apache.hadoop.hive.ql.udf.generic.GenericUDAFEvaluator.evaluate(GenericUDAFEvaluator.java:204) at org.apache.hadoop.hive.ql.exec.GroupByOperator.forward(GroupByOperator.java:1020) at org.apache.hadoop.hive.ql.exec.GroupByOperator.flush(GroupByOperator.java:1043) ... 15 more
Since iterate() accepts NULL inputs, drainQueue() should not fail; it has to return something like empty list.
Attachments
Issue Links
- links to