[HIVE-26340] Vectorized PTF operator fails if query has upper case window function

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 4.0.0-alpha-2
• Component/s: Vectorization
• Labels: pull-request-available
• Assignee: Krisztian Kasa
• Reporter: Krisztian Kasa
• Created: 17/Jun/22 10:47
• Updated: 16/Nov/22 13:50
• Resolved: 17/Jun/22 10:59

Description

SELECT ROW_NUMBER() OVER(order by age) AS rn FROM studentnull100;

2022-06-16T14:18:57,728 ERROR [pool-4-thread-1] jdbc.TestDriver: Error while compiling statement: FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.tez.TezTask. Vertex failed, vertexName=Reducer 7, vertexId=vertex_1655217967697_0062_1_01, diagnostics=[Task failed, taskId=task_1655217967697_0062_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Error while running task ( failure ) : attempt_1655217967697_0062_1_01_000000_0:java.lang.RuntimeException: java.lang.NullPointerException
	at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:298)
	at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:252)
	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:374)
	at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:75)
	at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:62)
	at java.base/java.security.AccessController.doPrivileged(Native Method)
	at java.base/javax.security.auth.Subject.doAs(Subject.java:423)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1898)
	at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:62)
	at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:38)
	at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
	at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125)
	at com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:69)
	at com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: java.lang.NullPointerException
	at org.apache.hadoop.hive.ql.plan.VectorPTFDesc.getEvaluator(VectorPTFDesc.java:165)
	at org.apache.hadoop.hive.ql.plan.VectorPTFDesc.getEvaluators(VectorPTFDesc.java:381)
	at org.apache.hadoop.hive.ql.exec.vector.ptf.VectorPTFOperator.initializeOp(VectorPTFOperator.java:317)
	at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:374)
	at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:571)
	at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:523)
	at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:384)
	at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordProcessor.init(ReduceRecordProcessor.java:211)
	at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:268)
	... 16 more

Issue Links

• is related to: HIVE-26274 No vectorization if query has upper case window function
• links to: GitHub Pull Request #3382