[HIVE-26466] NullPointerException on HiveQueryLifeTimeHook:checkAndRollbackCTAS

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 4.0.0-alpha-2
• Fix Version/s: 4.0.0-alpha-2
• Component/s: HiveServer2
• Labels: pull-request-available
• Target Version/s: 4.0.0-alpha-2
• Assignee: Butao Zhang
• Reporter: Butao Zhang
• Created: 12/Aug/22 03:17
• Updated: 16/Nov/22 13:50
• Resolved: 17/Aug/22 01:04

Description

Step to repro using beeline client:

1. Create database testdb, and table testdb.test

create database testdb;
create table testdb.test(id int);

 

2. Drop database testdb. This will fail because a table exists in testdb

drop database testdb;

 

3. NullPointerException on HiveQueryLifeTimeHook:checkAndRollbackCTAS

WARN  : Failed when invoking query after execution hook
java.lang.NullPointerException: null
        at org.apache.hadoop.hive.ql.HiveQueryLifeTimeHook.checkAndRollbackCTAS(HiveQueryLifeTimeHook.java:69) ~[hive-exec-4.0.0-alpha-2-SNAPSHOT.jar:4.0.0-alpha-2-SNAPSHOT]
        at org.apache.hadoop.hive.ql.HiveQueryLifeTimeHook.afterExecution(HiveQueryLifeTimeHook.java:62) ~[hive-exec-4.0.0-alpha-2-SNAPSHOT.jar:4.0.0-alpha-2-SNAPSHOT]
        at org.apache.hadoop.hive.ql.HookRunner.runAfterExecutionHook(HookRunner.java:185) ~[hive-exec-4.0.0-alpha-2-SNAPSHOT.jar:4.0.0-alpha-2-SNAPSHOT]
        at org.apache.hadoop.hive.ql.Executor.cleanUp(Executor.java:525) ~[hive-exec-4.0.0-alpha-2-SNAPSHOT.jar:4.0.0-alpha-2-SNAPSHOT]
        at org.apache.hadoop.hive.ql.Executor.execute(Executor.java:118) ~[hive-exec-4.0.0-alpha-2-SNAPSHOT.jar:4.0.0-alpha-2-SNAPSHOT]
        at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:355) ~[hive-exec-4.0.0-alpha-2-SNAPSHOT.jar:4.0.0-alpha-2-SNAPSHOT]
        at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:205) ~[hive-exec-4.0.0-alpha-2-SNAPSHOT.jar:4.0.0-alpha-2-SNAPSHOT]
        at org.apache.hadoop.hive.ql.Driver.run(Driver.java:154) ~[hive-exec-4.0.0-alpha-2-SNAPSHOT.jar:4.0.0-alpha-2-SNAPSHOT]
        at org.apache.hadoop.hive.ql.Driver.run(Driver.java:149) ~[hive-exec-4.0.0-alpha-2-SNAPSHOT.jar:4.0.0-alpha-2-SNAPSHOT]
        at org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:185) ~[hive-exec-4.0.0-alpha-2-SNAPSHOT.jar:4.0.0-alpha-2-SNAPSHOT]
        at org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:233) ~[hive-service-4.0.0-alpha-2-SNAPSHOT.jar:4.0.0-alpha-2-SNAPSHOT]
        at org.apache.hive.service.cli.operation.SQLOperation.access$500(SQLOperation.java:88) ~[hive-service-4.0.0-alpha-2-SNAPSHOT.jar:4.0.0-alpha-2-SNAPSHOT]
        at org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork$1.run(SQLOperation.java:336) ~[hive-service-4.0.0-alpha-2-SNAPSHOT.jar:4.0.0-alpha-2-SNAPSHOT]
        at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_112]
        at javax.security.auth.Subject.doAs(Subject.java:422) ~[?:1.8.0_112]
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1878) ~[hadoop-common-3.3.1.jar:?]
        at org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork.run(SQLOperation.java:356) ~[hive-service-4.0.0-alpha-2-SNAPSHOT.jar:4.0.0-alpha-2-SNAPSHOT]
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_112]
        at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_112]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) ~[?:1.8.0_112]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) ~[?:1.8.0_112]
        at java.lang.Thread.run(Thread.java:745) [?:1.8.0_112]
INFO  : Completed executing command(queryId=hive_20220812105501_c1beb730-1ded-4135-b1d7-6d0648926bfc); Time taken: 48.879 seconds
Error: Error while compiling statement: FAILED: Execution Error, return code 40000 from org.apache.hadoop.hive.ql.ddl.DDLTask. InvalidOperationException(message:Database testdb ty. One or more tables exist.) (state=08S01,code=40000)
 

 

This change is introduced by HIVE-26414, I think we shoud check NULL(queryPlan.getQueryProperties()) before invoke QueryProperties:isCTAS

Issue Links

• relates to: HIVE-26414 Aborted/Cancelled CTAS operations must initiate cleanup of uncommitted data
• links to: GitHub Pull Request #3515