[HIVE-16426] Query cancel: improve the way to handle files

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 3.0.0
• Assignee: Yongzhi Chen
• Reporter: Yongzhi Chen
• Created: 12/Apr/17 13:07
• Updated: 23/May/18 00:00
• Resolved: 24/Apr/17 14:51

Description

1. Add data structure support to make it is easy to check query cancel status.
2. Handle query cancel more gracefully. Remove possible file leaks caused by query cancel as shown in following stack:

2017-04-11 09:57:30,727 WARN  org.apache.hadoop.hive.ql.exec.Utilities: [HiveServer2-Background-Pool: Thread-149]: Failed to clean-up tmp directories.
java.io.InterruptedIOException: Call interrupted
        at org.apache.hadoop.ipc.Client.call(Client.java:1496)
        at org.apache.hadoop.ipc.Client.call(Client.java:1439)
        at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:230)
        at com.sun.proxy.$Proxy20.delete(Unknown Source)
        at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.delete(ClientNamenodeProtocolTranslatorPB.java:535)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:256)
        at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:104)
        at com.sun.proxy.$Proxy21.delete(Unknown Source)
        at org.apache.hadoop.hdfs.DFSClient.delete(DFSClient.java:2059)
        at org.apache.hadoop.hdfs.DistributedFileSystem$13.doCall(DistributedFileSystem.java:675)
        at org.apache.hadoop.hdfs.DistributedFileSystem$13.doCall(DistributedFileSystem.java:671)
        at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
        at org.apache.hadoop.hdfs.DistributedFileSystem.delete(DistributedFileSystem.java:671)
        at org.apache.hadoop.hive.ql.exec.Utilities.clearWork(Utilities.java:277)
        at org.apache.hadoop.hive.ql.exec.mr.ExecDriver.execute(ExecDriver.java:463)
        at org.apache.hadoop.hive.ql.exec.mr.MapRedTask.execute(MapRedTask.java:142)
        at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:214)
        at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100)
        at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1978)
        at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1691)
        at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1423)
        at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1207)
        at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1202)
        at org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:238)
        at org.apache.hive.service.cli.operation.SQLOperation.access$300(SQLOperation.java:88)
        at org.apache.hive.service.cli.operation.SQLOperation$3$1.run(SQLOperation.java:303)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:415)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1920)
        at org.apache.hive.service.cli.operation.SQLOperation$3.run(SQLOperation.java:316)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
        at java.util.concurrent.FutureTask.run(FutureTask.java:262)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)

3. Add checkpoints to related file operations to improve response time for query cancelling.

Issue Links

• links to: Review