[HIVE-19646] Filesystem closed error in HiveProtoLoggingHook

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 3.1.0, 3.0.1
• Component/s: HiveServer2
• Assignee: Harish JP
• Reporter: Harish JP
• Created: 22/May/18 10:03
• Updated: 31/Jul/18 01:33
• Resolved: 14/Jun/18 11:12

Description

Exception in proto logging hook on secure cluster.

2018-05-18T04:48:01,136 ERROR [Hive Hook Proto Log Writer 0]: hooks.HiveProtoLoggingHook (:()) - Error writing proto message for query hive_20180518043717_ca3ab4df-6cab-4920-aa44-2340ae246ad2, eventType: QUERY_SUBMITTED:
java.io.IOException: Filesystem closed
 at org.apache.hadoop.hdfs.DFSClient.checkOpen(DFSClient.java:472) ~[hadoop-hdfs-client-3.0.0.3.0.0.0-1298.jar:?]
 at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:1652) ~[hadoop-hdfs-client-3.0.0.3.0.0.0-1298.jar:?]
 at org.apache.hadoop.hdfs.DistributedFileSystem$29.doCall(DistributedFileSystem.java:1569) ~[hadoop-hdfs-client-3.0.0.3.0.0.0-1298.jar:?]
 at org.apache.hadoop.hdfs.DistributedFileSystem$29.doCall(DistributedFileSystem.java:1566) ~[hadoop-hdfs-client-3.0.0.3.0.0.0-1298.jar:?]
 at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) ~[hadoop-common-3.0.0.3.0.0.0-1298.jar:?]
 at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1581) ~[hadoop-hdfs-client-3.0.0.3.0.0.0-1298.jar:?]
 at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1734) ~[hadoop-common-3.0.0.3.0.0.0-1298.jar:?]
 at org.apache.hadoop.hive.ql.hooks.DatePartitionedLogger.getPathForDate(DatePartitionedLogger.java:89) ~[hive-exec-3.0.0.3.0.0.0-1298.jar:3.0.0.3.0.0.0-1298]
 at org.apache.hadoop.hive.ql.hooks.DatePartitionedLogger.getWriter(DatePartitionedLogger.java:73) ~[hive-exec-3.0.0.3.0.0.0-1298.jar:3.0.0.3.0.0.0-1298]
 at org.apache.hadoop.hive.ql.hooks.HiveProtoLoggingHook$EventLogger.writeEvent(HiveProtoLoggingHook.java:283) ~[hive-exec-3.0.0.3.0.0.0-1298.jar:3.0.0.3.0.0.0-1298]
 at org.apache.hadoop.hive.ql.hooks.HiveProtoLoggingHook$EventLogger.lambda$generateEvent$1(HiveProtoLoggingHook.java:274) ~[hive-exec-3.0.0.3.0.0.0-1298.jar:3.0.0.3.0.0.0-1298]
 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0_161]
 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0_161]
 at java.lang.Thread.run(Thread.java:748) [?:1.8.0_161]