[KYLIN-4459] Continuous print warning log-DFSInputStream has been closed already

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: v4.0.0-alpha
• Component/s: Storage - Parquet
• Assignee: Wang Rupeng
• Reporter: xuekaiqi
• Created: 17/Apr/20 10:07
• Updated: 13/Sep/20 12:21
• Resolved: 12/Sep/20 01:33

Description

when starting kylin with debug tomcat mode, we can see these logs.

2020-03-12 10:17:06,082 ERROR [pool-12-thread-1] curator.CuratorScheduler:205 : Node(127.0.0.1) job server state conflict. Is ZK leader: true; Is active job server: false
2020-03-12 10:17:06,830 WARN [Curator-LeaderSelector-0] hdfs.DFSClient:669 : DFSInputStream has been closed already
2020-03-12 10:17:06,830 WARN [Curator-LeaderSelector-0] hdfs.DFSClient:669 : DFSInputStream has been closed already
2020-03-12 10:17:06,830 WARN [Curator-LeaderSelector-0] hdfs.DFSClient:669 : DFSInputStream has been closed already
2020-03-12 10:17:08,717 WARN [Curator-LeaderSelector-0] hdfs.DFSClient:669 : DFSInputStream has been closed already
2020-03-12 10:17:08,717 WARN [Curator-LeaderSelector-0] hdfs.DFSClient:669 : DFSInputStream has been closed already
2020-03-12 10:17:08,717 WARN [Curator-LeaderSelector-0] hdfs.DFSClient:669 : DFSInputStream has been closed already
2020-03-12 10:17:12,936 WARN [Curator-LeaderSelector-0] hdfs.DFSClient:669 : DFSInputStream has been closed already
2020-03-12 10:17:12,936 WARN [Curator-LeaderSelector-0] hdfs.DFSClient:669 : DFSInputStream has been closed already
2020-03-12 10:17:12,936 WARN [Curator-LeaderSelector-0] hdfs.DFSClient:669 : DFSInputStream has been closed already
2020-03-12 10:17:14,152 WARN [Curator-LeaderSelector-0] hdfs.DFSClient:669 : DFSInputStream has been closed already
2020-03-12 10:17:14,152 WARN [Curator-LeaderSelector-0] hdfs.DFSClient:669 : DFSInputStream has been closed already
2020-03-12 10:17:14,152 WARN [Curator-LeaderSelector-0] hdfs.DFSClient:669 : DFSInputStream has been closed already
2020-03-12 10:17:17,603 WARN [Curator-LeaderSelector-0] hdfs.DFSClient:669 : DFSInputStream has been closed already
2020-03-12 10:17:17,603 WARN [Curator-LeaderSelector-0] hdfs.DFSClient:669 : DFSInputStream has been closed already
2020-03-12 10:17:17,603 WARN [Curator-LeaderSelector-0] hdfs.DFSClient:669 : DFSInputStream has been closed already
2020-03-12 10:17:17,603 INFO [Curator-LeaderSelector-0] threadpool.DefaultScheduler:166 : Finishing resume all running jobs.
2020-03-12 10:17:17,603 INFO [Curator-LeaderSelector-0] threadpool.DefaultScheduler:170 : Fetching jobs every 30 seconds
2020-03-12 10:17:17,603 INFO [Curator-LeaderSelector-0] threadpool.DefaultScheduler:180 : Creating fetcher pool instance:2094578449

Upgrade hadoop version to 2.7.2 can fix it, but need more test in case some unpredictable problems

https://community.cloudera.com/t5/Support-Questions/DFSInputStream-has-been-closed-already/td-p/125487