[MAPREDUCE-7307] Potential thread leak in LocatedFileStatusFetcher

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 3.3.1
• Component/s: job submission
• Assignee: Zhihua Deng
• Reporter: Zhihua Deng
• Created: 18/Nov/20 04:29
• Updated: 24/Nov/20 01:43
• Resolved: 23/Nov/20 16:34

Description

We see that when using LocatedFileStatusFetcher to get file infos In parallel, if the listStatus thread is interrupted,  the  executor service in LocatedFileStatusFetcher is left unclosed,  the thread stack will like this:

"GetFileInfo #63" #125 daemon prio=5 os_prio=0 tid=0x00007f6198106800 nid=0x881 waiting on condition [0x00007f60d9fde000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x0000000082e810a8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)

This caused by if condition.await() throws InterruptedException,  the method `shutDownNow` for the executor service would not be called as a result, should move such resource releasing call into the finally block.

Issue Links

• links to: PR