[HADOOP-15372] Race conditions and possible leaks in the Shell class

• Type: Bug
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Affects Version/s: 2.10.0, 3.2.0
• Assignee: Eric Badger
• Reporter: Miklos Szegedi
• Created: 07/Apr/18 01:40
• Updated: 08/Apr/19 19:35

Description

YARN-5641 introduced some cleanup code in the Shell class. It has a race condition. Shell.runCommand() can be called while/after Shell.getAllShells() returned all the shells to be cleaned up. This new thread can avoid the clean up, so that the process held by it can be leaked causing leaked localized files/etc.

I see another issue as well. Shell.runCommand() has a finally block with a process.destroy(); to clean up. However, the try catch block does not cover all instructions after the process is started, so for example we can exit the thread and leak the process, if timeOutTimer.schedule(timeoutTimerTask, timeOutInterval); causes an exception.

Issue Links

• is broken by: YARN-5641 Localizer leaves behind tarballs after container is complete