[HADOOP-6820] RunJar fails executing thousands JARs within single JVM with error "Too many open files"

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 0.20.2
• Component/s: util

Environment

OS:Linux, Linux-user limited by maximum number of open file descriptors (for example: ulimit -n shows 1024)

• Assignee: Unassigned
• Reporter: Alexander Bondar
• Created: 11/Jun/10 12:28
• Updated: 30/Jul/14 20:15
• Resolved: 30/Jul/14 20:15

Description

According to Sun JVM (up to 7) bug http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4167874 - The JarFile objects created by sun.net.www.protocol.jar.JarFileFactory never get garbage collected, even if the classloader that loaded them goes away.

So, if linux-user has limitation on maximum number of open file descriptors (for example: ulimit -n shows 1024) and performs RunJar.main(...) over thousands of JARs that include other nested JARs (also loaded by ClassLoader) within single JVM, RunJar.main(...) throws following exception: java.lang.RuntimeException: java.io.FileNotFoundException: /some-file.txt (Too many open files)