Details
-
Bug
-
Status: Closed
-
Major
-
Resolution: Fixed
-
None
-
None
-
None
Description
Jason Darrell Lowe found lot of lingering oozie launcher AMs which hung around 10 mins after successful completion and then jvm killed due to timeout. This happens when user code launches non-daemon threads, but mapreduce code does a System.exit() to avoid this from happening. Oozie does need the LauncherSecurityManager to capture exit codes when user applications do System.exit() . But it should reset it back to the originial security manager in the finally block so that AM code can proceed as expected in uber mode.
2016-05-12 12:26:48,514 INFO [Thread-107] org.apache.hadoop.util.ExitUtil: Exiting with status 1 2016-05-12 12:26:48,514 ERROR [Thread-107] org.apache.hadoop.yarn.YarnUncaughtExceptionHandler: Thread Thread[Thread-107,5,main] threw an Exception. java.lang.SecurityException: Intercepted System.exit(1) at org.apache.oozie.action.hadoop.LauncherSecurityManager.checkExit(LauncherMapper.java:637) at java.lang.Runtime.exit(Runtime.java:107) at java.lang.System.exit(System.java:971) at org.apache.hadoop.util.ExitUtil.terminate(ExitUtil.java:133) at org.apache.hadoop.util.ExitUtil.terminate(ExitUtil.java:170) at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.exitMRAppMaster(MRAppMaster.java:655) at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.shutDownJob(MRAppMaster.java:636) at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$JobFinishEventHandler$1.run(MRAppMaster.java:676)