[OOZIE-3307] [core] Limit heap usage of LauncherAM

• Type: Bug
• Status: Closed
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 5.0.0
• Fix Version/s: 5.1.0
• Component/s: core
• Assignee: Andras Piros
• Reporter: Sabir Naikwadi
• Created: 19/Jul/18 13:26
• Updated: 18/Apr/19 07:15
• Resolved: 26/Sep/18 10:26

Description

Application application_1531909575787_0039 failed 2 times due to AM Container for appattempt_1531909575787_0039_000002 exited with exitCode: -103
Failing this attempt.Diagnostics: Container [pid=11516,containerID=container_1531909575787_0039_02_000001] is running beyond virtual memory limits. Current usage: 469.8 MB of 2 GB physical memory used; 10.0 GB of 10 GB virtual memory used. Killing container.
Dump of the process-tree for container_1531909575787_0039_02_000001 :

PID PPID PGRPID SESSID CMD_NAME USER_MODE_TIME(MILLIS) SYSTEM_TIME(MILLIS) VMEM_USAGE(BYTES) RSSMEM_USAGE(PAGES) FULL_CMD_LINE

11516 11514 11516 11516 (bash) 1 3 115863552 682 /bin/bash -c /usr/lib/jvm/java-openjdk/bin/java -Dlog4j.configuration=container-log4j.properties -Dlog4j.debug=true -Dyarn.app.container.log.dir=/var/log/hadoop-yarn/containers/application_1531909575787_0039/container_1531909575787_0039_02_000001 -Dyarn.app.container.log.filesize=1048576 -Dhadoop.root.logger=INFO,CLA -Dhadoop.root.logfile=syslog -Dsubmitter.user=dev org.apache.oozie.action.hadoop.LauncherAM 1>/var/log/hadoop-yarn/containers/application_1531909575787_0039/container_1531909575787_0039_02_000001/stdout 2>/var/log/hadoop-yarn/containers/application_1531909575787_0039/container_1531909575787_0039_02_000001/stderr

11755 11516 11516 11516 (java) 1142 71 10658242560 119576 /usr/lib/jvm/java-openjdk/bin/java -Dlog4j.configuration=container-log4j.properties -Dlog4j.debug=true -Dyarn.app.container.log.dir=/var/log/hadoop-yarn/containers/application_1531909575787_0039/container_1531909575787_0039_02_000001 -Dyarn.app.container.log.filesize=1048576 -Dhadoop.root.logger=INFO,CLA -Dhadoop.root.logfile=syslog -Dsubmitter.user=dev org.apache.oozie.action.hadoop.LauncherAM Container killed on request. Exit code is 143 Container exited with a non-zero exit code 143 For more detailed output, check the application tracking page: http://ip-10-20-201-36.us-gov-west-1.compute.internal:8088/cluster/app/application_1531909575787_0039 Then click on links to logs of each attempt. . Failing the application.

Issue Links

• duplicates: OOZIE-2833 when using uber mode the regex pattern used in the extractHeapSizeMB method does not allow heap sizes specified in bytes.
• is cloned by: OOZIE-3388 When use oozie5.0.0 submit a Action on Yarn,Launcher filed
• links to: ReviewBoard review request