Uploaded image for project: 'Oozie'
  1. Oozie
  2. OOZIE-3307

[core] Limit heap usage of LauncherAM

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Critical
    • Resolution: Fixed
    • 5.0.0
    • 5.1.0
    • core
    • None

    Description

      Application application_1531909575787_0039 failed 2 times due to AM Container for appattempt_1531909575787_0039_000002 exited with exitCode: -103
      Failing this attempt.Diagnostics: Container [pid=11516,containerID=container_1531909575787_0039_02_000001] is running beyond virtual memory limits. Current usage: 469.8 MB of 2 GB physical memory used; 10.0 GB of 10 GB virtual memory used. Killing container.
      Dump of the process-tree for container_1531909575787_0039_02_000001 :

      • PID PPID PGRPID SESSID CMD_NAME USER_MODE_TIME(MILLIS) SYSTEM_TIME(MILLIS) VMEM_USAGE(BYTES) RSSMEM_USAGE(PAGES) FULL_CMD_LINE
      • 11516 11514 11516 11516 (bash) 1 3 115863552 682 /bin/bash -c /usr/lib/jvm/java-openjdk/bin/java -Dlog4j.configuration=container-log4j.properties -Dlog4j.debug=true -Dyarn.app.container.log.dir=/var/log/hadoop-yarn/containers/application_1531909575787_0039/container_1531909575787_0039_02_000001 -Dyarn.app.container.log.filesize=1048576 -Dhadoop.root.logger=INFO,CLA -Dhadoop.root.logfile=syslog -Dsubmitter.user=dev org.apache.oozie.action.hadoop.LauncherAM 1>/var/log/hadoop-yarn/containers/application_1531909575787_0039/container_1531909575787_0039_02_000001/stdout 2>/var/log/hadoop-yarn/containers/application_1531909575787_0039/container_1531909575787_0039_02_000001/stderr
      • 11755 11516 11516 11516 (java) 1142 71 10658242560 119576 /usr/lib/jvm/java-openjdk/bin/java -Dlog4j.configuration=container-log4j.properties -Dlog4j.debug=true -Dyarn.app.container.log.dir=/var/log/hadoop-yarn/containers/application_1531909575787_0039/container_1531909575787_0039_02_000001 -Dyarn.app.container.log.filesize=1048576 -Dhadoop.root.logger=INFO,CLA -Dhadoop.root.logfile=syslog -Dsubmitter.user=dev org.apache.oozie.action.hadoop.LauncherAM
        Container killed on request. Exit code is 143
        Container exited with a non-zero exit code 143
        For more detailed output, check the application tracking page: http://ip-10-20-201-36.us-gov-west-1.compute.internal:8088/cluster/app/application_1531909575787_0039 Then click on links to logs of each attempt.
        . Failing the application.

      Attachments

        1. OOZIE-3307.amend.001.patch
          3 kB
          Andras Piros
        2. OOZIE-3307.006.patch
          23 kB
          Andras Piros
        3. OOZIE-3307.005.patch
          27 kB
          Andras Piros
        4. OOZIE-3307.004.patch
          29 kB
          Andras Piros
        5. OOZIE-3307.003.patch
          28 kB
          Andras Piros
        6. OOZIE-3307.002.patch
          29 kB
          Andras Piros
        7. OOZIE-3307.001.patch
          27 kB
          Andras Piros

        Issue Links

          Activity

            People

              andras.piros Andras Piros
              sabir1 Sabir Naikwadi
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: