Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-8045

Reduce log output from container status calls

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 2.10.0, 3.2.0, 3.0.4, 2.8.6, 2.9.3, 3.1.3
    • None
    • None

    Description

      Each time a container's status is returned a log entry is produced in the NM from ContainerManagerImpl. The container status includes the diagnostics field for the container. If the diagnostics field contains an exception, it can appear as if the exception is logged repeatedly every second. The diagnostics message can also span many lines, which puts pressure on the logs and makes it harder to read.

      For example:

      2018-03-17 22:01:11,632 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl: Getting container-status for container_e01_1521323860653_0001_01_000005
      2018-03-17 22:01:11,632 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl: Returning ContainerStatus: [ContainerId: container_e01_1521323860653_0001_01_000005, ExecutionType: GUARANTEED, State: RUNNING, Capability: <memory:1024, vCores:1>, Diagnostics: [2018-03-17 22:01:00.675]Exception from container-launch.
      Container id: container_e01_1521323860653_0001_01_000005
      Exit code: -1
      Exception message: <unknown>
      Shell ouput: <unknown>
      
      
      [2018-03-17 22:01:00.750]Diagnostic message from attempt :
      [2018-03-17 22:01:00.750]Container exited with a non-zero exit code -1.
      , ExitStatus: -1, IP: null, Host: null, ContainerSubState: SCHEDULED]
      

      Attachments

        1. YARN-8045.001.patch
          3 kB
          Craig Condit
        2. YARN-8045.001-branch-2.8.patch
          2 kB
          Eric Badger

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            ccondit-target Craig Condit
            shanekumpf@gmail.com Shane Kumpf
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Issue deployment