Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-3339

Job is getting hanged indefinitely,if the child processes are killed on the NM. KILL_CONTAINER eventtype is continuosly sent to the containers that are not existing

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: 0.23.0
    • Fix Version/s: 0.23.1
    • Component/s: mrv2
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Fixed MR AM to stop considering node blacklisting after the number of nodes blacklisted crosses a threshold.

      Description

      I have only one NM running.
      I have submitted a job and all the child processes on the NM got killed continuosly.This made the Job to hang indefinitely.

      In the NM logs it is logging WARN message :org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl: Event EventType: KILL_CONTAINER sent to absent container container_1320301910500_0004_01_001359

      1. MR3339_v2.txt
        27 kB
        Siddharth Seth
      2. MR3339_v1.txt
        18 kB
        Siddharth Seth
      3. MAPREDUCE-3339-20111220.txt
        28 kB
        Vinod Kumar Vavilapalli

        Issue Links

          Activity

          Ramgopal N created issue -
          Arun C Murthy made changes -
          Field Original Value New Value
          Target Version/s 0.23.1 [ 12318883 ]
          Priority Major [ 3 ] Blocker [ 1 ]
          Siddharth Seth made changes -
          Assignee Siddharth Seth [ sseth ]
          Siddharth Seth made changes -
          Attachment MR3339_v1.txt [ 12506389 ]
          Siddharth Seth made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Siddharth Seth made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Siddharth Seth made changes -
          Attachment MR3339_v2.txt [ 12507620 ]
          Siddharth Seth made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Vinod Kumar Vavilapalli made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Fix Version/s 0.23.1 [ 12318883 ]
          Vinod Kumar Vavilapalli made changes -
          Attachment MAPREDUCE-3339-20111220.txt [ 12508138 ]
          Vinod Kumar Vavilapalli made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Hadoop Flags Reviewed [ 10343 ]
          Vinod Kumar Vavilapalli made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Release Note Fixed MR AM to stop considering node blacklisting after the number of nodes blacklisted crosses a threshold.
          Resolution Fixed [ 1 ]
          Arun C Murthy made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Zhijie Shen made changes -
          Link This issue is related to MAPREDUCE-5559 [ MAPREDUCE-5559 ]

            People

            • Assignee:
              Siddharth Seth
              Reporter:
              Ramgopal N
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development