Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-5260

Job failed because of JvmManager running into inconsistent state

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 1.1.2
    • Fix Version/s: 1.2.1
    • Component/s: tasktracker
    • Labels:
      None

      Description

      In our cluster, jobs failed due to randomly task initialization failed because of JvmManager running into inconsistent state and TaskTracker failed to exit:

      java.lang.Throwable: Child Error
      at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:271)
      Caused by: java.lang.NullPointerException
      at org.apache.hadoop.mapred.JvmManager$JvmManagerForType.getDetails(JvmManager.java:402)
      at org.apache.hadoop.mapred.JvmManager$JvmManagerForType.reapJvm(JvmManager.java:387)
      at org.apache.hadoop.mapred.JvmManager$JvmManagerForType.access$000(JvmManager.java:192)
      at org.apache.hadoop.mapred.JvmManager.launchJvm(JvmManager.java:125)
      at org.apache.hadoop.mapred.TaskRunner.launchJvmAndWait(TaskRunner.java:292)
      at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:251)

      -------
      java.lang.Throwable: Child Error
      at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:271)
      Caused by: java.lang.NullPointerException
      at org.apache.hadoop.mapred.JvmManager$JvmManagerForType.getDetails(JvmManager.java:402)
      at org.apache.hadoop.mapred.JvmManager$JvmManagerForType.reapJvm(JvmManager.java:387)
      at org.apache.hadoop.mapred.JvmManager$JvmManagerForType.access$000(JvmManager.java:192)
      at org.apache.hadoop.mapred.JvmManager.launchJvm(JvmManager.java:125)
      at org.apache.hadoop.mapred.TaskRunner.launchJvmAndWait(TaskRunner.java:292)
      at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:251)

        Issue Links

          Activity

          zhaoyunjiong created issue -
          zhaoyunjiong made changes -
          Field Original Value New Value
          Attachment MAPREDUCE-5260.patch [ 12583802 ]
          zhaoyunjiong made changes -
          Attachment MAPREDUCE-5260-branch-1.1.patch [ 12583808 ]
          zhaoyunjiong made changes -
          Attachment MAPREDUCE-5260.patch [ 12583802 ]
          Benoy Antony made changes -
          Assignee zhaoyunjiong [ zhaoyunjiong ]
          Benoy Antony made changes -
          Link This issue is related to MAPREDUCE-1397 [ MAPREDUCE-1397 ]
          zhaoyunjiong made changes -
          Status Open [ 1 ] In Progress [ 3 ]
          Arun C Murthy made changes -
          Status In Progress [ 3 ] Resolved [ 5 ]
          Fix Version/s 1.2.1 [ 12324149 ]
          Fix Version/s 1.1.3 [ 12324131 ]
          Resolution Fixed [ 1 ]
          Matt Foley made changes -
          Status Resolved [ 5 ] Closed [ 6 ]

            People

            • Assignee:
              zhaoyunjiong
              Reporter:
              zhaoyunjiong
            • Votes:
              1 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development