Hadoop Common
  1. Hadoop Common
  2. HADOOP-5285

JobTracker hangs for long periods of time

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: 0.19.1
    • Fix Version/s: 0.19.2, 0.20.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      On one of the larger clusters of 2000 nodes, JT hanged quite often, sometimes for times in the order of 10-15 minutes and once for one and a half hours. The stack trace shows that JobInProgress.obtainTaskCleanupTask() is waiting for lock on JobInProgress object which JobInProgress.initTasks() is holding for a long time waiting for DFS operations.

      1. trace.txt
        255 kB
        Vinod Kumar Vavilapalli
      2. 5285.patch
        13 kB
        Devaraj Das
      3. 5285.1.patch
        13 kB
        Vinod Kumar Vavilapalli

        Issue Links

          Activity

          Vinod Kumar Vavilapalli created issue -
          Vinod Kumar Vavilapalli made changes -
          Field Original Value New Value
          Attachment trace.txt [ 12400514 ]
          Devaraj Das made changes -
          Assignee Devaraj Das [ devaraj ]
          Devaraj Das made changes -
          Attachment 5285.patch [ 12400601 ]
          Vinod Kumar Vavilapalli made changes -
          Attachment 5285.1.patch [ 12400736 ]
          Devaraj Das made changes -
          Fix Version/s 0.21.0 [ 12313563 ]
          Hadoop Flags [Reviewed]
          Resolution Fixed [ 1 ]
          Status Open [ 1 ] Resolved [ 5 ]
          Vinod Kumar Vavilapalli made changes -
          Link This issue incorporates HADOOP-4375 [ HADOOP-4375 ]
          Devaraj Das made changes -
          Affects Version/s 0.19.2 [ 12313650 ]
          Devaraj Das made changes -
          Affects Version/s 0.20.0 [ 12313438 ]
          Affects Version/s 0.19.1 [ 12313473 ]
          Affects Version/s 0.19.2 [ 12313650 ]
          Fix Version/s 0.19.2 [ 12313650 ]
          Tsz Wo Nicholas Sze made changes -
          Link This issue is related to HADOOP-5483 [ HADOOP-5483 ]
          Nigel Daley made changes -
          Fix Version/s 0.21.0 [ 12313563 ]
          Nigel Daley made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Owen O'Malley made changes -
          Component/s mapred [ 12310690 ]

            People

            • Assignee:
              Devaraj Das
              Reporter:
              Vinod Kumar Vavilapalli
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development