Hadoop Common
  1. Hadoop Common
  2. HADOOP-5285

JobTracker hangs for long periods of time

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: 0.19.1
    • Fix Version/s: 0.19.2, 0.20.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      On one of the larger clusters of 2000 nodes, JT hanged quite often, sometimes for times in the order of 10-15 minutes and once for one and a half hours. The stack trace shows that JobInProgress.obtainTaskCleanupTask() is waiting for lock on JobInProgress object which JobInProgress.initTasks() is holding for a long time waiting for DFS operations.

      1. 5285.1.patch
        13 kB
        Vinod Kumar Vavilapalli
      2. 5285.patch
        13 kB
        Devaraj Das
      3. trace.txt
        255 kB
        Vinod Kumar Vavilapalli

        Issue Links

          Activity

          Owen O'Malley made changes -
          Component/s mapred [ 12310690 ]
          Nigel Daley made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Nigel Daley made changes -
          Fix Version/s 0.21.0 [ 12313563 ]
          Tsz Wo Nicholas Sze made changes -
          Link This issue is related to HADOOP-5483 [ HADOOP-5483 ]
          Devaraj Das made changes -
          Affects Version/s 0.20.0 [ 12313438 ]
          Affects Version/s 0.19.1 [ 12313473 ]
          Affects Version/s 0.19.2 [ 12313650 ]
          Fix Version/s 0.19.2 [ 12313650 ]
          Devaraj Das made changes -
          Affects Version/s 0.19.2 [ 12313650 ]
          Vinod Kumar Vavilapalli made changes -
          Link This issue incorporates HADOOP-4375 [ HADOOP-4375 ]
          Devaraj Das made changes -
          Fix Version/s 0.21.0 [ 12313563 ]
          Hadoop Flags [Reviewed]
          Resolution Fixed [ 1 ]
          Status Open [ 1 ] Resolved [ 5 ]
          Vinod Kumar Vavilapalli made changes -
          Attachment 5285.1.patch [ 12400736 ]
          Devaraj Das made changes -
          Attachment 5285.patch [ 12400601 ]
          Devaraj Das made changes -
          Assignee Devaraj Das [ devaraj ]
          Vinod Kumar Vavilapalli made changes -
          Field Original Value New Value
          Attachment trace.txt [ 12400514 ]
          Vinod Kumar Vavilapalli created issue -

            People

            • Assignee:
              Devaraj Das
              Reporter:
              Vinod Kumar Vavilapalli
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development