Hadoop Common
  1. Hadoop Common
  2. HADOOP-4906

TaskTracker running out of memory after running several tasks

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: 0.19.0
    • Fix Version/s: 0.19.1
    • Component/s: None
    • Labels:
      None
    • Release Note:
      Fix the tasktracker for OOM exception by sharing the jobconf properties across tasks of the same job. Earlier a new instance was held for each task. With this fix, the job level configuration properties are shared across tasks of the same job.

      Description

      Looks like the TaskTracker isn't cleaning up correctly after completed/failed tasks, I suspect that the JobConfs aren't being deallocated. Eventually the TaskTracker runs out of memory after running several tasks.

      1. 4906_v5.patch
        2 kB
        Sharad Agarwal
      2. 4906_v4.patch
        1.0 kB
        Sharad Agarwal
      3. 4906_v3.patch
        4 kB
        Sharad Agarwal
      4. 4906_v2.patch
        4 kB
        Sharad Agarwal
      5. 4906_v1.patch
        1 kB
        Sharad Agarwal

        Activity

        Owen O'Malley made changes -
        Component/s mapred [ 12310690 ]
        Nigel Daley made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Sharad Agarwal made changes -
        Release Note Fix the tasktracker for OOM exception by sharing the jobconf properties across tasks of the same job. Earlier a new instance was held for each task. With this fix, the job level configuration properties are shared across tasks of the same job.
        Arun C Murthy made changes -
        Resolution Fixed [ 1 ]
        Fix Version/s 0.20.0 [ 12313438 ]
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Fix Version/s 0.19.1 [ 12313473 ]
        Sharad Agarwal made changes -
        Attachment 4906_v5.patch [ 12397934 ]
        Sharad Agarwal made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Sharad Agarwal made changes -
        Attachment 4906_v4.patch [ 12397665 ]
        Sharad Agarwal made changes -
        Status Patch Available [ 10002 ] Open [ 1 ]
        Sharad Agarwal made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Sharad Agarwal made changes -
        Attachment 4906_v3.patch [ 12397114 ]
        Sharad Agarwal made changes -
        Attachment 4906_v2.patch [ 12396843 ]
        Sharad Agarwal made changes -
        Attachment 4906_v1.patch [ 12396669 ]
        Devaraj Das made changes -
        Field Original Value New Value
        Assignee Sharad Agarwal [ sharadag ]
        Arun C Murthy created issue -

          People

          • Assignee:
            Sharad Agarwal
            Reporter:
            Arun C Murthy
          • Votes:
            1 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development