Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-1482

Better handling of task diagnostic information stored in the TaskInProgress

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.21.0
    • Component/s: jobtracker
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Limit the size of diagnostics-string and state-string shipped as part of task status. This will help keep the JobTracker's memory usage under control. Diagnostic string and state string are capped to 1024 chars.

      Description

      Task diagnostic information can be very large at times eating up Jobtracker's memory. There should be some way to avoid storing large error strings in JobTracker.

      1. mapreduce-1482-v1.7.patch
        10 kB
        Amar Kamat
      2. mapreduce-1482-v1.3.patch
        10 kB
        Amar Kamat
      3. mapreduce-1482-v1.2.patch
        9 kB
        Amar Kamat
      4. mapreduce-1482-v1.1.patch
        9 kB
        Amar Kamat

        Activity

        Tom White made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Tom White made changes -
        Fix Version/s 0.21.0 [ 12314045 ]
        Fix Version/s 0.22.0 [ 12314184 ]
        Amar Kamat made changes -
        Release Note Limit the size of diagnostics-string and state-string shipped as part of task status. This will help keep the JobTracker's memory usage under control. Diagnostic string and state string are capped to 1024 chars.
        Tsz Wo Nicholas Sze made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Fix Version/s 0.22.0 [ 12314184 ]
        Resolution Fixed [ 1 ]
        Tsz Wo Nicholas Sze made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Amar Kamat made changes -
        Attachment mapreduce-1482-v1.7.patch [ 12438124 ]
        Amar Kamat made changes -
        Status Patch Available [ 10002 ] Open [ 1 ]
        Tsz Wo Nicholas Sze made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Amar Kamat made changes -
        Attachment mapreduce-1482-v1.3.patch [ 12437703 ]
        Amar Kamat made changes -
        Status Patch Available [ 10002 ] Open [ 1 ]
        Tsz Wo Nicholas Sze made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Amar Kamat made changes -
        Attachment mapreduce-1482-v1.2.patch [ 12437456 ]
        Tsz Wo Nicholas Sze made changes -
        Hadoop Flags [Reviewed]
        Assignee Amar Kamat [ amar_kamat ]
        Amar Kamat made changes -
        Field Original Value New Value
        Attachment mapreduce-1482-v1.1.patch [ 12437145 ]
        Amar Kamat created issue -

          People

          • Assignee:
            Amar Kamat
            Reporter:
            Amar Kamat
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development