Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-2469

Task counters should also report the total heap usage of the task

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.23.0
    • Fix Version/s: 0.23.0
    • Component/s: task
    • Labels:
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Task attempt's total heap usage gets recorded and published via counters as COMMITTED_HEAP_BYTES.
    • Tags:
      total-heap, task-counters

      Description

      Currently, the task counters report VSS and RSS usage of the task. The task counter should also report the total heap usage of the task also. The task might be configured with a max heap size of M but the task's total heap usage might only be H, where H < M. In such a case, knowing only M doesn't provide a complete picture of the task's memory usage.

      1. 2469.v0.1.patch
        2 kB
        Ravi Gummadi
      2. mapreduce-2469-v1.1.patch
        12 kB
        Amar Kamat
      3. mapreduce-2469-v1.2.patch
        13 kB
        Amar Kamat
      4. mapreduce-2469-v1.3.patch
        13 kB
        Amar Kamat
      5. mapreduce-2469-v1.4.patch
        13 kB
        Amar Kamat

        Issue Links

          Activity

          No work has yet been logged on this issue.

            People

            • Assignee:
              Amar Kamat
              Reporter:
              Amar Kamat
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development