Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-1195

Task resource utilization reporting for profiling and scheduling

    Details

    • Type: New Feature New Feature
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Duplicate
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      We can make TaskTracker reports its usage on CPU, memory, bandwidth to JobTracker. JobTracker can use the information for scheduling tasks and profiling jobs.

      One way to do this is to first make ProcfsProcessTree to collect the utilization information (CPU, mem...) and write the information in TaskTrackerStatus.taskReports and send them with the heartbeats. Then we can aggregate these information in JobInProgress to do job profiling and scheduling.

        Issue Links

          Activity

          No work has yet been logged on this issue.

            People

            • Assignee:
              Unassigned
              Reporter:
              Scott Chen
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development