Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-1195

Task resource utilization reporting for profiling and scheduling

    Details

    • Type: New Feature
    • Status: Resolved
    • Priority: Major
    • Resolution: Duplicate
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      We can make TaskTracker reports its usage on CPU, memory, bandwidth to JobTracker. JobTracker can use the information for scheduling tasks and profiling jobs.

      One way to do this is to first make ProcfsProcessTree to collect the utilization information (CPU, mem...) and write the information in TaskTrackerStatus.taskReports and send them with the heartbeats. Then we can aggregate these information in JobInProgress to do job profiling and scheduling.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                schen Scott Chen
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: