Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-1195

Task resource utilization reporting for profiling and scheduling

    Details

    • Type: New Feature New Feature
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Duplicate
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      We can make TaskTracker reports its usage on CPU, memory, bandwidth to JobTracker. JobTracker can use the information for scheduling tasks and profiling jobs.

      One way to do this is to first make ProcfsProcessTree to collect the utilization information (CPU, mem...) and write the information in TaskTrackerStatus.taskReports and send them with the heartbeats. Then we can aggregate these information in JobInProgress to do job profiling and scheduling.

        Issue Links

          Activity

          Hide
          Arun C Murthy added a comment -

          How is this different from MAPREDUCE-220 ?

          Show
          Arun C Murthy added a comment - How is this different from MAPREDUCE-220 ?
          Hide
          Scott Chen added a comment -

          Hi Arun, Thanks for pointing this out. I will work on that one and move my comments there.

          Show
          Scott Chen added a comment - Hi Arun, Thanks for pointing this out. I will work on that one and move my comments there.
          Hide
          Scott Chen added a comment -

          It is this same as MAPREDUCE-220. Thanks Aruns for pointing this out.

          Show
          Scott Chen added a comment - It is this same as MAPREDUCE-220 . Thanks Aruns for pointing this out.

            People

            • Assignee:
              Unassigned
              Reporter:
              Scott Chen
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development