Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-1739

Collecting CPU and memory usage for MapReduce jobs

    Details

    • Type: New Feature New Feature
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      MAPREDUCE-220 collects CPU and memory usage for each task.
      We can aggregate them to get the information per job. Such information can be used for scheduling, profiling or charging the users based on the resource they consumed.

      Here are some information that should be useful:
      1. Total CPU cycles (# of giga-cycles)
      2. Total Memory occupied time (GB-sec)
      3. Maximum peak memory on one task (GB)
      4. Maximum peak CPU on one task (GHz)

      Thoughts?

        Issue Links

          Activity

          Gavin made changes -
          Link This issue depends upon MAPREDUCE-220 [ MAPREDUCE-220 ]
          Gavin made changes -
          Link This issue depends on MAPREDUCE-220 [ MAPREDUCE-220 ]
          Jeff Hammerbacher made changes -
          Link This issue relates to HADOOP-6755 [ HADOOP-6755 ]
          Scott Chen made changes -
          Field Original Value New Value
          Link This issue depends on MAPREDUCE-220 [ MAPREDUCE-220 ]
          Scott Chen created issue -

            People

            • Assignee:
              Scott Chen
              Reporter:
              Scott Chen
            • Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

              • Created:
                Updated:

                Development