Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-2124

Add job counters for measuring time spent in three different phases in reducers

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: 0.22.0
    • Fix Version/s: None
    • Component/s: jobtracker
    • Labels:
      None

      Description

      We currently have SLOTS_MILLIS_REDUCES which measures the total slot time of reducer.
      It will be useful if we have

      SLOTS_MILLIS_REDUCES_COPY
      SLOTS_MILLIS_REDUCES_SORT
      SLOTS_MILLIS_REDUCES_REDUCE
      

      which measures three different phases of a reducer.
      This will help us identify the bottleneck of the reducers.

        Attachments

        1. test-result.txt
          54 kB
          Scott Chen
        2. MAPREDUCE-2124-v2.txt
          4 kB
          Scott Chen
        3. MAPREDUCE-2124.txt
          2 kB
          Scott Chen
        4. MAPREDUCE-2124.3.txt
          4 kB
          Scott Chen

          Activity

            People

            • Assignee:
              schen Scott Chen
              Reporter:
              schen Scott Chen
            • Votes:
              0 Vote for this issue
              Watchers:
              11 Start watching this issue

              Dates

              • Created:
                Updated: