Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-3779

Create hard and soft limits for job counters

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Won't Fix
    • Affects Version/s: 0.23.0, 1.0.0
    • Fix Version/s: None
    • Component/s: jobtracker, tasktracker
    • Labels:
      None

      Description

      The mapreduce.job.counters.limit is not overridable at the job level. While it is necessary to limit the number of counters to reduce overhead, there are times when exceeding the limit is required. Currently, the only solution is to increase the limit cluster wide.

      I would like to see a soft limit set in the mapred-site.xml that can be overridden at the job level, in addition to the hard limit that exists today.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              dave.shine Dave Shine
            • Votes:
              1 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: