Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-3341

Enhance logging of initalized queue limit values

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.23.0
    • Fix Version/s: 0.23.1
    • Component/s: mrv2
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Currently the RM log shows only a partial set of the limits that are configured when a queue is initialized / reinitialized.

      For example, this is what is currently shown in the RM log for an initialized queue:

      1. <datestamp> INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.LeafQueue: Initializing
        default, capacity=0.25, asboluteCapacity=0.25, maxCapacity=25.0, asboluteMaxCapacity=0.25, userLimit=100,
        userLimitFactor=20.0, maxApplications=2500, maxApplicationsPerUser=50000, state=RUNNING,
        acls=ADMINISTER_QUEUE:SUBMIT_JOB:*ADMINISTER_JOBS:

      Breaking down the line above, shows:

      capacity=0.25
      asboluteCapacity=0.25
      maxCapacity=25.0
      asboluteMaxCapacity=0.25
      userLimit=100
      userLimitFactor=20.0
      maxApplications=2500
      maxApplicationsPerUser=50000

      It might be nice if we could include more information such as maxActiveApplications, maxActiveApplicationsPerUser, utilization, and usedCapacity along with information on how each of these is computed (i.e. formulae used) (Thanks to Phil Su for requesting this).

        Attachments

          Activity

            People

            • Assignee:
              anupamseth Anupam Seth
              Reporter:
              anupamseth Anupam Seth
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: