Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-3341

Enhance logging of initalized queue limit values

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.23.0
    • Fix Version/s: 0.23.1
    • Component/s: mrv2
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Currently the RM log shows only a partial set of the limits that are configured when a queue is initialized / reinitialized.

      For example, this is what is currently shown in the RM log for an initialized queue:

      1. <datestamp> INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.LeafQueue: Initializing
        default, capacity=0.25, asboluteCapacity=0.25, maxCapacity=25.0, asboluteMaxCapacity=0.25, userLimit=100,
        userLimitFactor=20.0, maxApplications=2500, maxApplicationsPerUser=50000, state=RUNNING,
        acls=ADMINISTER_QUEUE:SUBMIT_JOB:*ADMINISTER_JOBS:

      Breaking down the line above, shows:

      capacity=0.25
      asboluteCapacity=0.25
      maxCapacity=25.0
      asboluteMaxCapacity=0.25
      userLimit=100
      userLimitFactor=20.0
      maxApplications=2500
      maxApplicationsPerUser=50000

      It might be nice if we could include more information such as maxActiveApplications, maxActiveApplicationsPerUser, utilization, and usedCapacity along with information on how each of these is computed (i.e. formulae used) (Thanks to Phil Su for requesting this).

        Activity

          People

          • Assignee:
            Anupam Seth
            Reporter:
            Anupam Seth
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development