Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-3683

Capacity scheduler LeafQueues maximum capacity calculation issues

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: 0.23.0
    • Fix Version/s: 0.23.1
    • Component/s: mrv2
    • Labels:
      None

      Description

      In the Capacity scheduler if you configure the queues to be hierarchical where you have root -> parent queue -> leaf queue, the leaf queue doesn't take into account its parents maximum capacity when calculate its own maximum capacity, instead it seems to use the parents capacity. Looking at the code its using the parents absoluteCapacity and I think it should be using the parents absoluteMaximumCapacity.

      It also seems to only use the parents capacity in the leaf queues max capacity calculation when the leaf queue has a max capacity configured. If the leaf queues maximum-capacity is not configured, then it can use 100% of the cluster.

      1. MAPREDUCE-3683.patch
        24 kB
        Arun C Murthy
      2. MAPREDUCE-3683.patch
        13 kB
        Arun C Murthy
      3. MAPREDUCE-3683.patch
        4 kB
        Arun C Murthy

        Activity

          People

          • Assignee:
            Arun C Murthy
            Reporter:
            Thomas Graves
          • Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development