Uploaded image for project: 'Mesos'
  1. Mesos
  2. MESOS-9691

Quota headroom calculation is off when subroles are involved.

    XMLWordPrintableJSON

    Details

    • Target Version/s:
    • Sprint:
      Resource Mgmt RI12 Sp 43
    • Story Points:
      3

      Description

      Quota "availableHeadroom" calculation:

      https://github.com/apache/mesos/blob/6276f7e73b0dbe7df49a7315cd1b83340d66f4ea/src/master/allocator/mesos/hierarchical.cpp#L1751-L1754

      is off when subroles are involved.

      Specifically, in the formula

      available headroom = total resources - allocated resources - (total reservations - allocated reservations) - unallocated revocable resources
      

      The "allocated resources" part is hierarchical-aware and aggregate that across all roles, thus allocations to subroles will be counted multiple times (in the case of "a/b", once for "a" and once for "a/b"). Looks like due to the presence of `INTERNAL` node, `roleSorter->allocationScalarQuantities(role)` is not hierarchical. Thus this is not an issue.

      (If role `a/b` consumes 1cpu and `a` consumes 1cpu, if we query `roleSorter->allocationScalarQuantities("a");` It will return 1cpu, which is correct. In the sorter, there are four nodes, root, `a` (internal, 1cpu), `a/.` (leaf, 1cpu), `a/b` (leaf, 1cpu). Query `a` will return `a/.`)

      The "total reservations" is correct, since today it is "flat" (reservations made to "a/b" are not counted to "a"). Thus all reservations are only counted once – which is the correct semantic here. However, once we fix MESOS-9688 (which likely requires reservation tracking to be hierarchical-aware), we need to ensure that the accounting is still correct.

      The "allocated reservations" is hierarchical-aware, thus overlap accounting would occur. Similar to the `"allocated resources"` above, this is also not an issue at the moment.

      Basically, when calculating the available headroom, we need to ensure "single-counting". Ideally, we only need to look at the root's consumptions.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                bmahler Benjamin Mahler
                Reporter:
                mzhu Meng Zhu
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: