Uploaded image for project: 'Mesos'
  1. Mesos
  2. MESOS-692

Reservations are not reported in master's state.json

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.23.0
    • Component/s: json api, master
    • Labels:

      Description

      For each resource type, state.json reports the sum of resources across all reservation roles (including unreserved '*'). Operators would love to have visibility into the reserved and unreserved resources on each node and across the cluster, so it makes sense to surface this information in state.json on both the slave and the master.

      This will become extra valuable as we expand the power of reservations with MESOS-2018 and MESOS-1791.

      Old description:
      ===========================
      When you dump state.json from the master, it lists out a single resources dict, e.g.

      >>> state['slaves'][0]
      {..., u'registered_time': 1378851991.87182, u'reregistered_time': 1378917907.3190701, u'id': u'201309042122-1937777162-5050-55755-120', u'resources': {u'mem': 21913, u'disk': 400000, u'cpus': 14, u'ports': u'[31000-32000]'}}
      

      Looking at the code, it looks like last value wins:

      // Returns a JSON object modeled on a Resources.
      JSON::Object model(const Resources& resources)
      {
        JSON::Object object;
      
        foreach (const Resource& resource, resources) {
          switch (resource.type()) {
            case Value::SCALAR:
              object.values[resource.name()] = resource.scalar().value();
              break;
            case Value::RANGES:
              object.values[resource.name()] = stringify(resource.ranges());
              break;
            case Value::SET:
              object.values[resource.name()] = stringify(resource.set());
              break;
            default:
              LOG(FATAL) << "Unexpected Value type: " << resource.type();
              break;
          }
        }
      
        return object;
      }
      

      So for example if you had role * with 15 cores and role "hdfs" with 1 cores, the resource dict might just report 1 core. Instead it should probably aggregate resources by role, and have resources = {'*':

      {ram, cpu, disk}

      , 'hdfs': {ram, cpu, disk}} etc.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                haosdent@gmail.com haosdent
                Reporter:
                wickman Brian Wickman
                Shepherd:
                Adam B
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: