Uploaded image for project: 'Mesos'
  1. Mesos
  2. MESOS-2332

Report per-container metrics for network bandwidth throttling

    Details

    • Target Version/s:
    • Sprint:
      Twitter Mesos Q1 Sprint 2, Twitter Mesos Q1 Sprint 3, Twitter Mesos Q1 Sprint 4, Twitter Mesos Q1 Sprint 5, Twitter Mesos Q1 Sprint 6, Twitter Q2 Sprint 1 - 4/13, Twitter Q2 Sprint 2
    • Story Points:
      5

      Description

      Export metrics from the network isolation to identify scope and duration of container throttling.

      Packet loss can be identified from the overlimits and requeues fields of the htb qdisc report for the virtual interface, e.g.

      $ tc -s -d qdisc show dev mesos19223
      qdisc pfifo_fast 0: root refcnt 2 bands 3 priomap  1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
       Sent 158213287452 bytes 1030876393 pkt (dropped 0, overlimits 0 requeues 0)
       backlog 0b 0p requeues 0
      qdisc ingress ffff: parent ffff:fff1 ----------------
       Sent 119381747824 bytes 1144549901 pkt (dropped 2044879, overlimits 0 requeues 0)
       backlog 0b 0p requeues 0
      

      Note that since a packet can be examined multiple times before transmission, overlimits can exceed total packets sent.

      Add to the port_mapping isolator usage() and the container statistics protobuf. Carefully consider the naming (esp tx/rx) + commenting of the protobuf fields so it's clear what these represent and how they are different to the existing dropped packet counts from the network stack.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                pbrett Paul Brett
                Reporter:
                pbrett Paul Brett
                Shepherd:
                Jie Yu
              • Votes:
                0 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: