Uploaded image for project: 'Mesos'
  1. Mesos
  2. MESOS-2332

Report per-container metrics for network bandwidth throttling

    XMLWordPrintableJSON

Details

    • Twitter Mesos Q1 Sprint 2, Twitter Mesos Q1 Sprint 3, Twitter Mesos Q1 Sprint 4, Twitter Mesos Q1 Sprint 5, Twitter Mesos Q1 Sprint 6, Twitter Q2 Sprint 1 - 4/13, Twitter Q2 Sprint 2
    • 5

    Description

      Export metrics from the network isolation to identify scope and duration of container throttling.

      Packet loss can be identified from the overlimits and requeues fields of the htb qdisc report for the virtual interface, e.g.

      $ tc -s -d qdisc show dev mesos19223
qdisc pfifo_fast 0: root refcnt 2 bands 3 priomap  1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
 Sent 158213287452 bytes 1030876393 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc ingress ffff: parent ffff:fff1 ----------------
 Sent 119381747824 bytes 1144549901 pkt (dropped 2044879, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0

      Note that since a packet can be examined multiple times before transmission, overlimits can exceed total packets sent.

      Add to the port_mapping isolator usage() and the container statistics protobuf. Carefully consider the naming (esp tx/rx) + commenting of the protobuf fields so it's clear what these represent and how they are different to the existing dropped packet counts from the network stack.

      Attachments

        Issue Links

          depends upon

          Bug - A problem which impairs or prevents the functions of the product. MESOS-2665 Fix queuing discipline wrapper in linux/routing/queueing

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. MESOS-2750 Extend queueing discipline wrappers to expose network isolator statistics

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. MESOS-2752 Add HTB queueing discipline wrapper class

          • Major - Major loss of function.
          • Resolved
          relates to

          Task - A task that needs to be done. MESOS-1598 Add advanced shaping controls to routing library

          • Major - Major loss of function.
          • Resolved

          Activity

            People

              pbrett Paul Brett
              pbrett Paul Brett
              Jie Yu Jie Yu
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: