Chukwa
  1. Chukwa
  2. CHUKWA-392

fair queuing for threads in collector

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Data Collection
    • Labels:
      None

      Description

      Right now, we don't write data in order received. This means that backpressure doesn't work right.

        Activity

        Hide
        Ari Rabkin added a comment -

        Patch ready for review, but I want to test at scale before committing.

        Show
        Ari Rabkin added a comment - Patch ready for review, but I want to test at scale before committing.
        Hide
        Ari Rabkin added a comment -

        OK. Tested and works ok on 200 nodes each at 100 kb/sec. Performance indistinguishable from baseline. I propose to commit this.

        Show
        Ari Rabkin added a comment - OK. Tested and works ok on 200 nodes each at 100 kb/sec. Performance indistinguishable from baseline. I propose to commit this.
        Hide
        Ari Rabkin added a comment -

        At higher data rates per node, this patch is actually a big win. I'm seeing drastic reduction in retransmits, and a consequent big improvement in goodput.

        Show
        Ari Rabkin added a comment - At higher data rates per node, this patch is actually a big win. I'm seeing drastic reduction in retransmits, and a consequent big improvement in goodput.
        Hide
        Eric Yang added a comment -

        +1 looks good.

        Show
        Eric Yang added a comment - +1 looks good.
        Hide
        Ari Rabkin added a comment -

        I just committed this.

        Show
        Ari Rabkin added a comment - I just committed this.
        Hide
        Hudson added a comment -

        Integrated in Chukwa-trunk #145 (See http://hudson.zones.apache.org/hudson/job/Chukwa-trunk/145/)
        . FIFO queueing of threads in collector.

        Show
        Hudson added a comment - Integrated in Chukwa-trunk #145 (See http://hudson.zones.apache.org/hudson/job/Chukwa-trunk/145/ ) . FIFO queueing of threads in collector.

          People

          • Assignee:
            Ari Rabkin
            Reporter:
            Ari Rabkin
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development