Kafka
  1. Kafka
  2. KAFKA-456

ProducerSendThread calls ListBuffer.size a whole bunch. That is a O(n) operation

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 0.8.0
    • Fix Version/s: 0.8.0
    • Component/s: core
    • Labels:
    • Environment:
      NA

      Description

      Hi all,

      So there are various statements throughout the async code that call 'events.size', mostly for debugging purposes.
      Problem is that this call is O, so it could add up if the batch size is high. (it's a ListBuffer)

      I see this in at least ProducerSendThread (x4), likely more. Will factor this out myself soon when I start hacking on the project, just wanted to put this somewhere.

      1. KAFKA-456.patch
        0.8 kB
        David Arthur

        Activity

        Hide
        David Arthur added a comment -

        I looked around in other parts of the code for similar situations (lots of calls to ListBuffer#size), didn't see any that were unnecessary.

        In the worst case, this saves two calls to size(); in the common case, it saves one call.

        Show
        David Arthur added a comment - I looked around in other parts of the code for similar situations (lots of calls to ListBuffer#size), didn't see any that were unnecessary. In the worst case, this saves two calls to size(); in the common case, it saves one call.
        Hide
        David Arthur added a comment -

        Only compatible with 0.8

        Show
        David Arthur added a comment - Only compatible with 0.8
        Hide
        Jun Rao added a comment -

        Thanks for the patch. Committed to 0.8.

        This performance issue seems to have been fixed in scala 2.9.1.
        https://issues.scala-lang.org/browse/SI-4933

        Show
        Jun Rao added a comment - Thanks for the patch. Committed to 0.8. This performance issue seems to have been fixed in scala 2.9.1. https://issues.scala-lang.org/browse/SI-4933
        Hide
        Jay Kreps added a comment -

        Why not just make this an ArrayBuffer so that there are 0 O operations? I suspect the ListBuffer thing was just a misunderstanding of scala collections underlying implementation (i.e. ListBuffer != ArrayList).

        Show
        Jay Kreps added a comment - Why not just make this an ArrayBuffer so that there are 0 O operations? I suspect the ListBuffer thing was just a misunderstanding of scala collections underlying implementation (i.e. ListBuffer != ArrayList).

          People

          • Assignee:
            David Arthur
            Reporter:
            Matthew Rathbone
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Time Tracking

              Estimated:
              Original Estimate - 2h
              2h
              Remaining:
              Remaining Estimate - 2h
              2h
              Logged:
              Time Spent - Not Specified
              Not Specified

                Development