Kafka
  1. Kafka
  2. KAFKA-118

Producer performance tool should use the new blocking async producer instead of the sleep timeout hack

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.7
    • Fix Version/s: 0.7
    • Component/s: None
    • Labels:
      None

      Description

      Before the blocking async producer feature, if you add new data to the async producer queue faster than it can handle, then it throws QueueFullException. To get around this in the producer performance tool, we had added a custom sleep option. This ends up reducing the producer throughput and also slows down the system test. After KAFKA-74, we have the option of blocking on the async producer, instead of throwing QueueFullException. This is useful to run stress and performance tests.

      1. KAFKA-118.patch
        5 kB
        Neha Narkhede

        Activity

        Neha Narkhede created issue -
        Hide
        Neha Narkhede added a comment -

        Modified the perf and system tools to use the blocking async producer, so that all system tests run successfully and fast.

        Show
        Neha Narkhede added a comment - Modified the perf and system tools to use the blocking async producer, so that all system tests run successfully and fast.
        Neha Narkhede made changes -
        Field Original Value New Value
        Attachment KAFKA-118.patch [ 12491419 ]
        Neha Narkhede made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Hide
        Jun Rao added a comment -

        +1 for the patch. The code in ProducerPerformance probably can be refactored better. Will create a separate jira.

        Show
        Jun Rao added a comment - +1 for the patch. The code in ProducerPerformance probably can be refactored better. Will create a separate jira.
        Jun Rao made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]
        Transition Time In Source Status Execution Times Last Executer Last Execution Date
        Open Open Patch Available Patch Available
        2m 47s 1 Neha Narkhede 24/Aug/11 01:15
        Patch Available Patch Available Resolved Resolved
        20d 1h 15m 1 Jun Rao 13/Sep/11 02:31

          People

          • Assignee:
            Unassigned
            Reporter:
            Neha Narkhede
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development