Kafka
  1. Kafka
  2. KAFKA-118

Producer performance tool should use the new blocking async producer instead of the sleep timeout hack

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.7
    • Fix Version/s: 0.7
    • Component/s: None
    • Labels:
      None

      Description

      Before the blocking async producer feature, if you add new data to the async producer queue faster than it can handle, then it throws QueueFullException. To get around this in the producer performance tool, we had added a custom sleep option. This ends up reducing the producer throughput and also slows down the system test. After KAFKA-74, we have the option of blocking on the async producer, instead of throwing QueueFullException. This is useful to run stress and performance tests.

      1. KAFKA-118.patch
        5 kB
        Neha Narkhede

        Activity

        Hide
        Neha Narkhede added a comment -

        Modified the perf and system tools to use the blocking async producer, so that all system tests run successfully and fast.

        Show
        Neha Narkhede added a comment - Modified the perf and system tools to use the blocking async producer, so that all system tests run successfully and fast.
        Hide
        Jun Rao added a comment -

        +1 for the patch. The code in ProducerPerformance probably can be refactored better. Will create a separate jira.

        Show
        Jun Rao added a comment - +1 for the patch. The code in ProducerPerformance probably can be refactored better. Will create a separate jira.

          People

          • Assignee:
            Unassigned
            Reporter:
            Neha Narkhede
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development