Kafka
  1. Kafka
  2. KAFKA-736

Add an option to the 0.8 producer to mimic 0.7 producer behavior

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: 0.8.0
    • Fix Version/s: None
    • Component/s: producer

      Description

      I profiled a producer throughput benchmark between a producer and a remote broker. It turns out that the background send threads spends ~97% of its time waiting to read the acknowledgement from the broker.

      I propose we change the current behavior of request.required.acks=0 to mean no acknowledgement from the broker. This will mimic the 0.7 producer behavior and will enable tuning the producer for very high throughput.

      1. kafka-736-draft.patch
        38 kB
        Neha Narkhede
      2. kafka-736-v1.patch
        32 kB
        Neha Narkhede
      3. kafka-736-v2.patch
        47 kB
        Neha Narkhede
      4. check-message-ordering.py
        2 kB
        Neha Narkhede
      5. kafka-736-v3.patch
        72 kB
        Neha Narkhede
      6. kafka-736-draft-producer-latency-20threads-acks1.out
        2 kB
        Neha Narkhede
      7. kafka-736-v3-producer-latency-20threads-acks1.out
        2 kB
        Neha Narkhede
      8. kafka-736-v4.patch
        48 kB
        Neha Narkhede

        Issue Links

          Activity

          Gavin made changes -
          Link This issue is depended upon by KAFKA-706 [ KAFKA-706 ]
          Gavin made changes -
          Link This issue blocks KAFKA-706 [ KAFKA-706 ]
          Neha Narkhede made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Neha Narkhede made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Resolution Fixed [ 1 ]
          Neha Narkhede made changes -
          Labels p2 replication-performance p1 replication-performance
          Neha Narkhede made changes -
          Attachment kafka-736-v4.patch [ 12570491 ]
          Neha Narkhede made changes -
          Status In Progress [ 3 ] Patch Available [ 10002 ]
          Neha Narkhede made changes -
          Link This issue blocks KAFKA-706 [ KAFKA-706 ]
          Sriram Subramanian made changes -
          Link This issue blocks KAFKA-706 [ KAFKA-706 ]
          Neha Narkhede made changes -
          Neha Narkhede made changes -
          Status Patch Available [ 10002 ] In Progress [ 3 ]
          Neha Narkhede made changes -
          Attachment kafka-736-v3.patch [ 12567777 ]
          Neha Narkhede made changes -
          Attachment check-message-ordering.py [ 12567697 ]
          Neha Narkhede made changes -
          Attachment kafka-736-v2.patch [ 12567201 ]
          Neha Narkhede made changes -
          Attachment kafka-736-v1.patch [ 12567092 ]
          Neha Narkhede made changes -
          Status In Progress [ 3 ] Patch Available [ 10002 ]
          Neha Narkhede made changes -
          Attachment kafka-736-draft.patch [ 12566701 ]
          Neha Narkhede made changes -
          Status Open [ 1 ] In Progress [ 3 ]
          Neha Narkhede made changes -
          Field Original Value New Value
          Labels replication-performance p2 replication-performance
          Neha Narkhede created issue -

            People

            • Assignee:
              Neha Narkhede
              Reporter:
              Neha Narkhede
            • Votes:
              1 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Time Tracking

                Estimated:
                Original Estimate - 24h
                24h
                Remaining:
                Remaining Estimate - 24h
                24h
                Logged:
                Time Spent - Not Specified
                Not Specified

                  Development