Kafka
  1. Kafka
  2. KAFKA-598

decouple fetch size from max message size

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Blocker Blocker
    • Resolution: Won't Fix
    • Affects Version/s: 0.8.0
    • Fix Version/s: None
    • Component/s: core
    • Labels:

      Description

      Currently, a consumer has to set fetch size larger than the max message size. This increases the memory footprint on the consumer, especially when a large number of topic/partition is subscribed. By decoupling the fetch size from max message size, we can use a smaller fetch size for normal consumption and when hitting a large message (hopefully rare), we automatically increase fetch size to max message size temporarily.

      1. KAFKA-598-v1.patch
        16 kB
        Joel Koshy
      2. KAFKA-598-v2.patch
        16 kB
        Joel Koshy
      3. KAFKA-598-v3.patch
        44 kB
        Joel Koshy

        Activity

        Gwen Shapira made changes -
        Status Open [ 1 ] Resolved [ 5 ]
        Resolution Won't Fix [ 2 ]
        Tony Stevenson made changes -
        Workflow Apache Kafka Workflow [ 13052870 ] no-reopen-closed, patch-avail [ 13055474 ]
        Tony Stevenson made changes -
        Workflow no-reopen-closed, patch-avail [ 12732298 ] Apache Kafka Workflow [ 13052870 ]
        Neha Narkhede made changes -
        Labels p4
        Joel Koshy made changes -
        Attachment KAFKA-598-v3.patch [ 12561081 ]
        Jun Rao made changes -
        Priority Major [ 3 ] Blocker [ 1 ]
        Joel Koshy made changes -
        Attachment KAFKA-598-v2.patch [ 12556060 ]
        Joel Koshy made changes -
        Attachment KAFKA-598-v1.patch [ 12555619 ]
        Joel Koshy made changes -
        Field Original Value New Value
        Assignee Joel Koshy [ jjkoshy ]
        Jun Rao created issue -

          People

          • Assignee:
            Joel Koshy
            Reporter:
            Jun Rao
          • Votes:
            3 Vote for this issue
            Watchers:
            8 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development