Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-598

decouple fetch size from max message size

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Blocker
    • Resolution: Won't Fix
    • 0.8.0
    • None
    • core

    Description

      Currently, a consumer has to set fetch size larger than the max message size. This increases the memory footprint on the consumer, especially when a large number of topic/partition is subscribed. By decoupling the fetch size from max message size, we can use a smaller fetch size for normal consumption and when hitting a large message (hopefully rare), we automatically increase fetch size to max message size temporarily.

      Attachments

        1. KAFKA-598-v1.patch
          16 kB
          Joel Jacob Koshy
        2. KAFKA-598-v2.patch
          16 kB
          Joel Jacob Koshy
        3. KAFKA-598-v3.patch
          44 kB
          Joel Jacob Koshy

        Activity

          People

            jjkoshy Joel Jacob Koshy
            junrao Jun Rao
            Votes:
            3 Vote for this issue
            Watchers:
            8 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: