1. Kafka
  2. KAFKA-79

Introduce the compression feature in Kafka


    • Type: New Feature New Feature
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.6
    • Fix Version/s: 0.7
    • Component/s: None
    • Labels:


      With this feature, we can enable end-to-end block compression in Kafka. The idea is to enable compression on the producer for some or all topics, write the data in compressed format on the server and make the consumers compression aware. The data will be decompressed only on the consumer side. Ideally, there should be a choice of compression codecs to be used by the producer. That means a change to the message header as well as the network byte format. On the consumer side, the state maintenance behavior of the zookeeper consumer changes. For compressed data, the consumed offset will be advanced one compressed message at a time. For uncompressed data, consumed offset will be advanced one message at a time.


        Neha Narkhede created issue -
        Alan Cabrera made changes -
        Field Original Value New Value
        Workflow jira [ 12625066 ] no-reopen-closed, patch-avail [ 12626217 ]
        Neha Narkhede made changes -
        Status Open [ 1 ] Resolved [ 5 ]
        Assignee Neha Narkhede [ nehanarkhede ]
        Resolution Fixed [ 1 ]
        Tony Stevenson made changes -
        Workflow no-reopen-closed, patch-avail [ 12626217 ] Apache Kafka Workflow [ 13052658 ]
        Tony Stevenson made changes -
        Workflow Apache Kafka Workflow [ 13052658 ] no-reopen-closed, patch-avail [ 13054929 ]


          • Assignee:
            Neha Narkhede
            Neha Narkhede
          • Votes:
            0 Vote for this issue
            2 Start watching this issue


            • Created: