Kafka
  1. Kafka
  2. KAFKA-79

Introduce the compression feature in Kafka

    Details

    • Type: New Feature New Feature
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.6
    • Fix Version/s: 0.7
    • Component/s: None
    • Labels:
      None

      Description

      With this feature, we can enable end-to-end block compression in Kafka. The idea is to enable compression on the producer for some or all topics, write the data in compressed format on the server and make the consumers compression aware. The data will be decompressed only on the consumer side. Ideally, there should be a choice of compression codecs to be used by the producer. That means a change to the message header as well as the network byte format. On the consumer side, the state maintenance behavior of the zookeeper consumer changes. For compressed data, the consumed offset will be advanced one compressed message at a time. For uncompressed data, consumed offset will be advanced one message at a time.

        Activity

        No work has yet been logged on this issue.

          People

          • Assignee:
            Neha Narkhede
            Reporter:
            Neha Narkhede
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development