Kafka
  1. Kafka
  2. KAFKA-406

Gzipped payload is a fully wrapped Message (with headers), not just payload

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Won't Fix
    • Affects Version/s: 0.7.1
    • Fix Version/s: None
    • Component/s: core
    • Labels:
      None
    • Environment:
      N/A

      Description

      When creating a gzipped MessageSet, the collection of Messages is passed to CompressionUtils.compress(), where each message is serialised [1] into a buffer (not just the payload, the full Message with headers, uncompressed), then gripped, and finally wrapped into another Message [2].

      In other words, the consumer has to unwrap the Message flagged as gzipped, unzip the payload, and unwrap the unzipped payload again as a non-compressed Message.

      Is this double-wrapping the intended behaviour?

      [1] messages.foreach(m => m.serializeTo(messageByteBuffer))

      [2] new Message(outputStream.toByteArray, compressionCodec)

        Activity

          People

          • Assignee:
            Unassigned
            Reporter:
            Lorenzo Alberton
          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development