[KAFKA-406] Gzipped payload is a fully wrapped Message (with headers), not just payload

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Won't Fix
• Affects Version/s: 0.7.1
• Component/s: core
• Environment: N/A
• Assignee: Unassigned
• Reporter: Lorenzo Alberton
• Created: 18/Jul/12 17:37
• Updated: 07/Feb/15 23:41
• Resolved: 07/Feb/15 23:41

Description

When creating a gzipped MessageSet, the collection of Messages is passed to CompressionUtils.compress(), where each message is serialised [1] into a buffer (not just the payload, the full Message with headers, uncompressed), then gripped, and finally wrapped into another Message [2].

In other words, the consumer has to unwrap the Message flagged as gzipped, unzip the payload, and unwrap the unzipped payload again as a non-compressed Message.

Is this double-wrapping the intended behaviour?

[1] messages.foreach(m => m.serializeTo(messageByteBuffer))

[2] new Message(outputStream.toByteArray, compressionCodec)