[KAFKA-2213] Log cleaner should write compacted messages using configured compression type

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Assignee: Manikumar
• Reporter: Joel Jacob Koshy
• Reviewer: Joel Jacob Koshy
• Created: 20/May/15 21:15
• Updated: 15/Jul/16 09:57
• Resolved: 15/Jul/16 09:57

Description

In KAFKA-1374 the log cleaner was improved to handle compressed messages. There were a couple of follow-ups from that:

• We write compacted messages using the original compression type in the compressed message-set. We should instead append all retained messages with the configured broker compression type of the topic.
• While compressing messages we should ideally do some batching before compression.
• Investigate the use of the client compressor. (See the discussion in the RBs for KAFKA-1374)

Issue Links

• is duplicated by: KAFKA-3252 compression type for a topic should be used during log compaction
• relates to: KAFKA-3252 compression type for a topic should be used during log compaction