[KAFKA-6834] log cleaner should handle the case when the size of a message set is larger than the max message size

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.0.0
• Assignee: Rajini Sivaram
• Reporter: Jun Rao
• Reviewer: Jason Gustafson
• Created: 27/Apr/18 21:51
• Updated: 13/Feb/20 05:06
• Resolved: 09/May/18 11:22

Description

In KAFKA-5316, we added the logic to allow a message (set) larger than the per topic message size to be written to the log during log cleaning. However, the buffer size in the log cleaner is still bounded by the per topic message size. This can cause the log cleaner to die and cause the broker to run out of disk space.

 

Issue Links

• relates to: KAFKA-6762 log-cleaner thread terminates due to java.lang.IllegalStateException
• links to: GitHub Pull Request #4953
• mentioned in: Page Loading...