[KAFKA-5584] Incorrect log size for topics larger than 2 GB

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 0.11.0.0
• Fix Version/s: 0.11.0.1
• Component/s: log
• Labels: reliability
• Assignee: Unassigned
• Reporter: Gregor Uhlenheuer
• Created: 12/Jul/17 07:23
• Updated: 14/Jul/17 10:46
• Resolved: 13/Jul/17 03:46

Description

The size of a Log is calculated incorrectly due to an Integer overflow. For large topics (larger than 2 GB) this value overflows.

This is easily observable in the reported metrics values of the path log.Log.partition.*.topic.<topicname>.Size (see attached screenshot).

Moreover I think this breaks the size-based retention (via log.retention.bytes and retention.bytes) of large topics as well.

I am not sure on the recommended workflow, should I open a pull request on github with a fix?

Issue Links

• links to: GitHub Pull Request #3521