[KAFKA-3802] log mtimes reset on broker restart

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.9.0.1
• Fix Version/s: 0.10.0.1
• Assignee: Moritz Siuts
• Reporter: Andrew Otto
• Created: 07/Jun/16 19:47
• Updated: 06/Jul/16 15:16
• Resolved: 06/Jul/16 15:14

Description

Folks over in http://mail-archives.apache.org/mod_mbox/kafka-users/201605.mbox/%3CCAO8=[e-mail]%3E are commenting about this issue.

In 0.9, any data log file that was on
disk before the broker has it's mtime modified to the time of the broker
restart.

This causes problems with log retention, as all the files then look like
they contain recent data to kafka. We use the default log retention of 7
days, but if all the files are touched at the same time, this can cause us
to retain up to 2 weeks of log data, which can fill up our disks.

This happens most of the time, but seemingly not all. We have seen broker restarts where mtimes were not changed.

Issue Links

• is related to: KAFKA-1379 Partition reassignment resets clock for time-based retention
• links to: GitHub Pull Request #1497