Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-3802

log mtimes reset on broker restart

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.9.0.1
    • Fix Version/s: 0.10.0.1
    • Component/s: None
    • Labels:
      None

      Description

      Folks over in http://mail-archives.apache.org/mod_mbox/kafka-users/201605.mbox/%3CCAO8=cz0RAGJAd1Acx4GEqcWJ+RKd1gMdAvkjWytwthKSzfgb5Q@mail.gmail.com%3E are commenting about this issue.

      In 0.9, any data log file that was on
      disk before the broker has it's mtime modified to the time of the broker
      restart.

      This causes problems with log retention, as all the files then look like
      they contain recent data to kafka. We use the default log retention of 7
      days, but if all the files are touched at the same time, this can cause us
      to retain up to 2 weeks of log data, which can fill up our disks.

      This happens most of the time, but seemingly not all. We have seen broker restarts where mtimes were not changed.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                moritz Moritz Siuts
                Reporter:
                ottomata Andrew Otto
              • Votes:
                0 Vote for this issue
                Watchers:
                8 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: