Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-10564

Continuous logging about deleting obsolete state directories

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 2.6.0
    • Fix Version/s: 2.7.0, 2.6.1
    • Component/s: streams
    • Labels:

      Description

      The internal process which automatically cleans up obsolete task state directories was modified in https://issues.apache.org/jira/browse/KAFKA-6647. The current logic in 2.6 is to remove all files from the task directory except the .lock file:

      https://github.com/apache/kafka/blob/2.6/streams/src/main/java/org/apache/kafka/streams/processor/internals/StateDirectory.java#L335

      However, the directory is only removed in its entirely for a manual cleanup:

      https://github.com/apache/kafka/blob/2.6/streams/src/main/java/org/apache/kafka/streams/processor/internals/StateDirectory.java#L349-L353

      The result of this is that Streams will continue revisiting this directory and trying to clean it up, since it determines what to clean based on last-modification time of the task directory (which is now no longer deleted during the automatic cleanup). So a user will see log messages like:
      stream-thread [app-c2d773e6-8ac3-4435-9777-378e0ec0ab82-CleanupThread] Deleting obsolete state directory 0_8 for task 0_8 as 60600001ms has elapsed (cleanup delay is 600000ms)
      repeated again and again.

      This issue doesn't seem to break anything - it's more about avoiding unnecessary logging and cleaning up empty task directories.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              mikebin Michael Bingham
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: