Details
Major Description
Kafka after achieving log.retention.hours dies every time with error.
############################# Log Retention Policy ############################# # The following configurations control the disposal of log segments. The policy can # be set to delete segments after a period of time, or after a given size has accumulated. # A segment will be deleted whenever *either* of these criteria are met. Deletion always happens # from the end of the log. # The minimum age of a log file to be eligible for deletion due to age log.retention.hours=168 # A size-based retention policy for logs. Segments are pruned from the log unless the remaining # segments drop below log.retention.bytes. Functions independently of log.retention.hours. #log.retention.bytes=1073741824 # The maximum size of a log segment file. When this size is reached a new log segment will be created. log.segment.bytes=1073741824 # The interval at which log segments are checked to see if they can be deleted according # to the retention policies log.retention.check.interval.ms=300000
Exception raised:
> C:\root\kafka_2.12-1.1.0\data\__consumer_offsets-3\00000000000000000000.log.swap: Proces nie mo┐e uzyskaŠ dostŕpu do pliku, poniewa┐ jest on u┐ywany przez inny proces. at sun.nio.fs.WindowsException.translateToIOException(Unknown Source) at sun.nio.fs.WindowsException.rethrowAsIOException(Unknown Source) at sun.nio.fs.WindowsFileCopy.move(Unknown Source) at sun.nio.fs.WindowsFileSystemProvider.move(Unknown Source) at java.nio.file.Files.move(Unknown Source) at org.apache.kafka.common.utils.Utils.atomicMoveWithFallback(Utils.java:697) at org.apache.kafka.common.record.FileRecords.renameTo(FileRecords.java:212) at kafka.log.LogSegment.changeFileSuffixes(LogSegment.scala:415) at kafka.log.Log.replaceSegments(Log.scala:1644) at kafka.log.Cleaner.cleanSegments(LogCleaner.scala:535) at kafka.log.Cleaner.$anonfun$doClean$6(LogCleaner.scala:462) at kafka.log.Cleaner.$anonfun$doClean$6$adapted(LogCleaner.scala:461) at scala.collection.immutable.List.foreach(List.scala:389) at kafka.log.Cleaner.doClean(LogCleaner.scala:461) at kafka.log.Cleaner.clean(LogCleaner.scala:438) at kafka.log.LogCleaner$CleanerThread.cleanOrSleep(LogCleaner.scala:305) at kafka.log.LogCleaner$CleanerThread.doWork(LogCleaner.scala:291) at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:82) Suppressed: java.nio.file.FileSystemException: C:\root\kafka_2.12-1.1.0\data\__consumer_offsets-3\00000000000000000000.log.cleaned -> C:\root\kafka_2.12-1.1.0\data\__consumer_offsets-3\00000000000000000000.log.swap: Proces nie mo┐e uzyskaŠ dostŕpu do pliku, poniewa┐ jest on u┐ywany przez inny proces. at sun.nio.fs.WindowsException.translateToIOException(Unknown Source) at sun.nio.fs.WindowsException.rethrowAsIOException(Unknown Source) at sun.nio.fs.WindowsFileCopy.move(Unknown Source) at sun.nio.fs.WindowsFileSystemProvider.move(Unknown Source) at java.nio.file.Files.move(Unknown Source) at org.apache.kafka.common.utils.Utils.atomicMoveWithFallback(Utils.java:694) ... 12 more [2018-06-21 13:06:34,196] INFO [ReplicaManager broker=0] Stopping serving replicas in dir C:\root\kafka_2.12-1.1.0\data (kafka.server.ReplicaManager) [2018-06-21 13:06:34,209] INFO [ReplicaFetcherManager on broker 0] Removed fetcher for partitions __consumer_offsets-22,test-14,__consumer_offsets-30,test-6,__consumer_offsets-8,__consumer_offsets-21,test-17,__consumer_offsets-4,INTEGRATION_TESTS_DEBUG.monitoring_events-0,test-20,__consumer_offsets-27,__consumer_offsets-7,__consumer_offsets-9,test-29,__consumer_offsets-46,test-23,test-24,test-11,test-10,__consumer_offsets-25,__consumer_offsets-35,__consumer_offsets-41,__consumer_offsets-33,__consumer_offsets-23,__consumer_offsets-49,test-28,__consumer_offsets-47,test-19,__consumer_offsets-16,test-0,__consumer_offsets-28,test-7,__consumer_offsets-31,__consumer_offsets-36,__consumer_offsets-42,__consumer_offsets-3,test-18,__consumer_offsets-18,test-22,test-25,__consumer_offsets-37,test-5,__consumer_offsets-15,__consumer_offsets-24,__consumer_offsets-38,__consumer_offsets-17,test-8,__consumer_offsets-48,test-1,__consumer_offsets-19,test-26,__consumer_offsets-11,__consumer_offsets-13,test-27,__consumer_offsets-2,__consumer_offsets-43,__consumer_offsets-6,__consumer_offsets-14,test-13,test-4,test-30,test-16,__consumer_offsets-20,__consumer_offsets-0,__consumer_offsets-44,test-9,__consumer_offsets-39,__consumer_offsets-12,__consumer_offsets-45,__consumer_offsets-1,__consumer_offsets-5,__consumer_offsets-26,__consumer_offsets-29,__consumer_offsets-34,__consumer_offsets-10,test-3,__consumer_offsets-32,test-21,test-12,test-15,__consumer_offsets-40,test-2,test-31 (kafka.server.ReplicaFetcherManager) [2018-06-21 13:06:34,216] INFO [ReplicaAlterLogDirsManager on broker 0] Removed fetcher for partitions __consumer_offsets-22,test-14,__consumer_offsets-30,test-6,__consumer_offsets-8,__consumer_offsets-21,test-17,__consumer_offsets-4,INTEGRATION_TESTS_DEBUG.monitoring_events-0,test-20,__consumer_offsets-27,__consumer_offsets-7,__consumer_offsets-9,test-29,__consumer_offsets-46,test-23,test-24,test-11,test-10,__consumer_offsets-25,__consumer_offsets-35,__consumer_offsets-41,__consumer_offsets-33,__consumer_offsets-23,__consumer_offsets-49,test-28,__consumer_offsets-47,test-19,__consumer_offsets-16,test-0,__consumer_offsets-28,test-7,__consumer_offsets-31,__consumer_offsets-36,__consumer_offsets-42,__consumer_offsets-3,test-18,__consumer_offsets-18,test-22,test-25,__consumer_offsets-37,test-5,__consumer_offsets-15,__consumer_offsets-24,__consumer_offsets-38,__consumer_offsets-17,test-8,__consumer_offsets-48,test-1,__consumer_offsets-19,test-26,__consumer_offsets-11,__consumer_offsets-13,test-27,__consumer_offsets-2,__consumer_offsets-43,__consumer_offsets-6,__consumer_offsets-14,test-13,test-4,test-30,test-16,__consumer_offsets-20,__consumer_offsets-0,__consumer_offsets-44,test-9,__consumer_offsets-39,__consumer_offsets-12,__consumer_offsets-45,__consumer_offsets-1,__consumer_offsets-5,__consumer_offsets-26,__consumer_offsets-29,__consumer_offsets-34,__consumer_offsets-10,test-3,__consumer_offsets-32,test-21,test-12,test-15,__consumer_offsets-40,test-2,test-31 (kafka.server.ReplicaAlterLogDirsManager) [2018-06-21 13:06:34,323] ERROR Failed to clean up log for __consumer_offsets-3 in dir C:\root\kafka_2.12-1.1.0\data due to IOException (kafka.server.LogDirFailureChannel) java.nio.file.FileSystemException: C:\root\kafka_2.12-1.1.0\data\__consumer_offsets-3\00000000000000000000.log.cleaned: Proces nie mo┐e uzyskaŠ dostŕpu do pliku, poniewa┐ jest on u┐ywany przez inny proces. at sun.nio.fs.WindowsException.translateToIOException(Unknown Source) at sun.nio.fs.WindowsException.rethrowAsIOException(Unknown Source) at sun.nio.fs.WindowsException.rethrowAsIOException(Unknown Source) at sun.nio.fs.WindowsFileSystemProvider.implDelete(Unknown Source) at sun.nio.fs.AbstractFileSystemProvider.deleteIfExists(Unknown Source) at java.nio.file.Files.deleteIfExists(Unknown Source) at kafka.log.Cleaner.deleteCleanedFileIfExists$1(LogCleaner.scala:488) at kafka.log.Cleaner.cleanSegments(LogCleaner.scala:493) at kafka.log.Cleaner.$anonfun$doClean$6(LogCleaner.scala:462) at kafka.log.Cleaner.$anonfun$doClean$6$adapted(LogCleaner.scala:461) at scala.collection.immutable.List.foreach(List.scala:389) at kafka.log.Cleaner.doClean(LogCleaner.scala:461) at kafka.log.Cleaner.clean(LogCleaner.scala:438) at kafka.log.LogCleaner$CleanerThread.cleanOrSleep(LogCleaner.scala:305) at kafka.log.LogCleaner$CleanerThread.doWork(LogCleaner.scala:291) at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:82) [2018-06-21 13:06:34,344] INFO [ReplicaManager broker=0] Broker 0 stopped fetcher for partitions __consumer_offsets-22,test-14,__consumer_offsets-30,test-6,__consumer_offsets-8,__consumer_offsets-21,test-17,__consumer_offsets-4,INTEGRATION_TESTS_DEBUG.monitoring_events-0,test-20,__consumer_offsets-27,__consumer_offsets-7,__consumer_offsets-9,test-29,__consumer_offsets-46,test-23,test-24,test-11,test-10,__consumer_offsets-25,__consumer_offsets-35,__consumer_offsets-41,__consumer_offsets-33,__consumer_offsets-23,__consumer_offsets-49,test-28,__consumer_offsets-47,test-19,__consumer_offsets-16,test-0,__consumer_offsets-28,test-7,__consumer_offsets-31,__consumer_offsets-36,__consumer_offsets-42,__consumer_offsets-3,test-18,__consumer_offsets-18,test-22,test-25,__consumer_offsets-37,test-5,__consumer_offsets-15,__consumer_offsets-24,__consumer_offsets-38,__consumer_offsets-17,test-8,__consumer_offsets-48,test-1,__consumer_offsets-19,test-26,__consumer_offsets-11,__consumer_offsets-13,test-27,__consumer_offsets-2,__consumer_offsets-43,__consumer_offsets-6,__consumer_offsets-14,test-13,test-4,test-30,test-16,__consumer_offsets-20,__consumer_offsets-0,__consumer_offsets-44,test-9,__consumer_offsets-39,__consumer_offsets-12,__consumer_offsets-45,__consumer_offsets-1,__consumer_offsets-5,__consumer_offsets-26,__consumer_offsets-29,__consumer_offsets-34,__consumer_offsets-10,test-3,__consumer_offsets-32,test-21,test-12,test-15,__consumer_offsets-40,test-2,test-31 and stopped moving logs for partitions because they are in the failed log directory C:\root\kafka_2.12-1.1.0\data. (kafka.server.ReplicaManager) [2018-06-21 13:06:34,644] INFO Stopping serving logs in dir C:\root\kafka_2.12-1.1.0\data (kafka.log.LogManager) [2018-06-21 13:06:34,759] ERROR Failed to clean up log for __consumer_offsets-3 in dir C:\root\kafka_2.12-1.1.0\data due to IOException (kafka.server.LogDirFailureChannel) java.nio.file.FileSystemException: C:\root\kafka_2.12-1.1.0\data\__consumer_offsets-3\00000000000000000000.log.cleaned: Proces nie mo┐e uzyskaŠ dostŕpu do pliku, poniewa┐ jest on u┐ywany przez inny proces. at sun.nio.fs.WindowsException.translateToIOException(Unknown Source) at sun.nio.fs.WindowsException.rethrowAsIOException(Unknown Source) at sun.nio.fs.WindowsException.rethrowAsIOException(Unknown Source) at sun.nio.fs.WindowsFileSystemProvider.implDelete(Unknown Source) at sun.nio.fs.AbstractFileSystemProvider.deleteIfExists(Unknown Source) at java.nio.file.Files.deleteIfExists(Unknown Source) at kafka.log.Cleaner.deleteCleanedFileIfExists$1(LogCleaner.scala:488) at kafka.log.Cleaner.cleanSegments(LogCleaner.scala:493) at kafka.log.Cleaner.$anonfun$doClean$6(LogCleaner.scala:462) at kafka.log.Cleaner.$anonfun$doClean$6$adapted(LogCleaner.scala:461) at scala.collection.immutable.List.foreach(List.scala:389) at kafka.log.Cleaner.doClean(LogCleaner.scala:461) at kafka.log.Cleaner.clean(LogCleaner.scala:438) at kafka.log.LogCleaner$CleanerThread.cleanOrSleep(LogCleaner.scala:305) at kafka.log.LogCleaner$CleanerThread.doWork(LogCleaner.scala:291) at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:82) [2018-06-21 13:06:34,766] ERROR Shutdown broker because all log dirs in C:\root\kafka_2.12-1.1.0\data have failed (kafka.log.LogManager)
Attachments
Issue Links
- relates to
-
KAFKA-9458 Kafka crashed in windows environment
-
- Patch Available
-