[KAFKA-13391] Failure on Windows due to AccessDeniedAcception when attempting to fsync the parent directory

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.0.0
• Fix Version/s: 3.1.0, 3.0.1
• Component/s: clients
• Assignee: Luke Chen
• Reporter: Andy Wilkinson
• Created: 21/Oct/21 09:40
• Updated: 16/Feb/22 08:46
• Resolved: 25/Oct/21 17:18

Description

There appears to be a regression in Kafka 3.0 due to the changes made for KAFKA-3968 that causes a failure on Windows. After upgrading to 3.0.0, we're seeing failures in Spring Boot's Windows CI such as the following:

Caused by: java.nio.file.AccessDeniedException: C:\Windows\TEMP\spring.kafka.915ab8c1-735c-4c88-8507-8d25fd050621920219824697516859	
	at sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:83)
	at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97)
	at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:102)
	at sun.nio.fs.WindowsFileSystemProvider.newFileChannel(WindowsFileSystemProvider.java:115)
	at java.nio.channels.FileChannel.open(FileChannel.java:287)
	at java.nio.channels.FileChannel.open(FileChannel.java:335)
	at org.apache.kafka.common.utils.Utils.flushDir(Utils.java:953)
	at org.apache.kafka.common.utils.Utils.atomicMoveWithFallback(Utils.java:941)
	at kafka.server.BrokerMetadataCheckpoint.liftedTree1$1(BrokerMetadataCheckpoint.scala:214)
	at kafka.server.BrokerMetadataCheckpoint.write(BrokerMetadataCheckpoint.scala:204)
	at kafka.server.KafkaServer.$anonfun$checkpointBrokerMetadata$2(KafkaServer.scala:772)
	at kafka.server.KafkaServer.$anonfun$checkpointBrokerMetadata$2$adapted(KafkaServer.scala:770)
	at scala.collection.IterableOnceOps.foreach(IterableOnce.scala:563)
	at scala.collection.IterableOnceOps.foreach$(IterableOnce.scala:561)
	at scala.collection.AbstractIterable.foreach(Iterable.scala:919)
	at scala.collection.IterableOps$WithFilter.foreach(Iterable.scala:889)
	at kafka.server.KafkaServer.checkpointBrokerMetadata(KafkaServer.scala:770)
	at kafka.server.KafkaServer.startup(KafkaServer.scala:322)
	at kafka.utils.TestUtils$.createServer(TestUtils.scala:175)
	at kafka.utils.TestUtils$.createServer(TestUtils.scala:170)
	at kafka.utils.TestUtils.createServer(TestUtils.scala)

While I'm aware that Windows isn't officially supported, I think this problem is likely to be a blocker for anyone who uses Windows for Kafka-based development work.

I suspect that the attempt to fsync the directory should just be skipped on Window. Alternatively, the failure could be ignored on Windows. Lucene added similar functionality in the past where it looks like they opted to ignore IOExceptions on Windows rather than skipping the attempt.

Issue Links

• is duplicated by: KAFKA-13390 Kafka 3 fails to run on Windows
• links to: GitHub Pull Request #11426