[KAFKA-9180] Broker won't start with empty log dir

• Type: Bug
• Status: Closed
• Priority: Blocker
• Resolution: Invalid
• Affects Version/s: 2.4.0
• Component/s: core
• Assignee: Ismael Juma
• Reporter: Magnus Edenhill
• Created: 13/Nov/19 16:23
• Updated: 18/Nov/19 16:43
• Resolved: 13/Nov/19 17:06

Description

On kafka trunk at commit 1675115ec193acf4c7d44e68a57272edfec0b455:

 

Attempting to start the broker with an existing but empty log dir yields the following error and terminates the process:

[2019-11-13 10:42:16,922] ERROR Failed to read meta.properties file under dir /Users/magnus/src/librdkafka/tests/tmp/LibrdkafkaTestCluster/73638126/KafkaBrokerApp/2/logs/meta.properties due to /Users/magnus/src/librdkafka/tests/tmp/LibrdkafkaTestCluster/73638126/KafkaBrokerApp/2/logs/meta.properties (No such file or directory) (kafka.server.BrokerMetadataCheckpoint)[2019-11-13 10:42:16,924] ERROR Fail to read meta.properties under log directory /Users/magnus/src/librdkafka/tests/tmp/LibrdkafkaTestCluster/73638126/KafkaBrokerApp/2/logs (kafka.server.KafkaServer)java.io.FileNotFoundException: /Users/magnus/src/librdkafka/tests/tmp/LibrdkafkaTestCluster/73638126/KafkaBrokerApp/2/logs/meta.properties (No such file or directory)        at java.io.FileInputStream.open0(Native Method)        at java.io.FileInputStream.open(FileInputStream.java:195)        at java.io.FileInputStream.<init>(FileInputStream.java:138)        at java.io.FileInputStream.<init>(FileInputStream.java:93)        at org.apache.kafka.common.utils.Utils.loadProps(Utils.java:512)        at kafka.server.BrokerMetadataCheckpoint.liftedTree2$1(BrokerMetadataCheckpoint.scala:73)        at kafka.server.BrokerMetadataCheckpoint.read(BrokerMetadataCheckpoint.scala:72)        at kafka.server.KafkaServer.$anonfun$getBrokerMetadataAndOfflineDirs$1(KafkaServer.scala:704)        at kafka.server.KafkaServer.$anonfun$getBrokerMetadataAndOfflineDirs$1$adapted(KafkaServer.scala:702)        at scala.collection.IndexedSeqOptimized.foreach(IndexedSeqOptimized.scala:36)        at scala.collection.IndexedSeqOptimized.foreach$(IndexedSeqOptimized.scala:33)        at scala.collection.mutable.WrappedArray.foreach(WrappedArray.scala:38)        at kafka.server.KafkaServer.getBrokerMetadataAndOfflineDirs(KafkaServer.scala:702)        at kafka.server.KafkaServer.startup(KafkaServer.scala:214)        at kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:44)        at kafka.Kafka$.main(Kafka.scala:84)        at kafka.Kafka.main(Kafka.scala) 

 
 
Changing the catch to FileNotFoundException fixes the issue, here:
https://github.com/apache/kafka/blob/trunk/core/src/main/scala/kafka/server/BrokerMetadataCheckpoint.scala#L84
 
 
This is a regression from 2.3.x.
 

Issue Links

• links to: GitHub Pull Request #7700