[KAFKA-15375] When running in KRaft mode, LogManager may creates CleanShutdown file by mistake

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.3.2, 3.4.1, 3.5.1
• Fix Version/s: 3.3.3, 3.6.0, 3.4.2, 3.5.2
• Assignee: Vincent Jiang
• Reporter: Vincent Jiang
• Created: 17/Aug/23 18:19
• Updated: 06/Sep/23 17:30
• Resolved: 06/Sep/23 17:30

Description

Consider following sequence when running Kafka in KRaft mode:

1. A partition log "log1" is created under "logDir1", and some records are appended to it.
2. Broker crashes. No clean shutdown file is created in "logDir1".
3. Broker is restarted. BrokerServer.startup is called.
4. On a different thread, LogManager.startup is called by BrokerMetadataPublisher.
5. Before LogManager.startup finishing recovering logs under "logDir1", fatal exception is thrown in BrokerServer.startup.
6. In exception hander, BrokerServer.startup calls LogManager.shutdown. As a result, a clean shutdown file is created under "logDir1"
7. Broker is restarted again. Due to the clean shutdown file created in step 6, recovery is skipped for logs under "logDir1", which is not right because "logDir1" was not fully recovered in step 5.

Issue Links

• links to: GitHub Pull Request #14239