[KAFKA-14127] KIP-858: Handle JBOD broker disk failure in KRaft

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 3.7.0
• Component/s: jbod, kraft
• Labels: 4.0-blocker kip-500 kraft
• Assignee: Igor Soarez
• Reporter: Igor Soarez
• Created: 01/Aug/22 12:23
• Updated: 22/Feb/24 12:08
• Resolved: 29/Dec/23 02:04

Description

Supporting configurations with multiple storage directories in KRaft mode

Issue Links

• relates to: KAFKA-16061 KRaft JBOD follow-ups and improvements
• mentioned in: Page Loading...
• mentioned in: Page Loading...

Sub-Tasks

• Message schema changes: Resolved
• Generate and persist directory IDs: Resolved
• Aggregate and propagate assignments: Resolved
• Send offline dirs in broker heartbeat: Resolved
• QueuedReplicaToDirAssignments metric: Resolved
• Include directory info in BrokerRegistration: Resolved
• Process and persist dir info with broker registration: Resolved
• Process and persist directory assignments: Resolved
• Resolve offline replicas in metadata cache: Resolved
• Broker log directory failure changes: Resolved
• Handle log directory failure in the Controller: Resolved
• Broker-side replica management changes: Resolved
• Controller-side replica management changes: Resolved
• Log directory failure integration test: Resolved
• Bug: Generated Persistent Directory IDs are overwritten on startup.: Resolved
• Data-loss on leader shutdown right after partition creation?: Resolved
• Broker stays fenced until all assignments are correct: Resolved
• Always specify directories for new partition registrations: Resolved
• Bump MetadataVersion for directory assignments: Resolved