[KAFKA-14470] Move log layer to storage module

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Assignee: Ismael Juma
• Reporter: Ismael Juma
• Created: 14/Dec/22 21:12
• Updated: 07/Dec/24 10:28

Description

We introduced the `storage` module as part of KIP-405, but the existing log layer remains in the `core` module. Moving the log layer to the `storage` module would be another step towards improved modularity and build times (similar to `metadata`, `raft` and `group-coordinator`).

We should do this in an incremental manner to make the code review process easier. I will create separate tasks, each one mapping to one pull request. In order to understand the feasibility, I tackled a few of the tasks myself.

Help from the community is appreciated for the unassigned tasks, but it probably makes sense to do that after the initial PRs have been submitted.

Issue Links

• blocks: KAFKA-14579 Move DumpLogSegments to tools
• blocks: KAFKA-14524 Modularize `core` monolith
• links to: GitHub Pull Request #18021

Sub-Tasks

• Move UnifiedLog to storage module: In Progress
• Move LogCleaner to storage module: Open
• Move LogCleanerManager to storage module: Open
• Move LogManager to storage module: Open
• Move log layer tests to storage module: Open
• Adjust visibility of classes moved to storage module: Open
• Consider using UncheckedIOException instead of IOException in the log layer: Open
• Move RemoteLogManager to the storage module: In Progress
• Move classes in kafka.raft from core module to raft module: Open
• Move RemoteIndexCacheTest to storage module: Open