[KAFKA-15857] Introduce LocalLogStartOffset and TieredOffset in OffsetSpec.

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 3.9.0
• Component/s: core
• Labels: need-kip tiered-storage
• Assignee: Christo Lolov
• Reporter: Satish Duggana
• Created: 20/Nov/23 11:30
• Updated: 05/Aug/24 02:42
• Resolved: 05/Aug/24 02:42

Description

Introduce EarliestLocalOffset and TieredOffset in OffsetSpec which will help in finding respective offsets while using AdminClient#listOffsets().

EarliestLocalOffset - local log start offset of a topic partition.
TieredOffset - Highest offset up to which the segments were copied to remote storage.

We can discuss further on naming and semantics of these offset specs.

 

KIP-1005: https://cwiki.apache.org/confluence/display/KAFKA/KIP-1005%3A+Expose+EarliestLocalOffset+and+TieredOffset

Issue Links

• is a child of: KAFKA-15420 Kafka Tiered Storage V1
• mentioned in: Page Loading...
• mentioned in: Page Loading...

Sub-Tasks

• Make broker changes to return an offset for LATEST_TIERED_TIMESTAMP: Resolved
• ListOffsets change should have an associated API/IBP version update: Resolved
• Update latestVersionUnstable of ListOffsetsRequest: Resolved
• Throw/return unsupported version exception if the server/client does NOT support EarliestLocalSpec and LatestTieredSpec: Resolved
• Add IT to make sure the production MV does not use unstable version of LIST_OFFSET: Resolved