[KAFKA-16385] Segment is rolled before segment.ms or segment.bytes breached - ASF JIRA

Attach files

Attach Screenshot

Voters

Watch issue

Watchers

Create sub-task

Link

Clone

Update Comment Author

Replace String in Comment

Update Comment Visibility

Delete Comments

XML

Word

Printable

JSON

Details

Type: Bug
Status: Resolved
Priority: Major
Resolution: Fixed
Affects Version/s: 3.5.1, 3.7.0
Fix Version/s: 3.8.0
Component/s: None
Labels:
None

Description

Steps to reproduce:
0. Startup a broker with `log.retention.check.interval.ms=1000` to speed up the test.
1. Creating a topic with the config: segment.ms=7days , segment.bytes=1GB, retention.ms=1sec .
2. Send a record "aaa" to the topic
3. Wait for 1 second

Will this segment will rolled? I thought no.
But what I have tested is it will roll:

[2024-03-19 15:23:13,924] INFO [LocalLog partition=t2-1, dir=/tmp/kafka-logs_jbod] Rolled new log segment at offset 1 in 3 ms. (kafka.log.LocalLog)
[2024-03-19 15:23:13,925] INFO [ProducerStateManager partition=t2-1] Wrote producer snapshot at offset 1 with 1 producer ids in 1 ms. (org.apache.kafka.storage.internals.log.ProducerStateManager)
[2024-03-19 15:23:13,925] INFO [UnifiedLog partition=t2-1, dir=/tmp/kafka-logs_jbod] Deleting segment LogSegment(baseOffset=0, size=71, lastModifiedTime=1710832993131, largestRecordTimestamp=1710832992125) due to log retention time 1000ms breach based on the largest record timestamp in the segment (kafka.log.UnifiedLog)

The segment is rolled due to log retention time 1000ms breached, which is unexpected.

Tested in v3.5.1, it has the same issue.