[KAFKA-16385] Segment is rolled before segment.ms or segment.bytes breached

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.5.1, 3.7.0
• Fix Version/s: 3.8.0
• Assignee: Kuan Po Tseng
• Reporter: Luke Chen
• Created: 19/Mar/24 08:20
• Updated: 20/May/24 14:07
• Resolved: 27/Mar/24 02:20

Description

Steps to reproduce:
0. Startup a broker with `log.retention.check.interval.ms=1000` to speed up the test.
1. Creating a topic with the config: segment.ms=7days , segment.bytes=1GB, retention.ms=1sec .
2. Send a record "aaa" to the topic
3. Wait for 1 second

Will this segment will rolled? I thought no.
But what I have tested is it will roll:

[2024-03-19 15:23:13,924] INFO [LocalLog partition=t2-1, dir=/tmp/kafka-logs_jbod] Rolled new log segment at offset 1 in 3 ms. (kafka.log.LocalLog)
[2024-03-19 15:23:13,925] INFO [ProducerStateManager partition=t2-1] Wrote producer snapshot at offset 1 with 1 producer ids in 1 ms. (org.apache.kafka.storage.internals.log.ProducerStateManager)
[2024-03-19 15:23:13,925] INFO [UnifiedLog partition=t2-1, dir=/tmp/kafka-logs_jbod] Deleting segment LogSegment(baseOffset=0, size=71, lastModifiedTime=1710832993131, largestRecordTimestamp=1710832992125) due to log retention time 1000ms breach based on the largest record timestamp in the segment (kafka.log.UnifiedLog)

The segment is rolled due to log retention time 1000ms breached, which is unexpected.

Tested in v3.5.1, it has the same issue.

Issue Links

• links to: GitHub Pull Request #15588

Sub-Tasks

• Enhance all configs which can trigger rolling of new segment: Open