[KAFKA-4451] Recovering empty replica yields negative offsets in index of compact partitions

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.9.0.1
• Fix Version/s: 0.10.2.0
• Labels: reliability
• Assignee: Unassigned
• Reporter: Michael Schiff
• Created: 28/Nov/16 01:20
• Updated: 15/Dec/16 23:47
• Resolved: 15/Dec/16 23:46

Description

Bringing up an empty broker.

the partition for a compact topic is not split into multiple log files. All data is written into a single log file, causing offsets to overflow.

A dump of the affected broker shortly after it started replicating:

michael.schiff@stats-kafka09:~$ sudo /opt/kafka/bin/kafka-run-class.sh kafka.tools.DumpLogSegments --files /kafka/attainment_event-0/00000000000000000000.index | head -n 10
Dumping /kafka/attainment_event-0/00000000000000000000.index
offset: 1022071124 position: 1037612
offset: -1713432120 position: 1348740
offset: -886291423 position: 2397130
offset: -644750126 position: 3445630
offset: -57889876 position: 4493972
offset: 433950099 position: 5388461
offset: 1071769472 position: 6436837
offset: 1746859069 position: 7485367
offset: 2090359736 position: 8533822
...

and the dump of the same log file from the leader of this partition

michael.schiff@stats-kafka12:~$ sudo /opt/kafka/bin/kafka-run-class.sh kafka.tools.DumpLogSegments --files /kafka/attainment_event-0/00000000000000000000.index
[sudo] password for michael.schiff:
Dumping /kafka/attainment_event-0/00000000000000000000.index
offset: 353690666 position: 262054
offset: 633140428 position: 523785
offset: 756537951 position: 785815

Issue Links

• is related to: KAFKA-3323 Negative offsets in Log Segment Index files due to Integer overflow when compaction is enabled
• links to: GitHub Pull Request #2210