Details
-
Improvement
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
None
-
None
Description
In the replica fetcher, we have logic to detect the case when the follower's offset is ahead of the leader's. If unclean leader election is not enabled, we raise a fatal error and kill the broker.
This behavior is inconsistent depending on the message format. With KIP-101/KIP-279, upon becoming a follower, the replica would use leader epoch information to reconcile the end of the log with the leader and simply truncate. Additionally, with the old format, the check is not really bulletproof for detecting data loss since the unclean leader's end offset might have already caught up to the follower's offset at the time of its initial fetch or when it queries for the current log end offset.
To make the logic consistent, we could raise a fatal error whenever the follower has to truncate below the high watermark. However, the fatal error is probably overkill and it would be better to log a warning since most of the damage is already done if the leader has already been elected and this causes a huge blast radius.
Attachments
Issue Links
- links to