Details
-
Improvement
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
None
-
None
-
None
Description
For KIP-392, we added logic to make sure that high watermark changes are propagated to followers without delay in order to improve end to end latency when fetching from followers. The downside of this change is that it increases the rate of fetch requests from followers which can have a noticeable impact on performance (see KAFKA-9731). To fix that problem, we modified the code so that we only propagate high watermark changes immediately when a replica selector is used (which is not the default). However, we should consider whether we want to go even further and disable immediate propagation of the high watermark more generally since it makes this feature risky to use. Instead, users can use the max wait time in order to control the worst-case latency. Note that this is typically only a problem for low-throughput topics since otherwise we will have a steady rate of high watermark updates.
Attachments
Issue Links
- links to