Details
-
Improvement
-
Status: Open
-
Major
-
Resolution: Unresolved
-
None
-
None
-
None
-
None
Description
We have seen several cases where down-conversion caused replicas to diverge from the leader in subtle ways. Generally speaking, even if we addressed all of the edge cases so that down-conversion worked correctly as far as consistency of offsets, it would probably still be a bad idea to permit down-conversion. For example, this can cause message timestamps to be lost if down-converting from v1 to v0, or transactional data could be lost if down-converting from v2 to v1 or v0.
With that in mind, it would better to forbid down-conversion for replica fetches. Following the normal upgrade procedure, down-conversion is not needed anyway, but users often skip updating the inter-broker version. It is probably better in these cases to let the ISR shrink until the replicas have been updated as well.