Details
-
Sub-task
-
Status: Open
-
Major
-
Resolution: Unresolved
-
0.11.0.0
-
None
-
None
-
None
Description
In the discussion of KIP-185: Make exactly once in order delivery per partition the default producer setting, it was realized that we don't have graceful handling when an idempotence-enabled producer is writing to a broker with a message format older than v2 (ie. the 0.11.0 message format).
In particular, if we enable idempotence, any produce requests to topics with an older message format will fail with an UnsupportedVersionException. Thus if the idempotent producer was to be made the default, the out of the box producer would fail to produce when used with clusters which haven't upgraded the message format yet.
This is particularly problematic since the recommended upgrade path is to upgrade broker code while keeping the message format at the older version, then upgrade all clients, and only finally upgrade the message format on the server. With the current behavior, the middle step is actually untenable if we enable idempotence as the default.
More details available at:
https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Exactly+Once+-+Dealing+with+older+message+formats+when+idempotence+is+enabled