[KAFKA-7174] Improve version probing of subscription info

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Component/s: streams
• Labels: compatibility
• Assignee: Unassigned
• Reporter: Ted Yu
• Created: 17/Jul/18 22:57
• Updated: 15/May/20 17:21
• Resolved: 15/May/20 17:21

Description

During code review for KAFKA-5037, guozhang made the following suggestion:

Currently the version probing works as the following:

when leader receives the subscription info encoded with a higher version that it can understand (e.g. the leader is on version 3, while one of the subscription received is encode with version 4), it will send back an empty assignment with the assignment encoded with version 3, and also latestSupportedVersion set to 3.

when the member receives the assignment, it checks if latestSupportedVersion is smaller than the version it used for encoding the sent subscription (i.e. the above logic). If it is smaller, then it means that leader cannot understand, in this case, version 4. It will then set the flag and then re-subscribe but with a down-graded encoding format of version 3.

NOW with PR #5322, we can let leader to clearly communicate this error via the error code, and upon receiving the assignment, if the error code is VERSION_PROBING, then the member can immediately know what happens, and hence can simplify the above logic.