[KAFKA-13197] KStream-GlobalKTable join semantics don't match documentation

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.7.0
• Fix Version/s: 3.6.0, 3.5.2
• Component/s: documentation, streams
• Assignee: Florin Akermann
• Reporter: Tommy Becker
• Created: 12/Aug/21 15:26
• Updated: 11/Aug/23 21:36
• Resolved: 11/Aug/23 18:47

Description

As part of KAFKA-10277, the behavior of KStream-GlobalKTable joins was changed. It appears the change was intended to merely relax a requirement but it actually broke backwards compatibility. Although it does allow null keys and values in the KStream to be joined, it now excludes null results of the KeyValueMapper. We have an application which can return null from the KeyValueMapper for non-null keys in the KStream, and relies on these nulls being passed to the ValueJoiner. Indeed the javadoc still explicitly says this is done:

If a KStream input record key or value is null the record will not be included in the join operation and thus no output record will be added to the resulting KStream.
If keyValueMapper returns null implying no match exists, a null value will be provided to ValueJoiner.

Both these statements are incorrect.

I think the new behavior is worse than the previous/documented behavior. It feels more reasonable to have a non-null stream record map to a null join key (our use-case is event-enhancement where the incoming record doesn't have the join field), than the reverse.

Issue Links

• relates to: KAFKA-12317 Relax non-null key requirement for left/outer KStream joins
• relates to: KAFKA-14049 Relax Non Null Requirement for KStreamGlobalKTable Left Join
• links to: GitHub Pull Request #14187