[KAFKA-4726] ValueMapper should have (read) access to key

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.10.1.1
• Fix Version/s: 1.1.0
• Component/s: streams
• Labels: kip
• Assignee: Jeyhun Karimov
• Reporter: Steven Schlansker
• Created: 02/Feb/17 20:52
• Updated: 18/Jan/19 18:37
• Resolved: 18/Jan/19 18:32

Description

ValueMapper should have read-only access to the key for the value it is mapping. Sometimes the value transformation will depend on the key.

It is possible to do this with a full blown KeyValueMapper but that loses the promise that you won't change the key – so you might introduce a re-keying phase that is totally unnecessary. It also requires you to return an identity KeyValue object which costs something to construct (unless we are lucky and the optimizer elides it).

[ If mapValues() is guaranteed to be no less efficient than map() the issue may be moot, but I presume there are some optimizations that are valid with the former but not latter. ]
 

KIP-149: https://cwiki.apache.org/confluence/display/KAFKA/KIP-149%3A+Enabling+key+access+in+ValueTransformer%2C+ValueMapper%2C+and+ValueJoiner

 

Issue Links

• Is contained by: KAFKA-4218 Enable access to key in ValueTransformer
• is related to: KAFKA-7842 Reducer should have (read) access to key
• is related to: KAFKA-7843 Initializer should have (read) access to key
• relates to: KAFKA-3745 Consider adding join key to ValueJoiner interface
• mentioned in: Page Loading...
• mentioned in: Page Loading...