[FLINK-9610] Add Kafka partitioner that uses the key to partition by

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Component/s: Connectors / Kafka
• Labels: pull-request-available
• Assignee: Niels Basjes
• Reporter: Niels Basjes
• Created: 18/Jun/18 19:15
• Updated: 14/Apr/20 13:01
• Resolved: 14/Apr/20 13:01

Description

The kafka connector package only contains the FlinkFixedPartitioner implementation of the FlinkKafkaPartitioner.

The most common usecase I have seen is the need to spread the records across the Kafka partitions while keeping all messages with the same key together.

I'll put up a pull request with a very simple implementation that should make this a lot easier for others to use and extend.

Issue Links

• is duplicated by: FLINK-11948 When kafka sink parallelism<kafka partition num，kafka data distribution unbalance
• links to: GitHub Pull Request #6181
• links to: GitHub Pull Request #6181