[FLINK-8516] Allow custom shard-to-subtask assignment for the FlinkKinesisConsumer

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.5.0
• Component/s: Connectors / Kinesis
• Assignee: Thomas Weise
• Reporter: Thomas Weise
• Created: 26/Jan/18 04:26
• Updated: 02/Oct/19 17:43
• Resolved: 15/Feb/18 18:23

Description

The hash code of the shard is used to distribute discovered shards over subtasks round robin. This works as long as shard identifiers are sequential. After shards are rebalanced in Kinesis, that may no longer be the case and the distribution become skewed.

Issue Links

• links to: GitHub Pull Request #5393