Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-8516

Allow custom shard-to-subtask assignment for the FlinkKinesisConsumer

    Details

    • Type: New Feature
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.5.0
    • Component/s: Kinesis Connector
    • Labels:
      None

      Description

      The hash code of the shard is used to distribute discovered shards over subtasks round robin. This works as long as shard identifiers are sequential. After shards are rebalanced in Kinesis, that may no longer be the case and the distribution become skewed.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                thw Thomas Weise
                Reporter:
                thw Thomas Weise
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: