[FLINK-5075] Kinesis consumer incorrectly determines shards as newly discovered when tested against Kinesalite

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.1.4, 1.2.0
• Component/s: Connectors / Kinesis
• Assignee: Tzu-Li (Gordon) Tai
• Reporter: Tzu-Li (Gordon) Tai
• Created: 16/Nov/16 06:15
• Updated: 24/Nov/16 10:58
• Resolved: 24/Nov/16 10:58

Description

A user reported that when our Kinesis connector is used against Kinesalite (https://github.com/mhart/kinesalite), we're incorrectly determining already found shards as newly discovered:
http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Subtask-keeps-on-discovering-new-Kinesis-shard-when-using-Kinesalite-td10133.html

I suspect the problem to be the mock Kinesis API implementations of Kinesalite doesn't completely match with the official AWS Kinesis behaviour.

Issue Links

• links to: GitHub Pull Request #2822