[FLINK-9691] Modify run loop in Kinesis ShardConsumer to not sleep for a fixed fetchIntervalMillis

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.4.2, 1.5.0
• Fix Version/s: 1.5.2, 1.6.0
• Component/s: Connectors / Kinesis
• Assignee: Jamie Grier
• Reporter: Lakshmi Rao
• Created: 28/Jun/18 21:50
• Updated: 06/Aug/18 13:38
• Resolved: 06/Aug/18 13:38

Description

Currently the ShardConsumer in the Kinesis connector sleeps for a fixed fetchIntervalMillis resulting in the shard consumer sleeping for more time than necessary and not optimally reading from Kinesis. It should only be sleeping for (fetchIntervalMillis - time taken to process records) before making the subsequent getRecords call.