[FLINK-9183] Kafka consumer docs to warn about idle partitions

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.4.3, 1.5.0
• Assignee: Juho Autio
• Reporter: Juho Autio
• Created: 16/Apr/18 13:35
• Updated: 16/Apr/18 19:47
• Resolved: 16/Apr/18 19:47

Description

Looks like the bug FLINK-5479 is entirely preventing FlinkKafkaConsumerBase#assignTimestampsAndWatermarks to be used if there are any idle partitions. It would be nice to mention in documentation that currently this requires all subscribed partitions to have a constant stream of data with growing timestamps. When watermark gets stalled on an idle partition it blocks everything.
 
Link to current documentation:
https://ci.apache.org/projects/flink/flink-docs-master/dev/connectors/kafka.html#kafka-consumers-and-timestamp-extractionwatermark-emission

Issue Links

• links to: GitHub Pull Request #5858