[FLINK-20194] KafkaSourceFetcherManager.commitOffsets() should handle the case when there is no split fetcher.

• Type: Bug
• Status: Closed
• Priority: Blocker
• Resolution: Fixed
• Fix Version/s: 1.11.3, 1.12.0
• Component/s: Connectors / Kafka
• Labels: pull-request-available
• Assignee: Jiangjie Qin
• Reporter: Jiangjie Qin
• Created: 17/Nov/20 15:25
• Updated: 24/Nov/20 13:13
• Resolved: 24/Nov/20 07:43

Description

The KafkaSourceFetcherManager.commitOffsets() may throw NPE when there is no split fetcher. Given that such cases should usually be rare and only happens when the source is shutting down or just started. We can create a new KafkaPartitionSplitReader to commit offset and close it immediately. The main thread maybe blocked for some time, but given that there is no event to process at that point, there should be no performance concerns.

Issue Links

• blocks: FLINK-20157 SourceCoordinatorProvider kills JobManager with IllegalStateException on job submission
• blocks: FLINK-20114 Fix a few KafkaSource-related bugs
• links to: GitHub Pull Request #14142