[FLINK-25509] FLIP-208: Add RecordEvaluator to dynamically stop source based on de-serialized records

• Type: New Feature
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Fix Version/s: kafka-4.0.0
• Component/s: Connectors / Common, Connectors / Kafka
• Labels: pull-request-available stale-assigned
• Assignee: Ruan Hang
• Reporter: Dong Lin
• Created: 04/Jan/22 07:02
• Updated: 04/Apr/24 12:04

Description

This feature is needed to migrate applications which uses KafkaDeserializationSchema::isEndOfStream() from using FlinkKafkaConsumer to using KafkaSource.

Please checkout https://cwiki.apache.org/confluence/display/FLINK/FLIP-208%3A+Add+RecordEvaluator+to+dynamically+stop+source+based+on+de-serialized+records for the motivation and the proposed changes.

Issue Links

• links to: GitHub Pull Request #76
• links to: GitHub Pull Request #21589

Sub-Tasks

• Adapt SplitFetcherManager#removeSplit for flink-connector-pulsar: Closed