[FLINK-28045] [umbrella] Blockers for SourceFunction API removal (in Flink 2.0)

• Type: Technical Debt
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: Connectors / Common
• Assignee: Alexander Fedulov
• Reporter: Alexander Fedulov
• Created: 14/Jun/22 10:11
• Updated: 20/Oct/23 12:58

Description

This ticket should only contain items where SourceFunction is already used in the existing code base. For new functionality and Source V2 API improvements, use FLINK-32692.
—
Current state of migration to Source V2 API for officially-supported connectors
  FileSystem (Public)
  Kafka  (PublicEvolving)
  Cassandra (PublicEvolving)
  Pulsar (PublicEvolving)
  HiveSource (PublicEvolving)
  HybridSource (PublicEvolving)
  MongoDB (PublicEvolving)
  Hive (PublicEvolving)
  Kinesis (Experimental)
  JDBC - not impacted (based on TableFunction)
  HBase - not impacted (based on TableFunction)
  RabbitMQ
  Google Pub Sub

 

Links to externalized connector repos: https://cwiki.apache.org/confluence/display/FLINK/Externalized+Connector+development

Issue Links

• is a parent of: FLINK-28061 create new tech blog for connector development based on Source API
• is duplicated by: FLINK-25852 Annotate SourceFunction and SinkFunction as deprecated
• mentioned in: Page Loading...

Sub-Tasks

• Annotate SourceFunction as deprecated: Resolved
• Cascade deprecation to classes that implement SourceFunction: Resolved
• Cascade deprecation to non-private methods that reference SourceFunction: Resolved
• Migrate ParallelSourceFunction classes to Source V2 API or prepare for removal: Open
• Annotate StreamExecutionEnvironment#readFile()/readTextFile(*) methods deprecated in favor of FileSource#forRecordStreamFormat/forBulkFileFormat: Closed
• Introduce Source API alternatives for StreamExecutionEnvironment#fromCollection() methods: Closed
• Introduce Source API alternative to SourceExecutionContext#fromElements(*) methods: Closed
• Introduce Source API alternative to FiniteTestSource: Resolved
• Introduce Source API utilities for throttling number of records emitted per second.: Resolved
• Migrate o.a.f.streaming.examples to the new Source API: Closed
• Promote ExternallyInducedSourceReader to non-experimental @Public: Open
• Migrate RabbitMQ connector to Source V2 API: Open
• Migrage Google PubSub connector to V2: Open
• Promote Kinesis connector support to PublicEvolving: Open
• Migrate SourceFunction in Tests to Source V2 API: Open
• Remove usage of FromElementsFunction: Open
• Migrate LoadSimulationPipeline in Flink K8S Operator to the new Source API: Open