Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-28045

[umbrella] Blockers for SourceFunction API removal (in Flink 2.0)

    XMLWordPrintableJSON

Details

    • Technical Debt
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • None
    • Connectors / Common
    • None

    Description

      This ticket should only contain items where SourceFunction is already used in the existing code base. For new functionality and Source V2 API improvements, use FLINK-32692.

      Current state of migration to Source V2 API for officially-supported connectors
        FileSystem (Public)
        Kafka  (PublicEvolving)
        Cassandra (PublicEvolving)
        Pulsar (PublicEvolving)
        HiveSource (PublicEvolving)
        HybridSource (PublicEvolving)
        MongoDB (PublicEvolving)
        Hive (PublicEvolving)
        Kinesis (Experimental)
        JDBC - not impacted (based on TableFunction)
        HBase - not impacted (based on TableFunction)
        RabbitMQ
        Google Pub Sub

       

      Links to externalized connector repos: https://cwiki.apache.org/confluence/display/FLINK/Externalized+Connector+development

      Attachments

        Issue Links

          1.
          Annotate SourceFunction as deprecated Sub-task Resolved Alexander Fedulov
          2.
          Cascade deprecation to classes that implement SourceFunction Sub-task Resolved Alexander Fedulov
          3.
          Cascade deprecation to non-private methods that reference SourceFunction Sub-task Resolved Alexander Fedulov
          4.
          Migrate ParallelSourceFunction classes to Source V2 API or prepare for removal Sub-task Open Alexander Fedulov
          5.
          Annotate StreamExecutionEnvironment#readFile()/readTextFile(*) methods deprecated in favor of FileSource#forRecordStreamFormat/forBulkFileFormat Sub-task Closed Alexander Fedulov
          6.
          Introduce Source API alternatives for StreamExecutionEnvironment#fromCollection() methods Sub-task Closed Alexander Fedulov
          7.
          Introduce Source API alternative to SourceExecutionContext#fromElements(*) methods Sub-task Closed Alexander Fedulov
          8.
          Introduce Source API alternative to FiniteTestSource Sub-task Open Alexander Fedulov
          9.
          Introduce Source API utilities for throttling number of records emitted per second. Sub-task Resolved Alexander Fedulov
          10.
          Migrate o.a.f.streaming.examples to the new Source API Sub-task Closed Alexander Fedulov
          11.
          Promote ExternallyInducedSourceReader to non-experimental @Public Sub-task Open Unassigned
          12.
          Migrate RabbitMQ connector to Source V2 API Sub-task Open Unassigned
          13.
          Migrage Google PubSub connector to V2 Sub-task Open Unassigned
          14.
          Promote Kinesis connector support to PublicEvolving Sub-task Open Unassigned
          15.
          Migrate SourceFunction in Tests to Source V2 API Sub-task Open Alexander Fedulov
          16.
          Remove usage of FromElementsFunction Sub-task Open Alexander Fedulov
          17.
          Migrate LoadSimulationPipeline in Flink K8S Operator to the new Source API Sub-task Open Unassigned

          Activity

            People

              afedulov Alexander Fedulov
              afedulov Alexander Fedulov
              Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

                Created:
                Updated: