Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-10740 FLIP-27: Refactor Source Interface
  3. FLINK-17699

Reduce scope for SourceOperator arguments and initialize more eagerly

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Closed
    • Minor
    • Resolution: Fixed
    • None
    • 1.11.0
    • API / DataStream
    • None

    Description

      Currently, the SourceOperator only gets a Source in the constructor.

      All actual components that the SourceOperator relies on when working are lazily initialized, in open() or via setters.

      Relying on something as broad as Source also means that a lot of redundant context has to be provided to the SourceOperator during initialization. The Source is, for example, also responsible for the SourceEnumerator, which is independent of the SourceOperator. However, it needs to be considered during testing, now, because the tests need to mock a full Source in order to instantiate a SourceOperator.

      The solution is to directly pass the collaborators of the SourceOperator directly eagerly into the constructor. It is not fully possible with the SourceReader, but for that we can at least reduce the scope by passing a targeted factory function.

      Attachments

        Activity

          People

            sewen Stephan Ewen
            sewen Stephan Ewen
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: