Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-19251

Avoid confusing queue handling in "SplitReader.handleSplitsChanges()"

    XMLWordPrintableJSON

    Details

      Description

      Currently, the method SplitReader.handleSplitsChanges() is passed a queue of split changes to handle. The method may decide to handle only a subset of them and is passes later all remaining changes.

      In practice, this ends up being confusing and problematic:

      • It is important to remove the elements from the queue, not accidentally iterate, or the splits will get handles multiple times
      • If the queue is not left empty, the task to handle the changes is immediately re-enqueued. No other operation can happen before all split changes from the queue are handled.

      A simpler and more efficient contract would be to simply pass a list of split changes directly and once, for the fetcher to handle. For all implementations so far, this was sufficient and easier.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                sewen Stephan Ewen
                Reporter:
                sewen Stephan Ewen
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: