[FLINK-19251] Avoid confusing queue handling in "SplitReader.handleSplitsChanges()"

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.11.3, 1.12.0
• Component/s: Connectors / Common
• Labels: pull-request-available
• Assignee: Stephan Ewen
• Reporter: Stephan Ewen
• Created: 15/Sep/20 21:43
• Updated: 15/Nov/20 22:11
• Resolved: 17/Sep/20 09:23

Description

Currently, the method SplitReader.handleSplitsChanges() is passed a queue of split changes to handle. The method may decide to handle only a subset of them and is passes later all remaining changes.

In practice, this ends up being confusing and problematic:

• It is important to remove the elements from the queue, not accidentally iterate, or the splits will get handles multiple times
• If the queue is not left empty, the task to handle the changes is immediately re-enqueued. No other operation can happen before all split changes from the queue are handled.

A simpler and more efficient contract would be to simply pass a list of split changes directly and once, for the fetcher to handle. For all implementations so far, this was sufficient and easier.

Issue Links

• links to: GitHub Pull Request #13400