[FLINK-12477] Change threading-model in StreamTask to a mailbox-based approach

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.10.0
• Component/s: Runtime / Task
• Labels: pull-request-available
• Assignee: Unassigned
• Reporter: Stefan Richter
• Created: 10/May/19 09:31
• Updated: 25/Feb/20 10:53
• Resolved: 11/Feb/20 12:03

Description

This is the umbrella issue to change the threading-model in StreamTask to a mailbox-based approach.

You can find the corresponding design doc here:

https://docs.google.com/document/d/1eDpsUKv2FqwZiS1Pm6gYO5eFHScBHfULKmH1-ZEWB4g

Please comment and ask first before assigning sub-task. Thanks!

Issue Links

• links to: GitHub Pull Request #8414

Sub-Tasks

• Decompose monolithic run-loops in StreamTask implementations into step-wise calls: Closed
• Make InputProcessors and StreamIterationHead non-blocking: Closed
• Introduce mailbox to StreamTask main-loop: Closed
• Support (legacy) SourceFunctions with mailbox: Closed
• Introduce mailbox-based ExecutorService: Closed
• Enhance mailbox executor with yield-to-downstream functionality: Closed
• Make processing time timer trigger run via the mailbox: Closed
• Make checkpoint trigger/notifyComplete run via the mailbox queue: Closed
• Drop unnecessary locking for code-paths that now use the mailbox: Closed
• Make BufferOrEventSequence#getNext() non-blocking: Closed
• Integrate AsyncWaitOperator with mailbox: Closed
• Make sure that SingleInputGate can always request partitions: Closed
• Integrate ContinuousFileReaderOperator with StreamTask mailbox execution model: Closed
• Execute/run processing timer triggers taking into account operator level mailbox loops: Closed
• Deprecate checkpoint lock from the Operators API: Closed