[SAMZA-2761] Shutdown after end-of-stream should wait for all messages in flight completion

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Assignee: Xinyu Liu
• Reporter: Xinyu Liu
• Created: 19/Sep/22 22:28
• Updated: 19/Sep/22 22:28

Description

When all input partitions reach end-of-stream, there might be messages still in flight, in case of async and multithreading. So we should wait till these messages being completely processed before shutting down the pipeline. Otherwise we will not have fully processed all the messages.