[FLINK-19547] Partial Record Cleanup after the Consumer Task Fails and Restart

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.12.0
• Fix Version/s: 1.12.0
• Component/s: Runtime / Network
• Labels: pull-request-available
• Assignee: Yuan Mei
• Reporter: Yuan Mei
• Created: 09/Oct/20 03:22
• Updated: 30/Oct/20 09:18
• Resolved: 27/Oct/20 08:01

Description

Partial records happen if a record can not fit into one buffer, then the remaining part of the same record is put into the next buffer. Hence partial records only exist at the beginning of a buffer. 

Partial record clean-up is needed in the mode of approximate local recovery. If a record is spanning over multiple buffers, and the first (several) buffers have got lost due to the failure of the receiver task, the remaining data belonging to the same record in transition should be cleaned up.

Issue Links

• links to: GitHub Pull Request #13614