[FLINK-1296] Add support for very large record for sorting

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.8.0
• Fix Version/s: 0.9
• Component/s: Runtime / Task
• Assignee: Stephan Ewen
• Reporter: Stephan Ewen
• Created: 01/Dec/14 15:51
• Updated: 04/Feb/15 22:15
• Resolved: 04/Feb/15 22:15

Description

Currently, very large records (multiple hundreds of megabytes) can break the sorter if the overflow the sort buffer.

Furthermore, if a merge is attempted of those records, pulling multiple of them concurrently into memory can break the machine memory.