[FLINK-2319] Collect size limitation due to akka.

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Later
• Affects Version/s: 0.10.0
• Component/s: Runtime / Coordination
• Flags: Important
• Assignee: Unassigned
• Reporter: Kostas Kloudas
• Created: 05/Jul/15 17:52
• Updated: 28/Feb/19 14:01
• Resolved: 03/Feb/17 19:34

Description

Each TaksManager keeps the results of a local task in a set of Accumulators. Upon termination of the task, the Accumulators are sent back to the JobManager, who merges them, before sending them back to the Client.

To exchange the Accumulators and their content, akka is used. This limits the size of the output of a task to no more than akka.framesize bytes. In other case, akka would drop the message.

This ticket is to propose the removal of this limitation so that results can be of arbitrary size.

Issue Links

• is superceded by: FLINK-4399 Add support for oversized messages