[SYSTEMDS-3363] Federated Communication - Buffer Allocation

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: SystemDS 3.0
• Component/s: federated
• Labels: Federated
• Assignee: Unassigned
• Reporter: David Weissteiner
• Created: 01/May/22 11:14
• Updated: 24/May/22 15:58
• Resolved: 24/May/22 15:58

Description

In netty's encoding process for sending federated requests and federated responses, the respective buffer for the serialized data is allocated with the default size of 256 bytes. This leads to a huge performance bottleneck when transferring large data files, since it has to allocate a new buffer every 256 bytes.

Issue Links

• links to: GitHub Pull Request #1601