[SYSTEMDS-3542] Compressed transfer to/from federated workers

• Type: Task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: SystemDS 3.2
• Labels: StudentProject
• Assignee: Unassigned
• Reporter: Matthias Boehm
• Created: 26/Apr/23 20:29
• Updated: 04/Dec/23 04:02
• Resolved: 04/Dec/23 04:02

Description

Currently, the transfer of data from and to a federated worker is send through our default serializers.
A consequence is slow transfer of a large amounts of data across limited network connections.
This project is to setup federated workers with limited network connectivity and experiment with
standard compression techniques of byte streams to improve the transfer.

Issue Links

• links to: GitHub Pull Request #1867