[SYSTEMDS-1637] Memory-efficiency csv/dataset - binary block converters

• Type: Sub-task
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Fix Version/s: SystemML 1.1
• Component/s: APIs, Runtime
• Assignee: Unassigned
• Reporter: Matthias Boehm
• Created: 27/May/17 05:11
• Updated: 21/Dec/17 06:04

Description

This task aims to make the converters for (1) csv to binary block, and (2) dataset to binary block more memory-efficient via a lazy iterator, which reduces the map output memory requirement from all blocks of a partition to a single row of blocks.