Details
-
Sub-task
-
Status: Open
-
Major
-
Resolution: Unresolved
-
None
-
None
Description
This task aims to make the converters for (1) csv to binary block, and (2) dataset to binary block more memory-efficient via a lazy iterator, which reduces the map output memory requirement from all blocks of a partition to a single row of blocks.