[SYSTEMDS-996] Unnecessary shuffle / block mismatch in frame right indexing

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: SystemML 0.11
• Fix Version/s: SystemML 0.11
• Epic Link: Frame data type support
• Assignee: Matthias Boehm
• Reporter: Matthias Boehm
• Created: 30/Sep/16 07:26
• Updated: 05/Oct/16 00:38
• Resolved: 30/Sep/16 20:40

Description

The implementation of our spark frame right indexing instruction unnecessarily tries to create aligned partial blocks, which causes unnecessary shuffle and merge issues (on merging these partial blocks) as frame can have variable block sizes. We should avoid merge/shuffle altogether and simply slice out the relevant sub blocks.

cc [e-mail] acs_s