[SYSTEMDS-2036] RemoveEmpty with selection vector updates nnz incorrectly

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: SystemML 1.0.0
• Epic Link: Perftest and QA for SystemML 1.0 release
• Assignee: Matthias Boehm
• Reporter: Matthias Boehm
• Created: 07/Dec/17 02:23
• Updated: 09/Dec/17 01:36
• Resolved: 09/Dec/17 01:36

Description

Detailed sparsity checks revealed an issue with removeEmpty and selection vectors, where not all rows with non zeros are selected. Since we only copy the nnz from the input, the output meta data is wrong which can lead to incorrect sparse/dense representations and in the worst-case incorrect results (e.g., for sum(X!=0)).

Caused by: org.apache.sysml.runtime.DMLRuntimeException: Matrix nnz meta data was incorrect: (_mVar7023, actual=4515, expected=4514, inst=CP°rmempty°margin=rows°select=_mVar7022°target=_mVar7009°bRmEmptyBC=false°_mVar7023·MATRIX·DOUBLE)
        at org.apache.sysml.runtime.controlprogram.ProgramBlock.checkSparsity(ProgramBlock.java:384)
        at org.apache.sysml.runtime.controlprogram.ProgramBlock.executeSingleInstruction(ProgramBlock.java:297)
        ... 10 more