[SYSTEMDS-2136] Text to binary block conversions fails for very wide, ultra-sparse data

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: SystemML 1.1
• Assignee: Matthias Boehm
• Reporter: Matthias Boehm
• Created: 08/Feb/18 00:36
• Updated: 24/Feb/18 09:38
• Resolved: 24/Feb/18 09:38

Description

org.apache.sysml.runtime.DMLRuntimeException: Unexpected error in sparseToDense().. limit < 0: 1000, -580983032, -1162447040
        at org.apache.sysml.runtime.matrix.data.MatrixBlock.sparseToDense(MatrixBlock.java:1143)
        at org.apache.sysml.runtime.matrix.data.MatrixBlock.examSparsity(MatrixBlock.java:1020)
        at org.apache.sysml.runtime.matrix.mapred.ReblockBuffer.outputBlock(ReblockBuffer.java:312)
        at org.apache.sysml.runtime.matrix.mapred.ReblockBuffer.flushBufferToBinaryBlocks(ReblockBuffer.java:264)
        at org.apache.sysml.runtime.instructions.spark.utils.RDDConverterUtils$CellToBinaryBlockFunction.flushBufferToList(RDDConverterUtils.java:506)
        at org.apache.sysml.runtime.instructions.spark.utils.RDDConverterUtils$TextToBinaryBlockFunction.call(RDDConverterUtils.java:549)
        at org.apache.sysml.runtime.instructions.spark.utils.RDDConverterUtils$TextToBinaryBlockFunction.call(RDDConverterUtils.java:511)