[SYSTEMDS-1818] Perftest: Kmeans train fails for 10Kx100, k=50, w/ forced singlenode

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: SystemML 0.15
• Assignee: Matthias Boehm
• Reporter: Matthias Boehm
• Created: 29/Jul/17 21:02
• Updated: 09/Sep/17 03:33
• Resolved: 29/Jul/17 22:50

Description

The kmeans algorithm fails when forced to singlenode with the following exception:

Caused by: org.apache.sysml.runtime.DMLRuntimeException: Invalid values for matrix indexing: dimensions of the source matrix [451x10] do not match the shape of the matrix specified by indices [1:10, 1:10].
        at org.apache.sysml.runtime.matrix.data.MatrixBlock.leftIndexingOperations(MatrixBlock.java:3654)
        at org.apache.sysml.runtime.matrix.data.MatrixBlock.leftIndexingOperations(MatrixBlock.java:3631)
        at org.apache.sysml.runtime.instructions.cp.MatrixIndexingCPInstruction.processInstruction(MatrixIndexingCPInstruction.java:95)
        at org.apache.sysml.runtime.controlprogram.ProgramBlock.executeSingleInstruction(ProgramBlock.java:286)

Furthermore, there seems to be an issue of unnecessary spark context creation - in hybrid_spark mode we do not instantiate the spark context, while in forced singlenode we do.