-
Type:
Bug
-
Status: Closed
-
Priority:
Major
-
Resolution: Fixed
-
Affects Version/s: None
-
Fix Version/s: SystemML 0.15
-
Component/s: Algorithms, ParFor, Runtime
-
Labels:None
-
Epic Link:
-
Sprint:Sprint 2
When running the distributed MNIST LeNet example , it works well in the hybrid mode. But in the Spark mode, there are some errors about
java.lang.NullPointerException and java.lang.ArrayIndexOutOfBoundsException: 1000 when reshaping the matrix. The involved functions are org.apache.sysml.runtime.matrix.data.LibMatrixReorg#reshapeSparse and org.apache.sysml.runtime.matrix.data.LibMatrixReorg#reshapeDense. The reason is that the output matrix index computed by org.apache.sysml.runtime.matrix.data.LibMatrixReorg#computeResultBlockIndex does not match the keys in the HashMap<MatrixIndexes,MatrixBlock> rix.
To reproduce the error, the attached scala file MNIST_Distrib_Sgd.scala could be used to run the distributed MNIST example.
In addition, if adding some codes to ignore the null output matrix block from MatrixBlock out = rix.get(ixtmp), the distributed MNIST example could run in the Spark mode, but the result may not be right.