[SYSTEMDS-1774] Improve Parfor parallelism for deep learning

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: SystemML 1.0.0
• Fix Version/s: SystemML 0.15
• Component/s: Algorithms, Compiler, ParFor
• Labels: deeplearning
• Epic Link: Deep Learning DML Library
• Sprint: Sprint 2
• Assignee: Fei Hu
• Reporter: Fei Hu
• Created: 17/Jul/17 18:42
• Updated: 09/Sep/17 05:30
• Resolved: 26/Jul/17 17:47

Description

When running the distributed MNIST LeNet example , each mini-batch could ideally run in parallel without interaction. We try to force parfor (j in 1:parallel_batches) at line 137 of nn/examples/mnist_lenet_distrib_sgd.dml to be parfor (j in 1:parallel_batches, mode=REMOTE_SPARK, opt=CONSTRAINED) use REMOTE_SPARK mode, but got some errors about org.apache.sysml.runtime.DMLRuntimeException: Not supported: Instructions of type other than CP instructions using the mode SPARK, and the error java.lang.NullPointerException using the mode HYBRID_SPARK. More log information can be found at the following comments.

Issue Links

• depends upon: SYSTEMDS-1782 Extend parfor block indexing analysis
• is a child of: SYSTEMDS-1760 Improve engine robustness of distributed SGD training