[SYSTEMDS-1142] ParFor execution crashes bc of singlethreaded warning in LibMatrixDNN

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: SystemML 0.11
• Component/s: ParFor
• Environment: ParFor Spark on Cluster with DeepLearning usecase
• Epic Link: Enable parfor on remote Spark
• Assignee: Niketan Pansare
• Reporter: Felix Schueler
• Created: 06/Dec/16 20:18
• Updated: 20/Dec/16 20:39
• Resolved: 06/Dec/16 21:31

Description

When using functions from the LibMatrixDNN in a Spark ParFor block an exception is thrown by the function `warnSingleThreaded()` due to single threaded execution of functions such as conv2d.

Disabling this warning resolves the crash but I am not sure this is the right way to go there. The question is also why this executes with one thread only and how to improve here.

cc niketanpansare

Issue Links

• is related to: SYSTEMDS-540 Deep Learning