[SYSTEMDS-1819] Create Keras2DML: Keras frontend to SystemML

• Type: New Feature
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: SystemML 1.0.0
• Epic Link: Deep Learning
• Assignee: Anooj Patel
• Reporter: Mike Dusenberry
• Created: 31/Jul/17 20:11
• Updated: 20/Sep/17 19:10
• Resolved: 20/Sep/17 19:10

Description

This task covers the creation of a "Keras2DML" frontend for SystemML, built upon the Caffe2DML infrastructure, that will allow users to define (and even train) models in Keras and then import them into SystemML for distributed training and prediction. As an initial set of thoughts, the input could be either (1) a Keras Model object, or (2) a saved Keras model hdf5 file, and the output of training could be either (1) a Keras Model object, (2) a saved Keras model hdf5 file, or (3) a SystemML model.

This would be a step towards a full-blown, official backend for Keras. The main goal here would be to allow users to be able to transparently make use of distributed training, without having to learn the details of SystemML.

Initial steps:
1. Learn Keras
2. Learn Caffe2DML: http://apache.github.io/systemml/beginners-guide-caffe2dml Basically, Caffe2DML lets users import Caffe models (architecture and trained weights if available) into SystemML and train/predict on Spark with a scikit-learn compatible API without the user having to learn SystemML. The main benefit is distributed training without the user needing to think about it much. A bunch of the infrastructure is in place that I think Keras2DML would use.
3. Import a simple Keras model definition with a single Dense layer, and focus on hooking up the new Keras2DML class to the existing infrastructure.
4. Add reading of trained weights from Keras for the simple model, and hook up to existing infrastructure.
5. Expand out to increasingly complex models, aiming to be able to import all of the pretrained models from Keras, starting with VGG16 & ResNet50. https://keras.io/applications/

Issue Links

• is related to: SYSTEMDS-1909 Keras2DML: Determine the loss of precision in large networks for pretrained models such as ResNet50.
• is related to: SYSTEMDS-1910 Keras2DML: Implement TEST and TRAIN when generating Caffe prototext and caffemodels.
• is related to: SYSTEMDS-1912 Keras2DML: Generalize the CaffeSolver class to pull optimizer information and other training/testing information from Keras models.
• is related to: SYSTEMDS-1911 Pull out the Caffe2DML into a generalized Framework2DML so Keras2DML and Caffe2DML are 2 separate entities.