[SINGA-10] Add Support for Recurrent Neural Networks (RNN)

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Assignee: Zheng Kaiping
• Reporter: wangwei
• Created: 13/Jun/15 07:25
• Updated: 26/Sep/15 15:10
• Resolved: 26/Sep/15 15:10

Description

The training algorithm for RNNs is Back-Propagation through time (BPTT). It is similar to the BP algorithm for feed-forward neural networks.

The model structures are quite different to feed-forward models. Hence, we may need to inherit the base NeuralNet class to create a RNN class. The RNN class overrides the SetupNeurlNet function to:

1. parse user configuration and create the RNN graph with (circles)
2. broke the circles and expand it through time.
3. create and setup layers

Model partitioning id not considered in this ticket.