[MAHOUT-976] Implement Multilayer Perceptron

• Type: New Feature
• Status: Closed
• Priority: Minor
• Resolution: Duplicate
• Affects Version/s: 0.7
• Fix Version/s: 0.9
• Labels: multilayer networks neural perceptron
• Assignee: Suneel Marthi
• Reporter: Christian Herta
• Created: 07/Feb/12 20:39
• Updated: 03/Feb/14 07:57
• Resolved: 20/Dec/13 21:17

Description

Implement a multi layer perceptron

• via Matrix Multiplication
• Learning by Backpropagation; implementing tricks by Yann LeCun et al.: "Efficent Backprop"
• arbitrary number of hidden layers (also 0 - just the linear model)
• connection between proximate layers only
• different cost and activation functions (different activation function in each layer)
• test of backprop by gradient checking
• normalization of the inputs (storeable) as part of the model

First:

• implementation "stocastic gradient descent" like gradient machine
• simple gradient descent incl. momentum

Later (new jira issues):

• Distributed Batch learning (see below)
• "Stacked (Denoising) Autoencoder" - Feature Learning
• advanced cost minimazation like 2nd order methods, conjugate gradient etc.

Distribution of learning can be done by (batch learning):
1 Partioning of the data in x chunks
2 Learning the weight changes as matrices in each chunk
3 Combining the matrixes and update of the weights - back to 2
Maybe this procedure can be done with random parts of the chunks (distributed quasi online learning).
Batch learning with delta-bar-delta heuristics for adapting the learning rates.

Issue Links

• is superceded by: MAHOUT-1265 Add Multilayer Perceptron