Details

      Description

      Implement a multi layer perceptron

      • via Matrix Multiplication
      • Learning by Backpropagation; implementing tricks by Yann LeCun et al.: "Efficent Backprop"
      • arbitrary number of hidden layers (also 0 - just the linear model)
      • connection between proximate layers only
      • different cost and activation functions (different activation function in each layer)
      • test of backprop by gradient checking
      • normalization of the inputs (storeable) as part of the model

      First:

      • implementation "stocastic gradient descent" like gradient machine
      • simple gradient descent incl. momentum

      Later (new jira issues):

      • Distributed Batch learning (see below)
      • "Stacked (Denoising) Autoencoder" - Feature Learning
      • advanced cost minimazation like 2nd order methods, conjugate gradient etc.

      Distribution of learning can be done by (batch learning):
      1 Partioning of the data in x chunks
      2 Learning the weight changes as matrices in each chunk
      3 Combining the matrixes and update of the weights - back to 2
      Maybe this procedure can be done with random parts of the chunks (distributed quasi online learning).
      Batch learning with delta-bar-delta heuristics for adapting the learning rates.

        Attachments

        1. MAHOUT-976.patch
          37 kB
          Christian Herta
        2. MAHOUT-976.patch
          30 kB
          Christian Herta
        3. MAHOUT-976.patch
          27 kB
          Christian Herta
        4. MAHOUT-976.patch
          19 kB
          Christian Herta

          Issue Links

            Activity

              People

              • Assignee:
                smarthi Suneel Marthi
                Reporter:
                chrisberlin Christian Herta
              • Votes:
                0 Vote for this issue
                Watchers:
                8 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - 80h
                  80h
                  Remaining:
                  Remaining Estimate - 80h
                  80h
                  Logged:
                  Time Spent - Not Specified
                  Not Specified