[FLINK-1994] Add different gain calculation schemes to SGD

• Type: Improvement
• Status: Closed
• Priority: Minor
• Resolution: Fixed
• Component/s: Library / Machine Learning
• Labels: ML Starter
• Assignee: Trevor Grant
• Reporter: Till Rohrmann
• Created: 08/May/15 10:17
• Updated: 22/Jan/16 15:06
• Resolved: 22/Jan/16 15:06

Description

The current SGD implementation uses as gain for the weight updates the formula stepsize/sqrt(iterationNumber). It would be good to make the gain calculation configurable and to provide different strategies for that. For example:

• stepsize/(1 + iterationNumber)
• stepsize*(1 + regularization * stepsize * iterationNumber)^(-3/4)

See also how to properly select the gains [1].

Resources:
[1] http://arxiv.org/pdf/1107.2490.pdf

Issue Links

• requires: FLINK-1807 Stochastic gradient descent optimizer for ML library