[SPARK-1157] L-BFGS Optimizer

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.0.0
• Assignee: DB Tsai
• Reporter: DB Tsai
• Created: 01/Mar/14 23:37
• Updated: 15/Apr/14 18:14
• Resolved: 15/Apr/14 18:14

Description

L-BFGS (Limited-memory BFGS) is an optimization algorithm like BFGS which uses an approximation to the inverse of Hessian matrix to steer its search through the variable space, but where BFGS stores a dense nxn approximation to the inverse Hessian, L-BFGS only stores a few vectors to represent the approximation.

For high dimensional optimization problems, the Newton method or BFGS is not applicable since the amount of memory needed to store the Hessian will grow exponentially, while L-BFGS only stores couple vectors.

One of the use case can be training large-scale logistic regression with so many features.

We'll use breeze implementation of L-BFGS.