[SPARK-10078] Vector-free L-BFGS

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Incomplete
• Component/s: ML
• Labels: bulk-closed
• Assignee: Yanbo Liang
• Reporter: Xiangrui Meng
• Created: 18/Aug/15 02:15
• Updated: 21/May/19 04:15
• Resolved: 21/May/19 04:15

Description

This is to implement a scalable version of vector-free L-BFGS (http://papers.nips.cc/paper/5333-large-scale-l-bfgs-using-mapreduce.pdf).

Design document:
https://docs.google.com/document/d/1VGKxhg-D-6-vZGUAZ93l3ze2f3LBvTjfHRFVpX68kaw/edit?usp=sharing

Issue Links

• is related to: SPARK-16592 Improving ml.Logistic Regression on speed and scalability
• relates to: SPARK-8520 Improve GLM's scalability on number of features
• supercedes: SPARK-6567 Large linear model parallelism via a join and reduceByKey