Details
-
Improvement
-
Status: Resolved
-
Major
-
Resolution: Won't Fix
-
1.0.0
-
None
Description
Current implementation of GradientDescent is inefficient in some aspects, especially in high-latency network. I propose a new implementation of GradientDescent, which follows a parallelism model called GradientDescentWithLocalUpdate, inspired by Jeff Dean's DistBelief and Eric Xing's SSP. With a few modifications of runMiniBatchSGD, the GradientDescentWithLocalUpdate can outperform the original sequential version by about 4x without sacrificing accuracy, and can be easily adopted by most classification and regression algorithms in MLlib.
