[SPARK-1969] Public available online summarizer for mean, variance, min, and max

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Done
• Fix Version/s: 1.1.0
• Component/s: MLlib
• Assignee: DB Tsai
• Reporter: DB Tsai
• Created: 30/May/14 00:18
• Updated: 12/Jul/14 06:06
• Resolved: 12/Jul/14 06:06

Description

It basically moved the private ColumnStatisticsAggregator class from RowMatrix to public available DeveloperApi.

Changes:
1) Moved the trait from org.apache.spark.mllib.stat.MultivariateStatisticalSummary to org.apache.spark.mllib.stats.Summarizer
2) Moved the private implementation from org.apache.spark.mllib.linalg. ColumnStatisticsAggregator to org.apache.spark.mllib.stats.OnlineSummarizer
3) When creating OnlineSummarizer object, the number of columns is not needed in the constructor. It's determined when users add the first sample.
4) Added the API documentation for OnlineSummarizer
5) Added the unittest for OnlineSummarizer