[FLINK-13924] Add summarizer and summary for sparse vector and dense vector.

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Won't Do
• Component/s: Library / Machine Learning
• Labels: pull-request-available
• Assignee: Unassigned
• Reporter: Xu Yang
• Created: 30/Aug/19 14:46
• Updated: 16/Oct/22 08:27
• Resolved: 16/Oct/22 08:27

Description

Summarizer is the class for calculating statistics, summary is the result class of summarizer. Summary defines methods to get statistics. Assuming that the data has dense vector and sparse vector, vectors size are not equal also, so if DenseVectorSummarizer visit a sparse vector, it will change to SparseVectorSummarizer.
Statistics include vectorSize, count, mean, variance, min, max, standardDeviation, normL1, normL2.

• Add SparseVectorSummarizer which will calculate statistics for sparse vector.
• Add SparseVectorSummary which can get statistics of sparse vector.
• Add DenseVectorSummarizer which will calculate statistics for dense vector.
• Add DenseVectorSummary which can get statistics of sparse vector.
• Add StatisticsUtil which provides utility functions for summarizer and summary.
• Add VectorSummarizerUtil which provides utility functions for VectorSummarizer.

Issue Links

• Dependent: FLINK-13667 Add the utility class for the Table
• Dependent: FLINK-13915 Add several base classes of summarizer.
• links to: GitHub Pull Request #9578