[SPARK-25549] High level API to collect RDD statistics

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Won't Fix
• Affects Version/s: 3.0.0
• Component/s: Spark Core, SQL
• Assignee: Unassigned
• Reporter: L. C. Hsieh
• Created: 27/Sep/18 03:28
• Updated: 25/Aug/19 22:39
• Resolved: 25/Aug/19 22:39

Description

We have low level API SparkContext.submitMapStage used for collecting statistics of RDD. However it is too low level and is not so easy to use. We need a high level API for that.