[SPARK-12896] Send only accumulator updates, not TaskMetrics, to the driver

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.0.0
• Component/s: Spark Core
• Target Version/s: 2.0.0
• Assignee: Andrew Or
• Reporter: Andrew Or
• Created: 19/Jan/16 01:16
• Updated: 27/Jan/16 19:59
• Resolved: 27/Jan/16 19:59

Description

Currently in Spark there are two different things that the executors send to the driver, accumulators and TaskMetrics. It would allow us to clean up a lot of things if we send ONLY accumulator updates instead of both that and TaskMetrics. Eventually we will be able to make TaskMetrics private and maybe even remove it.

This blocks on SPARK-12895.

Issue Links

• links to: [Github] Pull Request #10835 (andrewor14)
• links to: [Github] Pull Request #10857 (andrewor14)