[SPARK-14619] Track internal accumulators (metrics) by stage attempt rather than stage

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.0.0
• Component/s: Spark Core
• Target Version/s: 2.0.0
• Assignee: Reynold Xin
• Reporter: Reynold Xin
• Created: 14/Apr/16 01:17
• Updated: 14/Apr/16 17:55
• Resolved: 14/Apr/16 17:55

Description

When there are multiple attempts for a stage, we currently only reset internal accumulator values if all the tasks are resubmitted. It would make more sense to reset the accumulator values for each stage attempt. This will allow us to eventually get rid of the internal flag in the Accumulator class.

Issue Links

• links to: [Github] Pull Request #12378 (rxin)