[SPARK-21621] Reset numRecordsWritten after DiskBlockObjectWriter.commitAndGet called

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.2.0
• Fix Version/s: 2.2.1, 2.3.0
• Component/s: Spark Core
• Assignee: Xianyang Liu
• Reporter: Xianyang Liu
• Created: 03/Aug/17 06:03
• Updated: 07/Aug/17 09:06
• Resolved: 07/Aug/17 09:05

Description

We should reset numRecordsWritten to zero after DiskBlockObjectWriter.commitAndGet called.
Because when `revertPartialWritesAndClose` be called, we decrease the written records in `ShuffleWriteMetrics` . However, we decreased the written records to zero, this should be wrong, we should only decreased the number reords after the last `commitAndGet` called.