[SPARK-19306] Fix inconsistent state in DiskBlockObjectWriter when exception occurred

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 2.1.0
• Fix Version/s: 2.1.1, 2.2.0
• Component/s: Spark Core
• Assignee: Saisai Shao
• Reporter: Saisai Shao
• Created: 20/Jan/17 07:49
• Updated: 23/Jan/17 21:38
• Resolved: 23/Jan/17 21:38

Description

In DiskBlockObjectWriter, when some errors happened during writing, it will call revertPartialWritesAndClose, if this method again failed due to some hardware issues like out of disk, it will throw exception without resetting the state of this writer, also skipping the revert. So here propose to fix this issue to offer user a chance to recover from such issue.

Issue Links

• links to: [Github] Pull Request #16657 (jerryshao)