[SPARK-20045] Make sure SparkHadoopMapReduceWriter is resilient to failures of writers and committers

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Incomplete
• Affects Version/s: 2.2.0
• Component/s: Spark Core
• Labels: bulk-closed
• Assignee: Unassigned
• Reporter: Steve Loughran
• Created: 21/Mar/17 12:19
• Updated: 21/May/19 04:12
• Resolved: 21/May/19 04:12

Description

Following on from SPARK-20038: review SparkHadoopMapReduceWriter and ensure that it's failure handling code is itself resilient to follow on failures, especially in things like writer.close() and the abortTask/abortJob calls. That's to ensure as robust a cleanup as possible, and to stop the original exception getting lost.

At a quick glance

1. executeTask()'s catch logic should catch & log any failure in the writer.close()
2. The Hadoop commit protocol's abort* operation's can throw IOEs. Again, they need to be caught and logged

Should be testable with mocking, and worthwhile giving how important commit protocol resilience is.

Issue Links

• is related to: SPARK-21549 Spark fails to complete job correctly in case of OutputFormat which do not write into hdfs