[MAPREDUCE-7060] Cherry Pick PathOutputCommitter class/factory to branch-3.0

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 3.0.0, 2.10.0
• Fix Version/s: 3.0.3
• Target Version/s: 3.0.2
• Assignee: Steve Loughran
• Reporter: Steve Loughran
• Created: 26/Feb/18 15:02
• Updated: 09/May/18 20:53
• Resolved: 01/Mar/18 18:41

Description

It's easier for downstream apps like Spark to pick up the new PathOutputCommitter superclass if it is there on 2.10+, even if the S3A committer isn't there.

Adding the interface & binding stuff of MAPREDUCE-6956 allows for third party committers to be deployed.

I'm not proposing a backport of the HADOOP-13786 committer: that's Java 8, S3Guard, etc. Too traumatic. All I want here is to allow downstream code to be able to pick up the new interface and so be able to support it and other store committers when available

Issue Links

• is related to: MAPREDUCE-6956 FileOutputCommitter to gain abstract superclass PathOutputCommitter
• relates to: HADOOP-13786 Add S3A committers for zero-rename commits to S3 endpoints