[MAPREDUCE-7164] FileOutputCommitter does not report progress while merging paths.

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.0.3, 2.8.5, 2.9.2
• Fix Version/s: 3.0.4, 3.1.2, 3.3.0, 3.2.1
• Hadoop Flags: Reviewed
• Assignee: Kuhu Shukla
• Reporter: Kuhu Shukla
• Created: 26/Nov/18 20:45
• Updated: 29/Nov/18 17:24
• Resolved: 28/Nov/18 23:04

Description

In cases where the rename and merge path logic takes more time than usual, the committer does not report progress and can cause job failure. This behavior was not present in Hadoop 1.x. This JIRA will fix it so that the old behavior for 1.x is restored.

Issue Links

• is related to: HADOOP-15961 S3A committers: make sure there's regular progress() calls