[MAPREDUCE-5812] Make job context available to OutputCommitter.isRecoverySupported()

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.3.0
• Fix Version/s: 2.5.0
• Component/s: mr-am
• Target Version/s: 2.5.0
• Hadoop Flags: Reviewed
• Assignee: Mohammad Islam
• Reporter: Mohammad Islam
• Created: 27/Mar/14 06:38
• Updated: 03/Sep/14 20:33
• Resolved: 28/Apr/14 15:27

Description

Background
==========
The system like Hive provides its version of OutputCommitter. The custom implementation of isRecoverySupported() requires task context. From taskContext:getConfiguration(), hive checks if hive-defined specific property is set or not. Based on the property value, it returns true or false. However, in the current OutputCommitter:isRecoverySupported(), there is no way of getting task config. As a result, user can't turn on/off the MRAM recovery feature.

Proposed resolution:
===============
1. Pass Task Context into isRecoverySupported() method.
Pros: Easy and clean
Cons: Possible backward compatibility issue due to aPI changes. (Is it true?)

2. Call outputCommitter.setupTask(taskContext) from MRAM: The new OutputCommitter will store the context in the class level variable and use it from isRecoverySupported()

Props: No API changes. No backward compatibility issue. This call can be made from MRAppMaster.getOutputCommitter() method for old API case.
Cons: Might not be very clean solution due to class level variable.

Please give your comments.

Issue Links

• is related to: HIVE-6638 Hive needs to implement recovery for Application Master restart