[MAPREDUCE-6954] Disable erasure coding for files that are uploaded to the MR staging area

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.0.0-alpha4
• Fix Version/s: 3.0.0-beta1
• Component/s: client
• Hadoop Flags: Reviewed
• Assignee: Peter Bacsko
• Reporter: Peter Bacsko
• Created: 08/Sep/17 12:58
• Updated: 26/Feb/18 14:14
• Resolved: 15/Sep/17 19:07

Description

Depending on the encoder/decoder used and the type or MR workload, EC might negatively affect the performance of an MR job if too many files are localized.

In such a scenario, users might want to disable EC in the staging area to speed up the execution.

Issue Links

• breaks: MAPREDUCE-7059 Downward Compatibility issue: MR job fails because of unknown setErasureCodingPolicy method from 3.x client to HDFS 2.x cluster
• breaks: MAPREDUCE-6968 Staging directory erasure coding config property has a typo
• relates to: HDFS-12919 RBF: Support erasure coding methods in RouterRpcServer