[OOZIE-3054] Disable erasure coding for sharelib if Oozie runs on Hadoop 3

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 5.0.0b1
• Component/s: client
• Assignee: Peter Bacsko
• Reporter: Peter Bacsko
• Created: 11/Sep/17 17:18
• Updated: 25/Jan/18 21:00
• Resolved: 22/Sep/17 11:00

Description

Hadoop 3 will introduce Erasure Coding support for HDFS. This means less disk usage, but increased CPU load for reading/writing.

The contents of Oozie sharelib are localized every time an asynchronous action is executed. Therefore, to reduce the possible negative performance impact of EC, Oozie sharelib should not utilize it.

Since we're supporting Hadoop 2 as well, using the EC API will be done with Java reflection. Later, when we drop support for Hadoop 2, this can be rewritten.

Issue Links

• links to: ReviewBoard