[HADOOP-13665] Erasure Coding codec should support fallback coder

• Type: Sub-task
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Fix Version/s: 3.0.0-alpha4
• Component/s: io
• Labels: hdfs-ec-3.0-must-do
• Target Version/s: 3.0.0-alpha4
• Hadoop Flags: Incompatible change, Reviewed
• Release Note: Use configuration properties io.erasurecode.codec.{rs-legacy,rs,xor}.rawcoders to control erasure coding codec. These properties support codec fallback in case the previous codec is not loaded.
• Assignee: Kai
• Reporter: Wei-Chiu Chuang
• Created: 27/Sep/16 16:55
• Updated: 02/Oct/19 17:14
• Resolved: 11/Apr/17 14:36

Description

The current EC codec supports a single coder only (by default pure Java implementation). If the native coder is specified but is unavailable, it should fallback to pure Java implementation.

One possible solution is to follow the convention of existing Hadoop native codec, such as transport encryption (see CryptoCodec.java). It supports fallback by specifying two or multiple coders as the value of property, and loads coders in order.

Issue Links

• is depended upon by: HADOOP-14297 Update the documentation about the new ec codecs config keys
• is related to: HADOOP-13200 Implement customizable and configurable erasure coders
• relates to: HADOOP-13061 Refactor erasure coders