[HDFS-14679] Failed to add erasure code policies with example template

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 3.1.2
• Fix Version/s: 3.3.0
• Component/s: ec
• Hadoop Flags: Reviewed
• Assignee: Yuan Zhou
• Reporter: Yuan Zhou
• Created: 30/Jul/19 03:12
• Updated: 06/Aug/19 03:30
• Resolved: 06/Aug/19 03:30

Description

Hi Hadoop developers,

 

Trying to do some quick tests with erasure coding feature and ran into a issue on adding policies. The example on adding erasure code policies with provided template failed:

./bin/hdfs ec -addPolicies -policyFile /tmp/user_ec_policies.xml.template
2019-07-30 10:35:16,447 INFO util.ECPolicyLoader: Loading EC policy file /tmp/user_ec_policies.xml.template
Add ErasureCodingPolicy XOR-2-1-128k succeed.
Add ErasureCodingPolicy RS-LEGACY-12-4-256k failed and error message is Codec name RS-legacy is not supported

The issue seems due to be the mismatching codec(upper case vs lower case). The codec is in upper case in the example template[1] while all available codecs are lower case[2]. A way to fix maybe just converting the codec to lower case when parsing the policy schema. Also attached a simple patch here. 

[1] https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/conf/user_ec_policies.xml.template#L51

[2]https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/erasurecode/ErasureCodeConstants.java#L28-L33

Thanks, -yuan