Details
-
Improvement
-
Status: Open
-
Minor
-
Resolution: Unresolved
-
3.0.0-alpha2
-
None
Description
Potentially, multiple native erasure coding plugins will be available to be used from HDFS later on. These plugins should be tested as well. For example, the NativeRSRawErasureCoderFactory class - which is used for instantiating the native ISA-L plugin's encoder and decoder objects - are used in 5 test files under the hadoop/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/ directory. The files are:
- TestDFSStripedInputStream.java
- TestDFSStripedOutputStream.java
- TestDFSStripedOutputStreamWithFailure.java
- TestReconstructStripedFile.java
- TestUnsetAndChangeDirectoryEcPolicy.java
Other erasure coding plugins should be tested in these cases as well in a nice way (not by for example making a new file for every new erasure coding plugin). For this purpose parameterized tests might be used.
This is also true for the hadoop/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/erasurecode/rawcoder/ directory where this approach could be used for example for the interoperability tests (when it is checked that certain erasure coding implementations are compatible with each other by doing the encoding and decoding operations with different plugins and verifying their results). The plugin pairs which should be tested could be the parameters for the parameterized tests.
The parameterized test is just an idea, there can be other solutions as well.