[HADOOP-14479] Erasurecode testcase failures with native enabled

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 3.0.0-alpha3
• Fix Version/s: 3.0.0-alpha4
• Component/s: common
• Labels: hdfs-ec-3.0-must-do

Environment

x86_64 Ubuntu 16.04.02 LTS

• Target Version/s: 3.0.0-alpha4
• Hadoop Flags: Reviewed
• Assignee: Sammi Chen
• Reporter: Ayappan
• Created: 02/Jun/17 14:27
• Updated: 19/Aug/17 02:50
• Resolved: 29/Jun/17 03:28

Description

I built hadoop with ISA-L support. I took the ISA-L code from https://github.com/01org/isa-l (tag v2.18.0) and built it. While running the UTs , following three testcases are failing

1)TestHHXORErasureCoder

Tests run: 7, Failures: 3, Errors: 0, Skipped: 0, Time elapsed: 1.106 sec <<< FAILURE! - in org.apache.hadoop.io.erasurecode.coder.TestHHXORErasureCoder
testCodingDirectBuffer_10x4_erasing_p1(org.apache.hadoop.io.erasurecode.coder.TestHHXORErasureCoder) Time elapsed: 0.029 sec <<< FAILURE!
java.lang.AssertionError: Decoding and comparing failed.
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.assertTrue(Assert.java:41)
at org.apache.hadoop.io.erasurecode.TestCoderBase.compareAndVerify(TestCoderBase.java:170)
at org.apache.hadoop.io.erasurecode.coder.TestErasureCoderBase.compareAndVerify(TestErasureCoderBase.java:141)
at org.apache.hadoop.io.erasurecode.coder.TestErasureCoderBase.performTestCoding(TestErasureCoderBase.java:98)
at org.apache.hadoop.io.erasurecode.coder.TestErasureCoderBase.testCoding(TestErasureCoderBase.java:69)
at org.apache.hadoop.io.erasurecode.coder.TestHHXORErasureCoder.testCodingDirectBuffer_10x4_erasing_p1(TestHHXORErasureCoder.java:64)

2)TestRSErasureCoder

Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.591 sec - in org.apache.hadoop.io.erasurecode.coder.TestXORCoder
Running org.apache.hadoop.io.erasurecode.coder.TestRSErasureCoder
#

1. A fatal error has been detected by the Java Runtime Environment:
   #
2. SIGSEGV (0xb) at pc=0x00007f486a28a6e4, pid=8970, tid=0x00007f4850927700
   #
3. JRE version: OpenJDK Runtime Environment (8.0_121-b13) (build 1.8.0_121-8u121-b13-0ubuntu1.16.04.2-b13)
4. Java VM: OpenJDK 64-Bit Server VM (25.121-b13 mixed mode linux-amd64 compressed oops)
5. Problematic frame:
6. C [libc.so.6+0x8e6e4]
   #
7. Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
   #
8. An error report file with more information is saved as:
9. /home/ayappan/hadoop/hadoop-common-project/hadoop-common/hs_err_pid8970.log
   #
10. If you would like to submit a bug report, please visit:
11. http://bugreport.java.com/bugreport/crash.jsp
12. The crash happened outside the Java Virtual Machine in native code.
13. See problematic frame for where to report the bug.
    #

3)TestCodecRawCoderMapping

Running org.apache.hadoop.io.erasurecode.TestCodecRawCoderMapping
Tests run: 5, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.559 sec <<< FAILURE! - in org.apache.hadoop.io.erasurecode.TestCodecRawCoderMapping
testRSDefaultRawCoder(org.apache.hadoop.io.erasurecode.TestCodecRawCoderMapping) Time elapsed: 0.015 sec <<< FAILURE!
java.lang.AssertionError: null
at org.junit.Assert.fail(Assert.java:86)
at org.junit.Assert.assertTrue(Assert.java:41)
at org.junit.Assert.assertTrue(Assert.java:52)
at org.apache.hadoop.io.erasurecode.TestCodecRawCoderMapping.testRSDefaultRawCoder(TestCodecRawCoderMapping.java:58)

Issue Links

• breaks: HADOOP-14616 Client fails to read a block with erasure code (XOR, native) when one of the data block is lost
• is related to: HDFS-11066 Improve test coverage for ISA-L native coder