[HADOOP-12610] Tests lose stack traces on failures

• Type: Improvement
• Status: Reopened
• Priority: Minor
• Resolution: Unresolved
• Component/s: test
• Labels: supportability
• Assignee: Wei-Chiu Chuang
• Reporter: Wei-Chiu Chuang
• Created: 01/Dec/15 14:11
• Updated: 02/Oct/19 17:12

Description

I saw a test failure in TestCompressorDecompressor.testCompressorDecompressor

with the output
Error Message

testCompressorDecompressor error !!!java.lang.NullPointerException

Stacktrace

java.lang.AssertionError: testCompressorDecompressor error !!!java.lang.NullPointerException
	at org.junit.Assert.fail(Assert.java:88)
	at org.apache.hadoop.io.compress.TestCompressorDecompressor.testCompressorDecompressor(TestCompressorDecompressor.java:69)

However, this message does not expose the origin of the exception, nor does it print the error message. This makes debugging impossible. I also found the similar practice exists in multiple test files:

~/hadoop/hadoop-common-project/hadoop-common ((c21a801...)) $ grep -r "fail(.*+ e" .|less

./src/test/java/org/apache/hadoop/net/TestNetworkTopologyWithNodeGroup.java:        fail("Expecting IllegalArgumentException, but caught:" + e);
./src/test/java/org/apache/hadoop/io/nativeio/TestNativeIO.java:      fail("Got unexpected exception: " + e.getMessage());
./src/test/java/org/apache/hadoop/io/nativeio/TestNativeIO.java:        fail("Got unexpected exception: " + e.getMessage());
./src/test/java/org/apache/hadoop/io/nativeio/TestNativeIO.java:      fail("Got unexpected exception: " + e.getMessage());
./src/test/java/org/apache/hadoop/io/TestMapFile.java:      fail("testRename error " + ex);
./src/test/java/org/apache/hadoop/io/compress/snappy/TestSnappyCompressorDecompressor.java:      fail("testSnappyDirectBlockCompression ex !!!" + ex);
./src/test/java/org/apache/hadoop/io/compress/zlib/TestZlibCompressorDecompressor.java:      fail("testCompressorDecompressor error !!!" + ex);
./src/test/java/org/apache/hadoop/io/compress/zlib/TestZlibCompressorDecompressor.java:      fail("testCompressorDecompressorWithExeedBufferLimit error !!!" + ex);
./src/test/java/org/apache/hadoop/io/compress/zlib/TestZlibCompressorDecompressor.java:        fail("testZlibCompressorDecompressorWithConfiguration ex error " + ex);
./src/test/java/org/apache/hadoop/io/compress/zlib/TestZlibCompressorDecompressor.java:      fail("testZlibCompressDecompress ex !!!" + ex);
./src/test/java/org/apache/hadoop/io/compress/zlib/TestZlibCompressorDecompressor.java:      fail("testZlibDirectCompressDecompress ex !!!" + ex);
./src/test/java/org/apache/hadoop/io/compress/zlib/TestZlibCompressorDecompressor.java:      fail("testBuiltInGzipDecompressorExceptions npe error " + ex);
./src/test/java/org/apache/hadoop/io/compress/zlib/TestZlibCompressorDecompressor.java:      fail("testBuiltInGzipDecompressorExceptions aioob error" + ex);
./src/test/java/org/apache/hadoop/io/compress/zlib/TestZlibCompressorDecompressor.java:      fail("invalid 0 and 1 byte in gzip stream" + ex);
./src/test/java/org/apache/hadoop/io/compress/zlib/TestZlibCompressorDecompressor.java:      fail("invalid 2 byte in gzip stream" + ex);
./src/test/java/org/apache/hadoop/io/compress/zlib/TestZlibCompressorDecompressor.java:      fail("invalid 3 byte in gzip stream" + ex);
./src/test/java/org/apache/hadoop/io/compress/zlib/TestZlibCompressorDecompressor.java:      fail("invalid 3 byte make hasExtraField" + ex);
./src/test/java/org/apache/hadoop/io/compress/TestCompressorDecompressor.java:      fail("testCompressorDecompressor error !!!" + ex);
./src/test/java/org/apache/hadoop/io/compress/TestCompressorDecompressor.java:      fail("testCompressorDecompressorWithExeedBufferLimit error !!!" + ex);
./src/test/java/org/apache/hadoop/io/compress/TestBlockDecompressorStream.java:      fail("unexpected IOException : " + e);
./src/test/java/org/apache/hadoop/io/compress/bzip2/TestBzip2CompressorDecompressor.java:      fail("testBzip2CompressDecompress ex !!!" + ex);
./src/test/java/org/apache/hadoop/util/bloom/BloomFilterCommonTester.java:          Assert.fail("" + e);
./src/test/java/org/apache/hadoop/util/bloom/BloomFilterCommonTester.java:          Assert.fail("" + e);
./src/test/java/org/apache/hadoop/util/bloom/BloomFilterCommonTester.java:          Assert.fail("" + e);
./src/test/java/org/apache/hadoop/util/bloom/BloomFilterCommonTester.java:          Assert.fail("" + e);
./src/test/java/org/apache/hadoop/util/bloom/BloomFilterCommonTester.java:          Assert.fail("" + e);
./src/test/java/org/apache/hadoop/util/bloom/BloomFilterCommonTester.java:          Assert.fail("" + e);
./src/test/java/org/apache/hadoop/util/bloom/BloomFilterCommonTester.java:          Assert.fail("" + e);
./src/test/java/org/apache/hadoop/util/bloom/BloomFilterCommonTester.java:          Assert.fail("" + e);
./src/test/java/org/apache/hadoop/util/bloom/BloomFilterCommonTester.java:          Assert.fail("error ex !!!" + ex);
./src/test/java/org/apache/hadoop/util/TestShell.java:      fail("Expected an exception with the text `" + expectedText + "`"
./src/test/java/org/apache/hadoop/util/TestShell.java:      fail("Expected an exception with the text `" + expectedText + "`"
./src/test/java/org/apache/hadoop/tracing/SetSpanReceiver.java:      Assert.fail("timed out to get expected spans: " + e.getMessage());
./src/test/java/org/apache/hadoop/tracing/SetSpanReceiver.java:      Assert.fail("interrupted while waiting spans: " + e.getMessage());
./src/test/java/org/apache/hadoop/ipc/TestRPCCompatibility.java:      fail("Expected version mismatch but got " + ex);
./src/test/java/org/apache/hadoop/ipc/TestRPC.java:        fail("Expecting HadoopIllegalArgumentException but caught " + e);
./src/test/java/org/apache/hadoop/ipc/TestRPC.java:        fail("Expecting HadoopIllegalArgumentException but caught " + e);
./src/test/java/org/apache/hadoop/ipc/TestRPC.java:        fail("Expecting HadoopIllegalArgumentException but caught " + e);
./src/test/java/org/apache/hadoop/security/token/delegation/TestDelegationToken.java:      Assert.fail("action did not throw " + except);

I am filing this JIRA to make failure messages more verbose and make debugging easier. There are of course similar patterns in other part of Hadoop codebase, but I am starting with this one because I do see test failures here.

Issue Links

• depends upon: HADOOP-12590 TestCompressorDecompressor failing without stack traces
• is related to: HDFS-9514 TestDistributedFileSystem.testDFSClientPeerWriteTimeout failing; exception being swallowed