Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-12590

TestCompressorDecompressor failing without stack traces

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 2.8.0
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: test
    • Labels:
      None
    • Environment:

      jenkins

      Description

      Jenkins failing on TestCompressorDecompressor.

      The exception is being caught and converted to a fail so there is no stack trace of any value

      testCompressorDecompressor error !!!java.lang.NullPointerException
      Stacktrace
      
      java.lang.AssertionError: testCompressorDecompressor error !!!java.lang.NullPointerException
      	at org.junit.Assert.fail(Assert.java:88)
      	at org.apache.hadoop.io.compress.TestCompressorDecompressor.testCompressorDecompressor(TestCompressorDecompressor.java:69)
      

        Issue Links

          Activity

          Hide
          stevel@apache.org Steve Loughran added a comment -

          Same for the other test

          org.apache.hadoop.io.compress.TestCompressorDecompressor.testCompressorDecompressorWithExeedBufferLimit
          
          Failing for the past 6 builds (Since Failed#1186 )
          Took 60 ms.
          Error Message
          
          testCompressorDecompressorWithExeedBufferLimit error !!!java.lang.NullPointerException
          Stacktrace
          
          java.lang.AssertionError: testCompressorDecompressorWithExeedBufferLimit error !!!java.lang.NullPointerException
          	at org.junit.Assert.fail(Assert.java:88)
          	at org.apache.hadoop.io.compress.TestCompressorDecompressor.testCompressorDecompressorWithExeedBufferLimit(TestCompressorDecompressor.java:91)
          Standard Output
          
          2015-11-23 11:47:39,540 WARN  compress.CompressDecompressTester (CompressDecompressTester.java:isNativeSnappyLoadable(87)) - Snappy native library is available
          2015-11-23 11:47:39,540 INFO  compress.CompressDecompressTester (CompressDecompressTester.java:isNativeSnappyLoadable(92)) - Snappy native library loaded
          

          As well as the root cause, the test need to have their try/catch logic cut; JUnit will handle the exceptions being raised itself

          Show
          stevel@apache.org Steve Loughran added a comment - Same for the other test org.apache.hadoop.io.compress.TestCompressorDecompressor.testCompressorDecompressorWithExeedBufferLimit Failing for the past 6 builds (Since Failed#1186 ) Took 60 ms. Error Message testCompressorDecompressorWithExeedBufferLimit error !!!java.lang.NullPointerException Stacktrace java.lang.AssertionError: testCompressorDecompressorWithExeedBufferLimit error !!!java.lang.NullPointerException at org.junit.Assert.fail(Assert.java:88) at org.apache.hadoop.io.compress.TestCompressorDecompressor.testCompressorDecompressorWithExeedBufferLimit(TestCompressorDecompressor.java:91) Standard Output 2015-11-23 11:47:39,540 WARN compress.CompressDecompressTester (CompressDecompressTester.java:isNativeSnappyLoadable(87)) - Snappy native library is available 2015-11-23 11:47:39,540 INFO compress.CompressDecompressTester (CompressDecompressTester.java:isNativeSnappyLoadable(92)) - Snappy native library loaded As well as the root cause, the test need to have their try/catch logic cut; JUnit will handle the exceptions being raised itself
          Hide
          jojochuang Wei-Chiu Chuang added a comment -

          Hi Steve Loughran Do you also want to fix other tests as well? I am seeing other places where exceptions are swallowed with no stack track.

          Show
          jojochuang Wei-Chiu Chuang added a comment - Hi Steve Loughran Do you also want to fix other tests as well? I am seeing other places where exceptions are swallowed with no stack track.
          Hide
          stevel@apache.org Steve Loughran added a comment -

          hi re-opened the other JIRA. As I said there, it's big enough that its best dealt with test-by-test

          you may want to look at https://github.com/steveloughran/formality/blob/master/styleguide/styleguide.md to see my current thinking on what makes a good hadoop test

          Show
          stevel@apache.org Steve Loughran added a comment - hi re-opened the other JIRA. As I said there, it's big enough that its best dealt with test-by-test you may want to look at https://github.com/steveloughran/formality/blob/master/styleguide/styleguide.md to see my current thinking on what makes a good hadoop test
          Hide
          jojochuang Wei-Chiu Chuang added a comment -

          Thanks, that's a great article, and I am desperately looking for some thing like this.

          Show
          jojochuang Wei-Chiu Chuang added a comment - Thanks, that's a great article, and I am desperately looking for some thing like this.
          Hide
          stevel@apache.org Steve Loughran added a comment -

          thank you ... HADOOP-12143 proposes merging it in

          Show
          stevel@apache.org Steve Loughran added a comment - thank you ... HADOOP-12143 proposes merging it in
          Hide
          jzhuge John Zhuge added a comment -

          Steve Leland and [~ weichiu], let me know whether the fix is what you have in mind.

          Patch 001

          • Replace fail calls with GenericTestUtils.assertExceptionContains.
          Show
          jzhuge John Zhuge added a comment - Steve Leland and [~ weichiu] , let me know whether the fix is what you have in mind. Patch 001 Replace fail calls with GenericTestUtils.assertExceptionContains.
          Hide
          hadoopqa Hadoop QA added a comment -
          +1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 0s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 7m 27s trunk passed
          +1 compile 7m 43s trunk passed with JDK v1.8.0_66
          +1 compile 8m 37s trunk passed with JDK v1.7.0_91
          +1 checkstyle 0m 15s trunk passed
          +1 mvnsite 1m 0s trunk passed
          +1 mvneclipse 0m 13s trunk passed
          +1 findbugs 1m 45s trunk passed
          +1 javadoc 0m 51s trunk passed with JDK v1.8.0_66
          +1 javadoc 1m 1s trunk passed with JDK v1.7.0_91
          +1 mvninstall 1m 42s the patch passed
          +1 compile 7m 35s the patch passed with JDK v1.8.0_66
          +1 javac 7m 35s the patch passed
          +1 compile 8m 39s the patch passed with JDK v1.7.0_91
          +1 javac 8m 39s the patch passed
          +1 checkstyle 0m 15s the patch passed
          +1 mvnsite 1m 1s the patch passed
          +1 mvneclipse 0m 14s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 findbugs 1m 58s the patch passed
          +1 javadoc 0m 52s the patch passed with JDK v1.8.0_66
          +1 javadoc 1m 4s the patch passed with JDK v1.7.0_91
          +1 unit 6m 41s hadoop-common in the patch passed with JDK v1.8.0_66.
          +1 unit 6m 54s hadoop-common in the patch passed with JDK v1.7.0_91.
          +1 asflicense 0m 23s Patch does not generate ASF License warnings.
          67m 27s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:0ca8df7
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12781377/HADOOP-12590.001.patch
          JIRA Issue HADOOP-12590
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux dec2187d7e68 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / f0fa6d8
          Default Java 1.7.0_91
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91
          findbugs v3.0.0
          JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8374/testReport/
          modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
          Max memory used 76MB
          Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8374/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 7m 27s trunk passed +1 compile 7m 43s trunk passed with JDK v1.8.0_66 +1 compile 8m 37s trunk passed with JDK v1.7.0_91 +1 checkstyle 0m 15s trunk passed +1 mvnsite 1m 0s trunk passed +1 mvneclipse 0m 13s trunk passed +1 findbugs 1m 45s trunk passed +1 javadoc 0m 51s trunk passed with JDK v1.8.0_66 +1 javadoc 1m 1s trunk passed with JDK v1.7.0_91 +1 mvninstall 1m 42s the patch passed +1 compile 7m 35s the patch passed with JDK v1.8.0_66 +1 javac 7m 35s the patch passed +1 compile 8m 39s the patch passed with JDK v1.7.0_91 +1 javac 8m 39s the patch passed +1 checkstyle 0m 15s the patch passed +1 mvnsite 1m 1s the patch passed +1 mvneclipse 0m 14s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 1m 58s the patch passed +1 javadoc 0m 52s the patch passed with JDK v1.8.0_66 +1 javadoc 1m 4s the patch passed with JDK v1.7.0_91 +1 unit 6m 41s hadoop-common in the patch passed with JDK v1.8.0_66. +1 unit 6m 54s hadoop-common in the patch passed with JDK v1.7.0_91. +1 asflicense 0m 23s Patch does not generate ASF License warnings. 67m 27s Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12781377/HADOOP-12590.001.patch JIRA Issue HADOOP-12590 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux dec2187d7e68 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / f0fa6d8 Default Java 1.7.0_91 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91 findbugs v3.0.0 JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8374/testReport/ modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common Max memory used 76MB Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8374/console This message was automatically generated.
          Hide
          stevel@apache.org Steve Loughran added a comment -

          +1
          committed -thanks!

          Show
          stevel@apache.org Steve Loughran added a comment - +1 committed -thanks!
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #9077 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9077/)
          HADOOP-12590. TestCompressorDecompressor failing without stack traces (stevel: rev d7ed04758c1bdb1c7caf5cf3a03da3ad81701957)

          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/compress/TestCompressorDecompressor.java
          • hadoop-common-project/hadoop-common/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #9077 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9077/ ) HADOOP-12590 . TestCompressorDecompressor failing without stack traces (stevel: rev d7ed04758c1bdb1c7caf5cf3a03da3ad81701957) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/compress/TestCompressorDecompressor.java hadoop-common-project/hadoop-common/CHANGES.txt
          Hide
          jzhuge John Zhuge added a comment -

          Hi Steve Leland, TestCompressorDecompressor.java no longer needs to "import static org.junit.Assert.fail;". Should I fix it or leave it to whoever changes the file next?

          Show
          jzhuge John Zhuge added a comment - Hi Steve Leland , TestCompressorDecompressor.java no longer needs to "import static org.junit.Assert.fail;". Should I fix it or leave it to whoever changes the file next?
          Hide
          jzhuge John Zhuge added a comment -

          Hi Steve Loughran, TestCompressorDecompressor.java no longer needs to "import static org.junit.Assert.fail;". Should I fix it or leave it to whoever changes the file next?

          Show
          jzhuge John Zhuge added a comment - Hi Steve Loughran , TestCompressorDecompressor.java no longer needs to "import static org.junit.Assert.fail;". Should I fix it or leave it to whoever changes the file next?
          Hide
          stevel@apache.org Steve Loughran added a comment -

          oh, missed that. Lets' be tidy and clean up: file another JIRA for the one-liner

          Show
          stevel@apache.org Steve Loughran added a comment - oh, missed that. Lets' be tidy and clean up: file another JIRA for the one-liner
          Hide
          vinodkv Vinod Kumar Vavilapalli added a comment -

          Steve Loughran, there is a branch-2.8 where you need to land this patch for it to be in 2.8.0.

          Show
          vinodkv Vinod Kumar Vavilapalli added a comment - Steve Loughran , there is a branch-2.8 where you need to land this patch for it to be in 2.8.0.
          Hide
          jzhuge John Zhuge added a comment -

          Really appreciate the help.

          Please commit this patch to branch-2 and branch-2.8.
          Also please commit the followup HADOOP-12700 to branch-2 and branch-2.8 as well.

          And while at it, could someone review HADOOP-12701 so that problem like HADOOP-12700 would never happen again?

          Thanks,
          John

          Show
          jzhuge John Zhuge added a comment - Really appreciate the help. Please commit this patch to branch-2 and branch-2.8. Also please commit the followup HADOOP-12700 to branch-2 and branch-2.8 as well. And while at it, could someone review HADOOP-12701 so that problem like HADOOP-12700 would never happen again? Thanks, John
          Hide
          ajisakaa Akira Ajisaka added a comment -

          Committed this to branch-2.8 to fix the build failure after HADOOP-12700. Thanks Vinod and John.
          FYI: This patch has been already committed to branch-2.

          Show
          ajisakaa Akira Ajisaka added a comment - Committed this to branch-2.8 to fix the build failure after HADOOP-12700 . Thanks Vinod and John. FYI: This patch has been already committed to branch-2.
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #9117 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9117/)
          Move HADOOP-12590 from 2.9.0 to 2.8.0 in CHANGES.txt. (aajisaka: rev 1da762c745fa2bbb0a7a6d16bdc58ec9d4eb670d)

          • hadoop-common-project/hadoop-common/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #9117 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9117/ ) Move HADOOP-12590 from 2.9.0 to 2.8.0 in CHANGES.txt. (aajisaka: rev 1da762c745fa2bbb0a7a6d16bdc58ec9d4eb670d) hadoop-common-project/hadoop-common/CHANGES.txt
          Hide
          jzhuge John Zhuge added a comment -
          Show
          jzhuge John Zhuge added a comment - Thanks Akira Ajisaka

            People

            • Assignee:
              jzhuge John Zhuge
              Reporter:
              stevel@apache.org Steve Loughran
            • Votes:
              0 Vote for this issue
              Watchers:
              11 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development