Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-6419

TestBookKeeperHACheckpoints#TestSBNCheckpoints fails on trunk

    Details

    • Type: Test Test
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 2.5.0
    • Fix Version/s: 2.5.0
    • Component/s: None
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      TestBookKeerHACheckpoints#TestSBNCheckpoints fails on trunk.
      See https://builds.apache.org/job/PreCommit-HDFS-Build/6908//testReport/

      1. org.apache.hadoop.contrib.bkjournal.TestBookKeeperHACheckpoints.txt
        3 kB
        Akira AJISAKA
      2. HDFS-6419.patch
        1 kB
        Akira AJISAKA
      3. HDFS-6419.2.patch
        5 kB
        Akira AJISAKA

        Issue Links

          Activity

          Hide
          Akira AJISAKA added a comment -

          I reproduced the failure locally. Attaching the log.

          Show
          Akira AJISAKA added a comment - I reproduced the failure locally. Attaching the log.
          Hide
          Akira AJISAKA added a comment -

          The NPE occurs because tmpOivImgDir is not initialized. I attached a simple patch to initialize tmpOivImgDir and configure the directory to output a legacy fsimage.

          Show
          Akira AJISAKA added a comment - The NPE occurs because tmpOivImgDir is not initialized. I attached a simple patch to initialize tmpOivImgDir and configure the directory to output a legacy fsimage.
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12645283/HDFS-6419.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 1 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/6920//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/6920//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12645283/HDFS-6419.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 1 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/6920//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/6920//console This message is automatically generated.
          Hide
          Uma Maheswara Rao G added a comment -

          Hi Akira, Thanks a lot for filing it and posting the patch!
          Would you mind looking at the comments posted in HDFS-6427 by Yongjun Zhang

          Show
          Uma Maheswara Rao G added a comment - Hi Akira, Thanks a lot for filing it and posting the patch! Would you mind looking at the comments posted in HDFS-6427 by Yongjun Zhang
          Hide
          Akira AJISAKA added a comment -

          Hi Uma Maheswara Rao G, thanks for the comment! Updated the patch to reflect Yongjun Zhang's comment.

          Show
          Akira AJISAKA added a comment - Hi Uma Maheswara Rao G , thanks for the comment! Updated the patch to reflect Yongjun Zhang 's comment.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12645727/HDFS-6419.2.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 2 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal:

          org.apache.hadoop.hdfs.TestDistributedFileSystem
          org.apache.hadoop.hdfs.server.datanode.TestBPOfferService

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/6937//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/6937//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12645727/HDFS-6419.2.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 2 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal: org.apache.hadoop.hdfs.TestDistributedFileSystem org.apache.hadoop.hdfs.server.datanode.TestBPOfferService +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/6937//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/6937//console This message is automatically generated.
          Hide
          Akira AJISAKA added a comment -

          These test failures look unrelated to the patch. I cannot reproduce them.

          Show
          Akira AJISAKA added a comment - These test failures look unrelated to the patch. I cannot reproduce them.
          Hide
          Yongjun Zhang added a comment -

          Hi Akira AJISAKA,

          Thanks for finding and fixing the issue. The latest patch (.2 version) looks good to me. +1.

          BTW, The test failures appears to be not related to the change here, I'd suggest we file JIRAs for both issues (after this patch is committed), which might help us to to identify the source of the flakiness.

          Regression
          org.apache.hadoop.hdfs.TestDistributedFileSystem.testGetFileBlockStorageLocationsError
          Failing for the past 1 build (Since Failed#6937 )
          Took 2.9 sec.
          Error Message
          Expected one valid and one invalid volume
          Stacktrace
          java.lang.AssertionError: Expected one valid and one invalid volume
          	at org.junit.Assert.fail(Assert.java:88)
          	at org.junit.Assert.assertTrue(Assert.java:41)
          	at org.apache.hadoop.hdfs.TestDistributedFileSystem.testGetFileBlockStorageLocationsError(TestDistributedFileSystem.java:837)
          

          and

          Regression
          org.apache.hadoop.hdfs.server.datanode.TestBPOfferService.testBPInitErrorHandling
          Failing for the past 1 build (Since Failed#6937 )
          Took 0.2 sec.
          Error Message
          expected:<2> but was:<1>
          Stacktrace
          java.lang.AssertionError: expected:<2> but was:<1>
          	at org.junit.Assert.fail(Assert.java:88)
          	at org.junit.Assert.failNotEquals(Assert.java:743)
          	at org.junit.Assert.assertEquals(Assert.java:118)
          	at org.junit.Assert.assertEquals(Assert.java:555)
          	at org.junit.Assert.assertEquals(Assert.java:542)
          	at org.apache.hadoop.hdfs.server.datanode.TestBPOfferService.testBPInitErrorHandling(TestBPOfferService.java:334)
          
          Show
          Yongjun Zhang added a comment - Hi Akira AJISAKA , Thanks for finding and fixing the issue. The latest patch (.2 version) looks good to me. +1. BTW, The test failures appears to be not related to the change here, I'd suggest we file JIRAs for both issues (after this patch is committed), which might help us to to identify the source of the flakiness. Regression org.apache.hadoop.hdfs.TestDistributedFileSystem.testGetFileBlockStorageLocationsError Failing for the past 1 build (Since Failed#6937 ) Took 2.9 sec. Error Message Expected one valid and one invalid volume Stacktrace java.lang.AssertionError: Expected one valid and one invalid volume at org.junit.Assert.fail(Assert.java:88) at org.junit.Assert.assertTrue(Assert.java:41) at org.apache.hadoop.hdfs.TestDistributedFileSystem.testGetFileBlockStorageLocationsError(TestDistributedFileSystem.java:837) and Regression org.apache.hadoop.hdfs.server.datanode.TestBPOfferService.testBPInitErrorHandling Failing for the past 1 build (Since Failed#6937 ) Took 0.2 sec. Error Message expected:<2> but was:<1> Stacktrace java.lang.AssertionError: expected:<2> but was:<1> at org.junit.Assert.fail(Assert.java:88) at org.junit.Assert.failNotEquals(Assert.java:743) at org.junit.Assert.assertEquals(Assert.java:118) at org.junit.Assert.assertEquals(Assert.java:555) at org.junit.Assert.assertEquals(Assert.java:542) at org.apache.hadoop.hdfs.server.datanode.TestBPOfferService.testBPInitErrorHandling(TestBPOfferService.java:334)
          Hide
          Kihwal Lee added a comment -

          +1

          Show
          Kihwal Lee added a comment - +1
          Hide
          Kihwal Lee added a comment -

          Committed to trunk and branch-2. Thanks for fixing this, Akira.

          Show
          Kihwal Lee added a comment - Committed to trunk and branch-2. Thanks for fixing this, Akira.
          Hide
          Hudson added a comment -

          SUCCESS: Integrated in Hadoop-trunk-Commit #5607 (See https://builds.apache.org/job/Hadoop-trunk-Commit/5607/)
          HDFS-6419. TestBookKeeperHACheckpoints.TestSBNCheckpoints fails on trunk. Contributed by Akira AJISAKA. (kihwal: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1596344)

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java/org/apache/hadoop/contrib/bkjournal/TestBookKeeperHACheckpoints.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestStandbyCheckpoints.java
          Show
          Hudson added a comment - SUCCESS: Integrated in Hadoop-trunk-Commit #5607 (See https://builds.apache.org/job/Hadoop-trunk-Commit/5607/ ) HDFS-6419 . TestBookKeeperHACheckpoints.TestSBNCheckpoints fails on trunk. Contributed by Akira AJISAKA. (kihwal: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1596344 ) /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java/org/apache/hadoop/contrib/bkjournal/TestBookKeeperHACheckpoints.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestStandbyCheckpoints.java
          Hide
          Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk #563 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/563/)
          HDFS-6419. TestBookKeeperHACheckpoints.TestSBNCheckpoints fails on trunk. Contributed by Akira AJISAKA. (kihwal: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1596344)

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java/org/apache/hadoop/contrib/bkjournal/TestBookKeeperHACheckpoints.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestStandbyCheckpoints.java
          Show
          Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk #563 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/563/ ) HDFS-6419 . TestBookKeeperHACheckpoints.TestSBNCheckpoints fails on trunk. Contributed by Akira AJISAKA. (kihwal: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1596344 ) /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java/org/apache/hadoop/contrib/bkjournal/TestBookKeeperHACheckpoints.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestStandbyCheckpoints.java
          Hide
          Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk #1781 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1781/)
          HDFS-6419. TestBookKeeperHACheckpoints.TestSBNCheckpoints fails on trunk. Contributed by Akira AJISAKA. (kihwal: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1596344)

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java/org/apache/hadoop/contrib/bkjournal/TestBookKeeperHACheckpoints.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestStandbyCheckpoints.java
          Show
          Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #1781 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1781/ ) HDFS-6419 . TestBookKeeperHACheckpoints.TestSBNCheckpoints fails on trunk. Contributed by Akira AJISAKA. (kihwal: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1596344 ) /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java/org/apache/hadoop/contrib/bkjournal/TestBookKeeperHACheckpoints.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestStandbyCheckpoints.java
          Hide
          Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk #1755 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1755/)
          HDFS-6419. TestBookKeeperHACheckpoints.TestSBNCheckpoints fails on trunk. Contributed by Akira AJISAKA. (kihwal: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1596344)

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java/org/apache/hadoop/contrib/bkjournal/TestBookKeeperHACheckpoints.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestStandbyCheckpoints.java
          Show
          Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #1755 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1755/ ) HDFS-6419 . TestBookKeeperHACheckpoints.TestSBNCheckpoints fails on trunk. Contributed by Akira AJISAKA. (kihwal: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1596344 ) /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java/org/apache/hadoop/contrib/bkjournal/TestBookKeeperHACheckpoints.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestStandbyCheckpoints.java

            People

            • Assignee:
              Akira AJISAKA
              Reporter:
              Akira AJISAKA
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development