Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-4168

TestDFSUpgradeFromImage fails in branch-1

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.2.0
    • Component/s: namenode
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      java.lang.NullPointerException
      	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.removeBlocks(FSNamesystem.java:2212)
      	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.removePathAndBlocks(FSNamesystem.java:2225)
      	at org.apache.hadoop.hdfs.server.namenode.FSDirectory.unprotectedDelete(FSDirectory.java:645)
      	at org.apache.hadoop.hdfs.server.namenode.FSEditLog.loadFSEdits(FSEditLog.java:833)
      	at org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSEdits(FSImage.java:1024)
      	at org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:841)
      	at org.apache.hadoop.hdfs.server.namenode.FSImage.doUpgrade(FSImage.java:402)
      	at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:367)
      	at org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:104)
      	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:420)
      	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:388)
      	at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:285)
      	at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:546)
      	at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1444)
      	at org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:278)
      	at org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:173)
      	at org.apache.hadoop.hdfs.TestDFSUpgradeFromImage.testUpgradeFromImage(TestDFSUpgradeFromImage.java:185)
      

        Activity

        Hide
        Matt Foley added a comment -

        Closed upon release of Hadoop 1.2.0.

        Show
        Matt Foley added a comment - Closed upon release of Hadoop 1.2.0.
        Hide
        Tsz Wo Nicholas Sze added a comment -

        I have committed this. Thanks, Jing!

        Show
        Tsz Wo Nicholas Sze added a comment - I have committed this. Thanks, Jing!
        Hide
        Tsz Wo Nicholas Sze added a comment -

        +1 patch looks good. Thanks for fixing the bug promptly.

        Show
        Tsz Wo Nicholas Sze added a comment - +1 patch looks good. Thanks for fixing the bug promptly.
        Hide
        Jing Zhao added a comment -

        Patch uploaded. Have passed all the local unit tests.

        Show
        Jing Zhao added a comment - Patch uploaded. Have passed all the local unit tests.
        Hide
        Tsz Wo Nicholas Sze added a comment -

        All the following tests fail with the same NPE.

            [junit] Running org.apache.hadoop.hdfs.TestDFSUpgradeFromImage
            [junit] Tests run: 1, Failures: 0, Errors: 1, Time elapsed: 3.459 sec
        --
            [junit] Running org.apache.hadoop.hdfs.TestRestartDFS
            [junit] Tests run: 2, Failures: 0, Errors: 2, Time elapsed: 13.939 sec
        --
            [junit] Running org.apache.hadoop.hdfs.TestSafeMode
            [junit] Tests run: 3, Failures: 0, Errors: 2, Time elapsed: 9.124 sec
        --
            [junit] Running org.apache.hadoop.hdfs.server.namenode.TestCheckpoint
            [junit] Tests run: 3, Failures: 0, Errors: 3, Time elapsed: 6.36 sec
        --
            [junit] Running org.apache.hadoop.hdfs.server.namenode.TestStartup
            [junit] Tests run: 3, Failures: 2, Errors: 0, Time elapsed: 7.259 sec
        
        Show
        Tsz Wo Nicholas Sze added a comment - All the following tests fail with the same NPE. [junit] Running org.apache.hadoop.hdfs.TestDFSUpgradeFromImage [junit] Tests run: 1, Failures: 0, Errors: 1, Time elapsed: 3.459 sec -- [junit] Running org.apache.hadoop.hdfs.TestRestartDFS [junit] Tests run: 2, Failures: 0, Errors: 2, Time elapsed: 13.939 sec -- [junit] Running org.apache.hadoop.hdfs.TestSafeMode [junit] Tests run: 3, Failures: 0, Errors: 2, Time elapsed: 9.124 sec -- [junit] Running org.apache.hadoop.hdfs.server.namenode.TestCheckpoint [junit] Tests run: 3, Failures: 0, Errors: 3, Time elapsed: 6.36 sec -- [junit] Running org.apache.hadoop.hdfs.server.namenode.TestStartup [junit] Tests run: 3, Failures: 2, Errors: 0, Time elapsed: 7.259 sec
        Hide
        Jing Zhao added a comment -

        The NPE is because the pendingReplications has not been created before the loadFSImage call. We thus can check if pendingReplications is null before calling its remove method. Will run test first to verify.

        Show
        Jing Zhao added a comment - The NPE is because the pendingReplications has not been created before the loadFSImage call. We thus can check if pendingReplications is null before calling its remove method. Will run test first to verify.

          People

          • Assignee:
            Jing Zhao
            Reporter:
            Tsz Wo Nicholas Sze
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development