Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-4168

TestDFSUpgradeFromImage fails in branch-1

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.2.0
    • Component/s: namenode
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      java.lang.NullPointerException
      	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.removeBlocks(FSNamesystem.java:2212)
      	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.removePathAndBlocks(FSNamesystem.java:2225)
      	at org.apache.hadoop.hdfs.server.namenode.FSDirectory.unprotectedDelete(FSDirectory.java:645)
      	at org.apache.hadoop.hdfs.server.namenode.FSEditLog.loadFSEdits(FSEditLog.java:833)
      	at org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSEdits(FSImage.java:1024)
      	at org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:841)
      	at org.apache.hadoop.hdfs.server.namenode.FSImage.doUpgrade(FSImage.java:402)
      	at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:367)
      	at org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:104)
      	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:420)
      	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:388)
      	at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:285)
      	at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:546)
      	at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1444)
      	at org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:278)
      	at org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:173)
      	at org.apache.hadoop.hdfs.TestDFSUpgradeFromImage.testUpgradeFromImage(TestDFSUpgradeFromImage.java:185)
      

        Activity

        Hide
        mattf Matt Foley added a comment -

        Closed upon release of Hadoop 1.2.0.

        Show
        mattf Matt Foley added a comment - Closed upon release of Hadoop 1.2.0.
        Hide
        szetszwo Tsz Wo Nicholas Sze added a comment -

        I have committed this. Thanks, Jing!

        Show
        szetszwo Tsz Wo Nicholas Sze added a comment - I have committed this. Thanks, Jing!
        Hide
        szetszwo Tsz Wo Nicholas Sze added a comment -

        +1 patch looks good. Thanks for fixing the bug promptly.

        Show
        szetszwo Tsz Wo Nicholas Sze added a comment - +1 patch looks good. Thanks for fixing the bug promptly.
        Hide
        jingzhao Jing Zhao added a comment -

        Patch uploaded. Have passed all the local unit tests.

        Show
        jingzhao Jing Zhao added a comment - Patch uploaded. Have passed all the local unit tests.
        Hide
        szetszwo Tsz Wo Nicholas Sze added a comment -

        All the following tests fail with the same NPE.

            [junit] Running org.apache.hadoop.hdfs.TestDFSUpgradeFromImage
            [junit] Tests run: 1, Failures: 0, Errors: 1, Time elapsed: 3.459 sec
        --
            [junit] Running org.apache.hadoop.hdfs.TestRestartDFS
            [junit] Tests run: 2, Failures: 0, Errors: 2, Time elapsed: 13.939 sec
        --
            [junit] Running org.apache.hadoop.hdfs.TestSafeMode
            [junit] Tests run: 3, Failures: 0, Errors: 2, Time elapsed: 9.124 sec
        --
            [junit] Running org.apache.hadoop.hdfs.server.namenode.TestCheckpoint
            [junit] Tests run: 3, Failures: 0, Errors: 3, Time elapsed: 6.36 sec
        --
            [junit] Running org.apache.hadoop.hdfs.server.namenode.TestStartup
            [junit] Tests run: 3, Failures: 2, Errors: 0, Time elapsed: 7.259 sec
        
        Show
        szetszwo Tsz Wo Nicholas Sze added a comment - All the following tests fail with the same NPE. [junit] Running org.apache.hadoop.hdfs.TestDFSUpgradeFromImage [junit] Tests run: 1, Failures: 0, Errors: 1, Time elapsed: 3.459 sec -- [junit] Running org.apache.hadoop.hdfs.TestRestartDFS [junit] Tests run: 2, Failures: 0, Errors: 2, Time elapsed: 13.939 sec -- [junit] Running org.apache.hadoop.hdfs.TestSafeMode [junit] Tests run: 3, Failures: 0, Errors: 2, Time elapsed: 9.124 sec -- [junit] Running org.apache.hadoop.hdfs.server.namenode.TestCheckpoint [junit] Tests run: 3, Failures: 0, Errors: 3, Time elapsed: 6.36 sec -- [junit] Running org.apache.hadoop.hdfs.server.namenode.TestStartup [junit] Tests run: 3, Failures: 2, Errors: 0, Time elapsed: 7.259 sec
        Hide
        jingzhao Jing Zhao added a comment -

        The NPE is because the pendingReplications has not been created before the loadFSImage call. We thus can check if pendingReplications is null before calling its remove method. Will run test first to verify.

        Show
        jingzhao Jing Zhao added a comment - The NPE is because the pendingReplications has not been created before the loadFSImage call. We thus can check if pendingReplications is null before calling its remove method. Will run test first to verify.

          People

          • Assignee:
            jingzhao Jing Zhao
            Reporter:
            szetszwo Tsz Wo Nicholas Sze
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development