Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-7788

Post-2.6 namenode may not start up with an image containing inodes created with an old release.

    Details

    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      Before HDFS-4305, which was fixed in 2.1.0-beta, clients could specify arbitrarily small preferred block size for a file including 0. This was normally done by faulty clients or failed creates, but it was possible.

      Until 2.5, reading a fsimage containing inodes with 0 byte preferred block size was allowed. So if a fsimage contained such an inode, the namenode would come up fine. In 2.6, the preferred block size is required be > 0. Because of this change, the image that worked with 2.5 may not work with 2.6.

      If a cluster ran a version of hadoop earlier than 2.1.0-beta before, it is under this risk even if it worked fine with 2.5.

      1. HDFS-7788-2.6.0.patch
        6 kB
        Sangjin Lee
      2. HDFS-7788-binary.patch
        7 kB
        Rushabh S Shah
      3. image-with-zero-block-size.tar.gz
        1 kB
        Sangjin Lee
      4. rushabh.patch
        7 kB
        Kihwal Lee

        Activity

        Hide
        kihwal Kihwal Lee added a comment -

        The stack trace:

        2015-02-09 19:02:28,361 [main] FATAL namenode.NameNode: Failed to start
        namenode.java.lang.IllegalArgumentException: Illagal value: PREFERRED_BLOCK_SIZE = 0 <
        MIN = 1
        at org.apache.hadoop.hdfs.util.LongBitFormat.combine(LongBitFormat.java:58)
        at org.apache.hadoop.hdfs.server.namenode.INodeFile$HeaderFormat.toLong(INodeFile.java:106)
        at org.apache.hadoop.hdfs.server.namenode.INodeFile.<init>(INodeFile.java:128)
        at org.apache.hadoop.hdfs.server.namenode.FSImageFormatPBINode$Loader.loadINodeFile(FSImageFormatPBINode.java:291)
        at org.apache.hadoop.hdfs.server.namenode.FSImageFormatPBINode$Loader.loadINode(FSImageFormatPBINode.java:266)
        at org.apache.hadoop.hdfs.server.namenode.FSImageFormatPBINode$Loader.loadINodeSection(FSImageFormatPBINode.java:221)
        at org.apache.hadoop.hdfs.server.namenode.FSImageFormatProtobuf$Loader.loadInternal(FSImageFormatProtobuf.java:254)
        at org.apache.hadoop.hdfs.server.namenode.FSImageFormatProtobuf$Loader.load(FSImageFormatProtobuf.java:180)
        at org.apache.hadoop.hdfs.server.namenode.FSImageFormat$LoaderDelegator.load(FSImageFormat.java:226)
        at org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:926)
        at org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:910)
        at org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImageFile(FSImage.java:729)
        at org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:665)
        at org.apache.hadoop.hdfs.server.namenode.FSImage.doUpgrade(FSImage.java:374)
        at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:268)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:1053)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:757)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:538)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:597)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:764)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:748)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1441)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1507)
        2015-02-09 19:02:28,381 [main] INFO util.ExitUtil: Exiting with status 1
        2015-02-09 19:02:28,384 [Thread-1] INFO namenode.NameNode: SHUTDOWN_MSG:
        /************************************************************
        SHUTDOWN_MSG: Shutting down NameNode at xxxx
        ************************************************************/

        Show
        kihwal Kihwal Lee added a comment - The stack trace: 2015-02-09 19:02:28,361 [main] FATAL namenode.NameNode: Failed to start namenode.java.lang.IllegalArgumentException: Illagal value: PREFERRED_BLOCK_SIZE = 0 < MIN = 1 at org.apache.hadoop.hdfs.util.LongBitFormat.combine(LongBitFormat.java:58) at org.apache.hadoop.hdfs.server.namenode.INodeFile$HeaderFormat.toLong(INodeFile.java:106) at org.apache.hadoop.hdfs.server.namenode.INodeFile.<init>(INodeFile.java:128) at org.apache.hadoop.hdfs.server.namenode.FSImageFormatPBINode$Loader.loadINodeFile(FSImageFormatPBINode.java:291) at org.apache.hadoop.hdfs.server.namenode.FSImageFormatPBINode$Loader.loadINode(FSImageFormatPBINode.java:266) at org.apache.hadoop.hdfs.server.namenode.FSImageFormatPBINode$Loader.loadINodeSection(FSImageFormatPBINode.java:221) at org.apache.hadoop.hdfs.server.namenode.FSImageFormatProtobuf$Loader.loadInternal(FSImageFormatProtobuf.java:254) at org.apache.hadoop.hdfs.server.namenode.FSImageFormatProtobuf$Loader.load(FSImageFormatProtobuf.java:180) at org.apache.hadoop.hdfs.server.namenode.FSImageFormat$LoaderDelegator.load(FSImageFormat.java:226) at org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:926) at org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:910) at org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImageFile(FSImage.java:729) at org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:665) at org.apache.hadoop.hdfs.server.namenode.FSImage.doUpgrade(FSImage.java:374) at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:268) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:1053) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:757) at org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:538) at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:597) at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:764) at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:748) at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1441) at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1507) 2015-02-09 19:02:28,381 [main] INFO util.ExitUtil: Exiting with status 1 2015-02-09 19:02:28,384 [Thread-1] INFO namenode.NameNode: SHUTDOWN_MSG: /************************************************************ SHUTDOWN_MSG: Shutting down NameNode at xxxx ************************************************************/
        Hide
        kihwal Kihwal Lee added a comment -

        One option is to make namenode automatically convert such inodes to have the min block size as their preferred block size.

        Show
        kihwal Kihwal Lee added a comment - One option is to make namenode automatically convert such inodes to have the min block size as their preferred block size.
        Hide
        shahrs87 Rushabh S Shah added a comment -

        I tweaked the source code to let me create a file having zero blockSize.
        Then I saved that image and added that image to the resources folder.
        While reading the image, if the namenode encounters any inode with preferredBlockSize == 0 (which was possible pre 2.1.0.beta as Kihwal mentioned in the jira) , it will change to LongBitFormat.MIN so that namenode can read that image without any error.

        I have added a tarball which contains the image.
        But whenever I tried to apply the patch on my machine, it always copy into /hadoop-hdfs/src/test/resources/
        instead of hadoop-hdfs-project/hadoop-hdfs/src/test/resources/

        Show
        shahrs87 Rushabh S Shah added a comment - I tweaked the source code to let me create a file having zero blockSize. Then I saved that image and added that image to the resources folder. While reading the image, if the namenode encounters any inode with preferredBlockSize == 0 (which was possible pre 2.1.0.beta as Kihwal mentioned in the jira) , it will change to LongBitFormat.MIN so that namenode can read that image without any error. I have added a tarball which contains the image. But whenever I tried to apply the patch on my machine, it always copy into /hadoop-hdfs/src/test/resources/ instead of hadoop-hdfs-project/hadoop-hdfs/src/test/resources/
        Hide
        hadoopqa Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12699564/HDFS-7788-binary.patch
        against trunk revision 9a3e292.

        -1 patch. The patch command could not apply the patch.

        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/9614//console

        This message is automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12699564/HDFS-7788-binary.patch against trunk revision 9a3e292. -1 patch . The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/9614//console This message is automatically generated.
        Hide
        kihwal Kihwal Lee added a comment -

        I turned Rushabh's patch into "binary" patch using the regular diff. Let's see if the precommit can handle this.

        Show
        kihwal Kihwal Lee added a comment - I turned Rushabh's patch into "binary" patch using the regular diff. Let's see if the precommit can handle this.
        Hide
        kihwal Kihwal Lee added a comment -

        The patch was applied fine by the precommit.
        https://builds.apache.org/job/PreCommit-HDFS-Build/9621

        Show
        kihwal Kihwal Lee added a comment - The patch was applied fine by the precommit. https://builds.apache.org/job/PreCommit-HDFS-Build/9621
        Hide
        hadoopqa Hadoop QA added a comment -

        +1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12699735/rushabh.patch
        against trunk revision d49ae72.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 2 new or modified test files.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. There were no new javadoc warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in .

        Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/9621//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/9621//console

        This message is automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12699735/rushabh.patch against trunk revision d49ae72. +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 2 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in . Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/9621//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/9621//console This message is automatically generated.
        Hide
        kihwal Kihwal Lee added a comment -

        Although the precommit posted a +1, the test result is missing. Judging from the run-time, it ran many tests. So I ran hdfs tests locally overnight. Two test cases failed.

        TestDFSHAAdminMiniCluster.testFencer:163 expected:<0> but was:<-1>
        TestDFSUpgradeFromImage.testUpgradeFromRel1BBWImage:619->upgradeAndVerify:597->verifyFileSystem:225->verifyDir:210->dfsOpenFileWithRetries:174 » IO

        I just reran the tests and they are passing.

        +1 The change looks good.

        Show
        kihwal Kihwal Lee added a comment - Although the precommit posted a +1, the test result is missing. Judging from the run-time, it ran many tests. So I ran hdfs tests locally overnight. Two test cases failed. TestDFSHAAdminMiniCluster.testFencer:163 expected:<0> but was:<-1> TestDFSUpgradeFromImage.testUpgradeFromRel1BBWImage:619->upgradeAndVerify:597->verifyFileSystem:225->verifyDir:210->dfsOpenFileWithRetries:174 » IO I just reran the tests and they are passing. +1 The change looks good.
        Hide
        kihwal Kihwal Lee added a comment -

        After a git pull, TestDFSHAAdminMiniCluster#testFencer failing consistently. It looks like HDFS-7813, unrelated this change. I will review HDFS-7813.

        Show
        kihwal Kihwal Lee added a comment - After a git pull, TestDFSHAAdminMiniCluster#testFencer failing consistently. It looks like HDFS-7813 , unrelated this change. I will review HDFS-7813 .
        Hide
        kihwal Kihwal Lee added a comment -

        I've committed this to trunk and branch-2. Thanks for working on the fix, Rushabh S Shah.

        Show
        kihwal Kihwal Lee added a comment - I've committed this to trunk and branch-2. Thanks for working on the fix, Rushabh S Shah .
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-trunk-Commit #7161 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7161/)
        HDFS-7788. Post-2.6 namenode may not start up with an image containing inodes created with an old release. Contributed by Rushabh Shah. (kihwal: rev 7ae5255a1613ccfb43646f33eabacf1062c86e93)

        • hadoop-hdfs-project/hadoop-hdfs/src/test/resources/image-with-zero-block-size.tar.gz
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INodeFile.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFSImage.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/util/LongBitFormat.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #7161 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7161/ ) HDFS-7788 . Post-2.6 namenode may not start up with an image containing inodes created with an old release. Contributed by Rushabh Shah. (kihwal: rev 7ae5255a1613ccfb43646f33eabacf1062c86e93) hadoop-hdfs-project/hadoop-hdfs/src/test/resources/image-with-zero-block-size.tar.gz hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INodeFile.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFSImage.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/util/LongBitFormat.java
        Hide
        shahrs87 Rushabh S Shah added a comment -

        Thanks Kihwal for reviewing and committting !!

        Show
        shahrs87 Rushabh S Shah added a comment - Thanks Kihwal for reviewing and committting !!
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #111 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/111/)
        HDFS-7788. Post-2.6 namenode may not start up with an image containing inodes created with an old release. Contributed by Rushabh Shah. (kihwal: rev 7ae5255a1613ccfb43646f33eabacf1062c86e93)

        • hadoop-hdfs-project/hadoop-hdfs/src/test/resources/image-with-zero-block-size.tar.gz
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INodeFile.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/util/LongBitFormat.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFSImage.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #111 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/111/ ) HDFS-7788 . Post-2.6 namenode may not start up with an image containing inodes created with an old release. Contributed by Rushabh Shah. (kihwal: rev 7ae5255a1613ccfb43646f33eabacf1062c86e93) hadoop-hdfs-project/hadoop-hdfs/src/test/resources/image-with-zero-block-size.tar.gz hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INodeFile.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/util/LongBitFormat.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFSImage.java
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Hadoop-Yarn-trunk #845 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/845/)
        HDFS-7788. Post-2.6 namenode may not start up with an image containing inodes created with an old release. Contributed by Rushabh Shah. (kihwal: rev 7ae5255a1613ccfb43646f33eabacf1062c86e93)

        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INodeFile.java
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/util/LongBitFormat.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFSImage.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/test/resources/image-with-zero-block-size.tar.gz
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Yarn-trunk #845 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/845/ ) HDFS-7788 . Post-2.6 namenode may not start up with an image containing inodes created with an old release. Contributed by Rushabh Shah. (kihwal: rev 7ae5255a1613ccfb43646f33eabacf1062c86e93) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INodeFile.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/util/LongBitFormat.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFSImage.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/test/resources/image-with-zero-block-size.tar.gz
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Hdfs-trunk #2043 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2043/)
        HDFS-7788. Post-2.6 namenode may not start up with an image containing inodes created with an old release. Contributed by Rushabh Shah. (kihwal: rev 7ae5255a1613ccfb43646f33eabacf1062c86e93)

        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INodeFile.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/util/LongBitFormat.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFSImage.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/resources/image-with-zero-block-size.tar.gz
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2043 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2043/ ) HDFS-7788 . Post-2.6 namenode may not start up with an image containing inodes created with an old release. Contributed by Rushabh Shah. (kihwal: rev 7ae5255a1613ccfb43646f33eabacf1062c86e93) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INodeFile.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/util/LongBitFormat.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFSImage.java hadoop-hdfs-project/hadoop-hdfs/src/test/resources/image-with-zero-block-size.tar.gz
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #102 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/102/)
        HDFS-7788. Post-2.6 namenode may not start up with an image containing inodes created with an old release. Contributed by Rushabh Shah. (kihwal: rev 7ae5255a1613ccfb43646f33eabacf1062c86e93)

        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/util/LongBitFormat.java
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INodeFile.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFSImage.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/resources/image-with-zero-block-size.tar.gz
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #102 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/102/ ) HDFS-7788 . Post-2.6 namenode may not start up with an image containing inodes created with an old release. Contributed by Rushabh Shah. (kihwal: rev 7ae5255a1613ccfb43646f33eabacf1062c86e93) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/util/LongBitFormat.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INodeFile.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFSImage.java hadoop-hdfs-project/hadoop-hdfs/src/test/resources/image-with-zero-block-size.tar.gz
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #112 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/112/)
        HDFS-7788. Post-2.6 namenode may not start up with an image containing inodes created with an old release. Contributed by Rushabh Shah. (kihwal: rev 7ae5255a1613ccfb43646f33eabacf1062c86e93)

        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/test/resources/image-with-zero-block-size.tar.gz
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/util/LongBitFormat.java
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INodeFile.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFSImage.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #112 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/112/ ) HDFS-7788 . Post-2.6 namenode may not start up with an image containing inodes created with an old release. Contributed by Rushabh Shah. (kihwal: rev 7ae5255a1613ccfb43646f33eabacf1062c86e93) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/test/resources/image-with-zero-block-size.tar.gz hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/util/LongBitFormat.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INodeFile.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFSImage.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Mapreduce-trunk #2062 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2062/)
        HDFS-7788. Post-2.6 namenode may not start up with an image containing inodes created with an old release. Contributed by Rushabh Shah. (kihwal: rev 7ae5255a1613ccfb43646f33eabacf1062c86e93)

        • hadoop-hdfs-project/hadoop-hdfs/src/test/resources/image-with-zero-block-size.tar.gz
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFSImage.java
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INodeFile.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/util/LongBitFormat.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #2062 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2062/ ) HDFS-7788 . Post-2.6 namenode may not start up with an image containing inodes created with an old release. Contributed by Rushabh Shah. (kihwal: rev 7ae5255a1613ccfb43646f33eabacf1062c86e93) hadoop-hdfs-project/hadoop-hdfs/src/test/resources/image-with-zero-block-size.tar.gz hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFSImage.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INodeFile.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/util/LongBitFormat.java
        Hide
        sjlee0 Sangjin Lee added a comment -

        The backport to 2.6.0 is pretty trivial but the test FS image needs to be recreated for 2.6.0. Also it'd be good to rename the variable (HADOOP_2_7_ZER0_BLOCK_SIZE_TGZ) in TestFSImage for 2.6.

        I'll post a suggested backport patch for 2.6.0.

        Show
        sjlee0 Sangjin Lee added a comment - The backport to 2.6.0 is pretty trivial but the test FS image needs to be recreated for 2.6.0. Also it'd be good to rename the variable (HADOOP_2_7_ZER0_BLOCK_SIZE_TGZ) in TestFSImage for 2.6. I'll post a suggested backport patch for 2.6.0.
        Hide
        vinodkv Vinod Kumar Vavilapalli added a comment -

        Thanks Sangjin Lee. Pushed the fix into 2.6.1 after running compilation and test TestFSImage.

        Show
        vinodkv Vinod Kumar Vavilapalli added a comment - Thanks Sangjin Lee . Pushed the fix into 2.6.1 after running compilation and test TestFSImage.

          People

          • Assignee:
            shahrs87 Rushabh S Shah
            Reporter:
            kihwal Kihwal Lee
          • Votes:
            0 Vote for this issue
            Watchers:
            16 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development