Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-8384

Allow NN to startup if there are files having a lease but are not under construction

    Details

    • Hadoop Flags:
      Reviewed

      Description

      When there are files having a lease but are not under construction, NN will fail to start up with

      15/05/12 00:36:31 ERROR namenode.FSImage: Unable to save image for /hadoop/hdfs/namenode
      java.lang.IllegalStateException
              at com.google.common.base.Preconditions.checkState(Preconditions.java:129)
              at org.apache.hadoop.hdfs.server.namenode.LeaseManager.getINodesUnderConstruction(LeaseManager.java:412)
              at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getFilesUnderConstruction(FSNamesystem.java:7124)
              ...
      

      The actually problem is that the image could be corrupted by bugs like HDFS-7587. We should have an option/conf to allow NN to start up so that the problematic files could possibly be deleted.

      1. HDFS-8384.000.patch
        2 kB
        Jing Zhao
      2. HDFS-8384-branch-2.6.patch
        3 kB
        Jing Zhao
      3. HDFS-8384-branch-2.7.patch
        3 kB
        Jing Zhao

        Issue Links

          Activity

          Hide
          azulmarino Dean added a comment -

          Is there a workaround ?

          Show
          azulmarino Dean added a comment - Is there a workaround ?
          Hide
          vinayrpet Vinayakumar B added a comment -

          Is this improvement is to allow NNs to start with old images created before HDFS-7587 was in?
          Or in Some other cases also this situation (Lease without UC file) is possible?

          Show
          vinayrpet Vinayakumar B added a comment - Is this improvement is to allow NNs to start with old images created before HDFS-7587 was in? Or in Some other cases also this situation (Lease without UC file) is possible?
          Hide
          szetszwo Tsz Wo Nicholas Sze added a comment -

          Dean, I don't have a workaround except using patched jar.

          Vinay, yes it is for HDFS-7587. I don't aware if there are other possible cases.

          Show
          szetszwo Tsz Wo Nicholas Sze added a comment - Dean, I don't have a workaround except using patched jar. Vinay, yes it is for HDFS-7587 . I don't aware if there are other possible cases.
          Hide
          jingzhao Jing Zhao added a comment -

          Post a simple patch:

          1. Log a warning msg instead of throwing exception in LeaseManager#getNumUnderConstructionBlocks which is called by FSNamesystem#getCompleteBlocksTotal when NN starts
          2. Throw an IOException instead of an IllegalArgumentException in finalizeINodeFileUnderConstruction so that the IOException can be caught by the LeaseManager monitor.
          3. The exception mentioned in the description has been bypassed in HDFS-6757.
          Show
          jingzhao Jing Zhao added a comment - Post a simple patch: Log a warning msg instead of throwing exception in LeaseManager#getNumUnderConstructionBlocks which is called by FSNamesystem#getCompleteBlocksTotal when NN starts Throw an IOException instead of an IllegalArgumentException in finalizeINodeFileUnderConstruction so that the IOException can be caught by the LeaseManager monitor. The exception mentioned in the description has been bypassed in HDFS-6757 .
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 18m 7s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 javac 7m 55s There were no new javac warning messages.
          +1 javadoc 10m 3s There were no new javadoc warning messages.
          +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings.
          -1 checkstyle 1m 23s The applied patch generated 1 new checkstyle issues (total was 273, now 273).
          +1 whitespace 0m 0s The patch has no lines that end in whitespace.
          +1 install 1m 30s mvn install still works.
          +1 eclipse:eclipse 0m 31s The patch built with eclipse:eclipse.
          -1 findbugs 2m 8s Post-patch findbugs hadoop-hdfs-project/hadoop-hdfs compilation is broken.
          +1 findbugs 2m 8s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          +1 native 0m 24s Pre-build of native portion
          -1 hdfs tests 161m 24s Tests failed in hadoop-hdfs.
              203m 53s  



          Reason Tests
          Failed unit tests hadoop.hdfs.server.balancer.TestBalancerWithMultipleNameNodes
            hadoop.hdfs.TestLeaseRecovery2
            hadoop.hdfs.web.TestWebHDFSOAuth2
            hadoop.fs.TestHdfsNativeCodeLoader



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12754045/HDFS-8384.000.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 0ebc658
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/12286/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt
          hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12286/artifact/patchprocess/testrun_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12286/testReport/
          Java 1.7.0_55
          uname Linux asf906.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12286/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 18m 7s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac 7m 55s There were no new javac warning messages. +1 javadoc 10m 3s There were no new javadoc warning messages. +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 1m 23s The applied patch generated 1 new checkstyle issues (total was 273, now 273). +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 30s mvn install still works. +1 eclipse:eclipse 0m 31s The patch built with eclipse:eclipse. -1 findbugs 2m 8s Post-patch findbugs hadoop-hdfs-project/hadoop-hdfs compilation is broken. +1 findbugs 2m 8s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 native 0m 24s Pre-build of native portion -1 hdfs tests 161m 24s Tests failed in hadoop-hdfs.     203m 53s   Reason Tests Failed unit tests hadoop.hdfs.server.balancer.TestBalancerWithMultipleNameNodes   hadoop.hdfs.TestLeaseRecovery2   hadoop.hdfs.web.TestWebHDFSOAuth2   hadoop.fs.TestHdfsNativeCodeLoader Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12754045/HDFS-8384.000.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 0ebc658 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/12286/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12286/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12286/testReport/ Java 1.7.0_55 uname Linux asf906.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12286/console This message was automatically generated.
          Hide
          szetszwo Tsz Wo Nicholas Sze added a comment -

          +1 patch looks good.

          Do we need a different patch for pre-HDFS-6757?

          Show
          szetszwo Tsz Wo Nicholas Sze added a comment - +1 patch looks good. Do we need a different patch for pre- HDFS-6757 ?
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 17m 21s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 javac 7m 41s There were no new javac warning messages.
          +1 javadoc 9m 55s There were no new javadoc warning messages.
          +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings.
          -1 checkstyle 1m 20s The applied patch generated 1 new checkstyle issues (total was 273, now 273).
          +1 whitespace 0m 0s The patch has no lines that end in whitespace.
          +1 install 1m 28s mvn install still works.
          +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse.
          +1 findbugs 2m 28s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          +1 native 3m 10s Pre-build of native portion
          -1 hdfs tests 176m 58s Tests failed in hadoop-hdfs.
              221m 20s  



          Reason Tests
          Failed unit tests hadoop.hdfs.qjournal.client.TestQuorumJournalManager



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12754045/HDFS-8384.000.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / c83d13c
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/12304/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt
          hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12304/artifact/patchprocess/testrun_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12304/testReport/
          Java 1.7.0_55
          uname Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12304/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 17m 21s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac 7m 41s There were no new javac warning messages. +1 javadoc 9m 55s There were no new javadoc warning messages. +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 1m 20s The applied patch generated 1 new checkstyle issues (total was 273, now 273). +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 28s mvn install still works. +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse. +1 findbugs 2m 28s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 native 3m 10s Pre-build of native portion -1 hdfs tests 176m 58s Tests failed in hadoop-hdfs.     221m 20s   Reason Tests Failed unit tests hadoop.hdfs.qjournal.client.TestQuorumJournalManager Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12754045/HDFS-8384.000.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / c83d13c checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/12304/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12304/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12304/testReport/ Java 1.7.0_55 uname Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12304/console This message was automatically generated.
          Hide
          jingzhao Jing Zhao added a comment -

          Thanks for the review, Nicholas! Looks like the failed tests are unrelated. I will commit the patch shortly.

          I will also post a patch for 2.7.2/2.6.1 (before HDFS-6757).

          Show
          jingzhao Jing Zhao added a comment - Thanks for the review, Nicholas! Looks like the failed tests are unrelated. I will commit the patch shortly. I will also post a patch for 2.7.2/2.6.1 (before HDFS-6757 ).
          Hide
          jingzhao Jing Zhao added a comment -

          I've committed the patch to trunk and branch-2.

          The patches for 2.6.1 and 2.7.2 are also uploaded. Tsz Wo Nicholas Sze, could you please also take a look at them?

          Show
          jingzhao Jing Zhao added a comment - I've committed the patch to trunk and branch-2. The patches for 2.6.1 and 2.7.2 are also uploaded. Tsz Wo Nicholas Sze , could you please also take a look at them?
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #353 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/353/)
          HDFS-8384. Allow NN to startup if there are files having a lease but are not under construction. Contributed by Jing Zhao. (jing9: rev 8928729c80af0a154524e06fb13ed9b191986a78)

          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/LeaseManager.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #353 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/353/ ) HDFS-8384 . Allow NN to startup if there are files having a lease but are not under construction. Contributed by Jing Zhao. (jing9: rev 8928729c80af0a154524e06fb13ed9b191986a78) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/LeaseManager.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #8405 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8405/)
          HDFS-8384. Allow NN to startup if there are files having a lease but are not under construction. Contributed by Jing Zhao. (jing9: rev 8928729c80af0a154524e06fb13ed9b191986a78)

          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/LeaseManager.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #8405 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8405/ ) HDFS-8384 . Allow NN to startup if there are files having a lease but are not under construction. Contributed by Jing Zhao. (jing9: rev 8928729c80af0a154524e06fb13ed9b191986a78) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/LeaseManager.java
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          -1 patch 0m 0s The patch command could not apply the patch during dryrun.



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12754250/HDFS-8384-branch-2.7.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision branch-2 / 67bce1e
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12315/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment -1 patch 0m 0s The patch command could not apply the patch during dryrun. Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12754250/HDFS-8384-branch-2.7.patch Optional Tests javadoc javac unit findbugs checkstyle git revision branch-2 / 67bce1e Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12315/console This message was automatically generated.
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #347 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/347/)
          HDFS-8384. Allow NN to startup if there are files having a lease but are not under construction. Contributed by Jing Zhao. (jing9: rev 8928729c80af0a154524e06fb13ed9b191986a78)

          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/LeaseManager.java
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #347 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/347/ ) HDFS-8384 . Allow NN to startup if there are files having a lease but are not under construction. Contributed by Jing Zhao. (jing9: rev 8928729c80af0a154524e06fb13ed9b191986a78) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/LeaseManager.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk #2296 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2296/)
          HDFS-8384. Allow NN to startup if there are files having a lease but are not under construction. Contributed by Jing Zhao. (jing9: rev 8928729c80af0a154524e06fb13ed9b191986a78)

          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/LeaseManager.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #2296 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2296/ ) HDFS-8384 . Allow NN to startup if there are files having a lease but are not under construction. Contributed by Jing Zhao. (jing9: rev 8928729c80af0a154524e06fb13ed9b191986a78) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/LeaseManager.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk #2274 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2274/)
          HDFS-8384. Allow NN to startup if there are files having a lease but are not under construction. Contributed by Jing Zhao. (jing9: rev 8928729c80af0a154524e06fb13ed9b191986a78)

          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/LeaseManager.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2274 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2274/ ) HDFS-8384 . Allow NN to startup if there are files having a lease but are not under construction. Contributed by Jing Zhao. (jing9: rev 8928729c80af0a154524e06fb13ed9b191986a78) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/LeaseManager.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-Yarn-trunk #1085 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1085/)
          HDFS-8384. Allow NN to startup if there are files having a lease but are not under construction. Contributed by Jing Zhao. (jing9: rev 8928729c80af0a154524e06fb13ed9b191986a78)

          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/LeaseManager.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Yarn-trunk #1085 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1085/ ) HDFS-8384 . Allow NN to startup if there are files having a lease but are not under construction. Contributed by Jing Zhao. (jing9: rev 8928729c80af0a154524e06fb13ed9b191986a78) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/LeaseManager.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #336 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/336/)
          HDFS-8384. Allow NN to startup if there are files having a lease but are not under construction. Contributed by Jing Zhao. (jing9: rev 8928729c80af0a154524e06fb13ed9b191986a78)

          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/LeaseManager.java
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #336 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/336/ ) HDFS-8384 . Allow NN to startup if there are files having a lease but are not under construction. Contributed by Jing Zhao. (jing9: rev 8928729c80af0a154524e06fb13ed9b191986a78) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/LeaseManager.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
          Hide
          szetszwo Tsz Wo Nicholas Sze added a comment -

          +1 the 2.6 and 2.7 patches look good.

          Show
          szetszwo Tsz Wo Nicholas Sze added a comment - +1 the 2.6 and 2.7 patches look good.
          Hide
          jingzhao Jing Zhao added a comment -

          Thanks again for the review, Nicholas! I've committed the 2.7 patch.

          Vinod Kumar Vavilapalli, please see if you want to include this in 2.6.1.

          Show
          jingzhao Jing Zhao added a comment - Thanks again for the review, Nicholas! I've committed the 2.7 patch. Vinod Kumar Vavilapalli , please see if you want to include this in 2.6.1.
          Hide
          vinodkv Vinod Kumar Vavilapalli added a comment -

          Right when I was about to start closing down the release!

          Pulled this into 2.6.1. Ran compilation before the push.

          Show
          vinodkv Vinod Kumar Vavilapalli added a comment - Right when I was about to start closing down the release! Pulled this into 2.6.1. Ran compilation before the push.
          Hide
          yzhangal Yongjun Zhang added a comment -

          HI Tsz Wo Nicholas Sze, Jing Zhao ,

          Thanks for your earlier work on this issue. HDFS-8384 seems to be just a workaround for fsimages created by HDFS-7587, do you guys know if the real issue (to prevent this kind of fsimage to be created) is addressed somewhere?

          Thanks.

          Show
          yzhangal Yongjun Zhang added a comment - HI Tsz Wo Nicholas Sze , Jing Zhao , Thanks for your earlier work on this issue. HDFS-8384 seems to be just a workaround for fsimages created by HDFS-7587 , do you guys know if the real issue (to prevent this kind of fsimage to be created) is addressed somewhere? Thanks.

            People

            • Assignee:
              jingzhao Jing Zhao
              Reporter:
              szetszwo Tsz Wo Nicholas Sze
            • Votes:
              0 Vote for this issue
              Watchers:
              13 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development