Details

    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      java.lang.NullPointerException
      	at org.apache.hadoop.hdfs.server.datanode.BlockSender.<init>(BlockSender.java:264)
      	at org.apache.hadoop.hdfs.server.datanode.DataXceiver.readBlock(DataXceiver.java:506)
      	at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opReadBlock(Receiver.java:116)
      	at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:71)
      	at org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:249)
      	at java.lang.Thread.run(Thread.java:745)
      

      BlockSender.java:264 is shown below

            this.volumeRef = datanode.data.getVolume(block).obtainReference();
      
      1. HDFS-7884-002.patch
        1 kB
        Brahma Reddy Battula
      2. h7884_20150313.patch
        4 kB
        Tsz Wo Nicholas Sze
      3. HDFS-7884.patch
        1 kB
        Brahma Reddy Battula
      4. org.apache.hadoop.hdfs.TestAppendSnapshotTruncate-output.txt
        3.29 MB
        Tsz Wo Nicholas Sze

        Issue Links

          Activity

          Hide
          vinodkv Vinod Kumar Vavilapalli added a comment -

          Sangjin Lee backported this to 2.6.1. I just pushed the commit to 2.6.1 after running compilation.

          Show
          vinodkv Vinod Kumar Vavilapalli added a comment - Sangjin Lee backported this to 2.6.1. I just pushed the commit to 2.6.1 after running compilation.
          Hide
          sjlee0 Sangjin Lee added a comment -

          The patch applies to 2.6.0 cleanly.

          Show
          sjlee0 Sangjin Lee added a comment - The patch applies to 2.6.0 cleanly.
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #142 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/142/)
          HDFS-7884. Fix NullPointerException in BlockSender when the generation stamp provided by the client is larger than the one stored in the datanode. Contributed by Brahma Reddy Battula (szetszwo: rev d7e3c3364eb904f55a878bc14c331952f9dadab2)

          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockSender.java
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #142 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/142/ ) HDFS-7884 . Fix NullPointerException in BlockSender when the generation stamp provided by the client is larger than the one stored in the datanode. Contributed by Brahma Reddy Battula (szetszwo: rev d7e3c3364eb904f55a878bc14c331952f9dadab2) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockSender.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #133 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/133/)
          HDFS-7884. Fix NullPointerException in BlockSender when the generation stamp provided by the client is larger than the one stored in the datanode. Contributed by Brahma Reddy Battula (szetszwo: rev d7e3c3364eb904f55a878bc14c331952f9dadab2)

          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockSender.java
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #133 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/133/ ) HDFS-7884 . Fix NullPointerException in BlockSender when the generation stamp provided by the client is larger than the one stored in the datanode. Contributed by Brahma Reddy Battula (szetszwo: rev d7e3c3364eb904f55a878bc14c331952f9dadab2) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockSender.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk #2074 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2074/)
          HDFS-7884. Fix NullPointerException in BlockSender when the generation stamp provided by the client is larger than the one stored in the datanode. Contributed by Brahma Reddy Battula (szetszwo: rev d7e3c3364eb904f55a878bc14c331952f9dadab2)

          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockSender.java
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2074 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2074/ ) HDFS-7884 . Fix NullPointerException in BlockSender when the generation stamp provided by the client is larger than the one stored in the datanode. Contributed by Brahma Reddy Battula (szetszwo: rev d7e3c3364eb904f55a878bc14c331952f9dadab2) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockSender.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk #2092 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2092/)
          HDFS-7884. Fix NullPointerException in BlockSender when the generation stamp provided by the client is larger than the one stored in the datanode. Contributed by Brahma Reddy Battula (szetszwo: rev d7e3c3364eb904f55a878bc14c331952f9dadab2)

          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockSender.java
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #2092 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2092/ ) HDFS-7884 . Fix NullPointerException in BlockSender when the generation stamp provided by the client is larger than the one stored in the datanode. Contributed by Brahma Reddy Battula (szetszwo: rev d7e3c3364eb904f55a878bc14c331952f9dadab2) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockSender.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk #876 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/876/)
          HDFS-7884. Fix NullPointerException in BlockSender when the generation stamp provided by the client is larger than the one stored in the datanode. Contributed by Brahma Reddy Battula (szetszwo: rev d7e3c3364eb904f55a878bc14c331952f9dadab2)

          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockSender.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk #876 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/876/ ) HDFS-7884 . Fix NullPointerException in BlockSender when the generation stamp provided by the client is larger than the one stored in the datanode. Contributed by Brahma Reddy Battula (szetszwo: rev d7e3c3364eb904f55a878bc14c331952f9dadab2) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockSender.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #142 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/142/)
          HDFS-7884. Fix NullPointerException in BlockSender when the generation stamp provided by the client is larger than the one stored in the datanode. Contributed by Brahma Reddy Battula (szetszwo: rev d7e3c3364eb904f55a878bc14c331952f9dadab2)

          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockSender.java
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #142 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/142/ ) HDFS-7884 . Fix NullPointerException in BlockSender when the generation stamp provided by the client is larger than the one stored in the datanode. Contributed by Brahma Reddy Battula (szetszwo: rev d7e3c3364eb904f55a878bc14c331952f9dadab2) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockSender.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Hide
          brahmareddy Brahma Reddy Battula added a comment -

          Thanks a lot for reviews and commit !!!

          Show
          brahmareddy Brahma Reddy Battula added a comment - Thanks a lot for reviews and commit !!!
          Hide
          vinayrpet Vinayakumar B added a comment -
          Show
          vinayrpet Vinayakumar B added a comment - Thanks Tsz Wo Nicholas Sze and Brahma Reddy Battula
          Hide
          szetszwo Tsz Wo Nicholas Sze added a comment -

          I have committed this. Thanks, Brahma!

          Show
          szetszwo Tsz Wo Nicholas Sze added a comment - I have committed this. Thanks, Brahma!
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #7410 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7410/)
          HDFS-7884. Fix NullPointerException in BlockSender when the generation stamp provided by the client is larger than the one stored in the datanode. Contributed by Brahma Reddy Battula (szetszwo: rev d7e3c3364eb904f55a878bc14c331952f9dadab2)

          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockSender.java
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #7410 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7410/ ) HDFS-7884 . Fix NullPointerException in BlockSender when the generation stamp provided by the client is larger than the one stored in the datanode. Contributed by Brahma Reddy Battula (szetszwo: rev d7e3c3364eb904f55a878bc14c331952f9dadab2) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockSender.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Hide
          szetszwo Tsz Wo Nicholas Sze added a comment -

          > While rebasing the patch, found that test actually passes even though there is a NPE.

          You are right. The client will retry when there is a NPE so that the test indeed is passed. Let's commit Brahma's new patch first and add the test later.

          +1 on HDFS-7884-002.patch

          Show
          szetszwo Tsz Wo Nicholas Sze added a comment - > While rebasing the patch, found that test actually passes even though there is a NPE. You are right. The client will retry when there is a NPE so that the test indeed is passed. Let's commit Brahma's new patch first and add the test later. +1 on HDFS-7884 -002.patch
          Hide
          brahmareddy Brahma Reddy Battula added a comment -

          Testcase failures are unrelated this jira

          Show
          brahmareddy Brahma Reddy Battula added a comment - Testcase failures are unrelated this jira
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12706361/HDFS-7884-002.patch
          against trunk revision 4335429.

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs:

          org.apache.hadoop.tracing.TestTracing

          Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/10022//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/10022//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12706361/HDFS-7884-002.patch against trunk revision 4335429. +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs: org.apache.hadoop.tracing.TestTracing Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/10022//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/10022//console This message is automatically generated.
          Hide
          vinodkv Vinod Kumar Vavilapalli added a comment -

          Canceling the patch for rebasing.

          Can this be done soon? Otherwise, in the interest of 2.7's progress, I'd like to move this out to 2.8.

          Show
          vinodkv Vinod Kumar Vavilapalli added a comment - Canceling the patch for rebasing. Can this be done soon? Otherwise, in the interest of 2.7's progress, I'd like to move this out to 2.8.
          Hide
          brahmareddy Brahma Reddy Battula added a comment -

          ..

          Show
          brahmareddy Brahma Reddy Battula added a comment - ..
          Hide
          brahmareddy Brahma Reddy Battula added a comment -

          Thanks a lot for review and comments..will look into this one..( sorry for late reply,, I was in leave..)

          Show
          brahmareddy Brahma Reddy Battula added a comment - Thanks a lot for review and comments..will look into this one..( sorry for late reply,, I was in leave..)
          Hide
          vinayrpet Vinayakumar B added a comment -

          While rebasing the patch, found that test actually passes even though there is a NPE.

          Show
          vinayrpet Vinayakumar B added a comment - While rebasing the patch, found that test actually passes even though there is a NPE.
          Hide
          vinayrpet Vinayakumar B added a comment -

          Will commit the patch soon.

          Seems patch needs a rebase

          Show
          vinayrpet Vinayakumar B added a comment - Will commit the patch soon. Seems patch needs a rebase
          Hide
          vinayrpet Vinayakumar B added a comment -

          +1 for the patch h7884_20150313.patch and Test.

          test failures seems unrelated to current patch

          Will commit the patch soon.

          Show
          vinayrpet Vinayakumar B added a comment - +1 for the patch h7884_20150313.patch and Test. test failures seems unrelated to current patch Will commit the patch soon.
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12704227/h7884_20150313.patch
          against trunk revision 6dae6d1.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 2 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs:

          org.apache.hadoop.hdfs.server.blockmanagement.TestDatanodeManager
          org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestScrLazyPersistFiles

          Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/9858//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/9858//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12704227/h7884_20150313.patch against trunk revision 6dae6d1. +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 2 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs: org.apache.hadoop.hdfs.server.blockmanagement.TestDatanodeManager org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestScrLazyPersistFiles Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/9858//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/9858//console This message is automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12704227/h7884_20150313.patch
          against trunk revision 6dae6d1.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 2 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs:

          org.apache.hadoop.hdfs.server.blockmanagement.TestDatanodeManager

          Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/9859//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/9859//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12704227/h7884_20150313.patch against trunk revision 6dae6d1. +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 2 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs: org.apache.hadoop.hdfs.server.blockmanagement.TestDatanodeManager Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/9859//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/9859//console This message is automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12704227/h7884_20150313.patch
          against trunk revision 6dae6d1.

          -1 patch. Trunk compilation may be broken.

          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/9857//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12704227/h7884_20150313.patch against trunk revision 6dae6d1. -1 patch . Trunk compilation may be broken. Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/9857//console This message is automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12704227/h7884_20150313.patch
          against trunk revision 6dae6d1.

          -1 patch. Trunk compilation may be broken.

          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/9856//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12704227/h7884_20150313.patch against trunk revision 6dae6d1. -1 patch . Trunk compilation may be broken. Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/9856//console This message is automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12704227/h7884_20150313.patch
          against trunk revision 6dae6d1.

          -1 patch. Trunk compilation may be broken.

          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/9854//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12704227/h7884_20150313.patch against trunk revision 6dae6d1. -1 patch . Trunk compilation may be broken. Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/9854//console This message is automatically generated.
          Hide
          szetszwo Tsz Wo Nicholas Sze added a comment -

          h7884_20150313.patch: adds a test with truncate.

          Show
          szetszwo Tsz Wo Nicholas Sze added a comment - h7884_20150313.patch: adds a test with truncate.
          Hide
          szetszwo Tsz Wo Nicholas Sze added a comment -

          Vinayakumar B, you analysis makes sense. This bug is very similar to HDFS-7885.

          Brahma Reddy Battula, the patch looks good. Some minor comments:

          • We should check if LOG.isDebugEnabled() for before calling LOG.debug(..).
          • Is it easy to add a test? You may see if you could add a test similar to HDFS-7885.
          Show
          szetszwo Tsz Wo Nicholas Sze added a comment - Vinayakumar B , you analysis makes sense. This bug is very similar to HDFS-7885 . Brahma Reddy Battula , the patch looks good. Some minor comments: We should check if LOG.isDebugEnabled() for before calling LOG.debug(..). Is it easy to add a test? You may see if you could add a test similar to HDFS-7885 .
          Hide
          brahmareddy Brahma Reddy Battula added a comment -

          Thanks for update!! Even I ran locally before submitting the patch,All are passed...

          Show
          brahmareddy Brahma Reddy Battula added a comment - Thanks for update!! Even I ran locally before submitting the patch,All are passed...
          Hide
          eddyxu Lei (Eddy) Xu added a comment -

          I ran mvn test -Dtest=TestDatanodeManager on my laptop and test passes. TestRetryCacheWithHA is not relevant.

          The patch looks good to me. Non-binding +1.

          Show
          eddyxu Lei (Eddy) Xu added a comment - I ran mvn test -Dtest=TestDatanodeManager on my laptop and test passes. TestRetryCacheWithHA is not relevant. The patch looks good to me. Non-binding +1.
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12703410/HDFS-7884.patch
          against trunk revision 7711049.

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs:

          org.apache.hadoop.hdfs.server.namenode.ha.TestRetryCacheWithHA

          The following test timeouts occurred in hadoop-hdfs-project/hadoop-hdfs:

          org.apache.hadoop.hdfs.server.blockmanagement.TestDatanodeManager

          Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/9816//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/9816//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12703410/HDFS-7884.patch against trunk revision 7711049. +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs: org.apache.hadoop.hdfs.server.namenode.ha.TestRetryCacheWithHA The following test timeouts occurred in hadoop-hdfs-project/hadoop-hdfs: org.apache.hadoop.hdfs.server.blockmanagement.TestDatanodeManager Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/9816//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/9816//console This message is automatically generated.
          Hide
          vinayrpet Vinayakumar B added a comment -

          I think some problem with the cleaning of the jenkins workspace.

          [ERROR] Failed to execute goal org.apache.maven.plugins:maven-clean-plugin:2.5:clean (default-clean) on project hadoop-hdfs: Failed to clean project: Failed to delete /home/jenkins/jenkins-slave/workspace/PreCommit-HDFS-Build/hadoop-hdfs-project/hadoop-hdfs/target/test/data/dfs/data/data3 -> [Help 1]
          Show
          vinayrpet Vinayakumar B added a comment - I think some problem with the cleaning of the jenkins workspace. [ERROR] Failed to execute goal org.apache.maven.plugins:maven-clean-plugin:2.5:clean (default-clean) on project hadoop-hdfs: Failed to clean project: Failed to delete /home/jenkins/jenkins-slave/workspace/PreCommit-HDFS-Build/hadoop-hdfs-project/hadoop-hdfs/target/test/data/dfs/data/data3 -> [Help 1]
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12703410/HDFS-7884.patch
          against trunk revision 5578e22.

          -1 patch. Trunk compilation may be broken.

          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/9796//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12703410/HDFS-7884.patch against trunk revision 5578e22. -1 patch . Trunk compilation may be broken. Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/9796//console This message is automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12703410/HDFS-7884.patch
          against trunk revision 5578e22.

          -1 patch. Trunk compilation may be broken.

          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/9795//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12703410/HDFS-7884.patch against trunk revision 5578e22. -1 patch . Trunk compilation may be broken. Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/9795//console This message is automatically generated.
          Hide
          brahmareddy Brahma Reddy Battula added a comment -

          Thanks a lot for your detailed analysis, I also feel same..

          Cause : Reader is trying to read using the old block Id But the original block's gs was modified after appending.

          When client is latest and DN have old GS, then only throw exception. Othercase it should support read according to code.Here we missed the case when DN is having the lastest GS and client is having the old GS..( GS will not be checked while getting the replica and initializing the BlockSender)..Hence we need handle..

          Show
          brahmareddy Brahma Reddy Battula added a comment - Thanks a lot for your detailed analysis, I also feel same.. Cause : Reader is trying to read using the old block Id But the original block's gs was modified after appending. When client is latest and DN have old GS, then only throw exception. Othercase it should support read according to code.Here we missed the case when DN is having the lastest GS and client is having the old GS..( GS will not be checked while getting the replica and initializing the BlockSender)..Hence we need handle..
          Hide
          vinayrpet Vinayakumar B added a comment - - edited

          Earlier comment was a quick guess.

          The actual reason is, reader is trying to read using the old block Id (blk_1073741911_1102)
          But the original block's gs was modified after appending.

          2015-03-09 02:03:21,488 INFO  impl.FsDatasetImpl (FsDatasetImpl.java:append(1015)) - Appending to FinalizedReplica, blk_1073741911_1102, FINALIZED
          2015-03-09 02:03:21,501 INFO  namenode.FSNamesystem (FSNamesystem.java:updatePipeline(6199)) - updatePipeline(blk_1073741911_1102, newGS=1110, newLength=638, newNodes=[127.0.0.1:52069, 127.0.0.1:52065, 127.0.0.1:52074], client=DFSClient_NONMAPREDUCE_-727094507_1)

          For initializing the BlockSender, while getting the replica, GS was not checked.

                synchronized(datanode.data) { 
                  replica = getReplica(block, datanode);
                  replicaVisibleLength = replica.getVisibleLength();
                }

          GS was checked against client-passed GS only for latest. i.e. If client is latest and DN have old, then only throw exception. Othercase it should support read according to code.

               if (replica.getGenerationStamp() < block.getGenerationStamp()) {
                  throw new IOException("Replica gen stamp < block genstamp, block="
                      + block + ", replica=" + replica);
                }

          But while getting the Volume reference it will be checked down the line in ReplicaMap#get

            ReplicaInfo get(String bpid, Block block) {
              checkBlockPool(bpid);
              checkBlock(block);
              ReplicaInfo replicaInfo = get(bpid, block.getBlockId());
              if (replicaInfo != null && 
                  block.getGenerationStamp() == replicaInfo.getGenerationStamp()) {
                return replicaInfo;
              }
              return null;
            }

          So I think, In this case, If client-read needs to go through, then need to bump up the genstamp to latest, like below.

                if (replica.getGenerationStamp() < block.getGenerationStamp()) {
                  throw new IOException("Replica gen stamp < block genstamp, block="
                      + block + ", replica=" + replica);
                } else if (replica.getGenerationStamp() > block.getGenerationStamp()) {
                  DataNode.LOG.debug("Bumping up the client provided"
                      + " block's genstamp to latest " + replica.getGenerationStamp()
                      + " for block " + block);
                  block.setGenerationStamp(replica.getGenerationStamp());
                }

          Else needs to throw exception from here itself.

          Any thoughts.?

          Show
          vinayrpet Vinayakumar B added a comment - - edited Earlier comment was a quick guess. The actual reason is, reader is trying to read using the old block Id (blk_1073741911_1102) But the original block's gs was modified after appending. 2015-03-09 02:03:21,488 INFO impl.FsDatasetImpl (FsDatasetImpl.java:append(1015)) - Appending to FinalizedReplica, blk_1073741911_1102, FINALIZED 2015-03-09 02:03:21,501 INFO namenode.FSNamesystem (FSNamesystem.java:updatePipeline(6199)) - updatePipeline(blk_1073741911_1102, newGS=1110, newLength=638, newNodes=[127.0.0.1:52069, 127.0.0.1:52065, 127.0.0.1:52074], client=DFSClient_NONMAPREDUCE_-727094507_1) For initializing the BlockSender, while getting the replica, GS was not checked. synchronized (datanode.data) { replica = getReplica(block, datanode); replicaVisibleLength = replica.getVisibleLength(); } GS was checked against client-passed GS only for latest. i.e. If client is latest and DN have old, then only throw exception. Othercase it should support read according to code. if (replica.getGenerationStamp() < block.getGenerationStamp()) { throw new IOException( "Replica gen stamp < block genstamp, block=" + block + ", replica=" + replica); } But while getting the Volume reference it will be checked down the line in ReplicaMap#get ReplicaInfo get( String bpid, Block block) { checkBlockPool(bpid); checkBlock(block); ReplicaInfo replicaInfo = get(bpid, block.getBlockId()); if (replicaInfo != null && block.getGenerationStamp() == replicaInfo.getGenerationStamp()) { return replicaInfo; } return null ; } So I think, In this case, If client-read needs to go through, then need to bump up the genstamp to latest, like below. if (replica.getGenerationStamp() < block.getGenerationStamp()) { throw new IOException( "Replica gen stamp < block genstamp, block=" + block + ", replica=" + replica); } else if (replica.getGenerationStamp() > block.getGenerationStamp()) { DataNode.LOG.debug( "Bumping up the client provided" + " block's genstamp to latest " + replica.getGenerationStamp() + " for block " + block); block.setGenerationStamp(replica.getGenerationStamp()); } Else needs to throw exception from here itself. Any thoughts.?
          Hide
          brahmareddy Brahma Reddy Battula added a comment -

          Thanks a lot for your input...let me look around more...

          Show
          brahmareddy Brahma Reddy Battula added a comment - Thanks a lot for your input...let me look around more...
          Hide
          szetszwo Tsz Wo Nicholas Sze added a comment -

          org.apache.hadoop.hdfs.TestAppendSnapshotTruncate-output.txt: NPE inside.

          Show
          szetszwo Tsz Wo Nicholas Sze added a comment - org.apache.hadoop.hdfs.TestAppendSnapshotTruncate-output.txt: NPE inside.
          Hide
          szetszwo Tsz Wo Nicholas Sze added a comment -

          When I was working on HDFS-7746, the new test TestAppendSnapshotTruncate failed and I found NullPointerException in the log.

          Show
          szetszwo Tsz Wo Nicholas Sze added a comment - When I was working on HDFS-7746 , the new test TestAppendSnapshotTruncate failed and I found NullPointerException in the log.
          Hide
          eddyxu Lei (Eddy) Xu added a comment -

          Tsz Wo Nicholas Sze would you mind me to take this JIRA if you have not started on this yet.

          Moreover, could you post the context that generates this NullPointerException? Thanks!

          Show
          eddyxu Lei (Eddy) Xu added a comment - Tsz Wo Nicholas Sze would you mind me to take this JIRA if you have not started on this yet. Moreover, could you post the context that generates this NullPointerException ? Thanks!
          Hide
          brahmareddy Brahma Reddy Battula added a comment -

          Tsz Wo Nicholas Sze Please re-assign to yourself, if you started work on this jira...Thanks

          Show
          brahmareddy Brahma Reddy Battula added a comment - Tsz Wo Nicholas Sze Please re-assign to yourself, if you started work on this jira...Thanks

            People

            • Assignee:
              brahmareddy Brahma Reddy Battula
              Reporter:
              szetszwo Tsz Wo Nicholas Sze
            • Votes:
              0 Vote for this issue
              Watchers:
              11 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development