Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: datanode, hdfs
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    1. HDFS-10652.001.patch
      6 kB
      Yongjun Zhang
    2. HDFS-10652.003.patch
      9 kB
      Yongjun Zhang
    3. HDFS-10652.004.patch
      9 kB
      Vinayakumar B
    4. HDFS-10652.005.patch
      9 kB
      Yongjun Zhang
    5. HDFS-10652.006.patch
      10 kB
      Yongjun Zhang
    6. HDFS-10652.007.patch
      10 kB
      Yongjun Zhang
    7. HDFS-10652.008.patch
      11 kB
      Yongjun Zhang
    8. HDFS-10652-002.patch
      7 kB
      Vinayakumar B

      Issue Links

        Activity

        Hide
        yzhangal Yongjun Zhang added a comment -

        Thanks Vinayakumar B a lot for the help to create the testcase for the issue reported in HDFS-10587. Attaching it as rev 001 here, since it really creates one scenario we found in HDFS-10587, which got fixed by HDFS-4660. That is, reverting HDFS-4660, HDFS-9220, and HDFS-8722, the test failed as expected. So this test makes a perfect unit test for HDFS-4660.

        Show
        yzhangal Yongjun Zhang added a comment - Thanks Vinayakumar B a lot for the help to create the testcase for the issue reported in HDFS-10587 . Attaching it as rev 001 here, since it really creates one scenario we found in HDFS-10587 , which got fixed by HDFS-4660 . That is, reverting HDFS-4660 , HDFS-9220 , and HDFS-8722 , the test failed as expected. So this test makes a perfect unit test for HDFS-4660 .
        Hide
        vinayrpet Vinayakumar B added a comment -

        Thanks Yongjun Zhang.
        test given earlier was not of commit quality.
        May be we can refine the test to be able to commit.
        Like, removing "System.out.println("VINAY : read : "+count);" and some other improvements if required.

        Show
        vinayrpet Vinayakumar B added a comment - Thanks Yongjun Zhang . test given earlier was not of commit quality. May be we can refine the test to be able to commit. Like, removing " System.out.println("VINAY : read : "+count); " and some other improvements if required.
        Hide
        yzhangal Yongjun Zhang added a comment -

        Thanks Vinay, yes, agreed, and would you please add comments to indicate the steps involved too? Thanks.

        Show
        yzhangal Yongjun Zhang added a comment - Thanks Vinay, yes, agreed, and would you please add comments to indicate the steps involved too? Thanks.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 22s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 7m 14s trunk passed
        +1 compile 0m 49s trunk passed
        +1 checkstyle 0m 27s trunk passed
        +1 mvnsite 0m 55s trunk passed
        +1 mvneclipse 0m 12s trunk passed
        +1 findbugs 1m 46s trunk passed
        +1 javadoc 0m 56s trunk passed
        +1 mvninstall 0m 51s the patch passed
        +1 compile 0m 47s the patch passed
        +1 javac 0m 47s the patch passed
        -0 checkstyle 0m 25s hadoop-hdfs-project/hadoop-hdfs: The patch generated 2 new + 75 unchanged - 0 fixed = 77 total (was 75)
        +1 mvnsite 0m 55s the patch passed
        +1 mvneclipse 0m 10s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 1m 56s the patch passed
        +1 javadoc 0m 54s the patch passed
        -1 unit 77m 54s hadoop-hdfs in the patch failed.
        +1 asflicense 0m 21s The patch does not generate ASF License warnings.
        98m 13s



        Reason Tests
        Failed junit tests hadoop.hdfs.server.namenode.TestEditLogAutoroll



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12818759/HDFS-10652.001.patch
        JIRA Issue HDFS-10652
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 684e66c49d72 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 92fe2db
        Default Java 1.8.0_91
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/16090/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/16090/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16090/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16090/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 22s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 7m 14s trunk passed +1 compile 0m 49s trunk passed +1 checkstyle 0m 27s trunk passed +1 mvnsite 0m 55s trunk passed +1 mvneclipse 0m 12s trunk passed +1 findbugs 1m 46s trunk passed +1 javadoc 0m 56s trunk passed +1 mvninstall 0m 51s the patch passed +1 compile 0m 47s the patch passed +1 javac 0m 47s the patch passed -0 checkstyle 0m 25s hadoop-hdfs-project/hadoop-hdfs: The patch generated 2 new + 75 unchanged - 0 fixed = 77 total (was 75) +1 mvnsite 0m 55s the patch passed +1 mvneclipse 0m 10s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 56s the patch passed +1 javadoc 0m 54s the patch passed -1 unit 77m 54s hadoop-hdfs in the patch failed. +1 asflicense 0m 21s The patch does not generate ASF License warnings. 98m 13s Reason Tests Failed junit tests hadoop.hdfs.server.namenode.TestEditLogAutoroll Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12818759/HDFS-10652.001.patch JIRA Issue HDFS-10652 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 684e66c49d72 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 92fe2db Default Java 1.8.0_91 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/16090/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/16090/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16090/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16090/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        yzhangal Yongjun Zhang added a comment -

        HI Vinayakumar B,

        The test needs the following code to throw exception, but it's not guaranteed,

                    int count = 0;
                    while (count < 5) {
                      if (replicaInfo.getBytesAcked() < replicaInfo.getBytesOnDisk()) {
                        failed.set(true);
                        throw new IOException("Failing Pipeline " + replicaInfo
                            .getBytesAcked() + " : " + replicaInfo.getBytesOnDisk());
                      }
                      try {
                        Thread.sleep(200);
                      } catch (InterruptedException e) {
                      }
                      count++;
                    }
        

        Maybe we can make count a little larger, and fail the test if the exception is not generated after trying count times.

        Thanks.

        Show
        yzhangal Yongjun Zhang added a comment - HI Vinayakumar B , The test needs the following code to throw exception, but it's not guaranteed, int count = 0; while (count < 5) { if (replicaInfo.getBytesAcked() < replicaInfo.getBytesOnDisk()) { failed.set( true ); throw new IOException( "Failing Pipeline " + replicaInfo .getBytesAcked() + " : " + replicaInfo.getBytesOnDisk()); } try { Thread .sleep(200); } catch (InterruptedException e) { } count++; } Maybe we can make count a little larger, and fail the test if the exception is not generated after trying count times. Thanks.
        Hide
        vinayrpet Vinayakumar B added a comment -

        Updated the patch.
        Please review

        Show
        vinayrpet Vinayakumar B added a comment - Updated the patch. Please review
        Hide
        yzhangal Yongjun Zhang added a comment - - edited

        Thanks a lot for the updated rev Vinayakumar B!

        I'm uploading an updated rev (003) for ease for commenting:

        1, In the patch, we inject an error when detecting the condition (ackSize % 512 > 0 && ackSize < diskSize) at the second DN, the third DN is thrown away and we add a new DN. In pipeline recovery, when we do block transfer to the new DN, the copy source may be the first or the second DN. It's not deterministic in this test. I did quite a few rounds of test, and saw that using either one as a source seems fine, both nodes have the same ackSize and diskSize satisfying (ackSize % 512 > 0 && ackSize < diskSize), so this is good.

        However, I'd like to point this out, since in the real case that we examined, the copy source is the DN that satisfies the above mentioned condition.

        2. I replaced the hardcoded constants with constants, as

            final int CHUNK_SIZE = 512;
            final int ONE_WRITE_SIZE = 5000;
            final int TOTAL_SIZE = 2 * 1024 * 1024;
            final int ERROR_INJECTION_LOC = TOTAL_SIZE / 2;
        

        I thought the TOTAL_SIZE doesn't have to be 2M, and ERROR_INJECTION_LOC doesn't have to be half the total size, so I did the following change

            final int CHUNK_SIZE = 512;
            final int ONE_WRITE_SIZE = 5000;
            final int TOTAL_SIZE = 1024 * 1024;
            final int ERROR_INJECTION_LOC = 512;
        

        and that does work too.

        Would you please take a look?

        Thanks.

        Show
        yzhangal Yongjun Zhang added a comment - - edited Thanks a lot for the updated rev Vinayakumar B ! I'm uploading an updated rev (003) for ease for commenting: 1, In the patch, we inject an error when detecting the condition (ackSize % 512 > 0 && ackSize < diskSize) at the second DN, the third DN is thrown away and we add a new DN. In pipeline recovery, when we do block transfer to the new DN, the copy source may be the first or the second DN. It's not deterministic in this test. I did quite a few rounds of test, and saw that using either one as a source seems fine, both nodes have the same ackSize and diskSize satisfying (ackSize % 512 > 0 && ackSize < diskSize), so this is good. However, I'd like to point this out, since in the real case that we examined, the copy source is the DN that satisfies the above mentioned condition. 2. I replaced the hardcoded constants with constants, as final int CHUNK_SIZE = 512; final int ONE_WRITE_SIZE = 5000; final int TOTAL_SIZE = 2 * 1024 * 1024; final int ERROR_INJECTION_LOC = TOTAL_SIZE / 2; I thought the TOTAL_SIZE doesn't have to be 2M, and ERROR_INJECTION_LOC doesn't have to be half the total size, so I did the following change final int CHUNK_SIZE = 512; final int ONE_WRITE_SIZE = 5000; final int TOTAL_SIZE = 1024 * 1024; final int ERROR_INJECTION_LOC = 512; and that does work too. Would you please take a look? Thanks.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 29s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 9m 34s trunk passed
        +1 compile 0m 56s trunk passed
        +1 checkstyle 0m 40s trunk passed
        +1 mvnsite 1m 7s trunk passed
        +1 mvneclipse 0m 13s trunk passed
        +1 findbugs 1m 57s trunk passed
        +1 javadoc 1m 2s trunk passed
        +1 mvninstall 0m 59s the patch passed
        +1 compile 0m 52s the patch passed
        +1 javac 0m 52s the patch passed
        -0 checkstyle 0m 32s hadoop-hdfs-project/hadoop-hdfs: The patch generated 5 new + 365 unchanged - 0 fixed = 370 total (was 365)
        +1 mvnsite 1m 2s the patch passed
        +1 mvneclipse 0m 11s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 2m 1s the patch passed
        +1 javadoc 1m 0s the patch passed
        -1 unit 81m 18s hadoop-hdfs in the patch failed.
        +1 asflicense 0m 21s The patch does not generate ASF License warnings.
        105m 40s



        Reason Tests
        Failed junit tests hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure
          hadoop.hdfs.server.namenode.snapshot.TestSnapshotFileLength
          hadoop.metrics2.sink.TestRollingFileSystemSinkWithHdfs
        Timed out junit tests org.apache.hadoop.hdfs.TestLeaseRecovery2



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12819223/HDFS-10652.003.patch
        JIRA Issue HDFS-10652
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 59ed5ff39cd8 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 521f343
        Default Java 1.8.0_91
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/16130/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/16130/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16130/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16130/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 29s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 9m 34s trunk passed +1 compile 0m 56s trunk passed +1 checkstyle 0m 40s trunk passed +1 mvnsite 1m 7s trunk passed +1 mvneclipse 0m 13s trunk passed +1 findbugs 1m 57s trunk passed +1 javadoc 1m 2s trunk passed +1 mvninstall 0m 59s the patch passed +1 compile 0m 52s the patch passed +1 javac 0m 52s the patch passed -0 checkstyle 0m 32s hadoop-hdfs-project/hadoop-hdfs: The patch generated 5 new + 365 unchanged - 0 fixed = 370 total (was 365) +1 mvnsite 1m 2s the patch passed +1 mvneclipse 0m 11s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 2m 1s the patch passed +1 javadoc 1m 0s the patch passed -1 unit 81m 18s hadoop-hdfs in the patch failed. +1 asflicense 0m 21s The patch does not generate ASF License warnings. 105m 40s Reason Tests Failed junit tests hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure   hadoop.hdfs.server.namenode.snapshot.TestSnapshotFileLength   hadoop.metrics2.sink.TestRollingFileSystemSinkWithHdfs Timed out junit tests org.apache.hadoop.hdfs.TestLeaseRecovery2 Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12819223/HDFS-10652.003.patch JIRA Issue HDFS-10652 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 59ed5ff39cd8 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 521f343 Default Java 1.8.0_91 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/16130/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/16130/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16130/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16130/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        vinayrpet Vinayakumar B added a comment -

        v3 patch was looking good.
        Just updated the same with simple checkstyle fixes.

        Show
        vinayrpet Vinayakumar B added a comment - v3 patch was looking good. Just updated the same with simple checkstyle fixes.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 21s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 6m 58s trunk passed
        +1 compile 0m 47s trunk passed
        +1 checkstyle 0m 31s trunk passed
        +1 mvnsite 0m 53s trunk passed
        +1 mvneclipse 0m 12s trunk passed
        +1 findbugs 1m 44s trunk passed
        +1 javadoc 0m 56s trunk passed
        +1 mvninstall 0m 49s the patch passed
        +1 compile 0m 43s the patch passed
        +1 javac 0m 43s the patch passed
        +1 checkstyle 0m 28s the patch passed
        +1 mvnsite 0m 51s the patch passed
        +1 mvneclipse 0m 10s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 1m 49s the patch passed
        +1 javadoc 0m 55s the patch passed
        -1 unit 61m 57s hadoop-hdfs in the patch failed.
        +1 asflicense 0m 17s The patch does not generate ASF License warnings.
        81m 37s



        Reason Tests
        Failed junit tests hadoop.hdfs.server.balancer.TestBalancer



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12819273/HDFS-10652.004.patch
        JIRA Issue HDFS-10652
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 3f404cfa685f 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 557a245
        Default Java 1.8.0_91
        findbugs v3.0.0
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/16135/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16135/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16135/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 21s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 6m 58s trunk passed +1 compile 0m 47s trunk passed +1 checkstyle 0m 31s trunk passed +1 mvnsite 0m 53s trunk passed +1 mvneclipse 0m 12s trunk passed +1 findbugs 1m 44s trunk passed +1 javadoc 0m 56s trunk passed +1 mvninstall 0m 49s the patch passed +1 compile 0m 43s the patch passed +1 javac 0m 43s the patch passed +1 checkstyle 0m 28s the patch passed +1 mvnsite 0m 51s the patch passed +1 mvneclipse 0m 10s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 49s the patch passed +1 javadoc 0m 55s the patch passed -1 unit 61m 57s hadoop-hdfs in the patch failed. +1 asflicense 0m 17s The patch does not generate ASF License warnings. 81m 37s Reason Tests Failed junit tests hadoop.hdfs.server.balancer.TestBalancer Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12819273/HDFS-10652.004.patch JIRA Issue HDFS-10652 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 3f404cfa685f 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 557a245 Default Java 1.8.0_91 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/16135/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16135/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16135/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        yzhangal Yongjun Zhang added a comment -

        Hi Vinayakumar B,

        Thanks for the new rev. I have two comments:

        1. Improved msg printing in Datanode.java and FsDatasetImpl.java, where I added logging of current datanode.
        2. I change the condition replicaInfo.getBytesAcked() < replicaInfo.getBytesOnDisk() to

                  if ((replicaInfo.getBytesOnDisk() / chunkSize) -
                      (replicaInfo.getBytesAcked() / chunkSize) >= 1) {
        

        to make sure that not only the bytesAcked is smaller than the butesOnDisk, but the latter has to be equal to or bigger than the chunk end.

        Uploaded rev 005 to reflect these comments.

        BTW, I still saw a little flakiness, probably one out of 10 times, I would see that the test failed, reason is:

        unning org.apache.hadoop.hdfs.TestClientProtocolForPipelineRecovery
        Tests run: 9, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 79.142 sec <<< FAILURE! - in org.apache.hadoop.hdfs.TestClientProtocolForPipelineRecovery
        testPipelineRecoveryWithTransferBlock(org.apache.hadoop.hdfs.TestClientProtocolForPipelineRecovery)  Time elapsed: 21.456 sec  <<< ERROR!
        org.apache.hadoop.hdfs.BlockMissingException: Could not obtain block: BP-537180841-127.0.0.1-1469127392654:blk_1073741825_1002 file=/f
        	at org.apache.hadoop.hdfs.DFSInputStream.chooseDataNode(DFSInputStream.java:1039)
        	at org.apache.hadoop.hdfs.DFSInputStream.blockSeekTo(DFSInputStream.java:654)
        	at org.apache.hadoop.hdfs.DFSInputStream.readWithStrategy(DFSInputStream.java:946)
        	at org.apache.hadoop.hdfs.DFSInputStream.read(DFSInputStream.java:1001)
        	at java.io.DataInputStream.read(DataInputStream.java:100)
        	at org.apache.hadoop.io.IOUtils.copyBytes(IOUtils.java:88)
        	at org.apache.hadoop.io.IOUtils.copyBytes(IOUtils.java:62)
        	at org.apache.hadoop.hdfs.DFSTestUtil.readFileBuffer(DFSTestUtil.java:354)
        	at org.apache.hadoop.hdfs.DFSTestUtil.readFile(DFSTestUtil.java:346)
        	at org.apache.hadoop.hdfs.TestClientProtocolForPipelineRecovery.testPipelineRecoveryWithTransferBlock(TestClientProtocolForPipelineRecovery.java:588)
        

        Thanks.

        Show
        yzhangal Yongjun Zhang added a comment - Hi Vinayakumar B , Thanks for the new rev. I have two comments: 1. Improved msg printing in Datanode.java and FsDatasetImpl.java, where I added logging of current datanode. 2. I change the condition replicaInfo.getBytesAcked() < replicaInfo.getBytesOnDisk() to if ((replicaInfo.getBytesOnDisk() / chunkSize) - (replicaInfo.getBytesAcked() / chunkSize) >= 1) { to make sure that not only the bytesAcked is smaller than the butesOnDisk, but the latter has to be equal to or bigger than the chunk end. Uploaded rev 005 to reflect these comments. BTW, I still saw a little flakiness, probably one out of 10 times, I would see that the test failed, reason is: unning org.apache.hadoop.hdfs.TestClientProtocolForPipelineRecovery Tests run: 9, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 79.142 sec <<< FAILURE! - in org.apache.hadoop.hdfs.TestClientProtocolForPipelineRecovery testPipelineRecoveryWithTransferBlock(org.apache.hadoop.hdfs.TestClientProtocolForPipelineRecovery) Time elapsed: 21.456 sec <<< ERROR! org.apache.hadoop.hdfs.BlockMissingException: Could not obtain block: BP-537180841-127.0.0.1-1469127392654:blk_1073741825_1002 file=/f at org.apache.hadoop.hdfs.DFSInputStream.chooseDataNode(DFSInputStream.java:1039) at org.apache.hadoop.hdfs.DFSInputStream.blockSeekTo(DFSInputStream.java:654) at org.apache.hadoop.hdfs.DFSInputStream.readWithStrategy(DFSInputStream.java:946) at org.apache.hadoop.hdfs.DFSInputStream.read(DFSInputStream.java:1001) at java.io.DataInputStream.read(DataInputStream.java:100) at org.apache.hadoop.io.IOUtils.copyBytes(IOUtils.java:88) at org.apache.hadoop.io.IOUtils.copyBytes(IOUtils.java:62) at org.apache.hadoop.hdfs.DFSTestUtil.readFileBuffer(DFSTestUtil.java:354) at org.apache.hadoop.hdfs.DFSTestUtil.readFile(DFSTestUtil.java:346) at org.apache.hadoop.hdfs.TestClientProtocolForPipelineRecovery.testPipelineRecoveryWithTransferBlock(TestClientProtocolForPipelineRecovery.java:588) Thanks.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 17s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 12m 26s trunk passed
        +1 compile 0m 45s trunk passed
        +1 checkstyle 0m 32s trunk passed
        +1 mvnsite 0m 54s trunk passed
        +1 mvneclipse 0m 16s trunk passed
        +1 findbugs 1m 50s trunk passed
        +1 javadoc 0m 57s trunk passed
        +1 mvninstall 0m 47s the patch passed
        +1 compile 0m 42s the patch passed
        +1 javac 0m 42s the patch passed
        +1 checkstyle 0m 28s the patch passed
        +1 mvnsite 0m 52s the patch passed
        +1 mvneclipse 0m 9s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 1m 50s the patch passed
        +1 javadoc 0m 53s the patch passed
        -1 unit 68m 44s hadoop-hdfs in the patch failed.
        +1 asflicense 0m 19s The patch does not generate ASF License warnings.
        93m 56s



        Reason Tests
        Timed out junit tests org.apache.hadoop.hdfs.TestLeaseRecovery2



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12819562/HDFS-10652.005.patch
        JIRA Issue HDFS-10652
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 12033d555fe1 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 132deb4
        Default Java 1.8.0_91
        findbugs v3.0.0
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/16156/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16156/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16156/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 17s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 12m 26s trunk passed +1 compile 0m 45s trunk passed +1 checkstyle 0m 32s trunk passed +1 mvnsite 0m 54s trunk passed +1 mvneclipse 0m 16s trunk passed +1 findbugs 1m 50s trunk passed +1 javadoc 0m 57s trunk passed +1 mvninstall 0m 47s the patch passed +1 compile 0m 42s the patch passed +1 javac 0m 42s the patch passed +1 checkstyle 0m 28s the patch passed +1 mvnsite 0m 52s the patch passed +1 mvneclipse 0m 9s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 50s the patch passed +1 javadoc 0m 53s the patch passed -1 unit 68m 44s hadoop-hdfs in the patch failed. +1 asflicense 0m 19s The patch does not generate ASF License warnings. 93m 56s Reason Tests Timed out junit tests org.apache.hadoop.hdfs.TestLeaseRecovery2 Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12819562/HDFS-10652.005.patch JIRA Issue HDFS-10652 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 12033d555fe1 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 132deb4 Default Java 1.8.0_91 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/16156/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16156/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16156/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        vinayrpet Vinayakumar B added a comment -

        I change the condition replicaInfo.getBytesAcked() < replicaInfo.getBytesOnDisk() to make sure that not only the bytesAcked is smaller than the butesOnDisk, but the latter has to be equal to or bigger than the chunk end

        Since each time flushed content is 5000 bytes which is way more than chunk, bytesOnDisk would have entire last packet (~5000bytes) flushed to disk, when that exception thrown.
        Anyway, change is fine though. Making it clear.

        BTW, I still saw a little flakiness, probably one out of 10 times, I would see that the test failed,

        Can you post the log output whenever you see the failure? This could be due to shutting down wrong datanode?

        Show
        vinayrpet Vinayakumar B added a comment - I change the condition replicaInfo.getBytesAcked() < replicaInfo.getBytesOnDisk() to make sure that not only the bytesAcked is smaller than the butesOnDisk, but the latter has to be equal to or bigger than the chunk end Since each time flushed content is 5000 bytes which is way more than chunk, bytesOnDisk would have entire last packet (~5000bytes) flushed to disk, when that exception thrown. Anyway, change is fine though. Making it clear. BTW, I still saw a little flakiness, probably one out of 10 times, I would see that the test failed, Can you post the log output whenever you see the failure? This could be due to shutting down wrong datanode?
        Hide
        yzhangal Yongjun Zhang added a comment -

        Thanks Vinayakumar B, I will do some more runs and provide logs asap, early next week.

        Show
        yzhangal Yongjun Zhang added a comment - Thanks Vinayakumar B , I will do some more runs and provide logs asap, early next week.
        Hide
        yzhangal Yongjun Zhang added a comment -

        Hi Vinayakumar B,

        Sorry for the delay. I finally got to look at this case again.

        I found that it's not that we are shutting down wrong DNs. It's because the block report of the new DN did not make to NN when we do the read, from time to time. So I introduced some new code to trigger block report instead of

        DFSTestUtil.waitForReplication(fs, fileName, (short)3, 2000);
        

        which doesn't seem to guarantee block report is received by NN.

        With this change, the test appears to fail stably with checksum error when without the fix of HDFS-4660/HDFS-9220, and succeed stably with the fixes.

        Would you please take a look?

        Thanks.

        Show
        yzhangal Yongjun Zhang added a comment - Hi Vinayakumar B , Sorry for the delay. I finally got to look at this case again. I found that it's not that we are shutting down wrong DNs. It's because the block report of the new DN did not make to NN when we do the read, from time to time. So I introduced some new code to trigger block report instead of DFSTestUtil.waitForReplication(fs, fileName, ( short )3, 2000); which doesn't seem to guarantee block report is received by NN. With this change, the test appears to fail stably with checksum error when without the fix of HDFS-4660 / HDFS-9220 , and succeed stably with the fixes. Would you please take a look? Thanks.
        Hide
        hadoopqa Hadoop QA added a comment -
        +1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 19s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 7m 54s trunk passed
        +1 compile 0m 54s trunk passed
        +1 checkstyle 0m 32s trunk passed
        +1 mvnsite 0m 58s trunk passed
        +1 mvneclipse 0m 14s trunk passed
        +1 findbugs 2m 0s trunk passed
        +1 javadoc 1m 0s trunk passed
        +1 mvninstall 0m 59s the patch passed
        +1 compile 0m 54s the patch passed
        +1 javac 0m 54s the patch passed
        +1 checkstyle 0m 30s the patch passed
        +1 mvnsite 1m 0s the patch passed
        +1 mvneclipse 0m 11s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 2m 3s the patch passed
        +1 javadoc 0m 58s the patch passed
        +1 unit 65m 3s hadoop-hdfs in the patch passed.
        +1 asflicense 0m 17s The patch does not generate ASF License warnings.
        87m 10s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12825348/HDFS-10652.006.patch
        JIRA Issue HDFS-10652
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux c3aa9c71f181 3.13.0-92-generic #139-Ubuntu SMP Tue Jun 28 20:42:26 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / a1f3293
        Default Java 1.8.0_101
        findbugs v3.0.0
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16531/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16531/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 19s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 7m 54s trunk passed +1 compile 0m 54s trunk passed +1 checkstyle 0m 32s trunk passed +1 mvnsite 0m 58s trunk passed +1 mvneclipse 0m 14s trunk passed +1 findbugs 2m 0s trunk passed +1 javadoc 1m 0s trunk passed +1 mvninstall 0m 59s the patch passed +1 compile 0m 54s the patch passed +1 javac 0m 54s the patch passed +1 checkstyle 0m 30s the patch passed +1 mvnsite 1m 0s the patch passed +1 mvneclipse 0m 11s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 2m 3s the patch passed +1 javadoc 0m 58s the patch passed +1 unit 65m 3s hadoop-hdfs in the patch passed. +1 asflicense 0m 17s The patch does not generate ASF License warnings. 87m 10s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12825348/HDFS-10652.006.patch JIRA Issue HDFS-10652 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux c3aa9c71f181 3.13.0-92-generic #139-Ubuntu SMP Tue Jun 28 20:42:26 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / a1f3293 Default Java 1.8.0_101 findbugs v3.0.0 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16531/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16531/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        jojochuang Wei-Chiu Chuang added a comment -

        Thanks Yongjun Zhang that's great news!
        I had a quick glance at the patch and looks good overall. One comment: is it necessary to change the log message in FsDatasetImpl? I get that it helps debugging in the unit test, but doesn't seem to be needed for code. Similarly there's another log message change in DataNode that seems not necessary.

        Also I think if you can extract this

        if ((replicaInfo.getBytesOnDisk() / chunkSize) -
                          (replicaInfo.getBytesAcked() / chunkSize) >= 1)
        

        to a small wrapper method, it helps understand the condition it tests.

        Show
        jojochuang Wei-Chiu Chuang added a comment - Thanks Yongjun Zhang that's great news! I had a quick glance at the patch and looks good overall. One comment: is it necessary to change the log message in FsDatasetImpl ? I get that it helps debugging in the unit test, but doesn't seem to be needed for code. Similarly there's another log message change in DataNode that seems not necessary. Also I think if you can extract this if ((replicaInfo.getBytesOnDisk() / chunkSize) - (replicaInfo.getBytesAcked() / chunkSize) >= 1) to a small wrapper method, it helps understand the condition it tests.
        Hide
        yzhangal Yongjun Zhang added a comment - - edited

        Thanks Wei-Chiu Chuang for reviewing.

        I on-purposely added those messages. My thinking is, it's worthwhile since it helps debugging unit test, and it may help adding clarity in real cluster too. Do you agree?

        I will address your other comment soon.

        Thanks.

        Show
        yzhangal Yongjun Zhang added a comment - - edited Thanks Wei-Chiu Chuang for reviewing. I on-purposely added those messages. My thinking is, it's worthwhile since it helps debugging unit test, and it may help adding clarity in real cluster too. Do you agree? I will address your other comment soon. Thanks.
        Hide
        yzhangal Yongjun Zhang added a comment -

        Added comment instead to address the second comment from Wei-Chiu, and uploaded rev 007.

        Thanks for taking further look Wei-Chiu Chuang and Vinayakumar B.

        Show
        yzhangal Yongjun Zhang added a comment - Added comment instead to address the second comment from Wei-Chiu, and uploaded rev 007. Thanks for taking further look Wei-Chiu Chuang and Vinayakumar B .
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 14s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 7m 21s trunk passed
        +1 compile 0m 43s trunk passed
        +1 checkstyle 0m 29s trunk passed
        +1 mvnsite 0m 51s trunk passed
        +1 mvneclipse 0m 12s trunk passed
        +1 findbugs 1m 41s trunk passed
        +1 javadoc 0m 54s trunk passed
        +1 mvninstall 0m 45s the patch passed
        +1 compile 0m 41s the patch passed
        +1 javac 0m 41s the patch passed
        +1 checkstyle 0m 27s the patch passed
        +1 mvnsite 0m 47s the patch passed
        +1 mvneclipse 0m 9s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 1m 46s the patch passed
        +1 javadoc 0m 52s the patch passed
        -1 unit 57m 2s hadoop-hdfs in the patch failed.
        +1 asflicense 0m 18s The patch does not generate ASF License warnings.
        76m 28s



        Reason Tests
        Failed junit tests hadoop.hdfs.TestDFSShell
          hadoop.metrics2.sink.TestRollingFileSystemSinkWithHdfs



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Issue HDFS-10652
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12825562/HDFS-10652.007.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 58bdef09db21 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18 21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 81485db
        Default Java 1.8.0_101
        findbugs v3.0.0
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/16544/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16544/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16544/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 14s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 7m 21s trunk passed +1 compile 0m 43s trunk passed +1 checkstyle 0m 29s trunk passed +1 mvnsite 0m 51s trunk passed +1 mvneclipse 0m 12s trunk passed +1 findbugs 1m 41s trunk passed +1 javadoc 0m 54s trunk passed +1 mvninstall 0m 45s the patch passed +1 compile 0m 41s the patch passed +1 javac 0m 41s the patch passed +1 checkstyle 0m 27s the patch passed +1 mvnsite 0m 47s the patch passed +1 mvneclipse 0m 9s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 46s the patch passed +1 javadoc 0m 52s the patch passed -1 unit 57m 2s hadoop-hdfs in the patch failed. +1 asflicense 0m 18s The patch does not generate ASF License warnings. 76m 28s Reason Tests Failed junit tests hadoop.hdfs.TestDFSShell   hadoop.metrics2.sink.TestRollingFileSystemSinkWithHdfs Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Issue HDFS-10652 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12825562/HDFS-10652.007.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 58bdef09db21 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18 21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 81485db Default Java 1.8.0_101 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/16544/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16544/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16544/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        vinayrpet Vinayakumar B added a comment -

        Thanks Yongjun Zhang for finding the problem in test.

        Updated the test looks good to me.
        I think it could be better to change System.out.println(..) change to LOG.info() in test.
        Rest all looks fine.
        +1 once addressed.

        Sorry I am currently in travel and not able to update the patch myself. Thanks for taking care of this.

        Show
        vinayrpet Vinayakumar B added a comment - Thanks Yongjun Zhang for finding the problem in test. Updated the test looks good to me. I think it could be better to change System.out.println(..) change to LOG.info() in test. Rest all looks fine. +1 once addressed. Sorry I am currently in travel and not able to update the patch myself. Thanks for taking care of this.
        Hide
        yzhangal Yongjun Zhang added a comment -

        Thanks Vinayakumar B, uploaded rev 008 to address your logging comment.

        Show
        yzhangal Yongjun Zhang added a comment - Thanks Vinayakumar B , uploaded rev 008 to address your logging comment.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 9m 52s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 7m 50s trunk passed
        +1 compile 0m 50s trunk passed
        +1 checkstyle 0m 31s trunk passed
        +1 mvnsite 1m 2s trunk passed
        +1 mvneclipse 0m 13s trunk passed
        +1 findbugs 1m 46s trunk passed
        +1 javadoc 0m 59s trunk passed
        +1 mvninstall 0m 52s the patch passed
        +1 compile 0m 47s the patch passed
        +1 javac 0m 47s the patch passed
        +1 checkstyle 0m 28s the patch passed
        +1 mvnsite 0m 58s the patch passed
        +1 mvneclipse 0m 10s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 1m 53s the patch passed
        +1 javadoc 0m 55s the patch passed
        -1 unit 63m 29s hadoop-hdfs in the patch failed.
        +1 asflicense 0m 20s The patch does not generate ASF License warnings.
        94m 10s



        Reason Tests
        Failed junit tests hadoop.hdfs.server.blockmanagement.TestRBWBlockInvalidation



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Issue HDFS-10652
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12825719/HDFS-10652.008.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux c9bd48ce85f2 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 407b519
        Default Java 1.8.0_101
        findbugs v3.0.0
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/16553/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16553/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16553/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 9m 52s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 7m 50s trunk passed +1 compile 0m 50s trunk passed +1 checkstyle 0m 31s trunk passed +1 mvnsite 1m 2s trunk passed +1 mvneclipse 0m 13s trunk passed +1 findbugs 1m 46s trunk passed +1 javadoc 0m 59s trunk passed +1 mvninstall 0m 52s the patch passed +1 compile 0m 47s the patch passed +1 javac 0m 47s the patch passed +1 checkstyle 0m 28s the patch passed +1 mvnsite 0m 58s the patch passed +1 mvneclipse 0m 10s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 53s the patch passed +1 javadoc 0m 55s the patch passed -1 unit 63m 29s hadoop-hdfs in the patch failed. +1 asflicense 0m 20s The patch does not generate ASF License warnings. 94m 10s Reason Tests Failed junit tests hadoop.hdfs.server.blockmanagement.TestRBWBlockInvalidation Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Issue HDFS-10652 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12825719/HDFS-10652.008.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux c9bd48ce85f2 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 407b519 Default Java 1.8.0_101 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/16553/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16553/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16553/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        yzhangal Yongjun Zhang added a comment -

        The failed test appears to be unrelated and intermittent, I created HDFS-10811 and HDFS-10812 for them.

        Thanks Vinayakumar B for the earlier +1 (message logging is addressed in latest rev), I will commit soon.

        Show
        yzhangal Yongjun Zhang added a comment - The failed test appears to be unrelated and intermittent, I created HDFS-10811 and HDFS-10812 for them. Thanks Vinayakumar B for the earlier +1 (message logging is addressed in latest rev), I will commit soon.
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10363 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10363/)
        HDFS-10652. Add a unit test for HDFS-4660. Contributed by Vinayakumar (yzhang: rev c25817159af17753b398956cfe6ff14984801b01)

        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNodeFaultInjector.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestClientProtocolForPipelineRecovery.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockReceiver.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10363 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10363/ ) HDFS-10652 . Add a unit test for HDFS-4660 . Contributed by Vinayakumar (yzhang: rev c25817159af17753b398956cfe6ff14984801b01) (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNodeFaultInjector.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestClientProtocolForPipelineRecovery.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockReceiver.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
        Hide
        yzhangal Yongjun Zhang added a comment -

        Committed to trunk, branch-2, branch-2.8.

        Thanks Vinayakumar B and Wei-Chiu Chuang.

        Show
        yzhangal Yongjun Zhang added a comment - Committed to trunk, branch-2, branch-2.8. Thanks Vinayakumar B and Wei-Chiu Chuang .

          People

          • Assignee:
            vinayrpet Vinayakumar B
            Reporter:
            yzhangal Yongjun Zhang
          • Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development