Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-10913

Introduce fault injectors to simulate slow mirrors

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.9.0, 3.0.0-alpha2
    • Component/s: datanode, test
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      BlockReceiver#datanodeSlowLogThresholdMs is used as threshold to detect slow mirrors. BlockReceiver only writes some warning logs. In order to better test behaviors of slow mirrors, it necessitates introducing fault injectors.

      1. HDFS-10913.012.patch
        11 kB
        Xiaobing Zhou
      2. HDFS-10913.011.patch
        11 kB
        Xiaobing Zhou
      3. HDFS-10913.010.patch
        15 kB
        Xiaobing Zhou
      4. HDFS-10913.009.patch
        14 kB
        Xiaobing Zhou
      5. HDFS-10913.008.patch
        14 kB
        Xiaobing Zhou
      6. HDFS-10913.007.patch
        8 kB
        Xiaobing Zhou
      7. HDFS-10913.006.patch
        20 kB
        Xiaobing Zhou
      8. HDFS-10913.005.patch
        16 kB
        Xiaobing Zhou
      9. HDFS-10913.004.patch
        14 kB
        Xiaobing Zhou
      10. HDFS-10913.003.patch
        9 kB
        Xiaobing Zhou
      11. HDFS-10913.002.patch
        9 kB
        Xiaobing Zhou
      12. HDFS-10913.001.patch
        7 kB
        Xiaobing Zhou
      13. HDFS-10913.000.patch
        7 kB
        Xiaobing Zhou

        Activity

        Hide
        jojochuang Wei-Chiu Chuang added a comment -

        Thanks for the proposal. I think this is more like an improvement jira. Please revert if you think it's not. Thanks again.

        Show
        jojochuang Wei-Chiu Chuang added a comment - Thanks for the proposal. I think this is more like an improvement jira. Please revert if you think it's not. Thanks again.
        Hide
        xiaobingo Xiaobing Zhou added a comment -

        Wei-Chiu Chuang thanks for the correction.

        Show
        xiaobingo Xiaobing Zhou added a comment - Wei-Chiu Chuang thanks for the correction.
        Hide
        xiaobingo Xiaobing Zhou added a comment -

        I posted patch v000, please kindly review.

        Show
        xiaobingo Xiaobing Zhou added a comment - I posted patch v000, please kindly review.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 17s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 8m 22s trunk passed
        +1 compile 0m 49s trunk passed
        +1 checkstyle 0m 27s trunk passed
        +1 mvnsite 0m 51s trunk passed
        +1 mvneclipse 0m 12s trunk passed
        +1 findbugs 1m 42s trunk passed
        +1 javadoc 0m 56s trunk passed
        +1 mvninstall 0m 51s the patch passed
        +1 compile 0m 55s the patch passed
        +1 javac 0m 55s the patch passed
        -0 checkstyle 0m 24s hadoop-hdfs-project/hadoop-hdfs: The patch generated 5 new + 74 unchanged - 1 fixed = 79 total (was 75)
        +1 mvnsite 0m 51s the patch passed
        +1 mvneclipse 0m 9s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 1m 53s the patch passed
        +1 javadoc 0m 54s the patch passed
        -1 unit 63m 46s hadoop-hdfs in the patch failed.
        +1 asflicense 0m 18s The patch does not generate ASF License warnings.
        84m 57s



        Reason Tests
        Failed junit tests hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure
          hadoop.hdfs.server.datanode.TestDataNodeUUID
          hadoop.hdfs.TestSafeModeWithStripedFile



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Issue HDFS-10913
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12830596/HDFS-10913.000.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 74292d0fb7db 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 2acfb1e
        Default Java 1.8.0_101
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/16891/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/16891/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16891/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16891/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 17s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 8m 22s trunk passed +1 compile 0m 49s trunk passed +1 checkstyle 0m 27s trunk passed +1 mvnsite 0m 51s trunk passed +1 mvneclipse 0m 12s trunk passed +1 findbugs 1m 42s trunk passed +1 javadoc 0m 56s trunk passed +1 mvninstall 0m 51s the patch passed +1 compile 0m 55s the patch passed +1 javac 0m 55s the patch passed -0 checkstyle 0m 24s hadoop-hdfs-project/hadoop-hdfs: The patch generated 5 new + 74 unchanged - 1 fixed = 79 total (was 75) +1 mvnsite 0m 51s the patch passed +1 mvneclipse 0m 9s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 53s the patch passed +1 javadoc 0m 54s the patch passed -1 unit 63m 46s hadoop-hdfs in the patch failed. +1 asflicense 0m 18s The patch does not generate ASF License warnings. 84m 57s Reason Tests Failed junit tests hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure   hadoop.hdfs.server.datanode.TestDataNodeUUID   hadoop.hdfs.TestSafeModeWithStripedFile Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Issue HDFS-10913 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12830596/HDFS-10913.000.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 74292d0fb7db 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 2acfb1e Default Java 1.8.0_101 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/16891/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/16891/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16891/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16891/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        xiaobingo Xiaobing Zhou added a comment -

        v001 patch added brand new injector instance to avoid side effects from existing one.

        77	    final DataNodeFaultInjector dnInjector = new DataNodeFaultInjector();
        78	    DataNodeFaultInjector.set(dnInjector);
        
        Show
        xiaobingo Xiaobing Zhou added a comment - v001 patch added brand new injector instance to avoid side effects from existing one. 77 final DataNodeFaultInjector dnInjector = new DataNodeFaultInjector(); 78 DataNodeFaultInjector.set(dnInjector);
        Hide
        xyao Xiaoyu Yao added a comment -

        Thanks Xiaobing Zhou for reporting the issue and posting the patch for it. The patch looks good to me overall. I only have one question about the unit test:
        Can you override the new delay methods in DataNodeFaultInjector to include delay with verifications if possible. e.g.,

          final DataNodeFaultInjector dnInjector = new DataNodeFaultInjector() {
          @Override
            public void delayWritingDataToDisk() throws IOException {
              try { 
                 Thread.sleep(xxxx);    
              }catch (InterruptedException ie) {
                 throw new IOException("Interrupted while sleeping. Bailing out.");
              }
            }
          }
        
        Show
        xyao Xiaoyu Yao added a comment - Thanks Xiaobing Zhou for reporting the issue and posting the patch for it. The patch looks good to me overall. I only have one question about the unit test: Can you override the new delay methods in DataNodeFaultInjector to include delay with verifications if possible. e.g., final DataNodeFaultInjector dnInjector = new DataNodeFaultInjector() { @Override public void delayWritingDataToDisk() throws IOException { try { Thread .sleep(xxxx); } catch (InterruptedException ie) { throw new IOException( "Interrupted while sleeping. Bailing out." ); } } }
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 11s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 8m 25s trunk passed
        +1 compile 0m 46s trunk passed
        +1 checkstyle 0m 25s trunk passed
        +1 mvnsite 0m 52s trunk passed
        +1 mvneclipse 0m 12s trunk passed
        +1 findbugs 1m 43s trunk passed
        +1 javadoc 0m 53s trunk passed
        +1 mvninstall 0m 47s the patch passed
        +1 compile 0m 42s the patch passed
        +1 javac 0m 42s the patch passed
        -0 checkstyle 0m 23s hadoop-hdfs-project/hadoop-hdfs: The patch generated 5 new + 74 unchanged - 1 fixed = 79 total (was 75)
        +1 mvnsite 0m 51s the patch passed
        +1 mvneclipse 0m 11s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 1m 51s the patch passed
        +1 javadoc 0m 56s the patch passed
        -1 unit 76m 11s hadoop-hdfs in the patch failed.
        +1 asflicense 0m 17s The patch does not generate ASF License warnings.
        97m 2s



        Reason Tests
        Failed junit tests hadoop.hdfs.server.namenode.ha.TestEditLogTailer



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Issue HDFS-10913
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12830751/HDFS-10913.001.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux e3bf6e53ccf7 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / e19b37e
        Default Java 1.8.0_101
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/16913/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/16913/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16913/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16913/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 11s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 8m 25s trunk passed +1 compile 0m 46s trunk passed +1 checkstyle 0m 25s trunk passed +1 mvnsite 0m 52s trunk passed +1 mvneclipse 0m 12s trunk passed +1 findbugs 1m 43s trunk passed +1 javadoc 0m 53s trunk passed +1 mvninstall 0m 47s the patch passed +1 compile 0m 42s the patch passed +1 javac 0m 42s the patch passed -0 checkstyle 0m 23s hadoop-hdfs-project/hadoop-hdfs: The patch generated 5 new + 74 unchanged - 1 fixed = 79 total (was 75) +1 mvnsite 0m 51s the patch passed +1 mvneclipse 0m 11s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 51s the patch passed +1 javadoc 0m 56s the patch passed -1 unit 76m 11s hadoop-hdfs in the patch failed. +1 asflicense 0m 17s The patch does not generate ASF License warnings. 97m 2s Reason Tests Failed junit tests hadoop.hdfs.server.namenode.ha.TestEditLogTailer Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Issue HDFS-10913 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12830751/HDFS-10913.001.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux e3bf6e53ccf7 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / e19b37e Default Java 1.8.0_101 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/16913/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/16913/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16913/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16913/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        xiaobingo Xiaobing Zhou added a comment -

        Xiaoyu Yao thank you for reviews. I posted patch v002 with some enhanced tests.

        Show
        xiaobingo Xiaobing Zhou added a comment - Xiaoyu Yao thank you for reviews. I posted patch v002 with some enhanced tests.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 17s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 7m 40s trunk passed
        +1 compile 0m 51s trunk passed
        +1 checkstyle 0m 31s trunk passed
        +1 mvnsite 0m 59s trunk passed
        +1 mvneclipse 0m 13s trunk passed
        +1 findbugs 1m 49s trunk passed
        +1 javadoc 0m 55s trunk passed
        +1 mvninstall 0m 54s the patch passed
        +1 compile 0m 53s the patch passed
        +1 javac 0m 53s the patch passed
        -0 checkstyle 0m 24s hadoop-hdfs-project/hadoop-hdfs: The patch generated 3 new + 74 unchanged - 1 fixed = 77 total (was 75)
        +1 mvnsite 0m 58s the patch passed
        +1 mvneclipse 0m 11s the patch passed
        +1 whitespace 0m 1s The patch has no whitespace issues.
        +1 findbugs 2m 3s the patch passed
        +1 javadoc 0m 58s the patch passed
        -1 unit 64m 39s hadoop-hdfs in the patch failed.
        +1 asflicense 0m 20s The patch does not generate ASF License warnings.
        85m 59s



        Reason Tests
        Failed junit tests hadoop.hdfs.server.datanode.TestDataNodeUUID
          hadoop.hdfs.server.datanode.TestFsDatasetCache



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Issue HDFS-10913
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12830824/HDFS-10913.002.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 225ac8ffc967 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 47f8092
        Default Java 1.8.0_101
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/16917/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/16917/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16917/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16917/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 17s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 7m 40s trunk passed +1 compile 0m 51s trunk passed +1 checkstyle 0m 31s trunk passed +1 mvnsite 0m 59s trunk passed +1 mvneclipse 0m 13s trunk passed +1 findbugs 1m 49s trunk passed +1 javadoc 0m 55s trunk passed +1 mvninstall 0m 54s the patch passed +1 compile 0m 53s the patch passed +1 javac 0m 53s the patch passed -0 checkstyle 0m 24s hadoop-hdfs-project/hadoop-hdfs: The patch generated 3 new + 74 unchanged - 1 fixed = 77 total (was 75) +1 mvnsite 0m 58s the patch passed +1 mvneclipse 0m 11s the patch passed +1 whitespace 0m 1s The patch has no whitespace issues. +1 findbugs 2m 3s the patch passed +1 javadoc 0m 58s the patch passed -1 unit 64m 39s hadoop-hdfs in the patch failed. +1 asflicense 0m 20s The patch does not generate ASF License warnings. 85m 59s Reason Tests Failed junit tests hadoop.hdfs.server.datanode.TestDataNodeUUID   hadoop.hdfs.server.datanode.TestFsDatasetCache Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Issue HDFS-10913 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12830824/HDFS-10913.002.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 225ac8ffc967 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 47f8092 Default Java 1.8.0_101 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/16917/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/16917/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16917/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16917/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        xiaobingo Xiaobing Zhou added a comment -

        v003 fixed check style issues.

        Show
        xiaobingo Xiaobing Zhou added a comment - v003 fixed check style issues.
        Hide
        hadoopqa Hadoop QA added a comment -
        +1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 18s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 7m 0s trunk passed
        +1 compile 0m 45s trunk passed
        +1 checkstyle 0m 26s trunk passed
        +1 mvnsite 0m 51s trunk passed
        +1 mvneclipse 0m 12s trunk passed
        +1 findbugs 1m 42s trunk passed
        +1 javadoc 0m 55s trunk passed
        +1 mvninstall 0m 45s the patch passed
        +1 compile 0m 41s the patch passed
        +1 javac 0m 41s the patch passed
        -0 checkstyle 0m 24s hadoop-hdfs-project/hadoop-hdfs: The patch generated 1 new + 74 unchanged - 1 fixed = 75 total (was 75)
        +1 mvnsite 0m 47s the patch passed
        +1 mvneclipse 0m 10s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 1m 45s the patch passed
        +1 javadoc 0m 54s the patch passed
        +1 unit 58m 27s hadoop-hdfs in the patch passed.
        +1 asflicense 0m 19s The patch does not generate ASF License warnings.
        77m 31s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Issue HDFS-10913
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12830855/HDFS-10913.003.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 940bcdc53b63 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18 21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 47f8092
        Default Java 1.8.0_101
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/16920/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16920/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16920/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 18s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 7m 0s trunk passed +1 compile 0m 45s trunk passed +1 checkstyle 0m 26s trunk passed +1 mvnsite 0m 51s trunk passed +1 mvneclipse 0m 12s trunk passed +1 findbugs 1m 42s trunk passed +1 javadoc 0m 55s trunk passed +1 mvninstall 0m 45s the patch passed +1 compile 0m 41s the patch passed +1 javac 0m 41s the patch passed -0 checkstyle 0m 24s hadoop-hdfs-project/hadoop-hdfs: The patch generated 1 new + 74 unchanged - 1 fixed = 75 total (was 75) +1 mvnsite 0m 47s the patch passed +1 mvneclipse 0m 10s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 45s the patch passed +1 javadoc 0m 54s the patch passed +1 unit 58m 27s hadoop-hdfs in the patch passed. +1 asflicense 0m 19s The patch does not generate ASF License warnings. 77m 31s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Issue HDFS-10913 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12830855/HDFS-10913.003.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 940bcdc53b63 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18 21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 47f8092 Default Java 1.8.0_101 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/16920/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16920/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16920/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        xiaobingo Xiaobing Zhou added a comment -

        Posted v004 patch.

        1. added some log functions in DataNodeFaultInjector to make it easy to verify slow operations in terms of timing.
        2. added some extra tests to cover 4 scenarios of slow data nodes.

        I will to add test in next patch for delaySyncingWriterOsCache.

        Show
        xiaobingo Xiaobing Zhou added a comment - Posted v004 patch. added some log functions in DataNodeFaultInjector to make it easy to verify slow operations in terms of timing. added some extra tests to cover 4 scenarios of slow data nodes. I will to add test in next patch for delaySyncingWriterOsCache.
        Hide
        hadoopqa Hadoop QA added a comment -
        +1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 19s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 8m 49s trunk passed
        +1 compile 1m 2s trunk passed
        +1 checkstyle 0m 27s trunk passed
        +1 mvnsite 0m 54s trunk passed
        +1 mvneclipse 0m 13s trunk passed
        +1 findbugs 1m 44s trunk passed
        +1 javadoc 0m 42s trunk passed
        +1 mvninstall 0m 49s the patch passed
        +1 compile 0m 46s the patch passed
        +1 javac 0m 46s the patch passed
        -0 checkstyle 0m 24s hadoop-hdfs-project/hadoop-hdfs: The patch generated 3 new + 74 unchanged - 1 fixed = 77 total (was 75)
        +1 mvnsite 0m 53s the patch passed
        +1 mvneclipse 0m 12s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 1m 49s the patch passed
        +1 javadoc 0m 37s the patch passed
        +1 unit 92m 41s hadoop-hdfs in the patch passed.
        +1 asflicense 0m 18s The patch does not generate ASF License warnings.
        114m 4s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:a9ad5d6
        JIRA Issue HDFS-10913
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12840347/HDFS-10913.004.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 4a5ddf67c208 3.13.0-96-generic #143-Ubuntu SMP Mon Aug 29 20:15:20 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 0de0c32
        Default Java 1.8.0_111
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/17654/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17654/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17654/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 19s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 8m 49s trunk passed +1 compile 1m 2s trunk passed +1 checkstyle 0m 27s trunk passed +1 mvnsite 0m 54s trunk passed +1 mvneclipse 0m 13s trunk passed +1 findbugs 1m 44s trunk passed +1 javadoc 0m 42s trunk passed +1 mvninstall 0m 49s the patch passed +1 compile 0m 46s the patch passed +1 javac 0m 46s the patch passed -0 checkstyle 0m 24s hadoop-hdfs-project/hadoop-hdfs: The patch generated 3 new + 74 unchanged - 1 fixed = 77 total (was 75) +1 mvnsite 0m 53s the patch passed +1 mvneclipse 0m 12s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 49s the patch passed +1 javadoc 0m 37s the patch passed +1 unit 92m 41s hadoop-hdfs in the patch passed. +1 asflicense 0m 18s The patch does not generate ASF License warnings. 114m 4s Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HDFS-10913 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12840347/HDFS-10913.004.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 4a5ddf67c208 3.13.0-96-generic #143-Ubuntu SMP Mon Aug 29 20:15:20 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 0de0c32 Default Java 1.8.0_111 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/17654/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17654/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17654/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        arpitagarwal Arpit Agarwal added a comment -

        Thanks for this testability improvement Xiaobing Zhou. Few comments:

        1. delayWritingPacketToMirror looks unnecessary as we already have stopSendingPacketDownstream.
        2. Can you please add a hook for delayWritingChecksumToDisk also here?
                        checksumOut.write(buf);
          
        3. delaySyncingWriterOsCache should not introduce delay when syncBehindWritesInBackground is enabled. The delay should be in the else clause only in manageWriterOsCache:
                    if (syncBehindWritesInBackground) {
          ...
                    } else {
                      NativeIO.POSIX.syncFileRangeIfPossible(outFd,
          ...
                    }
          

        Also you may have to rebase the patch once HDFS-10930 is committed. Still reviewing the unit test.

        Show
        arpitagarwal Arpit Agarwal added a comment - Thanks for this testability improvement Xiaobing Zhou . Few comments: delayWritingPacketToMirror looks unnecessary as we already have stopSendingPacketDownstream. Can you please add a hook for delayWritingChecksumToDisk also here? checksumOut.write(buf); delaySyncingWriterOsCache should not introduce delay when syncBehindWritesInBackground is enabled. The delay should be in the else clause only in manageWriterOsCache: if (syncBehindWritesInBackground) { ... } else { NativeIO.POSIX.syncFileRangeIfPossible(outFd, ... } Also you may have to rebase the patch once HDFS-10930 is committed. Still reviewing the unit test.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 0s Docker mode activated.
        -1 patch 0m 5s HDFS-10913 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help.



        Subsystem Report/Notes
        JIRA Issue HDFS-10913
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12841087/HDFS-10913.005.patch
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17712/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. -1 patch 0m 5s HDFS-10913 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. Subsystem Report/Notes JIRA Issue HDFS-10913 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12841087/HDFS-10913.005.patch Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17712/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        xiaobingo Xiaobing Zhou added a comment -

        Posted v005 patch to address your comments, thanks Arpit Agarwal for reviews.

        Show
        xiaobingo Xiaobing Zhou added a comment - Posted v005 patch to address your comments, thanks Arpit Agarwal for reviews.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 23s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 7m 44s trunk passed
        +1 compile 0m 45s trunk passed
        +1 checkstyle 0m 26s trunk passed
        +1 mvnsite 0m 52s trunk passed
        +1 mvneclipse 0m 14s trunk passed
        -1 findbugs 1m 43s hadoop-hdfs-project/hadoop-hdfs in trunk has 1 extant Findbugs warnings.
        +1 javadoc 0m 39s trunk passed
        +1 mvninstall 0m 46s the patch passed
        +1 compile 0m 42s the patch passed
        +1 javac 0m 42s the patch passed
        -0 checkstyle 0m 24s hadoop-hdfs-project/hadoop-hdfs: The patch generated 3 new + 70 unchanged - 1 fixed = 73 total (was 71)
        +1 mvnsite 0m 49s the patch passed
        +1 mvneclipse 0m 10s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 1m 47s the patch passed
        +1 javadoc 0m 37s the patch passed
        -1 unit 93m 16s hadoop-hdfs in the patch failed.
        +1 asflicense 0m 33s The patch does not generate ASF License warnings.
        113m 4s



        Reason Tests
        Failed junit tests hadoop.hdfs.TestBlockStoragePolicy
          hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFS



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:a9ad5d6
        JIRA Issue HDFS-10913
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12841090/HDFS-10913.005.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux b74d6a632ce4 3.13.0-96-generic #143-Ubuntu SMP Mon Aug 29 20:15:20 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / be5a757
        Default Java 1.8.0_111
        findbugs v3.0.0
        findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/17713/artifact/patchprocess/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs-warnings.html
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/17713/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/17713/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17713/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17713/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 23s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 7m 44s trunk passed +1 compile 0m 45s trunk passed +1 checkstyle 0m 26s trunk passed +1 mvnsite 0m 52s trunk passed +1 mvneclipse 0m 14s trunk passed -1 findbugs 1m 43s hadoop-hdfs-project/hadoop-hdfs in trunk has 1 extant Findbugs warnings. +1 javadoc 0m 39s trunk passed +1 mvninstall 0m 46s the patch passed +1 compile 0m 42s the patch passed +1 javac 0m 42s the patch passed -0 checkstyle 0m 24s hadoop-hdfs-project/hadoop-hdfs: The patch generated 3 new + 70 unchanged - 1 fixed = 73 total (was 71) +1 mvnsite 0m 49s the patch passed +1 mvneclipse 0m 10s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 47s the patch passed +1 javadoc 0m 37s the patch passed -1 unit 93m 16s hadoop-hdfs in the patch failed. +1 asflicense 0m 33s The patch does not generate ASF License warnings. 113m 4s Reason Tests Failed junit tests hadoop.hdfs.TestBlockStoragePolicy   hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFS Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HDFS-10913 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12841090/HDFS-10913.005.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux b74d6a632ce4 3.13.0-96-generic #143-Ubuntu SMP Mon Aug 29 20:15:20 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / be5a757 Default Java 1.8.0_111 findbugs v3.0.0 findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/17713/artifact/patchprocess/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs-warnings.html checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/17713/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/17713/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17713/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17713/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        arpitagarwal Arpit Agarwal added a comment -

        The change looks good. I am reviewing the unit tests.

        Show
        arpitagarwal Arpit Agarwal added a comment - The change looks good. I am reviewing the unit tests.
        Hide
        xiaobingo Xiaobing Zhou added a comment -

        posted v006 patch.

        1. added separate test class TestDataNodeFaultInjector
        2. split the jumbo test into small tests
        Show
        xiaobingo Xiaobing Zhou added a comment - posted v006 patch. added separate test class TestDataNodeFaultInjector split the jumbo test into small tests
        Hide
        xiaobingo Xiaobing Zhou added a comment -

        v007 removed code that's not related to mirror operations.

        Show
        xiaobingo Xiaobing Zhou added a comment - v007 removed code that's not related to mirror operations.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 13s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 7m 48s trunk passed
        +1 compile 0m 49s trunk passed
        +1 checkstyle 0m 27s trunk passed
        +1 mvnsite 0m 59s trunk passed
        +1 mvneclipse 0m 13s trunk passed
        +1 findbugs 1m 47s trunk passed
        +1 javadoc 0m 40s trunk passed
        +1 mvninstall 0m 49s the patch passed
        +1 compile 0m 46s the patch passed
        +1 javac 0m 46s the patch passed
        -0 checkstyle 0m 25s hadoop-hdfs-project/hadoop-hdfs: The patch generated 5 new + 53 unchanged - 1 fixed = 58 total (was 54)
        +1 mvnsite 0m 53s the patch passed
        +1 mvneclipse 0m 11s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 1m 51s the patch passed
        +1 javadoc 0m 40s the patch passed
        -1 unit 73m 40s hadoop-hdfs in the patch failed.
        +1 asflicense 0m 21s The patch does not generate ASF License warnings.
        93m 44s



        Reason Tests
        Failed junit tests hadoop.hdfs.server.balancer.TestBalancer



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:a9ad5d6
        JIRA Issue HDFS-10913
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12843359/HDFS-10913.007.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux e8084fa70ca5 3.13.0-103-generic #150-Ubuntu SMP Thu Nov 24 10:34:17 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 64a2d5b
        Default Java 1.8.0_111
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/17863/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/17863/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17863/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17863/console
        Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 13s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 7m 48s trunk passed +1 compile 0m 49s trunk passed +1 checkstyle 0m 27s trunk passed +1 mvnsite 0m 59s trunk passed +1 mvneclipse 0m 13s trunk passed +1 findbugs 1m 47s trunk passed +1 javadoc 0m 40s trunk passed +1 mvninstall 0m 49s the patch passed +1 compile 0m 46s the patch passed +1 javac 0m 46s the patch passed -0 checkstyle 0m 25s hadoop-hdfs-project/hadoop-hdfs: The patch generated 5 new + 53 unchanged - 1 fixed = 58 total (was 54) +1 mvnsite 0m 53s the patch passed +1 mvneclipse 0m 11s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 51s the patch passed +1 javadoc 0m 40s the patch passed -1 unit 73m 40s hadoop-hdfs in the patch failed. +1 asflicense 0m 21s The patch does not generate ASF License warnings. 93m 44s Reason Tests Failed junit tests hadoop.hdfs.server.balancer.TestBalancer Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HDFS-10913 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12843359/HDFS-10913.007.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux e8084fa70ca5 3.13.0-103-generic #150-Ubuntu SMP Thu Nov 24 10:34:17 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 64a2d5b Default Java 1.8.0_111 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/17863/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/17863/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17863/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17863/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        xiaobingo Xiaobing Zhou added a comment -

        patch v008:

        1. added sending ack to upstream.
        2. added peer address to simulate slowing specific peer.
        3. fix check style issues.
        Show
        xiaobingo Xiaobing Zhou added a comment - patch v008: added sending ack to upstream. added peer address to simulate slowing specific peer. fix check style issues.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 23m 16s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 14m 27s trunk passed
        +1 compile 0m 56s trunk passed
        +1 checkstyle 0m 29s trunk passed
        +1 mvnsite 1m 1s trunk passed
        +1 mvneclipse 0m 14s trunk passed
        +1 findbugs 1m 56s trunk passed
        +1 javadoc 0m 43s trunk passed
        -1 mvninstall 0m 45s hadoop-hdfs in the patch failed.
        -1 compile 0m 41s hadoop-hdfs in the patch failed.
        -1 javac 0m 41s hadoop-hdfs in the patch failed.
        -0 checkstyle 0m 24s hadoop-hdfs-project/hadoop-hdfs: The patch generated 3 new + 111 unchanged - 1 fixed = 114 total (was 112)
        -1 mvnsite 0m 43s hadoop-hdfs in the patch failed.
        +1 mvneclipse 0m 10s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        -1 findbugs 0m 22s hadoop-hdfs in the patch failed.
        +1 javadoc 0m 38s the patch passed
        -1 unit 0m 43s hadoop-hdfs in the patch failed.
        +1 asflicense 0m 17s The patch does not generate ASF License warnings.
        48m 57s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:a9ad5d6
        JIRA Issue HDFS-10913
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12843456/HDFS-10913.008.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux ff8498321b93 3.13.0-96-generic #143-Ubuntu SMP Mon Aug 29 20:15:20 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 70ca1f1
        Default Java 1.8.0_111
        findbugs v3.0.0
        mvninstall https://builds.apache.org/job/PreCommit-HDFS-Build/17867/artifact/patchprocess/patch-mvninstall-hadoop-hdfs-project_hadoop-hdfs.txt
        compile https://builds.apache.org/job/PreCommit-HDFS-Build/17867/artifact/patchprocess/patch-compile-hadoop-hdfs-project_hadoop-hdfs.txt
        javac https://builds.apache.org/job/PreCommit-HDFS-Build/17867/artifact/patchprocess/patch-compile-hadoop-hdfs-project_hadoop-hdfs.txt
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/17867/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
        mvnsite https://builds.apache.org/job/PreCommit-HDFS-Build/17867/artifact/patchprocess/patch-mvnsite-hadoop-hdfs-project_hadoop-hdfs.txt
        findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/17867/artifact/patchprocess/patch-findbugs-hadoop-hdfs-project_hadoop-hdfs.txt
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/17867/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17867/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17867/console
        Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 23m 16s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 14m 27s trunk passed +1 compile 0m 56s trunk passed +1 checkstyle 0m 29s trunk passed +1 mvnsite 1m 1s trunk passed +1 mvneclipse 0m 14s trunk passed +1 findbugs 1m 56s trunk passed +1 javadoc 0m 43s trunk passed -1 mvninstall 0m 45s hadoop-hdfs in the patch failed. -1 compile 0m 41s hadoop-hdfs in the patch failed. -1 javac 0m 41s hadoop-hdfs in the patch failed. -0 checkstyle 0m 24s hadoop-hdfs-project/hadoop-hdfs: The patch generated 3 new + 111 unchanged - 1 fixed = 114 total (was 112) -1 mvnsite 0m 43s hadoop-hdfs in the patch failed. +1 mvneclipse 0m 10s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. -1 findbugs 0m 22s hadoop-hdfs in the patch failed. +1 javadoc 0m 38s the patch passed -1 unit 0m 43s hadoop-hdfs in the patch failed. +1 asflicense 0m 17s The patch does not generate ASF License warnings. 48m 57s Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HDFS-10913 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12843456/HDFS-10913.008.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux ff8498321b93 3.13.0-96-generic #143-Ubuntu SMP Mon Aug 29 20:15:20 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 70ca1f1 Default Java 1.8.0_111 findbugs v3.0.0 mvninstall https://builds.apache.org/job/PreCommit-HDFS-Build/17867/artifact/patchprocess/patch-mvninstall-hadoop-hdfs-project_hadoop-hdfs.txt compile https://builds.apache.org/job/PreCommit-HDFS-Build/17867/artifact/patchprocess/patch-compile-hadoop-hdfs-project_hadoop-hdfs.txt javac https://builds.apache.org/job/PreCommit-HDFS-Build/17867/artifact/patchprocess/patch-compile-hadoop-hdfs-project_hadoop-hdfs.txt checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/17867/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt mvnsite https://builds.apache.org/job/PreCommit-HDFS-Build/17867/artifact/patchprocess/patch-mvnsite-hadoop-hdfs-project_hadoop-hdfs.txt findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/17867/artifact/patchprocess/patch-findbugs-hadoop-hdfs-project_hadoop-hdfs.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/17867/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17867/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17867/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        xiaobingo Xiaobing Zhou added a comment -

        posted v9 to trigger Jenkins build. Also fixed a check style issue.

        Show
        xiaobingo Xiaobing Zhou added a comment - posted v9 to trigger Jenkins build. Also fixed a check style issue.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 11s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 13m 19s trunk passed
        +1 compile 0m 58s trunk passed
        +1 checkstyle 0m 31s trunk passed
        +1 mvnsite 0m 58s trunk passed
        +1 mvneclipse 0m 13s trunk passed
        +1 findbugs 1m 44s trunk passed
        +1 javadoc 0m 42s trunk passed
        -1 mvninstall 0m 40s hadoop-hdfs in the patch failed.
        -1 compile 0m 40s hadoop-hdfs in the patch failed.
        -1 javac 0m 40s hadoop-hdfs in the patch failed.
        -0 checkstyle 0m 24s hadoop-hdfs-project/hadoop-hdfs: The patch generated 2 new + 111 unchanged - 1 fixed = 113 total (was 112)
        -1 mvnsite 0m 42s hadoop-hdfs in the patch failed.
        +1 mvneclipse 0m 10s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        -1 findbugs 0m 21s hadoop-hdfs in the patch failed.
        +1 javadoc 0m 37s the patch passed
        -1 unit 0m 40s hadoop-hdfs in the patch failed.
        +1 asflicense 0m 16s The patch does not generate ASF License warnings.
        24m 12s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:a9ad5d6
        JIRA Issue HDFS-10913
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12843921/HDFS-10913.009.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux a59ac03198fe 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18 21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / ef2dd7b
        Default Java 1.8.0_111
        findbugs v3.0.0
        mvninstall https://builds.apache.org/job/PreCommit-HDFS-Build/17894/artifact/patchprocess/patch-mvninstall-hadoop-hdfs-project_hadoop-hdfs.txt
        compile https://builds.apache.org/job/PreCommit-HDFS-Build/17894/artifact/patchprocess/patch-compile-hadoop-hdfs-project_hadoop-hdfs.txt
        javac https://builds.apache.org/job/PreCommit-HDFS-Build/17894/artifact/patchprocess/patch-compile-hadoop-hdfs-project_hadoop-hdfs.txt
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/17894/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
        mvnsite https://builds.apache.org/job/PreCommit-HDFS-Build/17894/artifact/patchprocess/patch-mvnsite-hadoop-hdfs-project_hadoop-hdfs.txt
        findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/17894/artifact/patchprocess/patch-findbugs-hadoop-hdfs-project_hadoop-hdfs.txt
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/17894/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17894/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17894/console
        Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 11s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 13m 19s trunk passed +1 compile 0m 58s trunk passed +1 checkstyle 0m 31s trunk passed +1 mvnsite 0m 58s trunk passed +1 mvneclipse 0m 13s trunk passed +1 findbugs 1m 44s trunk passed +1 javadoc 0m 42s trunk passed -1 mvninstall 0m 40s hadoop-hdfs in the patch failed. -1 compile 0m 40s hadoop-hdfs in the patch failed. -1 javac 0m 40s hadoop-hdfs in the patch failed. -0 checkstyle 0m 24s hadoop-hdfs-project/hadoop-hdfs: The patch generated 2 new + 111 unchanged - 1 fixed = 113 total (was 112) -1 mvnsite 0m 42s hadoop-hdfs in the patch failed. +1 mvneclipse 0m 10s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. -1 findbugs 0m 21s hadoop-hdfs in the patch failed. +1 javadoc 0m 37s the patch passed -1 unit 0m 40s hadoop-hdfs in the patch failed. +1 asflicense 0m 16s The patch does not generate ASF License warnings. 24m 12s Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HDFS-10913 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12843921/HDFS-10913.009.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux a59ac03198fe 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18 21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / ef2dd7b Default Java 1.8.0_111 findbugs v3.0.0 mvninstall https://builds.apache.org/job/PreCommit-HDFS-Build/17894/artifact/patchprocess/patch-mvninstall-hadoop-hdfs-project_hadoop-hdfs.txt compile https://builds.apache.org/job/PreCommit-HDFS-Build/17894/artifact/patchprocess/patch-compile-hadoop-hdfs-project_hadoop-hdfs.txt javac https://builds.apache.org/job/PreCommit-HDFS-Build/17894/artifact/patchprocess/patch-compile-hadoop-hdfs-project_hadoop-hdfs.txt checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/17894/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt mvnsite https://builds.apache.org/job/PreCommit-HDFS-Build/17894/artifact/patchprocess/patch-mvnsite-hadoop-hdfs-project_hadoop-hdfs.txt findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/17894/artifact/patchprocess/patch-findbugs-hadoop-hdfs-project_hadoop-hdfs.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/17894/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17894/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17894/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        xiaobingo Xiaobing Zhou added a comment -

        v10 fixed the compile issue once missed in previous check.

        Show
        xiaobingo Xiaobing Zhou added a comment - v10 fixed the compile issue once missed in previous check.
        Hide
        xiaobingo Xiaobing Zhou added a comment -

        Posted v11.

        1. avoided passing replyAddr through BlockReceiver#receiveBlock since it's already passed as mirrIn through BlockReceiver constructor.
        2. adjusted delay time to accelerate test run.
        Show
        xiaobingo Xiaobing Zhou added a comment - Posted v11. avoided passing replyAddr through BlockReceiver#receiveBlock since it's already passed as mirrIn through BlockReceiver constructor. adjusted delay time to accelerate test run.
        Hide
        hadoopqa Hadoop QA added a comment -
        +1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 8s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
        +1 mvninstall 13m 40s trunk passed
        +1 compile 0m 47s trunk passed
        +1 checkstyle 0m 26s trunk passed
        +1 mvnsite 0m 50s trunk passed
        +1 mvneclipse 0m 13s trunk passed
        +1 findbugs 1m 43s trunk passed
        +1 javadoc 0m 38s trunk passed
        +1 mvninstall 0m 46s the patch passed
        +1 compile 0m 43s the patch passed
        +1 javac 0m 43s the patch passed
        -0 checkstyle 0m 23s hadoop-hdfs-project/hadoop-hdfs: The patch generated 1 new + 65 unchanged - 1 fixed = 66 total (was 66)
        +1 mvnsite 0m 47s the patch passed
        +1 mvneclipse 0m 10s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 1m 47s the patch passed
        +1 javadoc 0m 35s the patch passed
        +1 unit 65m 3s hadoop-hdfs in the patch passed.
        +1 asflicense 0m 18s The patch does not generate ASF License warnings.
        90m 14s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:a9ad5d6
        JIRA Issue HDFS-10913
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12843974/HDFS-10913.011.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 21f923fb2003 3.13.0-105-generic #152-Ubuntu SMP Fri Dec 2 15:37:11 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / ef2dd7b
        Default Java 1.8.0_111
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/17905/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17905/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17905/console
        Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 8s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. +1 mvninstall 13m 40s trunk passed +1 compile 0m 47s trunk passed +1 checkstyle 0m 26s trunk passed +1 mvnsite 0m 50s trunk passed +1 mvneclipse 0m 13s trunk passed +1 findbugs 1m 43s trunk passed +1 javadoc 0m 38s trunk passed +1 mvninstall 0m 46s the patch passed +1 compile 0m 43s the patch passed +1 javac 0m 43s the patch passed -0 checkstyle 0m 23s hadoop-hdfs-project/hadoop-hdfs: The patch generated 1 new + 65 unchanged - 1 fixed = 66 total (was 66) +1 mvnsite 0m 47s the patch passed +1 mvneclipse 0m 10s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 47s the patch passed +1 javadoc 0m 35s the patch passed +1 unit 65m 3s hadoop-hdfs in the patch passed. +1 asflicense 0m 18s The patch does not generate ASF License warnings. 90m 14s Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HDFS-10913 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12843974/HDFS-10913.011.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 21f923fb2003 3.13.0-105-generic #152-Ubuntu SMP Fri Dec 2 15:37:11 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / ef2dd7b Default Java 1.8.0_111 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/17905/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17905/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17905/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        hadoopqa Hadoop QA added a comment -
        +1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 10s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
        +1 mvninstall 14m 1s trunk passed
        +1 compile 0m 44s trunk passed
        +1 checkstyle 0m 25s trunk passed
        +1 mvnsite 0m 50s trunk passed
        +1 mvneclipse 0m 12s trunk passed
        +1 findbugs 1m 41s trunk passed
        +1 javadoc 0m 40s trunk passed
        +1 mvninstall 0m 45s the patch passed
        +1 compile 0m 42s the patch passed
        +1 javac 0m 42s the patch passed
        -0 checkstyle 0m 23s hadoop-hdfs-project/hadoop-hdfs: The patch generated 1 new + 65 unchanged - 1 fixed = 66 total (was 66)
        +1 mvnsite 0m 48s the patch passed
        +1 mvneclipse 0m 10s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 1m 46s the patch passed
        +1 javadoc 0m 37s the patch passed
        +1 unit 64m 43s hadoop-hdfs in the patch passed.
        +1 asflicense 0m 18s The patch does not generate ASF License warnings.
        90m 3s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:a9ad5d6
        JIRA Issue HDFS-10913
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12843978/HDFS-10913.011.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 647ffece7410 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / ef2dd7b
        Default Java 1.8.0_111
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/17906/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17906/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17906/console
        Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 10s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. +1 mvninstall 14m 1s trunk passed +1 compile 0m 44s trunk passed +1 checkstyle 0m 25s trunk passed +1 mvnsite 0m 50s trunk passed +1 mvneclipse 0m 12s trunk passed +1 findbugs 1m 41s trunk passed +1 javadoc 0m 40s trunk passed +1 mvninstall 0m 45s the patch passed +1 compile 0m 42s the patch passed +1 javac 0m 42s the patch passed -0 checkstyle 0m 23s hadoop-hdfs-project/hadoop-hdfs: The patch generated 1 new + 65 unchanged - 1 fixed = 66 total (was 66) +1 mvnsite 0m 48s the patch passed +1 mvneclipse 0m 10s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 46s the patch passed +1 javadoc 0m 37s the patch passed +1 unit 64m 43s hadoop-hdfs in the patch passed. +1 asflicense 0m 18s The patch does not generate ASF License warnings. 90m 3s Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HDFS-10913 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12843978/HDFS-10913.011.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 647ffece7410 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / ef2dd7b Default Java 1.8.0_111 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/17906/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17906/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17906/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        arpitagarwal Arpit Agarwal added a comment -

        The patch looks good. A few stylistic comments on the tests but +1 on the patch basically.

        1. delayOnce doesn't need to accept any parameters, right? It always injects one delay
        2. typo in test case. deplayMs should be delayMs.
        3. Nitpick: Unnecessary downcast in verifyFaultInjectionDelayPipeline. We can just make the parameter of type MetricsDataNodeFaultInjector.
          +    MetricsDataNodeFaultInjector mdnfi =
          +        (MetricsDataNodeFaultInjector) dnFaultInjector;
          
        Show
        arpitagarwal Arpit Agarwal added a comment - The patch looks good. A few stylistic comments on the tests but +1 on the patch basically. delayOnce doesn't need to accept any parameters, right? It always injects one delay typo in test case. deplayMs should be delayMs . Nitpick: Unnecessary downcast in verifyFaultInjectionDelayPipeline. We can just make the parameter of type MetricsDataNodeFaultInjector. + MetricsDataNodeFaultInjector mdnfi = + (MetricsDataNodeFaultInjector) dnFaultInjector;
        Hide
        xyao Xiaoyu Yao added a comment -

        Thanks Xiaobing Zhou for working on this. The latest patch looks good to me.
        Just have one comment in addition to Arpit Agarwal's comments above. +1 otherwise.

        TestDataNodeFaultInjector#verifyFaultInjectionDelayPipeline Line 165, "final FSDataOutputStream out" needs to be closed after each call.

        Show
        xyao Xiaoyu Yao added a comment - Thanks Xiaobing Zhou for working on this. The latest patch looks good to me. Just have one comment in addition to Arpit Agarwal 's comments above. +1 otherwise. TestDataNodeFaultInjector#verifyFaultInjectionDelayPipeline Line 165, "final FSDataOutputStream out" needs to be closed after each call.
        Hide
        xiaobingo Xiaobing Zhou added a comment - - edited

        Thanks Arpit Agarwal and Xiaoyu Yao for your reviews.
        Posted v12 patch to address your comments.

        Show
        xiaobingo Xiaobing Zhou added a comment - - edited Thanks Arpit Agarwal and Xiaoyu Yao for your reviews. Posted v12 patch to address your comments.
        Hide
        arpitagarwal Arpit Agarwal added a comment -

        +1 on the v12 patch, pending Jenkins. Thanks for addressing the feedback.

        Show
        arpitagarwal Arpit Agarwal added a comment - +1 on the v12 patch, pending Jenkins. Thanks for addressing the feedback.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 13s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
        +1 mvninstall 15m 41s trunk passed
        +1 compile 0m 55s trunk passed
        +1 checkstyle 0m 30s trunk passed
        +1 mvnsite 1m 3s trunk passed
        +1 mvneclipse 0m 15s trunk passed
        +1 findbugs 2m 5s trunk passed
        +1 javadoc 0m 49s trunk passed
        +1 mvninstall 0m 55s the patch passed
        +1 compile 0m 51s the patch passed
        +1 javac 0m 51s the patch passed
        -0 checkstyle 0m 27s hadoop-hdfs-project/hadoop-hdfs: The patch generated 1 new + 65 unchanged - 1 fixed = 66 total (was 66)
        +1 mvnsite 0m 57s the patch passed
        +1 mvneclipse 0m 12s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 2m 10s the patch passed
        +1 javadoc 0m 49s the patch passed
        -1 unit 70m 12s hadoop-hdfs in the patch failed.
        +1 asflicense 0m 21s The patch does not generate ASF License warnings.
        99m 56s



        Reason Tests
        Failed junit tests hadoop.hdfs.TestAclsEndToEnd
          hadoop.hdfs.TestBlockStoragePolicy



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:a9ad5d6
        JIRA Issue HDFS-10913
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12844105/HDFS-10913.012.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 92c85f7290e5 3.13.0-105-generic #152-Ubuntu SMP Fri Dec 2 15:37:11 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 1b401f6
        Default Java 1.8.0_111
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/17919/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/17919/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17919/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17919/console
        Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 13s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. +1 mvninstall 15m 41s trunk passed +1 compile 0m 55s trunk passed +1 checkstyle 0m 30s trunk passed +1 mvnsite 1m 3s trunk passed +1 mvneclipse 0m 15s trunk passed +1 findbugs 2m 5s trunk passed +1 javadoc 0m 49s trunk passed +1 mvninstall 0m 55s the patch passed +1 compile 0m 51s the patch passed +1 javac 0m 51s the patch passed -0 checkstyle 0m 27s hadoop-hdfs-project/hadoop-hdfs: The patch generated 1 new + 65 unchanged - 1 fixed = 66 total (was 66) +1 mvnsite 0m 57s the patch passed +1 mvneclipse 0m 12s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 2m 10s the patch passed +1 javadoc 0m 49s the patch passed -1 unit 70m 12s hadoop-hdfs in the patch failed. +1 asflicense 0m 21s The patch does not generate ASF License warnings. 99m 56s Reason Tests Failed junit tests hadoop.hdfs.TestAclsEndToEnd   hadoop.hdfs.TestBlockStoragePolicy Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HDFS-10913 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12844105/HDFS-10913.012.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 92c85f7290e5 3.13.0-105-generic #152-Ubuntu SMP Fri Dec 2 15:37:11 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 1b401f6 Default Java 1.8.0_111 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/17919/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/17919/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17919/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17919/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        hadoopqa Hadoop QA added a comment -
        +1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 27s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
        +1 mvninstall 16m 34s trunk passed
        +1 compile 1m 9s trunk passed
        +1 checkstyle 0m 34s trunk passed
        +1 mvnsite 1m 6s trunk passed
        +1 mvneclipse 0m 14s trunk passed
        +1 findbugs 2m 6s trunk passed
        +1 javadoc 0m 47s trunk passed
        +1 mvninstall 0m 53s the patch passed
        +1 compile 0m 53s the patch passed
        +1 javac 0m 53s the patch passed
        -0 checkstyle 0m 30s hadoop-hdfs-project/hadoop-hdfs: The patch generated 1 new + 65 unchanged - 1 fixed = 66 total (was 66)
        +1 mvnsite 1m 7s the patch passed
        +1 mvneclipse 0m 16s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 2m 12s the patch passed
        +1 javadoc 0m 47s the patch passed
        +1 unit 97m 11s hadoop-hdfs in the patch passed.
        +1 asflicense 0m 21s The patch does not generate ASF License warnings.
        128m 45s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:a9ad5d6
        JIRA Issue HDFS-10913
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12844107/HDFS-10913.012.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 52d59da0812c 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 1b401f6
        Default Java 1.8.0_111
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/17920/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17920/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17920/console
        Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 27s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. +1 mvninstall 16m 34s trunk passed +1 compile 1m 9s trunk passed +1 checkstyle 0m 34s trunk passed +1 mvnsite 1m 6s trunk passed +1 mvneclipse 0m 14s trunk passed +1 findbugs 2m 6s trunk passed +1 javadoc 0m 47s trunk passed +1 mvninstall 0m 53s the patch passed +1 compile 0m 53s the patch passed +1 javac 0m 53s the patch passed -0 checkstyle 0m 30s hadoop-hdfs-project/hadoop-hdfs: The patch generated 1 new + 65 unchanged - 1 fixed = 66 total (was 66) +1 mvnsite 1m 7s the patch passed +1 mvneclipse 0m 16s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 2m 12s the patch passed +1 javadoc 0m 47s the patch passed +1 unit 97m 11s hadoop-hdfs in the patch passed. +1 asflicense 0m 21s The patch does not generate ASF License warnings. 128m 45s Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HDFS-10913 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12844107/HDFS-10913.012.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 52d59da0812c 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 1b401f6 Default Java 1.8.0_111 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/17920/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17920/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17920/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        xyao Xiaoyu Yao added a comment -

        Thanks Xiaobing Zhou for the contribution and all for the reviews. I've commit the patch to trunk.

        Show
        xyao Xiaoyu Yao added a comment - Thanks Xiaobing Zhou for the contribution and all for the reviews. I've commit the patch to trunk.
        Hide
        xiaobingo Xiaobing Zhou added a comment -

        Thank you all for reviewing/committing the patches.

        Show
        xiaobingo Xiaobing Zhou added a comment - Thank you all for reviewing/committing the patches.
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11021 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11021/)
        HDFS-10913. Introduce fault injectors to simulate slow mirrors. (xyao: rev 5daa8d8631835de97d4e4979e507a080017ca159)

        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNodeFaultInjector.java
        • (add) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeFaultInjector.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestClientProtocolForPipelineRecovery.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockReceiver.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11021 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11021/ ) HDFS-10913 . Introduce fault injectors to simulate slow mirrors. (xyao: rev 5daa8d8631835de97d4e4979e507a080017ca159) (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNodeFaultInjector.java (add) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeFaultInjector.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestClientProtocolForPipelineRecovery.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockReceiver.java

          People

          • Assignee:
            xiaobingo Xiaobing Zhou
            Reporter:
            xiaobingo Xiaobing Zhou
          • Votes:
            0 Vote for this issue
            Watchers:
            9 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development