Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-10519

Add a configuration option to enable in-progress edit log tailing

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.0.0-alpha1
    • Component/s: ha
    • Labels:
      None
    • Target Version/s:
    • Release Note:
      Add a configuration option to enable in-progress edit tailing and a related unit test

      Description

      Standby Namenode has the option to do in-progress edit log tailing to improve the data freshness. In-progress tailing is already implemented, but it's not enabled as default configuration. And there's no related configuration key to turn it on.
      Adding a related configuration key to let Standby Namenode is reasonable and would be a basis for further improvement on Standby Namenode.

      1. HDFS-10519.013.patch
        49 kB
        Jiayi Zhou
      2. HDFS-10519.012.patch
        49 kB
        Jiayi Zhou
      3. HDFS-10519.011.patch
        49 kB
        Jiayi Zhou
      4. HDFS-10519.010.patch
        49 kB
        Jiayi Zhou
      5. HDFS-10519.009.patch
        45 kB
        Jiayi Zhou
      6. HDFS-10519.008.patch
        44 kB
        Jiayi Zhou
      7. HDFS-10519.007.patch
        43 kB
        Jiayi Zhou
      8. HDFS-10519.006.patch
        25 kB
        Jiayi Zhou
      9. HDFS-10519.005.patch
        23 kB
        Jiayi Zhou
      10. HDFS-10519.004.patch
        22 kB
        Jiayi Zhou
      11. HDFS-10519.003.patch
        7 kB
        Jiayi Zhou
      12. HDFS-10519.002.patch
        20 kB
        Jiayi Zhou
      13. HDFS-10519.001.patch
        21 kB
        Jiayi Zhou

        Issue Links

          Activity

          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 19s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 6m 44s trunk passed
          +1 compile 0m 44s trunk passed
          +1 checkstyle 0m 31s trunk passed
          +1 mvnsite 0m 51s trunk passed
          +1 mvneclipse 0m 12s trunk passed
          +1 findbugs 1m 41s trunk passed
          +1 javadoc 0m 54s trunk passed
          +1 mvninstall 0m 47s the patch passed
          +1 compile 0m 42s the patch passed
          +1 javac 0m 42s the patch passed
          -1 checkstyle 0m 30s hadoop-hdfs-project/hadoop-hdfs: The patch generated 218 new + 416 unchanged - 0 fixed = 634 total (was 416)
          +1 mvnsite 0m 48s the patch passed
          +1 mvneclipse 0m 9s the patch passed
          -1 whitespace 0m 0s The patch has 20 line(s) that end in whitespace. Use git apply --whitespace=fix.
          +1 xml 0m 1s The patch has no ill-formed XML file.
          +1 findbugs 1m 48s the patch passed
          +1 javadoc 0m 51s the patch passed
          -1 unit 69m 27s hadoop-hdfs in the patch failed.
          +1 asflicense 0m 19s The patch does not generate ASF License warnings.
          88m 36s



          Reason Tests
          Failed junit tests hadoop.hdfs.TestAsyncHDFSWithHA



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:2c91fd8
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12810072/HDFS-10519.001.patch
          JIRA Issue HDFS-10519
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml
          uname Linux 6535bede1f4c 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 709a814
          Default Java 1.8.0_91
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/15760/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
          whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/15760/artifact/patchprocess/whitespace-eol.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/15760/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
          unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/15760/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/15760/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/15760/console
          Powered by Apache Yetus 0.3.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 19s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 6m 44s trunk passed +1 compile 0m 44s trunk passed +1 checkstyle 0m 31s trunk passed +1 mvnsite 0m 51s trunk passed +1 mvneclipse 0m 12s trunk passed +1 findbugs 1m 41s trunk passed +1 javadoc 0m 54s trunk passed +1 mvninstall 0m 47s the patch passed +1 compile 0m 42s the patch passed +1 javac 0m 42s the patch passed -1 checkstyle 0m 30s hadoop-hdfs-project/hadoop-hdfs: The patch generated 218 new + 416 unchanged - 0 fixed = 634 total (was 416) +1 mvnsite 0m 48s the patch passed +1 mvneclipse 0m 9s the patch passed -1 whitespace 0m 0s The patch has 20 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 xml 0m 1s The patch has no ill-formed XML file. +1 findbugs 1m 48s the patch passed +1 javadoc 0m 51s the patch passed -1 unit 69m 27s hadoop-hdfs in the patch failed. +1 asflicense 0m 19s The patch does not generate ASF License warnings. 88m 36s Reason Tests Failed junit tests hadoop.hdfs.TestAsyncHDFSWithHA Subsystem Report/Notes Docker Image:yetus/hadoop:2c91fd8 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12810072/HDFS-10519.001.patch JIRA Issue HDFS-10519 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml uname Linux 6535bede1f4c 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 709a814 Default Java 1.8.0_91 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/15760/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/15760/artifact/patchprocess/whitespace-eol.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/15760/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/15760/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/15760/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/15760/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 16s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 6m 10s trunk passed
          +1 compile 0m 43s trunk passed
          +1 checkstyle 0m 30s trunk passed
          +1 mvnsite 0m 49s trunk passed
          +1 mvneclipse 0m 12s trunk passed
          +1 findbugs 1m 38s trunk passed
          +1 javadoc 0m 54s trunk passed
          +1 mvninstall 0m 46s the patch passed
          +1 compile 0m 41s the patch passed
          +1 javac 0m 41s the patch passed
          -1 checkstyle 0m 28s hadoop-hdfs-project/hadoop-hdfs: The patch generated 16 new + 415 unchanged - 0 fixed = 431 total (was 415)
          +1 mvnsite 0m 47s the patch passed
          +1 mvneclipse 0m 10s the patch passed
          -1 whitespace 0m 0s The patch has 20 line(s) that end in whitespace. Use git apply --whitespace=fix.
          +1 xml 0m 2s The patch has no ill-formed XML file.
          +1 findbugs 1m 45s the patch passed
          +1 javadoc 0m 51s the patch passed
          -1 unit 57m 22s hadoop-hdfs in the patch failed.
          +1 asflicense 0m 18s The patch does not generate ASF License warnings.
          75m 33s



          Reason Tests
          Failed junit tests hadoop.hdfs.server.namenode.TestEditLog



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:2c91fd8
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12810171/HDFS-10519.002.patch
          JIRA Issue HDFS-10519
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml
          uname Linux d75808aa1206 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 709a814
          Default Java 1.8.0_91
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/15763/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
          whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/15763/artifact/patchprocess/whitespace-eol.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/15763/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
          unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/15763/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/15763/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/15763/console
          Powered by Apache Yetus 0.3.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 16s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 6m 10s trunk passed +1 compile 0m 43s trunk passed +1 checkstyle 0m 30s trunk passed +1 mvnsite 0m 49s trunk passed +1 mvneclipse 0m 12s trunk passed +1 findbugs 1m 38s trunk passed +1 javadoc 0m 54s trunk passed +1 mvninstall 0m 46s the patch passed +1 compile 0m 41s the patch passed +1 javac 0m 41s the patch passed -1 checkstyle 0m 28s hadoop-hdfs-project/hadoop-hdfs: The patch generated 16 new + 415 unchanged - 0 fixed = 431 total (was 415) +1 mvnsite 0m 47s the patch passed +1 mvneclipse 0m 10s the patch passed -1 whitespace 0m 0s The patch has 20 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 xml 0m 2s The patch has no ill-formed XML file. +1 findbugs 1m 45s the patch passed +1 javadoc 0m 51s the patch passed -1 unit 57m 22s hadoop-hdfs in the patch failed. +1 asflicense 0m 18s The patch does not generate ASF License warnings. 75m 33s Reason Tests Failed junit tests hadoop.hdfs.server.namenode.TestEditLog Subsystem Report/Notes Docker Image:yetus/hadoop:2c91fd8 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12810171/HDFS-10519.002.patch JIRA Issue HDFS-10519 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml uname Linux d75808aa1206 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 709a814 Default Java 1.8.0_91 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/15763/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/15763/artifact/patchprocess/whitespace-eol.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/15763/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/15763/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/15763/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/15763/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
          Hide
          kihwal Kihwal Lee added a comment -

          How does it actually work? Can it replay the same edit log segment multiple times? It might have to replay the same in-progress segment multiple times or after it is finalized.
          I thought it would blow up with MetaRecoveryContext being null.

          Show
          kihwal Kihwal Lee added a comment - How does it actually work? Can it replay the same edit log segment multiple times? It might have to replay the same in-progress segment multiple times or after it is finalized. I thought it would blow up with MetaRecoveryContext being null.
          Hide
          clouderajiayi Jiayi Zhou added a comment -

          Maybe I got something wrong, but isn't the MetaRecoveryContext already null at the beginning?

          In doTailEdits() from EditLogTailer, we have
          streams = editLog.selectInputStreams(lastTxnId + 1, 0, null, inProgressTail);

          Here, MetaRecoverContext is null. And in image.LoadEdits(), we have
          public long loadEdits(Iterable<EditLogInputStream> editStreams,
          FSNamesystem target) throws IOException

          { return loadEdits(editStreams, target, null, null); }

          MetaRecoveryContext is also null.

          Also, I don't know why toAtLeastTxId is always set to 0 in the implementation, making the checkForGaps() method no use. If we set it to some other value, like lastTxnId + 1, the multiple replay should be rejected.

          Show
          clouderajiayi Jiayi Zhou added a comment - Maybe I got something wrong, but isn't the MetaRecoveryContext already null at the beginning? In doTailEdits() from EditLogTailer, we have streams = editLog.selectInputStreams(lastTxnId + 1, 0, null, inProgressTail); Here, MetaRecoverContext is null. And in image.LoadEdits(), we have public long loadEdits(Iterable<EditLogInputStream> editStreams, FSNamesystem target) throws IOException { return loadEdits(editStreams, target, null, null); } MetaRecoveryContext is also null. Also, I don't know why toAtLeastTxId is always set to 0 in the implementation, making the checkForGaps() method no use. If we set it to some other value, like lastTxnId + 1, the multiple replay should be rejected.
          Hide
          clouderajiayi Jiayi Zhou added a comment -

          Add a fake sendedits to make committed txn id more up-to-date. This means the standby namenode will only tail the in-progress edits which are committed. Thank you Todd Lipcon for the idea during an offline discussion.

          Show
          clouderajiayi Jiayi Zhou added a comment - Add a fake sendedits to make committed txn id more up-to-date. This means the standby namenode will only tail the in-progress edits which are committed. Thank you Todd Lipcon for the idea during an offline discussion.
          Hide
          clouderajiayi Jiayi Zhou added a comment - - edited

          Add a unit test which I forgot to add in the 003 patch.

          Show
          clouderajiayi Jiayi Zhou added a comment - - edited Add a unit test which I forgot to add in the 003 patch.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 16s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 6m 43s trunk passed
          +1 compile 0m 42s trunk passed
          +1 checkstyle 0m 31s trunk passed
          +1 mvnsite 0m 52s trunk passed
          +1 mvneclipse 0m 12s trunk passed
          +1 findbugs 1m 41s trunk passed
          +1 javadoc 0m 56s trunk passed
          +1 mvninstall 0m 46s the patch passed
          +1 compile 0m 43s the patch passed
          +1 javac 0m 43s the patch passed
          -0 checkstyle 0m 29s hadoop-hdfs-project/hadoop-hdfs: The patch generated 18 new + 456 unchanged - 0 fixed = 474 total (was 456)
          +1 mvnsite 0m 49s the patch passed
          +1 mvneclipse 0m 9s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 xml 0m 2s The patch has no ill-formed XML file.
          -1 findbugs 1m 47s hadoop-hdfs-project/hadoop-hdfs generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)
          +1 javadoc 0m 52s the patch passed
          -1 unit 59m 44s hadoop-hdfs in the patch failed.
          +1 asflicense 0m 19s The patch does not generate ASF License warnings.
          78m 47s



          Reason Tests
          FindBugs module:hadoop-hdfs-project/hadoop-hdfs
            Unused field:RemoteEditLogManifest.java
          Failed junit tests hadoop.hdfs.qjournal.client.TestQuorumJournalManagerUnit
            hadoop.hdfs.server.blockmanagement.TestComputeInvalidateWork



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12817525/HDFS-10519.004.patch
          JIRA Issue HDFS-10519
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml
          uname Linux 80d302a230db 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 10b704c
          Default Java 1.8.0_91
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/16032/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
          findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/16032/artifact/patchprocess/new-findbugs-hadoop-hdfs-project_hadoop-hdfs.html
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/16032/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16032/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16032/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 16s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 6m 43s trunk passed +1 compile 0m 42s trunk passed +1 checkstyle 0m 31s trunk passed +1 mvnsite 0m 52s trunk passed +1 mvneclipse 0m 12s trunk passed +1 findbugs 1m 41s trunk passed +1 javadoc 0m 56s trunk passed +1 mvninstall 0m 46s the patch passed +1 compile 0m 43s the patch passed +1 javac 0m 43s the patch passed -0 checkstyle 0m 29s hadoop-hdfs-project/hadoop-hdfs: The patch generated 18 new + 456 unchanged - 0 fixed = 474 total (was 456) +1 mvnsite 0m 49s the patch passed +1 mvneclipse 0m 9s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 2s The patch has no ill-formed XML file. -1 findbugs 1m 47s hadoop-hdfs-project/hadoop-hdfs generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) +1 javadoc 0m 52s the patch passed -1 unit 59m 44s hadoop-hdfs in the patch failed. +1 asflicense 0m 19s The patch does not generate ASF License warnings. 78m 47s Reason Tests FindBugs module:hadoop-hdfs-project/hadoop-hdfs   Unused field:RemoteEditLogManifest.java Failed junit tests hadoop.hdfs.qjournal.client.TestQuorumJournalManagerUnit   hadoop.hdfs.server.blockmanagement.TestComputeInvalidateWork Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12817525/HDFS-10519.004.patch JIRA Issue HDFS-10519 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml uname Linux 80d302a230db 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 10b704c Default Java 1.8.0_91 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/16032/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/16032/artifact/patchprocess/new-findbugs-hadoop-hdfs-project_hadoop-hdfs.html unit https://builds.apache.org/job/PreCommit-HDFS-Build/16032/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16032/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16032/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          clouderajiayi Jiayi Zhou added a comment -

          Fix bugs and checkstyle problems. Add a boolean flag in QuorumOutputStream so that fake send will be triggered only when in-progress tailing is enabled.

          Show
          clouderajiayi Jiayi Zhou added a comment - Fix bugs and checkstyle problems. Add a boolean flag in QuorumOutputStream so that fake send will be triggered only when in-progress tailing is enabled.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 26s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 8m 0s trunk passed
          +1 compile 0m 52s trunk passed
          +1 checkstyle 0m 33s trunk passed
          +1 mvnsite 1m 0s trunk passed
          +1 mvneclipse 0m 12s trunk passed
          +1 findbugs 1m 56s trunk passed
          +1 javadoc 1m 5s trunk passed
          +1 mvninstall 0m 53s the patch passed
          +1 compile 0m 47s the patch passed
          +1 javac 0m 47s the patch passed
          -0 checkstyle 0m 29s hadoop-hdfs-project/hadoop-hdfs: The patch generated 2 new + 451 unchanged - 0 fixed = 453 total (was 451)
          +1 mvnsite 0m 57s the patch passed
          +1 mvneclipse 0m 11s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 xml 0m 2s The patch has no ill-formed XML file.
          +1 findbugs 1m 57s the patch passed
          +1 javadoc 0m 52s the patch passed
          -1 unit 70m 7s hadoop-hdfs in the patch failed.
          +1 asflicense 0m 22s The patch does not generate ASF License warnings.
          91m 59s



          Reason Tests
          Failed junit tests hadoop.hdfs.server.namenode.ha.TestBootstrapStandbyWithQJM
            hadoop.hdfs.server.blockmanagement.TestPendingInvalidateBlock
            hadoop.hdfs.server.datanode.TestDataNodeUUID
            hadoop.hdfs.server.namenode.ha.TestDFSUpgradeWithHA



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12817562/HDFS-10519.005.patch
          JIRA Issue HDFS-10519
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml
          uname Linux 97ccef7f87ed 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / a290a98
          Default Java 1.8.0_91
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/16036/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/16036/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16036/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16036/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 26s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 8m 0s trunk passed +1 compile 0m 52s trunk passed +1 checkstyle 0m 33s trunk passed +1 mvnsite 1m 0s trunk passed +1 mvneclipse 0m 12s trunk passed +1 findbugs 1m 56s trunk passed +1 javadoc 1m 5s trunk passed +1 mvninstall 0m 53s the patch passed +1 compile 0m 47s the patch passed +1 javac 0m 47s the patch passed -0 checkstyle 0m 29s hadoop-hdfs-project/hadoop-hdfs: The patch generated 2 new + 451 unchanged - 0 fixed = 453 total (was 451) +1 mvnsite 0m 57s the patch passed +1 mvneclipse 0m 11s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 2s The patch has no ill-formed XML file. +1 findbugs 1m 57s the patch passed +1 javadoc 0m 52s the patch passed -1 unit 70m 7s hadoop-hdfs in the patch failed. +1 asflicense 0m 22s The patch does not generate ASF License warnings. 91m 59s Reason Tests Failed junit tests hadoop.hdfs.server.namenode.ha.TestBootstrapStandbyWithQJM   hadoop.hdfs.server.blockmanagement.TestPendingInvalidateBlock   hadoop.hdfs.server.datanode.TestDataNodeUUID   hadoop.hdfs.server.namenode.ha.TestDFSUpgradeWithHA Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12817562/HDFS-10519.005.patch JIRA Issue HDFS-10519 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml uname Linux 97ccef7f87ed 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / a290a98 Default Java 1.8.0_91 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/16036/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/16036/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16036/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16036/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          clouderajiayi Jiayi Zhou added a comment -

          Also add a boolean flag in Journal for the same purpose.

          Show
          clouderajiayi Jiayi Zhou added a comment - Also add a boolean flag in Journal for the same purpose.
          Hide
          hadoopqa Hadoop QA added a comment -
          +1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 22s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 7m 7s trunk passed
          +1 compile 0m 47s trunk passed
          +1 checkstyle 0m 31s trunk passed
          +1 mvnsite 0m 54s trunk passed
          +1 mvneclipse 0m 12s trunk passed
          +1 findbugs 1m 46s trunk passed
          +1 javadoc 0m 57s trunk passed
          +1 mvninstall 0m 53s the patch passed
          +1 compile 0m 49s the patch passed
          +1 javac 0m 49s the patch passed
          +1 checkstyle 0m 28s the patch passed
          +1 mvnsite 0m 52s the patch passed
          +1 mvneclipse 0m 10s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 xml 0m 1s The patch has no ill-formed XML file.
          +1 findbugs 1m 58s the patch passed
          +1 javadoc 0m 54s the patch passed
          +1 unit 61m 8s hadoop-hdfs in the patch passed.
          +1 asflicense 0m 18s The patch does not generate ASF License warnings.
          81m 23s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12817776/HDFS-10519.006.patch
          JIRA Issue HDFS-10519
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml
          uname Linux 73c8744b0f21 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / eb47163
          Default Java 1.8.0_91
          findbugs v3.0.0
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16048/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16048/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 22s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 7m 7s trunk passed +1 compile 0m 47s trunk passed +1 checkstyle 0m 31s trunk passed +1 mvnsite 0m 54s trunk passed +1 mvneclipse 0m 12s trunk passed +1 findbugs 1m 46s trunk passed +1 javadoc 0m 57s trunk passed +1 mvninstall 0m 53s the patch passed +1 compile 0m 49s the patch passed +1 javac 0m 49s the patch passed +1 checkstyle 0m 28s the patch passed +1 mvnsite 0m 52s the patch passed +1 mvneclipse 0m 10s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 1s The patch has no ill-formed XML file. +1 findbugs 1m 58s the patch passed +1 javadoc 0m 54s the patch passed +1 unit 61m 8s hadoop-hdfs in the patch passed. +1 asflicense 0m 18s The patch does not generate ASF License warnings. 81m 23s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12817776/HDFS-10519.006.patch JIRA Issue HDFS-10519 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml uname Linux 73c8744b0f21 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / eb47163 Default Java 1.8.0_91 findbugs v3.0.0 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16048/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16048/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          clouderajiayi Jiayi Zhou added a comment -

          Add a new parameter isTail to selectInputStream() on the NameNode side and a field in RemoteEditLogManifest. When we do in-progress tailing, we'll use committedTxnId rather than highestWrittenTxnId. This won't affect other parts which also need to select in-progress edits.

          Show
          clouderajiayi Jiayi Zhou added a comment - Add a new parameter isTail to selectInputStream() on the NameNode side and a field in RemoteEditLogManifest. When we do in-progress tailing, we'll use committedTxnId rather than highestWrittenTxnId. This won't affect other parts which also need to select in-progress edits.
          Hide
          andrew.wang Andrew Wang added a comment -

          Hi Jiayi, thanks for working on this. I took a look at the patch and have mostly style comments. It looks pretty good overall, appreciate the new tests.

          • config key should mention "edit log" somewhere. We can reuse an existing prefix and call it something like "dfs.ha.tail-edits.in-progress"
          • Would be good to explain a little more in hdfs-default.xml why a user might want to turn this on.
          • EditLogTailer, regarding the comment: edits themselves are not in-progress, an edit log segment is. Consider renaming the boolean also.
          • "isTail" is also not very descriptive, what it's really doing is bounding the tailing to the committed txID length right? Better to name it accordingly, rather than tie it to a single usecase like standby tailing.

          Journal.java

          • inProgress boolean is unused. Conceptually the JNs shouldn't have to be aware of the standby doing in-progress tailing anyway.
          • getCommittedTxnIdForTests isn't used only in tests, rename it?
          • Can you add a comment on the numTxns == 0 early return, explaining why we do this?
          • JournalSet, why is the correct committed txnid to set here 0 rather than the max txId from the set of logs?

          QuorumOutputStream

          • Let's pull the boolean out of the conf and pass it into the constructor rather than pass in the entire Configuration, this helps limit the scope. The boolean should also be named something more descriptive, like "updateCommittedTxnId".
          • The new conditional dummy flush also needs a comment for sure. How stale would the committed txnId be if we didn't write this 0-len segment? Any ideas on how to further optimize this (e.g. make it more async)?
          • QuorumJournalManager, the ternary can be written instead with Math.min yea? I'd prefer to see a little logic rework that avoids the need for the else also.

          RemoteEditLogManifest

          • Do we need the single-arg form of the constructor? We only use it once in a test, maybe just always require the two-arg form.
          • Initialize committedTxnId to some invalid txn id, and validate it in checkState? Should also be within bounds.
          • toString output should included the committedTxnId
          • TestBKSR, TestFJM, you don't need to modify the call since we kept the old overload.
          • TestNNSRManager, we should pass through the parameter in the mock, like the other args

          TestStandbyInProgressTail

          • Typo: edig
          • Typo: shoudl
          • Do we need the Thread.sleep for tailing? Can we trigger edit log tailing manually instead? Sleeps are unreliable and increase test runtime.
          • Looks like some of the startup/shutdown logic is shared, can we use @Before and @After annotated methods to share code? As an FYI we also normally do a null check on cluster before calling cluster.shutdown, as an extra guard.
          • New assert methods can be made static
          • Any thoughts on hooking this up to the randomized fault-injection tests originally developed for QJM? I think we'd all have more confidence in this feature if we ran the fuzzer for a while and it came back clean.
          Show
          andrew.wang Andrew Wang added a comment - Hi Jiayi, thanks for working on this. I took a look at the patch and have mostly style comments. It looks pretty good overall, appreciate the new tests. config key should mention "edit log" somewhere. We can reuse an existing prefix and call it something like "dfs.ha.tail-edits.in-progress" Would be good to explain a little more in hdfs-default.xml why a user might want to turn this on. EditLogTailer, regarding the comment: edits themselves are not in-progress, an edit log segment is. Consider renaming the boolean also. "isTail" is also not very descriptive, what it's really doing is bounding the tailing to the committed txID length right? Better to name it accordingly, rather than tie it to a single usecase like standby tailing. Journal.java inProgress boolean is unused. Conceptually the JNs shouldn't have to be aware of the standby doing in-progress tailing anyway. getCommittedTxnIdForTests isn't used only in tests, rename it? Can you add a comment on the numTxns == 0 early return, explaining why we do this? JournalSet, why is the correct committed txnid to set here 0 rather than the max txId from the set of logs? QuorumOutputStream Let's pull the boolean out of the conf and pass it into the constructor rather than pass in the entire Configuration, this helps limit the scope. The boolean should also be named something more descriptive, like "updateCommittedTxnId". The new conditional dummy flush also needs a comment for sure. How stale would the committed txnId be if we didn't write this 0-len segment? Any ideas on how to further optimize this (e.g. make it more async)? QuorumJournalManager, the ternary can be written instead with Math.min yea? I'd prefer to see a little logic rework that avoids the need for the else also. RemoteEditLogManifest Do we need the single-arg form of the constructor? We only use it once in a test, maybe just always require the two-arg form. Initialize committedTxnId to some invalid txn id, and validate it in checkState? Should also be within bounds. toString output should included the committedTxnId TestBKSR, TestFJM, you don't need to modify the call since we kept the old overload. TestNNSRManager, we should pass through the parameter in the mock, like the other args TestStandbyInProgressTail Typo: edig Typo: shoudl Do we need the Thread.sleep for tailing? Can we trigger edit log tailing manually instead? Sleeps are unreliable and increase test runtime. Looks like some of the startup/shutdown logic is shared, can we use @Before and @After annotated methods to share code? As an FYI we also normally do a null check on cluster before calling cluster.shutdown, as an extra guard. New assert methods can be made static Any thoughts on hooking this up to the randomized fault-injection tests originally developed for QJM? I think we'd all have more confidence in this feature if we ran the fuzzer for a while and it came back clean.
          Hide
          clouderajiayi Jiayi Zhou added a comment -

          Thanks Andrew for the review. I have attached a new patch to solve the problems.

          • Config key name fixed.
          • Comment added in hdfs-default.xml. Users want to turn it on if they want more up-to-date data on SBNN.

            EditLogTailer, regarding the comment: edits themselves are not in-progress, an edit log segment is. Consider renaming the boolean also.

          • Fixed. Rename to inProgressOk which is just the parameter name in selectInputStream()

            "isTail" is also not very descriptive, what it's really doing is bounding the tailing to the committed txID length right?

          • Change it to isBoundedByDurableTxId. It looks a bit long. Any suggestions?

          Journal.java

          • Unused boolean has been removed
          • Rename to getCommittedTxnId
          • Comment added. numTxn = 0 implies it's a dummy edits send, so we can return early.

            JournalSet, why is the correct committed txnid to set here 0 rather than the max txId from the set of logs?

          • Good point. I lazily set it to 0 since it would not be used. Change it to the max txId from the logs.

          QuorumOutputStream

          • Fixed. Boolean has been changed to updateCommittedTxnId
          • Comments added. Committed txnId could be one-batch stale if we don't have this dummy flush. A thread could be added to do this dummy flush, but setting a proper time period needs to be considered. For now, I prefer to leave it sync.

            QuorumJournalManager, the ternary can be written instead with Math.min yea? I'd prefer to see a little logic rework that avoids the need for the else also.

          • Fixed.

          RemoteEditLogManifest

          • Remove single-arg form of constructor.
          • Add a validation in checkState. committedTxnId should be larger than the start TxId of the logs.
          • toString() modified.

            TestBKSR, TestFJM, you don't need to modify the call since we kept the old overload.

          • Fixed. In TestFJM, parameter is JournalManager, changed it to FileJournalManager.

            TestNNSRManager, we should pass through the parameter in the mock, like the other args

          • Fixed.

          TestStandbyInProgressTail

          • Typos fixed
          • Thread.sleep removed. Use manual doTailEdits() instead.
          • @Before and @After methods added. Null check is included in tearDown()
          • Assert methods are static now.
          • Randomized tests could be tricky. I haven't come up with a good idea yet. A new jira might be filed for it in the future.
          Show
          clouderajiayi Jiayi Zhou added a comment - Thanks Andrew for the review. I have attached a new patch to solve the problems. Config key name fixed. Comment added in hdfs-default.xml. Users want to turn it on if they want more up-to-date data on SBNN. EditLogTailer, regarding the comment: edits themselves are not in-progress, an edit log segment is. Consider renaming the boolean also. Fixed. Rename to inProgressOk which is just the parameter name in selectInputStream() "isTail" is also not very descriptive, what it's really doing is bounding the tailing to the committed txID length right? Change it to isBoundedByDurableTxId. It looks a bit long. Any suggestions? Journal.java Unused boolean has been removed Rename to getCommittedTxnId Comment added. numTxn = 0 implies it's a dummy edits send, so we can return early. JournalSet, why is the correct committed txnid to set here 0 rather than the max txId from the set of logs? Good point. I lazily set it to 0 since it would not be used. Change it to the max txId from the logs. QuorumOutputStream Fixed. Boolean has been changed to updateCommittedTxnId Comments added. Committed txnId could be one-batch stale if we don't have this dummy flush. A thread could be added to do this dummy flush, but setting a proper time period needs to be considered. For now, I prefer to leave it sync. QuorumJournalManager, the ternary can be written instead with Math.min yea? I'd prefer to see a little logic rework that avoids the need for the else also. Fixed. RemoteEditLogManifest Remove single-arg form of constructor. Add a validation in checkState. committedTxnId should be larger than the start TxId of the logs. toString() modified. TestBKSR, TestFJM, you don't need to modify the call since we kept the old overload. Fixed. In TestFJM, parameter is JournalManager, changed it to FileJournalManager. TestNNSRManager, we should pass through the parameter in the mock, like the other args Fixed. TestStandbyInProgressTail Typos fixed Thread.sleep removed. Use manual doTailEdits() instead. @Before and @After methods added. Null check is included in tearDown() Assert methods are static now. Randomized tests could be tricky. I haven't come up with a good idea yet. A new jira might be filed for it in the future.
          Hide
          andrew.wang Andrew Wang added a comment -

          Thanks for revving Jiayi. Overall looks good, mostly just nits now. I think the next rev should do it.

          • in DFSConfigKeys, let's group the new config key with the other "dfs.ha.tail-edits" keys
          • Regarding isBoundedByDurableTxId, maybe shorten to onlyDurableTxns? Should update javadoc for JournalSet#selectInputStreams also to talk about the boolean. Would be good to explain what a "durable TxId" means also, and the tightness of the bound (it can be conservative).
          • Nit: in QuorumJournalManager, we already assigned remoteLog.getEditTxId() to endTxId. So we can re-use endTxId when doing Math.min.
          • QuorumOutputStream, recommend we add the new boolean at the end of the list of parameters, we generally try to put flags/options at the end
          • RemoteEditLogManifest, the manifest might not have any logs, then the logs.get(0) check will fail.

          I would like to think about those randomized tests more, but yea that can happen in another JIRA.

          Show
          andrew.wang Andrew Wang added a comment - Thanks for revving Jiayi. Overall looks good, mostly just nits now. I think the next rev should do it. in DFSConfigKeys, let's group the new config key with the other "dfs.ha.tail-edits" keys Regarding isBoundedByDurableTxId , maybe shorten to onlyDurableTxns ? Should update javadoc for JournalSet#selectInputStreams also to talk about the boolean. Would be good to explain what a "durable TxId" means also, and the tightness of the bound (it can be conservative). Nit: in QuorumJournalManager, we already assigned remoteLog.getEditTxId() to endTxId. So we can re-use endTxId when doing Math.min. QuorumOutputStream, recommend we add the new boolean at the end of the list of parameters, we generally try to put flags/options at the end RemoteEditLogManifest, the manifest might not have any logs, then the logs.get(0) check will fail. I would like to think about those randomized tests more, but yea that can happen in another JIRA.
          Hide
          clouderajiayi Jiayi Zhou added a comment -

          Fix some problems according to Andrew's recent comment.

          Show
          clouderajiayi Jiayi Zhou added a comment - Fix some problems according to Andrew's recent comment.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 19s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 6 new or modified test files.
          0 mvndep 0m 19s Maven dependency ordering for branch
          +1 mvninstall 8m 37s trunk passed
          +1 compile 0m 57s trunk passed
          +1 checkstyle 0m 39s trunk passed
          +1 mvnsite 1m 24s trunk passed
          +1 mvneclipse 0m 27s trunk passed
          +1 findbugs 2m 24s trunk passed
          +1 javadoc 1m 11s trunk passed
          0 mvndep 0m 7s Maven dependency ordering for patch
          +1 mvninstall 1m 9s the patch passed
          +1 compile 0m 44s the patch passed
          +1 cc 0m 44s the patch passed
          +1 javac 0m 44s the patch passed
          -0 checkstyle 0m 34s hadoop-hdfs-project/hadoop-hdfs: The patch generated 7 new + 822 unchanged - 0 fixed = 829 total (was 822)
          +1 mvnsite 1m 6s the patch passed
          +1 mvneclipse 0m 21s the patch passed
          -1 whitespace 0m 0s The patch has 2 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
          +1 xml 0m 1s The patch has no ill-formed XML file.
          +1 findbugs 2m 26s the patch passed
          +1 javadoc 1m 6s the patch passed
          -1 unit 59m 30s hadoop-hdfs in the patch failed.
          +1 unit 4m 20s bkjournal in the patch passed.
          +1 asflicense 0m 18s The patch does not generate ASF License warnings.
          89m 32s



          Reason Tests
          Failed junit tests hadoop.hdfs.server.namenode.TestEditLog
            hadoop.hdfs.qjournal.client.TestQuorumJournalManager
            hadoop.hdfs.server.namenode.ha.TestFailureToReadEdits
            hadoop.hdfs.qjournal.client.TestQJMWithFaults
            hadoop.hdfs.server.namenode.ha.TestDFSUpgradeWithHA
            hadoop.hdfs.TestDFSInotifyEventInputStream



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12819235/HDFS-10519.009.patch
          JIRA Issue HDFS-10519
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc xml
          uname Linux cd65b7e1b38d 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / e340064
          Default Java 1.8.0_91
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/16122/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
          whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/16122/artifact/patchprocess/whitespace-eol.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/16122/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16122/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal U: hadoop-hdfs-project/hadoop-hdfs
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16122/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 19s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 6 new or modified test files. 0 mvndep 0m 19s Maven dependency ordering for branch +1 mvninstall 8m 37s trunk passed +1 compile 0m 57s trunk passed +1 checkstyle 0m 39s trunk passed +1 mvnsite 1m 24s trunk passed +1 mvneclipse 0m 27s trunk passed +1 findbugs 2m 24s trunk passed +1 javadoc 1m 11s trunk passed 0 mvndep 0m 7s Maven dependency ordering for patch +1 mvninstall 1m 9s the patch passed +1 compile 0m 44s the patch passed +1 cc 0m 44s the patch passed +1 javac 0m 44s the patch passed -0 checkstyle 0m 34s hadoop-hdfs-project/hadoop-hdfs: The patch generated 7 new + 822 unchanged - 0 fixed = 829 total (was 822) +1 mvnsite 1m 6s the patch passed +1 mvneclipse 0m 21s the patch passed -1 whitespace 0m 0s The patch has 2 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply +1 xml 0m 1s The patch has no ill-formed XML file. +1 findbugs 2m 26s the patch passed +1 javadoc 1m 6s the patch passed -1 unit 59m 30s hadoop-hdfs in the patch failed. +1 unit 4m 20s bkjournal in the patch passed. +1 asflicense 0m 18s The patch does not generate ASF License warnings. 89m 32s Reason Tests Failed junit tests hadoop.hdfs.server.namenode.TestEditLog   hadoop.hdfs.qjournal.client.TestQuorumJournalManager   hadoop.hdfs.server.namenode.ha.TestFailureToReadEdits   hadoop.hdfs.qjournal.client.TestQJMWithFaults   hadoop.hdfs.server.namenode.ha.TestDFSUpgradeWithHA   hadoop.hdfs.TestDFSInotifyEventInputStream Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12819235/HDFS-10519.009.patch JIRA Issue HDFS-10519 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc xml uname Linux cd65b7e1b38d 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / e340064 Default Java 1.8.0_91 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/16122/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/16122/artifact/patchprocess/whitespace-eol.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/16122/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16122/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16122/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 26s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 6 new or modified test files.
          0 mvndep 0m 21s Maven dependency ordering for branch
          +1 mvninstall 7m 24s trunk passed
          +1 compile 0m 53s trunk passed
          +1 checkstyle 0m 37s trunk passed
          +1 mvnsite 1m 26s trunk passed
          +1 mvneclipse 0m 25s trunk passed
          +1 findbugs 2m 23s trunk passed
          +1 javadoc 1m 14s trunk passed
          0 mvndep 0m 6s Maven dependency ordering for patch
          +1 mvninstall 1m 11s the patch passed
          +1 compile 0m 50s the patch passed
          +1 cc 0m 50s the patch passed
          +1 javac 0m 50s the patch passed
          -0 checkstyle 0m 35s hadoop-hdfs-project/hadoop-hdfs: The patch generated 7 new + 823 unchanged - 0 fixed = 830 total (was 823)
          +1 mvnsite 1m 15s the patch passed
          +1 mvneclipse 0m 23s the patch passed
          -1 whitespace 0m 0s The patch has 2 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
          +1 xml 0m 1s The patch has no ill-formed XML file.
          +1 findbugs 2m 36s the patch passed
          +1 javadoc 1m 9s the patch passed
          -1 unit 68m 40s hadoop-hdfs in the patch failed.
          +1 unit 6m 57s bkjournal in the patch passed.
          +1 asflicense 0m 20s The patch does not generate ASF License warnings.
          100m 40s



          Reason Tests
          Failed junit tests hadoop.hdfs.qjournal.client.TestQJMWithFaults
            hadoop.hdfs.TestDFSInotifyEventInputStream
            hadoop.hdfs.server.datanode.TestDataNodeLifeline
            hadoop.hdfs.server.namenode.ha.TestFailureToReadEdits
            hadoop.hdfs.qjournal.client.TestQuorumJournalManager
            hadoop.hdfs.server.namenode.ha.TestDFSUpgradeWithHA
            hadoop.hdfs.server.namenode.TestEditLog



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12819235/HDFS-10519.009.patch
          JIRA Issue HDFS-10519
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc xml
          uname Linux 4b0cfd6bb825 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / e340064
          Default Java 1.8.0_91
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/16124/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
          whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/16124/artifact/patchprocess/whitespace-eol.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/16124/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16124/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal U: hadoop-hdfs-project/hadoop-hdfs
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16124/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 26s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 6 new or modified test files. 0 mvndep 0m 21s Maven dependency ordering for branch +1 mvninstall 7m 24s trunk passed +1 compile 0m 53s trunk passed +1 checkstyle 0m 37s trunk passed +1 mvnsite 1m 26s trunk passed +1 mvneclipse 0m 25s trunk passed +1 findbugs 2m 23s trunk passed +1 javadoc 1m 14s trunk passed 0 mvndep 0m 6s Maven dependency ordering for patch +1 mvninstall 1m 11s the patch passed +1 compile 0m 50s the patch passed +1 cc 0m 50s the patch passed +1 javac 0m 50s the patch passed -0 checkstyle 0m 35s hadoop-hdfs-project/hadoop-hdfs: The patch generated 7 new + 823 unchanged - 0 fixed = 830 total (was 823) +1 mvnsite 1m 15s the patch passed +1 mvneclipse 0m 23s the patch passed -1 whitespace 0m 0s The patch has 2 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply +1 xml 0m 1s The patch has no ill-formed XML file. +1 findbugs 2m 36s the patch passed +1 javadoc 1m 9s the patch passed -1 unit 68m 40s hadoop-hdfs in the patch failed. +1 unit 6m 57s bkjournal in the patch passed. +1 asflicense 0m 20s The patch does not generate ASF License warnings. 100m 40s Reason Tests Failed junit tests hadoop.hdfs.qjournal.client.TestQJMWithFaults   hadoop.hdfs.TestDFSInotifyEventInputStream   hadoop.hdfs.server.datanode.TestDataNodeLifeline   hadoop.hdfs.server.namenode.ha.TestFailureToReadEdits   hadoop.hdfs.qjournal.client.TestQuorumJournalManager   hadoop.hdfs.server.namenode.ha.TestDFSUpgradeWithHA   hadoop.hdfs.server.namenode.TestEditLog Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12819235/HDFS-10519.009.patch JIRA Issue HDFS-10519 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc xml uname Linux 4b0cfd6bb825 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / e340064 Default Java 1.8.0_91 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/16124/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/16124/artifact/patchprocess/whitespace-eol.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/16124/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16124/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16124/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 21s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 6 new or modified test files.
          0 mvndep 0m 6s Maven dependency ordering for branch
          +1 mvninstall 7m 42s trunk passed
          +1 compile 0m 54s trunk passed
          +1 checkstyle 0m 39s trunk passed
          +1 mvnsite 1m 21s trunk passed
          +1 mvneclipse 0m 26s trunk passed
          +1 findbugs 2m 20s trunk passed
          +1 javadoc 1m 13s trunk passed
          0 mvndep 0m 6s Maven dependency ordering for patch
          +1 mvninstall 1m 7s the patch passed
          +1 compile 0m 46s the patch passed
          +1 cc 0m 46s the patch passed
          +1 javac 0m 46s the patch passed
          -0 checkstyle 0m 34s hadoop-hdfs-project/hadoop-hdfs: The patch generated 7 new + 822 unchanged - 0 fixed = 829 total (was 822)
          +1 mvnsite 1m 6s the patch passed
          +1 mvneclipse 0m 20s the patch passed
          -1 whitespace 0m 0s The patch has 2 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
          +1 xml 0m 1s The patch has no ill-formed XML file.
          +1 findbugs 2m 16s the patch passed
          +1 javadoc 1m 3s the patch passed
          -1 unit 57m 51s hadoop-hdfs in the patch failed.
          +1 unit 4m 18s bkjournal in the patch passed.
          +1 asflicense 0m 19s The patch does not generate ASF License warnings.
          86m 12s



          Reason Tests
          Failed junit tests hadoop.hdfs.server.namenode.TestEditLog
            hadoop.hdfs.qjournal.client.TestQuorumJournalManager
            hadoop.hdfs.server.namenode.ha.TestFailureToReadEdits
            hadoop.hdfs.qjournal.client.TestQJMWithFaults
            hadoop.hdfs.server.namenode.ha.TestDFSUpgradeWithHA
            hadoop.hdfs.TestDFSInotifyEventInputStream



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12819235/HDFS-10519.009.patch
          JIRA Issue HDFS-10519
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc xml
          uname Linux 231c098f3992 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / e340064
          Default Java 1.8.0_91
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/16127/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
          whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/16127/artifact/patchprocess/whitespace-eol.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/16127/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16127/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal U: hadoop-hdfs-project/hadoop-hdfs
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16127/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 21s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 6 new or modified test files. 0 mvndep 0m 6s Maven dependency ordering for branch +1 mvninstall 7m 42s trunk passed +1 compile 0m 54s trunk passed +1 checkstyle 0m 39s trunk passed +1 mvnsite 1m 21s trunk passed +1 mvneclipse 0m 26s trunk passed +1 findbugs 2m 20s trunk passed +1 javadoc 1m 13s trunk passed 0 mvndep 0m 6s Maven dependency ordering for patch +1 mvninstall 1m 7s the patch passed +1 compile 0m 46s the patch passed +1 cc 0m 46s the patch passed +1 javac 0m 46s the patch passed -0 checkstyle 0m 34s hadoop-hdfs-project/hadoop-hdfs: The patch generated 7 new + 822 unchanged - 0 fixed = 829 total (was 822) +1 mvnsite 1m 6s the patch passed +1 mvneclipse 0m 20s the patch passed -1 whitespace 0m 0s The patch has 2 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply +1 xml 0m 1s The patch has no ill-formed XML file. +1 findbugs 2m 16s the patch passed +1 javadoc 1m 3s the patch passed -1 unit 57m 51s hadoop-hdfs in the patch failed. +1 unit 4m 18s bkjournal in the patch passed. +1 asflicense 0m 19s The patch does not generate ASF License warnings. 86m 12s Reason Tests Failed junit tests hadoop.hdfs.server.namenode.TestEditLog   hadoop.hdfs.qjournal.client.TestQuorumJournalManager   hadoop.hdfs.server.namenode.ha.TestFailureToReadEdits   hadoop.hdfs.qjournal.client.TestQJMWithFaults   hadoop.hdfs.server.namenode.ha.TestDFSUpgradeWithHA   hadoop.hdfs.TestDFSInotifyEventInputStream Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12819235/HDFS-10519.009.patch JIRA Issue HDFS-10519 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc xml uname Linux 231c098f3992 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / e340064 Default Java 1.8.0_91 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/16127/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/16127/artifact/patchprocess/whitespace-eol.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/16127/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16127/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16127/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          clouderajiayi Jiayi Zhou added a comment -

          Fix some style problems. Remove committedTxId check from RemoteEditLogManifest. Since we don't have dummy send in normal cases, committedTxId will not be up-to-date (especially for a newly-setup cluster).

          Show
          clouderajiayi Jiayi Zhou added a comment - Fix some style problems. Remove committedTxId check from RemoteEditLogManifest. Since we don't have dummy send in normal cases, committedTxId will not be up-to-date (especially for a newly-setup cluster).
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 0s Docker mode activated.
          -1 patch 0m 6s HDFS-10519 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help.



          Subsystem Report/Notes
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12819511/HDFS-10519.010.patch
          JIRA Issue HDFS-10519
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16175/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. -1 patch 0m 6s HDFS-10519 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. Subsystem Report/Notes JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12819511/HDFS-10519.010.patch JIRA Issue HDFS-10519 Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16175/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 13s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 8 new or modified test files.
          0 mvndep 0m 20s Maven dependency ordering for branch
          +1 mvninstall 6m 42s trunk passed
          +1 compile 0m 44s trunk passed
          +1 checkstyle 0m 37s trunk passed
          +1 mvnsite 1m 9s trunk passed
          +1 mvneclipse 0m 23s trunk passed
          +1 findbugs 2m 5s trunk passed
          +1 javadoc 1m 9s trunk passed
          0 mvndep 0m 6s Maven dependency ordering for patch
          +1 mvninstall 1m 0s the patch passed
          +1 compile 0m 42s the patch passed
          +1 cc 0m 42s the patch passed
          +1 javac 0m 42s the patch passed
          -0 checkstyle 0m 35s hadoop-hdfs-project/hadoop-hdfs: The patch generated 2 new + 968 unchanged - 0 fixed = 970 total (was 968)
          +1 mvnsite 1m 3s the patch passed
          +1 mvneclipse 0m 19s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 xml 0m 1s The patch has no ill-formed XML file.
          +1 findbugs 2m 15s the patch passed
          +1 javadoc 1m 4s the patch passed
          -1 unit 60m 23s hadoop-hdfs in the patch failed.
          +1 unit 4m 18s bkjournal in the patch passed.
          +1 asflicense 0m 17s The patch does not generate ASF License warnings.
          86m 55s



          Reason Tests
          Failed junit tests hadoop.hdfs.TestDFSShell



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12820056/HDFS-10519.011.patch
          JIRA Issue HDFS-10519
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc xml
          uname Linux d135ec04e615 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / d383bfd
          Default Java 1.8.0_101
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/16179/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/16179/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16179/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal U: hadoop-hdfs-project/hadoop-hdfs
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16179/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 13s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 8 new or modified test files. 0 mvndep 0m 20s Maven dependency ordering for branch +1 mvninstall 6m 42s trunk passed +1 compile 0m 44s trunk passed +1 checkstyle 0m 37s trunk passed +1 mvnsite 1m 9s trunk passed +1 mvneclipse 0m 23s trunk passed +1 findbugs 2m 5s trunk passed +1 javadoc 1m 9s trunk passed 0 mvndep 0m 6s Maven dependency ordering for patch +1 mvninstall 1m 0s the patch passed +1 compile 0m 42s the patch passed +1 cc 0m 42s the patch passed +1 javac 0m 42s the patch passed -0 checkstyle 0m 35s hadoop-hdfs-project/hadoop-hdfs: The patch generated 2 new + 968 unchanged - 0 fixed = 970 total (was 968) +1 mvnsite 1m 3s the patch passed +1 mvneclipse 0m 19s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 1s The patch has no ill-formed XML file. +1 findbugs 2m 15s the patch passed +1 javadoc 1m 4s the patch passed -1 unit 60m 23s hadoop-hdfs in the patch failed. +1 unit 4m 18s bkjournal in the patch passed. +1 asflicense 0m 17s The patch does not generate ASF License warnings. 86m 55s Reason Tests Failed junit tests hadoop.hdfs.TestDFSShell Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12820056/HDFS-10519.011.patch JIRA Issue HDFS-10519 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc xml uname Linux d135ec04e615 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / d383bfd Default Java 1.8.0_101 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/16179/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/16179/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16179/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16179/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          clouderajiayi Jiayi Zhou added a comment -

          Fix style problems. Failed test is unrelated.

          Show
          clouderajiayi Jiayi Zhou added a comment - Fix style problems. Failed test is unrelated.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 15s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 8 new or modified test files.
          0 mvndep 0m 6s Maven dependency ordering for branch
          +1 mvninstall 8m 28s trunk passed
          +1 compile 1m 0s trunk passed
          +1 checkstyle 0m 40s trunk passed
          +1 mvnsite 1m 27s trunk passed
          +1 mvneclipse 0m 28s trunk passed
          +1 findbugs 2m 30s trunk passed
          +1 javadoc 1m 8s trunk passed
          0 mvndep 0m 6s Maven dependency ordering for patch
          +1 mvninstall 1m 4s the patch passed
          +1 compile 0m 46s the patch passed
          +1 cc 0m 46s the patch passed
          +1 javac 0m 46s the patch passed
          +1 checkstyle 0m 37s the patch passed
          +1 mvnsite 1m 9s the patch passed
          +1 mvneclipse 0m 21s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 xml 0m 1s The patch has no ill-formed XML file.
          +1 findbugs 2m 18s the patch passed
          +1 javadoc 1m 9s the patch passed
          -1 unit 76m 50s hadoop-hdfs in the patch failed.
          +1 unit 7m 29s bkjournal in the patch passed.
          +1 asflicense 0m 21s The patch does not generate ASF License warnings.
          109m 42s



          Reason Tests
          Failed junit tests hadoop.cli.TestHDFSCLI
            hadoop.hdfs.server.namenode.ha.TestEditLogTailer



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12820202/HDFS-10519.012.patch
          JIRA Issue HDFS-10519
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc xml
          uname Linux a92d9f700c6a 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / da6adf5
          Default Java 1.8.0_101
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/16189/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16189/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal U: hadoop-hdfs-project/hadoop-hdfs
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16189/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 15s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 8 new or modified test files. 0 mvndep 0m 6s Maven dependency ordering for branch +1 mvninstall 8m 28s trunk passed +1 compile 1m 0s trunk passed +1 checkstyle 0m 40s trunk passed +1 mvnsite 1m 27s trunk passed +1 mvneclipse 0m 28s trunk passed +1 findbugs 2m 30s trunk passed +1 javadoc 1m 8s trunk passed 0 mvndep 0m 6s Maven dependency ordering for patch +1 mvninstall 1m 4s the patch passed +1 compile 0m 46s the patch passed +1 cc 0m 46s the patch passed +1 javac 0m 46s the patch passed +1 checkstyle 0m 37s the patch passed +1 mvnsite 1m 9s the patch passed +1 mvneclipse 0m 21s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 1s The patch has no ill-formed XML file. +1 findbugs 2m 18s the patch passed +1 javadoc 1m 9s the patch passed -1 unit 76m 50s hadoop-hdfs in the patch failed. +1 unit 7m 29s bkjournal in the patch passed. +1 asflicense 0m 21s The patch does not generate ASF License warnings. 109m 42s Reason Tests Failed junit tests hadoop.cli.TestHDFSCLI   hadoop.hdfs.server.namenode.ha.TestEditLogTailer Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12820202/HDFS-10519.012.patch JIRA Issue HDFS-10519 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc xml uname Linux a92d9f700c6a 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / da6adf5 Default Java 1.8.0_101 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/16189/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16189/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16189/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 14s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 8 new or modified test files.
          0 mvndep 0m 6s Maven dependency ordering for branch
          +1 mvninstall 7m 13s trunk passed
          +1 compile 0m 50s trunk passed
          +1 checkstyle 0m 37s trunk passed
          +1 mvnsite 1m 15s trunk passed
          +1 mvneclipse 0m 24s trunk passed
          +1 findbugs 2m 10s trunk passed
          +1 javadoc 1m 9s trunk passed
          0 mvndep 0m 6s Maven dependency ordering for patch
          +1 mvninstall 1m 5s the patch passed
          +1 compile 0m 46s the patch passed
          +1 cc 0m 46s the patch passed
          +1 javac 0m 46s the patch passed
          +1 checkstyle 0m 36s the patch passed
          +1 mvnsite 1m 11s the patch passed
          +1 mvneclipse 0m 19s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 xml 0m 2s The patch has no ill-formed XML file.
          +1 findbugs 2m 27s the patch passed
          +1 javadoc 1m 4s the patch passed
          -1 unit 65m 40s hadoop-hdfs in the patch failed.
          +1 unit 4m 30s bkjournal in the patch passed.
          +1 asflicense 0m 20s The patch does not generate ASF License warnings.
          93m 37s



          Reason Tests
          Failed junit tests hadoop.cli.TestHDFSCLI



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12820208/HDFS-10519.012.patch
          JIRA Issue HDFS-10519
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc xml
          uname Linux e1584f8c1dd2 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / da6adf5
          Default Java 1.8.0_101
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/16190/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16190/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal U: hadoop-hdfs-project/hadoop-hdfs
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16190/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 14s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 8 new or modified test files. 0 mvndep 0m 6s Maven dependency ordering for branch +1 mvninstall 7m 13s trunk passed +1 compile 0m 50s trunk passed +1 checkstyle 0m 37s trunk passed +1 mvnsite 1m 15s trunk passed +1 mvneclipse 0m 24s trunk passed +1 findbugs 2m 10s trunk passed +1 javadoc 1m 9s trunk passed 0 mvndep 0m 6s Maven dependency ordering for patch +1 mvninstall 1m 5s the patch passed +1 compile 0m 46s the patch passed +1 cc 0m 46s the patch passed +1 javac 0m 46s the patch passed +1 checkstyle 0m 36s the patch passed +1 mvnsite 1m 11s the patch passed +1 mvneclipse 0m 19s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 2s The patch has no ill-formed XML file. +1 findbugs 2m 27s the patch passed +1 javadoc 1m 4s the patch passed -1 unit 65m 40s hadoop-hdfs in the patch failed. +1 unit 4m 30s bkjournal in the patch passed. +1 asflicense 0m 20s The patch does not generate ASF License warnings. 93m 37s Reason Tests Failed junit tests hadoop.cli.TestHDFSCLI Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12820208/HDFS-10519.012.patch JIRA Issue HDFS-10519 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc xml uname Linux e1584f8c1dd2 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / da6adf5 Default Java 1.8.0_101 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/16190/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16190/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16190/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          clouderajiayi Jiayi Zhou added a comment - - edited

          Failed test is unrelated. It is filed in another jira. HDFS-10696
          Andrew Wang Can you take a look at the patch and see if we could commit it now?

          Show
          clouderajiayi Jiayi Zhou added a comment - - edited Failed test is unrelated. It is filed in another jira. HDFS-10696 Andrew Wang Can you take a look at the patch and see if we could commit it now?
          Hide
          andrew.wang Andrew Wang added a comment -

          Just some nits, I think this really is the last one:

          • Some extra whitespace changes in RemoteEditLogManifest. Looks like there are a few other places we could undo the whitespace changes too.
          • Rather than adding this new logString method, shall we update the test strings instead? This method doesn't seem useful except to avoid updating the tests.
          Show
          andrew.wang Andrew Wang added a comment - Just some nits, I think this really is the last one: Some extra whitespace changes in RemoteEditLogManifest. Looks like there are a few other places we could undo the whitespace changes too. Rather than adding this new logString method, shall we update the test strings instead? This method doesn't seem useful except to avoid updating the tests.
          Hide
          clouderajiayi Jiayi Zhou added a comment -

          Undo some whitespace changes and update test strings instead of adding a new method.

          Show
          clouderajiayi Jiayi Zhou added a comment - Undo some whitespace changes and update test strings instead of adding a new method.
          Hide
          andrew.wang Andrew Wang added a comment -

          LGTM, thanks Jiayi! +1 pending Jenkins.

          Show
          andrew.wang Andrew Wang added a comment - LGTM, thanks Jiayi! +1 pending Jenkins.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 17s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 8 new or modified test files.
          0 mvndep 0m 16s Maven dependency ordering for branch
          +1 mvninstall 9m 0s trunk passed
          +1 compile 0m 49s trunk passed
          +1 checkstyle 0m 40s trunk passed
          +1 mvnsite 1m 15s trunk passed
          +1 mvneclipse 0m 27s trunk passed
          +1 findbugs 2m 18s trunk passed
          +1 javadoc 1m 12s trunk passed
          0 mvndep 0m 6s Maven dependency ordering for patch
          +1 mvninstall 1m 2s the patch passed
          +1 compile 0m 45s the patch passed
          +1 cc 0m 45s the patch passed
          +1 javac 0m 45s the patch passed
          +1 checkstyle 0m 36s the patch passed
          +1 mvnsite 1m 7s the patch passed
          +1 mvneclipse 0m 20s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 xml 0m 1s The patch has no ill-formed XML file.
          +1 findbugs 2m 20s the patch passed
          +1 javadoc 1m 3s the patch passed
          -1 unit 60m 3s hadoop-hdfs in the patch failed.
          +1 unit 5m 32s bkjournal in the patch passed.
          +1 asflicense 0m 21s The patch does not generate ASF License warnings.
          91m 1s



          Reason Tests
          Failed junit tests hadoop.hdfs.server.namenode.TestReconstructStripedBlocks
            hadoop.cli.TestHDFSCLI



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12820588/HDFS-10519.013.patch
          JIRA Issue HDFS-10519
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc xml
          uname Linux 216c152974d2 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / b43de80
          Default Java 1.8.0_101
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/16222/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16222/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal U: hadoop-hdfs-project/hadoop-hdfs
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16222/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 17s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 8 new or modified test files. 0 mvndep 0m 16s Maven dependency ordering for branch +1 mvninstall 9m 0s trunk passed +1 compile 0m 49s trunk passed +1 checkstyle 0m 40s trunk passed +1 mvnsite 1m 15s trunk passed +1 mvneclipse 0m 27s trunk passed +1 findbugs 2m 18s trunk passed +1 javadoc 1m 12s trunk passed 0 mvndep 0m 6s Maven dependency ordering for patch +1 mvninstall 1m 2s the patch passed +1 compile 0m 45s the patch passed +1 cc 0m 45s the patch passed +1 javac 0m 45s the patch passed +1 checkstyle 0m 36s the patch passed +1 mvnsite 1m 7s the patch passed +1 mvneclipse 0m 20s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 1s The patch has no ill-formed XML file. +1 findbugs 2m 20s the patch passed +1 javadoc 1m 3s the patch passed -1 unit 60m 3s hadoop-hdfs in the patch failed. +1 unit 5m 32s bkjournal in the patch passed. +1 asflicense 0m 21s The patch does not generate ASF License warnings. 91m 1s Reason Tests Failed junit tests hadoop.hdfs.server.namenode.TestReconstructStripedBlocks   hadoop.cli.TestHDFSCLI Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12820588/HDFS-10519.013.patch JIRA Issue HDFS-10519 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc xml uname Linux 216c152974d2 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / b43de80 Default Java 1.8.0_101 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/16222/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16222/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16222/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          andrew.wang Andrew Wang added a comment -

          Committed to trunk and branch-3.0.0-alpha1. Thanks for the contribution Jiayi!

          Show
          andrew.wang Andrew Wang added a comment - Committed to trunk and branch-3.0.0-alpha1. Thanks for the contribution Jiayi!
          Hide
          andrew.wang Andrew Wang added a comment -

          I think this would be pretty easy to backport to branch-2 if there's demand, just a few small conflicts.

          Show
          andrew.wang Andrew Wang added a comment - I think this would be pretty easy to backport to branch-2 if there's demand, just a few small conflicts.
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-trunk-Commit #10166 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10166/)
          HDFS-10519. Add a configuration option to enable in-progress edit log (wang: rev 098ec2b11ff3f677eb823f75b147a1ac8dbf959e)

          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestGenericJournalConf.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NNStorageRetentionManager.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FileJournalManager.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/LogsPurgeable.java
          • hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/java/org/apache/hadoop/contrib/bkjournal/BookKeeperJournalManager.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/JournalSet.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/qjournal/server/Journal.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestFailureToReadEdits.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/PBHelper.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/protocol/RemoteEditLogManifest.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/BackupJournalManager.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/SecondaryNameNode.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/qjournal/client/QuorumJournalManager.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFileJournalManager.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/protocolPB/TestPBHelper.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLog.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestStandbyInProgressTail.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestEditLog.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNNStorageRetentionManager.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/qjournal/server/TestJournal.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/ha/EditLogTailer.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/qjournal/client/QuorumOutputStream.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/proto/HdfsServer.proto
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-trunk-Commit #10166 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10166/ ) HDFS-10519 . Add a configuration option to enable in-progress edit log (wang: rev 098ec2b11ff3f677eb823f75b147a1ac8dbf959e) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestGenericJournalConf.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NNStorageRetentionManager.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FileJournalManager.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/LogsPurgeable.java hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/java/org/apache/hadoop/contrib/bkjournal/BookKeeperJournalManager.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/JournalSet.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/qjournal/server/Journal.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestFailureToReadEdits.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/PBHelper.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/protocol/RemoteEditLogManifest.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/BackupJournalManager.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/SecondaryNameNode.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/qjournal/client/QuorumJournalManager.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFileJournalManager.java hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/protocolPB/TestPBHelper.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLog.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestStandbyInProgressTail.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestEditLog.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNNStorageRetentionManager.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/qjournal/server/TestJournal.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/ha/EditLogTailer.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/qjournal/client/QuorumOutputStream.java hadoop-hdfs-project/hadoop-hdfs/src/main/proto/HdfsServer.proto

            People

            • Assignee:
              clouderajiayi Jiayi Zhou
              Reporter:
              clouderajiayi Jiayi Zhou
            • Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development