Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-4176

EditLogTailer should call rollEdits with a timeout

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.0.2-alpha, 3.0.0-alpha1
    • Fix Version/s: 2.9.0, 3.0.0-alpha1
    • Component/s: ha, namenode
    • Labels:
      None

      Description

      When the EditLogTailer thread calls rollEdits() on the active NN via RPC, it currently does so without a timeout. So, if the active NN has frozen (but not actually crashed), this call can hang forever. This can then potentially prevent the standby from becoming active.

      This may actually considered a side effect of HADOOP-6762 – if the RPC were interruptible, that would also fix the issue.

      1. namenode.jstack4
        70 kB
        Marc Heide
      2. HDFS-4176.00.patch
        9 kB
        Lei (Eddy) Xu
      3. HDFS-4176.01.patch
        9 kB
        Lei (Eddy) Xu
      4. HDFS-4176.02.patch
        10 kB
        Lei (Eddy) Xu
      5. HDFS-4176.03.patch
        10 kB
        Lei (Eddy) Xu
      6. HDFS-4176.04.patch
        10 kB
        Lei (Eddy) Xu
      7. HDFS-4176-branch-2.0.patch
        9 kB
        Lei (Eddy) Xu
      8. HDFS-4176-branch-2.1.patch
        9 kB
        Lei (Eddy) Xu
      9. HDFS-4176-branch-2.2.patch
        9 kB
        Lei (Eddy) Xu
      10. HDFS-4176-branch-2.003.patch
        9 kB
        Lei (Eddy) Xu

        Issue Links

          Activity

          Hide
          eddyxu Lei (Eddy) Xu added a comment -

          Thanks for the reviews Xiao Chen Jing Zhao. I committed it to both branch-2 and trunk.

          And thanks [~Surendra Singh Lilhore] for the suggestions. I filed HDFS-10734 for changing the key names.

          Show
          eddyxu Lei (Eddy) Xu added a comment - Thanks for the reviews Xiao Chen Jing Zhao . I committed it to both branch-2 and trunk. And thanks [~Surendra Singh Lilhore] for the suggestions. I filed HDFS-10734 for changing the key names.
          Hide
          xiaochen Xiao Chen added a comment -

          I was pinged to review the branch-2 patch, 003 LGTM +1.
          Thanks Eddy and Jing for the nice work!

          Show
          xiaochen Xiao Chen added a comment - I was pinged to review the branch-2 patch, 003 LGTM +1. Thanks Eddy and Jing for the nice work!
          Hide
          eddyxu Lei (Eddy) Xu added a comment -

          Hi, Surendra Singh Lilhore
          The patch has already committed to trunk. Can we use a following JIRA to do the config key renaming?

          Show
          eddyxu Lei (Eddy) Xu added a comment - Hi, Surendra Singh Lilhore The patch has already committed to trunk. Can we use a following JIRA to do the config key renaming?
          Hide
          surendrasingh Surendra Singh Lilhore added a comment -

          Hi Lei (Eddy) Xu

          One minor comments.

          +  public static final String DFS_HA_TAILEDITS_ROLLEDITS_TIMEOUT_KEY =
          +      "dfs.ha.tail-edits.rolledits.timeout";
          

          Can we rename this property to dfs.ha.log-roll.execution.timeout ?. It will be in sync with dfs.ha.log-roll.rpc.timeout

          Show
          surendrasingh Surendra Singh Lilhore added a comment - Hi Lei (Eddy) Xu One minor comments. + public static final String DFS_HA_TAILEDITS_ROLLEDITS_TIMEOUT_KEY = + "dfs.ha.tail-edits.rolledits.timeout" ; Can we rename this property to dfs.ha.log-roll.execution.timeout ?. It will be in sync with dfs.ha.log-roll.rpc.timeout
          Hide
          hadoopqa Hadoop QA added a comment -
          +1 overall



          Vote Subsystem Runtime Comment
          0 reexec 16m 5s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 13m 23s branch-2 passed
          +1 compile 0m 55s branch-2 passed with JDK v1.8.0_101
          +1 compile 0m 49s branch-2 passed with JDK v1.7.0_101
          +1 checkstyle 0m 37s branch-2 passed
          +1 mvnsite 1m 37s branch-2 passed
          +1 mvneclipse 0m 20s branch-2 passed
          +1 findbugs 2m 24s branch-2 passed
          +1 javadoc 1m 5s branch-2 passed with JDK v1.8.0_101
          +1 javadoc 1m 50s branch-2 passed with JDK v1.7.0_101
          +1 mvninstall 0m 52s the patch passed
          +1 compile 0m 43s the patch passed with JDK v1.8.0_101
          +1 javac 0m 43s the patch passed
          +1 compile 0m 46s the patch passed with JDK v1.7.0_101
          +1 javac 0m 46s the patch passed
          +1 checkstyle 0m 33s the patch passed
          +1 mvnsite 0m 56s the patch passed
          +1 mvneclipse 0m 14s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 xml 0m 1s The patch has no ill-formed XML file.
          +1 findbugs 2m 24s the patch passed
          +1 javadoc 1m 2s the patch passed with JDK v1.8.0_101
          +1 javadoc 1m 47s the patch passed with JDK v1.7.0_101
          +1 unit 49m 36s hadoop-hdfs in the patch passed with JDK v1.7.0_101.
          +1 asflicense 0m 25s The patch does not generate ASF License warnings.
          154m 46s



          Reason Tests
          JDK v1.8.0_101 Failed junit tests hadoop.hdfs.server.datanode.TestDirectoryScanner



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:b59b8b7
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12821706/HDFS-4176-branch-2.003.patch
          JIRA Issue HDFS-4176
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml
          uname Linux 374c5269239f 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision branch-2 / 9850cf0
          Default Java 1.7.0_101
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_101 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_101
          findbugs v3.0.0
          JDK v1.7.0_101 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16299/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16299/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 16m 5s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 13m 23s branch-2 passed +1 compile 0m 55s branch-2 passed with JDK v1.8.0_101 +1 compile 0m 49s branch-2 passed with JDK v1.7.0_101 +1 checkstyle 0m 37s branch-2 passed +1 mvnsite 1m 37s branch-2 passed +1 mvneclipse 0m 20s branch-2 passed +1 findbugs 2m 24s branch-2 passed +1 javadoc 1m 5s branch-2 passed with JDK v1.8.0_101 +1 javadoc 1m 50s branch-2 passed with JDK v1.7.0_101 +1 mvninstall 0m 52s the patch passed +1 compile 0m 43s the patch passed with JDK v1.8.0_101 +1 javac 0m 43s the patch passed +1 compile 0m 46s the patch passed with JDK v1.7.0_101 +1 javac 0m 46s the patch passed +1 checkstyle 0m 33s the patch passed +1 mvnsite 0m 56s the patch passed +1 mvneclipse 0m 14s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 1s The patch has no ill-formed XML file. +1 findbugs 2m 24s the patch passed +1 javadoc 1m 2s the patch passed with JDK v1.8.0_101 +1 javadoc 1m 47s the patch passed with JDK v1.7.0_101 +1 unit 49m 36s hadoop-hdfs in the patch passed with JDK v1.7.0_101. +1 asflicense 0m 25s The patch does not generate ASF License warnings. 154m 46s Reason Tests JDK v1.8.0_101 Failed junit tests hadoop.hdfs.server.datanode.TestDirectoryScanner Subsystem Report/Notes Docker Image:yetus/hadoop:b59b8b7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12821706/HDFS-4176-branch-2.003.patch JIRA Issue HDFS-4176 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml uname Linux 374c5269239f 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision branch-2 / 9850cf0 Default Java 1.7.0_101 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_101 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_101 findbugs v3.0.0 JDK v1.7.0_101 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16299/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16299/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -
          +1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 11s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 7m 1s branch-2 passed
          +1 compile 0m 43s branch-2 passed with JDK v1.8.0_101
          +1 compile 0m 49s branch-2 passed with JDK v1.7.0_101
          +1 checkstyle 0m 33s branch-2 passed
          +1 mvnsite 0m 56s branch-2 passed
          +1 mvneclipse 0m 17s branch-2 passed
          +1 findbugs 2m 14s branch-2 passed
          +1 javadoc 0m 58s branch-2 passed with JDK v1.8.0_101
          +1 javadoc 1m 40s branch-2 passed with JDK v1.7.0_101
          +1 mvninstall 0m 50s the patch passed
          +1 compile 0m 40s the patch passed with JDK v1.8.0_101
          +1 javac 0m 40s the patch passed
          +1 compile 0m 45s the patch passed with JDK v1.7.0_101
          +1 javac 0m 45s the patch passed
          +1 checkstyle 0m 31s the patch passed
          +1 mvnsite 0m 54s the patch passed
          +1 mvneclipse 0m 14s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 xml 0m 0s The patch has no ill-formed XML file.
          +1 findbugs 2m 26s the patch passed
          +1 javadoc 0m 57s the patch passed with JDK v1.8.0_101
          +1 javadoc 1m 36s the patch passed with JDK v1.7.0_101
          +1 unit 51m 18s hadoop-hdfs in the patch passed with JDK v1.7.0_101.
          +1 asflicense 0m 22s The patch does not generate ASF License warnings.
          132m 8s



          Reason Tests
          JDK v1.8.0_101 Failed junit tests hadoop.hdfs.server.blockmanagement.TestPendingInvalidateBlock



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:b59b8b7
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12821706/HDFS-4176-branch-2.003.patch
          JIRA Issue HDFS-4176
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml
          uname Linux 77087e95dfe4 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision branch-2 / 9850cf0
          Default Java 1.7.0_101
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_101 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_101
          findbugs v3.0.0
          JDK v1.7.0_101 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16297/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16297/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 11s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 7m 1s branch-2 passed +1 compile 0m 43s branch-2 passed with JDK v1.8.0_101 +1 compile 0m 49s branch-2 passed with JDK v1.7.0_101 +1 checkstyle 0m 33s branch-2 passed +1 mvnsite 0m 56s branch-2 passed +1 mvneclipse 0m 17s branch-2 passed +1 findbugs 2m 14s branch-2 passed +1 javadoc 0m 58s branch-2 passed with JDK v1.8.0_101 +1 javadoc 1m 40s branch-2 passed with JDK v1.7.0_101 +1 mvninstall 0m 50s the patch passed +1 compile 0m 40s the patch passed with JDK v1.8.0_101 +1 javac 0m 40s the patch passed +1 compile 0m 45s the patch passed with JDK v1.7.0_101 +1 javac 0m 45s the patch passed +1 checkstyle 0m 31s the patch passed +1 mvnsite 0m 54s the patch passed +1 mvneclipse 0m 14s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 0s The patch has no ill-formed XML file. +1 findbugs 2m 26s the patch passed +1 javadoc 0m 57s the patch passed with JDK v1.8.0_101 +1 javadoc 1m 36s the patch passed with JDK v1.7.0_101 +1 unit 51m 18s hadoop-hdfs in the patch passed with JDK v1.7.0_101. +1 asflicense 0m 22s The patch does not generate ASF License warnings. 132m 8s Reason Tests JDK v1.8.0_101 Failed junit tests hadoop.hdfs.server.blockmanagement.TestPendingInvalidateBlock Subsystem Report/Notes Docker Image:yetus/hadoop:b59b8b7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12821706/HDFS-4176-branch-2.003.patch JIRA Issue HDFS-4176 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml uname Linux 77087e95dfe4 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision branch-2 / 9850cf0 Default Java 1.7.0_101 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_101 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_101 findbugs v3.0.0 JDK v1.7.0_101 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16297/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16297/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 0s Docker mode activated.
          -1 patch 0m 7s HDFS-4176 does not apply to branch-2.3. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help.



          Subsystem Report/Notes
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12821667/HDFS-4176-branch-2.3.patch
          JIRA Issue HDFS-4176
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16293/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. -1 patch 0m 7s HDFS-4176 does not apply to branch-2.3. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. Subsystem Report/Notes JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12821667/HDFS-4176-branch-2.3.patch JIRA Issue HDFS-4176 Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16293/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          eddyxu Lei (Eddy) Xu added a comment -

          rebase and upload

          Show
          eddyxu Lei (Eddy) Xu added a comment - rebase and upload
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 0s Docker mode activated.
          -1 patch 0m 11s HDFS-4176 does not apply to branch-2.2. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help.



          Subsystem Report/Notes
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12821662/HDFS-4176-branch-2.2.patch
          JIRA Issue HDFS-4176
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16292/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. -1 patch 0m 11s HDFS-4176 does not apply to branch-2.2. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. Subsystem Report/Notes JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12821662/HDFS-4176-branch-2.2.patch JIRA Issue HDFS-4176 Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16292/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          eddyxu Lei (Eddy) Xu added a comment -

          Update the patch, fix TestHdfsConfigFields.

          Jing Zhao would be much appreciated to have another review.

          Show
          eddyxu Lei (Eddy) Xu added a comment - Update the patch, fix TestHdfsConfigFields . Jing Zhao would be much appreciated to have another review.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 17s Docker mode activated.
          +1 @author 0m 1s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 7m 20s branch-2 passed
          +1 compile 0m 55s branch-2 passed with JDK v1.8.0_101
          +1 compile 0m 49s branch-2 passed with JDK v1.7.0_101
          +1 checkstyle 0m 32s branch-2 passed
          +1 mvnsite 0m 56s branch-2 passed
          +1 mvneclipse 0m 15s branch-2 passed
          +1 findbugs 2m 3s branch-2 passed
          +1 javadoc 1m 2s branch-2 passed with JDK v1.8.0_101
          +1 javadoc 1m 40s branch-2 passed with JDK v1.7.0_101
          +1 mvninstall 0m 46s the patch passed
          +1 compile 0m 38s the patch passed with JDK v1.8.0_101
          +1 javac 0m 38s the patch passed
          +1 compile 0m 42s the patch passed with JDK v1.7.0_101
          +1 javac 0m 42s the patch passed
          +1 checkstyle 0m 30s the patch passed
          +1 mvnsite 0m 53s the patch passed
          +1 mvneclipse 0m 12s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 xml 0m 1s The patch has no ill-formed XML file.
          +1 findbugs 2m 8s the patch passed
          +1 javadoc 0m 56s the patch passed with JDK v1.8.0_101
          +1 javadoc 1m 37s the patch passed with JDK v1.7.0_101
          -1 unit 65m 36s hadoop-hdfs in the patch failed with JDK v1.7.0_101.
          +1 asflicense 0m 21s The patch does not generate ASF License warnings.
          157m 59s



          Reason Tests
          JDK v1.8.0_101 Failed junit tests hadoop.tools.TestHdfsConfigFields
            hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure
            hadoop.hdfs.qjournal.TestSecureNNWithQJM
          JDK v1.7.0_101 Failed junit tests hadoop.tools.TestHdfsConfigFields
            hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure
            hadoop.hdfs.server.namenode.TestDecommissioningStatus



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:b59b8b7
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12821030/HDFS-4176-branch-2.1.patch
          JIRA Issue HDFS-4176
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml
          uname Linux b5e68ee57f9a 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision branch-2 / 47dd871
          Default Java 1.7.0_101
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_101 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_101
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/16256/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_101.txt
          JDK v1.7.0_101 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16256/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16256/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 17s Docker mode activated. +1 @author 0m 1s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 7m 20s branch-2 passed +1 compile 0m 55s branch-2 passed with JDK v1.8.0_101 +1 compile 0m 49s branch-2 passed with JDK v1.7.0_101 +1 checkstyle 0m 32s branch-2 passed +1 mvnsite 0m 56s branch-2 passed +1 mvneclipse 0m 15s branch-2 passed +1 findbugs 2m 3s branch-2 passed +1 javadoc 1m 2s branch-2 passed with JDK v1.8.0_101 +1 javadoc 1m 40s branch-2 passed with JDK v1.7.0_101 +1 mvninstall 0m 46s the patch passed +1 compile 0m 38s the patch passed with JDK v1.8.0_101 +1 javac 0m 38s the patch passed +1 compile 0m 42s the patch passed with JDK v1.7.0_101 +1 javac 0m 42s the patch passed +1 checkstyle 0m 30s the patch passed +1 mvnsite 0m 53s the patch passed +1 mvneclipse 0m 12s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 1s The patch has no ill-formed XML file. +1 findbugs 2m 8s the patch passed +1 javadoc 0m 56s the patch passed with JDK v1.8.0_101 +1 javadoc 1m 37s the patch passed with JDK v1.7.0_101 -1 unit 65m 36s hadoop-hdfs in the patch failed with JDK v1.7.0_101. +1 asflicense 0m 21s The patch does not generate ASF License warnings. 157m 59s Reason Tests JDK v1.8.0_101 Failed junit tests hadoop.tools.TestHdfsConfigFields   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure   hadoop.hdfs.qjournal.TestSecureNNWithQJM JDK v1.7.0_101 Failed junit tests hadoop.tools.TestHdfsConfigFields   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure   hadoop.hdfs.server.namenode.TestDecommissioningStatus Subsystem Report/Notes Docker Image:yetus/hadoop:b59b8b7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12821030/HDFS-4176-branch-2.1.patch JIRA Issue HDFS-4176 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml uname Linux b5e68ee57f9a 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision branch-2 / 47dd871 Default Java 1.7.0_101 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_101 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_101 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/16256/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_101.txt JDK v1.7.0_101 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16256/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16256/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          eddyxu Lei (Eddy) Xu added a comment -

          Hi, Jing Zhao

          I changed getNameNodeProxy to getRollEditsTask in branch-2. Could you take a look? Thanks.

          Show
          eddyxu Lei (Eddy) Xu added a comment - Hi, Jing Zhao I changed getNameNodeProxy to getRollEditsTask in branch-2. Could you take a look? Thanks.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 23s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 6m 59s branch-2 passed
          +1 compile 0m 44s branch-2 passed with JDK v1.8.0_101
          +1 compile 0m 45s branch-2 passed with JDK v1.7.0_101
          +1 checkstyle 0m 32s branch-2 passed
          +1 mvnsite 0m 53s branch-2 passed
          +1 mvneclipse 0m 16s branch-2 passed
          +1 findbugs 2m 0s branch-2 passed
          +1 javadoc 0m 55s branch-2 passed with JDK v1.8.0_101
          +1 javadoc 1m 36s branch-2 passed with JDK v1.7.0_101
          +1 mvninstall 0m 46s the patch passed
          +1 compile 0m 41s the patch passed with JDK v1.8.0_101
          +1 javac 0m 41s the patch passed
          +1 compile 0m 42s the patch passed with JDK v1.7.0_101
          +1 javac 0m 42s the patch passed
          +1 checkstyle 0m 27s the patch passed
          +1 mvnsite 0m 52s the patch passed
          +1 mvneclipse 0m 13s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 xml 0m 0s The patch has no ill-formed XML file.
          +1 findbugs 2m 8s the patch passed
          +1 javadoc 0m 53s the patch passed with JDK v1.8.0_101
          +1 javadoc 1m 33s the patch passed with JDK v1.7.0_101
          -1 unit 47m 51s hadoop-hdfs in the patch failed with JDK v1.7.0_101.
          +1 asflicense 0m 19s The patch does not generate ASF License warnings.
          125m 14s



          Reason Tests
          JDK v1.8.0_101 Failed junit tests hadoop.tools.TestHdfsConfigFields
            hadoop.hdfs.server.blockmanagement.TestComputeInvalidateWork
            hadoop.hdfs.server.datanode.TestLargeBlockReport
            hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure
          JDK v1.7.0_101 Failed junit tests hadoop.tools.TestHdfsConfigFields
            hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:b59b8b7
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12820805/HDFS-4176-branch-2.0.patch
          JIRA Issue HDFS-4176
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml
          uname Linux 892f71b69b0e 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision branch-2 / 2f854b8
          Default Java 1.7.0_101
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_101 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_101
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/16244/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_101.txt
          JDK v1.7.0_101 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16244/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16244/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 23s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 6m 59s branch-2 passed +1 compile 0m 44s branch-2 passed with JDK v1.8.0_101 +1 compile 0m 45s branch-2 passed with JDK v1.7.0_101 +1 checkstyle 0m 32s branch-2 passed +1 mvnsite 0m 53s branch-2 passed +1 mvneclipse 0m 16s branch-2 passed +1 findbugs 2m 0s branch-2 passed +1 javadoc 0m 55s branch-2 passed with JDK v1.8.0_101 +1 javadoc 1m 36s branch-2 passed with JDK v1.7.0_101 +1 mvninstall 0m 46s the patch passed +1 compile 0m 41s the patch passed with JDK v1.8.0_101 +1 javac 0m 41s the patch passed +1 compile 0m 42s the patch passed with JDK v1.7.0_101 +1 javac 0m 42s the patch passed +1 checkstyle 0m 27s the patch passed +1 mvnsite 0m 52s the patch passed +1 mvneclipse 0m 13s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 0s The patch has no ill-formed XML file. +1 findbugs 2m 8s the patch passed +1 javadoc 0m 53s the patch passed with JDK v1.8.0_101 +1 javadoc 1m 33s the patch passed with JDK v1.7.0_101 -1 unit 47m 51s hadoop-hdfs in the patch failed with JDK v1.7.0_101. +1 asflicense 0m 19s The patch does not generate ASF License warnings. 125m 14s Reason Tests JDK v1.8.0_101 Failed junit tests hadoop.tools.TestHdfsConfigFields   hadoop.hdfs.server.blockmanagement.TestComputeInvalidateWork   hadoop.hdfs.server.datanode.TestLargeBlockReport   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure JDK v1.7.0_101 Failed junit tests hadoop.tools.TestHdfsConfigFields   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure Subsystem Report/Notes Docker Image:yetus/hadoop:b59b8b7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12820805/HDFS-4176-branch-2.0.patch JIRA Issue HDFS-4176 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml uname Linux 892f71b69b0e 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision branch-2 / 2f854b8 Default Java 1.7.0_101 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_101 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_101 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/16244/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_101.txt JDK v1.7.0_101 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16244/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16244/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          jingzhao Jing Zhao added a comment -

          Thanks Lei (Eddy) Xu. The branch-2 patch also looks good to me. Maybe in branch-2's context, getNameNodeProxy can be renamed? Other than this +1.

          Show
          jingzhao Jing Zhao added a comment - Thanks Lei (Eddy) Xu . The branch-2 patch also looks good to me. Maybe in branch-2's context, getNameNodeProxy can be renamed? Other than this +1.
          Hide
          eddyxu Lei (Eddy) Xu added a comment -

          Hi, Jing Zhao

          I made a patch for branch-2. The conflicts between trunk and branch-2 are due to MultipleNameNodeProxy not exist on branch-2. So I put getActiveNodeProxy.rollEditLog() to a simple Callable#call().

          Would you mind to take another look? Thanks

          Show
          eddyxu Lei (Eddy) Xu added a comment - Hi, Jing Zhao I made a patch for branch-2 . The conflicts between trunk and branch-2 are due to MultipleNameNodeProxy not exist on branch-2. So I put getActiveNodeProxy.rollEditLog() to a simple Callable#call() . Would you mind to take another look? Thanks
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-trunk-Commit #10171 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10171/)
          HDFS-4176. EditLogTailer should call rollEdits with a timeout. (Lei Xu) (lei: rev 67406460f0b6c05edde1d1185aeb42b6324df202)

          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/ha/EditLogTailer.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestEditLogTailer.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-trunk-Commit #10171 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10171/ ) HDFS-4176 . EditLogTailer should call rollEdits with a timeout. (Lei Xu) (lei: rev 67406460f0b6c05edde1d1185aeb42b6324df202) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/ha/EditLogTailer.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestEditLogTailer.java hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
          Hide
          jingzhao Jing Zhao added a comment -

          +1 on the rebased patch.

          Show
          jingzhao Jing Zhao added a comment - +1 on the rebased patch.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 12s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 9m 1s trunk passed
          +1 compile 0m 57s trunk passed
          +1 checkstyle 0m 33s trunk passed
          +1 mvnsite 1m 5s trunk passed
          +1 mvneclipse 0m 15s trunk passed
          +1 findbugs 2m 2s trunk passed
          +1 javadoc 0m 59s trunk passed
          +1 mvninstall 1m 2s the patch passed
          +1 compile 0m 58s the patch passed
          +1 javac 0m 58s the patch passed
          +1 checkstyle 0m 32s hadoop-hdfs-project/hadoop-hdfs: The patch generated 0 new + 424 unchanged - 2 fixed = 424 total (was 426)
          +1 mvnsite 1m 3s the patch passed
          +1 mvneclipse 0m 11s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 xml 0m 2s The patch has no ill-formed XML file.
          +1 findbugs 2m 6s the patch passed
          +1 javadoc 0m 56s the patch passed
          -1 unit 71m 23s hadoop-hdfs in the patch failed.
          +1 asflicense 0m 19s The patch does not generate ASF License warnings.
          95m 1s



          Reason Tests
          Failed junit tests hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistReplicaRecovery
            hadoop.hdfs.server.datanode.TestDataNodeErasureCodingMetrics



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12820739/HDFS-4176.04.patch
          JIRA Issue HDFS-4176
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml
          uname Linux 623876c7d94c 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 26de4f0
          Default Java 1.8.0_101
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/16240/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16240/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16240/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 12s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 9m 1s trunk passed +1 compile 0m 57s trunk passed +1 checkstyle 0m 33s trunk passed +1 mvnsite 1m 5s trunk passed +1 mvneclipse 0m 15s trunk passed +1 findbugs 2m 2s trunk passed +1 javadoc 0m 59s trunk passed +1 mvninstall 1m 2s the patch passed +1 compile 0m 58s the patch passed +1 javac 0m 58s the patch passed +1 checkstyle 0m 32s hadoop-hdfs-project/hadoop-hdfs: The patch generated 0 new + 424 unchanged - 2 fixed = 424 total (was 426) +1 mvnsite 1m 3s the patch passed +1 mvneclipse 0m 11s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 2s The patch has no ill-formed XML file. +1 findbugs 2m 6s the patch passed +1 javadoc 0m 56s the patch passed -1 unit 71m 23s hadoop-hdfs in the patch failed. +1 asflicense 0m 19s The patch does not generate ASF License warnings. 95m 1s Reason Tests Failed junit tests hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistReplicaRecovery   hadoop.hdfs.server.datanode.TestDataNodeErasureCodingMetrics Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12820739/HDFS-4176.04.patch JIRA Issue HDFS-4176 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml uname Linux 623876c7d94c 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 26de4f0 Default Java 1.8.0_101 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/16240/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16240/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16240/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          eddyxu Lei (Eddy) Xu added a comment -

          Sure, Jing Zhao

          Updated the patch to fix the conflicts from HDFS-10519.

          Show
          eddyxu Lei (Eddy) Xu added a comment - Sure, Jing Zhao Updated the patch to fix the conflicts from HDFS-10519 .
          Hide
          jingzhao Jing Zhao added a comment -

          Lei (Eddy) Xu, looks like the patch needs a rebase due to the recent commit of HDFS-10519. Could you please update the patch accordingly? Thanks!

          Show
          jingzhao Jing Zhao added a comment - Lei (Eddy) Xu , looks like the patch needs a rebase due to the recent commit of HDFS-10519 . Could you please update the patch accordingly? Thanks!
          Hide
          eddyxu Lei (Eddy) Xu added a comment -

          Thanks for the review and commit, Jing Zhao!

          Show
          eddyxu Lei (Eddy) Xu added a comment - Thanks for the review and commit, Jing Zhao !
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 0s Docker mode activated.
          -1 patch 0m 5s HDFS-4176 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help.



          Subsystem Report/Notes
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12820581/HDFS-4176.03.patch
          JIRA Issue HDFS-4176
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16227/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. -1 patch 0m 5s HDFS-4176 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. Subsystem Report/Notes JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12820581/HDFS-4176.03.patch JIRA Issue HDFS-4176 Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16227/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          jingzhao Jing Zhao added a comment -

          TestHDFSCLI failure is tracked by HDFS-10696, the other two tests passed in my local machine and the failures should be unrelated. +1 on the latest patch. Thanks for the contribution, Lei (Eddy) Xu!

          Show
          jingzhao Jing Zhao added a comment - TestHDFSCLI failure is tracked by HDFS-10696 , the other two tests passed in my local machine and the failures should be unrelated. +1 on the latest patch. Thanks for the contribution, Lei (Eddy) Xu !
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 13s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 6m 55s trunk passed
          +1 compile 0m 45s trunk passed
          +1 checkstyle 0m 30s trunk passed
          +1 mvnsite 0m 51s trunk passed
          +1 mvneclipse 0m 12s trunk passed
          +1 findbugs 1m 44s trunk passed
          +1 javadoc 0m 56s trunk passed
          +1 mvninstall 0m 53s the patch passed
          +1 compile 0m 45s the patch passed
          +1 javac 0m 45s the patch passed
          +1 checkstyle 0m 28s hadoop-hdfs-project/hadoop-hdfs: The patch generated 0 new + 424 unchanged - 2 fixed = 424 total (was 426)
          +1 mvnsite 0m 51s the patch passed
          +1 mvneclipse 0m 10s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 xml 0m 1s The patch has no ill-formed XML file.
          +1 findbugs 2m 5s the patch passed
          +1 javadoc 0m 58s the patch passed
          -1 unit 73m 56s hadoop-hdfs in the patch failed.
          +1 asflicense 0m 20s The patch does not generate ASF License warnings.
          93m 57s



          Reason Tests
          Failed junit tests hadoop.cli.TestHDFSCLI
            hadoop.hdfs.server.datanode.TestDataNodeLifeline
          Timed out junit tests org.apache.hadoop.hdfs.TestLeaseRecovery2



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12820581/HDFS-4176.03.patch
          JIRA Issue HDFS-4176
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml
          uname Linux 1597bf5fcd77 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / eb7ff0c
          Default Java 1.8.0_101
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/16220/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16220/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16220/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 13s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 6m 55s trunk passed +1 compile 0m 45s trunk passed +1 checkstyle 0m 30s trunk passed +1 mvnsite 0m 51s trunk passed +1 mvneclipse 0m 12s trunk passed +1 findbugs 1m 44s trunk passed +1 javadoc 0m 56s trunk passed +1 mvninstall 0m 53s the patch passed +1 compile 0m 45s the patch passed +1 javac 0m 45s the patch passed +1 checkstyle 0m 28s hadoop-hdfs-project/hadoop-hdfs: The patch generated 0 new + 424 unchanged - 2 fixed = 424 total (was 426) +1 mvnsite 0m 51s the patch passed +1 mvneclipse 0m 10s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 1s The patch has no ill-formed XML file. +1 findbugs 2m 5s the patch passed +1 javadoc 0m 58s the patch passed -1 unit 73m 56s hadoop-hdfs in the patch failed. +1 asflicense 0m 20s The patch does not generate ASF License warnings. 93m 57s Reason Tests Failed junit tests hadoop.cli.TestHDFSCLI   hadoop.hdfs.server.datanode.TestDataNodeLifeline Timed out junit tests org.apache.hadoop.hdfs.TestLeaseRecovery2 Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12820581/HDFS-4176.03.patch JIRA Issue HDFS-4176 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml uname Linux 1597bf5fcd77 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / eb7ff0c Default Java 1.8.0_101 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/16220/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16220/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16220/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 14s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 8m 46s trunk passed
          +1 compile 0m 54s trunk passed
          +1 checkstyle 0m 30s trunk passed
          +1 mvnsite 1m 11s trunk passed
          +1 mvneclipse 0m 13s trunk passed
          +1 findbugs 1m 59s trunk passed
          +1 javadoc 1m 1s trunk passed
          +1 mvninstall 1m 5s the patch passed
          +1 compile 1m 0s the patch passed
          +1 javac 1m 0s the patch passed
          +1 checkstyle 0m 32s hadoop-hdfs-project/hadoop-hdfs: The patch generated 0 new + 424 unchanged - 2 fixed = 424 total (was 426)
          +1 mvnsite 1m 6s the patch passed
          +1 mvneclipse 0m 9s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 xml 0m 1s The patch has no ill-formed XML file.
          +1 findbugs 1m 50s the patch passed
          +1 javadoc 0m 56s the patch passed
          -1 unit 77m 29s hadoop-hdfs in the patch failed.
          +1 asflicense 0m 18s The patch does not generate ASF License warnings.
          100m 31s



          Reason Tests
          Failed junit tests hadoop.cli.TestHDFSCLI
            hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure
            hadoop.hdfs.server.namenode.TestEditLog



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12820576/HDFS-4176.02.patch
          JIRA Issue HDFS-4176
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml
          uname Linux aa44a1be660d 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / eb7ff0c
          Default Java 1.8.0_101
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/16219/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16219/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16219/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 14s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 8m 46s trunk passed +1 compile 0m 54s trunk passed +1 checkstyle 0m 30s trunk passed +1 mvnsite 1m 11s trunk passed +1 mvneclipse 0m 13s trunk passed +1 findbugs 1m 59s trunk passed +1 javadoc 1m 1s trunk passed +1 mvninstall 1m 5s the patch passed +1 compile 1m 0s the patch passed +1 javac 1m 0s the patch passed +1 checkstyle 0m 32s hadoop-hdfs-project/hadoop-hdfs: The patch generated 0 new + 424 unchanged - 2 fixed = 424 total (was 426) +1 mvnsite 1m 6s the patch passed +1 mvneclipse 0m 9s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 1s The patch has no ill-formed XML file. +1 findbugs 1m 50s the patch passed +1 javadoc 0m 56s the patch passed -1 unit 77m 29s hadoop-hdfs in the patch failed. +1 asflicense 0m 18s The patch does not generate ASF License warnings. 100m 31s Reason Tests Failed junit tests hadoop.cli.TestHDFSCLI   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure   hadoop.hdfs.server.namenode.TestEditLog Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12820576/HDFS-4176.02.patch JIRA Issue HDFS-4176 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml uname Linux aa44a1be660d 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / eb7ff0c Default Java 1.8.0_101 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/16219/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16219/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16219/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          eddyxu Lei (Eddy) Xu added a comment -

          Thanks, Jing Zhao.

          Updated the patch to use ThreadFactoryBuilder.

          Show
          eddyxu Lei (Eddy) Xu added a comment - Thanks, Jing Zhao . Updated the patch to use ThreadFactoryBuilder .
          Hide
          jingzhao Jing Zhao added a comment -

          Thanks for addressing the comments, Lei (Eddy) Xu

          Nit: We can use ThreadFactoryBuilder to simplify the following code. Other than this the patch looks good to me.

          177	    rollEditsRpcExecutor = Executors.newSingleThreadExecutor(
          178	        new ThreadFactory() {
          179	          @Override
          180	          public Thread newThread(Runnable r) {
          181	            Thread thread = Executors.defaultThreadFactory().newThread(r);
          182	            thread.setDaemon(true);
          183	            return thread;
          184	          }
          185	        });
          
          Show
          jingzhao Jing Zhao added a comment - Thanks for addressing the comments, Lei (Eddy) Xu Nit: We can use ThreadFactoryBuilder to simplify the following code. Other than this the patch looks good to me. 177 rollEditsRpcExecutor = Executors.newSingleThreadExecutor( 178 new ThreadFactory() { 179 @Override 180 public Thread newThread( Runnable r) { 181 Thread thread = Executors.defaultThreadFactory().newThread(r); 182 thread.setDaemon( true ); 183 return thread; 184 } 185 });
          Hide
          eddyxu Lei (Eddy) Xu added a comment -

          Thanks a lot for the great inputs, Jing Zhao

          I have updated the patch to address all your comments.

          The test failures are not relevant. TestDFSCLI fails on trunk as well.

          Show
          eddyxu Lei (Eddy) Xu added a comment - Thanks a lot for the great inputs, Jing Zhao I have updated the patch to address all your comments. The test failures are not relevant. TestDFSCLI fails on trunk as well.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 14s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 6m 31s trunk passed
          +1 compile 0m 44s trunk passed
          +1 checkstyle 0m 29s trunk passed
          +1 mvnsite 0m 51s trunk passed
          +1 mvneclipse 0m 12s trunk passed
          +1 findbugs 1m 40s trunk passed
          +1 javadoc 0m 55s trunk passed
          +1 mvninstall 0m 45s the patch passed
          +1 compile 0m 42s the patch passed
          +1 javac 0m 42s the patch passed
          +1 checkstyle 0m 28s hadoop-hdfs-project/hadoop-hdfs: The patch generated 0 new + 424 unchanged - 2 fixed = 424 total (was 426)
          +1 mvnsite 0m 47s the patch passed
          +1 mvneclipse 0m 10s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 xml 0m 1s The patch has no ill-formed XML file.
          +1 findbugs 1m 45s the patch passed
          +1 javadoc 0m 52s the patch passed
          -1 unit 57m 31s hadoop-hdfs in the patch failed.
          +1 asflicense 0m 18s The patch does not generate ASF License warnings.
          76m 6s



          Reason Tests
          Failed junit tests hadoop.hdfs.TestDFSShell
            hadoop.cli.TestHDFSCLI
            hadoop.hdfs.server.blockmanagement.TestPendingInvalidateBlock



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12820546/HDFS-4176.01.patch
          JIRA Issue HDFS-4176
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml
          uname Linux 8616328fb9d1 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 54fe17a
          Default Java 1.8.0_101
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/16218/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16218/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16218/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 14s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 6m 31s trunk passed +1 compile 0m 44s trunk passed +1 checkstyle 0m 29s trunk passed +1 mvnsite 0m 51s trunk passed +1 mvneclipse 0m 12s trunk passed +1 findbugs 1m 40s trunk passed +1 javadoc 0m 55s trunk passed +1 mvninstall 0m 45s the patch passed +1 compile 0m 42s the patch passed +1 javac 0m 42s the patch passed +1 checkstyle 0m 28s hadoop-hdfs-project/hadoop-hdfs: The patch generated 0 new + 424 unchanged - 2 fixed = 424 total (was 426) +1 mvnsite 0m 47s the patch passed +1 mvneclipse 0m 10s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 1s The patch has no ill-formed XML file. +1 findbugs 1m 45s the patch passed +1 javadoc 0m 52s the patch passed -1 unit 57m 31s hadoop-hdfs in the patch failed. +1 asflicense 0m 18s The patch does not generate ASF License warnings. 76m 6s Reason Tests Failed junit tests hadoop.hdfs.TestDFSShell   hadoop.cli.TestHDFSCLI   hadoop.hdfs.server.blockmanagement.TestPendingInvalidateBlock Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12820546/HDFS-4176.01.patch JIRA Issue HDFS-4176 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml uname Linux 8616328fb9d1 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 54fe17a Default Java 1.8.0_101 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/16218/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16218/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16218/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          jingzhao Jing Zhao added a comment -

          Thanks for working on this, Lei (Eddy) Xu! The patch looks good to me overall. Some minor comments:

          1. we should make sure the thread in the executor is daemon thread.
            327	    ExecutorService executor = Executors.newSingleThreadExecutor();
            
          2. We can try to reuse the ExecutorService, or for the current code we should call shutdown in the finally block.
          3. It's better to separate the exceptions in different catch sections, since they have different handling logic.
            333	    } catch (InterruptedException | ExecutionException | TimeoutException e) {
            
          Show
          jingzhao Jing Zhao added a comment - Thanks for working on this, Lei (Eddy) Xu ! The patch looks good to me overall. Some minor comments: we should make sure the thread in the executor is daemon thread. 327 ExecutorService executor = Executors.newSingleThreadExecutor(); We can try to reuse the ExecutorService, or for the current code we should call shutdown in the finally block. It's better to separate the exceptions in different catch sections, since they have different handling logic. 333 } catch (InterruptedException | ExecutionException | TimeoutException e) {
          Hide
          eddyxu Lei (Eddy) Xu added a comment -

          Fix check style errors and test failures.

          Show
          eddyxu Lei (Eddy) Xu added a comment - Fix check style errors and test failures.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 13s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 7m 46s trunk passed
          +1 compile 0m 49s trunk passed
          +1 checkstyle 0m 31s trunk passed
          +1 mvnsite 0m 56s trunk passed
          +1 mvneclipse 0m 12s trunk passed
          +1 findbugs 1m 51s trunk passed
          +1 javadoc 0m 57s trunk passed
          +1 mvninstall 0m 49s the patch passed
          +1 compile 0m 47s the patch passed
          +1 javac 0m 47s the patch passed
          -0 checkstyle 0m 29s hadoop-hdfs-project/hadoop-hdfs: The patch generated 4 new + 424 unchanged - 2 fixed = 428 total (was 426)
          +1 mvnsite 0m 55s the patch passed
          +1 mvneclipse 0m 10s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 1m 57s the patch passed
          +1 javadoc 0m 55s the patch passed
          -1 unit 70m 53s hadoop-hdfs in the patch failed.
          +1 asflicense 0m 24s The patch does not generate ASF License warnings.
          91m 58s



          Reason Tests
          Failed junit tests hadoop.cli.TestHDFSCLI
            hadoop.tools.TestHdfsConfigFields



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12820507/HDFS-4176.00.patch
          JIRA Issue HDFS-4176
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 4439d1a7da37 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 54fe17a
          Default Java 1.8.0_101
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/16213/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/16213/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16213/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16213/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 13s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 7m 46s trunk passed +1 compile 0m 49s trunk passed +1 checkstyle 0m 31s trunk passed +1 mvnsite 0m 56s trunk passed +1 mvneclipse 0m 12s trunk passed +1 findbugs 1m 51s trunk passed +1 javadoc 0m 57s trunk passed +1 mvninstall 0m 49s the patch passed +1 compile 0m 47s the patch passed +1 javac 0m 47s the patch passed -0 checkstyle 0m 29s hadoop-hdfs-project/hadoop-hdfs: The patch generated 4 new + 424 unchanged - 2 fixed = 428 total (was 426) +1 mvnsite 0m 55s the patch passed +1 mvneclipse 0m 10s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 57s the patch passed +1 javadoc 0m 55s the patch passed -1 unit 70m 53s hadoop-hdfs in the patch failed. +1 asflicense 0m 24s The patch does not generate ASF License warnings. 91m 58s Reason Tests Failed junit tests hadoop.cli.TestHDFSCLI   hadoop.tools.TestHdfsConfigFields Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12820507/HDFS-4176.00.patch JIRA Issue HDFS-4176 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 4439d1a7da37 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 54fe17a Default Java 1.8.0_101 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/16213/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/16213/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16213/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16213/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 14s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 7m 13s trunk passed
          +1 compile 0m 44s trunk passed
          +1 checkstyle 0m 31s trunk passed
          +1 mvnsite 0m 51s trunk passed
          +1 mvneclipse 0m 12s trunk passed
          +1 findbugs 1m 44s trunk passed
          +1 javadoc 0m 55s trunk passed
          +1 mvninstall 0m 45s the patch passed
          +1 compile 0m 42s the patch passed
          +1 javac 0m 42s the patch passed
          -0 checkstyle 0m 28s hadoop-hdfs-project/hadoop-hdfs: The patch generated 4 new + 424 unchanged - 2 fixed = 428 total (was 426)
          +1 mvnsite 0m 49s the patch passed
          +1 mvneclipse 0m 9s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 1m 45s the patch passed
          +1 javadoc 0m 52s the patch passed
          -1 unit 56m 48s hadoop-hdfs in the patch failed.
          +1 asflicense 0m 17s The patch does not generate ASF License warnings.
          76m 10s



          Reason Tests
          Failed junit tests hadoop.cli.TestHDFSCLI
            hadoop.hdfs.TestDistributedFileSystem
            hadoop.tools.TestHdfsConfigFields
            hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFSStriped



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12820507/HDFS-4176.00.patch
          JIRA Issue HDFS-4176
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 3abda960bb48 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 54fe17a
          Default Java 1.8.0_101
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/16212/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/16212/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16212/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16212/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 14s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 7m 13s trunk passed +1 compile 0m 44s trunk passed +1 checkstyle 0m 31s trunk passed +1 mvnsite 0m 51s trunk passed +1 mvneclipse 0m 12s trunk passed +1 findbugs 1m 44s trunk passed +1 javadoc 0m 55s trunk passed +1 mvninstall 0m 45s the patch passed +1 compile 0m 42s the patch passed +1 javac 0m 42s the patch passed -0 checkstyle 0m 28s hadoop-hdfs-project/hadoop-hdfs: The patch generated 4 new + 424 unchanged - 2 fixed = 428 total (was 426) +1 mvnsite 0m 49s the patch passed +1 mvneclipse 0m 9s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 45s the patch passed +1 javadoc 0m 52s the patch passed -1 unit 56m 48s hadoop-hdfs in the patch failed. +1 asflicense 0m 17s The patch does not generate ASF License warnings. 76m 10s Reason Tests Failed junit tests hadoop.cli.TestHDFSCLI   hadoop.hdfs.TestDistributedFileSystem   hadoop.tools.TestHdfsConfigFields   hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFSStriped Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12820507/HDFS-4176.00.patch JIRA Issue HDFS-4176 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 3abda960bb48 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 54fe17a Default Java 1.8.0_101 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/16212/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/16212/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16212/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16212/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          eddyxu Lei (Eddy) Xu added a comment -

          Upload a patch to use Future to wrap rollEdit with a configurable timeout.

          Show
          eddyxu Lei (Eddy) Xu added a comment - Upload a patch to use Future to wrap rollEdit with a configurable timeout.
          Hide
          eddyxu Lei (Eddy) Xu added a comment -

          it was not related to HDFS-9659.

          Show
          eddyxu Lei (Eddy) Xu added a comment - it was not related to HDFS-9659 .
          Hide
          eddyxu Lei (Eddy) Xu added a comment -

          HDFS-9659 adds timeout to RPC calls, thus fixed this issue.

          Show
          eddyxu Lei (Eddy) Xu added a comment - HDFS-9659 adds timeout to RPC calls, thus fixed this issue.
          Hide
          glottis Marc Heide added a comment -

          jstack of standby NameNode trying to get active

          Show
          glottis Marc Heide added a comment - jstack of standby NameNode trying to get active
          Hide
          glottis Marc Heide added a comment -

          So what became of this error?

          I am pretty that we have observed exactly this problem on one of our test clusters using Cloudera 4.5 (hadoop 2.0.0-cdh4.5.0) release in a Quorum Based HA mode. For a test we intentionally destroyed one of the active namenode's disks using Linux dd command (yeah, its ugly but so is life). The poor thing got stuck in an IO operation trying to close a file. The thread which got blocked, held locks which blocked then a lot of other threads (e.g. threads for incoming RPC calls). That had a fatal impact on the whole cluster, since everything stopped to work at once. HBase, HDFS and all commands did not work and either came back with a timeout or simply hang forever. Unfortunately the live checks from ZKFC seemed to work just fine, so the ZKFC did not detect failures and hence did not trigger a failover.

          So we tried to stop it manually. After doing a kill -2 and then a kill -9 on the NameNode process the ZKFC finally detected the error and tried to activate the standby NameNode on another machine. But this got stuck too. I have attached the pstack of this NameNode process as he tries to get active but never made it. As far as I can see he is not able to stop the EditLogTailerThread.

          The root cause is probably that the formerly active NameNode was not really dead. After searching around for some time we found that he had left a zombie (defunct process) running, which held the Port 8020 opened! You cannot kill such zombies in Linux without a reboot. So this is exaclty the situation described here. Former NN was frozen but not really dead. And the standby could not go active.

          Another sad story is that even the restart of this standby NameNode did not work. It became active, thats fine. But as long as this other zombie was running and kept his 8020 port open, all clients got stuck, so neither HBase started properly, nor could we access the HDFS with the dfs client commands. Just as we rebooted the former NN's machine, the cluster started up properly. But this is probably not part of this Jira. So working with interruptible RPC calls and using a timeout everywhere seems to be vital.

          Show
          glottis Marc Heide added a comment - So what became of this error? I am pretty that we have observed exactly this problem on one of our test clusters using Cloudera 4.5 (hadoop 2.0.0-cdh4.5.0) release in a Quorum Based HA mode. For a test we intentionally destroyed one of the active namenode's disks using Linux dd command (yeah, its ugly but so is life). The poor thing got stuck in an IO operation trying to close a file. The thread which got blocked, held locks which blocked then a lot of other threads (e.g. threads for incoming RPC calls). That had a fatal impact on the whole cluster, since everything stopped to work at once. HBase, HDFS and all commands did not work and either came back with a timeout or simply hang forever. Unfortunately the live checks from ZKFC seemed to work just fine, so the ZKFC did not detect failures and hence did not trigger a failover. So we tried to stop it manually. After doing a kill -2 and then a kill -9 on the NameNode process the ZKFC finally detected the error and tried to activate the standby NameNode on another machine. But this got stuck too. I have attached the pstack of this NameNode process as he tries to get active but never made it. As far as I can see he is not able to stop the EditLogTailerThread. The root cause is probably that the formerly active NameNode was not really dead. After searching around for some time we found that he had left a zombie (defunct process) running, which held the Port 8020 opened! You cannot kill such zombies in Linux without a reboot. So this is exaclty the situation described here. Former NN was frozen but not really dead. And the standby could not go active. Another sad story is that even the restart of this standby NameNode did not work. It became active, thats fine. But as long as this other zombie was running and kept his 8020 port open, all clients got stuck, so neither HBase started properly, nor could we access the HDFS with the dfs client commands. Just as we rebooted the former NN's machine, the cluster started up properly. But this is probably not part of this Jira. So working with interruptible RPC calls and using a timeout everywhere seems to be vital.

            People

            • Assignee:
              eddyxu Lei (Eddy) Xu
              Reporter:
              tlipcon Todd Lipcon
            • Votes:
              0 Vote for this issue
              Watchers:
              17 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development