Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-9347

Invariant assumption in TestQuorumJournalManager.shutdown() is wrong

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 2.7.3, 2.6.5, 3.0.0-alpha1
    • Component/s: test
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      The code

      TestTestQuorumJournalManager.java
      @After
        public void shutdown() throws IOException {
          IOUtils.cleanup(LOG, toClose.toArray(new Closeable[0]));
          
          // Should not leak clients between tests -- this can cause flaky tests.
          // (See HDFS-4643)
          GenericTestUtils.assertNoThreadsMatching(".*IPC Client.*");
          
          if (cluster != null) {
            cluster.shutdown();
          }
        }
      

      implicitly assumes when the call returns from IOUtils.cleanup() (which calls close() on QuorumJournalManager object), all IPC client connection threads are terminated. However, there is no internal implementation that enforces this assumption. Even if the bug reported in HADOOP-12532 is fixed, the internal code still only ensures IPC connections are terminated, but not the thread.

      1. HDFS-9347.006.patch
        5 kB
        Wei-Chiu Chuang
      2. HDFS-9347.005.patch
        5 kB
        Wei-Chiu Chuang
      3. HDFS-9347.004.patch
        5 kB
        Wei-Chiu Chuang
      4. HDFS-9347.003.patch
        6 kB
        Wei-Chiu Chuang
      5. HDFS-9347.002.patch
        6 kB
        Wei-Chiu Chuang
      6. HDFS-9347.001.patch
        3 kB
        Wei-Chiu Chuang

        Issue Links

          Activity

          Hide
          jojochuang Wei-Chiu Chuang added a comment -

          Rev1: After calling IOUtils.cleanup(), periodically check and wait for all ipc client threads to finish. This patch does not require the fix for HADOOP-12532.

          Show
          jojochuang Wei-Chiu Chuang added a comment - Rev1: After calling IOUtils.cleanup(), periodically check and wait for all ipc client threads to finish. This patch does not require the fix for HADOOP-12532 .
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 12s docker + precommit patch detected.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 4m 3s trunk passed
          +1 compile 0m 55s trunk passed with JDK v1.8.0_66
          +1 compile 0m 47s trunk passed with JDK v1.7.0_79
          +1 checkstyle 0m 20s trunk passed
          +1 mvnsite 0m 59s trunk passed
          +1 mvneclipse 0m 19s trunk passed
          +1 findbugs 2m 25s trunk passed
          +1 javadoc 1m 38s trunk passed with JDK v1.8.0_66
          +1 javadoc 2m 29s trunk passed with JDK v1.7.0_79
          +1 mvninstall 0m 56s the patch passed
          +1 compile 0m 50s the patch passed with JDK v1.8.0_66
          +1 javac 0m 50s the patch passed
          +1 compile 0m 43s the patch passed with JDK v1.7.0_79
          +1 javac 0m 43s the patch passed
          +1 checkstyle 0m 20s the patch passed
          +1 mvnsite 0m 57s the patch passed
          +1 mvneclipse 0m 18s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 findbugs 2m 39s the patch passed
          +1 javadoc 1m 36s the patch passed with JDK v1.8.0_66
          +1 javadoc 2m 30s the patch passed with JDK v1.7.0_79
          -1 unit 85m 30s hadoop-hdfs in the patch failed with JDK v1.8.0_66.
          -1 unit 71m 41s hadoop-hdfs in the patch failed with JDK v1.7.0_79.
          -1 asflicense 0m 22s Patch generated 56 ASF License warnings.
          185m 54s



          Reason Tests
          JDK v1.8.0_66 Failed junit tests hadoop.hdfs.web.TestWebHdfsTimeouts
            hadoop.hdfs.server.datanode.TestBlockScanner
            hadoop.hdfs.server.namenode.ha.TestEditLogTailer
            hadoop.hdfs.security.TestDelegationTokenForProxyUser
            hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure
            hadoop.hdfs.server.datanode.TestDataNodeHotSwapVolumes
            hadoop.hdfs.server.datanode.TestBlockReplacement
            hadoop.hdfs.server.namenode.ha.TestSeveralNameNodes
            hadoop.hdfs.server.namenode.ha.TestHAAppend
            hadoop.hdfs.TestDFSStripedOutputStreamWithFailure030
            hadoop.hdfs.server.datanode.TestDirectoryScanner
          JDK v1.7.0_79 Failed junit tests hadoop.hdfs.server.namenode.ha.TestEditLogTailer
            hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots
            hadoop.hdfs.server.namenode.TestBackupNode



          Subsystem Report/Notes
          Docker Client=1.7.0 Server=1.7.0 Image:test-patch-base-hadoop-date2015-11-14
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12771403/HDFS-9347.001.patch
          JIRA Issue HDFS-9347
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 4d4cf5e64713 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HDFS-Build/patchprocess/apache-yetus-fa12328/precommit/personality/hadoop.sh
          git revision trunk / c753617
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/13509/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/13509/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_79.txt
          unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/13509/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HDFS-Build/13509/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_79.txt
          JDK v1.7.0_79 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/13509/testReport/
          asflicense https://builds.apache.org/job/PreCommit-HDFS-Build/13509/artifact/patchprocess/patch-asflicense-problems.txt
          modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
          Max memory used 227MB
          Powered by Apache Yetus http://yetus.apache.org
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/13509/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 12s docker + precommit patch detected. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 4m 3s trunk passed +1 compile 0m 55s trunk passed with JDK v1.8.0_66 +1 compile 0m 47s trunk passed with JDK v1.7.0_79 +1 checkstyle 0m 20s trunk passed +1 mvnsite 0m 59s trunk passed +1 mvneclipse 0m 19s trunk passed +1 findbugs 2m 25s trunk passed +1 javadoc 1m 38s trunk passed with JDK v1.8.0_66 +1 javadoc 2m 29s trunk passed with JDK v1.7.0_79 +1 mvninstall 0m 56s the patch passed +1 compile 0m 50s the patch passed with JDK v1.8.0_66 +1 javac 0m 50s the patch passed +1 compile 0m 43s the patch passed with JDK v1.7.0_79 +1 javac 0m 43s the patch passed +1 checkstyle 0m 20s the patch passed +1 mvnsite 0m 57s the patch passed +1 mvneclipse 0m 18s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 2m 39s the patch passed +1 javadoc 1m 36s the patch passed with JDK v1.8.0_66 +1 javadoc 2m 30s the patch passed with JDK v1.7.0_79 -1 unit 85m 30s hadoop-hdfs in the patch failed with JDK v1.8.0_66. -1 unit 71m 41s hadoop-hdfs in the patch failed with JDK v1.7.0_79. -1 asflicense 0m 22s Patch generated 56 ASF License warnings. 185m 54s Reason Tests JDK v1.8.0_66 Failed junit tests hadoop.hdfs.web.TestWebHdfsTimeouts   hadoop.hdfs.server.datanode.TestBlockScanner   hadoop.hdfs.server.namenode.ha.TestEditLogTailer   hadoop.hdfs.security.TestDelegationTokenForProxyUser   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure   hadoop.hdfs.server.datanode.TestDataNodeHotSwapVolumes   hadoop.hdfs.server.datanode.TestBlockReplacement   hadoop.hdfs.server.namenode.ha.TestSeveralNameNodes   hadoop.hdfs.server.namenode.ha.TestHAAppend   hadoop.hdfs.TestDFSStripedOutputStreamWithFailure030   hadoop.hdfs.server.datanode.TestDirectoryScanner JDK v1.7.0_79 Failed junit tests hadoop.hdfs.server.namenode.ha.TestEditLogTailer   hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots   hadoop.hdfs.server.namenode.TestBackupNode Subsystem Report/Notes Docker Client=1.7.0 Server=1.7.0 Image:test-patch-base-hadoop-date2015-11-14 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12771403/HDFS-9347.001.patch JIRA Issue HDFS-9347 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 4d4cf5e64713 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HDFS-Build/patchprocess/apache-yetus-fa12328/precommit/personality/hadoop.sh git revision trunk / c753617 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/13509/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/13509/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_79.txt unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/13509/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HDFS-Build/13509/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_79.txt JDK v1.7.0_79 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/13509/testReport/ asflicense https://builds.apache.org/job/PreCommit-HDFS-Build/13509/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Max memory used 227MB Powered by Apache Yetus http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HDFS-Build/13509/console This message was automatically generated.
          Hide
          walter.k.su Walter Su added a comment -

          How about change assertNoThreadsMatching to a waitFor style? Here we need a waitFor with a longer timeout. In the other places, we use a waitFor with zero timeout so we keep the same logic.

          Show
          walter.k.su Walter Su added a comment - How about change assertNoThreadsMatching to a waitFor style? Here we need a waitFor with a longer timeout. In the other places, we use a waitFor with zero timeout so we keep the same logic.
          Hide
          jojochuang Wei-Chiu Chuang added a comment -

          Thanks for the comment. My rev01 patch did use waitFor() to wait for all IPC client threads to be destroyed. I still kept assertNoThreadsMatching() but this is actually not needed (just to test for regressions)

          Show
          jojochuang Wei-Chiu Chuang added a comment - Thanks for the comment. My rev01 patch did use waitFor() to wait for all IPC client threads to be destroyed. I still kept assertNoThreadsMatching() but this is actually not needed (just to test for regressions)
          Hide
          walter.k.su Walter Su added a comment -

          I know. It's just the 2 pieces of code is similar. We can remove the duplication. And changing it to a waitFor as a util func kept in GenericTestUtils could be useful for other tests.

          Show
          walter.k.su Walter Su added a comment - I know. It's just the 2 pieces of code is similar. We can remove the duplication. And changing it to a waitFor as a util func kept in GenericTestUtils could be useful for other tests.
          Hide
          jojochuang Wei-Chiu Chuang added a comment -

          Thanks Walter Su for the suggestion!
          I refactored the code to reduce duplication in my rev02 code.

          FYI, there are two currently two tests in TestCheckpoint and TestEditLogAutoroll that use GenericTestUtils.assertNoThreadsMatching().

          Show
          jojochuang Wei-Chiu Chuang added a comment - Thanks Walter Su for the suggestion! I refactored the code to reduce duplication in my rev02 code. FYI, there are two currently two tests in TestCheckpoint and TestEditLogAutoroll that use GenericTestUtils.assertNoThreadsMatching().
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 11m 13s docker + precommit patch detected.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
          +1 mvninstall 9m 5s trunk passed
          +1 compile 4m 16s trunk passed with JDK v1.8.0_66
          +1 compile 4m 8s trunk passed with JDK v1.7.0_85
          +1 checkstyle 0m 58s trunk passed
          +1 mvnsite 1m 44s trunk passed
          +1 mvneclipse 0m 33s trunk passed
          +1 findbugs 3m 38s trunk passed
          +1 javadoc 2m 3s trunk passed with JDK v1.8.0_66
          +1 javadoc 2m 52s trunk passed with JDK v1.7.0_85
          +1 mvninstall 2m 15s the patch passed
          +1 compile 4m 21s the patch passed with JDK v1.8.0_66
          +1 javac 4m 21s the patch passed
          +1 compile 4m 16s the patch passed with JDK v1.7.0_85
          +1 javac 4m 16s the patch passed
          +1 checkstyle 0m 56s the patch passed
          +1 mvnsite 1m 38s the patch passed
          +1 mvneclipse 0m 30s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 findbugs 3m 57s the patch passed
          +1 javadoc 2m 0s the patch passed with JDK v1.8.0_66
          +1 javadoc 2m 47s the patch passed with JDK v1.7.0_85
          -1 unit 6m 58s hadoop-common in the patch failed with JDK v1.8.0_66.
          -1 unit 51m 28s hadoop-hdfs in the patch failed with JDK v1.8.0_66.
          -1 unit 7m 26s hadoop-common in the patch failed with JDK v1.7.0_85.
          -1 unit 55m 15s hadoop-hdfs in the patch failed with JDK v1.7.0_85.
          -1 asflicense 0m 25s Patch generated 58 ASF License warnings.
          185m 58s



          Reason Tests
          JDK v1.8.0_66 Failed junit tests hadoop.fs.TestLocalFsFCStatistics
            hadoop.hdfs.server.namenode.TestNamenodeCapacityReport
            hadoop.hdfs.server.namenode.ha.TestHASafeMode
            hadoop.hdfs.server.namenode.TestNameNodeAcl
            hadoop.hdfs.TestDFSStripedOutputStreamWithFailure070
          JDK v1.7.0_85 Failed junit tests hadoop.metrics2.impl.TestGangliaMetrics
            hadoop.hdfs.TestWriteReadStripedFile



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:date2015-11-17
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12772794/HDFS-9347.002.patch
          JIRA Issue HDFS-9347
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 4e5be4e9c81b 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HDFS-Build/patchprocess/apache-yetus-4659377/precommit/personality/hadoop.sh
          git revision trunk / b82a6ce
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/13539/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_66.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/13539/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/13539/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_85.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/13539/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_85.txt
          unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/13539/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HDFS-Build/13539/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HDFS-Build/13539/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_85.txt https://builds.apache.org/job/PreCommit-HDFS-Build/13539/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_85.txt
          JDK v1.7.0_85 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/13539/testReport/
          asflicense https://builds.apache.org/job/PreCommit-HDFS-Build/13539/artifact/patchprocess/patch-asflicense-problems.txt
          modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: .
          Max memory used 231MB
          Powered by Apache Yetus http://yetus.apache.org
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/13539/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 11m 13s docker + precommit patch detected. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. +1 mvninstall 9m 5s trunk passed +1 compile 4m 16s trunk passed with JDK v1.8.0_66 +1 compile 4m 8s trunk passed with JDK v1.7.0_85 +1 checkstyle 0m 58s trunk passed +1 mvnsite 1m 44s trunk passed +1 mvneclipse 0m 33s trunk passed +1 findbugs 3m 38s trunk passed +1 javadoc 2m 3s trunk passed with JDK v1.8.0_66 +1 javadoc 2m 52s trunk passed with JDK v1.7.0_85 +1 mvninstall 2m 15s the patch passed +1 compile 4m 21s the patch passed with JDK v1.8.0_66 +1 javac 4m 21s the patch passed +1 compile 4m 16s the patch passed with JDK v1.7.0_85 +1 javac 4m 16s the patch passed +1 checkstyle 0m 56s the patch passed +1 mvnsite 1m 38s the patch passed +1 mvneclipse 0m 30s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 3m 57s the patch passed +1 javadoc 2m 0s the patch passed with JDK v1.8.0_66 +1 javadoc 2m 47s the patch passed with JDK v1.7.0_85 -1 unit 6m 58s hadoop-common in the patch failed with JDK v1.8.0_66. -1 unit 51m 28s hadoop-hdfs in the patch failed with JDK v1.8.0_66. -1 unit 7m 26s hadoop-common in the patch failed with JDK v1.7.0_85. -1 unit 55m 15s hadoop-hdfs in the patch failed with JDK v1.7.0_85. -1 asflicense 0m 25s Patch generated 58 ASF License warnings. 185m 58s Reason Tests JDK v1.8.0_66 Failed junit tests hadoop.fs.TestLocalFsFCStatistics   hadoop.hdfs.server.namenode.TestNamenodeCapacityReport   hadoop.hdfs.server.namenode.ha.TestHASafeMode   hadoop.hdfs.server.namenode.TestNameNodeAcl   hadoop.hdfs.TestDFSStripedOutputStreamWithFailure070 JDK v1.7.0_85 Failed junit tests hadoop.metrics2.impl.TestGangliaMetrics   hadoop.hdfs.TestWriteReadStripedFile Subsystem Report/Notes Docker Image:yetus/hadoop:date2015-11-17 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12772794/HDFS-9347.002.patch JIRA Issue HDFS-9347 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 4e5be4e9c81b 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HDFS-Build/patchprocess/apache-yetus-4659377/precommit/personality/hadoop.sh git revision trunk / b82a6ce findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/13539/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_66.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/13539/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/13539/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_85.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/13539/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_85.txt unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/13539/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HDFS-Build/13539/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HDFS-Build/13539/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_85.txt https://builds.apache.org/job/PreCommit-HDFS-Build/13539/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_85.txt JDK v1.7.0_85 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/13539/testReport/ asflicense https://builds.apache.org/job/PreCommit-HDFS-Build/13539/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: . Max memory used 231MB Powered by Apache Yetus http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HDFS-Build/13539/console This message was automatically generated.
          Hide
          jojochuang Wei-Chiu Chuang added a comment -

          Hi Walter Su Thank you very much for your last review.
          How do you feel about this version of patch?

          Thank you again.

          Show
          jojochuang Wei-Chiu Chuang added a comment - Hi Walter Su Thank you very much for your last review. How do you feel about this version of patch? Thank you again.
          Hide
          zhz Zhe Zhang added a comment -

          Thanks Wei-Chiu for the work and Walter for the comment. LGTM overall. A few comments:

          1. The semantics of waitForThreadTermination should be to throw TimeoutException if the specified waitForMillis expires. So assertNoThreadsMatching should catch this TimeoutException and Assert.fail. In other words assertNoThreadsMatching should be an actual assertion method, and waitForThreadTermination should be an actual wait method, without assertion logic. The current special handling of waitForMillis == 0 works, but creates unnecessary dependency b/w the 2 methods.
          2. There are some unnecessary white space changes.
          Show
          zhz Zhe Zhang added a comment - Thanks Wei-Chiu for the work and Walter for the comment. LGTM overall. A few comments: The semantics of waitForThreadTermination should be to throw TimeoutException if the specified waitForMillis expires. So assertNoThreadsMatching should catch this TimeoutException and Assert.fail . In other words assertNoThreadsMatching should be an actual assertion method, and waitForThreadTermination should be an actual wait method, without assertion logic. The current special handling of waitForMillis == 0 works, but creates unnecessary dependency b/w the 2 methods. There are some unnecessary white space changes.
          Hide
          jojochuang Wei-Chiu Chuang added a comment -

          Hi Zhe Zhang Thanks for the suggestion. Agreed the code was not clean. I refactored the code a bit, creating a new method anyThreadMatching() in order to make it cleaner. The idea is to assert name of the thread which matches the regular expression.

          An alternative approach would be to create a "assumeNoThreadsMatching()", and let it throw an exception if there's any thread that matches it.

          Show
          jojochuang Wei-Chiu Chuang added a comment - Hi Zhe Zhang Thanks for the suggestion. Agreed the code was not clean. I refactored the code a bit, creating a new method anyThreadMatching() in order to make it cleaner. The idea is to assert name of the thread which matches the regular expression. An alternative approach would be to create a "assumeNoThreadsMatching()", and let it throw an exception if there's any thread that matches it.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 0s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
          +1 mvninstall 7m 59s trunk passed
          +1 compile 8m 12s trunk passed with JDK v1.8.0_66
          +1 compile 8m 54s trunk passed with JDK v1.7.0_91
          +1 checkstyle 1m 0s trunk passed
          +1 mvnsite 1m 58s trunk passed
          +1 mvneclipse 0m 28s trunk passed
          +1 findbugs 3m 43s trunk passed
          +1 javadoc 1m 58s trunk passed with JDK v1.8.0_66
          +1 javadoc 2m 55s trunk passed with JDK v1.7.0_91
          +1 mvninstall 2m 33s the patch passed
          +1 compile 8m 6s the patch passed with JDK v1.8.0_66
          +1 javac 8m 6s the patch passed
          +1 compile 8m 59s the patch passed with JDK v1.7.0_91
          +1 javac 8m 59s the patch passed
          +1 checkstyle 1m 1s the patch passed
          +1 mvnsite 1m 54s the patch passed
          +1 mvneclipse 0m 28s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 findbugs 3m 59s the patch passed
          +1 javadoc 1m 59s the patch passed with JDK v1.8.0_66
          +1 javadoc 2m 57s the patch passed with JDK v1.7.0_91
          +1 unit 7m 33s hadoop-common in the patch passed with JDK v1.8.0_66.
          -1 unit 66m 2s hadoop-hdfs in the patch failed with JDK v1.8.0_66.
          -1 unit 7m 3s hadoop-common in the patch failed with JDK v1.7.0_91.
          -1 unit 68m 48s hadoop-hdfs in the patch failed with JDK v1.7.0_91.
          -1 asflicense 0m 26s Patch generated 58 ASF License warnings.
          220m 22s



          Reason Tests
          JDK v1.8.0_66 Failed junit tests hadoop.hdfs.TestDFSUpgradeFromImage
            hadoop.hdfs.server.namenode.snapshot.TestOpenFilesWithSnapshot
            hadoop.hdfs.TestRollingUpgrade
          JDK v1.7.0_91 Failed junit tests hadoop.ipc.TestIPC
            hadoop.hdfs.server.namenode.TestNameNodeResourceChecker
            hadoop.hdfs.server.namenode.TestNameNodeMetadataConsistency



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:0ca8df7
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12778141/HDFS-9347.003.patch
          JIRA Issue HDFS-9347
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 781fc66c30e0 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 3c0adac
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/13909/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/13909/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_91.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/13909/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_91.txt
          unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/13909/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HDFS-Build/13909/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_91.txt https://builds.apache.org/job/PreCommit-HDFS-Build/13909/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_91.txt
          JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/13909/testReport/
          asflicense https://builds.apache.org/job/PreCommit-HDFS-Build/13909/artifact/patchprocess/patch-asflicense-problems.txt
          modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: .
          Max memory used 75MB
          Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/13909/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. +1 mvninstall 7m 59s trunk passed +1 compile 8m 12s trunk passed with JDK v1.8.0_66 +1 compile 8m 54s trunk passed with JDK v1.7.0_91 +1 checkstyle 1m 0s trunk passed +1 mvnsite 1m 58s trunk passed +1 mvneclipse 0m 28s trunk passed +1 findbugs 3m 43s trunk passed +1 javadoc 1m 58s trunk passed with JDK v1.8.0_66 +1 javadoc 2m 55s trunk passed with JDK v1.7.0_91 +1 mvninstall 2m 33s the patch passed +1 compile 8m 6s the patch passed with JDK v1.8.0_66 +1 javac 8m 6s the patch passed +1 compile 8m 59s the patch passed with JDK v1.7.0_91 +1 javac 8m 59s the patch passed +1 checkstyle 1m 1s the patch passed +1 mvnsite 1m 54s the patch passed +1 mvneclipse 0m 28s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 3m 59s the patch passed +1 javadoc 1m 59s the patch passed with JDK v1.8.0_66 +1 javadoc 2m 57s the patch passed with JDK v1.7.0_91 +1 unit 7m 33s hadoop-common in the patch passed with JDK v1.8.0_66. -1 unit 66m 2s hadoop-hdfs in the patch failed with JDK v1.8.0_66. -1 unit 7m 3s hadoop-common in the patch failed with JDK v1.7.0_91. -1 unit 68m 48s hadoop-hdfs in the patch failed with JDK v1.7.0_91. -1 asflicense 0m 26s Patch generated 58 ASF License warnings. 220m 22s Reason Tests JDK v1.8.0_66 Failed junit tests hadoop.hdfs.TestDFSUpgradeFromImage   hadoop.hdfs.server.namenode.snapshot.TestOpenFilesWithSnapshot   hadoop.hdfs.TestRollingUpgrade JDK v1.7.0_91 Failed junit tests hadoop.ipc.TestIPC   hadoop.hdfs.server.namenode.TestNameNodeResourceChecker   hadoop.hdfs.server.namenode.TestNameNodeMetadataConsistency Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12778141/HDFS-9347.003.patch JIRA Issue HDFS-9347 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 781fc66c30e0 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 3c0adac findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/13909/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/13909/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_91.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/13909/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_91.txt unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/13909/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HDFS-Build/13909/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_91.txt https://builds.apache.org/job/PreCommit-HDFS-Build/13909/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_91.txt JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/13909/testReport/ asflicense https://builds.apache.org/job/PreCommit-HDFS-Build/13909/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: . Max memory used 75MB Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HDFS-Build/13909/console This message was automatically generated.
          Hide
          zhz Zhe Zhang added a comment -

          Thanks Wei-Chiu. New patch does look cleaner.

          Actually how about just let anyThreadMatching return a boolean, and let assertNoThreadsMatching fail if true is returned?

          +1 pending a conclusion on the above.

          Show
          zhz Zhe Zhang added a comment - Thanks Wei-Chiu. New patch does look cleaner. Actually how about just let anyThreadMatching return a boolean, and let assertNoThreadsMatching fail if true is returned? +1 pending a conclusion on the above.
          Hide
          jojochuang Wei-Chiu Chuang added a comment -

          Rev04: assert a failure in assertNoThreadsMatching() if anyThreadMatching() returns true.

          Show
          jojochuang Wei-Chiu Chuang added a comment - Rev04: assert a failure in assertNoThreadsMatching() if anyThreadMatching() returns true.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          -1 patch 0m 4s HDFS-9347 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help.



          Subsystem Report/Notes
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12778353/HDFS-9347.004.patch
          JIRA Issue HDFS-9347
          Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/13923/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment -1 patch 0m 4s HDFS-9347 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. Subsystem Report/Notes JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12778353/HDFS-9347.004.patch JIRA Issue HDFS-9347 Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HDFS-Build/13923/console This message was automatically generated.
          Hide
          jojochuang Wei-Chiu Chuang added a comment -

          Forgot the rebase my local repo. Re-attach the patch.

          Show
          jojochuang Wei-Chiu Chuang added a comment - Forgot the rebase my local repo. Re-attach the patch.
          Hide
          zhz Zhe Zhang added a comment -

          Thanks Wei-Chiu. +1 on the 04 patch pending a few minors:

          1. Unnecessary white space change in waitFor
          2. Unused imports in TestQuorumJournalManager
          3. In shutdown, can we preserve the existing comments, and just add "Wait for IPC clients to terminate ..."?
          Show
          zhz Zhe Zhang added a comment - Thanks Wei-Chiu. +1 on the 04 patch pending a few minors: Unnecessary white space change in waitFor Unused imports in TestQuorumJournalManager In shutdown , can we preserve the existing comments, and just add "Wait for IPC clients to terminate ..."?
          Hide
          jojochuang Wei-Chiu Chuang added a comment -

          Rev05: address Zhe's comments.
          IntelliJ automatically indent my code and some old code that does not conform to the coding standard was updated. Other than that, the latest patch address's Zhe's comments. Thank you!

          Show
          jojochuang Wei-Chiu Chuang added a comment - Rev05: address Zhe's comments. IntelliJ automatically indent my code and some old code that does not conform to the coding standard was updated. Other than that, the latest patch address's Zhe's comments. Thank you!
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          -1 patch 0m 6s HDFS-9347 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help.



          Subsystem Report/Notes
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12778380/HDFS-9347.005.patch
          JIRA Issue HDFS-9347
          Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/13925/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment -1 patch 0m 6s HDFS-9347 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. Subsystem Report/Notes JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12778380/HDFS-9347.005.patch JIRA Issue HDFS-9347 Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HDFS-Build/13925/console This message was automatically generated.
          Hide
          jojochuang Wei-Chiu Chuang added a comment -

          Rev06: rebase

          Show
          jojochuang Wei-Chiu Chuang added a comment - Rev06: rebase
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 0s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
          +1 mvninstall 7m 39s trunk passed
          +1 compile 7m 50s trunk passed with JDK v1.8.0_66
          +1 compile 8m 38s trunk passed with JDK v1.7.0_91
          +1 checkstyle 0m 59s trunk passed
          +1 mvnsite 1m 53s trunk passed
          +1 mvneclipse 0m 28s trunk passed
          +1 findbugs 3m 40s trunk passed
          +1 javadoc 1m 56s trunk passed with JDK v1.8.0_66
          +1 javadoc 2m 47s trunk passed with JDK v1.7.0_91
          +1 mvninstall 2m 35s the patch passed
          +1 compile 7m 49s the patch passed with JDK v1.8.0_66
          +1 javac 7m 49s the patch passed
          +1 compile 8m 40s the patch passed with JDK v1.7.0_91
          +1 javac 8m 40s the patch passed
          +1 checkstyle 0m 59s the patch passed
          +1 mvnsite 1m 52s the patch passed
          +1 mvneclipse 0m 27s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 findbugs 3m 55s the patch passed
          +1 javadoc 2m 2s the patch passed with JDK v1.8.0_66
          +1 javadoc 2m 49s the patch passed with JDK v1.7.0_91
          -1 unit 6m 46s hadoop-common in the patch failed with JDK v1.8.0_66.
          -1 unit 54m 46s hadoop-hdfs in the patch failed with JDK v1.8.0_66.
          +1 unit 8m 15s hadoop-common in the patch passed with JDK v1.7.0_91.
          -1 unit 55m 32s hadoop-hdfs in the patch failed with JDK v1.7.0_91.
          -1 asflicense 0m 26s Patch generated 58 ASF License warnings.
          193m 56s



          Reason Tests
          JDK v1.8.0_66 Failed junit tests hadoop.fs.shell.TestCopyPreserveFlag
            hadoop.ipc.TestIPC
            hadoop.hdfs.shortcircuit.TestShortCircuitCache
            hadoop.hdfs.TestLeaseRecovery2
            hadoop.hdfs.server.namenode.TestNNThroughputBenchmark
            hadoop.hdfs.TestDFSStorageStateRecovery
          JDK v1.7.0_91 Failed junit tests hadoop.hdfs.server.namenode.ha.TestHASafeMode



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:0ca8df7
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12778387/HDFS-9347.006.patch
          JIRA Issue HDFS-9347
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 5856a84a7ff4 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 1de56b0
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/13927/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_66.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/13927/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/13927/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_91.txt
          unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/13927/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HDFS-Build/13927/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HDFS-Build/13927/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_91.txt
          JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/13927/testReport/
          asflicense https://builds.apache.org/job/PreCommit-HDFS-Build/13927/artifact/patchprocess/patch-asflicense-problems.txt
          modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: .
          Max memory used 75MB
          Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/13927/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. +1 mvninstall 7m 39s trunk passed +1 compile 7m 50s trunk passed with JDK v1.8.0_66 +1 compile 8m 38s trunk passed with JDK v1.7.0_91 +1 checkstyle 0m 59s trunk passed +1 mvnsite 1m 53s trunk passed +1 mvneclipse 0m 28s trunk passed +1 findbugs 3m 40s trunk passed +1 javadoc 1m 56s trunk passed with JDK v1.8.0_66 +1 javadoc 2m 47s trunk passed with JDK v1.7.0_91 +1 mvninstall 2m 35s the patch passed +1 compile 7m 49s the patch passed with JDK v1.8.0_66 +1 javac 7m 49s the patch passed +1 compile 8m 40s the patch passed with JDK v1.7.0_91 +1 javac 8m 40s the patch passed +1 checkstyle 0m 59s the patch passed +1 mvnsite 1m 52s the patch passed +1 mvneclipse 0m 27s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 3m 55s the patch passed +1 javadoc 2m 2s the patch passed with JDK v1.8.0_66 +1 javadoc 2m 49s the patch passed with JDK v1.7.0_91 -1 unit 6m 46s hadoop-common in the patch failed with JDK v1.8.0_66. -1 unit 54m 46s hadoop-hdfs in the patch failed with JDK v1.8.0_66. +1 unit 8m 15s hadoop-common in the patch passed with JDK v1.7.0_91. -1 unit 55m 32s hadoop-hdfs in the patch failed with JDK v1.7.0_91. -1 asflicense 0m 26s Patch generated 58 ASF License warnings. 193m 56s Reason Tests JDK v1.8.0_66 Failed junit tests hadoop.fs.shell.TestCopyPreserveFlag   hadoop.ipc.TestIPC   hadoop.hdfs.shortcircuit.TestShortCircuitCache   hadoop.hdfs.TestLeaseRecovery2   hadoop.hdfs.server.namenode.TestNNThroughputBenchmark   hadoop.hdfs.TestDFSStorageStateRecovery JDK v1.7.0_91 Failed junit tests hadoop.hdfs.server.namenode.ha.TestHASafeMode Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12778387/HDFS-9347.006.patch JIRA Issue HDFS-9347 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 5856a84a7ff4 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 1de56b0 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/13927/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_66.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/13927/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/13927/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_91.txt unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/13927/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HDFS-Build/13927/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HDFS-Build/13927/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_91.txt JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/13927/testReport/ asflicense https://builds.apache.org/job/PreCommit-HDFS-Build/13927/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: . Max memory used 75MB Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HDFS-Build/13927/console This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 0s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
          +1 mvninstall 8m 6s trunk passed
          +1 compile 8m 49s trunk passed with JDK v1.8.0_66
          +1 compile 9m 33s trunk passed with JDK v1.7.0_91
          +1 checkstyle 1m 4s trunk passed
          +1 mvnsite 2m 4s trunk passed
          +1 mvneclipse 0m 29s trunk passed
          +1 findbugs 3m 59s trunk passed
          +1 javadoc 2m 5s trunk passed with JDK v1.8.0_66
          +1 javadoc 3m 2s trunk passed with JDK v1.7.0_91
          +1 mvninstall 2m 32s the patch passed
          +1 compile 9m 5s the patch passed with JDK v1.8.0_66
          +1 javac 9m 5s the patch passed
          +1 compile 9m 30s the patch passed with JDK v1.7.0_91
          +1 javac 9m 30s the patch passed
          +1 checkstyle 1m 3s the patch passed
          +1 mvnsite 2m 1s the patch passed
          +1 mvneclipse 0m 28s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 findbugs 4m 16s the patch passed
          +1 javadoc 2m 7s the patch passed with JDK v1.8.0_66
          +1 javadoc 3m 0s the patch passed with JDK v1.7.0_91
          -1 unit 17m 12s hadoop-common in the patch failed with JDK v1.8.0_66.
          -1 unit 57m 47s hadoop-hdfs in the patch failed with JDK v1.8.0_66.
          -1 unit 8m 7s hadoop-common in the patch failed with JDK v1.7.0_91.
          -1 unit 55m 45s hadoop-hdfs in the patch failed with JDK v1.7.0_91.
          -1 asflicense 0m 27s Patch generated 56 ASF License warnings.
          213m 50s



          Reason Tests
          JDK v1.8.0_66 Failed junit tests hadoop.hdfs.TestDatanodeRegistration
            hadoop.hdfs.server.datanode.TestBlockScanner
            hadoop.hdfs.TestReadStripedFileWithDecoding
            hadoop.hdfs.server.namenode.TestNNThroughputBenchmark
          JDK v1.8.0_66 Timed out junit tests org.apache.hadoop.http.TestHttpServerLifecycle
          JDK v1.7.0_91 Failed junit tests hadoop.metrics2.impl.TestGangliaMetrics
            hadoop.hdfs.server.datanode.TestBlockScanner
            hadoop.hdfs.server.namenode.TestNNThroughputBenchmark



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:0ca8df7
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12778387/HDFS-9347.006.patch
          JIRA Issue HDFS-9347
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 2a5024af0bf4 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 1de56b0
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/13928/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_66.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/13928/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/13928/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_91.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/13928/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_91.txt
          unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/13928/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HDFS-Build/13928/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HDFS-Build/13928/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_91.txt https://builds.apache.org/job/PreCommit-HDFS-Build/13928/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_91.txt
          JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/13928/testReport/
          asflicense https://builds.apache.org/job/PreCommit-HDFS-Build/13928/artifact/patchprocess/patch-asflicense-problems.txt
          modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: .
          Max memory used 76MB
          Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/13928/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. +1 mvninstall 8m 6s trunk passed +1 compile 8m 49s trunk passed with JDK v1.8.0_66 +1 compile 9m 33s trunk passed with JDK v1.7.0_91 +1 checkstyle 1m 4s trunk passed +1 mvnsite 2m 4s trunk passed +1 mvneclipse 0m 29s trunk passed +1 findbugs 3m 59s trunk passed +1 javadoc 2m 5s trunk passed with JDK v1.8.0_66 +1 javadoc 3m 2s trunk passed with JDK v1.7.0_91 +1 mvninstall 2m 32s the patch passed +1 compile 9m 5s the patch passed with JDK v1.8.0_66 +1 javac 9m 5s the patch passed +1 compile 9m 30s the patch passed with JDK v1.7.0_91 +1 javac 9m 30s the patch passed +1 checkstyle 1m 3s the patch passed +1 mvnsite 2m 1s the patch passed +1 mvneclipse 0m 28s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 4m 16s the patch passed +1 javadoc 2m 7s the patch passed with JDK v1.8.0_66 +1 javadoc 3m 0s the patch passed with JDK v1.7.0_91 -1 unit 17m 12s hadoop-common in the patch failed with JDK v1.8.0_66. -1 unit 57m 47s hadoop-hdfs in the patch failed with JDK v1.8.0_66. -1 unit 8m 7s hadoop-common in the patch failed with JDK v1.7.0_91. -1 unit 55m 45s hadoop-hdfs in the patch failed with JDK v1.7.0_91. -1 asflicense 0m 27s Patch generated 56 ASF License warnings. 213m 50s Reason Tests JDK v1.8.0_66 Failed junit tests hadoop.hdfs.TestDatanodeRegistration   hadoop.hdfs.server.datanode.TestBlockScanner   hadoop.hdfs.TestReadStripedFileWithDecoding   hadoop.hdfs.server.namenode.TestNNThroughputBenchmark JDK v1.8.0_66 Timed out junit tests org.apache.hadoop.http.TestHttpServerLifecycle JDK v1.7.0_91 Failed junit tests hadoop.metrics2.impl.TestGangliaMetrics   hadoop.hdfs.server.datanode.TestBlockScanner   hadoop.hdfs.server.namenode.TestNNThroughputBenchmark Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12778387/HDFS-9347.006.patch JIRA Issue HDFS-9347 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 2a5024af0bf4 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 1de56b0 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/13928/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_66.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/13928/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/13928/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_91.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/13928/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_91.txt unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/13928/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HDFS-Build/13928/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HDFS-Build/13928/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_91.txt https://builds.apache.org/job/PreCommit-HDFS-Build/13928/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_91.txt JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/13928/testReport/ asflicense https://builds.apache.org/job/PreCommit-HDFS-Build/13928/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: . Max memory used 76MB Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HDFS-Build/13928/console This message was automatically generated.
          Hide
          jojochuang Wei-Chiu Chuang added a comment -

          test failures look unrelated.

          Show
          jojochuang Wei-Chiu Chuang added a comment - test failures look unrelated.
          Hide
          zhz Zhe Zhang added a comment -

          Thanks Wei-Chiu! +1 on the latest patch. I just committed to trunk, branch-2, branch-2.8.

          Show
          zhz Zhe Zhang added a comment - Thanks Wei-Chiu! +1 on the latest patch. I just committed to trunk, branch-2, branch-2.8.
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-trunk-Commit #9001 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9001/)
          HDFS-9347. Invariant assumption in TestQuorumJournalManager.shutdown() (zhz: rev ef3f3f6bb14cf44bef1778f1091d8ed8a4b764a3)

          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/qjournal/client/TestQuorumJournalManager.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/test/GenericTestUtils.java
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-trunk-Commit #9001 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9001/ ) HDFS-9347 . Invariant assumption in TestQuorumJournalManager.shutdown() (zhz: rev ef3f3f6bb14cf44bef1778f1091d8ed8a4b764a3) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/qjournal/client/TestQuorumJournalManager.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/test/GenericTestUtils.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Hide
          jojochuang Wei-Chiu Chuang added a comment -

          Than you! Zhe Zhang and Walter Su for review

          Show
          jojochuang Wei-Chiu Chuang added a comment - Than you! Zhe Zhang and Walter Su for review
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #9265 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9265/)
          Move HDFS-9347 from 2.8.0 to 2.6.5 in CHANGES.txt. (waltersu4549: rev acac72911e59a54414cc72c788626792b24af2f3)

          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #9265 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9265/ ) Move HDFS-9347 from 2.8.0 to 2.6.5 in CHANGES.txt. (waltersu4549: rev acac72911e59a54414cc72c788626792b24af2f3) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Hide
          walter.k.su Walter Su added a comment -

          Thanks Wei-Chiu Chuang for the work. I just cherry-picked it to branch-2.7 and branch-2.6.

          Show
          walter.k.su Walter Su added a comment - Thanks Wei-Chiu Chuang for the work. I just cherry-picked it to branch-2.7 and branch-2.6.
          Hide
          vinodkv Vinod Kumar Vavilapalli added a comment -

          Closing the JIRA as part of 2.7.3 release.

          Show
          vinodkv Vinod Kumar Vavilapalli added a comment - Closing the JIRA as part of 2.7.3 release.

            People

            • Assignee:
              jojochuang Wei-Chiu Chuang
              Reporter:
              jojochuang Wei-Chiu Chuang
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development