Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-7553

fix the TestDFSUpgradeWithHA due to BindException

    Details

    • Type: Test
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.7.0
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: test
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      see https://builds.apache.org/job/PreCommit-HDFS-Build/9092//testReport/org.apache.hadoop.hdfs.server.namenode.ha/TestDFSUpgradeWithHA/testNfsUpgrade/ :
      Error Message

      Port in use: localhost:57896
      Stacktrace

      java.net.BindException: Port in use: localhost:57896
      at sun.nio.ch.Net.bind0(Native Method)
      at sun.nio.ch.Net.bind(Net.java:444)
      at sun.nio.ch.Net.bind(Net.java:436)
      at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:214)
      at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
      at org.mortbay.jetty.nio.SelectChannelConnector.open(SelectChannelConnector.java:216)
      at org.apache.hadoop.http.HttpServer2.openListeners(HttpServer2.java:868)
      at org.apache.hadoop.http.HttpServer2.start(HttpServer2.java:809)
      at org.apache.hadoop.hdfs.server.namenode.NameNodeHttpServer.start(NameNodeHttpServer.java:142)
      at org.apache.hadoop.hdfs.server.namenode.NameNode.startHttpServer(NameNode.java:704)
      at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:591)
      at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:763)
      at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:747)
      at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1443)
      at org.apache.hadoop.hdfs.MiniDFSCluster.restartNameNode(MiniDFSCluster.java:1815)
      at org.apache.hadoop.hdfs.MiniDFSCluster.restartNameNode(MiniDFSCluster.java:1796)
      at org.apache.hadoop.hdfs.server.namenode.ha.TestDFSUpgradeWithHA.testNfsUpgrade(TestDFSUpgradeWithHA.java:285)

      1. HDFS-7553.002.patch
        1 kB
        Xiao Chen
      2. HDFS-7553.03.patch
        3 kB
        Xiao Chen
      3. HDFS-7553.04.patch
        5 kB
        Xiao Chen
      4. HDFS-7553.05.patch
        5 kB
        Xiao Chen
      5. HDFS-7553.repro.patch
        7 kB
        Xiao Chen
      6. HDFS-7553-001.txt
        0.8 kB
        Liang Xie
      7. HDFS-7553-branch-2.05.patch
        5 kB
        Chris Nauroth

        Activity

        Hide
        xieliang007 Liang Xie added a comment -

        After this change, we can ensure HttpServer2.Builder call setFindPort(true), so it will retry once BindException jumped out. Please see DFSUtil.java:

            if (policy.isHttpEnabled()) {
              if (httpAddr.getPort() == 0) {
                builder.setFindPort(true);
              }
        
        Show
        xieliang007 Liang Xie added a comment - After this change, we can ensure HttpServer2.Builder call setFindPort(true), so it will retry once BindException jumped out. Please see DFSUtil.java: if (policy.isHttpEnabled()) { if (httpAddr.getPort() == 0) { builder.setFindPort( true ); }
        Hide
        hadoopqa Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12688300/HDFS-7553-001.txt
        against trunk revision 6635ccd.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 1 new or modified test files.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. There were no new javadoc warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        -1 findbugs. The patch appears to introduce 1 new Findbugs (version 2.0.3) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs:

        org.apache.hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureToleration
        org.apache.hadoop.hdfs.server.blockmanagement.TestDatanodeManager
        org.apache.hadoop.fs.TestSymlinkHdfsFileContext
        org.apache.hadoop.hdfs.server.namenode.TestCacheDirectives

        Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/9094//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/9094//artifact/patchprocess/newPatchFindbugsWarningshadoop-hdfs.html
        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/9094//console

        This message is automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12688300/HDFS-7553-001.txt against trunk revision 6635ccd. +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 1 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. -1 findbugs . The patch appears to introduce 1 new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs: org.apache.hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureToleration org.apache.hadoop.hdfs.server.blockmanagement.TestDatanodeManager org.apache.hadoop.fs.TestSymlinkHdfsFileContext org.apache.hadoop.hdfs.server.namenode.TestCacheDirectives Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/9094//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/9094//artifact/patchprocess/newPatchFindbugsWarningshadoop-hdfs.html Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/9094//console This message is automatically generated.
        Hide
        wheat9 Haohui Mai added a comment -

        I don't understand. The port is set at MiniDFSCluster#initNameNodeAddress(), why setting the port again will fix this issue?

        Show
        wheat9 Haohui Mai added a comment - I don't understand. The port is set at MiniDFSCluster#initNameNodeAddress() , why setting the port again will fix this issue?
        Hide
        cnauroth Chris Nauroth added a comment -

        Hi Liang Xie. If I understand correctly, you were trying to reset the port to 0, which would then retrigger the logic for dynamically selecting a new bind port. However, I don't think this is right, because all of this is happening in a Before method before each test runs. The failure report shows the bind failure part-way through one test after a NameNode restart, so it doesn't look like an interaction spanning multiple tests that run into a port conflict with each other.

        Another possibility here might be that NameNode shutdown waits for completion of the RPC server shutdown, but not the HTTP server shutdown. This happens in NameNode#join:

          public void join() {
            try {
              rpcServer.join();
            } catch (InterruptedException ie) {
              LOG.info("Caught interrupted exception ", ie);
            }
          }
        

        However, there is no call to HttpServer2#join. This would ultimately block waiting for Jetty's thread pool to shut down. Perhaps we have a race condition where Jetty remains running bound to that port for a brief window after NameNode shutdown. Then, when the restart happens, we have a conflict on that port.

        Show
        cnauroth Chris Nauroth added a comment - Hi Liang Xie . If I understand correctly, you were trying to reset the port to 0, which would then retrigger the logic for dynamically selecting a new bind port. However, I don't think this is right, because all of this is happening in a Before method before each test runs. The failure report shows the bind failure part-way through one test after a NameNode restart, so it doesn't look like an interaction spanning multiple tests that run into a port conflict with each other. Another possibility here might be that NameNode shutdown waits for completion of the RPC server shutdown, but not the HTTP server shutdown. This happens in NameNode#join : public void join() { try { rpcServer.join(); } catch (InterruptedException ie) { LOG.info( "Caught interrupted exception " , ie); } } However, there is no call to HttpServer2#join . This would ultimately block waiting for Jetty's thread pool to shut down. Perhaps we have a race condition where Jetty remains running bound to that port for a brief window after NameNode shutdown. Then, when the restart happens, we have a conflict on that port.
        Hide
        xiaochen Xiao Chen added a comment -

        I came across the same error:

        Error Message
        Port in use: localhost:36908
        Stacktrace

        java.net.BindException: Port in use: localhost:36908
        	at sun.nio.ch.Net.bind0(Native Method)
        	at sun.nio.ch.Net.bind(Net.java:444)
        	at sun.nio.ch.Net.bind(Net.java:436)
        	at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:214)
        	at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
        	at org.mortbay.jetty.nio.SelectChannelConnector.open(SelectChannelConnector.java:216)
        	at org.apache.hadoop.http.HttpServer2.openListeners(HttpServer2.java:939)
        	at org.apache.hadoop.http.HttpServer2.start(HttpServer2.java:880)
        	at org.apache.hadoop.hdfs.server.namenode.NameNodeHttpServer.start(NameNodeHttpServer.java:142)
        	at org.apache.hadoop.hdfs.server.namenode.NameNode.startHttpServer(NameNode.java:754)
        	at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:643)
        	at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:818)
        	at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:797)
        	at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1493)
        	at org.apache.hadoop.hdfs.MiniDFSCluster.restartNameNode(MiniDFSCluster.java:1820)
        	at org.apache.hadoop.hdfs.MiniDFSCluster.restartNameNode(MiniDFSCluster.java:1801)
        	at org.apache.hadoop.hdfs.server.namenode.ha.TestDFSUpgradeWithHA.testRollbackWithNfs(TestDFSUpgradeWithHA.java:593)
        

        And I agree with Chris Nauroth's speculation regarding the race condition. Patch 002 added the join method. I'm having a hard time reproducing the issue though, mainly because of the difficulty of reaching the BindException in jetty, so no unit test added.

        As a side note, setting DFS_NAMENODE_HTTP_ADDRESS_KEY at the beginning would not work, because in NameNodeHttpServer#start, we conf.set it to the address (including port) of the open connection. So when later in the test where cluster.restartNameNode, this configuration is read and will be used to start the http server. I feel no change is needed in this test since the RC is on the connection side anyways.

        Show
        xiaochen Xiao Chen added a comment - I came across the same error: Error Message Port in use: localhost:36908 Stacktrace java.net.BindException: Port in use: localhost:36908 at sun.nio.ch.Net.bind0(Native Method) at sun.nio.ch.Net.bind(Net.java:444) at sun.nio.ch.Net.bind(Net.java:436) at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:214) at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74) at org.mortbay.jetty.nio.SelectChannelConnector.open(SelectChannelConnector.java:216) at org.apache.hadoop.http.HttpServer2.openListeners(HttpServer2.java:939) at org.apache.hadoop.http.HttpServer2.start(HttpServer2.java:880) at org.apache.hadoop.hdfs.server.namenode.NameNodeHttpServer.start(NameNodeHttpServer.java:142) at org.apache.hadoop.hdfs.server.namenode.NameNode.startHttpServer(NameNode.java:754) at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:643) at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:818) at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:797) at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1493) at org.apache.hadoop.hdfs.MiniDFSCluster.restartNameNode(MiniDFSCluster.java:1820) at org.apache.hadoop.hdfs.MiniDFSCluster.restartNameNode(MiniDFSCluster.java:1801) at org.apache.hadoop.hdfs.server.namenode.ha.TestDFSUpgradeWithHA.testRollbackWithNfs(TestDFSUpgradeWithHA.java:593) And I agree with Chris Nauroth 's speculation regarding the race condition. Patch 002 added the join method. I'm having a hard time reproducing the issue though, mainly because of the difficulty of reaching the BindException in jetty, so no unit test added. As a side note, setting DFS_NAMENODE_HTTP_ADDRESS_KEY at the beginning would not work, because in NameNodeHttpServer#start , we conf.set it to the address (including port) of the open connection. So when later in the test where cluster.restartNameNode , this configuration is read and will be used to start the http server. I feel no change is needed in this test since the RC is on the connection side anyways.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 5s docker + precommit patch detected.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 mvninstall 3m 7s trunk passed
        +1 compile 0m 36s trunk passed with JDK v1.8.0_60
        +1 compile 0m 31s trunk passed with JDK v1.7.0_79
        +1 checkstyle 0m 15s trunk passed
        +1 mvneclipse 0m 14s trunk passed
        -1 findbugs 1m 53s hadoop-hdfs-project/hadoop-hdfs in trunk has 1 extant Findbugs warnings.
        +1 javadoc 1m 4s trunk passed with JDK v1.8.0_60
        +1 javadoc 1m 48s trunk passed with JDK v1.7.0_79
        +1 mvninstall 0m 39s the patch passed
        +1 compile 0m 31s the patch passed with JDK v1.8.0_60
        +1 javac 0m 31s the patch passed
        +1 compile 0m 32s the patch passed with JDK v1.7.0_79
        +1 javac 0m 32s the patch passed
        +1 checkstyle 0m 15s the patch passed
        +1 mvneclipse 0m 13s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        +1 findbugs 2m 1s the patch passed
        +1 javadoc 1m 6s the patch passed with JDK v1.8.0_60
        +1 javadoc 1m 46s the patch passed with JDK v1.7.0_79
        -1 unit 54m 21s hadoop-hdfs in the patch failed with JDK v1.8.0_60.
        -1 unit 50m 17s hadoop-hdfs in the patch failed with JDK v1.7.0_79.
        -1 asflicense 0m 22s Patch generated 58 ASF License warnings.
        124m 17s



        Reason Tests
        JDK v1.8.0_60 Failed junit tests hadoop.hdfs.server.namenode.TestDecommissioningStatus
          hadoop.hdfs.server.namenode.ha.TestSeveralNameNodes
          hadoop.hdfs.TestWriteReadStripedFile
        JDK v1.7.0_79 Failed junit tests hadoop.hdfs.TestDFSStripedOutputStreamWithFailure040
          hadoop.hdfs.TestRecoverStripedFile



        Subsystem Report/Notes
        Docker Client=1.7.1 Server=1.7.1 Image:test-patch-base-hadoop-date2015-11-09
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12771461/HDFS-7553.002.patch
        JIRA Issue HDFS-7553
        Optional Tests asflicense javac javadoc mvninstall unit findbugs checkstyle compile
        uname Linux 95b91f4157c7 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HDFS-Build/patchprocess/apache-yetus-ee5baeb/precommit/personality/hadoop.sh
        git revision trunk / 94a1833
        Default Java 1.7.0_79
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_60 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_79
        findbugs v3.0.0
        findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/13448/artifact/patchprocess/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs-warnings.html
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/13448/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_60.txt
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/13448/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_79.txt
        unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/13448/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_60.txt https://builds.apache.org/job/PreCommit-HDFS-Build/13448/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_79.txt
        JDK v1.7.0_79 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/13448/testReport/
        asflicense https://builds.apache.org/job/PreCommit-HDFS-Build/13448/artifact/patchprocess/patch-asflicense-problems.txt
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Max memory used 228MB
        Powered by Apache Yetus http://yetus.apache.org
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/13448/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 5s docker + precommit patch detected. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 3m 7s trunk passed +1 compile 0m 36s trunk passed with JDK v1.8.0_60 +1 compile 0m 31s trunk passed with JDK v1.7.0_79 +1 checkstyle 0m 15s trunk passed +1 mvneclipse 0m 14s trunk passed -1 findbugs 1m 53s hadoop-hdfs-project/hadoop-hdfs in trunk has 1 extant Findbugs warnings. +1 javadoc 1m 4s trunk passed with JDK v1.8.0_60 +1 javadoc 1m 48s trunk passed with JDK v1.7.0_79 +1 mvninstall 0m 39s the patch passed +1 compile 0m 31s the patch passed with JDK v1.8.0_60 +1 javac 0m 31s the patch passed +1 compile 0m 32s the patch passed with JDK v1.7.0_79 +1 javac 0m 32s the patch passed +1 checkstyle 0m 15s the patch passed +1 mvneclipse 0m 13s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 2m 1s the patch passed +1 javadoc 1m 6s the patch passed with JDK v1.8.0_60 +1 javadoc 1m 46s the patch passed with JDK v1.7.0_79 -1 unit 54m 21s hadoop-hdfs in the patch failed with JDK v1.8.0_60. -1 unit 50m 17s hadoop-hdfs in the patch failed with JDK v1.7.0_79. -1 asflicense 0m 22s Patch generated 58 ASF License warnings. 124m 17s Reason Tests JDK v1.8.0_60 Failed junit tests hadoop.hdfs.server.namenode.TestDecommissioningStatus   hadoop.hdfs.server.namenode.ha.TestSeveralNameNodes   hadoop.hdfs.TestWriteReadStripedFile JDK v1.7.0_79 Failed junit tests hadoop.hdfs.TestDFSStripedOutputStreamWithFailure040   hadoop.hdfs.TestRecoverStripedFile Subsystem Report/Notes Docker Client=1.7.1 Server=1.7.1 Image:test-patch-base-hadoop-date2015-11-09 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12771461/HDFS-7553.002.patch JIRA Issue HDFS-7553 Optional Tests asflicense javac javadoc mvninstall unit findbugs checkstyle compile uname Linux 95b91f4157c7 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HDFS-Build/patchprocess/apache-yetus-ee5baeb/precommit/personality/hadoop.sh git revision trunk / 94a1833 Default Java 1.7.0_79 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_60 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_79 findbugs v3.0.0 findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/13448/artifact/patchprocess/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs-warnings.html unit https://builds.apache.org/job/PreCommit-HDFS-Build/13448/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_60.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/13448/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_79.txt unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/13448/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_60.txt https://builds.apache.org/job/PreCommit-HDFS-Build/13448/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_79.txt JDK v1.7.0_79 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/13448/testReport/ asflicense https://builds.apache.org/job/PreCommit-HDFS-Build/13448/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Max memory used 228MB Powered by Apache Yetus http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HDFS-Build/13448/console This message was automatically generated.
        Hide
        xiaochen Xiao Chen added a comment -

        The findbugs warning seems to be HDFS-9401 which is fixed yesterday.
        The test failures and asflicense warnings are not related. Please review. Thanks!

        Show
        xiaochen Xiao Chen added a comment - The findbugs warning seems to be HDFS-9401 which is fixed yesterday. The test failures and asflicense warnings are not related. Please review. Thanks!
        Hide
        xiaochen Xiao Chen added a comment -

        I met a similar reoccurrence of this failure in a run of HDFS-9519:
        TestDFSUpgradeWithHA#testCannotUpgradeSecondNameNode failed with stack trace

        java.lang.AssertionError: Expected to find 'It looks like the shared log is already being upgraded' but got unexpected exception:java.net.BindException: Port in use: localhost:51012
        	at org.apache.hadoop.http.HttpServer2.openListeners(HttpServer2.java:910)
        	at org.apache.hadoop.http.HttpServer2.start(HttpServer2.java:847)
        	at org.apache.hadoop.hdfs.server.namenode.NameNodeHttpServer.start(NameNodeHttpServer.java:142)
        	at org.apache.hadoop.hdfs.server.namenode.NameNode.startHttpServer(NameNode.java:822)
        	at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:675)
        	at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:884)
        	at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:863)
        	at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1565)
        	at org.apache.hadoop.hdfs.MiniDFSCluster.restartNameNode(MiniDFSCluster.java:2017)
        	at org.apache.hadoop.hdfs.server.namenode.ha.TestDFSUpgradeWithHA.testCannotUpgradeSecondNameNode(TestDFSUpgradeWithHA.java:756)
        	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        	at java.lang.reflect.Method.invoke(Method.java:606)
        	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
        	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
        	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
        	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
        	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
        	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
        	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
        	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
        	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
        	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
        	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
        	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
        	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
        	at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
        	at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:264)
        	at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:153)
        	at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:124)
        	at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:200)
        	at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:153)
        	at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:103)
        Caused by: java.net.BindException: Address already in use
        	at sun.nio.ch.Net.bind0(Native Method)
        	at sun.nio.ch.Net.bind(Net.java:463)
        	at sun.nio.ch.Net.bind(Net.java:455)
        	at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223)
        	at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
        	at org.mortbay.jetty.nio.SelectChannelConnector.open(SelectChannelConnector.java:216)
        	at org.apache.hadoop.http.HttpServer2.openListeners(HttpServer2.java:905)
        	... 33 more
        
        	at sun.nio.ch.Net.bind0(Native Method)
        	at sun.nio.ch.Net.bind(Net.java:463)
        	at sun.nio.ch.Net.bind(Net.java:455)
        	at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223)
        	at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
        	at org.mortbay.jetty.nio.SelectChannelConnector.open(SelectChannelConnector.java:216)
        	at org.apache.hadoop.http.HttpServer2.openListeners(HttpServer2.java:905)
        	at org.apache.hadoop.http.HttpServer2.start(HttpServer2.java:847)
        	at org.apache.hadoop.hdfs.server.namenode.NameNodeHttpServer.start(NameNodeHttpServer.java:142)
        	at org.apache.hadoop.hdfs.server.namenode.NameNode.startHttpServer(NameNode.java:822)
        	at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:675)
        	at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:884)
        	at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:863)
        	at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1565)
        	at org.apache.hadoop.hdfs.MiniDFSCluster.restartNameNode(MiniDFSCluster.java:2017)
        	at org.apache.hadoop.hdfs.server.namenode.ha.TestDFSUpgradeWithHA.testCannotUpgradeSecondNameNode(TestDFSUpgradeWithHA.java:756)
        
        Show
        xiaochen Xiao Chen added a comment - I met a similar reoccurrence of this failure in a run of HDFS-9519 : TestDFSUpgradeWithHA#testCannotUpgradeSecondNameNode failed with stack trace java.lang.AssertionError: Expected to find 'It looks like the shared log is already being upgraded' but got unexpected exception:java.net.BindException: Port in use: localhost:51012 at org.apache.hadoop.http.HttpServer2.openListeners(HttpServer2.java:910) at org.apache.hadoop.http.HttpServer2.start(HttpServer2.java:847) at org.apache.hadoop.hdfs.server.namenode.NameNodeHttpServer.start(NameNodeHttpServer.java:142) at org.apache.hadoop.hdfs.server.namenode.NameNode.startHttpServer(NameNode.java:822) at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:675) at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:884) at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:863) at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1565) at org.apache.hadoop.hdfs.MiniDFSCluster.restartNameNode(MiniDFSCluster.java:2017) at org.apache.hadoop.hdfs.server.namenode.ha.TestDFSUpgradeWithHA.testCannotUpgradeSecondNameNode(TestDFSUpgradeWithHA.java:756) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229) at org.junit.runners.ParentRunner.run(ParentRunner.java:309) at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:264) at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:153) at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:124) at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:200) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:153) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:103) Caused by: java.net.BindException: Address already in use at sun.nio.ch.Net.bind0(Native Method) at sun.nio.ch.Net.bind(Net.java:463) at sun.nio.ch.Net.bind(Net.java:455) at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223) at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74) at org.mortbay.jetty.nio.SelectChannelConnector.open(SelectChannelConnector.java:216) at org.apache.hadoop.http.HttpServer2.openListeners(HttpServer2.java:905) ... 33 more at sun.nio.ch.Net.bind0(Native Method) at sun.nio.ch.Net.bind(Net.java:463) at sun.nio.ch.Net.bind(Net.java:455) at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223) at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74) at org.mortbay.jetty.nio.SelectChannelConnector.open(SelectChannelConnector.java:216) at org.apache.hadoop.http.HttpServer2.openListeners(HttpServer2.java:905) at org.apache.hadoop.http.HttpServer2.start(HttpServer2.java:847) at org.apache.hadoop.hdfs.server.namenode.NameNodeHttpServer.start(NameNodeHttpServer.java:142) at org.apache.hadoop.hdfs.server.namenode.NameNode.startHttpServer(NameNode.java:822) at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:675) at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:884) at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:863) at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1565) at org.apache.hadoop.hdfs.MiniDFSCluster.restartNameNode(MiniDFSCluster.java:2017) at org.apache.hadoop.hdfs.server.namenode.ha.TestDFSUpgradeWithHA.testCannotUpgradeSecondNameNode(TestDFSUpgradeWithHA.java:756)
        Hide
        xiaochen Xiao Chen added a comment -

        Not sure if Liang Xie is still working on this, but I want to self-promote patch 2 with a way to reproduce. Attached is the reproduce patch which includes patch 2, that fails with the bind exception when restarting NN. Uncommenting the httpServer.join(); at NameNode.java#932 makes the test pass. The test case I ran was testNfsUpgrade.

        In the repro patch, the exception is thrown from HttpServer2.openListeners, instead of the inner jetty + net classes. But IMHO this already proves that we need a join to have a successful immediate NN restart. (Arguably though, with this the NN shutdown will be slower, and potentially hang if somehow the webserver hangs in its shutdown. I think this would be the webserver's bug, and could be enhanced in a separate JIRA. E.g. add a timeout to the NN shutdown)

        Thanks Haohui Mai for helping understand the problem, and Chris Nauroth for the initial thought - turns out to be exactly the cause! Please feel free try out the repro patch and review patch 2.

        Show
        xiaochen Xiao Chen added a comment - Not sure if Liang Xie is still working on this, but I want to self-promote patch 2 with a way to reproduce. Attached is the reproduce patch which includes patch 2, that fails with the bind exception when restarting NN. Uncommenting the httpServer.join(); at NameNode.java#932 makes the test pass. The test case I ran was testNfsUpgrade . In the repro patch, the exception is thrown from HttpServer2.openListeners , instead of the inner jetty + net classes. But IMHO this already proves that we need a join to have a successful immediate NN restart. (Arguably though, with this the NN shutdown will be slower, and potentially hang if somehow the webserver hangs in its shutdown. I think this would be the webserver's bug, and could be enhanced in a separate JIRA. E.g. add a timeout to the NN shutdown) Thanks Haohui Mai for helping understand the problem, and Chris Nauroth for the initial thought - turns out to be exactly the cause! Please feel free try out the repro patch and review patch 2.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 0s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
        +1 mvninstall 10m 45s trunk passed
        +1 compile 13m 11s trunk passed with JDK v1.8.0_66
        +1 compile 12m 3s trunk passed with JDK v1.7.0_91
        +1 checkstyle 1m 16s trunk passed
        +1 mvnsite 2m 37s trunk passed
        +1 mvneclipse 0m 38s trunk passed
        +1 findbugs 5m 8s trunk passed
        +1 javadoc 2m 53s trunk passed with JDK v1.8.0_66
        +1 javadoc 3m 54s trunk passed with JDK v1.7.0_91
        +1 mvninstall 3m 10s the patch passed
        +1 compile 12m 36s the patch passed with JDK v1.8.0_66
        +1 javac 12m 36s the patch passed
        +1 compile 12m 9s the patch passed with JDK v1.7.0_91
        +1 javac 12m 9s the patch passed
        -1 checkstyle 1m 20s Patch generated 5 new checkstyle issues in root (total was 261, now 266).
        +1 mvnsite 2m 21s the patch passed
        +1 mvneclipse 0m 33s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        -1 findbugs 2m 24s hadoop-common-project/hadoop-common introduced 1 new FindBugs issues.
        +1 javadoc 2m 28s the patch passed with JDK v1.8.0_66
        +1 javadoc 3m 26s the patch passed with JDK v1.7.0_91
        -1 unit 8m 29s hadoop-common in the patch failed with JDK v1.8.0_66.
        -1 unit 81m 37s hadoop-hdfs in the patch failed with JDK v1.8.0_66.
        -1 unit 14m 1s hadoop-common in the patch failed with JDK v1.7.0_91.
        -1 unit 111m 4s hadoop-hdfs in the patch failed with JDK v1.7.0_91.
        -1 asflicense 0m 52s Patch generated 58 ASF License warnings.
        313m 0s



        Reason Tests
        FindBugs module:hadoop-common-project/hadoop-common
          Should org.apache.hadoop.http.HttpServer2$StopThread be a static inner class? At HttpServer2.java:inner class? At HttpServer2.java:[lines 120-136]
        JDK v1.8.0_66 Failed junit tests hadoop.http.TestHttpServer
          hadoop.http.TestHttpServerLifecycle
          hadoop.hdfs.server.namenode.TestINodeFile
          hadoop.hdfs.server.namenode.ha.TestHASafeMode
          hadoop.hdfs.server.namenode.ha.TestBootstrapStandby
          hadoop.hdfs.server.namenode.TestFSEditLogLoader
          hadoop.hdfs.server.datanode.TestDirectoryScanner
          hadoop.hdfs.server.namenode.TestDeleteRace
          hadoop.hdfs.server.namenode.TestStorageRestore
          hadoop.hdfs.TestEncryptionZones
          hadoop.hdfs.qjournal.TestNNWithQJM
          hadoop.hdfs.server.namenode.TestFileContextAcl
          hadoop.hdfs.TestDFSStripedOutputStreamWithFailure100
          hadoop.hdfs.qjournal.server.TestJournalNode
          hadoop.hdfs.server.namenode.ha.TestSeveralNameNodes
          hadoop.hdfs.server.namenode.TestNameNodeMetadataConsistency
          hadoop.hdfs.server.namenode.TestFSImageWithAcl
          hadoop.hdfs.server.namenode.TestAddBlock
          hadoop.hdfs.TestFileAppendRestart
          hadoop.hdfs.server.namenode.TestEditLog
          hadoop.hdfs.TestSafeMode
          hadoop.hdfs.server.namenode.TestHDFSConcat
          hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFS
          hadoop.hdfs.server.namenode.TestFSImageWithXAttr
          hadoop.hdfs.server.namenode.TestQuotaByStorageType
          hadoop.hdfs.TestFileCreation
          hadoop.hdfs.server.namenode.ha.TestQuotasWithHA
          hadoop.hdfs.qjournal.TestSecureNNWithQJM
          hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting
          hadoop.hdfs.server.namenode.ha.TestInitializeSharedEdits
          hadoop.hdfs.server.namenode.ha.TestHAStateTransitions
          hadoop.hdfs.server.namenode.ha.TestEditLogsDuringFailover
          hadoop.hdfs.TestLeaseRecovery2
          hadoop.hdfs.server.namenode.ha.TestEditLogTailer
          hadoop.hdfs.TestDistributedFileSystem
          hadoop.hdfs.server.datanode.TestDataNodeRollingUpgrade
          hadoop.hdfs.server.namenode.TestFileTruncate
          hadoop.hdfs.server.namenode.ha.TestStandbyCheckpoints
          hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistFiles
          hadoop.hdfs.server.namenode.ha.TestPendingCorruptDnMessages
          hadoop.hdfs.server.namenode.TestAddStripedBlocks
          hadoop.hdfs.server.namenode.TestDiskspaceQuotaUpdate
          hadoop.hdfs.qjournal.client.TestQuorumJournalManager
          hadoop.hdfs.server.blockmanagement.TestRBWBlockInvalidation
          hadoop.hdfs.server.namenode.TestCheckpoint
          hadoop.hdfs.server.namenode.ha.TestPipelinesFailover
          hadoop.hdfs.web.TestWebHDFSAcl
          hadoop.hdfs.server.namenode.snapshot.TestUpdatePipelineWithSnapshots
          hadoop.hdfs.TestErasureCodingPolicyWithSnapshot
          hadoop.hdfs.server.namenode.snapshot.TestOpenFilesWithSnapshot
          hadoop.hdfs.server.balancer.TestBalancerWithSaslDataTransfer
          hadoop.hdfs.tools.TestDFSAdminWithHA
          hadoop.hdfs.server.datanode.TestDataNodeMultipleRegistrations
          hadoop.hdfs.server.namenode.TestStartup
          hadoop.hdfs.server.namenode.ha.TestFailureToReadEdits
          hadoop.hdfs.security.TestDelegationTokenForProxyUser
          hadoop.hdfs.server.namenode.snapshot.TestSnapshotDeletion
          hadoop.hdfs.TestFileLengthOnClusterRestart
          hadoop.hdfs.TestFileAppend
          hadoop.hdfs.server.blockmanagement.TestPendingInvalidateBlock
          hadoop.hdfs.server.namenode.TestCacheDirectives
          hadoop.hdfs.server.namenode.snapshot.TestSnapshotBlocksMap
          hadoop.hdfs.TestRollingUpgradeRollback
          hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFSStriped
          hadoop.hdfs.server.namenode.ha.TestDFSUpgradeWithHA
          hadoop.hdfs.TestDFSUpgradeFromImage
          hadoop.hdfs.server.datanode.TestDataNodeMetrics
          hadoop.hdfs.TestPersistBlocks
          hadoop.hdfs.security.TestDelegationToken
          hadoop.hdfs.TestEncryptionZonesWithKMS
          hadoop.hdfs.server.namenode.TestRecoverStripedBlocks
          hadoop.hdfs.server.namenode.ha.TestHAMetrics
          hadoop.hdfs.server.namenode.TestFSImage
          hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistPolicy
          hadoop.hdfs.server.namenode.TestAddBlockRetry
          hadoop.hdfs.TestSafeModeWithStripedFile
          hadoop.hdfs.TestBlockStoragePolicy
          hadoop.hdfs.TestAbandonBlock
          hadoop.hdfs.server.namenode.TestNameNodeAcl
          hadoop.hdfs.TestEncryptedTransfer
          hadoop.hdfs.server.namenode.TestNamenodeRetryCache
          hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure
          hadoop.hdfs.server.namenode.TestListCorruptFileBlocks
          hadoop.hdfs.server.namenode.ha.TestDNFencing
        JDK v1.8.0_66 Timed out junit tests org.apache.hadoop.hdfs.server.namenode.ha.TestBootstrapStandbyWithQJM
          org.apache.hadoop.hdfs.TestDFSClientRetries
          org.apache.hadoop.hdfs.TestRollingUpgrade
          org.apache.hadoop.hdfs.server.balancer.TestBalancer
          org.apache.hadoop.hdfs.TestReadStripedFileWithDecoding
          org.apache.hadoop.hdfs.server.datanode.TestDirectoryScanner
        JDK v1.7.0_91 Failed junit tests hadoop.fs.shell.TestCopyPreserveFlag
          hadoop.http.TestHttpServer
          hadoop.ha.TestZKFailoverController
          hadoop.http.TestHttpServerLifecycle
          hadoop.ipc.TestIPC
          hadoop.fs.shell.find.TestPrint
          hadoop.fs.shell.find.TestName
          hadoop.test.TestTimedOutTestsListener
          hadoop.fs.shell.find.TestFind
          hadoop.hdfs.server.namenode.TestINodeFile
          hadoop.hdfs.server.namenode.ha.TestHASafeMode
          hadoop.hdfs.server.namenode.ha.TestBootstrapStandby
          hadoop.hdfs.TestDecommission
          hadoop.hdfs.server.namenode.TestFSEditLogLoader
          hadoop.hdfs.server.datanode.TestDirectoryScanner
          hadoop.hdfs.server.namenode.TestDeleteRace
          hadoop.hdfs.TestDFSRename
          hadoop.hdfs.server.namenode.TestStorageRestore
          hadoop.hdfs.TestEncryptionZones
          hadoop.hdfs.qjournal.TestNNWithQJM
          hadoop.hdfs.server.namenode.TestFileContextAcl
          hadoop.hdfs.qjournal.server.TestJournalNode
          hadoop.hdfs.server.namenode.TestNameNodeMetadataConsistency
          hadoop.hdfs.TestDFSStripedOutputStreamWithFailure060
          hadoop.hdfs.server.namenode.TestFSImageWithAcl
          hadoop.hdfs.server.namenode.TestAddBlock
          hadoop.hdfs.TestFileAppendRestart
          hadoop.hdfs.server.namenode.TestEditLog
          hadoop.hdfs.TestSafeMode
          hadoop.hdfs.server.namenode.TestHDFSConcat
          hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFS
          hadoop.hdfs.server.namenode.TestFSImageWithXAttr
          hadoop.hdfs.TestRollingUpgrade
          hadoop.hdfs.server.namenode.TestQuotaByStorageType
          hadoop.hdfs.TestFileCreation
          hadoop.hdfs.server.namenode.ha.TestQuotasWithHA
          hadoop.hdfs.qjournal.TestSecureNNWithQJM
          hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting
          hadoop.hdfs.server.namenode.ha.TestInitializeSharedEdits
          hadoop.hdfs.server.namenode.ha.TestHAStateTransitions
          hadoop.hdfs.TestRollingUpgradeDowngrade
          hadoop.hdfs.server.namenode.ha.TestEditLogsDuringFailover
          hadoop.hdfs.TestModTime
          hadoop.hdfs.TestLeaseRecovery2
          hadoop.hdfs.server.datanode.TestDataNodeRollingUpgrade
          hadoop.hdfs.server.namenode.TestFileTruncate
          hadoop.hdfs.server.namenode.ha.TestStandbyCheckpoints
          hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistFiles
          hadoop.hdfs.server.namenode.ha.TestPendingCorruptDnMessages
          hadoop.hdfs.server.namenode.TestAddStripedBlocks
          hadoop.hdfs.server.namenode.TestDiskspaceQuotaUpdate
          hadoop.hdfs.qjournal.client.TestQuorumJournalManager
          hadoop.hdfs.server.blockmanagement.TestRBWBlockInvalidation
          hadoop.hdfs.server.namenode.TestCheckpoint
          hadoop.hdfs.server.namenode.ha.TestPipelinesFailover
          hadoop.hdfs.web.TestWebHDFSAcl
          hadoop.hdfs.server.namenode.snapshot.TestUpdatePipelineWithSnapshots
          hadoop.hdfs.TestErasureCodingPolicyWithSnapshot
          hadoop.hdfs.server.namenode.snapshot.TestOpenFilesWithSnapshot
          hadoop.hdfs.server.balancer.TestBalancerWithSaslDataTransfer
          hadoop.hdfs.tools.TestDFSAdminWithHA
          hadoop.hdfs.server.datanode.TestDataNodeMultipleRegistrations
          hadoop.hdfs.server.namenode.TestStartup
          hadoop.hdfs.server.namenode.ha.TestFailureToReadEdits
          hadoop.hdfs.server.namenode.snapshot.TestSnapshotDeletion
          hadoop.hdfs.TestFileLengthOnClusterRestart
          hadoop.hdfs.TestFileAppend
          hadoop.hdfs.server.blockmanagement.TestPendingInvalidateBlock
          hadoop.hdfs.server.namenode.TestCacheDirectives
          hadoop.hdfs.server.namenode.snapshot.TestSnapshotBlocksMap
          hadoop.hdfs.TestRollingUpgradeRollback
          hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFSStriped
          hadoop.hdfs.server.namenode.ha.TestDFSUpgradeWithHA
          hadoop.hdfs.TestDFSUpgradeFromImage
          hadoop.hdfs.TestPersistBlocks
          hadoop.hdfs.TestDFSStripedOutputStreamWithFailure
          hadoop.hdfs.security.TestDelegationToken
          hadoop.hdfs.TestEncryptionZonesWithKMS
          hadoop.hdfs.TestDFSInotifyEventInputStream
          hadoop.hdfs.server.namenode.ha.TestHAMetrics
          hadoop.hdfs.server.namenode.TestFSImage
          hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistPolicy
          hadoop.hdfs.server.namenode.TestAddBlockRetry
          hadoop.hdfs.TestSafeModeWithStripedFile
          hadoop.hdfs.server.datanode.TestBlockScanner
          hadoop.hdfs.TestBlockStoragePolicy
          hadoop.hdfs.TestAbandonBlock
          hadoop.hdfs.server.namenode.TestNameNodeAcl
          hadoop.hdfs.TestEncryptedTransfer
          hadoop.hdfs.server.namenode.TestNamenodeRetryCache
          hadoop.hdfs.server.namenode.TestListCorruptFileBlocks
          hadoop.hdfs.server.namenode.ha.TestDNFencing
        JDK v1.7.0_91 Timed out junit tests org.apache.hadoop.hdfs.server.namenode.ha.TestBootstrapStandbyWithQJM



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:0ca8df7
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12776858/HDFS-7553.repro.patch
        JIRA Issue HDFS-7553
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux cb1982e3c1d5 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / f910e4f
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/13827/artifact/patchprocess/diff-checkstyle-root.txt
        findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/13827/artifact/patchprocess/new-findbugs-hadoop-common-project_hadoop-common.html
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/13827/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_66.txt
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/13827/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/13827/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_91.txt
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/13827/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_91.txt
        unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/13827/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HDFS-Build/13827/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HDFS-Build/13827/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_91.txt https://builds.apache.org/job/PreCommit-HDFS-Build/13827/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_91.txt
        JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/13827/testReport/
        asflicense https://builds.apache.org/job/PreCommit-HDFS-Build/13827/artifact/patchprocess/patch-asflicense-problems.txt
        modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: .
        Max memory used 75MB
        Powered by Apache Yetus 0.1.0-SNAPSHOT http://yetus.apache.org
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/13827/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. +1 mvninstall 10m 45s trunk passed +1 compile 13m 11s trunk passed with JDK v1.8.0_66 +1 compile 12m 3s trunk passed with JDK v1.7.0_91 +1 checkstyle 1m 16s trunk passed +1 mvnsite 2m 37s trunk passed +1 mvneclipse 0m 38s trunk passed +1 findbugs 5m 8s trunk passed +1 javadoc 2m 53s trunk passed with JDK v1.8.0_66 +1 javadoc 3m 54s trunk passed with JDK v1.7.0_91 +1 mvninstall 3m 10s the patch passed +1 compile 12m 36s the patch passed with JDK v1.8.0_66 +1 javac 12m 36s the patch passed +1 compile 12m 9s the patch passed with JDK v1.7.0_91 +1 javac 12m 9s the patch passed -1 checkstyle 1m 20s Patch generated 5 new checkstyle issues in root (total was 261, now 266). +1 mvnsite 2m 21s the patch passed +1 mvneclipse 0m 33s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. -1 findbugs 2m 24s hadoop-common-project/hadoop-common introduced 1 new FindBugs issues. +1 javadoc 2m 28s the patch passed with JDK v1.8.0_66 +1 javadoc 3m 26s the patch passed with JDK v1.7.0_91 -1 unit 8m 29s hadoop-common in the patch failed with JDK v1.8.0_66. -1 unit 81m 37s hadoop-hdfs in the patch failed with JDK v1.8.0_66. -1 unit 14m 1s hadoop-common in the patch failed with JDK v1.7.0_91. -1 unit 111m 4s hadoop-hdfs in the patch failed with JDK v1.7.0_91. -1 asflicense 0m 52s Patch generated 58 ASF License warnings. 313m 0s Reason Tests FindBugs module:hadoop-common-project/hadoop-common   Should org.apache.hadoop.http.HttpServer2$StopThread be a static inner class? At HttpServer2.java:inner class? At HttpServer2.java: [lines 120-136] JDK v1.8.0_66 Failed junit tests hadoop.http.TestHttpServer   hadoop.http.TestHttpServerLifecycle   hadoop.hdfs.server.namenode.TestINodeFile   hadoop.hdfs.server.namenode.ha.TestHASafeMode   hadoop.hdfs.server.namenode.ha.TestBootstrapStandby   hadoop.hdfs.server.namenode.TestFSEditLogLoader   hadoop.hdfs.server.datanode.TestDirectoryScanner   hadoop.hdfs.server.namenode.TestDeleteRace   hadoop.hdfs.server.namenode.TestStorageRestore   hadoop.hdfs.TestEncryptionZones   hadoop.hdfs.qjournal.TestNNWithQJM   hadoop.hdfs.server.namenode.TestFileContextAcl   hadoop.hdfs.TestDFSStripedOutputStreamWithFailure100   hadoop.hdfs.qjournal.server.TestJournalNode   hadoop.hdfs.server.namenode.ha.TestSeveralNameNodes   hadoop.hdfs.server.namenode.TestNameNodeMetadataConsistency   hadoop.hdfs.server.namenode.TestFSImageWithAcl   hadoop.hdfs.server.namenode.TestAddBlock   hadoop.hdfs.TestFileAppendRestart   hadoop.hdfs.server.namenode.TestEditLog   hadoop.hdfs.TestSafeMode   hadoop.hdfs.server.namenode.TestHDFSConcat   hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFS   hadoop.hdfs.server.namenode.TestFSImageWithXAttr   hadoop.hdfs.server.namenode.TestQuotaByStorageType   hadoop.hdfs.TestFileCreation   hadoop.hdfs.server.namenode.ha.TestQuotasWithHA   hadoop.hdfs.qjournal.TestSecureNNWithQJM   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting   hadoop.hdfs.server.namenode.ha.TestInitializeSharedEdits   hadoop.hdfs.server.namenode.ha.TestHAStateTransitions   hadoop.hdfs.server.namenode.ha.TestEditLogsDuringFailover   hadoop.hdfs.TestLeaseRecovery2   hadoop.hdfs.server.namenode.ha.TestEditLogTailer   hadoop.hdfs.TestDistributedFileSystem   hadoop.hdfs.server.datanode.TestDataNodeRollingUpgrade   hadoop.hdfs.server.namenode.TestFileTruncate   hadoop.hdfs.server.namenode.ha.TestStandbyCheckpoints   hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistFiles   hadoop.hdfs.server.namenode.ha.TestPendingCorruptDnMessages   hadoop.hdfs.server.namenode.TestAddStripedBlocks   hadoop.hdfs.server.namenode.TestDiskspaceQuotaUpdate   hadoop.hdfs.qjournal.client.TestQuorumJournalManager   hadoop.hdfs.server.blockmanagement.TestRBWBlockInvalidation   hadoop.hdfs.server.namenode.TestCheckpoint   hadoop.hdfs.server.namenode.ha.TestPipelinesFailover   hadoop.hdfs.web.TestWebHDFSAcl   hadoop.hdfs.server.namenode.snapshot.TestUpdatePipelineWithSnapshots   hadoop.hdfs.TestErasureCodingPolicyWithSnapshot   hadoop.hdfs.server.namenode.snapshot.TestOpenFilesWithSnapshot   hadoop.hdfs.server.balancer.TestBalancerWithSaslDataTransfer   hadoop.hdfs.tools.TestDFSAdminWithHA   hadoop.hdfs.server.datanode.TestDataNodeMultipleRegistrations   hadoop.hdfs.server.namenode.TestStartup   hadoop.hdfs.server.namenode.ha.TestFailureToReadEdits   hadoop.hdfs.security.TestDelegationTokenForProxyUser   hadoop.hdfs.server.namenode.snapshot.TestSnapshotDeletion   hadoop.hdfs.TestFileLengthOnClusterRestart   hadoop.hdfs.TestFileAppend   hadoop.hdfs.server.blockmanagement.TestPendingInvalidateBlock   hadoop.hdfs.server.namenode.TestCacheDirectives   hadoop.hdfs.server.namenode.snapshot.TestSnapshotBlocksMap   hadoop.hdfs.TestRollingUpgradeRollback   hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFSStriped   hadoop.hdfs.server.namenode.ha.TestDFSUpgradeWithHA   hadoop.hdfs.TestDFSUpgradeFromImage   hadoop.hdfs.server.datanode.TestDataNodeMetrics   hadoop.hdfs.TestPersistBlocks   hadoop.hdfs.security.TestDelegationToken   hadoop.hdfs.TestEncryptionZonesWithKMS   hadoop.hdfs.server.namenode.TestRecoverStripedBlocks   hadoop.hdfs.server.namenode.ha.TestHAMetrics   hadoop.hdfs.server.namenode.TestFSImage   hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistPolicy   hadoop.hdfs.server.namenode.TestAddBlockRetry   hadoop.hdfs.TestSafeModeWithStripedFile   hadoop.hdfs.TestBlockStoragePolicy   hadoop.hdfs.TestAbandonBlock   hadoop.hdfs.server.namenode.TestNameNodeAcl   hadoop.hdfs.TestEncryptedTransfer   hadoop.hdfs.server.namenode.TestNamenodeRetryCache   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure   hadoop.hdfs.server.namenode.TestListCorruptFileBlocks   hadoop.hdfs.server.namenode.ha.TestDNFencing JDK v1.8.0_66 Timed out junit tests org.apache.hadoop.hdfs.server.namenode.ha.TestBootstrapStandbyWithQJM   org.apache.hadoop.hdfs.TestDFSClientRetries   org.apache.hadoop.hdfs.TestRollingUpgrade   org.apache.hadoop.hdfs.server.balancer.TestBalancer   org.apache.hadoop.hdfs.TestReadStripedFileWithDecoding   org.apache.hadoop.hdfs.server.datanode.TestDirectoryScanner JDK v1.7.0_91 Failed junit tests hadoop.fs.shell.TestCopyPreserveFlag   hadoop.http.TestHttpServer   hadoop.ha.TestZKFailoverController   hadoop.http.TestHttpServerLifecycle   hadoop.ipc.TestIPC   hadoop.fs.shell.find.TestPrint   hadoop.fs.shell.find.TestName   hadoop.test.TestTimedOutTestsListener   hadoop.fs.shell.find.TestFind   hadoop.hdfs.server.namenode.TestINodeFile   hadoop.hdfs.server.namenode.ha.TestHASafeMode   hadoop.hdfs.server.namenode.ha.TestBootstrapStandby   hadoop.hdfs.TestDecommission   hadoop.hdfs.server.namenode.TestFSEditLogLoader   hadoop.hdfs.server.datanode.TestDirectoryScanner   hadoop.hdfs.server.namenode.TestDeleteRace   hadoop.hdfs.TestDFSRename   hadoop.hdfs.server.namenode.TestStorageRestore   hadoop.hdfs.TestEncryptionZones   hadoop.hdfs.qjournal.TestNNWithQJM   hadoop.hdfs.server.namenode.TestFileContextAcl   hadoop.hdfs.qjournal.server.TestJournalNode   hadoop.hdfs.server.namenode.TestNameNodeMetadataConsistency   hadoop.hdfs.TestDFSStripedOutputStreamWithFailure060   hadoop.hdfs.server.namenode.TestFSImageWithAcl   hadoop.hdfs.server.namenode.TestAddBlock   hadoop.hdfs.TestFileAppendRestart   hadoop.hdfs.server.namenode.TestEditLog   hadoop.hdfs.TestSafeMode   hadoop.hdfs.server.namenode.TestHDFSConcat   hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFS   hadoop.hdfs.server.namenode.TestFSImageWithXAttr   hadoop.hdfs.TestRollingUpgrade   hadoop.hdfs.server.namenode.TestQuotaByStorageType   hadoop.hdfs.TestFileCreation   hadoop.hdfs.server.namenode.ha.TestQuotasWithHA   hadoop.hdfs.qjournal.TestSecureNNWithQJM   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting   hadoop.hdfs.server.namenode.ha.TestInitializeSharedEdits   hadoop.hdfs.server.namenode.ha.TestHAStateTransitions   hadoop.hdfs.TestRollingUpgradeDowngrade   hadoop.hdfs.server.namenode.ha.TestEditLogsDuringFailover   hadoop.hdfs.TestModTime   hadoop.hdfs.TestLeaseRecovery2   hadoop.hdfs.server.datanode.TestDataNodeRollingUpgrade   hadoop.hdfs.server.namenode.TestFileTruncate   hadoop.hdfs.server.namenode.ha.TestStandbyCheckpoints   hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistFiles   hadoop.hdfs.server.namenode.ha.TestPendingCorruptDnMessages   hadoop.hdfs.server.namenode.TestAddStripedBlocks   hadoop.hdfs.server.namenode.TestDiskspaceQuotaUpdate   hadoop.hdfs.qjournal.client.TestQuorumJournalManager   hadoop.hdfs.server.blockmanagement.TestRBWBlockInvalidation   hadoop.hdfs.server.namenode.TestCheckpoint   hadoop.hdfs.server.namenode.ha.TestPipelinesFailover   hadoop.hdfs.web.TestWebHDFSAcl   hadoop.hdfs.server.namenode.snapshot.TestUpdatePipelineWithSnapshots   hadoop.hdfs.TestErasureCodingPolicyWithSnapshot   hadoop.hdfs.server.namenode.snapshot.TestOpenFilesWithSnapshot   hadoop.hdfs.server.balancer.TestBalancerWithSaslDataTransfer   hadoop.hdfs.tools.TestDFSAdminWithHA   hadoop.hdfs.server.datanode.TestDataNodeMultipleRegistrations   hadoop.hdfs.server.namenode.TestStartup   hadoop.hdfs.server.namenode.ha.TestFailureToReadEdits   hadoop.hdfs.server.namenode.snapshot.TestSnapshotDeletion   hadoop.hdfs.TestFileLengthOnClusterRestart   hadoop.hdfs.TestFileAppend   hadoop.hdfs.server.blockmanagement.TestPendingInvalidateBlock   hadoop.hdfs.server.namenode.TestCacheDirectives   hadoop.hdfs.server.namenode.snapshot.TestSnapshotBlocksMap   hadoop.hdfs.TestRollingUpgradeRollback   hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFSStriped   hadoop.hdfs.server.namenode.ha.TestDFSUpgradeWithHA   hadoop.hdfs.TestDFSUpgradeFromImage   hadoop.hdfs.TestPersistBlocks   hadoop.hdfs.TestDFSStripedOutputStreamWithFailure   hadoop.hdfs.security.TestDelegationToken   hadoop.hdfs.TestEncryptionZonesWithKMS   hadoop.hdfs.TestDFSInotifyEventInputStream   hadoop.hdfs.server.namenode.ha.TestHAMetrics   hadoop.hdfs.server.namenode.TestFSImage   hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistPolicy   hadoop.hdfs.server.namenode.TestAddBlockRetry   hadoop.hdfs.TestSafeModeWithStripedFile   hadoop.hdfs.server.datanode.TestBlockScanner   hadoop.hdfs.TestBlockStoragePolicy   hadoop.hdfs.TestAbandonBlock   hadoop.hdfs.server.namenode.TestNameNodeAcl   hadoop.hdfs.TestEncryptedTransfer   hadoop.hdfs.server.namenode.TestNamenodeRetryCache   hadoop.hdfs.server.namenode.TestListCorruptFileBlocks   hadoop.hdfs.server.namenode.ha.TestDNFencing JDK v1.7.0_91 Timed out junit tests org.apache.hadoop.hdfs.server.namenode.ha.TestBootstrapStandbyWithQJM Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12776858/HDFS-7553.repro.patch JIRA Issue HDFS-7553 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux cb1982e3c1d5 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / f910e4f findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/13827/artifact/patchprocess/diff-checkstyle-root.txt findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/13827/artifact/patchprocess/new-findbugs-hadoop-common-project_hadoop-common.html unit https://builds.apache.org/job/PreCommit-HDFS-Build/13827/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_66.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/13827/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/13827/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_91.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/13827/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_91.txt unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/13827/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HDFS-Build/13827/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HDFS-Build/13827/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_91.txt https://builds.apache.org/job/PreCommit-HDFS-Build/13827/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_91.txt JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/13827/testReport/ asflicense https://builds.apache.org/job/PreCommit-HDFS-Build/13827/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: . Max memory used 75MB Powered by Apache Yetus 0.1.0-SNAPSHOT http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HDFS-Build/13827/console This message was automatically generated.
        Hide
        xiaochen Xiao Chen added a comment -

        Jenkins failed desperately on the patch reproducing the exception.
        Please review patch 2. After that I think we can rerun Jenkins against patch 2 since the last run was 1 month ago (or modified patch 3 based on comments). Thank you!

        Show
        xiaochen Xiao Chen added a comment - Jenkins failed desperately on the patch reproducing the exception. Please review patch 2. After that I think we can rerun Jenkins against patch 2 since the last run was 1 month ago (or modified patch 3 based on comments). Thank you!
        Hide
        cnauroth Chris Nauroth added a comment -

        Xiao Chen, thank you for picking this up.

        Seeing the change now, I'm hesitant to commit the join call in NameNode. As you said, it could change shutdown behavior and introduce a slowdown or a hang. The need to join is an artificial requirement of the tests. In real production usage, this would rely on process shutdown at the OS level.

        Instead, maybe we could approach this as more of a test-only change. We could add a NameNode#getHttpServer method, annotated VisibleForTesting. Then, MiniDFSCluster#shutdown and MiniDFSCluster#shutdownNameNode could be changed to call that method after the current NameNode shutdown logic, and then call NameNodeHttpServer#join. This would help with shutdown for tests, but it would leave the main shutdown sequence unaltered.

        Let me know your thoughts.

        Show
        cnauroth Chris Nauroth added a comment - Xiao Chen , thank you for picking this up. Seeing the change now, I'm hesitant to commit the join call in NameNode . As you said, it could change shutdown behavior and introduce a slowdown or a hang. The need to join is an artificial requirement of the tests. In real production usage, this would rely on process shutdown at the OS level. Instead, maybe we could approach this as more of a test-only change. We could add a NameNode#getHttpServer method, annotated VisibleForTesting . Then, MiniDFSCluster#shutdown and MiniDFSCluster#shutdownNameNode could be changed to call that method after the current NameNode shutdown logic, and then call NameNodeHttpServer#join . This would help with shutdown for tests, but it would leave the main shutdown sequence unaltered. Let me know your thoughts.
        Hide
        xiaochen Xiao Chen added a comment -

        Hi Chris Nauroth,

        Thanks for reviewing! I agree that joining the httpserver is only required in the tests, and changing that may have too huge of impact.
        I will go with the advice of adding the join as a test-only fix. A patch is coming momentarily.

        Show
        xiaochen Xiao Chen added a comment - Hi Chris Nauroth , Thanks for reviewing! I agree that joining the httpserver is only required in the tests, and changing that may have too huge of impact. I will go with the advice of adding the join as a test-only fix. A patch is coming momentarily.
        Hide
        xiaochen Xiao Chen added a comment -

        Attached patch 3 to reflex the above discussion. I verified again with my ugly repro patch, the test fails without the join in MiniDFSCluster#shutdownNameNode, and passes with it.

        Show
        xiaochen Xiao Chen added a comment - Attached patch 3 to reflex the above discussion. I verified again with my ugly repro patch, the test fails without the join in MiniDFSCluster#shutdownNameNode , and passes with it.
        Hide
        cnauroth Chris Nauroth added a comment -

        I think patch v03 is the right idea. Here are a few comments.

        1. I'd like to make one more change in MiniDFSCluster. In the shutdown method, there is another spot that duplicates the calls to NameNode#stop and NameNode#join. I'd like to add the call to NameNodeHttpServer#join there too. This isn't directly related to the test failure that was reported originally, but it can help protect us from other similar problems in the future. Possibly consider a helper method to refactor some of the duplication.
        2. We should not swallow InterruptedException. The Hadoop code has a bad habit of doing this, but it's a bad practice, because it can cause harm to other parts of the code that expect to have visbility of the thread's interrupted status. In the catch (InterruptedException e) block, add a call to Thread.currentThread().interrupt() to restore the interrupted status.

        Thank you!

        Show
        cnauroth Chris Nauroth added a comment - I think patch v03 is the right idea. Here are a few comments. I'd like to make one more change in MiniDFSCluster . In the shutdown method, there is another spot that duplicates the calls to NameNode#stop and NameNode#join . I'd like to add the call to NameNodeHttpServer#join there too. This isn't directly related to the test failure that was reported originally, but it can help protect us from other similar problems in the future. Possibly consider a helper method to refactor some of the duplication. We should not swallow InterruptedException . The Hadoop code has a bad habit of doing this, but it's a bad practice, because it can cause harm to other parts of the code that expect to have visbility of the thread's interrupted status. In the catch (InterruptedException e) block, add a call to Thread.currentThread().interrupt() to restore the interrupted status. Thank you!
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 0s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 8m 0s trunk passed
        +1 compile 0m 37s trunk passed with JDK v1.8.0_66
        +1 compile 0m 41s trunk passed with JDK v1.7.0_91
        +1 checkstyle 0m 15s trunk passed
        +1 mvnsite 0m 52s trunk passed
        +1 mvneclipse 0m 14s trunk passed
        +1 findbugs 1m 54s trunk passed
        +1 javadoc 1m 5s trunk passed with JDK v1.8.0_66
        +1 javadoc 1m 47s trunk passed with JDK v1.7.0_91
        +1 mvninstall 0m 47s the patch passed
        +1 compile 0m 38s the patch passed with JDK v1.8.0_66
        +1 javac 0m 38s the patch passed
        +1 compile 0m 40s the patch passed with JDK v1.7.0_91
        +1 javac 0m 40s the patch passed
        +1 checkstyle 0m 14s the patch passed
        +1 mvnsite 0m 0s the patch passed
        +1 mvneclipse 0m 14s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        +1 findbugs 2m 0s the patch passed
        +1 javadoc 1m 5s the patch passed with JDK v1.8.0_66
        +1 javadoc 1m 45s the patch passed with JDK v1.7.0_91
        -1 unit 51m 43s hadoop-hdfs in the patch failed with JDK v1.8.0_66.
        -1 unit 51m 12s hadoop-hdfs in the patch failed with JDK v1.7.0_91.
        +1 asflicense 0m 26s Patch does not generate ASF License warnings.
        128m 45s



        Reason Tests
        JDK v1.8.0_66 Failed junit tests hadoop.hdfs.server.namenode.TestNNThroughputBenchmark
          hadoop.hdfs.server.datanode.TestDataNodeMetrics
          hadoop.hdfs.TestDFSUpgradeFromImage
        JDK v1.7.0_91 Failed junit tests hadoop.hdfs.TestDFSStripedOutputStreamWithFailure010
          hadoop.hdfs.server.datanode.TestBlockReplacement



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:0ca8df7
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12778932/HDFS-7553.03.patch
        JIRA Issue HDFS-7553
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 138ee840e772 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / a0de702
        findbugs v3.0.0
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/13967/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/13967/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_91.txt
        unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/13967/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HDFS-Build/13967/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_91.txt
        JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/13967/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Max memory used 76MB
        Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/13967/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 8m 0s trunk passed +1 compile 0m 37s trunk passed with JDK v1.8.0_66 +1 compile 0m 41s trunk passed with JDK v1.7.0_91 +1 checkstyle 0m 15s trunk passed +1 mvnsite 0m 52s trunk passed +1 mvneclipse 0m 14s trunk passed +1 findbugs 1m 54s trunk passed +1 javadoc 1m 5s trunk passed with JDK v1.8.0_66 +1 javadoc 1m 47s trunk passed with JDK v1.7.0_91 +1 mvninstall 0m 47s the patch passed +1 compile 0m 38s the patch passed with JDK v1.8.0_66 +1 javac 0m 38s the patch passed +1 compile 0m 40s the patch passed with JDK v1.7.0_91 +1 javac 0m 40s the patch passed +1 checkstyle 0m 14s the patch passed +1 mvnsite 0m 0s the patch passed +1 mvneclipse 0m 14s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 2m 0s the patch passed +1 javadoc 1m 5s the patch passed with JDK v1.8.0_66 +1 javadoc 1m 45s the patch passed with JDK v1.7.0_91 -1 unit 51m 43s hadoop-hdfs in the patch failed with JDK v1.8.0_66. -1 unit 51m 12s hadoop-hdfs in the patch failed with JDK v1.7.0_91. +1 asflicense 0m 26s Patch does not generate ASF License warnings. 128m 45s Reason Tests JDK v1.8.0_66 Failed junit tests hadoop.hdfs.server.namenode.TestNNThroughputBenchmark   hadoop.hdfs.server.datanode.TestDataNodeMetrics   hadoop.hdfs.TestDFSUpgradeFromImage JDK v1.7.0_91 Failed junit tests hadoop.hdfs.TestDFSStripedOutputStreamWithFailure010   hadoop.hdfs.server.datanode.TestBlockReplacement Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12778932/HDFS-7553.03.patch JIRA Issue HDFS-7553 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 138ee840e772 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / a0de702 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/13967/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/13967/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_91.txt unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/13967/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HDFS-Build/13967/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_91.txt JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/13967/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Max memory used 76MB Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HDFS-Build/13967/console This message was automatically generated.
        Hide
        xiaochen Xiao Chen added a comment -

        Thanks Chris. Both comments make sense to me, and patch 4 reflexes them.

        I also reviewed other usages of NameNode#stop and NameNode#join, and updated the code to provide a NameNode#joinHttpServer instead of a getter. This way, other tests can easily join the http server as needed. Currently I only see 1 place that needs this (TestStartup). The patch includes this as well.

        Show
        xiaochen Xiao Chen added a comment - Thanks Chris. Both comments make sense to me, and patch 4 reflexes them. I also reviewed other usages of NameNode#stop and NameNode#join , and updated the code to provide a NameNode#joinHttpServer instead of a getter. This way, other tests can easily join the http server as needed. Currently I only see 1 place that needs this (TestStartup). The patch includes this as well.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 0s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
        +1 mvninstall 7m 59s trunk passed
        +1 compile 0m 44s trunk passed with JDK v1.8.0_66
        +1 compile 0m 43s trunk passed with JDK v1.7.0_91
        +1 checkstyle 0m 16s trunk passed
        +1 mvnsite 0m 55s trunk passed
        +1 mvneclipse 0m 14s trunk passed
        +1 findbugs 1m 58s trunk passed
        +1 javadoc 1m 11s trunk passed with JDK v1.8.0_66
        +1 javadoc 1m 53s trunk passed with JDK v1.7.0_91
        +1 mvninstall 0m 51s the patch passed
        +1 compile 0m 45s the patch passed with JDK v1.8.0_66
        +1 javac 0m 45s the patch passed
        +1 compile 0m 44s the patch passed with JDK v1.7.0_91
        +1 javac 0m 44s the patch passed
        +1 checkstyle 0m 17s the patch passed
        +1 mvnsite 0m 55s the patch passed
        +1 mvneclipse 0m 14s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        +1 findbugs 2m 7s the patch passed
        +1 javadoc 1m 9s the patch passed with JDK v1.8.0_66
        +1 javadoc 1m 52s the patch passed with JDK v1.7.0_91
        -1 unit 56m 48s hadoop-hdfs in the patch failed with JDK v1.8.0_66.
        -1 unit 56m 21s hadoop-hdfs in the patch failed with JDK v1.7.0_91.
        +1 asflicense 0m 25s Patch does not generate ASF License warnings.
        141m 8s



        Reason Tests
        JDK v1.8.0_66 Failed junit tests hadoop.hdfs.TestDFSRollback
          hadoop.hdfs.server.balancer.TestBalancerWithMultipleNameNodes
          hadoop.hdfs.TestDFSUpgrade
          hadoop.hdfs.server.blockmanagement.TestBlockManager
          hadoop.hdfs.TestDFSFinalize
          hadoop.hdfs.server.datanode.TestBlockScanner
          hadoop.hdfs.server.namenode.TestBackupNode
          hadoop.hdfs.server.namenode.TestNNThroughputBenchmark
        JDK v1.7.0_91 Failed junit tests hadoop.hdfs.server.namenode.snapshot.TestNestedSnapshots
          hadoop.hdfs.TestDFSRollback
          hadoop.hdfs.TestDFSUpgrade
          hadoop.hdfs.TestDFSFinalize
          hadoop.hdfs.server.namenode.TestBackupNode
          hadoop.hdfs.server.namenode.TestNNThroughputBenchmark



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:0ca8df7
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12778967/HDFS-7553.04.patch
        JIRA Issue HDFS-7553
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 9cb8dc8107b2 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 114b590
        findbugs v3.0.0
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/13970/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/13970/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_91.txt
        unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/13970/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HDFS-Build/13970/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_91.txt
        JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/13970/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Max memory used 76MB
        Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/13970/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. +1 mvninstall 7m 59s trunk passed +1 compile 0m 44s trunk passed with JDK v1.8.0_66 +1 compile 0m 43s trunk passed with JDK v1.7.0_91 +1 checkstyle 0m 16s trunk passed +1 mvnsite 0m 55s trunk passed +1 mvneclipse 0m 14s trunk passed +1 findbugs 1m 58s trunk passed +1 javadoc 1m 11s trunk passed with JDK v1.8.0_66 +1 javadoc 1m 53s trunk passed with JDK v1.7.0_91 +1 mvninstall 0m 51s the patch passed +1 compile 0m 45s the patch passed with JDK v1.8.0_66 +1 javac 0m 45s the patch passed +1 compile 0m 44s the patch passed with JDK v1.7.0_91 +1 javac 0m 44s the patch passed +1 checkstyle 0m 17s the patch passed +1 mvnsite 0m 55s the patch passed +1 mvneclipse 0m 14s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 2m 7s the patch passed +1 javadoc 1m 9s the patch passed with JDK v1.8.0_66 +1 javadoc 1m 52s the patch passed with JDK v1.7.0_91 -1 unit 56m 48s hadoop-hdfs in the patch failed with JDK v1.8.0_66. -1 unit 56m 21s hadoop-hdfs in the patch failed with JDK v1.7.0_91. +1 asflicense 0m 25s Patch does not generate ASF License warnings. 141m 8s Reason Tests JDK v1.8.0_66 Failed junit tests hadoop.hdfs.TestDFSRollback   hadoop.hdfs.server.balancer.TestBalancerWithMultipleNameNodes   hadoop.hdfs.TestDFSUpgrade   hadoop.hdfs.server.blockmanagement.TestBlockManager   hadoop.hdfs.TestDFSFinalize   hadoop.hdfs.server.datanode.TestBlockScanner   hadoop.hdfs.server.namenode.TestBackupNode   hadoop.hdfs.server.namenode.TestNNThroughputBenchmark JDK v1.7.0_91 Failed junit tests hadoop.hdfs.server.namenode.snapshot.TestNestedSnapshots   hadoop.hdfs.TestDFSRollback   hadoop.hdfs.TestDFSUpgrade   hadoop.hdfs.TestDFSFinalize   hadoop.hdfs.server.namenode.TestBackupNode   hadoop.hdfs.server.namenode.TestNNThroughputBenchmark Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12778967/HDFS-7553.04.patch JIRA Issue HDFS-7553 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 9cb8dc8107b2 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 114b590 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/13970/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/13970/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_91.txt unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/13970/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HDFS-Build/13970/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_91.txt JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/13970/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Max memory used 76MB Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HDFS-Build/13970/console This message was automatically generated.
        Hide
        xiaochen Xiao Chen added a comment -

        Patch 5 fixed test failures. Specifically, patch 4 set nameNode to null after shutdown, but we have places in test that still reads config etc. from the object, which caused NPEs in the failed tests.

        Show
        xiaochen Xiao Chen added a comment - Patch 5 fixed test failures. Specifically, patch 4 set nameNode to null after shutdown, but we have places in test that still reads config etc. from the object, which caused NPEs in the failed tests.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 0s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
        +1 mvninstall 7m 55s trunk passed
        +1 compile 0m 42s trunk passed with JDK v1.8.0_66
        +1 compile 0m 44s trunk passed with JDK v1.7.0_91
        +1 checkstyle 0m 17s trunk passed
        +1 mvnsite 0m 55s trunk passed
        +1 mvneclipse 0m 14s trunk passed
        +1 findbugs 1m 59s trunk passed
        +1 javadoc 1m 9s trunk passed with JDK v1.8.0_66
        +1 javadoc 1m 50s trunk passed with JDK v1.7.0_91
        +1 mvninstall 0m 51s the patch passed
        +1 compile 0m 43s the patch passed with JDK v1.8.0_66
        +1 javac 0m 43s the patch passed
        +1 compile 0m 44s the patch passed with JDK v1.7.0_91
        +1 javac 0m 44s the patch passed
        +1 checkstyle 0m 16s the patch passed
        +1 mvnsite 0m 55s the patch passed
        +1 mvneclipse 0m 14s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        +1 findbugs 2m 7s the patch passed
        +1 javadoc 1m 11s the patch passed with JDK v1.8.0_66
        +1 javadoc 1m 53s the patch passed with JDK v1.7.0_91
        -1 unit 57m 24s hadoop-hdfs in the patch failed with JDK v1.8.0_66.
        -1 unit 55m 19s hadoop-hdfs in the patch failed with JDK v1.7.0_91.
        +1 asflicense 0m 30s Patch does not generate ASF License warnings.
        140m 36s



        Reason Tests
        JDK v1.8.0_66 Failed junit tests hadoop.hdfs.TestDFSClientRetries
          hadoop.hdfs.server.datanode.TestBlockScanner
          hadoop.hdfs.server.datanode.TestFsDatasetCache
          hadoop.hdfs.server.namenode.TestNNThroughputBenchmark
        JDK v1.7.0_91 Failed junit tests hadoop.hdfs.TestWriteReadStripedFile
          hadoop.hdfs.shortcircuit.TestShortCircuitCache
          hadoop.hdfs.TestDecommission
          hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:0ca8df7
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12778985/HDFS-7553.05.patch
        JIRA Issue HDFS-7553
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 986c6a9c0ba8 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 114b590
        findbugs v3.0.0
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/13971/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/13971/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_91.txt
        unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/13971/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HDFS-Build/13971/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_91.txt
        JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/13971/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Max memory used 76MB
        Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/13971/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. +1 mvninstall 7m 55s trunk passed +1 compile 0m 42s trunk passed with JDK v1.8.0_66 +1 compile 0m 44s trunk passed with JDK v1.7.0_91 +1 checkstyle 0m 17s trunk passed +1 mvnsite 0m 55s trunk passed +1 mvneclipse 0m 14s trunk passed +1 findbugs 1m 59s trunk passed +1 javadoc 1m 9s trunk passed with JDK v1.8.0_66 +1 javadoc 1m 50s trunk passed with JDK v1.7.0_91 +1 mvninstall 0m 51s the patch passed +1 compile 0m 43s the patch passed with JDK v1.8.0_66 +1 javac 0m 43s the patch passed +1 compile 0m 44s the patch passed with JDK v1.7.0_91 +1 javac 0m 44s the patch passed +1 checkstyle 0m 16s the patch passed +1 mvnsite 0m 55s the patch passed +1 mvneclipse 0m 14s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 2m 7s the patch passed +1 javadoc 1m 11s the patch passed with JDK v1.8.0_66 +1 javadoc 1m 53s the patch passed with JDK v1.7.0_91 -1 unit 57m 24s hadoop-hdfs in the patch failed with JDK v1.8.0_66. -1 unit 55m 19s hadoop-hdfs in the patch failed with JDK v1.7.0_91. +1 asflicense 0m 30s Patch does not generate ASF License warnings. 140m 36s Reason Tests JDK v1.8.0_66 Failed junit tests hadoop.hdfs.TestDFSClientRetries   hadoop.hdfs.server.datanode.TestBlockScanner   hadoop.hdfs.server.datanode.TestFsDatasetCache   hadoop.hdfs.server.namenode.TestNNThroughputBenchmark JDK v1.7.0_91 Failed junit tests hadoop.hdfs.TestWriteReadStripedFile   hadoop.hdfs.shortcircuit.TestShortCircuitCache   hadoop.hdfs.TestDecommission   hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12778985/HDFS-7553.05.patch JIRA Issue HDFS-7553 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 986c6a9c0ba8 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 114b590 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/13971/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/13971/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_91.txt unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/13971/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HDFS-Build/13971/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_91.txt JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/13971/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Max memory used 76MB Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HDFS-Build/13971/console This message was automatically generated.
        Hide
        xiaochen Xiao Chen added a comment -

        The failed tests looks unrelated to this patch.

        Show
        xiaochen Xiao Chen added a comment - The failed tests looks unrelated to this patch.
        Hide
        cnauroth Chris Nauroth added a comment -

        +1 for patch v05.

        Xiao Chen, I can commit this patch to trunk, but there is a conflict on the merge to branch-2 in MiniDFSCluster. Could you please post a separate branch-2 patch?

        Show
        cnauroth Chris Nauroth added a comment - +1 for patch v05. Xiao Chen , I can commit this patch to trunk, but there is a conflict on the merge to branch-2 in MiniDFSCluster . Could you please post a separate branch-2 patch?
        Hide
        xiaochen Xiao Chen added a comment -

        Attached branch-2 patch. The conflicts are in MiniDFSCluster#shutdownNameNode, due to HDFS-6440. I also pinged on that JIRA asking whether that will go to branch-2.
        Thanks again Chris Nauroth for the helpful review, commit and education!

        Show
        xiaochen Xiao Chen added a comment - Attached branch-2 patch. The conflicts are in MiniDFSCluster#shutdownNameNode , due to HDFS-6440 . I also pinged on that JIRA asking whether that will go to branch-2. Thanks again Chris Nauroth for the helpful review, commit and education!
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        -1 docker 0m 3s Docker failed to build yetus/hadoop:5d9212c.



        Subsystem Report/Notes
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12779146/HDFS-7553.branch-2.patch
        JIRA Issue HDFS-7553
        Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/13980/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment -1 docker 0m 3s Docker failed to build yetus/hadoop:5d9212c. Subsystem Report/Notes JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12779146/HDFS-7553.branch-2.patch JIRA Issue HDFS-7553 Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HDFS-Build/13980/console This message was automatically generated.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 0s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
        +1 mvninstall 14m 8s branch-2 passed
        +1 compile 1m 9s branch-2 passed with JDK v1.8.0_66
        +1 compile 0m 58s branch-2 passed with JDK v1.7.0_91
        +1 checkstyle 0m 23s branch-2 passed
        +1 mvnsite 1m 11s branch-2 passed
        +1 mvneclipse 0m 21s branch-2 passed
        +1 findbugs 2m 30s branch-2 passed
        +1 javadoc 1m 37s branch-2 passed with JDK v1.8.0_66
        +1 javadoc 2m 19s branch-2 passed with JDK v1.7.0_91
        +1 mvninstall 1m 0s the patch passed
        +1 compile 1m 5s the patch passed with JDK v1.8.0_66
        +1 javac 1m 5s the patch passed
        +1 compile 0m 54s the patch passed with JDK v1.7.0_91
        +1 javac 0m 54s the patch passed
        +1 checkstyle 0m 20s the patch passed
        +1 mvnsite 1m 8s the patch passed
        +1 mvneclipse 0m 13s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        +1 findbugs 2m 37s the patch passed
        +1 javadoc 1m 35s the patch passed with JDK v1.8.0_66
        +1 javadoc 2m 31s the patch passed with JDK v1.7.0_91
        -1 unit 68m 42s hadoop-hdfs in the patch failed with JDK v1.8.0_66.
        -1 unit 59m 23s hadoop-hdfs in the patch failed with JDK v1.7.0_91.
        +1 asflicense 0m 25s Patch does not generate ASF License warnings.
        168m 5s



        Reason Tests
        JDK v1.8.0_66 Failed junit tests hadoop.hdfs.TestDFSClientRetries
          hadoop.hdfs.security.TestDelegationTokenForProxyUser
          hadoop.hdfs.TestDFSUpgradeFromImage
          hadoop.hdfs.server.namenode.TestNNThroughputBenchmark
          hadoop.hdfs.TestEncryptionZones
          hadoop.hdfs.server.datanode.TestBlockScanner
          hadoop.hdfs.server.namenode.TestFSNamesystem
          hadoop.hdfs.TestHFlush
          hadoop.hdfs.shortcircuit.TestShortCircuitCache
          hadoop.hdfs.TestMissingBlocksAlert
        JDK v1.7.0_91 Failed junit tests hadoop.hdfs.server.datanode.TestBlockReplacement
          hadoop.hdfs.TestEncryptionZonesWithKMS
          hadoop.hdfs.shortcircuit.TestShortCircuitCache



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:babe025
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12779691/HDFS-7553-branch-2.05.patch
        JIRA Issue HDFS-7553
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 73544ca5e0a8 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision branch-2 / ca575e6
        Default Java 1.7.0_91
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91
        findbugs v3.0.0
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/14001/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/14001/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_91.txt
        unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/14001/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HDFS-Build/14001/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_91.txt
        JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/14001/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Max memory used 72MB
        Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/14001/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. +1 mvninstall 14m 8s branch-2 passed +1 compile 1m 9s branch-2 passed with JDK v1.8.0_66 +1 compile 0m 58s branch-2 passed with JDK v1.7.0_91 +1 checkstyle 0m 23s branch-2 passed +1 mvnsite 1m 11s branch-2 passed +1 mvneclipse 0m 21s branch-2 passed +1 findbugs 2m 30s branch-2 passed +1 javadoc 1m 37s branch-2 passed with JDK v1.8.0_66 +1 javadoc 2m 19s branch-2 passed with JDK v1.7.0_91 +1 mvninstall 1m 0s the patch passed +1 compile 1m 5s the patch passed with JDK v1.8.0_66 +1 javac 1m 5s the patch passed +1 compile 0m 54s the patch passed with JDK v1.7.0_91 +1 javac 0m 54s the patch passed +1 checkstyle 0m 20s the patch passed +1 mvnsite 1m 8s the patch passed +1 mvneclipse 0m 13s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 2m 37s the patch passed +1 javadoc 1m 35s the patch passed with JDK v1.8.0_66 +1 javadoc 2m 31s the patch passed with JDK v1.7.0_91 -1 unit 68m 42s hadoop-hdfs in the patch failed with JDK v1.8.0_66. -1 unit 59m 23s hadoop-hdfs in the patch failed with JDK v1.7.0_91. +1 asflicense 0m 25s Patch does not generate ASF License warnings. 168m 5s Reason Tests JDK v1.8.0_66 Failed junit tests hadoop.hdfs.TestDFSClientRetries   hadoop.hdfs.security.TestDelegationTokenForProxyUser   hadoop.hdfs.TestDFSUpgradeFromImage   hadoop.hdfs.server.namenode.TestNNThroughputBenchmark   hadoop.hdfs.TestEncryptionZones   hadoop.hdfs.server.datanode.TestBlockScanner   hadoop.hdfs.server.namenode.TestFSNamesystem   hadoop.hdfs.TestHFlush   hadoop.hdfs.shortcircuit.TestShortCircuitCache   hadoop.hdfs.TestMissingBlocksAlert JDK v1.7.0_91 Failed junit tests hadoop.hdfs.server.datanode.TestBlockReplacement   hadoop.hdfs.TestEncryptionZonesWithKMS   hadoop.hdfs.shortcircuit.TestShortCircuitCache Subsystem Report/Notes Docker Image:yetus/hadoop:babe025 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12779691/HDFS-7553-branch-2.05.patch JIRA Issue HDFS-7553 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 73544ca5e0a8 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision branch-2 / ca575e6 Default Java 1.7.0_91 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/14001/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/14001/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_91.txt unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/14001/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HDFS-Build/14001/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_91.txt JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/14001/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Max memory used 72MB Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HDFS-Build/14001/console This message was automatically generated.
        Hide
        cnauroth Chris Nauroth added a comment -

        I have committed this to trunk, branch-2 and branch-2.8. Xiao Chen, thank you for the patch.

        Show
        cnauroth Chris Nauroth added a comment - I have committed this to trunk, branch-2 and branch-2.8. Xiao Chen , thank you for the patch.
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-trunk-Commit #9033 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9033/)
        HDFS-7553. fix the TestDFSUpgradeWithHA due to BindException. (cnauroth: rev 99cf2ecee9c19231dea3620c053b2d8d71812fd6)

        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeHttpServer.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestStartup.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #9033 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9033/ ) HDFS-7553 . fix the TestDFSUpgradeWithHA due to BindException. (cnauroth: rev 99cf2ecee9c19231dea3620c053b2d8d71812fd6) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeHttpServer.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestStartup.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Hide
        xiaochen Xiao Chen added a comment -

        Thanks again Chris Nauroth for the review and commit! Also thanks for triggering a new jenkins. Happy new year.

        Show
        xiaochen Xiao Chen added a comment - Thanks again Chris Nauroth for the review and commit! Also thanks for triggering a new jenkins. Happy new year.

          People

          • Assignee:
            xiaochen Xiao Chen
            Reporter:
            xieliang007 Liang Xie
          • Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development