Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-10367

TestDFSShell.testMoveWithTargetPortEmpty fails with Address bind exception.

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: test
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Problem binding to [localhost:9820] java.net.BindException: Address already in use; For more details see:  http://wiki.apache.org/hadoop/BindException
      
      Stack Trace:
      java.net.BindException: Problem binding to [localhost:9820] java.net.BindException: Address already in use; For more details see:  http://wiki.apache.org/hadoop/BindException
      	at sun.nio.ch.Net.bind0(Native Method)
      	at sun.nio.ch.Net.bind(Net.java:444)
      	at sun.nio.ch.Net.bind(Net.java:436)
      	at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:214)
      	at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
      	at org.apache.hadoop.ipc.Server.bind(Server.java:530)
      	at org.apache.hadoop.ipc.Server$Listener.<init>(Server.java:793)
      	at org.apache.hadoop.ipc.Server.<init>(Server.java:2592)
      	at org.apache.hadoop.ipc.RPC$Server.<init>(RPC.java:958)
      	at org.apache.hadoop.ipc.ProtobufRpcEngine$Server.<init>(ProtobufRpcEngine.java:563)
      	at org.apache.hadoop.ipc.ProtobufRpcEngine.getServer(ProtobufRpcEngine.java:538)
      	at org.apache.hadoop.ipc.RPC$Builder.build(RPC.java:800)
      	at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.<init>(NameNodeRpcServer.java:426)
      	at org.apache.hadoop.hdfs.server.namenode.NameNode.createRpcServer(NameNode.java:783)
      	at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:710)
      	at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:924)
      	at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:903)
      	at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1620)
      	at org.apache.hadoop.hdfs.MiniDFSCluster.createNameNode(MiniDFSCluster.java:1247)
      	at org.apache.hadoop.hdfs.MiniDFSCluster.configureNameService(MiniDFSCluster.java:1016)
      	at org.apache.hadoop.hdfs.MiniDFSCluster.createNameNodesAndSetConf(MiniDFSCluster.java:891)
      	at org.apache.hadoop.hdfs.MiniDFSCluster.initMiniDFSCluster(MiniDFSCluster.java:823)
      	at org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:482)
      	at org.apache.hadoop.hdfs.MiniDFSCluster$Builder.build(MiniDFSCluster.java:441)
      	at org.apache.hadoop.hdfs.TestDFSShell.testMoveWithTargetPortEmpty(TestDFSShell.java:567)
      
      1. HDFS-10367.patch
        1 kB
        Brahma Reddy Battula
      2. HDFS-10367-002.patch
        3 kB
        Brahma Reddy Battula
      3. HDFS-10367-003.patch
        3 kB
        Brahma Reddy Battula
      4. HDFS-10367-004.patch
        3 kB
        Brahma Reddy Battula
      5. HDFS-10367-005.patch
        3 kB
        Brahma Reddy Battula
      6. HDFS-10367.005.patch
        3 kB
        Masatake Iwasaki

        Activity

        Hide
        brahmareddy Brahma Reddy Battula added a comment -

        Masatake Iwasaki thanks a lot for review and commit.. Raised HADOOP-13234 to handle random port improvement.

        Show
        brahmareddy Brahma Reddy Battula added a comment - Masatake Iwasaki thanks a lot for review and commit.. Raised HADOOP-13234 to handle random port improvement.
        Hide
        iwasakims Masatake Iwasaki added a comment -

        Committed. Thanks, Brahma Reddy Battula.

        Show
        iwasakims Masatake Iwasaki added a comment - Committed. Thanks, Brahma Reddy Battula .
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Hadoop-trunk-Commit #9899 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9899/)
        HDFS-10367. TestDFSShell.testMoveWithTargetPortEmpty fails with Address (iwasakims: rev aadb77e412ab9d4ad05a0bd8b37d547ba5adad03)

        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSShell.java
        • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/net/ServerSocketUtil.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-trunk-Commit #9899 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9899/ ) HDFS-10367 . TestDFSShell.testMoveWithTargetPortEmpty fails with Address (iwasakims: rev aadb77e412ab9d4ad05a0bd8b37d547ba5adad03) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSShell.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/net/ServerSocketUtil.java
        Hide
        iwasakims Masatake Iwasaki added a comment -

        +1, committing this.

        Show
        iwasakims Masatake Iwasaki added a comment - +1, committing this.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 25s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
        0 mvndep 0m 50s Maven dependency ordering for branch
        +1 mvninstall 7m 8s trunk passed
        +1 compile 7m 32s trunk passed
        +1 checkstyle 1m 35s trunk passed
        +1 mvnsite 2m 8s trunk passed
        +1 mvneclipse 0m 29s trunk passed
        +1 findbugs 3m 28s trunk passed
        +1 javadoc 2m 21s trunk passed
        0 mvndep 0m 13s Maven dependency ordering for patch
        +1 mvninstall 1m 47s the patch passed
        +1 compile 7m 37s the patch passed
        +1 javac 7m 37s the patch passed
        +1 checkstyle 1m 32s the patch passed
        +1 mvnsite 2m 0s the patch passed
        +1 mvneclipse 0m 28s the patch passed
        -1 whitespace 0m 0s The patch has 2 line(s) that end in whitespace. Use git apply --whitespace=fix.
        +1 findbugs 3m 35s the patch passed
        +1 javadoc 2m 19s the patch passed
        +1 unit 8m 52s hadoop-common in the patch passed.
        +1 unit 71m 55s hadoop-hdfs in the patch passed.
        +1 asflicense 0m 23s The patch does not generate ASF License warnings.
        127m 33s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:2c91fd8
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12807649/HDFS-10367.005.patch
        JIRA Issue HDFS-10367
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 36f338262329 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 16b1cc7
        Default Java 1.8.0_91
        findbugs v3.0.0
        whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/15632/artifact/patchprocess/whitespace-eol.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/15632/testReport/
        modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: .
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/15632/console
        Powered by Apache Yetus 0.3.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 25s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. 0 mvndep 0m 50s Maven dependency ordering for branch +1 mvninstall 7m 8s trunk passed +1 compile 7m 32s trunk passed +1 checkstyle 1m 35s trunk passed +1 mvnsite 2m 8s trunk passed +1 mvneclipse 0m 29s trunk passed +1 findbugs 3m 28s trunk passed +1 javadoc 2m 21s trunk passed 0 mvndep 0m 13s Maven dependency ordering for patch +1 mvninstall 1m 47s the patch passed +1 compile 7m 37s the patch passed +1 javac 7m 37s the patch passed +1 checkstyle 1m 32s the patch passed +1 mvnsite 2m 0s the patch passed +1 mvneclipse 0m 28s the patch passed -1 whitespace 0m 0s The patch has 2 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 findbugs 3m 35s the patch passed +1 javadoc 2m 19s the patch passed +1 unit 8m 52s hadoop-common in the patch passed. +1 unit 71m 55s hadoop-hdfs in the patch passed. +1 asflicense 0m 23s The patch does not generate ASF License warnings. 127m 33s Subsystem Report/Notes Docker Image:yetus/hadoop:2c91fd8 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12807649/HDFS-10367.005.patch JIRA Issue HDFS-10367 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 36f338262329 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 16b1cc7 Default Java 1.8.0_91 findbugs v3.0.0 whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/15632/artifact/patchprocess/whitespace-eol.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/15632/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: . Console output https://builds.apache.org/job/PreCommit-HDFS-Build/15632/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
        Hide
        brahmareddy Brahma Reddy Battula added a comment -

        Masatake Iwasaki I will raise randomport improvement tomorrow.hope this patch can be committed.

        Show
        brahmareddy Brahma Reddy Battula added a comment - Masatake Iwasaki I will raise randomport improvement tomorrow.hope this patch can be committed.
        Hide
        brahmareddy Brahma Reddy Battula added a comment -

        Uploaded the patch to address the above comment..

        Show
        brahmareddy Brahma Reddy Battula added a comment - Uploaded the patch to address the above comment..
        Hide
        iwasakims Masatake Iwasaki added a comment -

        Thanks for the update, Brahma Reddy Battula.

        +      } else {
        +        Thread.sleep(1000);
        +        tries++;
        +        if (tries >= retries) {
        +          throw new IOException(
        +              "Port is already in use; giving up after " + tries + " times.");
        +        }
        +      }
        

        should check tries before sleeping. I'm +1 if this is addressed.

        Show
        iwasakims Masatake Iwasaki added a comment - Thanks for the update, Brahma Reddy Battula . + } else { + Thread .sleep(1000); + tries++; + if (tries >= retries) { + throw new IOException( + "Port is already in use; giving up after " + tries + " times." ); + } + } should check tries before sleeping. I'm +1 if this is addressed.
        Hide
        brahmareddy Brahma Reddy Battula added a comment -

        Uploaded the patch to fix the checkstyle comment.

        Show
        brahmareddy Brahma Reddy Battula added a comment - Uploaded the patch to fix the checkstyle comment.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 11s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
        0 mvndep 0m 16s Maven dependency ordering for branch
        +1 mvninstall 7m 44s trunk passed
        +1 compile 7m 34s trunk passed
        +1 checkstyle 1m 27s trunk passed
        +1 mvnsite 2m 0s trunk passed
        +1 mvneclipse 0m 25s trunk passed
        +1 findbugs 3m 25s trunk passed
        +1 javadoc 2m 3s trunk passed
        0 mvndep 0m 13s Maven dependency ordering for patch
        +1 mvninstall 1m 34s the patch passed
        +1 compile 7m 30s the patch passed
        +1 javac 7m 30s the patch passed
        -1 checkstyle 1m 31s root: patch generated 1 new + 183 unchanged - 0 fixed = 184 total (was 183)
        +1 mvnsite 1m 59s the patch passed
        +1 mvneclipse 0m 26s the patch passed
        -1 whitespace 0m 0s The patch has 2 line(s) that end in whitespace. Use git apply --whitespace=fix.
        +1 findbugs 3m 56s the patch passed
        +1 javadoc 2m 11s the patch passed
        +1 unit 8m 20s hadoop-common in the patch passed.
        -1 unit 65m 12s hadoop-hdfs in the patch failed.
        +1 asflicense 0m 24s Patch does not generate ASF License warnings.
        119m 13s



        Reason Tests
        Failed junit tests hadoop.hdfs.TestAsyncDFSRename



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:2c91fd8
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12805821/HDFS-10367-003.patch
        JIRA Issue HDFS-10367
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 531dd6eadb8e 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / b4078bd
        Default Java 1.8.0_91
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/15537/artifact/patchprocess/diff-checkstyle-root.txt
        whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/15537/artifact/patchprocess/whitespace-eol.txt
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/15537/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
        unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/15537/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/15537/testReport/
        modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: .
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/15537/console
        Powered by Apache Yetus 0.2.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 11s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. 0 mvndep 0m 16s Maven dependency ordering for branch +1 mvninstall 7m 44s trunk passed +1 compile 7m 34s trunk passed +1 checkstyle 1m 27s trunk passed +1 mvnsite 2m 0s trunk passed +1 mvneclipse 0m 25s trunk passed +1 findbugs 3m 25s trunk passed +1 javadoc 2m 3s trunk passed 0 mvndep 0m 13s Maven dependency ordering for patch +1 mvninstall 1m 34s the patch passed +1 compile 7m 30s the patch passed +1 javac 7m 30s the patch passed -1 checkstyle 1m 31s root: patch generated 1 new + 183 unchanged - 0 fixed = 184 total (was 183) +1 mvnsite 1m 59s the patch passed +1 mvneclipse 0m 26s the patch passed -1 whitespace 0m 0s The patch has 2 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 findbugs 3m 56s the patch passed +1 javadoc 2m 11s the patch passed +1 unit 8m 20s hadoop-common in the patch passed. -1 unit 65m 12s hadoop-hdfs in the patch failed. +1 asflicense 0m 24s Patch does not generate ASF License warnings. 119m 13s Reason Tests Failed junit tests hadoop.hdfs.TestAsyncDFSRename Subsystem Report/Notes Docker Image:yetus/hadoop:2c91fd8 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12805821/HDFS-10367-003.patch JIRA Issue HDFS-10367 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 531dd6eadb8e 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / b4078bd Default Java 1.8.0_91 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/15537/artifact/patchprocess/diff-checkstyle-root.txt whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/15537/artifact/patchprocess/whitespace-eol.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/15537/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/15537/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/15537/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: . Console output https://builds.apache.org/job/PreCommit-HDFS-Build/15537/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
        Hide
        brahmareddy Brahma Reddy Battula added a comment - - edited

        Masatake Iwasaki

        we can add new method like waitPort(int port) by using isPortAvailable.

        uploaded the patch..kindly review.

        Assuming ServerSocketUtil#getPort could be implemented like below

        Make sense to me,will handle in separate jira..

        Show
        brahmareddy Brahma Reddy Battula added a comment - - edited Masatake Iwasaki we can add new method like waitPort(int port) by using isPortAvailable. uploaded the patch..kindly review. Assuming ServerSocketUtil#getPort could be implemented like below Make sense to me,will handle in separate jira..
        Hide
        iwasakims Masatake Iwasaki added a comment -
            while (true) {
              if ((tries > 0 || tryPort == 0) && !needSamePort) {
                tryPort = port + rand.nextInt(65535 - port);
              }
              if (tryPort == 0) {
                continue;
              }
              try (ServerSocket s = new ServerSocket(tryPort)) {
                LOG.info("Using port " + tryPort);
                return tryPort;
        

        Do you know the reason why manually choosing random number? I think we can get available random port by new ServerSocket(0).getLocalPort() and it's more portable. Assuming ServerSocketUtil#getPort could be implemented like below, we can add new method like waitPort(int port) by using isPortAvailable.

          public static int getPort(int port) throws IOException {
            if (isPortAvailable(port)) {
              return port;
            } else {
              return getRandomPort();
            }
          }
        
          private static boolean isPortAvailable(int port) {
            try (ServerSocket s = new ServerSocket(port)) {
              return true;
            } catch (IOException e) {
              return false;
            }
          }
        
          public static int getRandomPort() throws IOException {
            try (ServerSocket s = new ServerSocket(0)) {
              int port = s.getLocalPort();
              LOG.info("Using port " + port);
              return port;
            }
          }
        

        Though the code of ServerSocketUtil is not from this issue...

        Show
        iwasakims Masatake Iwasaki added a comment - while ( true ) { if ((tries > 0 || tryPort == 0) && !needSamePort) { tryPort = port + rand.nextInt(65535 - port); } if (tryPort == 0) { continue ; } try (ServerSocket s = new ServerSocket(tryPort)) { LOG.info( "Using port " + tryPort); return tryPort; Do you know the reason why manually choosing random number? I think we can get available random port by new ServerSocket(0).getLocalPort() and it's more portable. Assuming ServerSocketUtil#getPort could be implemented like below, we can add new method like waitPort(int port) by using isPortAvailable . public static int getPort( int port) throws IOException { if (isPortAvailable(port)) { return port; } else { return getRandomPort(); } } private static boolean isPortAvailable( int port) { try (ServerSocket s = new ServerSocket(port)) { return true ; } catch (IOException e) { return false ; } } public static int getRandomPort() throws IOException { try (ServerSocket s = new ServerSocket(0)) { int port = s.getLocalPort(); LOG.info( "Using port " + port); return port; } } Though the code of ServerSocketUtil is not from this issue...
        Hide
        brahmareddy Brahma Reddy Battula added a comment -

        Masatake Iwasaki and Mingliang Liu thanks for taking a look into this issue...

        Yes, we need to try with same port.. I missed that.. Uploaded the patch,kindly review..

        Show
        brahmareddy Brahma Reddy Battula added a comment - Masatake Iwasaki and Mingliang Liu thanks for taking a look into this issue... Yes, we need to try with same port.. I missed that.. Uploaded the patch,kindly review..
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 11s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 6m 40s trunk passed
        +1 compile 0m 40s trunk passed with JDK v1.8.0_91
        +1 compile 0m 41s trunk passed with JDK v1.7.0_95
        +1 checkstyle 0m 21s trunk passed
        +1 mvnsite 0m 51s trunk passed
        +1 mvneclipse 0m 14s trunk passed
        +1 findbugs 1m 55s trunk passed
        +1 javadoc 1m 5s trunk passed with JDK v1.8.0_91
        +1 javadoc 1m 44s trunk passed with JDK v1.7.0_95
        +1 mvninstall 0m 45s the patch passed
        +1 compile 0m 36s the patch passed with JDK v1.8.0_91
        +1 javac 0m 36s the patch passed
        +1 compile 0m 38s the patch passed with JDK v1.7.0_95
        +1 javac 0m 38s the patch passed
        +1 checkstyle 0m 18s the patch passed
        +1 mvnsite 0m 48s the patch passed
        +1 mvneclipse 0m 12s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        +1 findbugs 2m 6s the patch passed
        +1 javadoc 1m 1s the patch passed with JDK v1.8.0_91
        +1 javadoc 1m 44s the patch passed with JDK v1.7.0_95
        -1 unit 56m 35s hadoop-hdfs in the patch failed with JDK v1.8.0_91.
        -1 unit 54m 14s hadoop-hdfs in the patch failed with JDK v1.7.0_95.
        +1 asflicense 0m 21s Patch does not generate ASF License warnings.
        135m 38s



        Reason Tests
        JDK v1.8.0_91 Failed junit tests hadoop.hdfs.TestFileAppend
          hadoop.hdfs.TestRenameWhileOpen
          hadoop.hdfs.TestHFlush
        JDK v1.7.0_95 Failed junit tests hadoop.hdfs.TestHFlush
          hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:cf2ee45
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12802348/HDFS-10367.patch
        JIRA Issue HDFS-10367
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux df490bf45ecd 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 1268cf5
        Default Java 1.7.0_95
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_91 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
        findbugs v3.0.0
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/15367/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_91.txt
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/15367/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt
        unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/15367/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_91.txt https://builds.apache.org/job/PreCommit-HDFS-Build/15367/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt
        JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/15367/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/15367/console
        Powered by Apache Yetus 0.2.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 11s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 6m 40s trunk passed +1 compile 0m 40s trunk passed with JDK v1.8.0_91 +1 compile 0m 41s trunk passed with JDK v1.7.0_95 +1 checkstyle 0m 21s trunk passed +1 mvnsite 0m 51s trunk passed +1 mvneclipse 0m 14s trunk passed +1 findbugs 1m 55s trunk passed +1 javadoc 1m 5s trunk passed with JDK v1.8.0_91 +1 javadoc 1m 44s trunk passed with JDK v1.7.0_95 +1 mvninstall 0m 45s the patch passed +1 compile 0m 36s the patch passed with JDK v1.8.0_91 +1 javac 0m 36s the patch passed +1 compile 0m 38s the patch passed with JDK v1.7.0_95 +1 javac 0m 38s the patch passed +1 checkstyle 0m 18s the patch passed +1 mvnsite 0m 48s the patch passed +1 mvneclipse 0m 12s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 2m 6s the patch passed +1 javadoc 1m 1s the patch passed with JDK v1.8.0_91 +1 javadoc 1m 44s the patch passed with JDK v1.7.0_95 -1 unit 56m 35s hadoop-hdfs in the patch failed with JDK v1.8.0_91. -1 unit 54m 14s hadoop-hdfs in the patch failed with JDK v1.7.0_95. +1 asflicense 0m 21s Patch does not generate ASF License warnings. 135m 38s Reason Tests JDK v1.8.0_91 Failed junit tests hadoop.hdfs.TestFileAppend   hadoop.hdfs.TestRenameWhileOpen   hadoop.hdfs.TestHFlush JDK v1.7.0_95 Failed junit tests hadoop.hdfs.TestHFlush   hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl Subsystem Report/Notes Docker Image:yetus/hadoop:cf2ee45 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12802348/HDFS-10367.patch JIRA Issue HDFS-10367 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux df490bf45ecd 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 1268cf5 Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_91 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/15367/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_91.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/15367/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/15367/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_91.txt https://builds.apache.org/job/PreCommit-HDFS-Build/15367/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/15367/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/15367/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
        Hide
        liuml07 Mingliang Liu added a comment -

        Thanks for your comment Masatake Iwasaki. I was fooled by the ServerSocketUtil#getPort()'s name. Yes we need retrying the same port (9820 in this case). Perhaps we need a new util method for this.

        Show
        liuml07 Mingliang Liu added a comment - Thanks for your comment Masatake Iwasaki . I was fooled by the ServerSocketUtil#getPort() 's name. Yes we need retrying the same port (9820 in this case). Perhaps we need a new util method for this.
        Hide
        iwasakims Masatake Iwasaki added a comment -

        We can not use ServerSocketUtil#getPort which returns a random number when it do retries. The test fails if the port is not 9820. I think quick fix is just retrying 9820 until it becomes available..

        Show
        iwasakims Masatake Iwasaki added a comment - We can not use ServerSocketUtil#getPort which returns a random number when it do retries. The test fails if the port is not 9820. I think quick fix is just retrying 9820 until it becomes available..
        Hide
        liuml07 Mingliang Liu added a comment -

        This is a workaround fix. I see no simple fix for this test as the port number matters. Basically we can't use an ephemeral one.

        I think we can still use the config deafult HdfsClientConfigKeys.DFS_NAMENODE_RPC_PORT_DEFAULT instead of the literal number 9820.

        Show
        liuml07 Mingliang Liu added a comment - This is a workaround fix. I see no simple fix for this test as the port number matters. Basically we can't use an ephemeral one. I think we can still use the config deafult HdfsClientConfigKeys.DFS_NAMENODE_RPC_PORT_DEFAULT instead of the literal number 9820.
        Hide
        brahmareddy Brahma Reddy Battula added a comment -

        Uploading the patch..Kindly Review..

        Show
        brahmareddy Brahma Reddy Battula added a comment - Uploading the patch..Kindly Review..

          People

          • Assignee:
            brahmareddy Brahma Reddy Battula
            Reporter:
            brahmareddy Brahma Reddy Battula
          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development