Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-3568

TestAMRMTokens should use some random port

    Details

    • Type: Test
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.8.0
    • Fix Version/s: 2.8.0, 3.0.0-alpha2
    • Component/s: test
    • Labels:
    • Hadoop Flags:
      Reviewed

      Description

      Since the default port is used for yarn.resourcemanager.scheduler.address, if we already run a pseudo-distributed cluster on the same development machine, the test fails like this:

      testMasterKeyRollOver[0](org.apache.hadoop.yarn.server.resourcemanager.security.TestAMRMTokens)  Time elapsed: 1.511 sec  <<< ERROR!
      org.apache.hadoop.yarn.exceptions.YarnRuntimeException: java.net.BindException: Problem binding to [0.0.0.0:8030] java.net.BindException: Address already in use; For more details see:  http://wiki.apache.org/hadoop/BindException
      	at sun.nio.ch.Net.bind0(Native Method)
      	at sun.nio.ch.Net.bind(Net.java:444)
      	at sun.nio.ch.Net.bind(Net.java:436)
      	at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:214)
      	at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
      	at org.apache.hadoop.ipc.Server.bind(Server.java:413)
      	at org.apache.hadoop.ipc.Server$Listener.<init>(Server.java:590)
      	at org.apache.hadoop.ipc.Server.<init>(Server.java:2340)
      	at org.apache.hadoop.ipc.RPC$Server.<init>(RPC.java:945)
      	at org.apache.hadoop.ipc.ProtobufRpcEngine$Server.<init>(ProtobufRpcEngine.java:534)
      	at org.apache.hadoop.ipc.ProtobufRpcEngine.getServer(ProtobufRpcEngine.java:509)
      	at org.apache.hadoop.ipc.RPC$Builder.build(RPC.java:787)
      	at org.apache.hadoop.yarn.factories.impl.pb.RpcServerFactoryPBImpl.createServer(RpcServerFactoryPBImpl.java:169)
      	at org.apache.hadoop.yarn.factories.impl.pb.RpcServerFactoryPBImpl.getServer(RpcServerFactoryPBImpl.java:132)
      	at org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC.getServer(HadoopYarnProtoRPC.java:65)
      	at org.apache.hadoop.yarn.ipc.YarnRPC.getServer(YarnRPC.java:54)
      	at org.apache.hadoop.yarn.server.resourcemanager.ApplicationMasterService.serviceStart(ApplicationMasterService.java:140)
      	at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
      	at org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:120)
      	at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMActiveServices.serviceStart(ResourceManager.java:586)
      	at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
      	at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.startActiveServices(ResourceManager.java:996)
      	at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$1.run(ResourceManager.java:1037)
      	at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$1.run(ResourceManager.java:1033)
      	at java.security.AccessController.doPrivileged(Native Method)
      	at javax.security.auth.Subject.doAs(Subject.java:415)
      	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1669)
      	at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.transitionToActive(ResourceManager.java:1033)
      	at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceStart(ResourceManager.java:1073)
      	at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
      	at org.apache.hadoop.yarn.server.resourcemanager.security.TestAMRMTokens.testMasterKeyRollOver(TestAMRMTokens.java:235)
      
      1. YARN-3568.1.patch
        2 kB
        Takashi Ohnishi

        Activity

        Hide
        bwtakacy Takashi Ohnishi added a comment -

        Hi.

        I have created a patch for this.

        Show
        bwtakacy Takashi Ohnishi added a comment - Hi. I have created a patch for this.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 15s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 6m 52s trunk passed
        +1 compile 0m 27s trunk passed with JDK v1.8.0_77
        +1 compile 0m 30s trunk passed with JDK v1.7.0_95
        +1 checkstyle 0m 18s trunk passed
        +1 mvnsite 0m 37s trunk passed
        +1 mvneclipse 0m 15s trunk passed
        +1 findbugs 1m 8s trunk passed
        +1 javadoc 0m 21s trunk passed with JDK v1.8.0_77
        +1 javadoc 0m 27s trunk passed with JDK v1.7.0_95
        +1 mvninstall 0m 30s the patch passed
        +1 compile 0m 25s the patch passed with JDK v1.8.0_77
        +1 javac 0m 25s the patch passed
        +1 compile 0m 27s the patch passed with JDK v1.7.0_95
        +1 javac 0m 27s the patch passed
        +1 checkstyle 0m 15s the patch passed
        +1 mvnsite 0m 34s the patch passed
        +1 mvneclipse 0m 14s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        +1 findbugs 1m 16s the patch passed
        +1 javadoc 0m 19s the patch passed with JDK v1.8.0_77
        +1 javadoc 0m 25s the patch passed with JDK v1.7.0_95
        -1 unit 81m 6s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_77.
        -1 unit 82m 48s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_95.
        +1 asflicense 0m 18s Patch does not generate ASF License warnings.
        180m 50s



        Reason Tests
        JDK v1.8.0_77 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens
          hadoop.yarn.server.resourcemanager.TestAMAuthorization
        JDK v1.8.0_77 Timed out junit tests org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesNodes
        JDK v1.7.0_95 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens
          hadoop.yarn.server.resourcemanager.TestAMAuthorization
        JDK v1.7.0_95 Timed out junit tests org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesNodes



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:fbe3e86
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12800517/YARN-3568.1.patch
        JIRA Issue YARN-3568
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 370f0617319c 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / bec5b4c
        Default Java 1.7.0_95
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_77 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
        findbugs v3.0.0
        unit https://builds.apache.org/job/PreCommit-YARN-Build/11204/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_77.txt
        unit https://builds.apache.org/job/PreCommit-YARN-Build/11204/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt
        unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/11204/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_77.txt https://builds.apache.org/job/PreCommit-YARN-Build/11204/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt
        JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/11204/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/11204/console
        Powered by Apache Yetus 0.2.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 15s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 6m 52s trunk passed +1 compile 0m 27s trunk passed with JDK v1.8.0_77 +1 compile 0m 30s trunk passed with JDK v1.7.0_95 +1 checkstyle 0m 18s trunk passed +1 mvnsite 0m 37s trunk passed +1 mvneclipse 0m 15s trunk passed +1 findbugs 1m 8s trunk passed +1 javadoc 0m 21s trunk passed with JDK v1.8.0_77 +1 javadoc 0m 27s trunk passed with JDK v1.7.0_95 +1 mvninstall 0m 30s the patch passed +1 compile 0m 25s the patch passed with JDK v1.8.0_77 +1 javac 0m 25s the patch passed +1 compile 0m 27s the patch passed with JDK v1.7.0_95 +1 javac 0m 27s the patch passed +1 checkstyle 0m 15s the patch passed +1 mvnsite 0m 34s the patch passed +1 mvneclipse 0m 14s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 1m 16s the patch passed +1 javadoc 0m 19s the patch passed with JDK v1.8.0_77 +1 javadoc 0m 25s the patch passed with JDK v1.7.0_95 -1 unit 81m 6s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_77. -1 unit 82m 48s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_95. +1 asflicense 0m 18s Patch does not generate ASF License warnings. 180m 50s Reason Tests JDK v1.8.0_77 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens   hadoop.yarn.server.resourcemanager.TestAMAuthorization JDK v1.8.0_77 Timed out junit tests org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesNodes JDK v1.7.0_95 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens   hadoop.yarn.server.resourcemanager.TestAMAuthorization JDK v1.7.0_95 Timed out junit tests org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesNodes Subsystem Report/Notes Docker Image:yetus/hadoop:fbe3e86 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12800517/YARN-3568.1.patch JIRA Issue YARN-3568 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 370f0617319c 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / bec5b4c Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_77 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-YARN-Build/11204/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_77.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/11204/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/11204/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_77.txt https://builds.apache.org/job/PreCommit-YARN-Build/11204/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/11204/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager Console output https://builds.apache.org/job/PreCommit-YARN-Build/11204/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
        Hide
        subru Subru Krishnan added a comment -

        Thanks Takashi Ohnishi for the patch, I committed this to trunk, branch-2 and branch-2.8.

        Show
        subru Subru Krishnan added a comment - Thanks Takashi Ohnishi for the patch, I committed this to trunk, branch-2 and branch-2.8.
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10711 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10711/)
        YARN-3568. TestAMRMTokens should use some random port. (Takashi Ohnishi (subu: rev 79ae78dcbec183ab53b26de408b4517e5a151878)

        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/security/TestAMRMTokens.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10711 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10711/ ) YARN-3568 . TestAMRMTokens should use some random port. (Takashi Ohnishi (subu: rev 79ae78dcbec183ab53b26de408b4517e5a151878) (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/security/TestAMRMTokens.java
        Hide
        bwtakacy Takashi Ohnishi added a comment -

        Thanks Subru Krishnan for committing !

        Show
        bwtakacy Takashi Ohnishi added a comment - Thanks Subru Krishnan for committing !

          People

          • Assignee:
            bwtakacy Takashi Ohnishi
            Reporter:
            jira.shegalov Gera Shegalov
          • Votes:
            0 Vote for this issue
            Watchers:
            9 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development