Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-10261

TestBookKeeperHACheckpoints doesn't handle ephemeral HTTP ports

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 2.7.3, 3.0.0-alpha1
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      The MiniDFSCluster HTTP ports are hard-coded to 10001 and 10002. This makes it impossible to run these tests simultaneously and also allows for failures if those ports are in use.

      1. HDFS-10261.001.patch
        3 kB
        Eric Badger
      2. HDFS-10261.002.patch
        3 kB
        Eric Badger

        Activity

        Hide
        vinodkv Vinod Kumar Vavilapalli added a comment -

        Closing the JIRA as part of 2.7.3 release.

        Show
        vinodkv Vinod Kumar Vavilapalli added a comment - Closing the JIRA as part of 2.7.3 release.
        Hide
        kihwal Kihwal Lee added a comment -

        Committed this to trunk through branch-2.7. branch-2 and up had conflict due to absence of HDFS-6440. Thanks for working on this, Eric Badger.

        Show
        kihwal Kihwal Lee added a comment - Committed this to trunk through branch-2.7. branch-2 and up had conflict due to absence of HDFS-6440 . Thanks for working on this, Eric Badger .
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-trunk-Commit #9564 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9564/)
        HDFS-10261. TestBookKeeperHACheckpoints doesn't handle ephemeral HTTP (kihwal: rev 9ba1e5af06070ba01dcf46e1a4c66713a1d43352)

        • hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java/org/apache/hadoop/contrib/bkjournal/TestBookKeeperHACheckpoints.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #9564 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9564/ ) HDFS-10261 . TestBookKeeperHACheckpoints doesn't handle ephemeral HTTP (kihwal: rev 9ba1e5af06070ba01dcf46e1a4c66713a1d43352) hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java/org/apache/hadoop/contrib/bkjournal/TestBookKeeperHACheckpoints.java
        Hide
        kihwal Kihwal Lee added a comment -

        +1 the latest patch looks good.

        Show
        kihwal Kihwal Lee added a comment - +1 the latest patch looks good.
        Hide
        hadoopqa Hadoop QA added a comment -
        +1 overall



        Vote Subsystem Runtime Comment
        0 reexec 15m 11s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 11m 51s trunk passed
        +1 compile 0m 36s trunk passed with JDK v1.8.0_77
        +1 compile 0m 28s trunk passed with JDK v1.7.0_95
        +1 checkstyle 0m 23s trunk passed
        +1 mvnsite 0m 35s trunk passed
        +1 mvneclipse 0m 23s trunk passed
        +1 findbugs 0m 48s trunk passed
        +1 javadoc 0m 31s trunk passed with JDK v1.8.0_77
        +1 javadoc 0m 27s trunk passed with JDK v1.7.0_95
        +1 mvninstall 0m 27s the patch passed
        +1 compile 0m 30s the patch passed with JDK v1.8.0_77
        +1 javac 0m 30s the patch passed
        +1 compile 0m 23s the patch passed with JDK v1.7.0_95
        +1 javac 0m 23s the patch passed
        +1 checkstyle 0m 15s the patch passed
        +1 mvnsite 0m 27s the patch passed
        +1 mvneclipse 0m 18s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        +1 findbugs 1m 2s the patch passed
        +1 javadoc 0m 29s the patch passed with JDK v1.8.0_77
        +1 javadoc 0m 23s the patch passed with JDK v1.7.0_95
        +1 unit 7m 1s bkjournal in the patch passed with JDK v1.8.0_77.
        +1 unit 5m 45s bkjournal in the patch passed with JDK v1.7.0_95.
        +1 asflicense 0m 30s Patch does not generate ASF License warnings.
        52m 48s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:fbe3e86
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12797158/HDFS-10261.002.patch
        JIRA Issue HDFS-10261
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 9fc951d323c5 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 0cd320a
        Default Java 1.7.0_95
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_77 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
        findbugs v3.0.0
        JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/15074/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal U: hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/15074/console
        Powered by Apache Yetus 0.2.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 15m 11s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 11m 51s trunk passed +1 compile 0m 36s trunk passed with JDK v1.8.0_77 +1 compile 0m 28s trunk passed with JDK v1.7.0_95 +1 checkstyle 0m 23s trunk passed +1 mvnsite 0m 35s trunk passed +1 mvneclipse 0m 23s trunk passed +1 findbugs 0m 48s trunk passed +1 javadoc 0m 31s trunk passed with JDK v1.8.0_77 +1 javadoc 0m 27s trunk passed with JDK v1.7.0_95 +1 mvninstall 0m 27s the patch passed +1 compile 0m 30s the patch passed with JDK v1.8.0_77 +1 javac 0m 30s the patch passed +1 compile 0m 23s the patch passed with JDK v1.7.0_95 +1 javac 0m 23s the patch passed +1 checkstyle 0m 15s the patch passed +1 mvnsite 0m 27s the patch passed +1 mvneclipse 0m 18s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 1m 2s the patch passed +1 javadoc 0m 29s the patch passed with JDK v1.8.0_77 +1 javadoc 0m 23s the patch passed with JDK v1.7.0_95 +1 unit 7m 1s bkjournal in the patch passed with JDK v1.8.0_77. +1 unit 5m 45s bkjournal in the patch passed with JDK v1.7.0_95. +1 asflicense 0m 30s Patch does not generate ASF License warnings. 52m 48s Subsystem Report/Notes Docker Image:yetus/hadoop:fbe3e86 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12797158/HDFS-10261.002.patch JIRA Issue HDFS-10261 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 9fc951d323c5 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 0cd320a Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_77 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/15074/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal U: hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal Console output https://builds.apache.org/job/PreCommit-HDFS-Build/15074/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
        Hide
        ebadger Eric Badger added a comment -

        Well that's embarrassing. Good catch, Kihwal Lee. Submitting new patch with the ports actually updated.

        Show
        ebadger Eric Badger added a comment - Well that's embarrassing. Good catch, Kihwal Lee . Submitting new patch with the ports actually updated.
        Hide
        kihwal Kihwal Lee added a comment - - edited

        Did you mean to do this?

                    .addNN(new MiniDFSNNTopology.NNConf("nn1").setHttpPort(basePort))
                    .addNN(new MiniDFSNNTopology.NNConf("nn2").setHttpPort(basePort+1)));
        
        Show
        kihwal Kihwal Lee added a comment - - edited Did you mean to do this? .addNN( new MiniDFSNNTopology.NNConf( "nn1" ).setHttpPort(basePort)) .addNN( new MiniDFSNNTopology.NNConf( "nn2" ).setHttpPort(basePort+1)));
        Hide
        hadoopqa Hadoop QA added a comment -
        +1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 8s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 6m 32s trunk passed
        +1 compile 0m 13s trunk passed with JDK v1.8.0_77
        +1 compile 0m 15s trunk passed with JDK v1.7.0_95
        +1 checkstyle 0m 13s trunk passed
        +1 mvnsite 0m 19s trunk passed
        +1 mvneclipse 0m 13s trunk passed
        +1 findbugs 0m 27s trunk passed
        +1 javadoc 0m 13s trunk passed with JDK v1.8.0_77
        +1 javadoc 0m 15s trunk passed with JDK v1.7.0_95
        +1 mvninstall 0m 15s the patch passed
        +1 compile 0m 11s the patch passed with JDK v1.8.0_77
        +1 javac 0m 11s the patch passed
        +1 compile 0m 13s the patch passed with JDK v1.7.0_95
        +1 javac 0m 13s the patch passed
        +1 checkstyle 0m 10s the patch passed
        +1 mvnsite 0m 17s the patch passed
        +1 mvneclipse 0m 10s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        +1 findbugs 0m 36s the patch passed
        +1 javadoc 0m 10s the patch passed with JDK v1.8.0_77
        +1 javadoc 0m 12s the patch passed with JDK v1.7.0_95
        +1 unit 4m 17s bkjournal in the patch passed with JDK v1.8.0_77.
        +1 unit 4m 21s bkjournal in the patch passed with JDK v1.7.0_95.
        +1 asflicense 0m 19s Patch does not generate ASF License warnings.
        21m 53s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:fbe3e86
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12797120/HDFS-10261.001.patch
        JIRA Issue HDFS-10261
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 4258da663bd1 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 0005816
        Default Java 1.7.0_95
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_77 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
        findbugs v3.0.0
        JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/15069/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal U: hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/15069/console
        Powered by Apache Yetus 0.2.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 8s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 6m 32s trunk passed +1 compile 0m 13s trunk passed with JDK v1.8.0_77 +1 compile 0m 15s trunk passed with JDK v1.7.0_95 +1 checkstyle 0m 13s trunk passed +1 mvnsite 0m 19s trunk passed +1 mvneclipse 0m 13s trunk passed +1 findbugs 0m 27s trunk passed +1 javadoc 0m 13s trunk passed with JDK v1.8.0_77 +1 javadoc 0m 15s trunk passed with JDK v1.7.0_95 +1 mvninstall 0m 15s the patch passed +1 compile 0m 11s the patch passed with JDK v1.8.0_77 +1 javac 0m 11s the patch passed +1 compile 0m 13s the patch passed with JDK v1.7.0_95 +1 javac 0m 13s the patch passed +1 checkstyle 0m 10s the patch passed +1 mvnsite 0m 17s the patch passed +1 mvneclipse 0m 10s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 0m 36s the patch passed +1 javadoc 0m 10s the patch passed with JDK v1.8.0_77 +1 javadoc 0m 12s the patch passed with JDK v1.7.0_95 +1 unit 4m 17s bkjournal in the patch passed with JDK v1.8.0_77. +1 unit 4m 21s bkjournal in the patch passed with JDK v1.7.0_95. +1 asflicense 0m 19s Patch does not generate ASF License warnings. 21m 53s Subsystem Report/Notes Docker Image:yetus/hadoop:fbe3e86 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12797120/HDFS-10261.001.patch JIRA Issue HDFS-10261 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 4258da663bd1 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 0005816 Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_77 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/15069/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal U: hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal Console output https://builds.apache.org/job/PreCommit-HDFS-Build/15069/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
        Hide
        ebadger Eric Badger added a comment -

        Daryn Sharp Please review.

        Show
        ebadger Eric Badger added a comment - Daryn Sharp Please review.
        Hide
        ebadger Eric Badger added a comment -

        Using the port randomization policy that is used in TestStandbyCheckpoints (which TestBookKeeperHACheckpoints extends).

        Show
        ebadger Eric Badger added a comment - Using the port randomization policy that is used in TestStandbyCheckpoints (which TestBookKeeperHACheckpoints extends).

          People

          • Assignee:
            ebadger Eric Badger
            Reporter:
            ebadger Eric Badger
          • Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development