Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-12655

TestHttpServer.testBindAddress bind port range is wider than expected

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.9.0, 3.0.0-alpha1
    • Fix Version/s: 2.9.0, 3.0.0-alpha1
    • Component/s: None
    • Labels:
    • Environment:

      Jenkins

      Description

      Saw it in a pre-commit jenkins job https://builds.apache.org/job/PreCommit-HADOOP-Build/8242/testReport/org.apache.hadoop.http/TestHttpServer/testBindAddress/

      It also appeared previously in Hadoop-common-trunk-Java8 jenkins on Oct 21.

      In the following case, the first server bound to port 53212, and the second one bound to port 53225, which violated the assertion in the test case (the second port is supposed to be no more than the first + 8)

      Stacktrace
      
      java.lang.AssertionError: null
      	at org.junit.Assert.fail(Assert.java:86)
      	at org.junit.Assert.assertTrue(Assert.java:41)
      	at org.junit.Assert.assertTrue(Assert.java:52)
      	at org.apache.hadoop.http.TestHttpServer.checkBindAddress(TestHttpServer.java:539)
      	at org.apache.hadoop.http.TestHttpServer.testBindAddress(TestHttpServer.java:503)
      
      
      Standard Output
      
      2015-12-15 05:26:44,123 INFO  server.AuthenticationFilter (AuthenticationFilter.java:constructSecretProvider(304)) - Unable to initialize FileSignerSecretProvider, falling back to use random secrets.
      2015-12-15 05:26:44,125 INFO  http.HttpRequestLog (HttpRequestLog.java:getRequestLog(80)) - Http request log for http.requests.test is not defined
      2015-12-15 05:26:44,126 INFO  http.HttpServer2 (HttpServer2.java:addGlobalFilter(701)) - Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter)
      2015-12-15 05:26:44,127 INFO  http.HttpServer2 (HttpServer2.java:addFilter(676)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context test
      2015-12-15 05:26:44,128 INFO  http.HttpServer2 (HttpServer2.java:addFilter(684)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context static
      2015-12-15 05:26:44,128 INFO  http.HttpServer2 (HttpServer2.java:addFilter(684)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs
      2015-12-15 05:26:44,131 INFO  http.HttpServer2 (HttpServer2.java:openListeners(906)) - Jetty bound to port 41406
      2015-12-15 05:26:44,155 INFO  server.AuthenticationFilter (AuthenticationFilter.java:constructSecretProvider(304)) - Unable to initialize FileSignerSecretProvider, falling back to use random secrets.
      2015-12-15 05:26:44,157 INFO  http.HttpRequestLog (HttpRequestLog.java:getRequestLog(80)) - Http request log for http.requests.test is not defined
      2015-12-15 05:26:44,158 INFO  http.HttpServer2 (HttpServer2.java:addGlobalFilter(701)) - Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter)
      2015-12-15 05:26:44,159 INFO  http.HttpServer2 (HttpServer2.java:addFilter(676)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context test
      2015-12-15 05:26:44,159 INFO  http.HttpServer2 (HttpServer2.java:addFilter(684)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context static
      2015-12-15 05:26:44,160 INFO  http.HttpServer2 (HttpServer2.java:addFilter(684)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs
      2015-12-15 05:26:44,161 INFO  http.HttpServer2 (HttpServer2.java:openListeners(906)) - Jetty bound to port 53212
      2015-12-15 05:26:44,188 INFO  server.AuthenticationFilter (AuthenticationFilter.java:constructSecretProvider(304)) - Unable to initialize FileSignerSecretProvider, falling back to use random secrets.
      2015-12-15 05:26:44,189 INFO  http.HttpRequestLog (HttpRequestLog.java:getRequestLog(80)) - Http request log for http.requests.test is not defined
      2015-12-15 05:26:44,190 INFO  http.HttpServer2 (HttpServer2.java:addGlobalFilter(701)) - Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter)
      2015-12-15 05:26:44,191 INFO  http.HttpServer2 (HttpServer2.java:addFilter(676)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context test
      2015-12-15 05:26:44,191 INFO  http.HttpServer2 (HttpServer2.java:addFilter(684)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context static
      2015-12-15 05:26:44,192 INFO  http.HttpServer2 (HttpServer2.java:addFilter(684)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs
      2015-12-15 05:26:45,500 INFO  http.HttpServer2 (HttpServer2.java:openListeners(906)) - Jetty bound to port 53225
      
      
      1. HADOOP-12655.001.patch
        0.9 kB
        Wei-Chiu Chuang

        Activity

        Hide
        jojochuang Wei-Chiu Chuang added a comment -

        Rev01: The assumption that boundPort must be no more than/equal to 8 + port is not correct. So remove the assertion.

        Show
        jojochuang Wei-Chiu Chuang added a comment - Rev01: The assumption that boundPort must be no more than/equal to 8 + port is not correct. So remove the assertion.
        Hide
        hadoopqa Hadoop QA added a comment -
        +1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 0s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 7m 56s trunk passed
        +1 compile 9m 35s trunk passed with JDK v1.8.0_66
        +1 compile 10m 12s trunk passed with JDK v1.7.0_91
        +1 checkstyle 0m 18s trunk passed
        +1 mvnsite 1m 12s trunk passed
        +1 mvneclipse 0m 15s trunk passed
        +1 findbugs 2m 5s trunk passed
        +1 javadoc 0m 55s trunk passed with JDK v1.8.0_66
        +1 javadoc 1m 9s trunk passed with JDK v1.7.0_91
        +1 mvninstall 1m 41s the patch passed
        +1 compile 8m 51s the patch passed with JDK v1.8.0_66
        +1 javac 8m 51s the patch passed
        +1 compile 9m 35s the patch passed with JDK v1.7.0_91
        +1 javac 9m 35s the patch passed
        +1 checkstyle 0m 18s the patch passed
        +1 mvnsite 1m 8s the patch passed
        +1 mvneclipse 0m 15s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        +1 findbugs 2m 6s the patch passed
        +1 javadoc 1m 3s the patch passed with JDK v1.8.0_66
        +1 javadoc 1m 11s the patch passed with JDK v1.7.0_91
        +1 unit 7m 42s hadoop-common in the patch passed with JDK v1.8.0_66.
        +1 unit 7m 44s hadoop-common in the patch passed with JDK v1.7.0_91.
        +1 asflicense 0m 23s Patch does not generate ASF License warnings.
        76m 43s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:0ca8df7
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12778269/HADOOP-12655.001.patch
        JIRA Issue HADOOP-12655
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 14e0dda788df 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / d90625e
        findbugs v3.0.0
        JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8264/testReport/
        modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
        Max memory used 76MB
        Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org
        Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8264/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 7m 56s trunk passed +1 compile 9m 35s trunk passed with JDK v1.8.0_66 +1 compile 10m 12s trunk passed with JDK v1.7.0_91 +1 checkstyle 0m 18s trunk passed +1 mvnsite 1m 12s trunk passed +1 mvneclipse 0m 15s trunk passed +1 findbugs 2m 5s trunk passed +1 javadoc 0m 55s trunk passed with JDK v1.8.0_66 +1 javadoc 1m 9s trunk passed with JDK v1.7.0_91 +1 mvninstall 1m 41s the patch passed +1 compile 8m 51s the patch passed with JDK v1.8.0_66 +1 javac 8m 51s the patch passed +1 compile 9m 35s the patch passed with JDK v1.7.0_91 +1 javac 9m 35s the patch passed +1 checkstyle 0m 18s the patch passed +1 mvnsite 1m 8s the patch passed +1 mvneclipse 0m 15s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 2m 6s the patch passed +1 javadoc 1m 3s the patch passed with JDK v1.8.0_66 +1 javadoc 1m 11s the patch passed with JDK v1.7.0_91 +1 unit 7m 42s hadoop-common in the patch passed with JDK v1.8.0_66. +1 unit 7m 44s hadoop-common in the patch passed with JDK v1.7.0_91. +1 asflicense 0m 23s Patch does not generate ASF License warnings. 76m 43s Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12778269/HADOOP-12655.001.patch JIRA Issue HADOOP-12655 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 14e0dda788df 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / d90625e findbugs v3.0.0 JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8264/testReport/ modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common Max memory used 76MB Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8264/console This message was automatically generated.
        Hide
        stevel@apache.org Steve Loughran added a comment -

        +1, patch applied —thanks!

        Show
        stevel@apache.org Steve Loughran added a comment - +1, patch applied —thanks!
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-trunk-Commit #9068 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9068/)
        HADOOP-12655. TestHttpServer.testBindAddress bind port range is wider (stevel: rev 3190534814587a0e6cb7d99b01cca4480425458b)

        • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/http/TestHttpServer.java
        • hadoop-common-project/hadoop-common/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #9068 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9068/ ) HADOOP-12655 . TestHttpServer.testBindAddress bind port range is wider (stevel: rev 3190534814587a0e6cb7d99b01cca4480425458b) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/http/TestHttpServer.java hadoop-common-project/hadoop-common/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-trunk-Commit #9069 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9069/)
        HADOOP-12655. TestHttpServer.testBindAddress patch - move from (stevel: rev 7914f569ef4cd8a2b2d0f625beec4b83eb6d1ab1)

        • hadoop-common-project/hadoop-common/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #9069 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9069/ ) HADOOP-12655 . TestHttpServer.testBindAddress patch - move from (stevel: rev 7914f569ef4cd8a2b2d0f625beec4b83eb6d1ab1) hadoop-common-project/hadoop-common/CHANGES.txt
        Hide
        jojochuang Wei-Chiu Chuang added a comment -

        Thanks Steve Loughran for the commit!

        Show
        jojochuang Wei-Chiu Chuang added a comment - Thanks Steve Loughran for the commit!

          People

          • Assignee:
            jojochuang Wei-Chiu Chuang
            Reporter:
            jojochuang Wei-Chiu Chuang
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development