Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-13183

S3A proxy tests fail after httpclient/httpcore upgrade.

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: fs/s3
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      HADOOP-12767 upgraded the httpclient and httpcore dependency versions. After that, I started seeing failures in S3A tests related to proxy handling. If I revert that patch locally, the tests pass again.

        Issue Links

          Activity

          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-trunk-Commit #9830 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9830/)
          HADOOP-13183. S3A proxy tests fail after httpclient/httpcore upgrade. (cnauroth: rev 93258459faf56bc84121ba99d20eaef95273329e)

          • hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/TestS3AConfiguration.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-trunk-Commit #9830 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9830/ ) HADOOP-13183 . S3A proxy tests fail after httpclient/httpcore upgrade. (cnauroth: rev 93258459faf56bc84121ba99d20eaef95273329e) hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/TestS3AConfiguration.java
          Hide
          cnauroth Chris Nauroth added a comment -

          +1 for the patch. I had a successful full test run against S3 buckets in US-west-2. The license warning was unrelated. I committed this to trunk, branch-2 and branch-2.8. Steve Loughran, thank you for the patch.

          Show
          cnauroth Chris Nauroth added a comment - +1 for the patch. I had a successful full test run against S3 buckets in US-west-2. The license warning was unrelated. I committed this to trunk, branch-2 and branch-2.8. Steve Loughran , thank you for the patch.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 11s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 6m 25s branch-2 passed
          +1 compile 0m 12s branch-2 passed with JDK v1.8.0_91
          +1 compile 0m 13s branch-2 passed with JDK v1.7.0_101
          +1 checkstyle 0m 13s branch-2 passed
          +1 mvnsite 0m 18s branch-2 passed
          +1 mvneclipse 0m 15s branch-2 passed
          +1 findbugs 0m 29s branch-2 passed
          +1 javadoc 0m 12s branch-2 passed with JDK v1.8.0_91
          +1 javadoc 0m 14s branch-2 passed with JDK v1.7.0_101
          +1 mvninstall 0m 13s the patch passed
          +1 compile 0m 9s the patch passed with JDK v1.8.0_91
          +1 javac 0m 9s the patch passed
          +1 compile 0m 11s the patch passed with JDK v1.7.0_101
          +1 javac 0m 11s the patch passed
          +1 checkstyle 0m 11s the patch passed
          +1 mvnsite 0m 15s the patch passed
          +1 mvneclipse 0m 10s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 0m 39s the patch passed
          +1 javadoc 0m 9s the patch passed with JDK v1.8.0_91
          +1 javadoc 0m 12s the patch passed with JDK v1.7.0_101
          +1 unit 0m 10s hadoop-aws in the patch passed with JDK v1.8.0_91.
          +1 unit 0m 10s hadoop-aws in the patch passed with JDK v1.7.0_101.
          -1 asflicense 0m 17s The patch generated 1 ASF License warnings.
          12m 27s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:babe025
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12804996/HADOOP-13183-branch-2-001.patch
          JIRA Issue HADOOP-13183
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux ec693a0f0ccf 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision branch-2 / f7ffa1a
          Default Java 1.7.0_101
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_91 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_101
          findbugs v3.0.0
          JDK v1.7.0_101 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9523/testReport/
          asflicense https://builds.apache.org/job/PreCommit-HADOOP-Build/9523/artifact/patchprocess/patch-asflicense-problems.txt
          modules C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9523/console
          Powered by Apache Yetus 0.3.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 11s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 6m 25s branch-2 passed +1 compile 0m 12s branch-2 passed with JDK v1.8.0_91 +1 compile 0m 13s branch-2 passed with JDK v1.7.0_101 +1 checkstyle 0m 13s branch-2 passed +1 mvnsite 0m 18s branch-2 passed +1 mvneclipse 0m 15s branch-2 passed +1 findbugs 0m 29s branch-2 passed +1 javadoc 0m 12s branch-2 passed with JDK v1.8.0_91 +1 javadoc 0m 14s branch-2 passed with JDK v1.7.0_101 +1 mvninstall 0m 13s the patch passed +1 compile 0m 9s the patch passed with JDK v1.8.0_91 +1 javac 0m 9s the patch passed +1 compile 0m 11s the patch passed with JDK v1.7.0_101 +1 javac 0m 11s the patch passed +1 checkstyle 0m 11s the patch passed +1 mvnsite 0m 15s the patch passed +1 mvneclipse 0m 10s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 0m 39s the patch passed +1 javadoc 0m 9s the patch passed with JDK v1.8.0_91 +1 javadoc 0m 12s the patch passed with JDK v1.7.0_101 +1 unit 0m 10s hadoop-aws in the patch passed with JDK v1.8.0_91. +1 unit 0m 10s hadoop-aws in the patch passed with JDK v1.7.0_101. -1 asflicense 0m 17s The patch generated 1 ASF License warnings. 12m 27s Subsystem Report/Notes Docker Image:yetus/hadoop:babe025 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12804996/HADOOP-13183-branch-2-001.patch JIRA Issue HADOOP-13183 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux ec693a0f0ccf 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision branch-2 / f7ffa1a Default Java 1.7.0_101 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_91 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_101 findbugs v3.0.0 JDK v1.7.0_101 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9523/testReport/ asflicense https://builds.apache.org/job/PreCommit-HADOOP-Build/9523/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9523/console Powered by Apache Yetus 0.3.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          stevel@apache.org Steve Loughran added a comment -

          here: removed text checks. They were looking for the hostname:port to be in the exception strings —they aren't any more.

          Show
          stevel@apache.org Steve Loughran added a comment - here: removed text checks. They were looking for the hostname:port to be in the exception strings —they aren't any more.
          Hide
          stevel@apache.org Steve Loughran added a comment -

          Chris: I can replicate this on a linux Java 8 JVM; I'm not seeing it in the IOE exception branch, so think maybe I've somehow addressed it there...and perhaps why I didn't catch this earlier.

          What's probably happening is httpclient is returning a different error string; the two proxy tests are looking for specific text/numbers in them. Fix? I think we should just downgrade to catching the exception and not look at the text inside.

          Show
          stevel@apache.org Steve Loughran added a comment - Chris: I can replicate this on a linux Java 8 JVM; I'm not seeing it in the IOE exception branch, so think maybe I've somehow addressed it there...and perhaps why I didn't catch this earlier. What's probably happening is httpclient is returning a different error string; the two proxy tests are looking for specific text/numbers in them. Fix? I think we should just downgrade to catching the exception and not look at the text inside.
          Hide
          cnauroth Chris Nauroth added a comment -

          I see the problem on both trunk (Java 8) and branch-2.8 (Java 7).

          Show
          cnauroth Chris Nauroth added a comment - I see the problem on both trunk (Java 8) and branch-2.8 (Java 7).
          Hide
          cnauroth Chris Nauroth added a comment -

          If I git checkout 22ff9e68d1d9f27d62455c15fb1242538551aca9 (which is the HADOOP-12767 commit), then I see failures in the TestS3AConfiguration proxy tests. If I go one commit back by running git checkout 22ff9e68d1d9f27d62455c15fb1242538551aca9^, then the tests pass again. See below for stack traces.

          Running org.apache.hadoop.fs.s3a.TestS3AConfiguration
          Tests run: 14, Failures: 0, Errors: 2, Skipped: 0, Time elapsed: 4.814 sec <<< FAILURE! - in org.apache.hadoop.fs.s3a.TestS3AConfiguration
          testProxyConnection(org.apache.hadoop.fs.s3a.TestS3AConfiguration)  Time elapsed: 2.007 sec  <<< ERROR!
          com.amazonaws.AmazonClientException: Unable to execute HTTP request: Connection refused
          	at java.net.PlainSocketImpl.socketConnect(Native Method)
          	at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
          	at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
          	at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
          	at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
          	at java.net.Socket.connect(Socket.java:589)
          	at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:120)
          	at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:179)
          	at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:328)
          	at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:612)
          	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:447)
          	at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:884)
          	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
          	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
          	at com.amazonaws.http.AmazonHttpClient.executeOneRequest(AmazonHttpClient.java:728)
          	at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:489)
          	at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:310)
          	at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:3785)
          	at com.amazonaws.services.s3.AmazonS3Client.headBucket(AmazonS3Client.java:1107)
          	at com.amazonaws.services.s3.AmazonS3Client.doesBucketExist(AmazonS3Client.java:1070)
          	at org.apache.hadoop.fs.s3a.S3AFileSystem.initialize(S3AFileSystem.java:208)
          	at org.apache.hadoop.fs.s3a.S3ATestUtils.createTestFileSystem(S3ATestUtils.java:52)
          	at org.apache.hadoop.fs.s3a.TestS3AConfiguration.testProxyConnection(TestS3AConfiguration.java:123)
          
          testAutomaticProxyPortSelection(org.apache.hadoop.fs.s3a.TestS3AConfiguration)  Time elapsed: 0.936 sec  <<< ERROR!
          com.amazonaws.AmazonClientException: Unable to execute HTTP request: Connection refused
          	at java.net.PlainSocketImpl.socketConnect(Native Method)
          	at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
          	at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
          	at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
          	at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
          	at java.net.Socket.connect(Socket.java:589)
          	at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:120)
          	at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:179)
          	at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:328)
          	at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:612)
          	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:447)
          	at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:884)
          	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
          	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
          	at com.amazonaws.http.AmazonHttpClient.executeOneRequest(AmazonHttpClient.java:728)
          	at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:489)
          	at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:310)
          	at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:3785)
          	at com.amazonaws.services.s3.AmazonS3Client.headBucket(AmazonS3Client.java:1107)
          	at com.amazonaws.services.s3.AmazonS3Client.doesBucketExist(AmazonS3Client.java:1070)
          	at org.apache.hadoop.fs.s3a.S3AFileSystem.initialize(S3AFileSystem.java:208)
          	at org.apache.hadoop.fs.s3a.S3ATestUtils.createTestFileSystem(S3ATestUtils.java:52)
          	at org.apache.hadoop.fs.s3a.TestS3AConfiguration.testAutomaticProxyPortSelection(TestS3AConfiguration.java:158)
          
          Show
          cnauroth Chris Nauroth added a comment - If I git checkout 22ff9e68d1d9f27d62455c15fb1242538551aca9 (which is the HADOOP-12767 commit), then I see failures in the TestS3AConfiguration proxy tests. If I go one commit back by running git checkout 22ff9e68d1d9f27d62455c15fb1242538551aca9^ , then the tests pass again. See below for stack traces. Running org.apache.hadoop.fs.s3a.TestS3AConfiguration Tests run: 14, Failures: 0, Errors: 2, Skipped: 0, Time elapsed: 4.814 sec <<< FAILURE! - in org.apache.hadoop.fs.s3a.TestS3AConfiguration testProxyConnection(org.apache.hadoop.fs.s3a.TestS3AConfiguration) Time elapsed: 2.007 sec <<< ERROR! com.amazonaws.AmazonClientException: Unable to execute HTTP request: Connection refused at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:589) at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:120) at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:179) at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:328) at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:612) at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:447) at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:884) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55) at com.amazonaws.http.AmazonHttpClient.executeOneRequest(AmazonHttpClient.java:728) at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:489) at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:310) at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:3785) at com.amazonaws.services.s3.AmazonS3Client.headBucket(AmazonS3Client.java:1107) at com.amazonaws.services.s3.AmazonS3Client.doesBucketExist(AmazonS3Client.java:1070) at org.apache.hadoop.fs.s3a.S3AFileSystem.initialize(S3AFileSystem.java:208) at org.apache.hadoop.fs.s3a.S3ATestUtils.createTestFileSystem(S3ATestUtils.java:52) at org.apache.hadoop.fs.s3a.TestS3AConfiguration.testProxyConnection(TestS3AConfiguration.java:123) testAutomaticProxyPortSelection(org.apache.hadoop.fs.s3a.TestS3AConfiguration) Time elapsed: 0.936 sec <<< ERROR! com.amazonaws.AmazonClientException: Unable to execute HTTP request: Connection refused at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:589) at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:120) at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:179) at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:328) at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:612) at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:447) at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:884) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55) at com.amazonaws.http.AmazonHttpClient.executeOneRequest(AmazonHttpClient.java:728) at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:489) at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:310) at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:3785) at com.amazonaws.services.s3.AmazonS3Client.headBucket(AmazonS3Client.java:1107) at com.amazonaws.services.s3.AmazonS3Client.doesBucketExist(AmazonS3Client.java:1070) at org.apache.hadoop.fs.s3a.S3AFileSystem.initialize(S3AFileSystem.java:208) at org.apache.hadoop.fs.s3a.S3ATestUtils.createTestFileSystem(S3ATestUtils.java:52) at org.apache.hadoop.fs.s3a.TestS3AConfiguration.testAutomaticProxyPortSelection(TestS3AConfiguration.java:158)

            People

            • Assignee:
              stevel@apache.org Steve Loughran
              Reporter:
              cnauroth Chris Nauroth
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development