Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-3638

Commons HTTP client dependency conflict in extras/kinesis-asl module

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

      Description

      Followed instructions as mentioned @ https://github.com/apache/spark/blob/master/docs/streaming-kinesis-integration.md and when running the example, I get the following error:

      Caused by: java.lang.NoSuchMethodError: org.apache.http.impl.conn.DefaultClientConnectionOperator.<init>(Lorg/apache/http/conn/scheme/SchemeRegistry;Lorg/apache/http/conn/DnsResolver;)V
              at org.apache.http.impl.conn.PoolingClientConnectionManager.createConnectionOperator(PoolingClientConnectionManager.java:140)
              at org.apache.http.impl.conn.PoolingClientConnectionManager.<init>(PoolingClientConnectionManager.java:114)
              at org.apache.http.impl.conn.PoolingClientConnectionManager.<init>(PoolingClientConnectionManager.java:99)
              at com.amazonaws.http.ConnectionManagerFactory.createPoolingClientConnManager(ConnectionManagerFactory.java:29)
              at com.amazonaws.http.HttpClientFactory.createHttpClient(HttpClientFactory.java:97)
              at com.amazonaws.http.AmazonHttpClient.<init>(AmazonHttpClient.java:181)
              at com.amazonaws.AmazonWebServiceClient.<init>(AmazonWebServiceClient.java:119)
              at com.amazonaws.AmazonWebServiceClient.<init>(AmazonWebServiceClient.java:103)
              at com.amazonaws.services.kinesis.AmazonKinesisClient.<init>(AmazonKinesisClient.java:136)
              at com.amazonaws.services.kinesis.AmazonKinesisClient.<init>(AmazonKinesisClient.java:117)
              at com.amazonaws.services.kinesis.AmazonKinesisAsyncClient.<init>(AmazonKinesisAsyncClient.java:132)
      

      I believe this is due to the dependency conflict as described @ http://mail-archives.apache.org/mod_mbox/spark-dev/201409.mbox/%3CCAJOb8btdXks-7-spJJ5jMNw0XsnrjwDpCQqtjht1hUn6j4zb_g@mail.gmail.com%3E

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              aniket Aniket Bhatnagar

              Dates

              • Created:
                Updated:
                Resolved:

                Issue deployment