Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-26455

Spark Kinesis Integration with no SSL

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Invalid
    • 2.3.0
    • None
    • Spark Submit
    • None

    Description

      Hi, 

      we are trying access the endpoint thought library mentioned below and we get the SSL error i think internally it use KCL library. if you look at the error, so if I have to skip the certificate is it possible through KCL utils call ? because I do not find any provision to do that with set SSL as false within spark streaming kinesis library like we do with KCL. Can you please help me with the same.

      compile("org.apache.spark:spark-streaming-kinesis-asl_2.11:2.3.0")

      { exclude group: 'org.apache.spark', module: 'spark-streaming_2.11' }

      Caused by: javax.net.ssl.SSLPeerUnverifiedException: Certificate for kinesis-endpoint> doesn't match any of the subject alternative names: [kinesis-fips.us-east-1.amazonaws.com, *.kinesis.us-east-1.vpce.amazonaws.com, kinesis.us-east-1.amazonaws.com]
      at org.apache.http.conn.ssl.SSLConnectionSocketFactory.verifyHostname(SSLConnectionSocketFactory.java:467)
      at org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:397)
      at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:355)
      at shade.com.amazonaws.http.conn.ssl.SdkTLSSocketFactory.connectSocket(SdkTLSSocketFactory.java:132)
      at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142)
      at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:373)
      at sun.reflect.GeneratedMethodAccessor19.invoke(Unknown Source)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      at java.lang.reflect.Method.invoke(Method.java:498)
      at shade.com.amazonaws.http.conn.ClientConnectionManagerFactory$Handler.invoke(ClientConnectionManagerFactory.java:76)
      at shade.com.amazonaws.http.conn.$Proxy18.connect(Unknown Source)
      at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381)
      at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237)
      at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
      at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
      at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
      at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
      at shade.com.amazonaws.http.apache.client.impl.SdkHttpClient.execute(SdkHttpClient.java:72)
      at shade.com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1238)
      at shade.com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1058)
      ... 20 more

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            Unassigned Unassigned
            shzshi Shashikant Bangera
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment