Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-26455

Spark Kinesis Integration with no SSL

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Invalid
    • Affects Version/s: 2.3.0
    • Fix Version/s: None
    • Component/s: Spark Submit
    • Labels:
      None

      Description

      Hi, 

      we are trying access the endpoint thought library mentioned below and we get the SSL error i think internally it use KCL library. if you look at the error, so if I have to skip the certificate is it possible through KCL utils call ? because I do not find any provision to do that with set SSL as false within spark streaming kinesis library like we do with KCL. Can you please help me with the same.

      compile("org.apache.spark:spark-streaming-kinesis-asl_2.11:2.3.0")

      { exclude group: 'org.apache.spark', module: 'spark-streaming_2.11' }

      Caused by: javax.net.ssl.SSLPeerUnverifiedException: Certificate for kinesis-endpoint> doesn't match any of the subject alternative names: [kinesis-fips.us-east-1.amazonaws.com, *.kinesis.us-east-1.vpce.amazonaws.com, kinesis.us-east-1.amazonaws.com]
      at org.apache.http.conn.ssl.SSLConnectionSocketFactory.verifyHostname(SSLConnectionSocketFactory.java:467)
      at org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:397)
      at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:355)
      at shade.com.amazonaws.http.conn.ssl.SdkTLSSocketFactory.connectSocket(SdkTLSSocketFactory.java:132)
      at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142)
      at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:373)
      at sun.reflect.GeneratedMethodAccessor19.invoke(Unknown Source)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      at java.lang.reflect.Method.invoke(Method.java:498)
      at shade.com.amazonaws.http.conn.ClientConnectionManagerFactory$Handler.invoke(ClientConnectionManagerFactory.java:76)
      at shade.com.amazonaws.http.conn.$Proxy18.connect(Unknown Source)
      at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381)
      at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237)
      at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
      at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
      at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
      at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
      at shade.com.amazonaws.http.apache.client.impl.SdkHttpClient.execute(SdkHttpClient.java:72)
      at shade.com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1238)
      at shade.com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1058)
      ... 20 more

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              shzshi Shashikant Bangera
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: