[FLINK-32031] Flink GCP Connector having issues with Conscrypt library

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.15.4
• Fix Version/s: gcp-pubsub-3.0.2, gcp-pubsub-3.1.0
• Component/s: Connectors / Google Cloud PubSub
• Labels: pubsub pull-request-available
• Assignee: Jayadeep Jayaraman
• Reporter: Jayadeep Jayaraman
• Created: 08/May/23 10:31
• Updated: 03/Nov/23 10:59
• Resolved: 04/Jul/23 09:08

Description

When using the current pubsub connector it is not using the latest libraries bom due to which when the connector is used in Cloud Dataproc it is failing with conscrypt related issues.

 

Caused by: repackaged.io.grpc.netty.shaded.io.netty.channel.AbstractChannel$AnnotatedSocketException: Network is unreachable: pubsub.googleapis.com/2607:f8b0:4001:c23:0:0:0:5f:443
Caused by: java.net.SocketException: Network is unreachable
    at java.base/sun.nio.ch.Net.connect0(Native Method)
    at java.base/sun.nio.ch.Net.connect(Net.java:483)
    at java.base/sun.nio.ch.Net.connect(Net.java:472)
    at java.base/sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:692)
    at repackaged.io.grpc.netty.shaded.io.netty.util.internal.SocketUtils$3.run(SocketUtils.java:91)
    at repackaged.io.grpc.netty.shaded.io.netty.util.internal.SocketUtils$3.run(SocketUtils.java:88)
    at java.base/java.security.AccessController.doPrivileged(Native Method)
    at repackaged.io.grpc.netty.shaded.io.netty.util.internal.SocketUtils.connect(SocketUtils.java:88)
    at repackaged.io.grpc.netty.shaded.io.netty.channel.socket.nio.NioSocketChannel.doConnect(NioSocketChannel.java:315)
    at repackaged.io.grpc.netty.shaded.io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.connect(AbstractNioChannel.java:248)
    at repackaged.io.grpc.netty.shaded.io.netty.channel.DefaultChannelPipeline$HeadContext.connect(DefaultChannelPipeline.java:1342)
    at repackaged.io.grpc.netty.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeConnect(AbstractChannelHandlerContext.java:548)
    at repackaged.io.grpc.netty.shaded.io.netty.channel.AbstractChannelHandlerContext.connect(AbstractChannelHandlerContext.java:533)
    at repackaged.io.grpc.netty.shaded.io.netty.channel.ChannelDuplexHandler.connect(ChannelDuplexHandler.java:54)
    at repackaged.io.grpc.netty.shaded.io.grpc.netty.WriteBufferingAndExceptionHandler.connect(WriteBufferingAndExceptionHandler.java:150)
    at repackaged.io.grpc.netty.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeConnect(AbstractChannelHandlerContext.java:548)
    at repackaged.io.grpc.netty.shaded.io.netty.channel.AbstractChannelHandlerContext.access$1000(AbstractChannelHandlerContext.java:61)
    at repackaged.io.grpc.netty.shaded.io.netty.channel.AbstractChannelHandlerContext$9.run(AbstractChannelHandlerContext.java:538)
    at repackaged.io.grpc.netty.shaded.io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
    at repackaged.io.grpc.netty.shaded.io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472)
    at repackaged.io.grpc.netty.shaded.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:500)
    at repackaged.io.grpc.netty.shaded.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
    at repackaged.io.grpc.netty.shaded.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
    at repackaged.io.grpc.netty.shaded.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
    at java.base/java.lang.Thread.run(Thread.java:829)

On disabling ipv6 getting the following error:-

Channel Pipeline: [SslHandler#0, ProtocolNegotiators$ClientTlsHandler#0, WriteBufferingAndExceptionHandler#0, DefaultChannelPipeline$TailContext#0]
    at io.grpc.stub.ClientCalls.toStatusRuntimeException(ClientCalls.java:244)
    at io.grpc.stub.ClientCalls.getUnchecked(ClientCalls.java:225)
    at io.grpc.stub.ClientCalls.blockingUnaryCall(ClientCalls.java:142)
    at com.google.pubsub.v1.SubscriberGrpc$SubscriberBlockingStub.pull(SubscriberGrpc.java:1641)
    at org.apache.flink.streaming.connectors.gcp.pubsub.BlockingGrpcPubSubSubscriber.pull(BlockingGrpcPubSubSubscriber.java:73)
    at org.apache.flink.streaming.connectors.gcp.pubsub.BlockingGrpcPubSubSubscriber.pull(BlockingGrpcPubSubSubscriber.java:77)
    at org.apache.flink.streaming.connectors.gcp.pubsub.BlockingGrpcPubSubSubscriber.pull(BlockingGrpcPubSubSubscriber.java:77)
    at org.apache.flink.streaming.connectors.gcp.pubsub.BlockingGrpcPubSubSubscriber.pull(BlockingGrpcPubSubSubscriber.java:77)
    at org.apache.flink.streaming.connectors.gcp.pubsub.BlockingGrpcPubSubSubscriber.pull(BlockingGrpcPubSubSubscriber.java:67)
    at org.apache.flink.streaming.connectors.gcp.pubsub.PubSubSource.run(PubSubSource.java:128)
    at org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:110)
    at org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:67)
    at org.apache.flink.streaming.runtime.tasks.SourceStreamTask$LegacySourceFunctionThread.run(SourceStreamTask.java:332)
Caused by: javax.net.ssl.SSLHandshakeException: No subjectAltNames on the certificate match
    at org.conscrypt.SSLUtils.toSSLHandshakeException(SSLUtils.java:361)
    at org.conscrypt.ConscryptEngine.convertException(ConscryptEngine.java:1135)
    at org.conscrypt.ConscryptEngine.readPlaintextData(ConscryptEngine.java:1090)
    at org.conscrypt.ConscryptEngine.unwrap(ConscryptEngine.java:867)
    at org.conscrypt.ConscryptEngine.unwrap(ConscryptEngine.java:738)
    at org.conscrypt.Java8EngineWrapper.unwrap(Java8EngineWrapper.java:252)
    at org.conscrypt.Conscrypt.unwrap(Conscrypt.java:605)    

 

Issue Links

• links to: GitHub Pull Request #13
• links to: GitHub Pull Request #15
• links to: GitHub Pull Request #17