[FLINK-20944] Launching in application mode requesting a ClusterIP rest service type results in an Exception

• Type: Bug
• Status: Closed
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 1.12.0
• Fix Version/s: 1.12.2, 1.13.0
• Component/s: Deployment / Kubernetes
• Labels: pull-request-available

Environment

Kubernetes 1.17

Flink 1.12

Running ./bin/flink from an Ubuntu 18.04 host

• Assignee: Yang Wang
• Reporter: Jason Brome
• Created: 12/Jan/21 21:33
• Updated: 23/Aug/21 13:54
• Resolved: 21/Jan/21 09:33

Description

Run a Flink job in Kubernetes in application mode, specifying kubernetes.rest-service.exposed.type=ClusterIP, results in the job being started, however the call to ./bin/flink throws an UnknownHostException Exception on the client.

Command line:

./bin/flink run-application --target kubernetes-application -Dkubernetes.cluster-id=myjob-qa -Dkubernetes.container.image=SOME_REDACTED_PATH/somrepo/someimage -Dkubernetes.service-account=flink-service-account -Dkubernetes.namespace=myjob-qa -Dkubernetes.rest-service.exposed.type=ClusterIP local:///opt/flink
/usrlib/my-job.jar

Output:

2021-01-12 20:29:19,047 INFO org.apache.flink.kubernetes.utils.KubernetesUtils [] - Kubernetes deployment requires a fixed port. Configuration blob.server.port will be set to 6124
2021-01-12 20:29:19,048 INFO org.apache.flink.kubernetes.utils.KubernetesUtils [] - Kubernetes deployment requires a fixed port. Configuration taskmanager.rpc.port will be set to 6122
2021-01-12 20:29:20,369 ERROR org.apache.flink.kubernetes.kubeclient.Fabric8FlinkKubeClient [] - A Kubernetes exception occurred.
java.net.UnknownHostException: myjob-qa-rest.myjob-qa: Name or service not known
at java.net.Inet4AddressImpl.lookupAllHostAddr(Native Method) ~[?:1.8.0_275]
at java.net.InetAddress$2.lookupAllHostAddr(InetAddress.java:929) ~[?:1.8.0_275]
at java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1324) ~[?:1.8.0_275]
at java.net.InetAddress.getAllByName0(InetAddress.java:1277) ~[?:1.8.0_275]
at java.net.InetAddress.getAllByName(InetAddress.java:1193) ~[?:1.8.0_275]
at java.net.InetAddress.getAllByName(InetAddress.java:1127) ~[?:1.8.0_275]
at java.net.InetAddress.getByName(InetAddress.java:1077) ~[?:1.8.0_275]
at org.apache.flink.runtime.highavailability.HighAvailabilityServicesUtils.getWebMonitorAddress(HighAvailabilityServicesUtils.java:193) ~[flink-dist_2.12-1.12.0.jar:1.12.0]
at org.apache.flink.kubernetes.KubernetesClusterDescriptor.lambda$createClusterClientProvider$0(KubernetesClusterDescriptor.java:114) ~[flink-dist_2.12-1.12.0.jar:1.12.0]
at org.apache.flink.kubernetes.KubernetesClusterDescriptor.deployApplicationCluster(KubernetesClusterDescriptor.java:185) ~[flink-dist_2.12-1.12.0.jar:1.12.0]
at org.apache.flink.client.deployment.application.cli.ApplicationClusterDeployer.run(ApplicationClusterDeployer.java:64) ~[flink-dist_2.12-1.12.0.jar:1.12.0]
at org.apache.flink.client.cli.CliFrontend.runApplication(CliFrontend.java:207) ~[flink-dist_2.12-1.12.0.jar:1.12.0]
at org.apache.flink.client.cli.CliFrontend.parseAndRun(CliFrontend.java:974) ~[flink-dist_2.12-1.12.0.jar:1.12.0]
at org.apache.flink.client.cli.CliFrontend.lambda$main$10(CliFrontend.java:1047) ~[flink-dist_2.12-1.12.0.jar:1.12.0]
at org.apache.flink.runtime.security.contexts.NoOpSecurityContext.runSecured(NoOpSecurityContext.java:30) [flink-dist_2.12-1.12.0.jar:1.12.0]
at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1047) [flink-dist_2.12-1.12.0.jar:1.12.0]

------------------------------------------------------------
The program finished with the following exception:

java.lang.RuntimeException: org.apache.flink.client.deployment.ClusterRetrieveException: Could not create the RestClusterClient.
at org.apache.flink.kubernetes.KubernetesClusterDescriptor.lambda$createClusterClientProvider$0(KubernetesClusterDescriptor.java:118)
at org.apache.flink.kubernetes.KubernetesClusterDescriptor.deployApplicationCluster(KubernetesClusterDescriptor.java:185)
at org.apache.flink.client.deployment.application.cli.ApplicationClusterDeployer.run(ApplicationClusterDeployer.java:64)
at org.apache.flink.client.cli.CliFrontend.runApplication(CliFrontend.java:207)
at org.apache.flink.client.cli.CliFrontend.parseAndRun(CliFrontend.java:974)
at org.apache.flink.client.cli.CliFrontend.lambda$main$10(CliFrontend.java:1047)
at org.apache.flink.runtime.security.contexts.NoOpSecurityContext.runSecured(NoOpSecurityContext.java:30)
at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1047)
Caused by: org.apache.flink.client.deployment.ClusterRetrieveException: Could not create the RestClusterClient.
... 8 more
Caused by: java.net.UnknownHostException: myjob-qa-rest.myjob-qa: Name or service not known
at java.net.Inet4AddressImpl.lookupAllHostAddr(Native Method)
at java.net.InetAddress$2.lookupAllHostAddr(InetAddress.java:929)
at java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1324)
at java.net.InetAddress.getAllByName0(InetAddress.java:1277)
at java.net.InetAddress.getAllByName(InetAddress.java:1193)
at java.net.InetAddress.getAllByName(InetAddress.java:1127)
at java.net.InetAddress.getByName(InetAddress.java:1077)
at org.apache.flink.runtime.highavailability.HighAvailabilityServicesUtils.getWebMonitorAddress(HighAvailabilityServicesUtils.java:193)
at org.apache.flink.kubernetes.KubernetesClusterDescriptor.lambda$createClusterClientProvider$0(KubernetesClusterDescriptor.java:114)
... 7 more

 

Issue Links

• links to: GitHub Pull Request #14692