[FLINK-16025] Service could expose blob server port mismatched with JM Container

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.10.0
• Fix Version/s: 1.10.1
• Component/s: Deployment / Kubernetes
• Labels: pull-request-available
• Assignee: Canbin Zheng
• Reporter: Canbin Zheng
• Created: 13/Feb/20 01:56
• Updated: 10/Mar/20 10:33
• Resolved: 10/Mar/20 10:33

Description

The Service would always expose 6124 port if it should expose that port, and while building ServicePort we do not explicitly specify a target port, so the target port would always be 6124 too.

// From ServiceDecorator.java

servicePorts.add(getServicePort(
 getPortName(BlobServerOptions.PORT.key()),
 Constants.BLOB_SERVER_PORT));

private ServicePort getServicePort(String name, int port) {
   return new ServicePortBuilder()
      .withName(name)
      .withPort(port)
      .build();
}

 

meanwhile, the Container of the JM would expose the blob server port which is configured in the Flink Configuration,

// From FlinkMasterDeploymentDecorator.java

final int blobServerPort = KubernetesUtils.parsePort(flinkConfig, BlobServerOptions.PORT);

...

final Container container = createJobManagerContainer(flinkConfig, mainClass, hasLogback, hasLog4j, blobServerPort);

 

so there is a risk that in non-HA mode the TM could not execute Task due to dependencies fetching failure if the Service exposes a blob server port which is different from the JM Container when one configures the blob server port with a value different from 6124.

 

Issue Links

• is part of: FLINK-16194 Refactor the Kubernetes decorator design
• links to: GitHub Pull Request #11349