[BEAM-11957] Worker endpoints are unset in StartWorkerRequest.

• Type: Bug
• Status: Open
• Priority: P3
• Resolution: Unresolved
• Component/s: java-fn-execution
• Assignee: Unassigned
• Reporter: Kyle Weaver
• Created: 10/Mar/21 21:43
• Updated: 04/Jun/22 19:58

Description

User reported that worker endpoints are unset when attempting to start a worker in a worker pool using Kubernetes. https://stackoverflow.com/questions/66498209/how-to-configure-beam-python-sdk-with-spark-in-a-kubernetes-environment

Starting worker with command ['/opt/apache/beam/boot', '--id=1-1', '--logging_endpoint=', '--artifact_endpoint=', '--provision_endpoint=', '--control_endpoint=']

These endpoints are expected to be set to use localhost and some random allocated port. The server is started here: https://github.com/apache/beam/blob/b101064f18e71adb81c8fe6ba7beae53e9b13b08/runners/java-fn-execution/src/main/java/org/apache/beam/runners/fnexecution/control/DefaultJobBundleFactory.java#L672-L697
And the StartWorkerRequest is constructed here: https://github.com/apache/beam/blob/master/runners/java-fn-execution/src/main/java/org/apache/beam/runners/fnexecution/environment/ExternalEnvironmentFactory.java