[BEAM-12244] Failure to connect to expansion service with GRPC local secure channel

• Type: Bug
• Status: Open
• Priority: P3
• Resolution: Unresolved
• Component/s: cross-language, io-py-kafka
• Assignee: Unassigned
• Reporter: Nir Gazit
• Created: 28/Apr/21 17:15
• Updated: 04/Jun/22 20:43

Description

There seem to be a discrepancy between how Beam Python SDK connects to the job endpoint and the artifact endpoint (where it always uses insecure GRPC connection), and the expansion service (where it tries to use local secure channel if available in the GRPC module).

This has caused an error to be thrown when trying to connect to a remote Flink cluster: 

File "/Users/nir.gazit/.pyenv/versions/beam/lib/python3.7/site-packages/apache_beam/transforms/ptransform.py", line 573, in __ror__
    result = p.apply(self, pvalueish, label)
  File "/Users/nir.gazit/.pyenv/versions/beam/lib/python3.7/site-packages/apache_beam/pipeline.py", line 689, in apply
    pvalueish_result = self.runner.apply(transform, pvalueish, self._options)
  File "/Users/nir.gazit/.pyenv/versions/beam/lib/python3.7/site-packages/apache_beam/runners/runner.py", line 188, in apply
    return m(transform, input, options)
  File "/Users/nir.gazit/.pyenv/versions/beam/lib/python3.7/site-packages/apache_beam/runners/runner.py", line 218, in apply_PTransform
    return transform.expand(input)
  File "/Users/nir.gazit/.pyenv/versions/beam/lib/python3.7/site-packages/apache_beam/transforms/external.py", line 325, in expand
    response = service.Expand(request)
  File "/Users/nir.gazit/.pyenv/versions/beam/lib/python3.7/site-packages/grpc/_channel.py", line 923, in __call__
    return _end_unary_response_blocking(state, call, False, None)
  File "/Users/nir.gazit/.pyenv/versions/beam/lib/python3.7/site-packages/grpc/_channel.py", line 826, in _end_unary_response_blocking
    raise _InactiveRpcError(state)
grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
        status = StatusCode.UNAVAILABLE
        details = "failed to connect to all addresses"
        debug_error_string = "{"created":"@1619629304.651319000","description":"Failed to pick subchannel","file":"src/core/ext/filters/client_channel/client_channel.cc","file_line":5397,"referenced_errors":[{"created":"@1619629304.651278000","description":"failed to connect to all addresses","file":"src/core/ext/filters/client_channel/lb_policy/pick_first/pick_first.cc","file_line":398,"grpc_status":14}]}"

I think all GRPC services (job, artifact, expansion) should use the same way to connect (currently insecure_channel)

Issue Links

• links to: GitHub Pull Request #14671