[SPARK-48083] session.copyFromLocalToFs failure with 3.5 client <> 4.0 server

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 4.0.0
• Fix Version/s: 3.5.2
• Component/s: Connect, ML, PySpark
• Labels: pull-request-available
• Assignee: Weichen Xu
• Reporter: Hyukjin Kwon
• Created: 02/May/24 09:33
• Updated: 07/May/24 06:58
• Resolved: 07/May/24 06:58

Description

======================================================================
ERROR [1.120s]: test_save_load (pyspark.ml.tests.connect.test_connect_classification.ClassificationTestsOnConnect.test_save_load)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/runner/work/spark/spark-3.5/python/pyspark/ml/tests/connect/test_legacy_mode_classification.py", line 144, in test_save_load
    estimator.save(fs_path)
  File "/home/runner/work/spark/spark-3.5/python/pyspark/ml/connect/io_utils.py", line 248, in save
    _copy_dir_from_local_to_fs(tmp_local_dir, path)
  File "/home/runner/work/spark/spark-3.5/python/pyspark/ml/connect/io_utils.py", line 57, in _copy_dir_from_local_to_fs
    _copy_file_from_local_to_fs(file_path, dest_file_path)
  File "/home/runner/work/spark/spark-3.5/python/pyspark/ml/connect/io_utils.py", line 39, in _copy_file_from_local_to_fs
    session.copyFromLocalToFs(local_path, dest_path)
  File "/home/runner/work/spark/spark-3.5/python/pyspark/sql/connect/session.py", line 756, in copyFromLocalToFs
    self._client.copy_from_local_to_fs(local_path, dest_path)
  File "/home/runner/work/spark/spark-3.5/python/pyspark/sql/connect/client/core.py", line 1549, in copy_from_local_to_fs
    self._artifact_manager._add_forward_to_fs_artifacts(local_path, dest_path)
  File "/home/runner/work/spark/spark-3.5/python/pyspark/sql/connect/client/artifact.py", line 280, in _add_forward_to_fs_artifacts
    self._request_add_artifacts(requests)
  File "/home/runner/work/spark/spark-3.5/python/pyspark/sql/connect/client/artifact.py", line 259, in _request_add_artifacts
    response: proto.AddArtifactsResponse = self._retrieve_responses(requests)
                                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/runner/work/spark/spark-3.5/python/pyspark/sql/connect/client/artifact.py", line 256, in _retrieve_responses
    return self._stub.AddArtifacts(requests, metadata=self._metadata)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/grpc/_channel.py", line 1536, in __call__
    return _end_unary_response_blocking(state, call, False, None)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/grpc/_channel.py", line 1006, in _end_unary_response_blocking
    raise _InactiveRpcError(state)  # pytype: disable=not-instantiable
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
	status = StatusCode.INTERNAL
	details = "Uploading artifact file to local file system destination path is not supported."
	debug_error_string = "UNKNOWN:Error received from peer  {grpc_message:"Uploading artifact file to local file system destination path is not supported.", grpc_status:13, created_time:"2024-05-01T03:01:32.[558](https://github.com/HyukjinKwon/spark/actions/runs/8904629949/job/24454181142#step:9:559)489983+00:00"}"
>

Issue Links

• links to: GitHub Pull Request #46419