Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-45981 Improve Python language test coverage
  3. SPARK-45987

Fix `pyspark.sql.tests.connect.test_connect_basic` in Python 3.11

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 4.0.0
    • 4.0.0
    • PySpark
    • None

    Description

      https://github.com/apache/spark/actions/runs/6914662405/job/18812759788

      ======================================================================
      ERROR [3.529s]: test_recursion_handling_for_plan_logging (pyspark.sql.tests.connect.test_connect_basic.SparkConnectBasicTests.test_recursion_handling_for_plan_logging)
      SPARK-45852 - Test that we can handle recursion in plan logging.
      ----------------------------------------------------------------------
      Traceback (most recent call last):
        File "/__w/spark/spark/python/pyspark/sql/tests/connect/test_connect_basic.py", line 171, in test_recursion_handling_for_plan_logging
          self.assertIsNotNone(cdf.schema)
                               ^^^^^^^^^^
        File "/__w/spark/spark/python/pyspark/sql/connect/dataframe.py", line 1735, in schema
          return self._session.client.schema(query)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/__w/spark/spark/python/pyspark/sql/connect/client/core.py", line 924, in schema
          schema = self._analyze(method="schema", plan=plan).schema
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/__w/spark/spark/python/pyspark/sql/connect/client/core.py", line 1110, in _analyze
          self._handle_error(error)
        File "/__w/spark/spark/python/pyspark/sql/connect/client/core.py", line 1499, in _handle_error
          self._handle_rpc_error(error)
        File "/__w/spark/spark/python/pyspark/sql/connect/client/core.py", line 1570, in _handle_rpc_error
          raise SparkConnectGrpcException(str(rpc_error)) from None
      pyspark.errors.exceptions.connect.SparkConnectGrpcException: <_InactiveRpcError of RPC that terminated with:
      	status = StatusCode.INTERNAL
      	details = "Exception serializing request!"
      	debug_error_string = "None"
      >
      
      ----------------------------------------------------------------------
      Ran 141 tests in 86.259s
      
      FAILED (errors=1)
      

      Attachments

        Issue Links

          Activity

            People

              dongjoon Dongjoon Hyun
              dongjoon Dongjoon Hyun
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: