[SPARK-44011] Enable FrameParityBinaryOpsTests.test_binary_operator_multiply

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Not A Problem
• Affects Version/s: 3.5.0
• Component/s: Pandas API on Spark, Tests
• Assignee: Unassigned
• Reporter: Ruifeng Zheng
• Created: 09/Jun/23 02:17
• Updated: 09/Jun/23 02:23
• Resolved: 09/Jun/23 02:23

Description

======================================================================
ERROR [4.378s]: test_binary_operator_multiply (pyspark.pandas.tests.connect.computation.test_parity_binary_ops.FrameParityBinaryOpsTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/ruifeng.zheng/Dev/spark/python/pyspark/pandas/tests/computation/test_binary_ops.py", line 169, in test_binary_operator_multiply
    self.assert_eq(psdf["a"] * psdf["b"], pdf["a"] * pdf["b"])
  File "/Users/ruifeng.zheng/Dev/spark/python/pyspark/pandas/base.py", line 325, in __mul__
    return self._dtype_op.mul(self, other)
  File "/Users/ruifeng.zheng/Dev/spark/python/pyspark/pandas/data_type_ops/string_ops.py", line 78, in mul
    return column_op(SF.repeat)(left, right)
  File "/Users/ruifeng.zheng/Dev/spark/python/pyspark/pandas/base.py", line 222, in wrapper
    scol = f(
  File "/Users/ruifeng.zheng/Dev/spark/python/pyspark/pandas/spark/functions.py", line 94, in repeat
    n = _to_java_column(n) if isinstance(n, Column) else _create_column_from_literal(n)
  File "/Users/ruifeng.zheng/Dev/spark/python/pyspark/sql/column.py", line 51, in _create_column_from_literal
    return cast(JVMView, sc._jvm).functions.lit(literal)
  File "/Users/ruifeng.zheng/Dev/spark/python/lib/py4j-0.10.9.7-src.zip/py4j/java_gateway.py", line 1314, in __call__
    args_command, temp_args = self._build_args(*args)
  File "/Users/ruifeng.zheng/Dev/spark/python/lib/py4j-0.10.9.7-src.zip/py4j/java_gateway.py", line 1277, in _build_args
    (new_args, temp_args) = self._get_args(args)
  File "/Users/ruifeng.zheng/Dev/spark/python/lib/py4j-0.10.9.7-src.zip/py4j/java_gateway.py", line 1264, in _get_args
    temp_arg = converter.convert(arg, self.gateway_client)
  File "/Users/ruifeng.zheng/Dev/spark/python/lib/py4j-0.10.9.7-src.zip/py4j/java_collections.py", line 510, in convert
    for element in object:
  File "/Users/ruifeng.zheng/Dev/spark/python/pyspark/sql/connect/column.py", line 468, in __iter__
    raise PySparkTypeError(
pyspark.errors.exceptions.base.PySparkTypeError: [NOT_ITERABLE] Column is not iterable.