Description
Python-only UDTs can't work well. One example is:
import pyspark.sql.group from pyspark.sql.tests import PythonOnlyPoint, PythonOnlyUDT from pyspark.sql.types import * schema = StructType().add("key", LongType()).add("val", PythonOnlyUDT()) df = spark.createDataFrame([(i % 3, PythonOnlyPoint(float(i), float(i))) for i in range(10)], schema=schema) df.collect() # this works df.show() # this doesn't work
Attachments
Issue Links
- is related to
-
SPARK-15989 PySpark SQL python-only UDTs don't support nested types
-
- Resolved
-
- links to
