It would be valuable for applications that use Arrow to be able to
- Convert between Spark DataFrames/Datasets and Java Arrow vectors
- Send / Receive Arrow record batches / Arrow file format RPCs to / from Spark
- Allow PySpark to use Arrow for messaging in UDF evaluation
- relates to
-
SPARK-13534 Implement Apache Arrow serializer for Spark DataFrame for use in DataFrame.toPandas
-
- Resolved
-