Details
-
Improvement
-
Status: In Progress
-
Major
-
Resolution: Unresolved
-
3.0.1
-
None
Description
I am opening this issue to track pull request here: https://github.com/apache/spark/pull/31744
The R arrow package now supports FloatType, BinaryType, ArrayType,StructType. This was brought to my attention by Neal Richardson, maintainer of the R Arrow package, in the comments on this issue: https://issues.apache.org/jira/browse/ARROW-3783
The proposed changes allows SparkR users to leverage arrow optimization for the additional types supported. Documentation for currently supported types can be viewed at https://arrow.apache.org/docs/r/articles/arrow.html#arrow-to-r