Arrow 0.12.0 is release and it contains R API. We could optimize Spark DaraFrame <> R DataFrame interoperability.
For instance see the examples below:
- R DataFrame -> Spark DataFrame
- Spark DataFrame -> R DataFrame
Currently, some of communication path between R side and JVM side has to buffer the data and flush it at once due to ARROW-4512. I don't target to fix it under this umbrella.