Details
-
Umbrella
-
Status: Resolved
-
Major
-
Resolution: Resolved
-
3.4.0
-
None
-
None
Description
There are derivative APIs which depend on the implementation of Pandas UDFs: Pandas Function APIs and Arrow Function APIs, as shown below:
Spark Connect Python Client (SCPC), as a client and server interface for PySpark will eventually replace the legacy API of PySpark. Supporting PySpark UDFs is essential for Spark Connect to reach parity with the PySpark legacy API.
See design doc here.
Attachments
Attachments
Issue Links
- depends upon
-
-
- Resolved
-
1.
|
Implement Grouped Map API |
|
Resolved | Xinrong Meng |
2.
|
Standardize `returnType` property of UserDefinedFunction |
|
Resolved | Takuya Ueshin |
3.
|
Implement `DataFrame.mapInPandas` |
|
Resolved | Xinrong Meng |
4.
|
Rename FrameMap proto to MapPartitions |
|
Resolved | Xinrong Meng |
5.
|
Implement `DataFrame.mapInArrow` |
|
Resolved | Xinrong Meng |
6.
|
Implement CoGrouped Map API |
|
Resolved | Xinrong Meng |