Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-42393

Support for Pandas/Arrow Functions API

    XMLWordPrintableJSON

Details

    • Umbrella
    • Status: Resolved
    • Major
    • Resolution: Resolved
    • 3.4.0
    • None
    • Connect, PySpark
    • None

    Description

      There are derivative APIs which depend on the implementation of Pandas UDFs: Pandas Function APIs and Arrow Function APIs, as shown below:

       

      Spark Connect Python Client (SCPC), as a client and server interface for PySpark will eventually replace the legacy API of PySpark. Supporting PySpark UDFs is essential for Spark Connect to reach parity with the PySpark legacy API.

      See design doc here.

      Attachments

        Issue Links

          There are no Sub-Tasks for this issue.

          Activity

            People

              XinrongM Xinrong Meng
              XinrongM Xinrong Meng
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: