Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-22125

Enable Arrow Stream format for vectorized UDF.

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 2.3.0
    • 2.3.0
    • PySpark, SQL
    • None

    Description

      Currently we use Arrow File format to communicate with Python worker when invoking vectorized UDF but we can use Arrow Stream format.

      Attachments

        Issue Links

          Activity

            People

              ueshin Takuya Ueshin
              ueshin Takuya Ueshin
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: