Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-21187 Complete support for remaining Spark data types in Arrow Converters
  3. SPARK-29493

Add MapType support for Arrow Java

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 3.0.0
    • 3.0.0
    • SQL
    • None

    Description

      This will add MapType support for Arrow in Spark ArrowConverters. This can happen after the Arrow 0.15.0 upgrade, but MapType is not available in pyarrow yet, so pyspark and pandas_udf will come later.

      Attachments

        Issue Links

          depends upon

          Improvement - An improvement or enhancement to an existing feature or task. SPARK-29376 Upgrade Apache Arrow to 0.15.1

          • Major - Major loss of function.
          • Resolved
          is depended upon by

          Sub-task - The sub-task of the issue SPARK-24554 Add MapType Support for Arrow in PySpark

          • Major - Major loss of function.
          • Resolved
          links to

          Web Link GitHub Pull Request #26512

          Activity

            People

              jalpan.randeri Jalpan Randeri
              bryanc Bryan Cutler
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: