Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-39089

Make unionByName null-filling behavior work with Array/Array of Structs columns

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 3.1.0
    • 3.1.0
    • SQL
    • None

    Description

      https://issues.apache.org/jira/browse/SPARK-29358 added support for unionByName to work when the two datasets didn't necessarily have the same schema, but it does not work with nested array columns.

      This has been solved for Structs. https://issues.apache.org/jira/browse/SPARK-32376

      Attachments

        Issue Links

          is a clone of

          Improvement - An improvement or enhancement to an existing feature or task. SPARK-32376 Make unionByName null-filling behavior work with struct columns

          • Major - Major loss of function.
          • Resolved

          Activity

            People

              viirya L. C. Hsieh
              workingWithTrips V. Nguyen
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated: