Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-17414

Set type is not supported for creating data frames

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 2.0.0
    • Fix Version/s: 2.3.0
    • Component/s: SQL
    • Labels:
      None

      Description

      For a case class that has a field of type Set, createDataFrame() method throws an exception saying "Schema for type Set is not supported". Exception is raised by the org.apache.spark.sql.catalyst.ScalaReflection class where Array, Seq and Map types are supported but Set is not. It would be nice to support Set here by default instead of having to write a custom Encoder.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                emre.colak Emre Colak
              • Votes:
                1 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: