Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-19595

from_json produces only a single row when input is a json array

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • 2.2.0
    • 2.2.0
    • SQL
    • None

    Description

      Currently, from_json reads a single row when it is a json array. For example,

      import org.apache.spark.sql.functions._
      import org.apache.spark.sql.types._
      val schema = StructType(StructField("a", IntegerType) :: Nil)
      Seq(("""[{"a": 1}, {"a": 2}]""")).toDF("struct").select(from_json(col("struct"), schema)).show()
      +--------------------+
      |jsontostruct(struct)|
      +--------------------+
      |                 [1]|
      +--------------------+
      

      Maybe we should not support this in that function or it should work like a generator expression.

      Attachments

        Issue Links

          Activity

            People

              hyukjin.kwon Hyukjin Kwon
              hyukjin.kwon Hyukjin Kwon
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: