Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-10840

SparkSQL doesn't work well with JSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Minor
    • Resolution: Duplicate
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: SQL
    • Labels:

      Description

      Well formed JSON doesn't work with the 1.5.1 version while using sqlContext.read.json("<json-file>"):

      {
      "employees": {
      "employee": [

      { "name": "Mia",<newline> "surname": "Radison",<newline> "mobile": "7295913821",<newline> "email": "miaradison@sparky.com" }

      ,

      { "name": "Thor",<newline> "surname": "Kovaskz",<newline> "mobile": "8829177193",<newline> "email": "tkovaskz@sparky.com" }

      ,

      { "name": "Bindy",<newline> "surname": "Kvuls",<newline> "mobile": "5033828845",<newline> "email": "bindykk@sparky.com" }

      ]
      }
      }

      For the above following error is obtained:
      ERROR Executor: Exception in task 0.0 in stage 1.0 (TID 2)
      scala.MatchError: (VALUE_STRING,StructType()) (of class scala.Tuple2)

      Where as, this works fine because all components are in the same line:

      [

      {"name": "Mia","surname": "Radison","mobile": "7295913821","email": "miaradison@sparky.com"}

      ,

      {"name": "Thor","surname": "Kovaskz","mobile": "8829177193","email": "tkovaskz@sparky.com"}

      ,

      {"name": "Bindy","surname": "Kvuls","mobile": "5033828845","email": "bindykk@sparky.com"}

      ]

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                djordan Jordan Sarraf
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: