Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-25514

Generating pretty JSON by to_json

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 2.4.0
    • Fix Version/s: 3.0.0
    • Component/s: SQL
    • Labels:
      None

      Description

      It would be nice to have an option, for example "pretty", which enable special output mode for the to_json function. In the mode, produced JSON string will have easily readable representation. For example:

      val json = """[{"book":{"publisher":[{"country":"NL","year":[1981,1986,1999]}]}}]"""
      to_json(from_json('col, ...), Map("pretty" -> "true")))
      
      [ {
        "book" : {
          "publisher" : [ {
            "country" : "NL",
            "year" : [ 1981, 1986, 1999 ]
          } ]
        }
      } ]
      

      There are at least two use cases:

      1. Exploring content of nested columns. For example, a result of your query is a few rows, and some columns have deep nested structure. And you want to analyze and find a value of one of nested fields.
      2. You already have an JSON in one of columns, and want to explore the JSON records. New option will allow to do that easily without copy-past JSON content to an editor by combining from_json and to_json functions.

        Attachments

          Activity

            People

            • Assignee:
              maxgekk Max Gekk
              Reporter:
              maxgekk Max Gekk

              Dates

              • Created:
                Updated:
                Resolved:

                Issue deployment