Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-25514

Generating pretty JSON by to_json

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • 2.4.0
    • 3.0.0
    • SQL
    • None

    Description

      It would be nice to have an option, for example "pretty", which enable special output mode for the to_json function. In the mode, produced JSON string will have easily readable representation. For example:

      val json = """[{"book":{"publisher":[{"country":"NL","year":[1981,1986,1999]}]}}]"""
      to_json(from_json('col, ...), Map("pretty" -> "true")))
      
      [ {
        "book" : {
          "publisher" : [ {
            "country" : "NL",
            "year" : [ 1981, 1986, 1999 ]
          } ]
        }
      } ]
      

      There are at least two use cases:

      1. Exploring content of nested columns. For example, a result of your query is a few rows, and some columns have deep nested structure. And you want to analyze and find a value of one of nested fields.
      2. You already have an JSON in one of columns, and want to explore the JSON records. New option will allow to do that easily without copy-past JSON content to an editor by combining from_json and to_json functions.

      Attachments

        Activity

          People

            maxgekk Max Gekk
            maxgekk Max Gekk
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: