Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-16434

Avoid record-per type dispatch in JSON when reading

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 2.0.0
    • 2.1.0
    • SQL
    • None

    Description

      Currently, JacksonParser.parse is doing type dispatch for each row to read appropriate values.

      It might not have to be done like this because the schema of DataFrame is already there.

      So, appropriate converters can be created first according to the schema, and then apply them to each row.

      Attachments

        Activity

          People

            gurwls223 Hyukjin Kwon
            gurwls223 Hyukjin Kwon
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: