Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-23194

from_json in FAILFAST mode doesn't fail fast, instead it just returns nulls

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Duplicate
    • 2.3.0
    • None
    • SQL
    • None

    Description

      from_json accepts Json parsing options such as being PERMISSIVE to parsing errors or failing fast. It seems from the code that even though the default option is to fail fast, we catch that exception and return nulls.

       

      In order to not change behavior, we should remove that try-catch block and change the default to permissive, but allow failfast mode to indeed fail.

      Attachments

        Issue Links

          duplicates

          Improvement - An improvement or enhancement to an existing feature or task. SPARK-25243 Use FailureSafeParser in from_json

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Activity

            People

              Unassigned Unassigned
              brkyvz Burak Yavuz
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: