Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-4522

Failure to read parquet schema with missing metadata.

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: SQL
    • Labels:
      None
    • Target Version/s:

      Description

      Our JSON parsing code is very strict and will fail, even if optional fields like metadata are missing. This means that people who are reading parquet files written with code from between 1.1 and 1.2 might loose type information for no reason.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                marmbrus Michael Armbrust
                Reporter:
                marmbrus Michael Armbrust
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: