Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-13070

Points out which physical file is the trouble maker when Parquet schema merging fails

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • 2.0.0
    • 2.0.0
    • SQL
    • None

    Description

      As a user, I'd like to know which physical file is the trouble maker when Parquet schema merging fails. Currently, we only have an error message like this:

      Failed to merge incompatible data types LongType and IntegerType

      Would be nice to add the file path and the actual schema.

      Attachments

        Activity

          People

            lian cheng Cheng Lian
            lian cheng Cheng Lian
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: