Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-36861

Partition columns are overly eagerly parsed as dates

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Blocker
    • Resolution: Fixed
    • 3.3.0
    • 3.3.0
    • SQL
    • None

    Description

      I have an input directory with subdirs:

      • hour=2021-01-01T00
      • hour=2021-01-01T01
      • hour=2021-01-01T02
      • ...

      in spark 3.1 the 'hour' column is parsed as a string type, but in 3.2 RC it is parsed as date type and the hour part is lost.

      Attachments

        Activity

          People

            maxgekk Max Gekk
            tanelk Tanel Kiis
            Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: