Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-26651 Use Proleptic Gregorian calendar
  3. SPARK-26503

Get rid of spark.sql.legacy.timeParser.enabled

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • 3.0.0
    • 3.0.0
    • SQL
    • None

    Description

      The flag is used in CSV/JSON datasources as well in time related functions to control parsing/formatting date/timestamps. By default, DateTimeFormat is used for the purpose but the flag allow switch back to SimpleDateFormat and some fallback. In the major release 3.0, the flag should be removed, and new formatters DateFormatter/TimestampFormatter should be used by default.

      Attachments

        Issue Links

          is related to

          Bug - A problem which impairs or prevents the functions of the product. SPARK-30668 to_timestamp failed to parse 2020-01-27T20:06:11.847-0800 using pattern "yyyy-MM-dd'T'HH:mm:ss.SSSz"

          • Blocker - Blocks development and/or testing work, production could not run
          • Resolved
          relates to

          Sub-task - The sub-task of the issue SPARK-26374 Support new date/timestamp parser in HadoopFsRelationTest

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved
          links to

          Web Link GitHub Pull Request #23411

          Web Link GitHub Pull Request #23495

          Web Link GitHub Pull Request #23529

          Activity

            People

              srowen Sean R. Owen
              maxgekk Max Gekk
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: