Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-31286

Specify formats of time zone ID for JSON/CSV option and from/to_utc_timestamp

    XMLWordPrintableJSON

Details

    • Documentation
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 2.4.5, 3.0.0
    • 3.0.0
    • SQL
    • None

    Description

      There are two distinct types of ID (see https://docs.oracle.com/javase/8/docs/api/java/time/ZoneId.html):

      1. Fixed offsets - a fully resolved offset from UTC/Greenwich, that uses the same offset for all local date-times
      2. Geographical regions - an area where a specific set of rules for finding the offset from UTC/Greenwich apply

      For example three-letter time zone IDs are ambitious, and depend on the locale. They have been already deprecated in JDK, see https://docs.oracle.com/javase/8/docs/api/java/util/TimeZone.html :

      For compatibility with JDK 1.1.x, some other three-letter time zone IDs (such as "PST", "CTT", "AST") are also supported. However, their use is deprecated because the same abbreviation is often used for multiple time zones (for example, "CST" could be U.S. "Central Standard Time" and "China Standard Time"), and the Java platform can then only recognize one of them.
      

      The ticket aims to specify formats of the `timeZone` option in JSON/CSV datasource, and the `tz` parameter of the from_utc_timestamp() and to_utc_timestamp() functions.

      Attachments

        Issue Links

          Activity

            People

              maxgekk Max Gekk
              maxgekk Max Gekk
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: