Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-34392

Invalid ID for offset-based ZoneId since Spark 3.0

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 3.0.0, 3.0.1
    • 3.0.3, 3.1.2, 3.2.0
    • SQL
    • None

    Description

      How to reproduce this issue:

      select to_utc_timestamp("2020-02-07 16:00:00", "GMT+8:00");
      

      Spark 2.4:

      spark-sql> select to_utc_timestamp("2020-02-07 16:00:00", "GMT+8:00");
      2020-02-07 08:00:00
      Time taken: 0.089 seconds, Fetched 1 row(s)
      

      Spark 3.x:

      spark-sql> select to_utc_timestamp("2020-02-07 16:00:00", "GMT+8:00");
      21/02/07 01:24:32 ERROR SparkSQLDriver: Failed in [select to_utc_timestamp("2020-02-07 16:00:00", "GMT+8:00")]
      java.time.DateTimeException: Invalid ID for offset-based ZoneId: GMT+8:00
      	at java.time.ZoneId.ofWithPrefix(ZoneId.java:437)
      	at java.time.ZoneId.of(ZoneId.java:407)
      	at java.time.ZoneId.of(ZoneId.java:359)
      	at java.time.ZoneId.of(ZoneId.java:315)
      	at org.apache.spark.sql.catalyst.util.DateTimeUtils$.getZoneId(DateTimeUtils.scala:53)
      	at org.apache.spark.sql.catalyst.util.DateTimeUtils$.toUTCTime(DateTimeUtils.scala:814)
      

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            wangshikai karl wang
            yumwang Yuming Wang
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment