Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-18350

Support session local timezone

Log workAgile BoardRank to TopRank to BottomAttach filesAttach ScreenshotBulk Copy AttachmentsBulk Move AttachmentsVotersWatch issueWatchersCreate sub-taskMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete CommentsDelete
    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 2.2.0
    • SQL

    Description

      As of Spark 2.1, Spark SQL assumes the machine timezone for datetime manipulation, which is bad if users are not in the same timezones as the machines, or if different users have different timezones.

      We should introduce a session local timezone setting that is used for execution.

      An explicit non-goal is locale handling.

      Attachments

        Issue Links

        relates to

        Task - A task that needs to be done. SPARK-12297 Add work-around for Parquet/Hive int96 timestamp bug.

        • Major - Major loss of function.
        • Resolved

        Umbrella - An overarching type made of sub-tasks SPARK-15834 Time zone / locale sensitivity umbrella

        • Major - Major loss of function.
        • Resolved
        links to

        Pull request #16308 [Github] Pull Request #16308 (ueshin)

        Activity
          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            ueshin Takuya Ueshin Assign to me
            rxin Reynold Xin
            Votes:
            3 Vote for this issue
            Watchers:
            21 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment