Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-18350

Support session local timezone

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 2.2.0
    • SQL

    Description

      As of Spark 2.1, Spark SQL assumes the machine timezone for datetime manipulation, which is bad if users are not in the same timezones as the machines, or if different users have different timezones.

      We should introduce a session local timezone setting that is used for execution.

      An explicit non-goal is locale handling.

      Attachments

        1. sample.csv
          0.1 kB
          Vinayak

        Issue Links

          relates to

          Task - A task that needs to be done. SPARK-12297 Add work-around for Parquet/Hive int96 timestamp bug.

          • Major - Major loss of function.
          • Resolved

          Umbrella - An overarching type made of sub-tasks SPARK-15834 Time zone / locale sensitivity umbrella

          • Major - Major loss of function.
          • Resolved
          links to

          Pull request #16308 [Github] Pull Request #16308 (ueshin)

          There are no Sub-Tasks for this issue.

          Activity

            People

              ueshin Takuya Ueshin
              rxin Reynold Xin
              Votes:
              3 Vote for this issue
              Watchers:
              21 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: