Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-15834 Time zone / locale sensitivity umbrella
  3. SPARK-14057

sql time stamps do not respect time zones

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Minor
    • Resolution: Duplicate
    • 1.6.0
    • None
    • SQL
    • None

    Description

      we have time stamp data. The time stamp data is UTC how ever when we load the data into spark data frames, the system assume the time stamps are in the local time zone. This causes problems for our data scientists. Often they pull data from our data center into their local macs. The data centers run UTC. There computers are typically in PST or EST.

      It is possible to hack around this problem

      This cause a lot of errors in their analysis

      A complete description of this issue can be found in the following mail msg

      https://www.mail-archive.com/user@spark.apache.org/msg48121.html

      Attachments

        Issue Links

          duplicates

          Sub-task - The sub-task of the issue SPARK-18936 Infrastructure for session local timezone support

          • Major - Major loss of function.
          • Resolved

          Activity

            People

              Unassigned Unassigned
              aedwip Andrew Davidson
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: