XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 3.0.0
    • Fix Version/s: 3.0.0
    • Component/s: SQL
    • Labels:

      Description

      By default, collect() returns java.sql.Timestamp/Date instances with offsets derived from internal values of Catalyst's TIMESTAMP/DATE that store microseconds since the epoch. The conversion from internal values to java.sql.Timestamp/Date based on Proleptic Gregorian calendar but converting the resulted values before 1582 year to strings produces timestamp/date string in Julian calendar. For example:

      scala> sql("select date '1100-10-10'").collect()
      res1: Array[org.apache.spark.sql.Row] = Array([1100-10-03])
      

      This can be fixed if internal Catalyst's values are converted to local date-time in Gregorian calendar, and construct local date-time from the resulted year, month, ..., seconds in Julian calendar.

        Attachments

          Activity

            People

            • Assignee:
              maxgekk Maxim Gekk
              Reporter:
              maxgekk Maxim Gekk
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: