Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-16609

Single function for parsing timestamps/dates

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 2.2.0
    • SQL
    • None

    Description

      Today, if you want to parse a date or timestamp, you have to use the unix time function and then cast to a timestamp. Its a little odd there isn't a single function that does both. I propose we add

      to_date(<input>, <pattern>)/to_timestamp(<input>, <pattern>).
      

      For reference, in other systems there are:

      MS SQL: convert(<input>, <pattern id>). See: https://technet.microsoft.com/en-us/library/ms174450(v=sql.110).aspx
      Netezza: to_timestamp(<input>, <pattern>). See: https://www.ibm.com/support/knowledgecenter/SSULQD_7.0.3/com.ibm.nz.dbu.doc/r_dbuser_ntz_sql_extns_conversion_funcs.html
      Teradata has special casting functionality: cast(<input> as timestamp format '<pattern>')
      MySql: STR_TO_DATE(<input>, <pattern>). This returns a datetime when you define both date and time parts. See: https://dev.mysql.com/doc/refman/5.5/en/date-and-time-functions.html

      Attachments

        Issue Links

          Activity

            People

              bill_chambers Bill Chambers
              marmbrus Michael Armbrust
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: