Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-25098

Trim the string when cast stringToTimestamp and stringToDate

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 2.3.0
    • 3.0.0
    • SQL
    • None

    Description

      UDF ‘Cast’ will return NULL when input string starts/ends with special character, but hive doesn't.

      For examle, we get hour from a string ends with a blank :

      hive:

      ```
      hive> SELECT CAST(' 2018-08-13' AS DATE);//starts with a blank
      OK
      2018-08-13

      hive> SELECT HOUR('2018-08-13 17:20:07 );//ends with a blank
      OK
      17
      ```

      spark-sql:

      ```
      spark-sql> SELECT CAST(' 2018-08-13' AS DATE);//starts with a blank
      NULL

      spark-sql> SELECT HOUR('2018-08-13 17:20:07 );//ends with a blank
      NULL
      ```

      All of the following UDFs will be affected:
      ```
      year
      month
      day
      hour
      minute
      second
      date_add
      date_sub
      ```

      Attachments

        Issue Links

          Activity

            People

              yumwang Yuming Wang
              a283791411 ice bai
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: