Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-8010

Implict promote Numeric type to String type in HiveTypeCoercion

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 1.3.1
    • 1.5.0
    • SQL
    • None

    Description

      1. Given a query
      `select coalesce(null, 1, '1') from dual` will cause exception:

      java.lang.RuntimeException: Could not determine return type of Coalesce for IntegerType,StringType

      2. Given a query:
      `select case when true then 1 else '1' end from dual` will cause exception:

      java.lang.RuntimeException: Types in CASE WHEN must be the same or coercible to a common type: StringType != IntegerType

      I checked the code, the main cause is the HiveTypeCoercion doesn't do implicit convert when there is a IntegerType and StringType.

      Numeric types can be promoted to string type in case throw exceptions.

      Since Hive will always do this. It need to be fixed.

      Attachments

        Activity

          People

            OopsOutOfMemory OopsOutOfMemory
            OopsOutOfMemory OopsOutOfMemory
            Yin Huai Yin Huai
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Time Tracking

                Estimated:
                Original Estimate - 48h
                48h
                Remaining:
                Remaining Estimate - 48h
                48h
                Logged:
                Time Spent - Not Specified
                Not Specified