Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-34246 New type coercion syntax rules in ANSI mode
  3. SPARK-34769

AnsiTypeCoercion: return narrowest convertible type among TypeCollection

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 3.2.0
    • 3.2.0
    • SQL
    • None

    Description

      Currently, when implicit casting a data type to a `TypeCollection`, Spark returns the first convertible data type among `TypeCollection`.
      In ANSI mode, we can make the behavior more reasonable by returning the narrowest convertible data type in `TypeCollection`.

      In details, we first try to find the all the expected types we can implicitly cast:
      1. if there is no convertible data types, return None;
      2. if there is only one convertible data type, cast input as it;
      3. otherwise if there are multiple convertible data types, find the narrowest common data
      type among them. If there is no such narrowest common data type, return None.

      Note that if the narrowest common type is Float type and the convertible types contains Double ype, simply return Double type as the narrowest common type to avoid potential
      precision loss on converting the Integral type as Float type.

      Attachments

        Activity

          People

            Gengliang.Wang Gengliang Wang
            Gengliang.Wang Gengliang Wang
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: