Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-34246 New type coercion syntax rules in ANSI mode
  3. SPARK-34769

AnsiTypeCoercion: return narrowest convertible type among TypeCollection

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 3.2.0
    • Fix Version/s: 3.2.0
    • Component/s: SQL
    • Labels:
      None

      Description

      Currently, when implicit casting a data type to a `TypeCollection`, Spark returns the first convertible data type among `TypeCollection`.
      In ANSI mode, we can make the behavior more reasonable by returning the narrowest convertible data type in `TypeCollection`.

      In details, we first try to find the all the expected types we can implicitly cast:
      1. if there is no convertible data types, return None;
      2. if there is only one convertible data type, cast input as it;
      3. otherwise if there are multiple convertible data types, find the narrowest common data
      type among them. If there is no such narrowest common data type, return None.

      Note that if the narrowest common type is Float type and the convertible types contains Double ype, simply return Double type as the narrowest common type to avoid potential
      precision loss on converting the Integral type as Float type.

        Attachments

          Activity

            People

            • Assignee:
              Gengliang.Wang Gengliang Wang
              Reporter:
              Gengliang.Wang Gengliang Wang
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: