Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-25417

ArrayContains function may return incorrect result when right expression is implicitly down casted

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.4.0
    • Fix Version/s: 2.4.0
    • Component/s: SQL
    • Labels:
      None

      Description

      In ArrayContains, we currently cast the right hand side expression to match the element type of the left hand side Array. This may result in down casting and may return wrong result or questionable result.

      Example :

      spark-sql> select array_position(array(1), 1.34);
      true
      
      
      

       

      spark-sql> select array_position(array(1), 'foo');
      null
      

        Attachments

          Activity

            People

            • Assignee:
              dkbiswal Dilip Biswal
              Reporter:
              dkbiswal Dilip Biswal
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: