Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-37935 Migrate onto error classes
  3. SPARK-40372

Migrate failures of array type checks onto error classes

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 3.4.0
    • 3.4.0
    • SQL
    • None

    Description

      Replace TypeCheckFailure by DataTypeMismatch in type checks in collection expressions:
      1. SortArray (3): https://github.com/apache/spark/blob/master/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala#L1027-L1035
      2. ArrayContains (2): https://github.com/apache/spark/blob/master/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala#L1259-L1264
      3. ArrayPosition (1): https://github.com/apache/spark/blob/master/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala#L2035
      4. ElementAt (3): https://github.com/apache/spark/blob/master/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala#L2177-L2187
      5. Concat (1): https://github.com/apache/spark/blob/master/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala#L2385-L2388
      6. Flatten (1): https://github.com/apache/spark/blob/master/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala#L2593-L2595
      7. Sequence (1): https://github.com/apache/spark/blob/master/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala#L2773
      8. ArrayRemove (1): https://github.com/apache/spark/blob/master/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala#L3445-L3447
      9. ArrayDistinct (1): https://github.com/apache/spark/blob/master/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala#L3642

      Attachments

        Issue Links

          is a clone of

          Sub-task - The sub-task of the issue SPARK-40358 Migrate collection type check failures onto error classes

          • Major - Major loss of function.
          • Resolved
          is cloned by

          Sub-task - The sub-task of the issue SPARK-40374 Migrate type check failures of type creators onto error classes

          • Major - Major loss of function.
          • Resolved
          links to

          Web Link https://github.com/apache/spark/pull/38507

          Activity

            People

              LuciferYang Yang Jie
              maxgekk Max Gekk
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: