Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-39419

When the comparator of ArraySort returns null, it should fail.

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 3.3.0
    • 3.2.2, 3.3.1
    • SQL
    • None

    Description

      When the comparator of ArraySort returns null, currently it handles it as 0 (equal).

      According to the doc,

      It returns -1, 0, or 1 as the first element is less than, equal to, or greater than the second element. If the comparator function returns other values (including null), the function will fail and raise an error.

      It's fine to return non -1, 0, 1 integers to follow the Java convention (still need to update the doc, though), but it should throw an exception for null result.

      Attachments

        Activity

          People

            ueshin Takuya Ueshin
            ueshin Takuya Ueshin
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: