Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-33275

ANSI mode: runtime errors instead of returning null on invalid inputs

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Resolved
    • Major
    • Resolution: Done
    • 3.1.0
    • None
    • SQL
    • None

    Description

      We should respect the ANSI mode in more places. What we have done so far are mostly the overflow check in various operators. This ticket is to track a category of ANSI mode behaviors: operators should throw runtime errors instead of returning null when the input is invalid.

      Attachments

        1.
        		 Division by zero on divide-like operations returns incorrect result Sub-task Resolved Lu Lu
        2.
        		 Accessing array elements should failed if index is out of bound. Sub-task Resolved Leanken.Lin
        3.
        		 Accessing map values should fail if key is not found. Sub-task Resolved Leanken.Lin
        4.
        		 ParseUrl should fail if input string is not a valid url Sub-task Resolved XiDuo You
        5.
        		 Datetime parsing should fail if the input string can't be parsed, or the pattern string is invalid Sub-task Resolved Leanken.Lin
        6.
        		 Datetime building should fail if the year, month, ..., second combination is invalid Sub-task Resolved zhoukeyong
        7.
        		 Fix the constant folding rule to skip it if the expression fails to execute Sub-task In Progress Unassigned
        8.
        		 ANSI mode: Casting string to date should throw exception on parse error Sub-task Resolved Gengliang Wang
        9.
        		 next_day function should throw runtime exception when receiving invalid input under ANSI mode Sub-task Resolved Chongguang LIU
        10.
        		 ANSI mode: Casting String to Boolean should throw exception on parse error Sub-task Resolved Gengliang Wang
        11.
        		 ANSI mode: Abs throws exception if input is out of range Sub-task Resolved Gengliang Wang
        12.
        		 ANSI mode: IntegralDivide throws exception on overflow Sub-task Resolved Gengliang Wang
        13.
        		 The ANSI flag of Cast should be kept after being copied Sub-task Resolved Gengliang Wang
        14.
        		 The ANSI flags of Sum and Avg should be kept after being copied Sub-task Resolved Gengliang Wang
        15.
        		 ANSI mode: always return null on invalid access to map column Sub-task Resolved Gengliang Wang
        16.
        		 ANSI mode: allow explicitly casting fraction strings as Integral types Sub-task Resolved Gengliang Wang
        17.
        		 Round/Bround should return an error on integral overflow Sub-task Resolved Gengliang Wang
        18.
        		 Conv should return an error if the internal conversion overflows Sub-task Resolved Gengliang Wang

        Activity

          People

            apachespark Apache Spark
            cloud_fan Wenchen Fan
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: