[SPARK-32115] Incorrect results for SUBSTRING when overflow

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 1.6.3, 2.0.2, 2.1.3, 2.2.3, 2.3.4, 2.4.6, 3.0.0
• Fix Version/s: 2.4.7, 3.0.1, 3.1.0
• Component/s: SQL
• Labels: correctness
• Target Version/s: 2.4.7, 3.0.1
• Assignee: Yuanjian Li
• Reporter: Yuanjian Li
• Created: 28/Jun/20 04:28
• Updated: 30/Jun/20 08:06
• Resolved: 28/Jun/20 19:30

Description

SQL query SELECT SUBSTRING("abc", -1207959552, -1207959552) incorrectly returns "abc" against expected output of "".
This is a result of integer overflow in addition https://github.com/apache/spark/blob/8c44d744631516a5cdaf63406e69a9dd11e5b878/common/unsafe/src/main/java/org/apache/spark/unsafe/types/UTF8String.java#L345

Issue Links

• links to: [Github] Pull Request #28937 (xuanyuanking)