[SPARK-23976] UTF8String.concat() or ByteArray.concat() may allocate shorter structure.

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 2.3.0
• Fix Version/s: 2.4.0
• Component/s: Spark Core
• Assignee: Kazuaki Ishizaki
• Reporter: Kazuaki Ishizaki
• Created: 13/Apr/18 11:57
• Updated: 19/Apr/18 16:08
• Resolved: 19/Apr/18 16:08

Description

When the three inputs has `0x7FFF_FF00`, `0x7FFF_FF00`, and `0xE00`, the current algorithm allocate the result structure with 0x1000 length due to integer sum overflow.

We should detect overflow.

Issue Links

• links to: [Github] Pull Request #21064 (kiszk)