[SPARK-9620] generated UnsafeProjection does not support many columns or large exressions

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Fix Version/s: 1.5.0
• Component/s: SQL
• Target Version/s: 1.5.0
• Assignee: Davies Liu
• Reporter: Davies Liu
• Created: 04/Aug/15 22:46
• Updated: 22/Apr/16 04:10
• Resolved: 10/Aug/15 20:52

Description

We put all the expressions in one function of UnsafeProjection, that could reach the 65k code size limit in JVM.

We should split them into multiple functions, like that we do for MutableProjection and SafeProjection.

Issue Links

• is related to: SPARK-14793 Code generation for large complex type exceeds JVM size limit.
• links to: [Github] Pull Request #8044 (davies)