[SPARK-13242] Moderately complex `when` expression causes code generation failure

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.6.0
• Fix Version/s: 1.6.2, 2.0.0
• Component/s: SQL
• Assignee: Davies Liu
• Reporter: Joe Halliwell
• Created: 09/Feb/16 15:54
• Updated: 14/Nov/17 22:27
• Resolved: 09/Mar/16 20:05

Description

Moderately complex `when` expressions produce generated code that busts the 64KB method limit. This causes code generation to fail.

Here's a test case exhibiting the problem: https://github.com/joehalliwell/spark/commit/4dbdf6e15d1116b8e1eb44822fd29ead9b7d817d

I'm interested in working on a fix. I'm thinking it may be possible to split the expressions along the lines of SPARK-8443, but any pointers would be welcome!

Issue Links

• is related to: SPARK-22520 Support code generation also for complex CASE WHEN
• links to: [Github] Pull Request #11221 (joehalliwell)
• links to: [Github] Pull Request #11243 (joehalliwell)
• links to: [Github] Pull Request #11592 (davies)
• links to: [Github] Pull Request #11606 (davies)