[SPARK-22520] Support code generation also for complex CASE WHEN

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 2.3.0
• Fix Version/s: 2.3.0
• Component/s: SQL
• Assignee: Marco Gaido
• Reporter: Marco Gaido
• Created: 14/Nov/17 22:26
• Updated: 02/Jan/18 00:20
• Resolved: 27/Nov/17 23:46

Description

Code generation is disabled for CaseWhen when the number of branches is higher than spark.sql.codegen.maxCaseBranches (which defaults to 20). This was done in SPARK-13242 to prevent the well known 64KB method limit exception.

This tickets proposes to support code generation also in those cases (without causing exceptions of course). As a side effect, we could get rid of the spark.sql.codegen.maxCaseBranches configuration.

Issue Links

• is duplicated by: SPARK-21413 Multiple projections with CASE WHEN fails to run generated codes
• relates to: SPARK-13242 Moderately complex `when` expression causes code generation failure
• relates to: SPARK-14577 spark.sql.codegen.maxCaseBranches config option
• links to: [Github] Pull Request #19752 (mgaido91)
• links to: [Github] Pull Request #19830 (cloud-fan)