[SPARK-29100] Codegen with switch in InSet expression causes compilation error

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.0.0
• Fix Version/s: 3.0.0
• Component/s: SQL
• Assignee: L. C. Hsieh
• Reporter: L. C. Hsieh
• Created: 16/Sep/19 18:47
• Updated: 17/Sep/19 04:02
• Resolved: 17/Sep/19 03:06

Description

SPARK-26205 adds an optimization to InSet that generates Java switch condition for certain cases. When the given set is empty, it is possibly that codegen causes compilation error:

 

[info] - SPARK-29100: InSet with empty input set *** FAILED *** (58 milliseconds)                                      

[info]   Code generation of input[0, int, true] INSET () failed:                                                                        

[info]   org.codehaus.janino.InternalCompilerException: failed to compile: org.codehaus.janino.InternalCompilerException: Compiling "GeneratedClass" in "generated.java": Compiling "apply(java.lang.Object _i)"; apply(java.lang.Object _i): Operand stack inconsistent at offset 45: Previous size 0, now 1                                                                                                           

[info]   org.codehaus.janino.InternalCompilerException: failed to compile: org.codehaus.janino.InternalCompilerException: Compiling "GeneratedClass" in "generated.java": Compiling "apply(java.lang.Object _i)"; apply(java.lang.Object _i): Operand stack inconsistent at offset 45: Previous size 0, now 1                                                                                                           

[info]         at org.apache.spark.sql.catalyst.expressions.codegen.CodeGenerator$.org$apache$spark$sql$catalyst$expressions$codegen$CodeGenerator$$doCompile(CodeGenerator.scala:1308)                                                                                        

[info]         at org.apache.spark.sql.catalyst.expressions.codegen.CodeGenerator$$anon$1.load(CodeGenerator.scala:1386)               

[info]         at org.apache.spark.sql.catalyst.expressions.codegen.CodeGenerator$$anon$1.load(CodeGenerator.scala:1383)

Issue Links

• is caused by: SPARK-26205 Optimize InSet expression for bytes, shorts, ints, dates
• links to: GitHub Pull Request #25806