[SPARK-14887] Generated SpecificUnsafeProjection Exceeds JVM Code Size Limits

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Incomplete
• Affects Version/s: 1.5.2
• Component/s: SQL
• Labels: bulk-closed
• Assignee: Davies Liu
• Reporter: fang fang chen
• Created: 25/Apr/16 05:59
• Updated: 21/May/19 04:12
• Resolved: 21/May/19 04:12

Description

Similiar issue with SPARK-14138 and SPARK-8443:
With large sql syntax(673K), following error happened:

Caused by: org.codehaus.janino.JaninoRuntimeException: Code of method "(Lorg/apache/spark/sql/catalyst/expressions/GeneratedClass$SpecificUnsafeProjection;Lorg/apache/spark/sql/catalyst/InternalRow;)V" of class "org.apache.spark.sql.catalyst.expressions.GeneratedClass$SpecificUnsafeProjection" grows beyond 64 KB
at org.codehaus.janino.CodeContext.makeSpace(CodeContext.java:941)
at org.codehaus.janino.CodeContext.write(CodeContext.java:854)

Issue Links

• is duplicated by: SPARK-16624 Generated SpecificColumnarIterator code can exceed JVM size limit for cached DataFrames
• is duplicated by: SPARK-17936 "CodeGenerator - failed to compile: org.codehaus.janino.JaninoRuntimeException: Code of" method Error