[SPARK-21393] spark (pyspark) crashes unpredictably when using show() or toPandas() - ASF JIRA

XML

Word

Printable

JSON

Details

Type: Bug
Status: Resolved
Priority: Major
Resolution: Duplicate
Affects Version/s: 2.1.1, 2.2.0
Fix Version/s: None
Component/s: PySpark
Labels:
None
Environment:

Windows 10
python 2.7

Description

unpredictbly run into this error either when using `pyspark.sql.DataFrame.show()` or `pyspark.sql.DataFrame.toPandas()`

error log starts with (truncated) :

17/07/12 16:03:09 ERROR CodeGenerator: failed to compile: org.codehaus.janino.JaninoRuntimeException: Code of method "apply_47$(Lorg/apache/spark/sql/catalyst/expressions/GeneratedClass$SpecificUnsafeProjection;Lorg/apache/spark/sql/catalyst/InternalRow;)V" of class "org.apache.spark.sql.catalyst.expressions.GeneratedClass$SpecificUnsafeProjection" grows beyond 64 KB
/* 001 */ public java.lang.Object generate(Object[] references) {
/* 002 */   return new SpecificUnsafeProjection(references);
/* 003 */ }
/* 004 */
/* 005 */ class SpecificUnsafeProjection extends org.apache.spark.sql.catalyst.expressions.UnsafeProjection {
/* 006 */
/* 007 */   private Object[] references;
/* 008 */   private scala.collection.immutable.Set set;
/* 009 */   private scala.collection.immutable.Set set1;
/* 010 */   private scala.collection.immutable.Set set2;
/* 011 */   private scala.collection.immutable.Set set3;
/* 012 */   private UTF8String.IntWrapper wrapper;
/* 013 */   private UTF8String.IntWrapper wrapper1;
/* 014 */   private scala.collection.immutable.Set set4;
/* 015 */   private UTF8String.IntWrapper wrapper2;
/* 016 */   private UTF8String.IntWrapper wrapper3;
/* 017 */   private scala.collection.immutable.Set set5;
/* 018 */   private scala.collection.immutable.Set set6;
/* 019 */   private scala.collection.immutable.Set set7;
/* 020 */   private UTF8String.IntWrapper wrapper4;
/* 021 */   private UTF8String.IntWrapper wrapper5;
/* 022 */   private scala.collection.immutable.Set set8;
/* 023 */   private UTF8String.IntWrapper wrapper6;
/* 024 */   private UTF8String.IntWrapper wrapper7;
/* 025 */   private scala.collection.immutable.Set set9;
/* 026 */   private scala.collection.immutable.Set set10;
/* 027 */   private scala.collection.immutable.Set set11;
/* 028 */   private UTF8String.IntWrapper wrapper8;
/* 029 */   private UTF8String.IntWrapper wrapper9;
/* 030 */   private scala.collection.immutable.Set set12;
/* 031 */   private UTF8String.IntWrapper wrapper10;
/* 032 */   private UTF8String.IntWrapper wrapper11;
/* 033 */   private scala.collection.immutable.Set set13;
/* 034 */   private scala.collection.immutable.Set set14;
/* 035 */   private scala.collection.immutable.Set set15;
/* 036 */   private UTF8String.IntWrapper wrapper12;
/* 037 */   private UTF8String.IntWrapper wrapper13;
/* 038 */   private scala.collection.immutable.Set set16;
/* 039 */   private UTF8String.IntWrapper wrapper14;
/* 040 */   private UTF8String.IntWrapper wrapper15;
/* 041 */   private scala.collection.immutable.Set set17;
/* 042 */   private scala.collection.immutable.Set set18;
/* 043 */   private scala.collection.immutable.Set set19;
/* 044 */   private UTF8String.IntWrapper wrapper16;
/* 045 */   private UTF8String.IntWrapper wrapper17;
/* 046 */   private scala.collection.immutable.Set set20;
/* 047 */   private UTF8String.IntWrapper wrapper18;
/* 048 */   private UTF8String.IntWrapper wrapper19;
/* 049 */   private scala.collection.immutable.Set set21;
/* 050 */   private scala.collection.immutable.Set set22;
/* 051 */   private scala.collection.immutable.Set set23;
/* 052 */   private UTF8String.IntWrapper wrapper20;
/* 053 */   private UTF8String.IntWrapper wrapper21;
/* 054 */   private scala.collection.immutable.Set set24;
/* 055 */   private UTF8String.IntWrapper wrapper22;
/* 056 */   private UTF8String.IntWrapper wrapper23;
/* 057 */   private scala.collection.immutable.Set set25;
/* 058 */   private scala.collection.immutable.Set set26;
/* 059 */   private scala.collection.immutable.Set set27;
/* 060 */   private UTF8String.IntWrapper wrapper24;
/* 061 */   private UTF8String.IntWrapper wrapper25;
/* 062 */   private scala.collection.immutable.Set set28;
/* 063 */   private UTF8String.IntWrapper wrapper26;
/* 064 */   private UTF8String.IntWrapper wrapper27;
/* 065 */   private scala.collection.immutable.Set set29;
/* 066 */   private scala.collection.immutable.Set set30;
/* 067 */   private scala.collection.immutable.Set set31;
/* 068 */   private UTF8String.IntWrapper wrapper28;
/* 069 */   private UTF8String.IntWrapper wrapper29;
/* 070 */   private scala.collection.immutable.Set set32;
/* 071 */   private UTF8String.IntWrapper wrapper30;
/* 072 */   private UTF8String.IntWrapper wrapper31;
/* 073 */   private scala.collection.immutable.Set set33;
/* 074 */   private scala.collection.immutable.Set set34;
/* 075 */   private scala.collection.immutable.Set set35;
/* 076 */   private scala.collection.immutable.Set set36;
/* 077 */   private UTF8String.IntWrapper wrapper32;
/* 078 */   private UTF8String.IntWrapper wrapper33;
/* 079 */   private scala.collection.immutable.Set set37;
/* 080 */   private UTF8String.IntWrapper wrapper34;
/* 081 */   private UTF8String.IntWrapper wrapper35;
/* 082 */   private scala.collection.immutable.Set set38;
/* 083 */   private scala.collection.immutable.Set set39;
/* 084 */   private scala.collection.immutable.Set set40;
/* 085 */   private UTF8String.IntWrapper wrapper36;
/* 086 */   private UTF8String.IntWrapper wrapper37;
/* 087 */   private scala.collection.immutable.Set set41;
/* 088 */   private UTF8String.IntWrapper wrapper38;
/* 089 */   private UTF8String.IntWrapper wrapper39;
/* 090 */   private scala.collection.immutable.Set set42;
/* 091 */   private scala.collection.immutable.Set set43;
/* 092 */   private scala.collection.immutable.Set set44;
/* 093 */   private UTF8String.IntWrapper wrapper40;
/* 094 */   private UTF8String.IntWrapper wrapper41;
/* 095 */   private scala.collection.immutable.Set set45;
/* 096 */   private UTF8String.IntWrapper wrapper42;
/* 097 */   private UTF8String.IntWrapper wrapper43;
/* 098 */   private scala.collection.immutable.Set set46;
/* 099 */   private scala.collection.immutable.Set set47;
/* 100 */   private scala.collection.immutable.Set set48;
/* 101 */   private UTF8String.IntWrapper wrapper44;
/* 102 */   private UTF8String.IntWrapper wrapper45;
/* 103 */   private scala.collection.immutable.Set set49;
/* 104 */   private UTF8String.IntWrapper wrapper46;
/* 105 */   private UTF8String.IntWrapper wrapper47;
/* 106 */   private scala.collection.immut

org.codehaus.janino.JaninoRuntimeException: Code of method "apply_47$(Lorg/apache/spark/sql/catalyst/expressions/GeneratedClass$SpecificUnsafeProjection;Lorg/apache/spark/sql/catalyst/InternalRow;)V" of class "org.apache.spark.sql.catalyst.expressions.GeneratedClass$SpecificUnsafeProjection" grows beyond 64 KB
	at org.codehaus.janino.CodeContext.makeSpace(CodeContext.java:949)
	at org.codehaus.janino.CodeContext.write(CodeContext.java:839)
	at org.codehaus.janino.UnitCompiler.writeOpcode(UnitCompiler.java:11081)
	at org.codehaus.janino.UnitCompiler.store(UnitCompiler.java:10770)
	at org.codehaus.janino.UnitCompiler.store(UnitCompiler.java:10753)
	at org.codehaus.janino.UnitCompiler.compileSet2(UnitCompiler.java:5629)
	at org.codehaus.janino.UnitCompiler.access$12700(UnitCompiler.java:206)
	at org.codehaus.janino.UnitCompiler$17.visitLocalVariableAccess(UnitCompiler.java:5619)
	at org.codehaus.janino.UnitCompiler$17.visitLocalVariableAccess(UnitCompiler.java:5611)
	at org.codehaus.janino.Java$LocalVariableAccess.accept(Java.java:3675)
	at org.codehaus.janino.UnitCompiler.compileSet(UnitCompiler.java:5611)
	at org.codehaus.janino.UnitCompiler.compileSet2(UnitCompiler.java:5625)
	at org.codehaus.janino.UnitCompiler.access$12200(UnitCompiler.java:206)
	at org.codehaus.janino.UnitCompiler$17.visitAmbiguousName(UnitCompiler.java:5614)
	at org.codehaus.janino.UnitCompiler$17.visitAmbiguousName(UnitCompiler.java:5611)
	at org.codehaus.janino.Java$AmbiguousName.accept(Java.java:3633)
	at org.codehaus.janino.UnitCompiler.compileSet(UnitCompiler.java:5611)
	at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:3193)
	at org.codehaus.janino.UnitCompiler.access$5100(UnitCompiler.java:206)
	at org.codehaus.janino.UnitCompiler$9.visitAssignment(UnitCompiler.java:3143)
	at org.codehaus.janino.UnitCompiler$9.visitAssignment(UnitCompiler.java:3139)
	at org.codehaus.janino.Java$Assignment.accept(Java.java:3847)
	at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:3139)
	at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:2112)
	at org.codehaus.janino.UnitCompiler.access$1700(UnitCompiler.java:206)
	at org.codehaus.janino.UnitCompiler$6.visitExpressionStatement(UnitCompiler.java:1377)
	at org.codehaus.janino.UnitCompiler$6.visitExpressionStatement(UnitCompiler.java:1370)
	at org.codehaus.janino.Java$ExpressionStatement.accept(Java.java:2558)
	at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:1370)
	at org.codehaus.janino.UnitCompiler.compileStatements(UnitCompiler.java:1450)
	at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:1436)
	at org.codehaus.janino.UnitCompiler.access$1600(UnitCompiler.java:206)
	at org.codehaus.janino.UnitCompiler$6.visitBlock(UnitCompiler.java:1376)
	at org.codehaus.janino.UnitCompiler$6.visitBlock(UnitCompiler.java:1370)
	at org.codehaus.janino.Java$Block.accept(Java.java:2471)
	at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:1370)
	at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:2220)
	at org.codehaus.janino.UnitCompiler.access$1800(UnitCompiler.java:206)
	at org.codehaus.janino.UnitCompiler$6.visitIfStatement(UnitCompiler.java:1378)
	at org.codehaus.janino.UnitCompiler$6.visitIfStatement(UnitCompiler.java:1370)
	at org.codehaus.janino.Java$IfStatement.accept(Java.java:2621)
	at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:1370)
	at org.codehaus.janino.UnitCompiler.compileStatements(UnitCompiler.java:1450)
	at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:1436)
	at org.codehaus.janino.UnitCompiler.access$1600(UnitCompiler.java:206)
	at org.codehaus.janino.UnitCompiler$6.visitBlock(UnitCompiler.java:1376)
	at org.codehaus.janino.UnitCompiler$6.visitBlock(UnitCompiler.java:1370)
	at org.codehaus.janino.Java$Block.accept(Java.java:2471)
	at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:1370)
	at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:2238)
	at org.codehaus.janino.UnitCompiler.access$1800(UnitCompiler.java:206)
	at org.codehaus.janino.UnitCompiler$6.visitIfStatement(UnitCompiler.java:1378)
	at org.codehaus.janino.UnitCompiler$6.visitIfStatement(UnitCompiler.java:1370)
	at org.codehaus.janino.Java$IfStatement.accept(Java.java:2621)
	at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:1370)
	at org.codehaus.janino.UnitCompiler.compileStatements(UnitCompiler.java:1450)
	at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:1436)
	at org.codehaus.janino.UnitCompiler.access$1600(UnitCompiler.java:206)
	at org.codehaus.janino.UnitCompiler$6.visitBlock(UnitCompiler.java:1376)
	at org.codehaus.janino.UnitCompiler$6.visitBlock(UnitCompiler.java:1370)
	at org.codehaus.janino.Java$Block.accept(Java.java:2471)
	at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:1370)
	at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:2238)
	at org.codehaus.janino.UnitCompiler.access$1800(UnitCompiler.java:206)
	at org.codehaus.janino.UnitCompiler$6.visitIfStatement(UnitCompiler.java:1378)
	at org.codehaus.janino.UnitCompiler$6.visitIfStatement(UnitCompiler.java:1370)
	at org.codehaus.janino.Java$IfStatement.accept(Java.java:2621)
	at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:1370)
	at org.codehaus.janino.UnitCompiler.compileStatements(UnitCompiler.java:1450)
	at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:1436)
	at org.codehaus.janino.UnitCompiler.access$1600(UnitCompiler.java:206)
	at org.codehaus.janino.UnitCompiler$6.visitBlock(UnitCompiler.java:1376)
	at org.codehaus.janino.UnitCompiler$6.visitBlock(UnitCompiler.java:1370)
	at org.codehaus.janino.Java$Block.accept(Java.java:2471)
	at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:1370)
	at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:2220)
	at org.codehaus.janino.UnitCompiler.access$1800(UnitCompiler.java:206)
	at org.codehaus.janino.UnitCompiler$6.visitIfStatement(UnitCompiler.java:1378)
	at org.codehaus.janino.UnitCompiler$6.visitIfStatement(UnitCompiler.java:1370)
	at org.codehaus.janino.Java$IfStatement.accept(Java.java:2621)
	at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:1370)
	at org.codehaus.janino.UnitCompiler.compileStatements(UnitCompiler.java:1450)
	at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:1436)
	at org.codehaus.janino.UnitCompiler.access$1600(UnitCompiler.java:206)
	at org.codehaus.janino.UnitCompiler$6.visitBlock(UnitCompiler.java:1376)
	at org.codehaus.janino.UnitCompiler$6.visitBlock(UnitCompiler.java:1370)
	at org.codehaus.janino.Java$Block.accept(Java.java:2471)
	at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:1370)
	at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:2238)
	at org.codehaus.janino.UnitCompiler.access$1800(UnitCompiler.java:206)
	at org.codehaus.janino.UnitCompiler$6.visitIfStatement(UnitCompiler.java:1378)
	at org.codehaus.janino.UnitCompiler$6.visitIfStatement(UnitCompiler.java:1370)
	at org.codehaus.janino.Java$IfStatement.accept(Java.java:2621)
	at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:1370)
	at org.codehaus.janino.UnitCompiler.compileStatements(UnitCompiler.java:1450)
	at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:1436)
	at org.codehaus.janino.UnitCompiler.access$1600(UnitCompiler.java:206)
	at org.codehaus.janino.UnitCompiler$6.visitBlock(UnitCompiler.java:1376)
	at org.codehaus.janino.UnitCompiler$6.visitBlock(UnitCompiler.java:1370)
	at org.codehaus.janino.Java$Block.accept(Java.java:2471)
	at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:1370)
	at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:2238)
	at org.codehaus.janino.UnitCompiler.access$1800(UnitCompiler.java:206)
	at org.codehaus.janino.UnitCompiler$6.visitIfStatement(UnitCompiler.java:1378)
	at org.codehaus.janino.UnitCompiler$6.visitIfStatement(UnitCompiler.java:1370)
	at org.codehaus.janino.Java$IfStatement.accept(Java.java:2621)
	at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:1370)
	at org.codehaus.janino.UnitCompiler.compileStatements(UnitCompiler.java:1450)
	at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:1436)
	at org.codehaus.janino.UnitCompiler.access$1600(UnitCompiler.java:206)
	at org.codehaus.janino.UnitCompiler$6.visitBlock(UnitCompiler.java:1376)
	at org.codehaus.janino.UnitCompiler$6.visitBlock(UnitCompiler.java:1370)
	at org.codehaus.janino.Java$Block.accept(Java.java:2471)
	at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:1370)
	at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:2220)
	at org.codehaus.janino.UnitCompiler.access$1800(UnitCompiler.java:206)
	at org.codehaus.janino.UnitCompiler$6.visitIfStatement(UnitCompiler.java:1378)
	at org.codehaus.janino.UnitCompiler$6.visitIfStatement(UnitCompiler.java:1370)
	at org.codehaus.janino.Java$IfStatement.accept(Java.java:2621)
	at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:1370)
	at org.codehaus.janino.UnitCompiler.compileStatements(UnitCompiler.java:1450)
	at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:1436)
	at org.codehaus.janino.UnitCompiler.access$1600(UnitCompiler.java:206)
	at org.codehaus.janino.UnitCompiler$6.visitBlock(UnitCompiler.java:1376)
	at org.codehaus.janino.UnitCompiler$6.visitBlock(UnitCompiler.java:1370)
	at org.codehaus.janino.Java$Block.accept(Java.java:2471)
	at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:1370)
	at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:2220)
	at org.codehaus.janino.UnitCompiler.access$1800(UnitCompiler.java:206)
	at org.codehaus.janino.UnitCompiler$6.visitIfStatement(UnitCompiler.java:1378)
	at org.codehaus.janino.UnitCompiler$6.visitIfStatement(UnitCompiler.java:1370)
	at org.codehaus.janino.Java$IfStatement.accept(Java.java:2621)
	at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:1370)
	at org.codehaus.janino.UnitCompiler.compileStatements(UnitCompiler.java:1450)
	at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:1436)
	at org.codehaus.janino.UnitCompiler.access$1600(UnitCompiler.java:206)
	at org.codehaus.janino.UnitCompiler$6.visitBlock(UnitCompiler.java:1376)
	at org.codehaus.janino.UnitCompiler$6.visitBlock(UnitCompiler.java:1370)
	at org.codehaus.janino.Java$Block.accept(Java.java:2471)
	at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:1370)
	at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:2238)
	at org.codehaus.janino.UnitCompiler.access$1800(UnitCompiler.java:206)
	at org.codehaus.janino.UnitCompiler$6.visitIfStatement(UnitCompiler.java:1378)
	at org.codehaus.janino.UnitCompiler$6.visitIfStatement(UnitCompiler.java:1370)
	at org.codehaus.janino.Java$IfStatement.accept(Java.java:2621)
	at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:1370)
	at org.codehaus.janino.UnitCompiler.compileStatements(UnitCompiler.java:1450)
	at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:1436)
	at org.codehaus.janino.UnitCompiler.access$1600(UnitCompiler.java:206)
	at org.codehaus.janino.UnitCompiler$6.visitBlock(UnitCompiler.java:1376)
	at org.codehaus.janino.UnitCompiler$6.visitBlock(UnitCompiler.java:1370)
	at org.codehaus.janino.Java$Block.accept(Java.java:2471)
	at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:1370)
	at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:2238)
	at org.codehaus.janino.UnitCompiler.access$1800(UnitCompiler.java:206)
	at org.codehaus.janino.UnitCompiler$6.visitIfStatement(UnitCompiler.java:1378)
	at org.codehaus.janino.UnitCompiler$6.visitIfStatement(UnitCompiler.java:1370)
	at org.codehaus.janino.Java$IfStatement.accept(Java.java:2621)
	at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:1370)
	at org.codehaus.janino.UnitCompiler.compileStatements(UnitCompiler.java:1450)
	at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:1436)
	at org.codehaus.janino.UnitCompiler.access$1600(UnitCompiler.java:206)
	at org.codehaus.janino.UnitCompiler$6.visitBlock(UnitCompiler.java:1376)
	at org.codehaus.janino.UnitCompiler$6.visitBlock(UnitCompiler.java:1370)
	at org.codehaus.janino.Java$Block.accept(Java.java:2471)
	at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:1370)
	at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:2238)
	at org.codehaus.janino.UnitCompiler.access$1800(UnitCompiler.java:206)
	at org.codehaus.janino.UnitCompiler$6.visitIfStatement(UnitCompiler.java:1378)
	at org.codehaus.janino.UnitCompiler$6.visitIfStatement(UnitCompiler.java:1370)
	at org.codehaus.janino.Java$IfStatement.accept(Java.java:2621)
	at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:1370)
	at org.codehaus.janino.UnitCompiler.compileStatements(UnitCompiler.java:1450)
	at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:2811)
	at org.codehaus.janino.UnitCompiler.compileDeclaredMethods(UnitCompiler.java:1262)
	at org.codehaus.janino.UnitCompiler.compileDeclaredMethods(UnitCompiler.java:1234)
	at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:538)
	at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:890)
	at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:894)
	at org.codehaus.janino.UnitCompiler.access$600(UnitCompiler.java:206)
	at org.codehaus.janino.UnitCompiler$2.visitMemberClassDeclaration(UnitCompiler.java:377)
	at org.codehaus.janino.UnitCompiler$2.visitMemberClassDeclaration(UnitCompiler.java:369)
	at org.codehaus.janino.Java$MemberClassDeclaration.accept(Java.java:1128)
	at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:369)
	at org.codehaus.janino.UnitCompiler.compileDeclaredMemberTypes(UnitCompiler.java:1209)
	at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:564)
	at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:420)
	at org.codehaus.janino.UnitCompiler.access$400(UnitCompiler.java:206)
	at org.codehaus.janino.UnitCompiler$2.visitPackageMemberClassDeclaration(UnitCompiler.java:374)
	at org.codehaus.janino.UnitCompiler$2.visitPackageMemberClassDeclaration(UnitCompiler.java:369)
	at org.codehaus.janino.Java$AbstractPackageMemberClassDeclaration.accept(Java.java:1309)
	at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:369)
	at org.codehaus.janino.UnitCompiler.compileUnit(UnitCompiler.java:345)
	at org.codehaus.janino.SimpleCompiler.compileToClassLoader(SimpleCompiler.java:396)
	at org.codehaus.janino.ClassBodyEvaluator.compileToClass(ClassBodyEvaluator.java:311)
	at org.codehaus.janino.ClassBodyEvaluator.cook(ClassBodyEvaluator.java:229)
	at org.codehaus.janino.SimpleCompiler.cook(SimpleCompiler.java:196)
	at org.codehaus.commons.compiler.Cookable.cook(Cookable.java:91)
	at org.apache.spark.sql.catalyst.expressions.codegen.CodeGenerator$.org$apache$spark$sql$catalyst$expressions$codegen$CodeGenerator$$doCompile(CodeGenerator.scala:960)
	at org.apache.spark.sql.catalyst.expressions.codegen.CodeGenerator$$anon$1.load(CodeGenerator.scala:1027)
	at org.apache.spark.sql.catalyst.expressions.codegen.CodeGenerator$$anon$1.load(CodeGenerator.scala:1024)
	at org.spark_project.guava.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3599)
	at org.spark_project.guava.cache.LocalCache$Segment.loadSync(LocalCache.java:2379)
	at org.spark_project.guava.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2342)
	at org.spark_project.guava.cache.LocalCache$Segment.get(LocalCache.java:2257)
	at org.spark_project.guava.cache.LocalCache.get(LocalCache.java:4000)
	at org.spark_project.guava.cache.LocalCache.getOrLoad(LocalCache.java:4004)
	at org.spark_project.guava.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4874)
	at org.apache.spark.sql.catalyst.expressions.codegen.CodeGenerator$.compile(CodeGenerator.scala:906)
	at org.apache.spark.sql.catalyst.expressions.codegen.GenerateUnsafeProjection$.create(GenerateUnsafeProjection.scala:412)
	at org.apache.spark.sql.catalyst.expressions.codegen.GenerateUnsafeProjection$.generate(GenerateUnsafeProjection.scala:362)
	at org.apache.spark.sql.catalyst.expressions.UnsafeProjection$.create(Projection.scala:155)
	at org.apache.spark.sql.execution.ProjectExec$$anonfun$9.apply(basicPhysicalOperators.scala:74)
	at org.apache.spark.sql.execution.ProjectExec$$anonfun$9.apply(basicPhysicalOperators.scala:73)
	at org.apache.spark.rdd.RDD$$anonfun$mapPartitionsWithIndexInternal$1$$anonfun$apply$24.apply(RDD.scala:815)
	at org.apache.spark.rdd.RDD$$anonfun$mapPartitionsWithIndexInternal$1$$anonfun$apply$24.apply(RDD.scala:815)
	at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
	at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:323)
	at org.apache.spark.rdd.RDD.iterator(RDD.scala:287)
	at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
	at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:323)
	at org.apache.spark.rdd.RDD.iterator(RDD.scala:287)
	at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:87)
	at org.apache.spark.scheduler.Task.run(Task.scala:108)
	at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:335)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)
17/07/12 16:03:09 ERROR Executor: Exception in task 0.0 in stage 2.0 (TID 2)
org.codehaus.janino.JaninoRuntimeException: failed to compile: org.codehaus.janino.JaninoRuntimeException: Code of method "apply_47$(Lorg/apache/spark/sql/catalyst/expressions/GeneratedClass$SpecificUnsafeProjection;Lorg/apache/spark/sql/catalyst/InternalRow;)V" of class "org.apache.spark.sql.catalyst.expressions.GeneratedClass$SpecificUnsafeProjection" grows beyond 64 KB

and there is bunch of lines .... like :

/* 001 */ public java.lang.Object generate(Object[] references) {
/* 002 */   return new SpecificUnsafeProjection(references);
/* 003 */ }
/* 004 */
/* 005 */ class SpecificUnsafeProjection extends org.apache.spark.sql.catalyst.expressions.UnsafeProjection {
/* 006 */
/* 007 */   private Object[] references;
/* 008 */   private scala.collection.immutable.Set set;
/* 009 */   private scala.collection.immutable.Set set1;
/* 010 */   private scala.collection.immutable.Set set2;
/* 011 */   private scala.collection.immutable.Set set3;
/* 012 */   private UTF8String.IntWrapper wrapper;
/* 013 */   private UTF8String.IntWrapper wrapper1;
/* 014 */   private scala.collection.immutable.Set set4;
/* 015 */   private UTF8String.IntWrapper wrapper2;
/* 016 */   private UTF8String.IntWrapper wrapper3;
/* 017 */   private scala.collection.immutable.Set set5;
/* 018 */   private scala.collection.immutable.Set set6;
/* 019 */   private scala.collection.immutable.Set set7;
/* 020 */   private UTF8String.IntWrapper wrapper4;

at the end ...

	at org.apache.spark.sql.catalyst.expressions.codegen.CodeGenerator$.org$apache$spark$sql$catalyst$expressions$codegen$CodeGenerator$$doCompile(CodeGenerator.scala:966)

	at org.apache.spark.sql.catalyst.expressions.codegen.CodeGenerator$$anon$1.load(CodeGenerator.scala:1027)

	at org.apache.spark.sql.catalyst.expressions.codegen.CodeGenerator$$anon$1.load(CodeGenerator.scala:1024)

	at org.spark_project.guava.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3599)

	at org.spark_project.guava.cache.LocalCache$Segment.loadSync(LocalCache.java:2379)

	at org.spark_project.guava.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2342)

	at org.spark_project.guava.cache.LocalCache$Segment.get(LocalCache.java:2257)

	at org.spark_project.guava.cache.LocalCache.get(LocalCache.java:4000)

	at org.spark_project.guava.cache.LocalCache.getOrLoad(LocalCache.java:4004)

	at org.spark_project.guava.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4874)

	at org.apache.spark.sql.catalyst.expressions.codegen.CodeGenerator$.compile(CodeGenerator.scala:906)

	at org.apache.spark.sql.catalyst.expressions.codegen.GenerateUnsafeProjection$.create(GenerateUnsafeProjection.scala:412)

	at org.apache.spark.sql.catalyst.expressions.codegen.GenerateUnsafeProjection$.generate(GenerateUnsafeProjection.scala:362)

	at org.apache.spark.sql.catalyst.expressions.UnsafeProjection$.create(Projection.scala:155)

	at org.apache.spark.sql.execution.ProjectExec$$anonfun$9.apply(basicPhysicalOperators.scala:74)

	at org.apache.spark.sql.execution.ProjectExec$$anonfun$9.apply(basicPhysicalOperators.scala:73)

	at org.apache.spark.rdd.RDD$$anonfun$mapPartitionsWithIndexInternal$1$$anonfun$apply$24.apply(RDD.scala:815)

	at org.apache.spark.rdd.RDD$$anonfun$mapPartitionsWithIndexInternal$1$$anonfun$apply$24.apply(RDD.scala:815)

	at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)

	at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:323)

	at org.apache.spark.rdd.RDD.iterator(RDD.scala:287)

	at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)

	at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:323)

	at org.apache.spark.rdd.RDD.iterator(RDD.scala:287)

	at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:87)

	at org.apache.spark.scheduler.Task.run(Task.scala:108)

	at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:335)

	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)

	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)

	... 1 more

Caused by: org.codehaus.janino.JaninoRuntimeException: Code of method "apply_47$(Lorg/apache/spark/sql/catalyst/expressions/GeneratedClass$SpecificUnsafeProjection;Lorg/apache/spark/sql/catalyst/InternalRow;)V" of class "org.apache.spark.sql.catalyst.expressions.GeneratedClass$SpecificUnsafeProjection" grows beyond 64 KB

	at org.codehaus.janino.CodeContext.makeSpace(CodeContext.java:949)

	at org.codehaus.janino.CodeContext.write(CodeContext.java:839)

	at org.codehaus.janino.UnitCompiler.writeOpcode(UnitCompiler.java:11081)

	at org.codehaus.janino.UnitCompiler.store(UnitCompiler.java:10770)

	at org.codehaus.janino.UnitCompiler.store(UnitCompiler.java:10753)

	at org.codehaus.janino.UnitCompiler.compileSet2(UnitCompiler.java:5629)

	at org.codehaus.janino.UnitCompiler.access$12700(UnitCompiler.java:206)

	at org.codehaus.janino.UnitCompiler$17.visitLocalVariableAccess(UnitCompiler.java:5619)

	at org.codehaus.janino.UnitCompiler$17.visitLocalVariableAccess(UnitCompiler.java:5611)

	at org.codehaus.janino.Java$LocalVariableAccess.accept(Java.java:3675)

	at org.codehaus.janino.UnitCompiler.compileSet(UnitCompiler.java:5611)

	at org.codehaus.janino.UnitCompiler.compileSet2(UnitCompiler.java:5625)

	at org.codehaus.janino.UnitCompiler.access$12200(UnitCompiler.java:206)

	at org.codehaus.janino.UnitCompiler$17.visitAmbiguousName(UnitCompiler.java:5614)

	at org.codehaus.janino.UnitCompiler$17.visitAmbiguousName(UnitCompiler.java:5611)

	at org.codehaus.janino.Java$AmbiguousName.accept(Java.java:3633)

	at org.codehaus.janino.UnitCompiler.compileSet(UnitCompiler.java:5611)

	at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:3193)

	at org.codehaus.janino.UnitCompiler.access$5100(UnitCompiler.java:206)

	at org.codehaus.janino.UnitCompiler$9.visitAssignment(UnitCompiler.java:3143)

	at org.codehaus.janino.UnitCompiler$9.visitAssignment(UnitCompiler.java:3139)

	at org.codehaus.janino.Java$Assignment.accept(Java.java:3847)

	at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:3139)

	at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:2112)

	at org.codehaus.janino.UnitCompiler.access$1700(UnitCompiler.java:206)

	at org.codehaus.janino.UnitCompiler$6.visitExpressionStatement(UnitCompiler.java:1377)

	at org.codehaus.janino.UnitCompiler$6.visitExpressionStatement(UnitCompiler.java:1370)

	at org.codehaus.janino.Java$ExpressionStatement.accept(Java.java:2558)

	at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:1370)

	at org.codehaus.janino.UnitCompiler.compileStatements(UnitCompiler.java:1450)

	at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:1436)

	at org.codehaus.janino.UnitCompiler.access$1600(UnitCompiler.java:206)

	at org.codehaus.janino.UnitCompiler$6.visitBlock(UnitCompiler.java:1376)

	at org.codehaus.janino.UnitCompiler$6.visitBlock(UnitCompiler.java:1370)

	at org.codehaus.janino.Java$Block.accept(Java.java:2471)

	at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:1370)

	at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:2220)

	at org.codehaus.janino.UnitCompiler.access$1800(UnitCompiler.java:206)

	at org.codehaus.janino.UnitCompiler$6.visitIfStatement(UnitCompiler.java:1378)

	at org.codehaus.janino.UnitCompiler$6.visitIfStatement(UnitCompiler.java:1370)

	at org.codehaus.janino.Java$IfStatement.accept(Java.java:2621)

	at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:1370)

	at org.codehaus.janino.UnitCompiler.compileStatements(UnitCompiler.java:1450)

	at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:1436)

	at org.codehaus.janino.UnitCompiler.access$1600(UnitCompiler.java:206)

	at org.codehaus.janino.UnitCompiler$6.visitBlock(UnitCompiler.java:1376)

	at org.codehaus.janino.UnitCompiler$6.visitBlock(UnitCompiler.java:1370)

	at org.codehaus.janino.Java$Block.accept(Java.java:2471)

	at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:1370)

	at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:2238)

	at org.codehaus.janino.UnitCompiler.access$1800(UnitCompiler.java:206)

	at org.codehaus.janino.UnitCompiler$6.visitIfStatement(UnitCompiler.java:1378)

	at org.codehaus.janino.UnitCompiler$6.visitIfStatement(UnitCompiler.java:1370)

	at org.codehaus.janino.Java$IfStatement.accept(Java.java:2621)

	at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:1370)

	at org.codehaus.janino.UnitCompiler.compileStatements(UnitCompiler.java:1450)

	at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:1436)

	at org.codehaus.janino.UnitCompiler.access$1600(UnitCompiler.java:206)

	at org.codehaus.janino.UnitCompiler$6.visitBlock(UnitCompiler.java:1376)

	at org.codehaus.janino.UnitCompiler$6.visitBlock(UnitCompiler.java:1370)

	at org.codehaus.janino.Java$Block.accept(Java.java:2471)

	at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:1370)

	at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:2238)

	at org.codehaus.janino.UnitCompiler.access$1800(UnitCompiler.java:206)

	at org.codehaus.janino.UnitCompiler$6.visitIfStatement(UnitCompiler.java:1378)

	at org.codehaus.janino.UnitCompiler$6.visitIfStatement(UnitCompiler.java:1370)

	at org.codehaus.janino.Java$IfStatement.accept(Java.java:2621)

	at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:1370)

	at org.codehaus.janino.UnitCompiler.compileStatements(UnitCompiler.java:1450)

	at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:1436)

	at org.codehaus.janino.UnitCompiler.access$1600(UnitCompiler.java:206)

	at org.codehaus.janino.UnitCompiler$6.visitBlock(UnitCompiler.java:1376)

	at org.codehaus.janino.UnitCompiler$6.visitBlock(UnitCompiler.java:1370)

	at org.codehaus.janino.Java$Block.accept(Java.java:2471)

	at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:1370)

	at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:2220)

	at org.codehaus.janino.UnitCompiler.access$1800(UnitCompiler.java:206)

	at org.codehaus.janino.UnitCompiler$6.visitIfStatement(UnitCompiler.java:1378)

	at org.codehaus.janino.UnitCompiler$6.visitIfStatement(UnitCompiler.java:1370)

	at org.codehaus.janino.Java$IfStatement.accept(Java.java:2621)

	at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:1370)

	at org.codehaus.janino.UnitCompiler.compileStatements(UnitCompiler.java:1450)

	at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:1436)

	at org.codehaus.janino.UnitCompiler.access$1600(UnitCompiler.java:206)

	at org.codehaus.janino.UnitCompiler$6.visitBlock(UnitCompiler.java:1376)

	at org.codehaus.janino.UnitCompiler$6.visitBlock(UnitCompiler.java:1370)

	at org.codehaus.janino.Java$Block.accept(Java.java:2471)

	at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:1370)

	at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:2238)

	at org.codehaus.janino.UnitCompiler.access$1800(UnitCompiler.java:206)

	at org.codehaus.janino.UnitCompiler$6.visitIfStatement(UnitCompiler.java:1378)

	at org.codehaus.janino.UnitCompiler$6.visitIfStatement(UnitCompiler.java:1370)

	at org.codehaus.janino.Java$IfStatement.accept(Java.java:2621)

	at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:1370)

	at org.codehaus.janino.UnitCompiler.compileStatements(UnitCompiler.java:1450)

	at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:1436)

	at org.codehaus.janino.UnitCompiler.access$1600(UnitCompiler.java:206)

	at org.codehaus.janino.UnitCompiler$6.visitBlock(UnitCompiler.java:1376)

	at org.codehaus.janino.UnitCompiler$6.visitBlock(UnitCompiler.java:1370)

	at org.codehaus.janino.Java$Block.accept(Java.java:2471)

	at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:1370)

	at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:2238)

	at org.codehaus.janino.UnitCompiler.access$1800(UnitCompiler.java:206)

	at org.codehaus.janino.UnitCompiler$6.visitIfStatement(UnitCompiler.java:1378)

	at org.codehaus.janino.UnitCompiler$6.visitIfStatement(UnitCompiler.java:1370)

	at org.codehaus.janino.Java$IfStatement.accept(Java.java:2621)

	at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:1370)

	at org.codehaus.janino.UnitCompiler.compileStatements(UnitCompiler.java:1450)

	at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:1436)

	at org.codehaus.janino.UnitCompiler.access$1600(UnitCompiler.java:206)

	at org.codehaus.janino.UnitCompiler$6.visitBlock(UnitCompiler.java:1376)

	at org.codehaus.janino.UnitCompiler$6.visitBlock(UnitCompiler.java:1370)

	at org.codehaus.janino.Java$Block.accept(Java.java:2471)

	at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:1370)

	at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:2220)

	at org.codehaus.janino.UnitCompiler.access$1800(UnitCompiler.java:206)

	at org.codehaus.janino.UnitCompiler$6.visitIfStatement(UnitCompiler.java:1378)

	at org.codehaus.janino.UnitCompiler$6.visitIfStatement(UnitCompiler.java:1370)

	at org.codehaus.janino.Java$IfStatement.accept(Java.java:2621)

	at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:1370)

	at org.codehaus.janino.UnitCompiler.compileStatements(UnitCompiler.java:1450)

	at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:1436)

	at org.codehaus.janino.UnitCompiler.access$1600(UnitCompiler.java:206)

	at org.codehaus.janino.UnitCompiler$6.visitBlock(UnitCompiler.java:1376)

	at org.codehaus.janino.UnitCompiler$6.visitBlock(UnitCompiler.java:1370)

	at org.codehaus.janino.Java$Block.accept(Java.java:2471)

	at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:1370)

	at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:2220)

	at org.codehaus.janino.UnitCompiler.access$1800(UnitCompiler.java:206)

	at org.codehaus.janino.UnitCompiler$6.visitIfStatement(UnitCompiler.java:1378)

	at org.codehaus.janino.UnitCompiler$6.visitIfStatement(UnitCompiler.java:1370)

	at org.codehaus.janino.Java$IfStatement.accept(Java.java:2621)

	at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:1370)

	at org.codehaus.janino.UnitCompiler.compileStatements(UnitCompiler.java:1450)

	at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:1436)

	at org.codehaus.janino.UnitCompiler.access$1600(UnitCompiler.java:206)

	at org.codehaus.janino.UnitCompiler$6.visitBlock(UnitCompiler.java:1376)

	at org.codehaus.janino.UnitCompiler$6.visitBlock(UnitCompiler.java:1370)

	at org.codehaus.janino.Java$Block.accept(Java.java:2471)

	at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:1370)

	at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:2238)

	at org.codehaus.janino.UnitCompiler.access$1800(UnitCompiler.java:206)

	at org.codehaus.janino.UnitCompiler$6.visitIfStatement(UnitCompiler.java:1378)

	at org.codehaus.janino.UnitCompiler$6.visitIfStatement(UnitCompiler.java:1370)

	at org.codehaus.janino.Java$IfStatement.accept(Java.java:2621)

	at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:1370)

	at org.codehaus.janino.UnitCompiler.compileStatements(UnitCompiler.java:1450)

	at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:1436)

	at org.codehaus.janino.UnitCompiler.access$1600(UnitCompiler.java:206)

	at org.codehaus.janino.UnitCompiler$6.visitBlock(UnitCompiler.java:1376)

	at org.codehaus.janino.UnitCompiler$6.visitBlock(UnitCompiler.java:1370)

	at org.codehaus.janino.Java$Block.accept(Java.java:2471)

	at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:1370)

	at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:2238)

	at org.codehaus.janino.UnitCompiler.access$1800(UnitCompiler.java:206)

	at org.codehaus.janino.UnitCompiler$6.visitIfStatement(UnitCompiler.java:1378)

	at org.codehaus.janino.UnitCompiler$6.visitIfStatement(UnitCompiler.java:1370)

	at org.codehaus.janino.Java$IfStatement.accept(Java.java:2621)

	at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:1370)

	at org.codehaus.janino.UnitCompiler.compileStatements(UnitCompiler.java:1450)

	at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:1436)

	at org.codehaus.janino.UnitCompiler.access$1600(UnitCompiler.java:206)

	at org.codehaus.janino.UnitCompiler$6.visitBlock(UnitCompiler.java:1376)

	at org.codehaus.janino.UnitCompiler$6.visitBlock(UnitCompiler.java:1370)

	at org.codehaus.janino.Java$Block.accept(Java.java:2471)

	at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:1370)

	at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:2238)

	at org.codehaus.janino.UnitCompiler.access$1800(UnitCompiler.java:206)

	at org.codehaus.janino.UnitCompiler$6.visitIfStatement(UnitCompiler.java:1378)

	at org.codehaus.janino.UnitCompiler$6.visitIfStatement(UnitCompiler.java:1370)

	at org.codehaus.janino.Java$IfStatement.accept(Java.java:2621)

	at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:1370)

	at org.codehaus.janino.UnitCompiler.compileStatements(UnitCompiler.java:1450)

	at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:2811)

	at org.codehaus.janino.UnitCompiler.compileDeclaredMethods(UnitCompiler.java:1262)

	at org.codehaus.janino.UnitCompiler.compileDeclaredMethods(UnitCompiler.java:1234)

	at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:538)

	at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:890)

	at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:894)

	at org.codehaus.janino.UnitCompiler.access$600(UnitCompiler.java:206)

	at org.codehaus.janino.UnitCompiler$2.visitMemberClassDeclaration(UnitCompiler.java:377)

	at org.codehaus.janino.UnitCompiler$2.visitMemberClassDeclaration(UnitCompiler.java:369)

	at org.codehaus.janino.Java$MemberClassDeclaration.accept(Java.java:1128)

	at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:369)

	at org.codehaus.janino.UnitCompiler.compileDeclaredMemberTypes(UnitCompiler.java:1209)

	at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:564)

	at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:420)

	at org.codehaus.janino.UnitCompiler.access$400(UnitCompiler.java:206)

	at org.codehaus.janino.UnitCompiler$2.visitPackageMemberClassDeclaration(UnitCompiler.java:374)

	at org.codehaus.janino.UnitCompiler$2.visitPackageMemberClassDeclaration(UnitCompiler.java:369)

	at org.codehaus.janino.Java$AbstractPackageMemberClassDeclaration.accept(Java.java:1309)

	at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:369)

	at org.codehaus.janino.UnitCompiler.compileUnit(UnitCompiler.java:345)

	at org.codehaus.janino.SimpleCompiler.compileToClassLoader(SimpleCompiler.java:396)

	at org.codehaus.janino.ClassBodyEvaluator.compileToClass(ClassBodyEvaluator.java:311)

	at org.codehaus.janino.ClassBodyEvaluator.cook(ClassBodyEvaluator.java:229)

	at org.codehaus.janino.SimpleCompiler.cook(SimpleCompiler.java:196)

	at org.codehaus.commons.compiler.Cookable.cook(Cookable.java:91)

	at org.apache.spark.sql.catalyst.expressions.codegen.CodeGenerator$.org$apache$spark$sql$catalyst$expressions$codegen$CodeGenerator$$doCompile(CodeGenerator.scala:960)

	... 29 more

Attachments

- Sort By Name
- Sort By Date
- Ascending
- Descending

Data.zip
13/Jul/17 17:12
2.28 MB
Zahra
working_ST_pyspark.py
13/Jul/17 13:41
8 kB
Zahra

Issue Links

duplicates

SPARK-21413 Multiple projections with CASE WHEN fails to run generated codes

Resolved

Activity

People

Assignee:: Unassigned

Reporter:: Zahra

Votes:: 0 Vote for this issue

Watchers:: 3 Start watching this issue

Dates

Created:: 12/Jul/17 20:47

Updated:: 12/Dec/22 18:11

Resolved:: 14/Jul/17 04:54