[CALCITE-3094] Code of method grows beyond 64 KB when joining two tables with many fields

• Type: Bug
• Status: Closed
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 1.19.0
• Fix Version/s: 1.38.0
• Component/s: core, jdbc-adapter
• Labels: pull-request-available
• Assignee: James Duong
• Reporter: Pablo
• Created: 27/May/19 15:53
• Updated: 17/Oct/24 22:41
• Resolved: 17/Oct/24 22:41

Description

I am running simple queries like these:

SELECT * FROM t0 INNER JOIN t1 ON t0.`f000000` = t1.`f000003`"

 

They are running ok, but, when t1 and t0 have about 3290 fields, I get this error. 

Is there anything I can do? I can't do joins with this amount of fields?

 

Thank you in advance

 

Caused by: java.lang.RuntimeException: Error while compiling generated Java code:
public org.apache.calcite.linq4j.Enumerable bind(final org.apache.calcite.DataContext root) {
final org.apache.calcite.rel.RelNode v2stashed = (org.apache.calcite.rel.RelNode) root.get("v2stashed");
final org.apache.calcite.rel.RelNode v1stashed = (org.apache.calcite.rel.RelNode) root.get("v1stashed");
final org.apache.calcite.interpreter.Interpreter interpreter = new org.apache.calcite.interpreter.Interpreter(
root,
v1stashed);
final org.apache.calcite.interpreter.Interpreter interpreter0 = new org.apache.calcite.interpreter.Interpreter(
root,
v2stashed);
return interpreter.join(interpreter0, new org.apache.calcite.linq4j.function.Function1() {
public Byte apply(Object[] v1) {
return (Byte) v1[0];
}
public Object apply(Object v1) {
return apply(
(Object[]) v1);
}
}
, new org.apache.calcite.linq4j.function.Function1() {
public Byte apply(Object[] v1) {
return (Byte) v1[3];
}
public Object apply(Object v1) {
return apply(
(Object[]) v1);
}
}
, new org.apache.calcite.linq4j.function.Function2() {
public Object[] apply(Object[] left, Object[] right) {
return new Object[] {
left[0],
left[1],
left[2],
[...elided ...]
left[3298],
left[3299],
right[0],
right[1],
[...elided...]]
right[3295],
right[3296],
right[3297],
right[3298],
right[3299]};
}
public Object[] apply(Object left, Object right) {
return apply(
(Object[]) left,
(Object[]) right);
}
}
, null, false, false);
}


public Class getElementType() {
return java.lang.Object[].class;
}

at org.apache.calcite.avatica.Helper.wrap(Helper.java:37)
at org.apache.calcite.adapter.enumerable.EnumerableInterpretable.toBindable(EnumerableInterpretable.java:128)
at org.apache.calcite.prepare.CalcitePrepareImpl$CalcitePreparingStmt.implement(CalcitePrepareImpl.java:1233)
at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:332)
at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:231)
at org.apache.calcite.prepare.CalcitePrepareImpl.prepare2_(CalcitePrepareImpl.java:767)
at org.apache.calcite.prepare.CalcitePrepareImpl.prepare_(CalcitePrepareImpl.java:631)
at org.apache.calcite.prepare.CalcitePrepareImpl.prepareSql(CalcitePrepareImpl.java:601)
at org.apache.calcite.jdbc.CalciteConnectionImpl.parseQuery(CalciteConnectionImpl.java:229)
at org.apache.calcite.jdbc.CalciteConnectionImpl.prepareStatement_(CalciteConnectionImpl.java:211)
... 80 more
Caused by: org.codehaus.janino.InternalCompilerException: Compiling "Baz": Code of method "apply([Ljava/lang/Object;[Ljava/lang/Object;)[Ljava/lang/Object;" of class "Baz$3" grows beyond 64 KB
at org.codehaus.janino.UnitCompiler.compileUnit(UnitCompiler.java:382)
at org.codehaus.janino.SimpleCompiler.cook(SimpleCompiler.java:237)
at org.codehaus.janino.SimpleCompiler.compileToClassLoader(SimpleCompiler.java:465)
at org.codehaus.janino.ClassBodyEvaluator.compileToClass(ClassBodyEvaluator.java:313)
at org.codehaus.janino.ClassBodyEvaluator.cook(ClassBodyEvaluator.java:235)
at org.codehaus.janino.SimpleCompiler.cook(SimpleCompiler.java:207)
at org.codehaus.commons.compiler.Cookable.cook(Cookable.java:50)
at org.codehaus.janino.ClassBodyEvaluator.createInstance(ClassBodyEvaluator.java:347)
at org.apache.calcite.adapter.enumerable.EnumerableInterpretable.getBindable(EnumerableInterpretable.java:162)
at org.apache.calcite.adapter.enumerable.EnumerableInterpretable.toBindable(EnumerableInterpretable.java:125)
... 88 more
Caused by: org.codehaus.janino.InternalCompilerException: Code of method "apply([Ljava/lang/Object;[Ljava/lang/Object;)[Ljava/lang/Object;" of class "Baz$3" grows beyond 64 KB
at org.codehaus.janino.CodeContext.makeSpace(CodeContext.java:1048)
at org.codehaus.janino.CodeContext.write(CodeContext.java:925)
at org.codehaus.janino.UnitCompiler.writeOpcode(UnitCompiler.java:12291)
at org.codehaus.janino.UnitCompiler.pushConstant(UnitCompiler.java:10746)
at org.codehaus.janino.UnitCompiler.compileGetValue(UnitCompiler.java:5616)
at org.codehaus.janino.UnitCompiler.compileGet2(UnitCompiler.java:5592)
at org.codehaus.janino.UnitCompiler.access$9700(UnitCompiler.java:215)
at org.codehaus.janino.UnitCompiler$16.visitNewInitializedArray(UnitCompiler.java:4434)
at org.codehaus.janino.UnitCompiler$16.visitNewInitializedArray(UnitCompiler.java:4396)
at org.codehaus.janino.Java$NewInitializedArray.accept(Java.java:5373)
at org.codehaus.janino.UnitCompiler.compileGet(UnitCompiler.java:4396)
at org.codehaus.janino.UnitCompiler.compileGetValue(UnitCompiler.java:5662)
at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:2649)
at org.codehaus.janino.UnitCompiler.access$2800(UnitCompiler.java:215)
at org.codehaus.janino.UnitCompiler$6.visitReturnStatement(UnitCompiler.java:1504)
at org.codehaus.janino.UnitCompiler$6.visitReturnStatement(UnitCompiler.java:1487)
at org.codehaus.janino.Java$ReturnStatement.accept(Java.java:3563)
at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:1487)
at org.codehaus.janino.UnitCompiler.compileStatements(UnitCompiler.java:1567)
at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:3388)
at org.codehaus.janino.UnitCompiler.compileDeclaredMethods(UnitCompiler.java:1357)
at org.codehaus.janino.UnitCompiler.compileDeclaredMethods(UnitCompiler.java:1330)
at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:822)
at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:981)
at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:951)
at org.codehaus.janino.UnitCompiler.access$200(UnitCompiler.java:215)
at org.codehaus.janino.UnitCompiler$2.visitAnonymousClassDeclaration(UnitCompiler.java:409)
at org.codehaus.janino.UnitCompiler$2.visitAnonymousClassDeclaration(UnitCompiler.java:406)
at org.codehaus.janino.Java$AnonymousClassDeclaration.accept(Java.java:1149)
at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:406)
at org.codehaus.janino.UnitCompiler.compileGet2(UnitCompiler.java:5509)
at org.codehaus.janino.UnitCompiler.access$9500(UnitCompiler.java:215)
at org.codehaus.janino.UnitCompiler$16.visitNewAnonymousClassInstance(UnitCompiler.java:4432)
at org.codehaus.janino.UnitCompiler$16.visitNewAnonymousClassInstance(UnitCompiler.java:4396)
at org.codehaus.janino.Java$NewAnonymousClassInstance.accept(Java.java:5238)
at org.codehaus.janino.UnitCompiler.compileGet(UnitCompiler.java:4396)
at org.codehaus.janino.UnitCompiler.compileGetValue(UnitCompiler.java:5662)
at org.codehaus.janino.UnitCompiler.compileGet2(UnitCompiler.java:5182)
at org.codehaus.janino.UnitCompiler.access$9100(UnitCompiler.java:215)
at org.codehaus.janino.UnitCompiler$16.visitMethodInvocation(UnitCompiler.java:4423)
at org.codehaus.janino.UnitCompiler$16.visitMethodInvocation(UnitCompiler.java:4396)
at org.codehaus.janino.Java$MethodInvocation.accept(Java.java:5073)
at org.codehaus.janino.UnitCompiler.compileGet(UnitCompiler.java:4396)
at org.codehaus.janino.UnitCompiler.compileGetValue(UnitCompiler.java:5662)
at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:2649)
at org.codehaus.janino.UnitCompiler.access$2800(UnitCompiler.java:215)
at org.codehaus.janino.UnitCompiler$6.visitReturnStatement(UnitCompiler.java:1504)
at org.codehaus.janino.UnitCompiler$6.visitReturnStatement(UnitCompiler.java:1487)
at org.codehaus.janino.Java$ReturnStatement.accept(Java.java:3563)
at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:1487)
at org.codehaus.janino.UnitCompiler.compileStatements(UnitCompiler.java:1567)
at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:3388)
at org.codehaus.janino.UnitCompiler.compileDeclaredMethods(UnitCompiler.java:1357)
at org.codehaus.janino.UnitCompiler.compileDeclaredMethods(UnitCompiler.java:1330)
at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:822)
at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:432)
at org.codehaus.janino.UnitCompiler.access$400(UnitCompiler.java:215)
at org.codehaus.janino.UnitCompiler$2.visitPackageMemberClassDeclaration(UnitCompiler.java:411)
at org.codehaus.janino.UnitCompiler$2.visitPackageMemberClassDeclaration(UnitCompiler.java:406)
at org.codehaus.janino.Java$PackageMemberClassDeclaration.accept(Java.java:1414)
at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:406)
at org.codehaus.janino.UnitCompiler.compileUnit(UnitCompiler.java:378)
... 97 more

 

 

Issue Links

• causes: CALCITE-6593 NPE when outer joining tables with many fields and unmatching rows
• is related to: KYLIN-3657 sql length over 64 KB cause query fail
• relates to: CALCITE-2792 Stackoverflow while evaluating filter with large number of OR conditions
• links to: GitHub Pull Request #3826