[CALCITE-2900] RelStructuredTypeFlattener generates wrong types on nested columns

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.18.0
• Fix Version/s: 1.20.0
• Component/s: core
• Labels: pull-request-available
• Assignee: Will Yu
• Reporter: Will Yu
• Created: 07/Mar/19 01:42
• Updated: 24/Jun/19 15:01
• Resolved: 29/Mar/19 23:01

Description

When we leverage Calcite to parse a SQL and generates a RelNode tree on top of a table with structured columns, Calcite throws an error about type mismatch shown at bottom.

After a bit of debugging, the issue is suspected to happen in RewriteRexShuttle.visitFieldAccess. When the input is RexInputRef, such as $36.target_column, the newField type is calculated based on ONLY the index 36, WITHOUT considering the offset of target_column. The result is new field type is the type of the first sub-column of column $36, which causes the type mismatch.

 

Error and stacktrace:

java.lang.AssertionError: type mismatch:
ref:
INTEGER NOT NULL
input:
VARCHAR CHARACTER SET "ISO-8859-1" COLLATE "ISO-8859-1$en_US$primary" NOT NULL

at org.apache.calcite.util.Litmus$1.fail(Litmus.java:31)
at org.apache.calcite.plan.RelOptUtil.eq(RelOptUtil.java:1832)
at org.apache.calcite.rex.RexChecker.visitInputRef(RexChecker.java:125)
at org.apache.calcite.rex.RexChecker.visitInputRef(RexChecker.java:57)
at org.apache.calcite.rex.RexInputRef.accept(RexInputRef.java:112)
at org.apache.calcite.rel.core.Project.isValid(Project.java:186)
at org.apache.calcite.rel.core.Project.<init>(Project.java:83)
at org.apache.calcite.rel.logical.LogicalProject.<init>(LogicalProject.java:62)
at org.apache.calcite.rel.logical.LogicalProject.create(LogicalProject.java:112)
at org.apache.calcite.rel.logical.LogicalProject.create(LogicalProject.java:100)
at org.apache.calcite.rel.core.RelFactories$ProjectFactoryImpl.createProject(RelFactories.java:140)
at org.apache.calcite.tools.RelBuilder.project(RelBuilder.java:1248)
at org.apache.calcite.tools.RelBuilder.projectNamed(RelBuilder.java:1304)
at org.apache.calcite.sql2rel.RelStructuredTypeFlattener.rewriteRel(RelStructuredTypeFlattener.java:506)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.calcite.util.ReflectUtil.invokeVisitorInternal(ReflectUtil.java:257)
at org.apache.calcite.util.ReflectUtil.invokeVisitor(ReflectUtil.java:214)
at org.apache.calcite.util.ReflectUtil$1.invokeVisitor(ReflectUtil.java:464)
at org.apache.calcite.sql2rel.RelStructuredTypeFlattener$RewriteRelVisitor.visit(RelStructuredTypeFlattener.java:759)
at org.apache.calcite.rel.SingleRel.childrenAccept(SingleRel.java:72)
at org.apache.calcite.rel.RelVisitor.visit(RelVisitor.java:44)
at org.apache.calcite.sql2rel.RelStructuredTypeFlattener$RewriteRelVisitor.visit(RelStructuredTypeFlattener.java:754)
at org.apache.calcite.sql2rel.RelStructuredTypeFlattener.rewrite(RelStructuredTypeFlattener.java:194)
at org.apache.calcite.sql2rel.SqlToRelConverter.flattenTypes(SqlToRelConverter.java:466)
at org.apache.calcite.prepare.PlannerImpl.rel(PlannerImpl.java:236)

 

Issue Links

• is related to: CALCITE-2963 IndexOutOfBoundsException in RelStructuredTypeFlattener when accessing nested fields
• is related to: CALCITE-2962 RelStructuredTypeFlattener generates wrong types for nested column when flattenProjection
• links to: GitHub Pull Request #1104