Details
-
Bug
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
1.14.0
Description
When we run query with unnest referencing a list within a nested map type then query fails with IOB. But if list if not nested in map it works fine. For example:
Failure Case: with schema
customer {
element {
c_orders: [O1, O2]
}
}
SELECT tt.og, tt.ag FROM dfs.testData.`/*` t, LATERAL (SELECT l.o as og, l.o as ag FROM UNNEST(t.element.c_orders) l(o)) tt
Success Case:
customer {
c_orders: [O1, O2]
}
SELECT tt.og, tt.ag FROM dfs.testData.`/*` t, LATERAL (SELECT l.o as og, l.o as ag FROM UNNEST(t.c_orders) l(o)) tt
Exception:
2018-09-28 11:13:03,633 [245190cf-da1c-02cd-977e-ded27e7abcff:foreman] INFO o.a.drill.exec.work.foreman.Foreman - Query text for query with id 245190cf-da1c-02cd-977e-ded27e7abcff issued by anonymous: SELECT tt.og, tt.ag FROM dfs.testData.`/*` t, LATERAL (SELECT l.o as og, l.o as ag FROM UNNEST(t.element.c_orders) l(o)) tt 2018-09-28 11:13:03,725 [245190cf-da1c-02cd-977e-ded27e7abcff:foreman] ERROR o.a.drill.exec.work.foreman.Foreman - SYSTEM ERROR: IndexOutOfBoundsException: index (3) must be less than size (3) [Error Id: d5a21cc8-581f-4c35-9950-c7cfbd7de194 on 172.30.8.49:31010] org.apache.drill.common.exceptions.UserException: SYSTEM ERROR: IndexOutOfBoundsException: index (3) must be less than size (3) [Error Id: d5a21cc8-581f-4c35-9950-c7cfbd7de194 on 172.30.8.49:31010] at org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:633) ~[drill-common-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT] at org.apache.drill.exec.work.foreman.Foreman$ForemanResult.close(Foreman.java:779) [drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT] at org.apache.drill.exec.work.foreman.QueryStateProcessor.checkCommonStates(QueryStateProcessor.java:325) [drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT] at org.apache.drill.exec.work.foreman.QueryStateProcessor.planning(QueryStateProcessor.java:221) [drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT] at org.apache.drill.exec.work.foreman.QueryStateProcessor.moveToState(QueryStateProcessor.java:83) [drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT] at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:299) [drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_102] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_102] at java.lang.Thread.run(Thread.java:745) [na:1.8.0_102] Caused by: org.apache.drill.exec.work.foreman.ForemanException: Unexpected exception during fragment initialization: Error while applying rule DrillMergeProjectRule:force_mode, args [rel#4423:LogicalProject.NONE.ANY([]).[](input=rel#4422:Subset#8.NONE.ANY([]).[],og=$2,ag=$3), rel#4421:LogicalProject.NONE.ANY([]).[](input=rel#4420:Subset#7.NONE.ANY([]).[],**=$0,c_orders=$1,og=$3)] at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:300) [drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT] ... 3 common frames omitted Caused by: java.lang.RuntimeException: Error while applying rule DrillMergeProjectRule:force_mode, args [rel#4423:LogicalProject.NONE.ANY([]).[](input=rel#4422:Subset#8.NONE.ANY([]).[],og=$2,ag=$3), rel#4421:LogicalProject.NONE.ANY([]).[](input=rel#4420:Subset#7.NONE.ANY([]).[],**=$0,c_orders=$1,og=$3)] at org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch(VolcanoRuleCall.java:236) ~[calcite-core-1.17.0-drill-r0.jar:1.17.0-drill-r0] at org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp(VolcanoPlanner.java:648) ~[calcite-core-1.17.0-drill-r0.jar:1.17.0-drill-r0] at org.apache.calcite.tools.Programs$RuleSetProgram.run(Programs.java:339) ~[calcite-core-1.17.0-drill-r0.jar:1.17.0-drill-r0] at org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.transform(DefaultSqlHandler.java:425) ~[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT] at org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.transform(DefaultSqlHandler.java:365) ~[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT] at org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToRawDrel(DefaultSqlHandler.java:252) ~[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT] at org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToDrel(DefaultSqlHandler.java:314) ~[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT] at org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.getPlan(DefaultSqlHandler.java:179) ~[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT] at org.apache.drill.exec.planner.sql.DrillSqlWorker.getQueryPlan(DrillSqlWorker.java:145) ~[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT] at org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan(DrillSqlWorker.java:83) ~[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT] at org.apache.drill.exec.work.foreman.Foreman.runSQL(Foreman.java:584) [drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT] at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:272) [drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT] ... 3 common frames omitted Caused by: java.lang.IndexOutOfBoundsException: index (3) must be less than size (3) at com.google.common.base.Preconditions.checkElementIndex(Preconditions.java:310) ~[guava-19.0.jar:na] at com.google.common.base.Preconditions.checkElementIndex(Preconditions.java:293) ~[guava-19.0.jar:na] at com.google.common.collect.RegularImmutableList.get(RegularImmutableList.java:67) ~[guava-19.0.jar:na] at org.apache.calcite.plan.RelOptUtil$2.visitInputRef(RelOptUtil.java:2638) ~[calcite-core-1.17.0-drill-r0.jar:1.17.0-drill-r0] at org.apache.calcite.plan.RelOptUtil$2.visitInputRef(RelOptUtil.java:2636) ~[calcite-core-1.17.0-drill-r0.jar:1.17.0-drill-r0] at org.apache.calcite.rex.RexInputRef.accept(RexInputRef.java:112) ~[calcite-core-1.17.0-drill-r0.jar:1.17.0-drill-r0] at org.apache.calcite.rex.RexShuttle.visitList(RexShuttle.java:166) ~[calcite-core-1.17.0-drill-r0.jar:1.17.0-drill-r0] at org.apache.calcite.plan.RelOptUtil.pushPastProject(RelOptUtil.java:2631) ~[calcite-core-1.17.0-drill-r0.jar:1.17.0-drill-r0] at org.apache.drill.exec.planner.logical.DrillMergeProjectRule.onMatch(DrillMergeProjectRule.java:134) ~[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT] at org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch(VolcanoRuleCall.java:212) ~[calcite-core-1.17.0-drill-r0.jar:1.17.0-drill-r0] ... 14 common frames omitted 2018-09-28 11:13:03,734 [qtp224809804-247] ERROR o.a.d.e.server.rest.QueryResources - Query from Web UI Failed org.apache.drill.common.exceptions.UserRemoteException: SYSTEM ERROR: IndexOutOfBoundsException: index (3) must be less than size (3) [Error Id: d5a21cc8-581f-4c35-9950-c7cfbd7de194 on 172.30.8.49:31010] at org.apache.drill.exec.server.rest.QueryWrapper.run(QueryWrapper.java:115) ~[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT] at org.apache.drill.exec.server.rest.QueryResources.submitQueryJSON(QueryResources.java:72) ~[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT] at org.apache.drill.exec.server.rest.QueryResources.submitQuery(QueryResources.java:87) ~[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT] at sun.reflect.GeneratedMethodAccessor109.invoke(Unknown Source) ~[na:na] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_102] at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_102] at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory$1.invoke(ResourceMethodInvocationHandlerFactory.java:81) [jersey-server-2.8.jar:na] at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:151) [jersey-server-2.8.jar:na] at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:171) [jersey-server-2.8.jar:na] at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:195) [jersey-server-2.8.jar:na] at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:104) [jersey-server-2.8.jar:na] at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:387) [jersey-server-2.8.jar:na] at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:331) [jersey-server-2.8.jar:na] at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:103) [jersey-server-2.8.jar:na] at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:269) [jersey-server-2.8.jar:na] at org.glassfish.jersey.internal.Errors$1.call(Errors.java:271) [jersey-common-2.8.jar:na] at org.glassfish.jersey.internal.Errors$1.call(Errors.java:267) [jersey-common-2.8.jar:na] at org.glassfish.jersey.internal.Errors.process(Errors.java:315) [jersey-common-2.8.jar:na] at org.glassfish.jersey.internal.Errors.process(Errors.java:297) [jersey-common-2.8.jar:na] at org.glassfish.jersey.internal.Errors.process(Errors.java:267) [jersey-common-2.8.jar:na] at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:297) [jersey-common-2.8.jar:na] at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:252) [jersey-server-2.8.jar:na] at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1023) [jersey-server-2.8.jar:na] at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:372) [jersey-container-servlet-core-2.8.jar:na] at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:382) [jersey-container-servlet-core-2.8.jar:na] at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:345) [jersey-container-servlet-core-2.8.jar:na] at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:220) [jersey-container-servlet-core-2.8.jar:na] at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:738) [jetty-servlet-9.1.5.v20140505.jar:9.1.5.v20140505] at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:551) [jetty-servlet-9.1.5.v20140505.jar:9.1.5.v20140505] at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:219) [jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505] at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1111) [jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505] at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:478) [jetty-servlet-9.1.5.v20140505.jar:9.1.5.v20140505] at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:183) [jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505] at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1045) [jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505] at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) [jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505] at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97) [jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505] at org.eclipse.jetty.server.Server.handle(Server.java:462) [jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505] at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:279) [jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505] at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:232) [jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505] at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:534) [jetty-io-9.1.5.v20140505.jar:9.1.5.v20140505] at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:607) [jetty-util-9.1.5.v20140505.jar:9.1.5.v20140505] at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:536) [jetty-util-9.1.5.v20140505.jar:9.1.5.v20140505] at java.lang.Thread.run(Thread.java:745) [na:1.8.0_102]
Attachments
Issue Links
- links to