Details
-
Bug
-
Status: Closed
-
Minor
-
Resolution: Not A Bug
-
1.17.0
-
None
-
None
Description
Steps to reproduce:
1. Create hive table:
create external table if not exists customer_complex ( c_custkey int, c_name string, c_address string, c_nation struct< n_name:string, n_comment:string, n_region:map<string,string>>, c_phone string, c_acctbal double, c_mktsegment string, c_comment string, c_orders array<struct< o_orderstatus:string, o_totalprice:double, o_orderdate:date, o_orderpriority:string, o_clerk:string, o_shippriority:int, o_comment:string, o_lineitems:array<struct< l_part:struct< p_name:string, p_mfgr:string, p_brand:string, p_type:string, p_size:int, p_container:string, p_retailprice:double, p_comment:string>, l_supplier:struct< s_name:string, s_address:string, s_nationkey:int, s_phone:string, s_acctbal:double, s_comment:string>, l_linenumber:int, l_quantity:double, l_extendedprice:double, l_discount:double, l_tax:double, l_returnflag:string, l_linestatus:string, l_shipdate:date, l_commitdate:date, l_receiptdate:date, l_shipinstruct:string, l_shipmode:string, l_comment:string>>>> ) STORED AS parquet LOCATION '/drill/customer_complex';
2. Add folder customer_complex. ZIP attached.
3. Query hive table using Drill:
SELECT c_orders FROM hive.customer_complex;
Query fails with error:
org.apache.drill.common.exceptions.UserRemoteException: INTERNAL_ERROR ERROR: For input string: "361"
Fragment: 0:0
Stack trace:
2020-06-25 08:30:51,116 [210b3cd3-8532-fa13-9853-50f22d16be5f:foreman] INFO o.a.drill.exec.work.foreman.Foreman - Query text for query with id 210b3cd3-8532-fa13-9853-50f22d16be5f issued by anonymous: SELECT c_orders FROM hive.customer_complex 2020-06-25 08:30:51,444 [210b3cd3-8532-fa13-9853-50f22d16be5f:foreman] INFO o.a.drill.exec.store.PluginHandle - Creating storage plugin for hive 2020-06-25 08:30:52,669 [210b3cd3-8532-fa13-9853-50f22d16be5f:foreman] INFO o.a.drill.exec.store.PluginHandle - Creating storage plugin for dfs 2020-06-25 08:30:52,768 [210b3cd3-8532-fa13-9853-50f22d16be5f:foreman] INFO o.a.d.c.s.persistence.ScanResult - loading 18 classes for org.apache.drill.exec.store.dfs.FormatPlugin took 48ms 2020-06-25 08:30:52,854 [210b3cd3-8532-fa13-9853-50f22d16be5f:foreman] INFO o.a.d.c.s.persistence.ScanResult - loading 19 classes for org.apache.drill.common.logical.FormatPluginConfig took 0ms 2020-06-25 08:30:52,855 [210b3cd3-8532-fa13-9853-50f22d16be5f:foreman] INFO o.a.d.c.s.persistence.ScanResult - loading 19 classes for org.apache.drill.common.logical.FormatPluginConfig took 0ms 2020-06-25 08:30:52,856 [210b3cd3-8532-fa13-9853-50f22d16be5f:foreman] INFO o.a.d.c.s.persistence.ScanResult - loading 19 classes for org.apache.drill.common.logical.FormatPluginConfig took 0ms 2020-06-25 08:30:52,856 [210b3cd3-8532-fa13-9853-50f22d16be5f:foreman] INFO o.a.d.c.s.persistence.ScanResult - loading 19 classes for org.apache.drill.common.logical.FormatPluginConfig took 0ms 2020-06-25 08:30:52,856 [210b3cd3-8532-fa13-9853-50f22d16be5f:foreman] INFO o.a.d.c.s.persistence.ScanResult - loading 19 classes for org.apache.drill.common.logical.FormatPluginConfig took 0ms 2020-06-25 08:30:52,857 [210b3cd3-8532-fa13-9853-50f22d16be5f:foreman] INFO o.a.d.c.s.persistence.ScanResult - loading 19 classes for org.apache.drill.common.logical.FormatPluginConfig took 0ms 2020-06-25 08:30:52,857 [210b3cd3-8532-fa13-9853-50f22d16be5f:foreman] INFO o.a.drill.exec.store.PluginHandle - Creating storage plugin for maprdb 2020-06-25 08:30:53,888 [210b3cd3-8532-fa13-9853-50f22d16be5f:frag:0:0] INFO o.a.d.e.w.fragment.FragmentExecutor - 210b3cd3-8532-fa13-9853-50f22d16be5f:0:0: State change requested AWAITING_ALLOCATION --> RUNNING 2020-06-25 08:30:53,889 [210b3cd3-8532-fa13-9853-50f22d16be5f:frag:0:0] INFO o.a.d.e.w.f.FragmentStatusReporter - 210b3cd3-8532-fa13-9853-50f22d16be5f:0:0: State to report: RUNNING 2020-06-25 08:30:54,761 [210b3cd3-8532-fa13-9853-50f22d16be5f:frag:0:0] INFO o.a.d.exec.physical.impl.ScanBatch - User Error Occurred: For input string: "361" (For input string: "361") org.apache.drill.common.exceptions.UserException: INTERNAL_ERROR ERROR: For input string: "361" Please, refer to logs for more information.[Error Id: d6b8f2ba-f111-4205-8229-24d1739cd024 ] at org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:657) at org.apache.drill.exec.physical.impl.ScanBatch.next(ScanBatch.java:305) at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:119) at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:111) at org.apache.drill.exec.record.AbstractUnaryRecordBatch.innerNext(AbstractUnaryRecordBatch.java:59) at org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext(ProjectRecordBatch.java:85) at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:170) at org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:103) at org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.innerNext(ScreenCreator.java:81) at org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:93) at org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentExecutor.java:323) at org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentExecutor.java:310) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1669) at org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:310) at org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: java.lang.NumberFormatException: For input string: "361" at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65) at java.lang.Integer.parseInt(Integer.java:569) at java.lang.Integer.parseInt(Integer.java:615) at java.sql.Date.valueOf(Date.java:133) at org.apache.drill.exec.store.hive.writers.primitive.HiveDateWriter.write(HiveDateWriter.java:37) at org.apache.drill.exec.store.hive.writers.complex.HiveStructWriter.write(HiveStructWriter.java:50) at org.apache.drill.exec.store.hive.writers.complex.HiveListWriter.write(HiveListWriter.java:52) at org.apache.drill.exec.store.hive.readers.HiveDefaultRecordReader.readHiveRecordAndInsertIntoRecordBatch(HiveDefaultRecordReader.java:413) at org.apache.drill.exec.store.hive.readers.HiveDefaultRecordReader.next(HiveDefaultRecordReader.java:399) at org.apache.drill.exec.physical.impl.ScanBatch.internalNext(ScanBatch.java:234) at org.apache.drill.exec.physical.impl.ScanBatch.next(ScanBatch.java:298) ... 18 common frames omitted 2020-06-25 08:30:54,762 [210b3cd3-8532-fa13-9853-50f22d16be5f:frag:0:0] ERROR o.a.d.e.physical.impl.BaseRootExec - Batch dump started: dumping last 2 failed batches 2020-06-25 08:30:54,762 [210b3cd3-8532-fa13-9853-50f22d16be5f:frag:0:0] ERROR o.a.d.exec.physical.impl.ScanBatch - ScanBatch[container=org.apache.drill.exec.record.VectorContainer@143d73f6[recordCount = 0, schemaChanged = true, schema = null, wrappers = [org.apache.drill.exec.vector.complex.RepeatedMapVector@5578670e], ...], currentReader=org.apache.drill.exec.store.hive.readers.HiveDefaultRecordReader@6ce3b5e4[columns = [`c_orders`], isStarQuery = false, isSkipQuery = false], schema=null] 2020-06-25 08:30:54,762 [210b3cd3-8532-fa13-9853-50f22d16be5f:frag:0:0] ERROR o.a.d.e.p.i.p.ProjectRecordBatch - ProjectRecordBatch[projector=null, hasRemainder=false, remainderIndex=0, recordCount=0, container=org.apache.drill.exec.record.VectorContainer@793aa3d5[recordCount = 0, schemaChanged = true, schema = null, wrappers = [], ...]] 2020-06-25 08:30:54,762 [210b3cd3-8532-fa13-9853-50f22d16be5f:frag:0:0] ERROR o.a.d.e.physical.impl.BaseRootExec - Batch dump completed. 2020-06-25 08:30:54,762 [210b3cd3-8532-fa13-9853-50f22d16be5f:frag:0:0] INFO o.a.d.e.w.fragment.FragmentExecutor - 210b3cd3-8532-fa13-9853-50f22d16be5f:0:0: State change requested RUNNING --> FAILED 2020-06-25 08:30:54,765 [210b3cd3-8532-fa13-9853-50f22d16be5f:frag:0:0] INFO o.a.d.e.w.fragment.FragmentExecutor - 210b3cd3-8532-fa13-9853-50f22d16be5f:0:0: State change requested FAILED --> FINISHED 2020-06-25 08:30:54,780 [210b3cd3-8532-fa13-9853-50f22d16be5f:frag:0:0] WARN o.a.d.exec.rpc.control.WorkEventBus - Fragment 210b3cd3-8532-fa13-9853-50f22d16be5f:0:0 manager is not found in the work bus. 2020-06-25 08:30:54,814 [qtp1007076085-61] ERROR o.a.d.e.server.rest.QueryResources - Query from Web UI Failed: {} org.apache.drill.common.exceptions.UserRemoteException: INTERNAL_ERROR ERROR: For input string: "361"Fragment: 0:0Please, refer to logs for more information.[Error Id: d6b8f2ba-f111-4205-8229-24d1739cd024 on maprdemo:31010] at org.apache.drill.exec.server.rest.RestQueryRunner.submitQuery(RestQueryRunner.java:181) at org.apache.drill.exec.server.rest.RestQueryRunner.run(RestQueryRunner.java:70) at org.apache.drill.exec.server.rest.QueryResources.submitQueryJSON(QueryResources.java:96) at org.apache.drill.exec.server.rest.QueryResources.submitQuery(QueryResources.java:114) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory$1.invoke(ResourceMethodInvocationHandlerFactory.java:81) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:144) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:161) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:205) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:99) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:389) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:347) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:102) at org.glassfish.jersey.server.ServerRuntime$2.run(ServerRuntime.java:326) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:271) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:267) at org.glassfish.jersey.internal.Errors.process(Errors.java:315) at org.glassfish.jersey.internal.Errors.process(Errors.java:297) at org.glassfish.jersey.internal.Errors.process(Errors.java:267) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:317) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:305) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1154) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:473) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:427) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:388) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:341) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:228) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:848) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1780) at org.apache.drill.exec.server.rest.header.ResponseHeadersSettingFilter.doFilter(ResponseHeadersSettingFilter.java:71) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1767) at org.apache.drill.exec.server.rest.CsrfTokenValidateFilter.doFilter(CsrfTokenValidateFilter.java:55) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1767) at org.apache.drill.exec.server.rest.CsrfTokenInjectFilter.doFilter(CsrfTokenInjectFilter.java:54) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1767) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:583) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:513) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134) at org.eclipse.jetty.server.Server.handle(Server.java:539) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:333) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108) at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589) at java.lang.Thread.run(Thread.java:748) Caused by: java.lang.NumberFormatException: For input string: "361" at .......(:0) at org.apache.drill.exec.store.hive.writers.primitive.HiveDateWriter.write(HiveDateWriter.java:37) at org.apache.drill.exec.store.hive.writers.complex.HiveStructWriter.write(HiveStructWriter.java:50) at org.apache.drill.exec.store.hive.writers.complex.HiveListWriter.write(HiveListWriter.java:52) at org.apache.drill.exec.store.hive.readers.HiveDefaultRecordReader.readHiveRecordAndInsertIntoRecordBatch(HiveDefaultRecordReader.java:413) at org.apache.drill.exec.store.hive.readers.HiveDefaultRecordReader.next(HiveDefaultRecordReader.java:399) at org.apache.drill.exec.physical.impl.ScanBatch.internalNext(ScanBatch.java:234) at org.apache.drill.exec.physical.impl.ScanBatch.next(ScanBatch.java:298) at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:119) at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:111) at org.apache.drill.exec.record.AbstractUnaryRecordBatch.innerNext(AbstractUnaryRecordBatch.java:59) at org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext(ProjectRecordBatch.java:85) at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:170) at org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:103) at org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.innerNext(ScreenCreator.java:81) at org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:93) at org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentExecutor.java:323) at org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentExecutor.java:310) at .......(:0) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1669) at org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:310) at org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38) at .......(:0) 2020-06-25 08:30:54,946 [210b3cd3-8532-fa13-9853-50f22d16be5f:frag:0:0] WARN o.a.d.e.w.f.QueryStateProcessor - Dropping request to move to COMPLETED state as query is already at FAILED state (which is terminal).