[DRILL-7752] Query fails on hive table with date column - ASF JIRA

XML

Word

Printable

JSON

Details

Type: Bug
Status: Closed
Priority: Minor
Resolution: Not A Bug
Affects Version/s: 1.17.0
Fix Version/s: None
Component/s: Storage - Hive
Labels:
None

Description

Steps to reproduce:
1. Create hive table:

create  external table if not exists customer_complex (
    c_custkey int,
    c_name string,
    c_address string,
    c_nation struct<
        n_name:string,
        n_comment:string,
        n_region:map<string,string>>,
    c_phone string,
    c_acctbal double,
    c_mktsegment string,
    c_comment string,
    c_orders array<struct<
        o_orderstatus:string,
        o_totalprice:double,
        o_orderdate:date,
        o_orderpriority:string,
        o_clerk:string,
        o_shippriority:int,
        o_comment:string,
        o_lineitems:array<struct<
            l_part:struct<
                p_name:string,
                p_mfgr:string,
                p_brand:string,
                p_type:string,
                p_size:int,
                p_container:string,
                p_retailprice:double,
                p_comment:string>,
            l_supplier:struct<
                s_name:string,
                s_address:string,
                s_nationkey:int,
                s_phone:string,
                s_acctbal:double,
                s_comment:string>,
            l_linenumber:int,
            l_quantity:double,
            l_extendedprice:double,
            l_discount:double,
            l_tax:double,
            l_returnflag:string,
            l_linestatus:string,
            l_shipdate:date,
            l_commitdate:date,
            l_receiptdate:date,
            l_shipinstruct:string,
            l_shipmode:string,
            l_comment:string>>>>
)
STORED AS parquet
LOCATION '/drill/customer_complex';

2. Add folder customer_complex. ZIP attached.

3. Query hive table using Drill:

SELECT c_orders FROM  hive.customer_complex;

Query fails with error:

org.apache.drill.common.exceptions.UserRemoteException: INTERNAL_ERROR ERROR: For input string: "361"

Fragment: 0:0

Stack trace:

2020-06-25 08:30:51,116 [210b3cd3-8532-fa13-9853-50f22d16be5f:foreman] INFO  o.a.drill.exec.work.foreman.Foreman - Query text for query with id 210b3cd3-8532-fa13-9853-50f22d16be5f issued by anonymous: SELECT c_orders FROM  hive.customer_complex
2020-06-25 08:30:51,444 [210b3cd3-8532-fa13-9853-50f22d16be5f:foreman] INFO  o.a.drill.exec.store.PluginHandle - Creating storage plugin for hive
2020-06-25 08:30:52,669 [210b3cd3-8532-fa13-9853-50f22d16be5f:foreman] INFO  o.a.drill.exec.store.PluginHandle - Creating storage plugin for dfs
2020-06-25 08:30:52,768 [210b3cd3-8532-fa13-9853-50f22d16be5f:foreman] INFO  o.a.d.c.s.persistence.ScanResult - loading 18 classes for org.apache.drill.exec.store.dfs.FormatPlugin took 48ms
2020-06-25 08:30:52,854 [210b3cd3-8532-fa13-9853-50f22d16be5f:foreman] INFO  o.a.d.c.s.persistence.ScanResult - loading 19 classes for org.apache.drill.common.logical.FormatPluginConfig took 0ms
2020-06-25 08:30:52,855 [210b3cd3-8532-fa13-9853-50f22d16be5f:foreman] INFO  o.a.d.c.s.persistence.ScanResult - loading 19 classes for org.apache.drill.common.logical.FormatPluginConfig took 0ms
2020-06-25 08:30:52,856 [210b3cd3-8532-fa13-9853-50f22d16be5f:foreman] INFO  o.a.d.c.s.persistence.ScanResult - loading 19 classes for org.apache.drill.common.logical.FormatPluginConfig took 0ms
2020-06-25 08:30:52,856 [210b3cd3-8532-fa13-9853-50f22d16be5f:foreman] INFO  o.a.d.c.s.persistence.ScanResult - loading 19 classes for org.apache.drill.common.logical.FormatPluginConfig took 0ms
2020-06-25 08:30:52,856 [210b3cd3-8532-fa13-9853-50f22d16be5f:foreman] INFO  o.a.d.c.s.persistence.ScanResult - loading 19 classes for org.apache.drill.common.logical.FormatPluginConfig took 0ms
2020-06-25 08:30:52,857 [210b3cd3-8532-fa13-9853-50f22d16be5f:foreman] INFO  o.a.d.c.s.persistence.ScanResult - loading 19 classes for org.apache.drill.common.logical.FormatPluginConfig took 0ms
2020-06-25 08:30:52,857 [210b3cd3-8532-fa13-9853-50f22d16be5f:foreman] INFO  o.a.drill.exec.store.PluginHandle - Creating storage plugin for maprdb
2020-06-25 08:30:53,888 [210b3cd3-8532-fa13-9853-50f22d16be5f:frag:0:0] INFO  o.a.d.e.w.fragment.FragmentExecutor - 210b3cd3-8532-fa13-9853-50f22d16be5f:0:0: State change requested AWAITING_ALLOCATION --> RUNNING
2020-06-25 08:30:53,889 [210b3cd3-8532-fa13-9853-50f22d16be5f:frag:0:0] INFO  o.a.d.e.w.f.FragmentStatusReporter - 210b3cd3-8532-fa13-9853-50f22d16be5f:0:0: State to report: RUNNING
2020-06-25 08:30:54,761 [210b3cd3-8532-fa13-9853-50f22d16be5f:frag:0:0] INFO  o.a.d.exec.physical.impl.ScanBatch - User Error Occurred: For input string: "361" (For input string: "361")
org.apache.drill.common.exceptions.UserException: INTERNAL_ERROR ERROR: For input string: "361"
Please, refer to logs for more information.[Error Id: d6b8f2ba-f111-4205-8229-24d1739cd024 ]
    at org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:657)
    at org.apache.drill.exec.physical.impl.ScanBatch.next(ScanBatch.java:305)
    at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:119)
    at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:111)
    at org.apache.drill.exec.record.AbstractUnaryRecordBatch.innerNext(AbstractUnaryRecordBatch.java:59)
    at org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext(ProjectRecordBatch.java:85)
    at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:170)
    at org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:103)
    at org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.innerNext(ScreenCreator.java:81)
    at org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:93)
    at org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentExecutor.java:323)
    at org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentExecutor.java:310)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:422)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1669)
    at org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:310)
    at org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.NumberFormatException: For input string: "361"
    at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
    at java.lang.Integer.parseInt(Integer.java:569)
    at java.lang.Integer.parseInt(Integer.java:615)
    at java.sql.Date.valueOf(Date.java:133)
    at org.apache.drill.exec.store.hive.writers.primitive.HiveDateWriter.write(HiveDateWriter.java:37)
    at org.apache.drill.exec.store.hive.writers.complex.HiveStructWriter.write(HiveStructWriter.java:50)
    at org.apache.drill.exec.store.hive.writers.complex.HiveListWriter.write(HiveListWriter.java:52)
    at org.apache.drill.exec.store.hive.readers.HiveDefaultRecordReader.readHiveRecordAndInsertIntoRecordBatch(HiveDefaultRecordReader.java:413)
    at org.apache.drill.exec.store.hive.readers.HiveDefaultRecordReader.next(HiveDefaultRecordReader.java:399)
    at org.apache.drill.exec.physical.impl.ScanBatch.internalNext(ScanBatch.java:234)
    at org.apache.drill.exec.physical.impl.ScanBatch.next(ScanBatch.java:298)
    ... 18 common frames omitted
2020-06-25 08:30:54,762 [210b3cd3-8532-fa13-9853-50f22d16be5f:frag:0:0] ERROR o.a.d.e.physical.impl.BaseRootExec - Batch dump started: dumping last 2 failed batches
2020-06-25 08:30:54,762 [210b3cd3-8532-fa13-9853-50f22d16be5f:frag:0:0] ERROR o.a.d.exec.physical.impl.ScanBatch - ScanBatch[container=org.apache.drill.exec.record.VectorContainer@143d73f6[recordCount = 0, schemaChanged = true, schema = null, wrappers = [org.apache.drill.exec.vector.complex.RepeatedMapVector@5578670e], ...], currentReader=org.apache.drill.exec.store.hive.readers.HiveDefaultRecordReader@6ce3b5e4[columns = [`c_orders`], isStarQuery = false, isSkipQuery = false], schema=null]
2020-06-25 08:30:54,762 [210b3cd3-8532-fa13-9853-50f22d16be5f:frag:0:0] ERROR o.a.d.e.p.i.p.ProjectRecordBatch - ProjectRecordBatch[projector=null, hasRemainder=false, remainderIndex=0, recordCount=0, container=org.apache.drill.exec.record.VectorContainer@793aa3d5[recordCount = 0, schemaChanged = true, schema = null, wrappers = [], ...]]
2020-06-25 08:30:54,762 [210b3cd3-8532-fa13-9853-50f22d16be5f:frag:0:0] ERROR o.a.d.e.physical.impl.BaseRootExec - Batch dump completed.
2020-06-25 08:30:54,762 [210b3cd3-8532-fa13-9853-50f22d16be5f:frag:0:0] INFO  o.a.d.e.w.fragment.FragmentExecutor - 210b3cd3-8532-fa13-9853-50f22d16be5f:0:0: State change requested RUNNING --> FAILED
2020-06-25 08:30:54,765 [210b3cd3-8532-fa13-9853-50f22d16be5f:frag:0:0] INFO  o.a.d.e.w.fragment.FragmentExecutor - 210b3cd3-8532-fa13-9853-50f22d16be5f:0:0: State change requested FAILED --> FINISHED
2020-06-25 08:30:54,780 [210b3cd3-8532-fa13-9853-50f22d16be5f:frag:0:0] WARN  o.a.d.exec.rpc.control.WorkEventBus - Fragment 210b3cd3-8532-fa13-9853-50f22d16be5f:0:0 manager is not found in the work bus.
2020-06-25 08:30:54,814 [qtp1007076085-61] ERROR o.a.d.e.server.rest.QueryResources - Query from Web UI Failed: {}
org.apache.drill.common.exceptions.UserRemoteException: INTERNAL_ERROR ERROR: For input string: "361"Fragment: 0:0Please, refer to logs for more information.[Error Id: d6b8f2ba-f111-4205-8229-24d1739cd024 on maprdemo:31010]
    at org.apache.drill.exec.server.rest.RestQueryRunner.submitQuery(RestQueryRunner.java:181)
    at org.apache.drill.exec.server.rest.RestQueryRunner.run(RestQueryRunner.java:70)
    at org.apache.drill.exec.server.rest.QueryResources.submitQueryJSON(QueryResources.java:96)
    at org.apache.drill.exec.server.rest.QueryResources.submitQuery(QueryResources.java:114)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory$1.invoke(ResourceMethodInvocationHandlerFactory.java:81)
    at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:144)
    at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:161)
    at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:205)
    at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:99)
    at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:389)
    at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:347)
    at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:102)
    at org.glassfish.jersey.server.ServerRuntime$2.run(ServerRuntime.java:326)
    at org.glassfish.jersey.internal.Errors$1.call(Errors.java:271)
    at org.glassfish.jersey.internal.Errors$1.call(Errors.java:267)
    at org.glassfish.jersey.internal.Errors.process(Errors.java:315)
    at org.glassfish.jersey.internal.Errors.process(Errors.java:297)
    at org.glassfish.jersey.internal.Errors.process(Errors.java:267)
    at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:317)
    at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:305)
    at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1154)
    at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:473)
    at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:427)
    at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:388)
    at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:341)
    at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:228)
    at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:848)
    at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1780)
    at org.apache.drill.exec.server.rest.header.ResponseHeadersSettingFilter.doFilter(ResponseHeadersSettingFilter.java:71)
    at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1767)
    at org.apache.drill.exec.server.rest.CsrfTokenValidateFilter.doFilter(CsrfTokenValidateFilter.java:55)
    at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1767)
    at org.apache.drill.exec.server.rest.CsrfTokenInjectFilter.doFilter(CsrfTokenInjectFilter.java:54)
    at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1767)
    at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:583)
    at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
    at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
    at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:513)
    at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
    at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
    at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
    at org.eclipse.jetty.server.Server.handle(Server.java:539)
    at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:333)
    at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
    at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
    at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
    at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
    at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
    at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
    at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
    at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
    at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
    at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.NumberFormatException: For input string: "361"
    at .......(:0)
    at org.apache.drill.exec.store.hive.writers.primitive.HiveDateWriter.write(HiveDateWriter.java:37)
    at org.apache.drill.exec.store.hive.writers.complex.HiveStructWriter.write(HiveStructWriter.java:50)
    at org.apache.drill.exec.store.hive.writers.complex.HiveListWriter.write(HiveListWriter.java:52)
    at org.apache.drill.exec.store.hive.readers.HiveDefaultRecordReader.readHiveRecordAndInsertIntoRecordBatch(HiveDefaultRecordReader.java:413)
    at org.apache.drill.exec.store.hive.readers.HiveDefaultRecordReader.next(HiveDefaultRecordReader.java:399)
    at org.apache.drill.exec.physical.impl.ScanBatch.internalNext(ScanBatch.java:234)
    at org.apache.drill.exec.physical.impl.ScanBatch.next(ScanBatch.java:298)
    at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:119)
    at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:111)
    at org.apache.drill.exec.record.AbstractUnaryRecordBatch.innerNext(AbstractUnaryRecordBatch.java:59)
    at org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext(ProjectRecordBatch.java:85)
    at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:170)
    at org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:103)
    at org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.innerNext(ScreenCreator.java:81)
    at org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:93)
    at org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentExecutor.java:323)
    at org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentExecutor.java:310)
    at .......(:0)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1669)
    at org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:310)
    at org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38)
    at .......(:0)
2020-06-25 08:30:54,946 [210b3cd3-8532-fa13-9853-50f22d16be5f:frag:0:0] WARN  o.a.d.e.w.f.QueryStateProcessor - Dropping request to move to COMPLETED state as query is already at FAILED state (which is terminal).

Attachments

- Sort By Name
- Sort By Date
- Ascending
- Descending

customer_complex.zip
26/Jun/20 12:58
2.56 MB
Maksym Rymar

Activity

People

Assignee:: Maksym Rymar

Reporter:: Maksym Rymar

Votes:: 0 Vote for this issue

Watchers:: 2 Start watching this issue

Dates

Created:: 26/Jun/20 13:01

Updated:: 03/Jul/20 10:36

Resolved:: 03/Jul/20 10:36