[DRILL-5166] Select with options returns NPE - ASF JIRA

XML

Word

Printable

JSON

Details

Type: Bug
Status: Closed
Priority: Major
Resolution: Fixed
Affects Version/s: 1.9.0
Fix Version/s: 1.12.0
Component/s: None
Labels:
None

Description

When querying two csv files:

First file (2 records):

key_header, value_header
key_1,value_1

Second file (500000 records):

key_header, value_header
key_1,value_1
.......
key_499999,value_499999

Select with options returns NPE:

select * from table(dfs.root.`/home/arina/files/ver/*.csv`(type => 'text',extractHeader => true, fieldDelimiter => ',')) limit 10;

Querying without options works file:

select  * from dfs.root.`/home/arina/files/ver/*.csv` limit 10;

Error:

Caused by: org.apache.drill.common.exceptions.UserRemoteException: SYSTEM ERROR: NullPointerException

Fragment 1:0

[Error Id: b789f5f8-f090-4097-b7ff-9f4efd3d01e8 on localhost:31013]

  (com.fasterxml.jackson.databind.JsonMappingException) Instantiation of [simple type, class org.apache.drill.exec.store.dfs.easy.EasySubScan] value failed (java.lang.NullPointerException): null
 at [Source: {
  "pop" : "single-sender",
  "@id" : 0,
  "receiver-major-fragment" : 0,
  "receiver-minor-fragment" : 0,
  "child" : {
    "pop" : "selection-vector-remover",
    "@id" : 1,
    "child" : {
      "pop" : "limit",
      "@id" : 2,
      "child" : {
        "pop" : "fs-sub-scan",
        "@id" : 3,
        "userName" : "arina",
        "files" : [ {
          "start" : 0,
          "length" : 11777804,
          "path" : "file:/home/arina/files/ver/key_value_500000.csv"
        } ],
        "storage" : {
          "type" : "file",
          "enabled" : true,
          "connection" : "file:///",
          "config" : null,
          "workspaces" : {
            "root" : {
              "location" : "/",
              "writable" : false,
              "defaultInputFormat" : null
            },
            "tmp" : {
              "location" : "/tmp",
              "writable" : false,
              "defaultInputFormat" : null
            }
          },
          "formats" : {
            "psv" : {
              "type" : "text",
              "extensions" : [ "tbl" ],
              "delimiter" : "|"
            },
            "csv" : {
              "type" : "text",
              "extensions" : [ "csv" ],
              "delimiter" : ","
            },
            "tsv" : {
              "type" : "text",
              "extensions" : [ "tsv" ],
              "delimiter" : "\t"
            },
            "httpd" : {
              "type" : "httpd",
              "logFormat" : "%h %t \"%r\" %>s %b \"%{Referer}i\"",
              "timestampFormat" : null
            },
            "parquet" : {
              "type" : "parquet"
            },
            "json" : {
              "type" : "json",
              "extensions" : [ "json" ]
            },
            "avro" : {
              "type" : "avro"
            },
            "sequencefile" : {
              "type" : "sequencefile",
              "extensions" : [ "seq" ]
            },
            "csvh" : {
              "type" : "text",
              "extensions" : [ "csvh" ],
              "extractHeader" : true,
              "delimiter" : ","
            }
          }
        },
        "format" : {
          "type" : "named",
          "name" : "text"
        },
        "columns" : [ "`*`" ],
        "selectionRoot" : "file:/home/arina/files/ver",
        "initialAllocation" : 1000000,
        "maxAllocation" : 10000000000,
        "cost" : 0.0
      },
      "first" : 0,
      "last" : 10,
      "initialAllocation" : 1000000,
      "maxAllocation" : 10000000000,
      "cost" : 10.0
    },
    "initialAllocation" : 1000000,
    "maxAllocation" : 10000000000,
    "cost" : 10.0
  },
  "destination" : "Cglsb2NhbGhvc3QQpfIBGKbyASCn8gEyDzEuMTAuMC1TTkFQU0hPVA==",
  "initialAllocation" : 1000000,
  "maxAllocation" : 10000000000,
  "cost" : 10.0
}; line: 90, column: 7] (through reference chain: org.apache.drill.exec.physical.config.SingleSender["child"]->org.apache.drill.exec.physical.config.SelectionVectorRemover["child"]->org.apache.drill.exec.physical.config.Limit["child"])
    com.fasterxml.jackson.databind.JsonMappingException.from():223
    com.fasterxml.jackson.databind.deser.std.StdValueInstantiator.wrapAsJsonMappingException():445
    com.fasterxml.jackson.databind.deser.std.StdValueInstantiator.rewrapCtorProblem():464
    com.fasterxml.jackson.databind.deser.std.StdValueInstantiator.createFromObjectWith():258
    com.fasterxml.jackson.databind.deser.impl.PropertyBasedCreator.build():135
    com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased():444
    com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault():1123
    com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject():298
    com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeWithObjectId():1094
    com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeOther():166
    com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize():135
    com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer._deserializeTypedForId():120
    com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer.deserializeTypedFromObject():91
    com.fasterxml.jackson.databind.deser.AbstractDeserializer.deserializeWithType():142
    com.fasterxml.jackson.databind.deser.SettableBeanProperty.deserialize():488
    com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeWithErrorWrapping():465
    com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased():380
    com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault():1123
    com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject():298
    com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeWithObjectId():1094
    com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeOther():166
    com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize():135
    com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer._deserializeTypedForId():120
    com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer.deserializeTypedFromObject():91
    com.fasterxml.jackson.databind.deser.AbstractDeserializer.deserializeWithType():142
    com.fasterxml.jackson.databind.deser.SettableBeanProperty.deserialize():488
    com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeWithErrorWrapping():465
    com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased():380
    com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault():1123
    com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject():298
    com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeWithObjectId():1094
    com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeOther():166
    com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize():135
    com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer._deserializeTypedForId():120
    com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer.deserializeTypedFromObject():91
    com.fasterxml.jackson.databind.deser.AbstractDeserializer.deserializeWithType():142
    com.fasterxml.jackson.databind.deser.SettableBeanProperty.deserialize():488
    com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeWithErrorWrapping():465
    com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased():380
    com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault():1123
    com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject():298
    com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeWithObjectId():1094
    com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeOther():166
    com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize():135
    com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer._deserializeTypedForId():120
    com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer.deserializeTypedFromObject():91
    com.fasterxml.jackson.databind.deser.AbstractDeserializer.deserializeWithType():142
    com.fasterxml.jackson.databind.deser.impl.TypeWrappedDeserializer.deserialize():42
    com.fasterxml.jackson.databind.ObjectReader._bindAndClose():1575
    com.fasterxml.jackson.databind.ObjectReader.readValue():1200
    org.apache.drill.exec.planner.PhysicalPlanReader.readFragmentOperator():94
    org.apache.drill.exec.work.fragment.FragmentExecutor.run():203
    org.apache.drill.common.SelfCleaningRunnable.run():38
    java.util.concurrent.ThreadPoolExecutor.runWorker():1145
    java.util.concurrent.ThreadPoolExecutor$Worker.run():615
    java.lang.Thread.run():745
  Caused By (java.lang.NullPointerException) null
    com.google.common.base.Preconditions.checkNotNull():210
    org.apache.drill.exec.store.dfs.easy.EasySubScan.<init>():60
    sun.reflect.NativeConstructorAccessorImpl.newInstance0():-2
    sun.reflect.NativeConstructorAccessorImpl.newInstance():57
    sun.reflect.DelegatingConstructorAccessorImpl.newInstance():45
    java.lang.reflect.Constructor.newInstance():526
    com.fasterxml.jackson.databind.introspect.AnnotatedConstructor.call():114
    com.fasterxml.jackson.databind.deser.std.StdValueInstantiator.createFromObjectWith():256
    com.fasterxml.jackson.databind.deser.impl.PropertyBasedCreator.build():135
    com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased():444
    com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault():1123
    com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject():298
    com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeWithObjectId():1094
    com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeOther():166
    com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize():135
    com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer._deserializeTypedForId():120
    com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer.deserializeTypedFromObject():91
    com.fasterxml.jackson.databind.deser.AbstractDeserializer.deserializeWithType():142
    com.fasterxml.jackson.databind.deser.SettableBeanProperty.deserialize():488
    com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeWithErrorWrapping():465
    com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased():380
    com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault():1123
    com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject():298
    com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeWithObjectId():1094
    com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeOther():166
    com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize():135
    com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer._deserializeTypedForId():120
    com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer.deserializeTypedFromObject():91
    com.fasterxml.jackson.databind.deser.AbstractDeserializer.deserializeWithType():142
    com.fasterxml.jackson.databind.deser.SettableBeanProperty.deserialize():488
    com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeWithErrorWrapping():465
    com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased():380
    com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault():1123
    com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject():298
    com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeWithObjectId():1094
    com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeOther():166
    com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize():135
    com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer._deserializeTypedForId():120
    com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer.deserializeTypedFromObject():91
    com.fasterxml.jackson.databind.deser.AbstractDeserializer.deserializeWithType():142
    com.fasterxml.jackson.databind.deser.SettableBeanProperty.deserialize():488
    com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeWithErrorWrapping():465
    com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased():380
    com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault():1123
    com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject():298
    com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeWithObjectId():1094
    com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeOther():166
    com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize():135
    com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer._deserializeTypedForId():120
    com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer.deserializeTypedFromObject():91
    com.fasterxml.jackson.databind.deser.AbstractDeserializer.deserializeWithType():142
    com.fasterxml.jackson.databind.deser.impl.TypeWrappedDeserializer.deserialize():42
    com.fasterxml.jackson.databind.ObjectReader._bindAndClose():1575
    com.fasterxml.jackson.databind.ObjectReader.readValue():1200
    org.apache.drill.exec.planner.PhysicalPlanReader.readFragmentOperator():94
    org.apache.drill.exec.work.fragment.FragmentExecutor.run():203
    org.apache.drill.common.SelfCleaningRunnable.run():38
    java.util.concurrent.ThreadPoolExecutor.runWorker():1145
    java.util.concurrent.ThreadPoolExecutor$Worker.run():615
    java.lang.Thread.run():745

	at org.apache.drill.exec.rpc.user.QueryResultHandler.resultArrived(QueryResultHandler.java:123)
	at org.apache.drill.exec.rpc.user.UserClient.handleReponse(UserClient.java:145)
	at org.apache.drill.exec.rpc.BasicClientWithConnection.handle(BasicClientWithConnection.java:46)
	at org.apache.drill.exec.rpc.BasicClientWithConnection.handle(BasicClientWithConnection.java:31)
	at org.apache.drill.exec.rpc.RpcBus.handle(RpcBus.java:65)
	at org.apache.drill.exec.rpc.RpcBus$RequestEvent.run(RpcBus.java:363)
	at org.apache.drill.common.SerializedExecutor$RunnableProcessor.run(SerializedExecutor.java:89)
	at org.apache.drill.exec.rpc.RpcBus$SameExecutor.execute(RpcBus.java:240)
	at org.apache.drill.common.SerializedExecutor.execute(SerializedExecutor.java:123)
	at org.apache.drill.exec.rpc.RpcBus$InboundHandler.decode(RpcBus.java:274)
	at org.apache.drill.exec.rpc.RpcBus$InboundHandler.decode(RpcBus.java:245)
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:89)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:324)
	at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:254)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:324)
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:324)
	at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:242)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:324)
	at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:324)
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:847)
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:131)
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:511)
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468)
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382)
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354)
	at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:111)
	at java.lang.Thread.run(Thread.java:745)

Attachments

Issue Links

is part of

DRILL-5771 Fix serDe errors for format plugins

Resolved

Activity

People

Assignee:: Arina Ielchiieva

Reporter:: Arina Ielchiieva

Votes:: 0 Vote for this issue

Watchers:: 2 Start watching this issue

Dates

Created:: 28/Dec/16 11:36

Updated:: 19/Jan/18 11:54

Resolved:: 23/Nov/17 13:23