Apache Drill
  1. Apache Drill
  2. DRILL-459

Hive source table containing varchar fails in drill

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.4.0
    • Component/s: None
    • Labels:
      None

      Description

      Query a hive table as below

      hive> desc nation;
      OK
      n_nationkey int None
      n_name varchar(25) None
      n_regionkey int None
      n_comment varchar(152) None
      Time taken: 1.697 seconds, Fetched: 4 row(s)

      0: jdbc:drill:> select * from hive.nation;
      message: "Failure while parsing sql. < UnsupportedOperationException:[ Could not determine type ]"

      Stacktrace:
      java.lang.UnsupportedOperationException: Could not determine type
      at org.apache.drill.exec.store.hive.HiveRecordReader.getMajorType(HiveRecordReader.java:374) ~[drill-java-exec-1.0.0-m2-incubating-SNAPSHOT-rebuffed.jar:1.0.0-m2-incubating-SNAPSHOT]
      at org.apache.drill.exec.store.hive.HiveRecordReader.setup(HiveRecordReader.java:198) ~[drill-java-exec-1.0.0-m2-incubating-SNAPSHOT-rebuffed.jar:1.0.0-m2-incubating-SNAPSHOT]
      at org.apache.drill.exec.physical.impl.ScanBatch.<init>(ScanBatch.java:65) ~[drill-java-exec-1.0.0-m2-incubating-SNAPSHOT-rebuffed.jar:1.0.0-m2-incubating-SNAPSHOT]
      at org.apache.drill.exec.store.hive.HiveScanBatchCreator.getBatch(HiveScanBatchCreator.java:69) ~[drill-java-exec-1.0.0-m2-incubating-SNAPSHOT-rebuffed.jar:1.0.0-m2-incubating-SNAPSHOT]
      at org.apache.drill.exec.store.hive.HiveScanBatchCreator.getBatch(HiveScanBatchCreator.java:35) ~[drill-java-exec-1.0.0-m2-incubating-SNAPSHOT-rebuffed.jar:1.0.0-m2-incubating-SNAPSHOT]
      at org.apache.drill.exec.physical.impl.ImplCreator.visitOp(ImplCreator.java:60) ~[drill-java-exec-1.0.0-m2-incubating-SNAPSHOT-rebuffed.jar:1.0.0-m2-incubating-SNAPSHOT]
      at org.apache.drill.exec.physical.impl.ImplCreator.visitOp(ImplCreator.java:37) ~[drill-java-exec-1.0.0-m2-incubating-SNAPSHOT-rebuffed.jar:1.0.0-m2-incubating-SNAPSHOT]
      at org.apache.drill.exec.physical.base.AbstractPhysicalVisitor.visitSubScan(AbstractPhysicalVisitor.java:82) ~[drill-java-exec-1.0.0-m2-incubating-SNAPSHOT-rebuffed.jar:1.0.0-m2-incubating-SNAPSHOT]

        Activity

        Hide
        Steven Phillips added a comment -

        could you run describe extended nation

        and post the output here?

        Show
        Steven Phillips added a comment - could you run describe extended nation and post the output here?
        Hide
        Ramana Inukonda Nagaraj added a comment -

        Detailed Table Information Table(tableName:nation, dbName:default, owner:root, createTime:1395871704, lastAccessTime:0, retention:0, sd:StorageDescriptor(cols:[FieldSchema(name:n_nationkey, type:int, comment:null), FieldSchema(name:n_name, type:varchar(25), comment:null), FieldSchema(name:n_regionkey, type:int, comment:null), FieldSchema(name:n_comment, type:varchar(152), comment:null)], location:maprfs:/user/hive/warehouse/nation, inputFormat:org.apache.hadoop.mapred.TextInputFormat, outputFormat:org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, compressed:false, numBuckets:-1, serdeInfo:SerDeInfo(name:null, serializationLib:org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, parameters:

        {serialization.format=|, field.delim=|}

        ), bucketCols:[], sortCols:[], parameters:{}, skewedInfo:SkewedInfo(skewedColNames:[], skewedColValues:[], skewedColValueLocationMaps:{}), storedAsSubDirectories:false), partitionKeys:[], parameters:

        {transient_lastDdlTime=1395871704}

        , viewOriginalText:null, viewExpandedText:null, tableType:MANAGED_TABLE)

        Show
        Ramana Inukonda Nagaraj added a comment - Detailed Table Information Table(tableName:nation, dbName:default, owner:root, createTime:1395871704, lastAccessTime:0, retention:0, sd:StorageDescriptor(cols: [FieldSchema(name:n_nationkey, type:int, comment:null), FieldSchema(name:n_name, type:varchar(25), comment:null), FieldSchema(name:n_regionkey, type:int, comment:null), FieldSchema(name:n_comment, type:varchar(152), comment:null)] , location:maprfs:/user/hive/warehouse/nation, inputFormat:org.apache.hadoop.mapred.TextInputFormat, outputFormat:org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, compressed:false, numBuckets:-1, serdeInfo:SerDeInfo(name:null, serializationLib:org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, parameters: {serialization.format=|, field.delim=|} ), bucketCols:[], sortCols:[], parameters:{}, skewedInfo:SkewedInfo(skewedColNames:[], skewedColValues:[], skewedColValueLocationMaps:{}), storedAsSubDirectories:false), partitionKeys:[], parameters: {transient_lastDdlTime=1395871704} , viewOriginalText:null, viewExpandedText:null, tableType:MANAGED_TABLE)
        Hide
        Jacques Nadeau added a comment -

        should be fixed. complex data still isn't supported, though.

        Show
        Jacques Nadeau added a comment - should be fixed. complex data still isn't supported, though.

          People

          • Assignee:
            Unassigned
            Reporter:
            Ramana Inukonda Nagaraj
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development