Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-2958

GROUP BY causing ClassCastException [LazyDioInteger cannot be cast LazyInteger]

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: 0.9.0
    • Fix Version/s: 0.9.0
    • Component/s: HBase Handler
    • Labels:
      None
    • Environment:

      HBase 0.90.4, Hive 0.90 snapshot (trunk) built today

      Description

      This relates to https://issues.apache.org/jira/browse/HIVE-1634.

      The following work fine:

      CREATE EXTERNAL TABLE tim_hbase_occurrence ( 
        id int,
        scientific_name string,
        data_resource_id int
      ) STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' WITH SERDEPROPERTIES (
        "hbase.columns.mapping" = ":key#b,v:scientific_name#s,v:data_resource_id#b"
      ) TBLPROPERTIES(
        "hbase.table.name" = "mini_occurrences", 
        "hbase.table.default.storage.type" = "binary"
      );
      SELECT * FROM tim_hbase_occurrence LIMIT 3;
      SELECT * FROM tim_hbase_occurrence WHERE data_resource_id=1081 LIMIT 3;
      

      However, the following fails:

      SELECT data_resource_id, count(*) FROM tim_hbase_occurrence GROUP BY data_resource_id;
      

      The error given:

      0 TS
      2012-04-17 16:58:45,693 INFO org.apache.hadoop.hive.ql.exec.MapOperator: Initialization Done 7 MAP
      2012-04-17 16:58:45,714 INFO org.apache.hadoop.hive.ql.exec.MapOperator: Processing alias tim_hbase_occurrence for file hdfs://c1n2.gbif.org/user/hive/warehouse/tim_hbase_occurrence
      2012-04-17 16:58:45,714 INFO org.apache.hadoop.hive.ql.exec.MapOperator: 7 forwarding 1 rows
      2012-04-17 16:58:45,714 INFO org.apache.hadoop.hive.ql.exec.TableScanOperator: 0 forwarding 1 rows
      2012-04-17 16:58:45,716 INFO org.apache.hadoop.hive.ql.exec.SelectOperator: 1 forwarding 1 rows
      2012-04-17 16:58:45,723 FATAL ExecMapper: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row {"id":1444,"scientific_name":null,"data_resource_id":1081}
      	at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:548)
      	at org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:143)
      	at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
      	at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:391)
      	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:325)
      	at org.apache.hadoop.mapred.Child$4.run(Child.java:270)
      	at java.security.AccessController.doPrivileged(Native Method)
      	at javax.security.auth.Subject.doAs(Subject.java:396)
      	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1157)
      	at org.apache.hadoop.mapred.Child.main(Child.java:264)
      Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.ClassCastException: org.apache.hadoop.hive.serde2.lazydio.LazyDioInteger cannot be cast to org.apache.hadoop.hive.serde2.lazy.LazyInteger
      	at org.apache.hadoop.hive.ql.exec.GroupByOperator.processOp(GroupByOperator.java:737)
      	at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
      	at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:762)
      	at org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(SelectOperator.java:84)
      	at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
      	at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:762)
      	at org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:83)
      	at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
      	at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:762)
      	at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:529)
      	... 9 more
      Caused by: java.lang.ClassCastException: org.apache.hadoop.hive.serde2.lazydio.LazyDioInteger cannot be cast to org.apache.hadoop.hive.serde2.lazy.LazyInteger
      	at org.apache.hadoop.hive.serde2.lazy.objectinspector.primitive.LazyIntObjectInspector.copyObject(LazyIntObjectInspector.java:43)
      	at org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils.copyToStandardObject(ObjectInspectorUtils.java:239)
      	at org.apache.hadoop.hive.ql.exec.KeyWrapperFactory$ListKeyWrapper.deepCopyElements(KeyWrapperFactory.java:150)
      	at org.apache.hadoop.hive.ql.exec.KeyWrapperFactory$ListKeyWrapper.deepCopyElements(KeyWrapperFactory.java:142)
      	at org.apache.hadoop.hive.ql.exec.KeyWrapperFactory$ListKeyWrapper.copyKey(KeyWrapperFactory.java:119)
      	at org.apache.hadoop.hive.ql.exec.GroupByOperator.processHashAggr(GroupByOperator.java:750)
      	at org.apache.hadoop.hive.ql.exec.GroupByOperator.processOp(GroupByOperator.java:722)
      	... 18 more
      

        Attachments

          Activity

            People

            • Assignee:
              navis Navis
              Reporter:
              timrobertson100 Tim Robertson
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: