Uploaded image for project: 'Apache Drill'
  1. Apache Drill
  2. DRILL-4686

Aggregation query over HBase table results in IllegalStateException: Failure while reading vector

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.7.0
    • Fix Version/s: 1.12.0
    • Labels:
      None
    • Environment:

      4 node cluster CentOS

      Description

      Aggregation query over HBase table from Drill 1.7.0 returns IllegalStateException
      Drill version 1.7.0-SNAPSHOT, commit ID : 09b26277

      put 'browser_action2', '1','v:e0', 'abc1';
      put 'browser_action2', '2','v:e0', 'abc2';
      put 'browser_action2', '3','v:e0', 'abc3';
      put 'browser_action2', '4','v:e0', 'abc4';
      put 'browser_action2', '5','v:e0', 'abc5';
      put 'browser_action2', '6','v:e0', 'abc6';
      put 'browser_action2', '7','v:e0', 'abc7';
      put 'browser_action2', '8','v:e0', 'abc8';
      put 'browser_action2', '9','v:e0', 'abc9';
      put 'browser_action2', '10','v:e0', 'abc10';
      
      [root@centos-01 ~]# hbase shell
      HBase Shell; enter 'help<RETURN>' for list of supported commands.
      Type "exit<RETURN>" to leave the HBase Shell
      Version 1.1.1-mapr-1602-SNAPSHOT, r05ceb750d7ac9decac18e92650fedc0e86d85c7a, Mon Mar 28 18:32:45 UTC 2016
      
      Not all HBase shell commands are applicable to MapR tables.
      Consult MapR documentation for the list of supported commands.
      
      hbase(main):001:0> scan 'browser_action2'
      ROW                                COLUMN+CELL
       1                                 column=v:e0, timestamp=1463589516782, value=abc1
       10                                column=v:e0, timestamp=1463589516916, value=abc10
       2                                 column=v:e0, timestamp=1463589516809, value=abc2
       3                                 column=v:e0, timestamp=1463589516829, value=abc3
       4                                 column=v:e0, timestamp=1463589516834, value=abc4
       5                                 column=v:e0, timestamp=1463589516847, value=abc5
       6                                 column=v:e0, timestamp=1463589516861, value=abc6
       7                                 column=v:e0, timestamp=1463589516874, value=abc7
       8                                 column=v:e0, timestamp=1463589516896, value=abc8
       9                                 column=v:e0, timestamp=1463589516905, value=abc9
      10 row(s) in 0.7970 seconds
      
      hbase(main):002:0>
      
      0: jdbc:drill:schema=dfs.tmp> use hbase;
      +-------+------------------------------------+
      |  ok   |              summary               |
      +-------+------------------------------------+
      | true  | Default schema changed to [hbase]  |
      +-------+------------------------------------+
      1 row selected (0.327 seconds)
      0: jdbc:drill:schema=dfs.tmp> show tables;
      +---------------+---------------------+
      | TABLE_SCHEMA  |     TABLE_NAME      |
      +---------------+---------------------+
      | hbase         | browser_action2     |
      | hbase         | fewtypes_null       |
      | hbase         | fewtypes_null_hive  |
      | hbase         | hbase_student       |
      | hbase         | hbase_voter         |
      | hbase         | hbvoter             |
      | hbase         | hive_hbase          |
      | hbase         | student             |
      | hbase         | voter               |
      +---------------+---------------------+
      9 rows selected (1.597 seconds)
      0: jdbc:drill:schema=dfs.tmp> describe browser_action2;
      +--------------+------------+--------------+
      | COLUMN_NAME  | DATA_TYPE  | IS_NULLABLE  |
      +--------------+------------+--------------+
      | row_key      | ANY        | NO           |
      | v            | MAP        | NO           |
      +--------------+------------+--------------+
      2 rows selected (1.669 seconds)
      0: jdbc:drill:schema=dfs.tmp> select  CONVERT_FROM(a.`v`.`e0`, 'UTF8') as k from hbase.browser_action2 a;
      +--------+
      |   k    |
      +--------+
      | abc6   |
      | abc5   |
      | abc3   |
      | abc7   |
      | abc1   |
      | abc10  |
      | abc9   |
      | abc2   |
      | abc8   |
      | abc4   |
      +--------+
      10 rows selected (0.924 seconds)
      0: jdbc:drill:schema=dfs.tmp> select CONVERT_FROM(a.`v`.`e0`, 'UTF8') as k, count(a.`v`.`e0`) p from hbase.browser_action2 a where a.row_key > '0'  group by a.`v`.`e0`;
      Error: SYSTEM ERROR: IllegalStateException: Failure while reading vector.  Expected vector class of org.apache.drill.exec.vector.NullableIntVector but was holding vector class org.apache.drill.exec.vector.NullableVarBinaryVector, field= $f0(VARBINARY:OPTIONAL)[$bits$(UINT1:REQUIRED), $f0(VARBINARY:OPTIONAL)[$offsets$(UINT4:REQUIRED)]]
      
      Fragment 2:3
      
      [Error Id: 5c0c8dc2-141b-4a2c-ba7e-008693fe4273 on centos-02.qa.lab:31010] (state=,code=0)
      0: jdbc:drill:schema=dfs.tmp>
      

      Explain plan for the above failing query

      0: jdbc:drill:schema=dfs.tmp> explain plan for select CONVERT_FROM(a.`v`.`e0`, 'UTF8') as k, count(a.`v`.`e0`) p from hbase.browser_action2 a where a.row_key > '0'  group by a.`v`.`e0`;
      +------+------+
      | text | json |
      +------+------+
      | 00-00    Screen
      00-01      Project(k=[$0], p=[$1])
      00-02        UnionExchange
      01-01          Project(k=[CONVERT_FROMUTF8($0)], p=[$1])
      01-02            HashAgg(group=[{0}], p=[$SUM0($1)])
      01-03              Project($f0=[$0], p=[$1])
      01-04                HashToRandomExchange(dist0=[[$0]])
      02-01                  UnorderedMuxExchange
      03-01                    Project($f0=[$0], p=[$1], E_X_P_R_H_A_S_H_F_I_E_L_D=[hash32AsDouble($0)])
      03-02                      HashAgg(group=[{0}], p=[COUNT($0)])
      03-03                        Project($f0=[ITEM($1, 'e0')])
      03-04                          Scan(groupscan=[HBaseGroupScan [HBaseScanSpec=HBaseScanSpec [tableName=browser_action2, startRow=0\x00, stopRow=, filter=null], columns=[`*`]]])
      

      Java code used to create table in HBase

      public class PutIntDataToHBase {
          public static void main(String args[]) throws IOException {
              Configuration conf = HBaseConfiguration.create();
              conf.set("hbase.zookeeper.property.clientPort","5181");
              HBaseAdmin admin = new HBaseAdmin(conf);
              if (admin.tableExists("browser_action2")) {
                  admin.disableTable("browser_action2");
                  admin.deleteTable("browser_action2");
              }
      
              byte[][] SPLIT_KEYS = {{'0'},{'1'},{'2'},{'3'},{'4'},{'5'},{'6'},{'7'},{'8'},{'9'}};
              HTableDescriptor tableDesc = new
                  HTableDescriptor(TableName.valueOf("browser_action2"));
      
              tableDesc.addFamily(new HColumnDescriptor("v"));
              admin.createTable(tableDesc,SPLIT_KEYS);
      
          }
      }
      

      Stack trace from drillbit.log

      2016-05-19 06:11:31,906 [28c2a6ec-c264-23a9-c1d3-d8d0d174ca57:frag:2:3] ERROR o.a.d.e.w.fragment.FragmentExecutor - SYSTEM ERROR: IllegalStateException: Failure while reading vector.  Expected vector class of org.apache.drill.exec.vector.NullableIntVector but was holding vector class org.apache.drill.exec.vector.NullableVarBinaryVector, field= $f0(VARBINARY:OPTIONAL)[$bits$(UINT1:REQUIRED), $f0(VARBINARY:OPTIONAL)[$offsets$(UINT4:REQUIRED)]]
      
      Fragment 2:3
      
      [Error Id: 5c0c8dc2-141b-4a2c-ba7e-008693fe4273 on centos-02.qa.lab:31010]
      org.apache.drill.common.exceptions.UserException: SYSTEM ERROR: IllegalStateException: Failure while reading vector.  Expected vector class of org.apache.drill.exec.vector.NullableIntVector but was holding vector class org.apache.drill.exec.vector.NullableVarBinaryVector, field= $f0(VARBINARY:OPTIONAL)[$bits$(UINT1:REQUIRED), $f0(VARBINARY:OPTIONAL)[$offsets$(UINT4:REQUIRED)]]
      
      Fragment 2:3
      
      [Error Id: 5c0c8dc2-141b-4a2c-ba7e-008693fe4273 on centos-02.qa.lab:31010]
              at org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:543) ~[drill-common-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
              at org.apache.drill.exec.work.fragment.FragmentExecutor.sendFinalState(FragmentExecutor.java:318) [drill-java-exec-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
              at org.apache.drill.exec.work.fragment.FragmentExecutor.cleanup(FragmentExecutor.java:185) [drill-java-exec-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
              at org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:287) [drill-java-exec-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
              at org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38) [drill-common-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [na:1.7.0_45]
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_45]
              at java.lang.Thread.run(Thread.java:744) [na:1.7.0_45]
      Caused by: java.lang.IllegalStateException: Failure while reading vector.  Expected vector class of org.apache.drill.exec.vector.NullableIntVector but was holding vector class org.apache.drill.exec.vector.NullableVarBinaryVector, field= $f0(VARBINARY:OPTIONAL)[$bits$(UINT1:REQUIRED), $f0(VARBINARY:OPTIONAL)[$offsets$(UINT4:REQUIRED)]]
              at org.apache.drill.exec.record.VectorContainer.getValueAccessorById(VectorContainer.java:290) ~[drill-java-exec-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
              at org.apache.drill.exec.record.RecordBatchLoader.getValueAccessorById(RecordBatchLoader.java:178) ~[drill-java-exec-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
              at org.apache.drill.exec.physical.impl.unorderedreceiver.UnorderedReceiverBatch.getValueAccessorById(UnorderedReceiverBatch.java:135) ~[drill-java-exec-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
              at org.apache.drill.exec.test.generated.PartitionerGen953$OutgoingRecordBatch.doSetup(PartitionerTemplate.java:64) ~[na:na]
              at org.apache.drill.exec.test.generated.PartitionerGen953$OutgoingRecordBatch.initializeBatch(PartitionerTemplate.java:358) ~[na:na]
              at org.apache.drill.exec.test.generated.PartitionerGen953.flushOutgoingBatches(PartitionerTemplate.java:163) ~[na:na]
              at org.apache.drill.exec.physical.impl.partitionsender.PartitionerDecorator$FlushBatchesHandlingClass.execute(PartitionerDecorator.java:266) ~[drill-java-exec-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
              at org.apache.drill.exec.physical.impl.partitionsender.PartitionerDecorator.executeMethodLogic(PartitionerDecorator.java:138) ~[drill-java-exec-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
              at org.apache.drill.exec.physical.impl.partitionsender.PartitionerDecorator.flushOutgoingBatches(PartitionerDecorator.java:82) ~[drill-java-exec-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
              at org.apache.drill.exec.physical.impl.partitionsender.PartitionSenderRootExec.innerNext(PartitionSenderRootExec.java:183) ~[drill-java-exec-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
              at org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:94) ~[drill-java-exec-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
              at org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentExecutor.java:257) ~[drill-java-exec-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
              at org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentExecutor.java:251) ~[drill-java-exec-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
              at java.security.AccessController.doPrivileged(Native Method) ~[na:1.7.0_45]
              at javax.security.auth.Subject.doAs(Subject.java:415) ~[na:1.7.0_45]
              at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1595) ~[hadoop-common-2.7.0-mapr-1602.jar:na]
              at org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:251) [drill-java-exec-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
              ... 4 common frames omitted
      

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                khfaraaz Khurram Faraaz
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: