[HCATALOG-151] HBaseInputFormat assumes hbase table name is hcat table name

• Type: Bug
• Status: Closed
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 0.3
• Fix Version/s: 0.3
• Component/s: hbase
• Labels: hbase
• Assignee: Vandana Ayyalasomayajula
• Reporter: David Capwell
• Created: 01/Nov/11 18:16
• Updated: 17/May/12 01:20
• Resolved: 04/Nov/11 00:23

Description

If you create a db then a table within that db, the hbase default name is $dbName.$tableName. When you launch a job that reads from this table, the HBaseInputformat assumes the table name is just $tableName

CREATE DATABASE readhcatdatatest_1320170553442 LOCATION '/tmp/hcat_hbase;

USE readhcatdatatest_1320170553442 ; CREATE TABLE readhcatdatatest (key string, value map<string, string>) STORED BY 'org.apache.hcatalog.hbase.HBaseHCatStorageHandler' TBLPROPERTIES ('hbase.columns.mapping'=':key,cf:','hcat.osd'='org.apache.hcatalog.hbase.HBaseOutputStorageDriver','hcat.isd'='org.apache.hcatalog.hbase.HBaseInputStorageDriver');

...

1. client mr job
   InputJobInfo inputJobInfo = InputJobInfo.create(dbName, tableName, null,
   job.getConfiguration().get(HCatConstants.HCAT_METASTORE_URI),
   job.getConfiguration().get(HCatConstants.HCAT_METASTORE_PRINCIPAL));
   HCatInputFormat.setInput(job, inputJobInfo);

dbName = readhcatdatatest_1320170553442
tableName = readhcatdatatest

Here is the error you get:

ERROR 18:02:45,586 [pool-1-thread-1] TableInputFormat:93 - org.apache.hadoop.hbase.TableNotFoundException: readhcatdatatest
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:724)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:593)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:558)
at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:172)
at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:146)
at org.apache.hadoop.hbase.mapreduce.TableInputFormat.setConf(TableInputFormat.java:91)
at org.apache.hcatalog.hbase.HBaseInputFormat.setConf(HBaseInputFormat.java:89)
at org.apache.hcatalog.hbase.HBaseInputStorageDriver.getInputFormat(HBaseInputStorageDriver.java:106)
at org.apache.hcatalog.mapreduce.HCatBaseInputFormat.getSplits(HCatBaseInputFormat.java:114)
at org.apache.hadoop.mapred.JobClient.writeNewSplits(JobClient.java:919)
at org.apache.hadoop.mapred.JobClient.writeSplits(JobClient.java:936)
at org.apache.hadoop.mapred.JobClient.access$500(JobClient.java:170)
at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:854)
at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:807)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059)
at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:807)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:465)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:495)

This is blocking all testing

Issue Links

• is related to: HCATALOG-157 HBaseOutputFormat assumes hbase table name is hcat table name