HCatalog
  1. HCatalog
  2. HCATALOG-301

Reading from a table created with upper case table name throws error

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.4
    • Fix Version/s: 0.4.1
    • Component/s: hbase
    • Labels:
      None

      Description

      if you create a new table then try to read from it, you will get the exception below. Ether the error should propagate up and be clear what happened (no data to read) or the job should run with empty input (fail fast imo).

      ERROR 18:29:05,057 [main] Launcher:171 - Backend error message during job
      submission
      org.apache.pig.backend.executionengine.ExecException: ERROR 2118:
      java.lang.IllegalStateException: Error while configuring job properties
      at
      org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.getSplits(PigInputFormat.java:280)
      at
      org.apache.hadoop.mapred.JobClient.writeNewSplits(JobClient.java:962)
      [...]
      Caused by: org.apache.zookeeper.KeeperException$NoNodeException:
      KeeperErrorCode = NoNode for
      /revision-management/data/hcatreadhbasetabletest_85330477246989847.hcatreadhbasetabletest_85330477285774847
      at
      org.apache.zookeeper.KeeperException.create(KeeperException.java:111)
      at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
      at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1448)
      at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1476)
      at
      org.apache.hcatalog.hbase.snapshot.ZKUtil.getColumnFamiliesOfTable(ZKUtil.java:130)
      ... 25 more

      1. HCATALOG-301.patch
        13 kB
        Rohini Palaniswamy
      2. HCATALOG-301-2.patch
        30 kB
        Rohini Palaniswamy

        Activity

        Hide
        Vandana Ayyalasomayajula added a comment -

        Hey David, could you post the exact command that lead to this error ?
        Thanks, Vandana

        Show
        Vandana Ayyalasomayajula added a comment - Hey David, could you post the exact command that lead to this error ? Thanks, Vandana
        Hide
        David Capwell added a comment -

        [INFO] 03:15:06,580 [main] CLIHCatClient:164 - Executing cmd: CREATE DATABASE IF NOT EXISTS HCatReadHBaseTableTest_85362060752406847 LOCATION '/tmp/hcat-qe/HCatReadHBaseTableTest_85362060752406847'

        [INFO] 03:15:10,717 [main] HCatReadHBaseTableTest:73 - Creating Table CLITable{name=HCatReadHBaseTableTest_85362060781502847, database=CLIDatabase

        {name=HCatReadHBaseTableTest_85362060752406847, description=null, location=/tmp/hcat-qe/HCatReadHBaseTableTest_85362060752406847, properties=null}

        , type=INTERNAL, properties=

        {hbase.columns.mapping=:key,cf:}

        , location=null, columns=[CLIField

        {name=key, type=string}

        , CLIField

        {name=value, type=map<string, string>}

        ], storage-engine=CLIStorageEngine{name=org.apache.hcatalog.hbase.HBaseHCatStorageHandler, type=HANDLER, serde-properties=null}}

        [INFO] 03:15:10,718 [main] CLIHCatClient:164 - Executing cmd: USE HCatReadHBaseTableTest_85362060752406847; CREATE TABLE IF NOT EXISTS HCatReadHBaseTableTest_85362060781502847 (key string,value map<string, string>) STORED BY 'org.apache.hcatalog.hbase.HBaseHCatStorageHandler' TBLPROPERTIES ('hbase.columns.mapping'=':key,cf:')

        The funny thing, if you do an ls of /revion-managment/data it will show the path there, but if you ls the path it says its not there.

        Show
        David Capwell added a comment - [INFO] 03:15:06,580 [main] CLIHCatClient:164 - Executing cmd: CREATE DATABASE IF NOT EXISTS HCatReadHBaseTableTest_85362060752406847 LOCATION '/tmp/hcat-qe/HCatReadHBaseTableTest_85362060752406847' [INFO] 03:15:10,717 [main] HCatReadHBaseTableTest:73 - Creating Table CLITable{name=HCatReadHBaseTableTest_85362060781502847, database=CLIDatabase {name=HCatReadHBaseTableTest_85362060752406847, description=null, location=/tmp/hcat-qe/HCatReadHBaseTableTest_85362060752406847, properties=null} , type=INTERNAL, properties= {hbase.columns.mapping=:key,cf:} , location=null, columns=[CLIField {name=key, type=string} , CLIField {name=value, type=map<string, string>} ], storage-engine=CLIStorageEngine{name=org.apache.hcatalog.hbase.HBaseHCatStorageHandler, type=HANDLER, serde-properties=null}} [INFO] 03:15:10,718 [main] CLIHCatClient:164 - Executing cmd: USE HCatReadHBaseTableTest_85362060752406847; CREATE TABLE IF NOT EXISTS HCatReadHBaseTableTest_85362060781502847 (key string,value map<string, string>) STORED BY 'org.apache.hcatalog.hbase.HBaseHCatStorageHandler' TBLPROPERTIES ('hbase.columns.mapping'=':key,cf:') The funny thing, if you do an ls of /revion-managment/data it will show the path there, but if you ls the path it says its not there.
        Hide
        Rohini Palaniswamy added a comment -

        HCat keeps all the table names in lowercase even if the table was created with upper case. But when we setup the znodes for revision mgmt during table creation we don't make it lower case. That is causing the problem. Will fix after confirming with team that HCat will always handle database and table names in
        lower case.

        Show
        Rohini Palaniswamy added a comment - HCat keeps all the table names in lowercase even if the table was created with upper case. But when we setup the znodes for revision mgmt during table creation we don't make it lower case. That is causing the problem. Will fix after confirming with team that HCat will always handle database and table names in lower case.
        Hide
        Rohini Palaniswamy added a comment -

        Introduced a new table property "hbase.table.name" similar to Hive to store hbase table names retaining the case. This will be used to access hbase and zookeeper which are case sensitive from within the HbaseHCatStorageHandler.

        Show
        Rohini Palaniswamy added a comment - Introduced a new table property "hbase.table.name" similar to Hive to store hbase table names retaining the case. This will be used to access hbase and zookeeper which are case sensitive from within the HbaseHCatStorageHandler.
        Hide
        jiraposter@reviews.apache.org added a comment -

        -----------------------------------------------------------
        This is an automatically generated e-mail. To reply, visit:
        https://reviews.apache.org/r/4416/
        -----------------------------------------------------------

        Review request for Francis Liu and Vandana Ayyalasomayajula.

        Summary
        -------

        Introduced a new table property "hbase.table.name" similar to Hive to store hbase table names retaining the case. This will be used to access hbase and zookeeper which are case sensitive from within the HbaseHCatStorageHandler.

        Added two new APIs setupTable and dropTable to RevisionManager and removed public APIs for setting up and dropping znodes in ZKBasedRevisionManager.

        This addresses bug HCATALOG-301.
        https://issues.apache.org/jira/browse/HCATALOG-301

        Diffs


        http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/java/org/apache/hcatalog/hbase/HBaseHCatStorageHandler.java 1303016
        http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/java/org/apache/hcatalog/hbase/snapshot/RevisionManager.java 1303016
        http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/java/org/apache/hcatalog/hbase/snapshot/ZKBasedRevisionManager.java 1303016
        http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/test/org/apache/hcatalog/hbase/TestHBaseDirectOutputFormat.java 1303016
        http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/test/org/apache/hcatalog/hbase/TestHBaseHCatStorageHandler.java 1303016
        http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/test/org/apache/hcatalog/hbase/TestHBaseInputFormat.java 1303016

        Diff: https://reviews.apache.org/r/4416/diff

        Testing
        -------

        Unit tests added

        Thanks,

        Rohini

        Show
        jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4416/ ----------------------------------------------------------- Review request for Francis Liu and Vandana Ayyalasomayajula. Summary ------- Introduced a new table property "hbase.table.name" similar to Hive to store hbase table names retaining the case. This will be used to access hbase and zookeeper which are case sensitive from within the HbaseHCatStorageHandler. Added two new APIs setupTable and dropTable to RevisionManager and removed public APIs for setting up and dropping znodes in ZKBasedRevisionManager. This addresses bug HCATALOG-301 . https://issues.apache.org/jira/browse/HCATALOG-301 Diffs http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/java/org/apache/hcatalog/hbase/HBaseHCatStorageHandler.java 1303016 http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/java/org/apache/hcatalog/hbase/snapshot/RevisionManager.java 1303016 http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/java/org/apache/hcatalog/hbase/snapshot/ZKBasedRevisionManager.java 1303016 http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/test/org/apache/hcatalog/hbase/TestHBaseDirectOutputFormat.java 1303016 http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/test/org/apache/hcatalog/hbase/TestHBaseHCatStorageHandler.java 1303016 http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/test/org/apache/hcatalog/hbase/TestHBaseInputFormat.java 1303016 Diff: https://reviews.apache.org/r/4416/diff Testing ------- Unit tests added Thanks, Rohini
        Hide
        jiraposter@reviews.apache.org added a comment -

        -----------------------------------------------------------
        This is an automatically generated e-mail. To reply, visit:
        https://reviews.apache.org/r/4416/
        -----------------------------------------------------------

        (Updated 2012-03-21 18:18:10.028032)

        Review request for Francis Liu and Vandana Ayyalasomayajula.

        Changes
        -------

        Updating patch with fix to create table. Changes to setConf() for map jobs has broken create table.

        Summary
        -------

        Introduced a new table property "hbase.table.name" similar to Hive to store hbase table names retaining the case. This will be used to access hbase and zookeeper which are case sensitive from within the HbaseHCatStorageHandler.

        Added two new APIs setupTable and dropTable to RevisionManager and removed public APIs for setting up and dropping znodes in ZKBasedRevisionManager.

        This addresses bug HCATALOG-301.
        https://issues.apache.org/jira/browse/HCATALOG-301

        Diffs (updated)


        http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/java/org/apache/hcatalog/hbase/HBaseHCatStorageHandler.java 1303371
        http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/java/org/apache/hcatalog/hbase/snapshot/RevisionManager.java 1303371
        http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/java/org/apache/hcatalog/hbase/snapshot/ZKBasedRevisionManager.java 1303371
        http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/test/org/apache/hcatalog/hbase/TestHBaseBulkOutputFormat.java 1303371
        http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/test/org/apache/hcatalog/hbase/TestHBaseDirectOutputFormat.java 1303371
        http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/test/org/apache/hcatalog/hbase/TestHBaseHCatStorageHandler.java 1303371
        http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/test/org/apache/hcatalog/hbase/TestHBaseInputFormat.java 1303371

        Diff: https://reviews.apache.org/r/4416/diff

        Testing
        -------

        Unit tests added

        Thanks,

        Rohini

        Show
        jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4416/ ----------------------------------------------------------- (Updated 2012-03-21 18:18:10.028032) Review request for Francis Liu and Vandana Ayyalasomayajula. Changes ------- Updating patch with fix to create table. Changes to setConf() for map jobs has broken create table. Summary ------- Introduced a new table property "hbase.table.name" similar to Hive to store hbase table names retaining the case. This will be used to access hbase and zookeeper which are case sensitive from within the HbaseHCatStorageHandler. Added two new APIs setupTable and dropTable to RevisionManager and removed public APIs for setting up and dropping znodes in ZKBasedRevisionManager. This addresses bug HCATALOG-301 . https://issues.apache.org/jira/browse/HCATALOG-301 Diffs (updated) http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/java/org/apache/hcatalog/hbase/HBaseHCatStorageHandler.java 1303371 http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/java/org/apache/hcatalog/hbase/snapshot/RevisionManager.java 1303371 http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/java/org/apache/hcatalog/hbase/snapshot/ZKBasedRevisionManager.java 1303371 http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/test/org/apache/hcatalog/hbase/TestHBaseBulkOutputFormat.java 1303371 http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/test/org/apache/hcatalog/hbase/TestHBaseDirectOutputFormat.java 1303371 http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/test/org/apache/hcatalog/hbase/TestHBaseHCatStorageHandler.java 1303371 http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/test/org/apache/hcatalog/hbase/TestHBaseInputFormat.java 1303371 Diff: https://reviews.apache.org/r/4416/diff Testing ------- Unit tests added Thanks, Rohini
        Hide
        jiraposter@reviews.apache.org added a comment -

        -----------------------------------------------------------
        This is an automatically generated e-mail. To reply, visit:
        https://reviews.apache.org/r/4416/#review6177
        -----------------------------------------------------------

        http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/java/org/apache/hcatalog/hbase/HBaseHCatStorageHandler.java
        <https://reviews.apache.org/r/4416/#comment13299>

        The copyOfConf is based on jobConf, which has hbase defaults in it. So you don't have to do HBaseConfiguration.addHbaseResources(copyOfConf); again.

        http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/java/org/apache/hcatalog/hbase/HBaseHCatStorageHandler.java
        <https://reviews.apache.org/r/4416/#comment13300>

        same as above comment.

        http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/java/org/apache/hcatalog/hbase/HBaseHCatStorageHandler.java
        <https://reviews.apache.org/r/4416/#comment13301>

        Do we need this additional method ? Can we do, getFullyQualifiedHBaseTableName(HCatTableInfo.valueOf(tbl).

        Also, can in my opinion we should be using only lower case names. So "deafult.Mytable" should be equivalent to "default.mytable". The nodes in the revision manager will have the fully qualified name ( in lower case).

        http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/java/org/apache/hcatalog/hbase/HBaseHCatStorageHandler.java
        <https://reviews.apache.org/r/4416/#comment13302>

        This method should convert the dbname.tablename to lower case. Any user should be using this method get obtain table names, so that we do not run into errors.

        • Vandana

        On 2012-03-21 18:18:10, Rohini Palaniswamy wrote:

        -----------------------------------------------------------

        This is an automatically generated e-mail. To reply, visit:

        https://reviews.apache.org/r/4416/

        -----------------------------------------------------------

        (Updated 2012-03-21 18:18:10)

        Review request for Francis Liu and Vandana Ayyalasomayajula.

        Summary

        -------

        Introduced a new table property "hbase.table.name" similar to Hive to store hbase table names retaining the case. This will be used to access hbase and zookeeper which are case sensitive from within the HbaseHCatStorageHandler.

        Added two new APIs setupTable and dropTable to RevisionManager and removed public APIs for setting up and dropping znodes in ZKBasedRevisionManager.

        This addresses bug HCATALOG-301.

        https://issues.apache.org/jira/browse/HCATALOG-301

        Diffs

        -----

        http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/java/org/apache/hcatalog/hbase/HBaseHCatStorageHandler.java 1303371

        http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/java/org/apache/hcatalog/hbase/snapshot/RevisionManager.java 1303371

        http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/java/org/apache/hcatalog/hbase/snapshot/ZKBasedRevisionManager.java 1303371

        http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/test/org/apache/hcatalog/hbase/TestHBaseBulkOutputFormat.java 1303371

        http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/test/org/apache/hcatalog/hbase/TestHBaseDirectOutputFormat.java 1303371

        http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/test/org/apache/hcatalog/hbase/TestHBaseHCatStorageHandler.java 1303371

        http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/test/org/apache/hcatalog/hbase/TestHBaseInputFormat.java 1303371

        Diff: https://reviews.apache.org/r/4416/diff

        Testing

        -------

        Unit tests added

        Thanks,

        Rohini

        Show
        jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4416/#review6177 ----------------------------------------------------------- http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/java/org/apache/hcatalog/hbase/HBaseHCatStorageHandler.java < https://reviews.apache.org/r/4416/#comment13299 > The copyOfConf is based on jobConf, which has hbase defaults in it. So you don't have to do HBaseConfiguration.addHbaseResources(copyOfConf); again. http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/java/org/apache/hcatalog/hbase/HBaseHCatStorageHandler.java < https://reviews.apache.org/r/4416/#comment13300 > same as above comment. http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/java/org/apache/hcatalog/hbase/HBaseHCatStorageHandler.java < https://reviews.apache.org/r/4416/#comment13301 > Do we need this additional method ? Can we do, getFullyQualifiedHBaseTableName(HCatTableInfo.valueOf(tbl). Also, can in my opinion we should be using only lower case names. So "deafult.Mytable" should be equivalent to "default.mytable". The nodes in the revision manager will have the fully qualified name ( in lower case). http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/java/org/apache/hcatalog/hbase/HBaseHCatStorageHandler.java < https://reviews.apache.org/r/4416/#comment13302 > This method should convert the dbname.tablename to lower case. Any user should be using this method get obtain table names, so that we do not run into errors. Vandana On 2012-03-21 18:18:10, Rohini Palaniswamy wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4416/ ----------------------------------------------------------- (Updated 2012-03-21 18:18:10) Review request for Francis Liu and Vandana Ayyalasomayajula. Summary ------- Introduced a new table property "hbase.table.name" similar to Hive to store hbase table names retaining the case. This will be used to access hbase and zookeeper which are case sensitive from within the HbaseHCatStorageHandler. Added two new APIs setupTable and dropTable to RevisionManager and removed public APIs for setting up and dropping znodes in ZKBasedRevisionManager. This addresses bug HCATALOG-301 . https://issues.apache.org/jira/browse/HCATALOG-301 Diffs ----- http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/java/org/apache/hcatalog/hbase/HBaseHCatStorageHandler.java 1303371 http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/java/org/apache/hcatalog/hbase/snapshot/RevisionManager.java 1303371 http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/java/org/apache/hcatalog/hbase/snapshot/ZKBasedRevisionManager.java 1303371 http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/test/org/apache/hcatalog/hbase/TestHBaseBulkOutputFormat.java 1303371 http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/test/org/apache/hcatalog/hbase/TestHBaseDirectOutputFormat.java 1303371 http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/test/org/apache/hcatalog/hbase/TestHBaseHCatStorageHandler.java 1303371 http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/test/org/apache/hcatalog/hbase/TestHBaseInputFormat.java 1303371 Diff: https://reviews.apache.org/r/4416/diff Testing ------- Unit tests added Thanks, Rohini
        Hide
        jiraposter@reviews.apache.org added a comment -

        On 2012-03-21 19:08:35, Vandana Ayyalasomayajula wrote:

        > http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/java/org/apache/hcatalog/hbase/HBaseHCatStorageHandler.java, line 433

        > <https://reviews.apache.org/r/4416/diff/2/?file=94212#file94212line433>

        >

        > This method should convert the dbname.tablename to lower case. Any user should be using this method get obtain table names, so that we do not run into errors.

        Need to maintain case for working with external hbase tables that were created with uppercase name.

        On 2012-03-21 19:08:35, Vandana Ayyalasomayajula wrote:

        > http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/java/org/apache/hcatalog/hbase/HBaseHCatStorageHandler.java, line 417

        > <https://reviews.apache.org/r/4416/diff/2/?file=94212#file94212line417>

        >

        > Do we need this additional method ? Can we do, getFullyQualifiedHBaseTableName(HCatTableInfo.valueOf(tbl).

        >

        > Also, can in my opinion we should be using only lower case names. So "deafult.Mytable" should be equivalent to "default.mytable". The nodes in the revision manager will have the fully qualified name ( in lower case).

        Can do. But it will save on the additional copying and construction of objects. We already have too much of that in the code.

        We need to maintain the case of the hbase tables. If we are handling only lower case then we cannot work with external hbase tables.

        On 2012-03-21 19:08:35, Vandana Ayyalasomayajula wrote:

        > http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/java/org/apache/hcatalog/hbase/HBaseHCatStorageHandler.java, line 103

        > <https://reviews.apache.org/r/4416/diff/2/?file=94212#file94212line103>

        >

        > The copyOfConf is based on jobConf, which has hbase defaults in it. So you don't have to do HBaseConfiguration.addHbaseResources(copyOfConf); again.

        The jobConf does not have the defaults added in them. If the defaults are added to the the jobConf directly then any overrides set in the job for hbase will not apply. For eg: The unit tests have different zookeeper port set in jobConf. But if hbase-default.xml is added to it, it overrides it with default port. That's why resorting to this.

        • Rohini

        -----------------------------------------------------------
        This is an automatically generated e-mail. To reply, visit:
        https://reviews.apache.org/r/4416/#review6177
        -----------------------------------------------------------

        On 2012-03-21 18:18:10, Rohini Palaniswamy wrote:

        -----------------------------------------------------------

        This is an automatically generated e-mail. To reply, visit:

        https://reviews.apache.org/r/4416/

        -----------------------------------------------------------

        (Updated 2012-03-21 18:18:10)

        Review request for Francis Liu and Vandana Ayyalasomayajula.

        Summary

        -------

        Introduced a new table property "hbase.table.name" similar to Hive to store hbase table names retaining the case. This will be used to access hbase and zookeeper which are case sensitive from within the HbaseHCatStorageHandler.

        Added two new APIs setupTable and dropTable to RevisionManager and removed public APIs for setting up and dropping znodes in ZKBasedRevisionManager.

        This addresses bug HCATALOG-301.

        https://issues.apache.org/jira/browse/HCATALOG-301

        Diffs

        -----

        http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/java/org/apache/hcatalog/hbase/HBaseHCatStorageHandler.java 1303371

        http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/java/org/apache/hcatalog/hbase/snapshot/RevisionManager.java 1303371

        http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/java/org/apache/hcatalog/hbase/snapshot/ZKBasedRevisionManager.java 1303371

        http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/test/org/apache/hcatalog/hbase/TestHBaseBulkOutputFormat.java 1303371

        http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/test/org/apache/hcatalog/hbase/TestHBaseDirectOutputFormat.java 1303371

        http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/test/org/apache/hcatalog/hbase/TestHBaseHCatStorageHandler.java 1303371

        http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/test/org/apache/hcatalog/hbase/TestHBaseInputFormat.java 1303371

        Diff: https://reviews.apache.org/r/4416/diff

        Testing

        -------

        Unit tests added

        Thanks,

        Rohini

        Show
        jiraposter@reviews.apache.org added a comment - On 2012-03-21 19:08:35, Vandana Ayyalasomayajula wrote: > http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/java/org/apache/hcatalog/hbase/HBaseHCatStorageHandler.java , line 433 > < https://reviews.apache.org/r/4416/diff/2/?file=94212#file94212line433 > > > This method should convert the dbname.tablename to lower case. Any user should be using this method get obtain table names, so that we do not run into errors. Need to maintain case for working with external hbase tables that were created with uppercase name. On 2012-03-21 19:08:35, Vandana Ayyalasomayajula wrote: > http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/java/org/apache/hcatalog/hbase/HBaseHCatStorageHandler.java , line 417 > < https://reviews.apache.org/r/4416/diff/2/?file=94212#file94212line417 > > > Do we need this additional method ? Can we do, getFullyQualifiedHBaseTableName(HCatTableInfo.valueOf(tbl). > > Also, can in my opinion we should be using only lower case names. So "deafult.Mytable" should be equivalent to "default.mytable". The nodes in the revision manager will have the fully qualified name ( in lower case). Can do. But it will save on the additional copying and construction of objects. We already have too much of that in the code. We need to maintain the case of the hbase tables. If we are handling only lower case then we cannot work with external hbase tables. On 2012-03-21 19:08:35, Vandana Ayyalasomayajula wrote: > http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/java/org/apache/hcatalog/hbase/HBaseHCatStorageHandler.java , line 103 > < https://reviews.apache.org/r/4416/diff/2/?file=94212#file94212line103 > > > The copyOfConf is based on jobConf, which has hbase defaults in it. So you don't have to do HBaseConfiguration.addHbaseResources(copyOfConf); again. The jobConf does not have the defaults added in them. If the defaults are added to the the jobConf directly then any overrides set in the job for hbase will not apply. For eg: The unit tests have different zookeeper port set in jobConf. But if hbase-default.xml is added to it, it overrides it with default port. That's why resorting to this. Rohini ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4416/#review6177 ----------------------------------------------------------- On 2012-03-21 18:18:10, Rohini Palaniswamy wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4416/ ----------------------------------------------------------- (Updated 2012-03-21 18:18:10) Review request for Francis Liu and Vandana Ayyalasomayajula. Summary ------- Introduced a new table property "hbase.table.name" similar to Hive to store hbase table names retaining the case. This will be used to access hbase and zookeeper which are case sensitive from within the HbaseHCatStorageHandler. Added two new APIs setupTable and dropTable to RevisionManager and removed public APIs for setting up and dropping znodes in ZKBasedRevisionManager. This addresses bug HCATALOG-301 . https://issues.apache.org/jira/browse/HCATALOG-301 Diffs ----- http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/java/org/apache/hcatalog/hbase/HBaseHCatStorageHandler.java 1303371 http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/java/org/apache/hcatalog/hbase/snapshot/RevisionManager.java 1303371 http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/java/org/apache/hcatalog/hbase/snapshot/ZKBasedRevisionManager.java 1303371 http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/test/org/apache/hcatalog/hbase/TestHBaseBulkOutputFormat.java 1303371 http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/test/org/apache/hcatalog/hbase/TestHBaseDirectOutputFormat.java 1303371 http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/test/org/apache/hcatalog/hbase/TestHBaseHCatStorageHandler.java 1303371 http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/test/org/apache/hcatalog/hbase/TestHBaseInputFormat.java 1303371 Diff: https://reviews.apache.org/r/4416/diff Testing ------- Unit tests added Thanks, Rohini
        Hide
        jiraposter@reviews.apache.org added a comment -

        -----------------------------------------------------------
        This is an automatically generated e-mail. To reply, visit:
        https://reviews.apache.org/r/4416/#review6235
        -----------------------------------------------------------

        http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/java/org/apache/hcatalog/hbase/HBaseHCatStorageHandler.java
        <https://reviews.apache.org/r/4416/#comment13457>

        Instead of depending on unintentional behvaior why not just make it:

        addToken(copyOfConf)
        if(jobConf instanceof JobConf)

        { //copy credential contents }

        http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/java/org/apache/hcatalog/hbase/HBaseHCatStorageHandler.java
        <https://reviews.apache.org/r/4416/#comment13690>

        Add comment on why this is needed. It's a bit hard to discern just by reading the code.

        http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/java/org/apache/hcatalog/hbase/snapshot/RevisionManager.java
        <https://reviews.apache.org/r/4416/#comment13462>

        createTable sounds better?

        • Francis

        On 2012-03-21 18:18:10, Rohini Palaniswamy wrote:

        -----------------------------------------------------------

        This is an automatically generated e-mail. To reply, visit:

        https://reviews.apache.org/r/4416/

        -----------------------------------------------------------

        (Updated 2012-03-21 18:18:10)

        Review request for Francis Liu and Vandana Ayyalasomayajula.

        Summary

        -------

        Introduced a new table property "hbase.table.name" similar to Hive to store hbase table names retaining the case. This will be used to access hbase and zookeeper which are case sensitive from within the HbaseHCatStorageHandler.

        Added two new APIs setupTable and dropTable to RevisionManager and removed public APIs for setting up and dropping znodes in ZKBasedRevisionManager.

        This addresses bug HCATALOG-301.

        https://issues.apache.org/jira/browse/HCATALOG-301

        Diffs

        -----

        http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/java/org/apache/hcatalog/hbase/HBaseHCatStorageHandler.java 1303371

        http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/java/org/apache/hcatalog/hbase/snapshot/RevisionManager.java 1303371

        http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/java/org/apache/hcatalog/hbase/snapshot/ZKBasedRevisionManager.java 1303371

        http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/test/org/apache/hcatalog/hbase/TestHBaseBulkOutputFormat.java 1303371

        http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/test/org/apache/hcatalog/hbase/TestHBaseDirectOutputFormat.java 1303371

        http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/test/org/apache/hcatalog/hbase/TestHBaseHCatStorageHandler.java 1303371

        http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/test/org/apache/hcatalog/hbase/TestHBaseInputFormat.java 1303371

        Diff: https://reviews.apache.org/r/4416/diff

        Testing

        -------

        Unit tests added

        Thanks,

        Rohini

        Show
        jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4416/#review6235 ----------------------------------------------------------- http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/java/org/apache/hcatalog/hbase/HBaseHCatStorageHandler.java < https://reviews.apache.org/r/4416/#comment13457 > Instead of depending on unintentional behvaior why not just make it: addToken(copyOfConf) if(jobConf instanceof JobConf) { //copy credential contents } http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/java/org/apache/hcatalog/hbase/HBaseHCatStorageHandler.java < https://reviews.apache.org/r/4416/#comment13690 > Add comment on why this is needed. It's a bit hard to discern just by reading the code. http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/java/org/apache/hcatalog/hbase/snapshot/RevisionManager.java < https://reviews.apache.org/r/4416/#comment13462 > createTable sounds better? Francis On 2012-03-21 18:18:10, Rohini Palaniswamy wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4416/ ----------------------------------------------------------- (Updated 2012-03-21 18:18:10) Review request for Francis Liu and Vandana Ayyalasomayajula. Summary ------- Introduced a new table property "hbase.table.name" similar to Hive to store hbase table names retaining the case. This will be used to access hbase and zookeeper which are case sensitive from within the HbaseHCatStorageHandler. Added two new APIs setupTable and dropTable to RevisionManager and removed public APIs for setting up and dropping znodes in ZKBasedRevisionManager. This addresses bug HCATALOG-301 . https://issues.apache.org/jira/browse/HCATALOG-301 Diffs ----- http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/java/org/apache/hcatalog/hbase/HBaseHCatStorageHandler.java 1303371 http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/java/org/apache/hcatalog/hbase/snapshot/RevisionManager.java 1303371 http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/java/org/apache/hcatalog/hbase/snapshot/ZKBasedRevisionManager.java 1303371 http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/test/org/apache/hcatalog/hbase/TestHBaseBulkOutputFormat.java 1303371 http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/test/org/apache/hcatalog/hbase/TestHBaseDirectOutputFormat.java 1303371 http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/test/org/apache/hcatalog/hbase/TestHBaseHCatStorageHandler.java 1303371 http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/test/org/apache/hcatalog/hbase/TestHBaseInputFormat.java 1303371 Diff: https://reviews.apache.org/r/4416/diff Testing ------- Unit tests added Thanks, Rohini
        Hide
        jiraposter@reviews.apache.org added a comment -

        -----------------------------------------------------------
        This is an automatically generated e-mail. To reply, visit:
        https://reviews.apache.org/r/4416/
        -----------------------------------------------------------

        (Updated 2012-03-24 00:03:54.806109)

        Review request for Francis Liu and Vandana Ayyalasomayajula.

        Changes
        -------

        Updated patch with
        1) Changes needed for HCATALOG-310 (RM as a coprocessor).
        2) Incorporated Francis's review comments

        • Renamed setupTable to createTable
        • Changed the behaviour to create the hbase table in lower case by default if not specified by hbase.table.name property.

        Summary
        -------

        Introduced a new table property "hbase.table.name" similar to Hive to store hbase table names retaining the case. This will be used to access hbase and zookeeper which are case sensitive from within the HbaseHCatStorageHandler.

        Added two new APIs setupTable and dropTable to RevisionManager and removed public APIs for setting up and dropping znodes in ZKBasedRevisionManager.

        This addresses bug HCATALOG-301.
        https://issues.apache.org/jira/browse/HCATALOG-301

        Diffs (updated)


        http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/java/org/apache/hcatalog/hbase/HBaseHCatStorageHandler.java 1304584
        http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/java/org/apache/hcatalog/hbase/snapshot/RevisionManager.java 1304584
        http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/java/org/apache/hcatalog/hbase/snapshot/RevisionManagerEndpoint.java 1304584
        http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/java/org/apache/hcatalog/hbase/snapshot/RevisionManagerEndpointClient.java 1304584
        http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/java/org/apache/hcatalog/hbase/snapshot/ZKBasedRevisionManager.java 1304584
        http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/test/org/apache/hcatalog/hbase/TestHBaseBulkOutputFormat.java 1304584
        http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/test/org/apache/hcatalog/hbase/TestHBaseDirectOutputFormat.java 1304584
        http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/test/org/apache/hcatalog/hbase/TestHBaseHCatStorageHandler.java 1304584
        http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/test/org/apache/hcatalog/hbase/TestHBaseInputFormat.java 1304584
        http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/test/org/apache/hcatalog/hbase/snapshot/TestRevisionManagerEndpoint.java 1304584

        Diff: https://reviews.apache.org/r/4416/diff

        Testing
        -------

        Unit tests added

        Thanks,

        Rohini

        Show
        jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4416/ ----------------------------------------------------------- (Updated 2012-03-24 00:03:54.806109) Review request for Francis Liu and Vandana Ayyalasomayajula. Changes ------- Updated patch with 1) Changes needed for HCATALOG-310 (RM as a coprocessor). 2) Incorporated Francis's review comments Renamed setupTable to createTable Changed the behaviour to create the hbase table in lower case by default if not specified by hbase.table.name property. Summary ------- Introduced a new table property "hbase.table.name" similar to Hive to store hbase table names retaining the case. This will be used to access hbase and zookeeper which are case sensitive from within the HbaseHCatStorageHandler. Added two new APIs setupTable and dropTable to RevisionManager and removed public APIs for setting up and dropping znodes in ZKBasedRevisionManager. This addresses bug HCATALOG-301 . https://issues.apache.org/jira/browse/HCATALOG-301 Diffs (updated) http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/java/org/apache/hcatalog/hbase/HBaseHCatStorageHandler.java 1304584 http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/java/org/apache/hcatalog/hbase/snapshot/RevisionManager.java 1304584 http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/java/org/apache/hcatalog/hbase/snapshot/RevisionManagerEndpoint.java 1304584 http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/java/org/apache/hcatalog/hbase/snapshot/RevisionManagerEndpointClient.java 1304584 http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/java/org/apache/hcatalog/hbase/snapshot/ZKBasedRevisionManager.java 1304584 http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/test/org/apache/hcatalog/hbase/TestHBaseBulkOutputFormat.java 1304584 http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/test/org/apache/hcatalog/hbase/TestHBaseDirectOutputFormat.java 1304584 http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/test/org/apache/hcatalog/hbase/TestHBaseHCatStorageHandler.java 1304584 http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/test/org/apache/hcatalog/hbase/TestHBaseInputFormat.java 1304584 http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/test/org/apache/hcatalog/hbase/snapshot/TestRevisionManagerEndpoint.java 1304584 Diff: https://reviews.apache.org/r/4416/diff Testing ------- Unit tests added Thanks, Rohini
        Hide
        jiraposter@reviews.apache.org added a comment -

        -----------------------------------------------------------
        This is an automatically generated e-mail. To reply, visit:
        https://reviews.apache.org/r/4416/#review6350
        -----------------------------------------------------------

        Ship it!

        • Vandana

        On 2012-03-24 00:03:54, Rohini Palaniswamy wrote:

        -----------------------------------------------------------

        This is an automatically generated e-mail. To reply, visit:

        https://reviews.apache.org/r/4416/

        -----------------------------------------------------------

        (Updated 2012-03-24 00:03:54)

        Review request for Francis Liu and Vandana Ayyalasomayajula.

        Summary

        -------

        Introduced a new table property "hbase.table.name" similar to Hive to store hbase table names retaining the case. This will be used to access hbase and zookeeper which are case sensitive from within the HbaseHCatStorageHandler.

        Added two new APIs setupTable and dropTable to RevisionManager and removed public APIs for setting up and dropping znodes in ZKBasedRevisionManager.

        This addresses bug HCATALOG-301.

        https://issues.apache.org/jira/browse/HCATALOG-301

        Diffs

        -----

        http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/java/org/apache/hcatalog/hbase/HBaseHCatStorageHandler.java 1304584

        http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/java/org/apache/hcatalog/hbase/snapshot/RevisionManager.java 1304584

        http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/java/org/apache/hcatalog/hbase/snapshot/RevisionManagerEndpoint.java 1304584

        http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/java/org/apache/hcatalog/hbase/snapshot/RevisionManagerEndpointClient.java 1304584

        http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/java/org/apache/hcatalog/hbase/snapshot/ZKBasedRevisionManager.java 1304584

        http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/test/org/apache/hcatalog/hbase/TestHBaseBulkOutputFormat.java 1304584

        http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/test/org/apache/hcatalog/hbase/TestHBaseDirectOutputFormat.java 1304584

        http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/test/org/apache/hcatalog/hbase/TestHBaseHCatStorageHandler.java 1304584

        http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/test/org/apache/hcatalog/hbase/TestHBaseInputFormat.java 1304584

        http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/test/org/apache/hcatalog/hbase/snapshot/TestRevisionManagerEndpoint.java 1304584

        Diff: https://reviews.apache.org/r/4416/diff

        Testing

        -------

        Unit tests added

        Thanks,

        Rohini

        Show
        jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4416/#review6350 ----------------------------------------------------------- Ship it! Vandana On 2012-03-24 00:03:54, Rohini Palaniswamy wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4416/ ----------------------------------------------------------- (Updated 2012-03-24 00:03:54) Review request for Francis Liu and Vandana Ayyalasomayajula. Summary ------- Introduced a new table property "hbase.table.name" similar to Hive to store hbase table names retaining the case. This will be used to access hbase and zookeeper which are case sensitive from within the HbaseHCatStorageHandler. Added two new APIs setupTable and dropTable to RevisionManager and removed public APIs for setting up and dropping znodes in ZKBasedRevisionManager. This addresses bug HCATALOG-301 . https://issues.apache.org/jira/browse/HCATALOG-301 Diffs ----- http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/java/org/apache/hcatalog/hbase/HBaseHCatStorageHandler.java 1304584 http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/java/org/apache/hcatalog/hbase/snapshot/RevisionManager.java 1304584 http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/java/org/apache/hcatalog/hbase/snapshot/RevisionManagerEndpoint.java 1304584 http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/java/org/apache/hcatalog/hbase/snapshot/RevisionManagerEndpointClient.java 1304584 http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/java/org/apache/hcatalog/hbase/snapshot/ZKBasedRevisionManager.java 1304584 http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/test/org/apache/hcatalog/hbase/TestHBaseBulkOutputFormat.java 1304584 http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/test/org/apache/hcatalog/hbase/TestHBaseDirectOutputFormat.java 1304584 http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/test/org/apache/hcatalog/hbase/TestHBaseHCatStorageHandler.java 1304584 http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/test/org/apache/hcatalog/hbase/TestHBaseInputFormat.java 1304584 http://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.4/storage-handlers/hbase/src/test/org/apache/hcatalog/hbase/snapshot/TestRevisionManagerEndpoint.java 1304584 Diff: https://reviews.apache.org/r/4416/diff Testing ------- Unit tests added Thanks, Rohini
        Hide
        Alan Gates added a comment -

        All the end-to-end tests, including the the HBase ones, pass with this patch.

        Show
        Alan Gates added a comment - All the end-to-end tests, including the the HBase ones, pass with this patch.
        Hide
        Francis Liu added a comment -

        With security enabled pig hbase tests pass. MR tests (including non-hbase) are failing due to client not using sasl and metastore expecting sasl which an e2e harness issue. Failure is unlrelated, patch has been tested to fix addressed problems and showed no signs of regression given aggregate testing of security and non-secure setup. Checking this in. e2e tests can be debugged and address in a separate jira.

        Show
        Francis Liu added a comment - With security enabled pig hbase tests pass. MR tests (including non-hbase) are failing due to client not using sasl and metastore expecting sasl which an e2e harness issue. Failure is unlrelated, patch has been tested to fix addressed problems and showed no signs of regression given aggregate testing of security and non-secure setup. Checking this in. e2e tests can be debugged and address in a separate jira.
        Hide
        Francis Liu added a comment -

        Checked in to trunk and branch-0.4. Thanks Rohini.

        Show
        Francis Liu added a comment - Checked in to trunk and branch-0.4. Thanks Rohini.

          People

          • Assignee:
            Rohini Palaniswamy
            Reporter:
            David Capwell
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development