Hive
  1. Hive
  2. HIVE-675

Add Database/Schema support to Hive QL

    Details

    • Type: New Feature New Feature
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.6.0
    • Component/s: Metastore, Query Processor
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      This ticket adds the ability to CREATE DATABASE, DROP DATABASE, and USE <database>. Please note that Hive does not yet support the ability to run a query that selects from tables in more than one database.

      Description

      Currently all Hive tables reside in single namespace (default). Hive should support multiple namespaces (databases or schemas) such that users can create tables in their specific namespaces. These name spaces can have different warehouse directories (with a default naming scheme) and possibly different properties.

      There is already some support for this in metastore but Hive query parser should have this feature as well.

      1. HIVE-675.10.patch.txt
        433 kB
        Carl Steinbach
      2. HIVE-675.11.patch.txt
        429 kB
        Carl Steinbach
      3. HIVE-675.12.patch.txt
        835 kB
        Carl Steinbach
      4. HIVE-675.13.patch.txt
        842 kB
        Carl Steinbach
      5. hive-675-2009-9-16.patch
        65 kB
        He Yongqiang
      6. hive-675-2009-9-19.patch
        66 kB
        He Yongqiang
      7. hive-675-2009-9-21.patch
        64 kB
        He Yongqiang
      8. hive-675-2009-9-23.patch
        66 kB
        He Yongqiang
      9. hive-675-2009-9-7.patch
        54 kB
        He Yongqiang
      10. hive-675-2009-9-8.patch
        63 kB
        He Yongqiang
      11. HIVE-675-2010-08-16.patch.txt
        425 kB
        Carl Steinbach
      12. HIVE-675-2010-7-16.patch.txt
        121 kB
        Carl Steinbach
      13. HIVE-675-2010-8-4.patch.txt
        369 kB
        Carl Steinbach
      14. HIVE-675-backport-v6.1.patch.txt
        1.10 MB
        Carl Steinbach
      15. HIVE-675-backport-v6.2.patch.txt
        1.11 MB
        Carl Steinbach

        Issue Links

          Activity

          Hide
          Ning Zhang added a comment -

          That works. Thanks Carl!

          Show
          Ning Zhang added a comment - That works. Thanks Carl!
          Hide
          Carl Steinbach added a comment -

          @Ning: Can you please delete metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaStoreRemote.java?

          In the backport patch this file is deleted and replaced with TestRemoteHiveMetaStore.java, but it looks like for some reason
          this file was not actually deleted when the patch was applied.

          Show
          Carl Steinbach added a comment - @Ning: Can you please delete metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaStoreRemote.java? In the backport patch this file is deleted and replaced with TestRemoteHiveMetaStore.java, but it looks like for some reason this file was not actually deleted when the patch was applied.
          Hide
          Carl Steinbach added a comment -

          Hi Ning, I'm looking into it.

          Show
          Carl Steinbach added a comment - Hi Ning, I'm looking into it.
          Hide
          Ning Zhang added a comment -

          Hi Carl,

          Branch 0.6 currently is broken when running a unit test. The error is as follows:

          compile-test:
          [javac] /data/users/nzhang/reviews/0.6/branch-0.6/build-common.xml:307: warning: 'includeantruntime' was not set, defaulting to build.sysclasspath=last; set to false for repeatable builds
          [javac] Compiling 4 source files to /data/users/nzhang/reviews/0.6/branch-0.6/build/metastore/test/classes
          [javac] /data/users/nzhang/reviews/0.6/branch-0.6/metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaStoreRemote.java:76: partitionTester(org.apache.hadoop.hive.metastore.HiveMetaStoreClient,org.apache.hadoop.hive.conf.HiveConf) in org.apache.hadoop.hive.metastore.TestHiveMetaStore cannot be applied to (org.apache.hadoop.hive.metastore.HiveMetaStoreClient,org.apache.hadoop.hive.conf.HiveConf,boolean)
          [javac] TestHiveMetaStore.partitionTester(client, hiveConf, true);
          [javac] ^
          [javac] 1 error

          It seems the last patch that touches TestHiveMetaStore and TestHiveMetaStoreRemote is this patch. Can you take a look?

          Show
          Ning Zhang added a comment - Hi Carl, Branch 0.6 currently is broken when running a unit test. The error is as follows: compile-test: [javac] /data/users/nzhang/reviews/0.6/branch-0.6/build-common.xml:307: warning: 'includeantruntime' was not set, defaulting to build.sysclasspath=last; set to false for repeatable builds [javac] Compiling 4 source files to /data/users/nzhang/reviews/0.6/branch-0.6/build/metastore/test/classes [javac] /data/users/nzhang/reviews/0.6/branch-0.6/metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaStoreRemote.java:76: partitionTester(org.apache.hadoop.hive.metastore.HiveMetaStoreClient,org.apache.hadoop.hive.conf.HiveConf) in org.apache.hadoop.hive.metastore.TestHiveMetaStore cannot be applied to (org.apache.hadoop.hive.metastore.HiveMetaStoreClient,org.apache.hadoop.hive.conf.HiveConf,boolean) [javac] TestHiveMetaStore.partitionTester(client, hiveConf, true); [javac] ^ [javac] 1 error It seems the last patch that touches TestHiveMetaStore and TestHiveMetaStoreRemote is this patch. Can you take a look?
          Hide
          Namit Jain added a comment -

          Committed in 0.6. Thanks Carl

          Show
          Namit Jain added a comment - Committed in 0.6. Thanks Carl
          Hide
          Namit Jain added a comment -

          I have take a look

          Show
          Namit Jain added a comment - I have take a look
          Hide
          Carl Steinbach added a comment -

          HIVE-675-backport-v6.2.patch.txt includes HIVE-1607.

          Show
          Carl Steinbach added a comment - HIVE-675 -backport-v6.2.patch.txt includes HIVE-1607 .
          Hide
          John Sichi added a comment -

          I think it's fine to squash in this case.

          Show
          John Sichi added a comment - I think it's fine to squash in this case.
          Hide
          Carl Steinbach added a comment -

          Backport for 0.6.0.

          Should I squash HIVE-1607 into this or backport it separately?

          Show
          Carl Steinbach added a comment - Backport for 0.6.0. Should I squash HIVE-1607 into this or backport it separately?
          Hide
          John Sichi added a comment -

          I think a DEFAULT clause is needed for DB_LOCATION_URI since there's an existing row which needs to be filled in.

          The .jdo has allows-null=false, so the column definition should be NOT NULL.

          Show
          John Sichi added a comment - I think a DEFAULT clause is needed for DB_LOCATION_URI since there's an existing row which needs to be filled in. The .jdo has allows-null=false, so the column definition should be NOT NULL.
          Hide
          Carl Steinbach added a comment -

          @Paul: No, not yet, but I think the following script should work:

          ALTER TABLE DBS MODIFY DESC VARCHAR(4000);
          ALTER TABLE DBS ADD COLUMN DB_LOCATION_URI VARCHAR(4000);
          
          Show
          Carl Steinbach added a comment - @Paul: No, not yet, but I think the following script should work: ALTER TABLE DBS MODIFY DESC VARCHAR(4000); ALTER TABLE DBS ADD COLUMN DB_LOCATION_URI VARCHAR(4000);
          Hide
          Paul Yang added a comment -

          @Carl, do you happen to have a MySQL upgrade script for adding the new tables / columns required to support this feature?

          Show
          Paul Yang added a comment - @Carl, do you happen to have a MySQL upgrade script for adding the new tables / columns required to support this feature?
          Hide
          Carl Steinbach added a comment -

          Working on a backport for 0.6.0

          Show
          Carl Steinbach added a comment - Working on a backport for 0.6.0
          Hide
          Ning Zhang added a comment -

          Great! Thanks Carl!

          Show
          Ning Zhang added a comment - Great! Thanks Carl!
          Hide
          Carl Steinbach added a comment -

          @Ning: I filed HIVE-1607. Patch to follow shortly. Sorry for inconvenience.

          Show
          Carl Steinbach added a comment - @Ning: I filed HIVE-1607 . Patch to follow shortly. Sorry for inconvenience.
          Hide
          Ning Zhang added a comment -

          @Carl, I got compiling errors for tableExists() and getTable(). It would be nice to bring back those methods (as well as other methods like dropTable) without the database argument. Putting deprecation warning is fine for me.

          Show
          Ning Zhang added a comment - @Carl, I got compiling errors for tableExists() and getTable(). It would be nice to bring back those methods (as well as other methods like dropTable) without the database argument. Putting deprecation warning is fine for me.
          Hide
          John Sichi added a comment -

          Going through a deprecation phase helps ease transitions in deployment environments (like Facebook's) where not all upgrades can be done in lockstep.

          Show
          John Sichi added a comment - Going through a deprecation phase helps ease transitions in deployment environments (like Facebook's) where not all upgrades can be done in lockstep.
          Hide
          Carl Steinbach added a comment -

          I removed these methods since they implicitly target the default database. I can put them
          back with a deprecation warning, but I also want to point out that old code that depends on
          these methods is probably no longer correct now that Hive supports multiple databases.
          Removing these methods entirely may be the easiest way to help people find these errors.

          Show
          Carl Steinbach added a comment - I removed these methods since they implicitly target the default database. I can put them back with a deprecation warning, but I also want to point out that old code that depends on these methods is probably no longer correct now that Hive supports multiple databases. Removing these methods entirely may be the easiest way to help people find these errors.
          Hide
          John Sichi added a comment -

          Ning hit the broken dependencies; he may be able to tell whether this list is complete.

          Show
          John Sichi added a comment - Ning hit the broken dependencies; he may be able to tell whether this list is complete.
          Hide
          John Sichi added a comment -

          Oops, I misspoke; actually the thrift API is fine, it's the removal/change of some methods from Java interface IMetaStoreClient which broke dependencies.

          -  public void dropTable(String tableName, boolean deleteData)
          -      throws MetaException, UnknownTableException, TException,
          -      NoSuchObjectException;
          
          -  public boolean tableExists(String tableName) throws MetaException,
          +  public boolean tableExists(String databaseName, String tableName) throws MetaException,
                 TException, UnknownDBException;
           
          -  public Table getTable(String tableName) throws MetaException, TException,
          -      NoSuchObjectException;
          
          Show
          John Sichi added a comment - Oops, I misspoke; actually the thrift API is fine, it's the removal/change of some methods from Java interface IMetaStoreClient which broke dependencies. - public void dropTable(String tableName, boolean deleteData) - throws MetaException, UnknownTableException, TException, - NoSuchObjectException; - public boolean tableExists(String tableName) throws MetaException, + public boolean tableExists(String databaseName, String tableName) throws MetaException, TException, UnknownDBException; - public Table getTable(String tableName) throws MetaException, TException, - NoSuchObjectException;
          Hide
          Carl Steinbach added a comment -

          @John: Are you referring to the changes I made to create_database/get_database/get_databases/drop_database in hive_metastore.thrift?

          In that file I replaced

          list<string> get_databases() throws(1:MetaException o1)

          with

          list<string> get_databases(1:string pattern) throws(1:MetaException o1)

          I can easily revert this change, but want to know if there are are other things you think I need to fix.

          I think the changes I made to create_database and drop_database should not be an issue since these calls weren't actually supported in previous versions.

          Show
          Carl Steinbach added a comment - @John: Are you referring to the changes I made to create_database/get_database/get_databases/drop_database in hive_metastore.thrift? In that file I replaced list<string> get_databases() throws(1:MetaException o1) with list<string> get_databases(1:string pattern) throws(1:MetaException o1) I can easily revert this change, but want to know if there are are other things you think I need to fix. I think the changes I made to create_database and drop_database should not be an issue since these calls weren't actually supported in previous versions.
          Hide
          Ning Zhang added a comment -

          I think we should add backward-compatibility since there are a lot of scripts/programs are using the old API.

          Show
          Ning Zhang added a comment - I think we should add backward-compatibility since there are a lot of scripts/programs are using the old API.
          Hide
          John Sichi added a comment -

          The metastore thrift API change is a breaking change, so we should either add another a patch which restores backwards-compatibility, or else mention this in the release notes.

          Show
          John Sichi added a comment - The metastore thrift API change is a breaking change, so we should either add another a patch which restores backwards-compatibility, or else mention this in the release notes.
          Hide
          Namit Jain added a comment -

          Committed in trunk - Thanks Carl

          Show
          Namit Jain added a comment - Committed in trunk - Thanks Carl
          Hide
          Namit Jain added a comment -

          My bad in applying the patch - running tests again

          Show
          Namit Jain added a comment - My bad in applying the patch - running tests again
          Hide
          Carl Steinbach added a comment -

          Cited the wrong lines from the patch. Here are the correct ones:

          ...
          diff --git metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaStoreRemote.java metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaStoreRemote.java
          deleted file mode 100644
          index bc950b9..0000000
          --- metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaStoreRemote.java
          +++ /dev/null
          ...
          
          Show
          Carl Steinbach added a comment - Cited the wrong lines from the patch. Here are the correct ones: ... diff --git metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaStoreRemote.java metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaStoreRemote.java deleted file mode 100644 index bc950b9..0000000 --- metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaStoreRemote.java +++ /dev/null ...
          Hide
          Carl Steinbach added a comment -

          @Namit: the patch I supplied deletes TestHiveMetaStoreRemote.java and replaces it with TestRemoteHiveMetaStore and TestEmbeddedHiveMetaStore. I'm not sure why this not working for you. I tried applying the patch again to a fresh checkout of the Hive svn repo and confirmed that TestHiveMetaStoreRemote.java was deleted.

          Here are the relevant lines in the patch:

          diff --git eclipse-templates/TestRemoteHiveMetaStore.launchtemplate eclipse-templates/TestRemoteHiveMetaStore.launchtemplate
          new file mode 100644
          index 0000000..3600e5c
          --- /dev/null
          
          Show
          Carl Steinbach added a comment - @Namit: the patch I supplied deletes TestHiveMetaStoreRemote.java and replaces it with TestRemoteHiveMetaStore and TestEmbeddedHiveMetaStore. I'm not sure why this not working for you. I tried applying the patch again to a fresh checkout of the Hive svn repo and confirmed that TestHiveMetaStoreRemote.java was deleted. Here are the relevant lines in the patch: diff --git eclipse-templates/TestRemoteHiveMetaStore.launchtemplate eclipse-templates/TestRemoteHiveMetaStore.launchtemplate new file mode 100644 index 0000000..3600e5c --- /dev/null
          Hide
          Namit Jain added a comment -

          compile-test:
          [javac] Compiling 4 source files to /data/users/njain/hive_commit2/hive_commit2/build/metastore/test/classes
          [javac] /data/users/njain/hive_commit2/hive_commit2/metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaStoreRemote.java:76: partitionTester(org.apache.hadoop.hive.metastore.HiveMetaStoreClient,org.apache.hadoop.hive.conf.HiveConf) in org.apache.hadoop.hive.metastore.TestHiveMetaStore cannot be applied to (org.apache.hadoop.hive.metastore.HiveMetaStoreClient,org.apache.hadoop.hive.conf.HiveConf,boolean)
          [javac] TestHiveMetaStore.partitionTester(client, hiveConf, true);
          [javac] ^
          [javac] 1 error

          Show
          Namit Jain added a comment - compile-test: [javac] Compiling 4 source files to /data/users/njain/hive_commit2/hive_commit2/build/metastore/test/classes [javac] /data/users/njain/hive_commit2/hive_commit2/metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaStoreRemote.java:76: partitionTester(org.apache.hadoop.hive.metastore.HiveMetaStoreClient,org.apache.hadoop.hive.conf.HiveConf) in org.apache.hadoop.hive.metastore.TestHiveMetaStore cannot be applied to (org.apache.hadoop.hive.metastore.HiveMetaStoreClient,org.apache.hadoop.hive.conf.HiveConf,boolean) [javac] TestHiveMetaStore.partitionTester(client, hiveConf, true); [javac] ^ [javac] 1 error
          Hide
          Carl Steinbach added a comment -

          @Namit: What is the compilation error? I applied HIVE-675.13.patch.txt to a fresh checkout of trunk and did not run into any problems.

          Show
          Carl Steinbach added a comment - @Namit: What is the compilation error? I applied HIVE-675 .13.patch.txt to a fresh checkout of trunk and did not run into any problems.
          Hide
          Namit Jain added a comment -

          I am getting a compilation error

          Show
          Namit Jain added a comment - I am getting a compilation error
          Hide
          Carl Steinbach added a comment -

          That's correct. I will start work on the backport patch after this gets committed to trunk.

          Show
          Carl Steinbach added a comment - That's correct. I will start work on the backport patch after this gets committed to trunk.
          Hide
          Namit Jain added a comment -

          I will take a look again -

          Also, I think you would also like to have in 0.6 branch ?
          In that case, can you also prepare a patch for 0.6

          Show
          Namit Jain added a comment - I will take a look again - Also, I think you would also like to have in 0.6 branch ? In that case, can you also prepare a patch for 0.6
          Hide
          Carl Steinbach added a comment -

          @Namit: Looks like I was missing some tab characters in the test outputs. I verified that all tests pass with HIVE-675.13.patch.txt

          Show
          Carl Steinbach added a comment - @Namit: Looks like I was missing some tab characters in the test outputs. I verified that all tests pass with HIVE-675 .13.patch.txt
          Hide
          Namit Jain added a comment -

          The following tests failed (TestCliDriver):

          rename_column.q
          database.q
          alter1.q
          alter2.q
          alter3.q

          Can you run the tests again ?

          Show
          Namit Jain added a comment - The following tests failed (TestCliDriver): rename_column.q database.q alter1.q alter2.q alter3.q Can you run the tests again ?
          Hide
          Namit Jain added a comment -

          +1

          will commit if tests pass.
          Needs support to lock the database, but will file a follow-up for that

          Show
          Namit Jain added a comment - +1 will commit if tests pass. Needs support to lock the database, but will file a follow-up for that
          Hide
          HBase Review Board added a comment -

          Message from: "Ted Yu" <ted_yu@yahoo.com>

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          http://review.cloudera.org/r/508/#review1024
          -----------------------------------------------------------

          metastore/src/model/package.jdo
          <http://review.cloudera.org/r/508/#comment3249>

          name= should be changed as well.

          • Ted
          Show
          HBase Review Board added a comment - Message from: "Ted Yu" <ted_yu@yahoo.com> ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: http://review.cloudera.org/r/508/#review1024 ----------------------------------------------------------- metastore/src/model/package.jdo < http://review.cloudera.org/r/508/#comment3249 > name= should be changed as well. Ted
          Hide
          HBase Review Board added a comment -

          Message from: "Carl Steinbach" <carl@cloudera.com>

          On 2010-08-24 14:23:09, Ted Yu wrote:

          > metastore/src/model/package.jdo, line 15

          > <http://review.cloudera.org/r/508/diff/4/?file=7227#file7227line15>

          >

          > name= should be changed as well.

          "DESC" is the name of a column that already exists in the schema. The name is not optimal, but changing it presents a bigger inconvenience.

          • Carl

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          http://review.cloudera.org/r/508/#review1024
          -----------------------------------------------------------

          Show
          HBase Review Board added a comment - Message from: "Carl Steinbach" <carl@cloudera.com> On 2010-08-24 14:23:09, Ted Yu wrote: > metastore/src/model/package.jdo, line 15 > < http://review.cloudera.org/r/508/diff/4/?file=7227#file7227line15 > > > name= should be changed as well. "DESC" is the name of a column that already exists in the schema. The name is not optimal, but changing it presents a bigger inconvenience. Carl ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: http://review.cloudera.org/r/508/#review1024 -----------------------------------------------------------
          Hide
          HBase Review Board added a comment -

          Message from: "Carl Steinbach" <carl@cloudera.com>

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          http://review.cloudera.org/r/508/
          -----------------------------------------------------------

          (Updated 2010-08-24 13:55:16.686174)

          Review request for Hive Developers.

          Changes
          -------

          HIVE-675.12.patch.txt:

          • Added HiveMetaStore.getAllTables(String dbName) and HiveMetaStore.getAllDatabases()
          • Added ORDER BY clause to ObjectStore.getDatabases()
          • Reverted Database.getComment() to Database.getDescription()
          • Replaced a couple more hardcoded references to DEFAULT_DATABASE_NAME

          Summary
          -------

          Database/Scheme support for Hive.

          • Implemented 'SHOW DATABASES' command
          • Refactored TestHiveMetaStore and enabled tests for remote metastore client.
          • Added launch configurations for TestHiveMetaStore and TestHiveMetaStoreRemote

          This addresses bug HIVE-675.
          http://issues.apache.org/jira/browse/HIVE-675

          Diffs (updated)


          .gitignore 376cdc6
          build-common.xml 00aa470
          eclipse-templates/TestCliDriver.launchtemplate c304161
          eclipse-templates/TestEmbeddedHiveMetaStore.launchtemplate PRE-CREATION
          eclipse-templates/TestHive.launchtemplate 24efc12
          eclipse-templates/TestHiveMetaStoreChecker.launchtemplate PRE-CREATION
          eclipse-templates/TestRemoteHiveMetaStore.launchtemplate PRE-CREATION
          metastore/if/hive_metastore.thrift 478d0af
          metastore/src/gen-cpp/ThriftHiveMetastore.h e2538fb
          metastore/src/gen-cpp/ThriftHiveMetastore.cpp f945a3a
          metastore/src/gen-cpp/ThriftHiveMetastore_server.skeleton.cpp ed2bb99
          metastore/src/gen-cpp/hive_metastore_types.h 1b0c706
          metastore/src/gen-cpp/hive_metastore_types.cpp b5a403d
          metastore/src/gen-javabean/org/apache/hadoop/hive/metastore/api/Database.java 78c78d9
          metastore/src/gen-javabean/org/apache/hadoop/hive/metastore/api/ThriftHiveMetastore.java 25408d9
          metastore/src/gen-php/ThriftHiveMetastore.php ea4add5
          metastore/src/gen-php/hive_metastore_types.php 61872a0
          metastore/src/gen-py/hive_metastore/ThriftHiveMetastore-remote fc06cba
          metastore/src/gen-py/hive_metastore/ThriftHiveMetastore.py 4a0bc67
          metastore/src/gen-py/hive_metastore/ttypes.py ea7269e
          metastore/src/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java 39dbd52
          metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 4fb296a
          metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java c6541af
          metastore/src/java/org/apache/hadoop/hive/metastore/IMetaStoreClient.java 6013644
          metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java 0818689
          metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java a06384c
          metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java 4951bd6
          metastore/src/java/org/apache/hadoop/hive/metastore/Warehouse.java 4488f94
          metastore/src/model/org/apache/hadoop/hive/metastore/model/MDatabase.java b3e098d
          metastore/src/model/package.jdo 206ba75
          metastore/src/test/org/apache/hadoop/hive/metastore/TestEmbeddedHiveMetaStore.java PRE-CREATION
          metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java fff6aad
          metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaStoreRemote.java bc950b9
          metastore/src/test/org/apache/hadoop/hive/metastore/TestRemoteHiveMetaStore.java PRE-CREATION
          ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java 28b4d2a
          ql/src/java/org/apache/hadoop/hive/ql/exec/MoveTask.java d59f48c
          ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java ce5d903
          ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java 352af89
          ql/src/java/org/apache/hadoop/hive/ql/parse/BaseSemanticAnalyzer.java eedf9e3
          ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java 658eea5
          ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g fca7ff6
          ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java cba4a66
          ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzerFactory.java 881290c
          ql/src/java/org/apache/hadoop/hive/ql/plan/CreateDatabaseDesc.java PRE-CREATION
          ql/src/java/org/apache/hadoop/hive/ql/plan/DDLWork.java 0c848e6
          ql/src/java/org/apache/hadoop/hive/ql/plan/DropDatabaseDesc.java PRE-CREATION
          ql/src/java/org/apache/hadoop/hive/ql/plan/ShowDatabasesDesc.java PRE-CREATION
          ql/src/java/org/apache/hadoop/hive/ql/plan/SwitchDatabaseDesc.java PRE-CREATION
          ql/src/test/org/apache/hadoop/hive/ql/QTestUtil.java 70d64ef
          ql/src/test/org/apache/hadoop/hive/ql/hooks/EnforceReadOnlyTables.java 8cff958
          ql/src/test/org/apache/hadoop/hive/ql/metadata/TestHive.java ab39ca4
          ql/src/test/org/apache/hadoop/hive/ql/metadata/TestHiveMetaStoreChecker.java 26cc71a
          ql/src/test/org/apache/hadoop/hive/ql/metadata/TestPartition.java 790abf6
          ql/src/test/queries/clientnegative/database_create_already_exists.q PRE-CREATION
          ql/src/test/queries/clientnegative/database_create_invalid_name.q PRE-CREATION
          ql/src/test/queries/clientnegative/database_drop_does_not_exist.q PRE-CREATION
          ql/src/test/queries/clientnegative/database_drop_not_empty.q PRE-CREATION
          ql/src/test/queries/clientnegative/database_switch_does_not_exist.q PRE-CREATION
          ql/src/test/queries/clientpositive/add_part_exist.q 54d0096
          ql/src/test/queries/clientpositive/alter1.q 6f95685
          ql/src/test/queries/clientpositive/alter2.q 2388e21
          ql/src/test/queries/clientpositive/alter3.q d9d3b07
          ql/src/test/queries/clientpositive/alter4.q 35fa441
          ql/src/test/queries/clientpositive/database.q PRE-CREATION
          ql/src/test/queries/clientpositive/rename_column.q ce82ff7
          ql/src/test/results/clientnegative/database_already_exists.q.out PRE-CREATION
          ql/src/test/results/clientnegative/database_create_already_exists.q.out PRE-CREATION
          ql/src/test/results/clientnegative/database_create_invalid_name.q.out PRE-CREATION
          ql/src/test/results/clientnegative/database_drop_does_not_exist.q.out PRE-CREATION
          ql/src/test/results/clientnegative/database_drop_not_empty.q.out PRE-CREATION
          ql/src/test/results/clientnegative/database_switch_does_not_exist.q.out PRE-CREATION
          ql/src/test/results/clientpositive/add_part_exist.q.out 5f76c9d
          ql/src/test/results/clientpositive/alter1.q.out 73c5978
          ql/src/test/results/clientpositive/alter2.q.out 06a2bab
          ql/src/test/results/clientpositive/alter3.q.out 3f91306
          ql/src/test/results/clientpositive/alter4.q.out 058d25d
          ql/src/test/results/clientpositive/database.q.out PRE-CREATION
          ql/src/test/results/clientpositive/rename_column.q.out 252d681

          Diff: http://review.cloudera.org/r/508/diff

          Testing
          -------

          Thanks,

          Carl

          Show
          HBase Review Board added a comment - Message from: "Carl Steinbach" <carl@cloudera.com> ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: http://review.cloudera.org/r/508/ ----------------------------------------------------------- (Updated 2010-08-24 13:55:16.686174) Review request for Hive Developers. Changes ------- HIVE-675 .12.patch.txt: Added HiveMetaStore.getAllTables(String dbName) and HiveMetaStore.getAllDatabases() Added ORDER BY clause to ObjectStore.getDatabases() Reverted Database.getComment() to Database.getDescription() Replaced a couple more hardcoded references to DEFAULT_DATABASE_NAME Summary ------- Database/Scheme support for Hive. Implemented 'SHOW DATABASES' command Refactored TestHiveMetaStore and enabled tests for remote metastore client. Added launch configurations for TestHiveMetaStore and TestHiveMetaStoreRemote This addresses bug HIVE-675 . http://issues.apache.org/jira/browse/HIVE-675 Diffs (updated) .gitignore 376cdc6 build-common.xml 00aa470 eclipse-templates/TestCliDriver.launchtemplate c304161 eclipse-templates/TestEmbeddedHiveMetaStore.launchtemplate PRE-CREATION eclipse-templates/TestHive.launchtemplate 24efc12 eclipse-templates/TestHiveMetaStoreChecker.launchtemplate PRE-CREATION eclipse-templates/TestRemoteHiveMetaStore.launchtemplate PRE-CREATION metastore/if/hive_metastore.thrift 478d0af metastore/src/gen-cpp/ThriftHiveMetastore.h e2538fb metastore/src/gen-cpp/ThriftHiveMetastore.cpp f945a3a metastore/src/gen-cpp/ThriftHiveMetastore_server.skeleton.cpp ed2bb99 metastore/src/gen-cpp/hive_metastore_types.h 1b0c706 metastore/src/gen-cpp/hive_metastore_types.cpp b5a403d metastore/src/gen-javabean/org/apache/hadoop/hive/metastore/api/Database.java 78c78d9 metastore/src/gen-javabean/org/apache/hadoop/hive/metastore/api/ThriftHiveMetastore.java 25408d9 metastore/src/gen-php/ThriftHiveMetastore.php ea4add5 metastore/src/gen-php/hive_metastore_types.php 61872a0 metastore/src/gen-py/hive_metastore/ThriftHiveMetastore-remote fc06cba metastore/src/gen-py/hive_metastore/ThriftHiveMetastore.py 4a0bc67 metastore/src/gen-py/hive_metastore/ttypes.py ea7269e metastore/src/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java 39dbd52 metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 4fb296a metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java c6541af metastore/src/java/org/apache/hadoop/hive/metastore/IMetaStoreClient.java 6013644 metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java 0818689 metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java a06384c metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java 4951bd6 metastore/src/java/org/apache/hadoop/hive/metastore/Warehouse.java 4488f94 metastore/src/model/org/apache/hadoop/hive/metastore/model/MDatabase.java b3e098d metastore/src/model/package.jdo 206ba75 metastore/src/test/org/apache/hadoop/hive/metastore/TestEmbeddedHiveMetaStore.java PRE-CREATION metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java fff6aad metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaStoreRemote.java bc950b9 metastore/src/test/org/apache/hadoop/hive/metastore/TestRemoteHiveMetaStore.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java 28b4d2a ql/src/java/org/apache/hadoop/hive/ql/exec/MoveTask.java d59f48c ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java ce5d903 ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java 352af89 ql/src/java/org/apache/hadoop/hive/ql/parse/BaseSemanticAnalyzer.java eedf9e3 ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java 658eea5 ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g fca7ff6 ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java cba4a66 ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzerFactory.java 881290c ql/src/java/org/apache/hadoop/hive/ql/plan/CreateDatabaseDesc.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/plan/DDLWork.java 0c848e6 ql/src/java/org/apache/hadoop/hive/ql/plan/DropDatabaseDesc.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/plan/ShowDatabasesDesc.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/plan/SwitchDatabaseDesc.java PRE-CREATION ql/src/test/org/apache/hadoop/hive/ql/QTestUtil.java 70d64ef ql/src/test/org/apache/hadoop/hive/ql/hooks/EnforceReadOnlyTables.java 8cff958 ql/src/test/org/apache/hadoop/hive/ql/metadata/TestHive.java ab39ca4 ql/src/test/org/apache/hadoop/hive/ql/metadata/TestHiveMetaStoreChecker.java 26cc71a ql/src/test/org/apache/hadoop/hive/ql/metadata/TestPartition.java 790abf6 ql/src/test/queries/clientnegative/database_create_already_exists.q PRE-CREATION ql/src/test/queries/clientnegative/database_create_invalid_name.q PRE-CREATION ql/src/test/queries/clientnegative/database_drop_does_not_exist.q PRE-CREATION ql/src/test/queries/clientnegative/database_drop_not_empty.q PRE-CREATION ql/src/test/queries/clientnegative/database_switch_does_not_exist.q PRE-CREATION ql/src/test/queries/clientpositive/add_part_exist.q 54d0096 ql/src/test/queries/clientpositive/alter1.q 6f95685 ql/src/test/queries/clientpositive/alter2.q 2388e21 ql/src/test/queries/clientpositive/alter3.q d9d3b07 ql/src/test/queries/clientpositive/alter4.q 35fa441 ql/src/test/queries/clientpositive/database.q PRE-CREATION ql/src/test/queries/clientpositive/rename_column.q ce82ff7 ql/src/test/results/clientnegative/database_already_exists.q.out PRE-CREATION ql/src/test/results/clientnegative/database_create_already_exists.q.out PRE-CREATION ql/src/test/results/clientnegative/database_create_invalid_name.q.out PRE-CREATION ql/src/test/results/clientnegative/database_drop_does_not_exist.q.out PRE-CREATION ql/src/test/results/clientnegative/database_drop_not_empty.q.out PRE-CREATION ql/src/test/results/clientnegative/database_switch_does_not_exist.q.out PRE-CREATION ql/src/test/results/clientpositive/add_part_exist.q.out 5f76c9d ql/src/test/results/clientpositive/alter1.q.out 73c5978 ql/src/test/results/clientpositive/alter2.q.out 06a2bab ql/src/test/results/clientpositive/alter3.q.out 3f91306 ql/src/test/results/clientpositive/alter4.q.out 058d25d ql/src/test/results/clientpositive/database.q.out PRE-CREATION ql/src/test/results/clientpositive/rename_column.q.out 252d681 Diff: http://review.cloudera.org/r/508/diff Testing ------- Thanks, Carl
          Hide
          HBase Review Board added a comment -

          Message from: "Carl Steinbach" <carl@cloudera.com>

          On 2010-08-17 15:32:59, namit jain wrote:

          > metastore/src/model/package.jdo, line 14

          > <http://review.cloudera.org/r/508/diff/4/?file=7227#file7227line14>

          >

          > Why do you want to change the field name ?

          I changed it since this refers to the value of the COMMENT clause in the CREATE DATABASE statement. I changed it back to the original value just to keep things simple.

          • Carl

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          http://review.cloudera.org/r/508/#review934
          -----------------------------------------------------------

          Show
          HBase Review Board added a comment - Message from: "Carl Steinbach" <carl@cloudera.com> On 2010-08-17 15:32:59, namit jain wrote: > metastore/src/model/package.jdo, line 14 > < http://review.cloudera.org/r/508/diff/4/?file=7227#file7227line14 > > > Why do you want to change the field name ? I changed it since this refers to the value of the COMMENT clause in the CREATE DATABASE statement. I changed it back to the original value just to keep things simple. Carl ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: http://review.cloudera.org/r/508/#review934 -----------------------------------------------------------
          Hide
          Carl Steinbach added a comment -

          Namit, will do.

          Show
          Carl Steinbach added a comment - Namit, will do.
          Hide
          Namit Jain added a comment -

          Carl, The changes look good, but can you regenerate the patch ?
          I am getting some conflicts on trunk.

          Show
          Namit Jain added a comment - Carl, The changes look good, but can you regenerate the patch ? I am getting some conflicts on trunk.
          Hide
          HBase Review Board added a comment -

          Message from: "Carl Steinbach" <carl@cloudera.com>

          On 2010-08-11 06:57:04, namit jain wrote:

          > metastore/if/hive_metastore.thrift, line 35

          > <http://review.cloudera.org/r/508/diff/2/?file=5171#file5171line35>

          >

          > Same as John's earlier comment - do we need

          > to change the name.

          >

          > No-one is using this table in the metastore currently,

          > so it might be OK - but not sure, how JDO would

          > handle renaming a column

          Done.

          On 2010-08-11 06:57:04, namit jain wrote:

          > ql/src/test/queries/clientpositive/database.q, line 15

          > <http://review.cloudera.org/r/508/diff/2/?file=5216#file5216line15>

          >

          > also add a test for 'show databases for a pattern'

          Done.

          • Carl

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          http://review.cloudera.org/r/508/#review838
          -----------------------------------------------------------

          Show
          HBase Review Board added a comment - Message from: "Carl Steinbach" <carl@cloudera.com> On 2010-08-11 06:57:04, namit jain wrote: > metastore/if/hive_metastore.thrift, line 35 > < http://review.cloudera.org/r/508/diff/2/?file=5171#file5171line35 > > > Same as John's earlier comment - do we need > to change the name. > > No-one is using this table in the metastore currently, > so it might be OK - but not sure, how JDO would > handle renaming a column Done. On 2010-08-11 06:57:04, namit jain wrote: > ql/src/test/queries/clientpositive/database.q, line 15 > < http://review.cloudera.org/r/508/diff/2/?file=5216#file5216line15 > > > also add a test for 'show databases for a pattern' Done. Carl ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: http://review.cloudera.org/r/508/#review838 -----------------------------------------------------------
          Hide
          Namit Jain added a comment -

          Also, can you update the patch ?
          I am getting some conflicts after https://issues.apache.org/jira/browse/HIVE-1548

          Show
          Namit Jain added a comment - Also, can you update the patch ? I am getting some conflicts after https://issues.apache.org/jira/browse/HIVE-1548
          Hide
          HBase Review Board added a comment -

          Message from: "namit jain" <njain@facebook.com>

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          http://review.cloudera.org/r/508/#review934
          -----------------------------------------------------------

          metastore/src/model/package.jdo
          <http://review.cloudera.org/r/508/#comment3080>

          Why do you want to change the field name ?

          • namit
          Show
          HBase Review Board added a comment - Message from: "namit jain" <njain@facebook.com> ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: http://review.cloudera.org/r/508/#review934 ----------------------------------------------------------- metastore/src/model/package.jdo < http://review.cloudera.org/r/508/#comment3080 > Why do you want to change the field name ? namit
          Hide
          Namit Jain added a comment -

          Added comments at http://review.cloudera.org/r/508/diff

          Also, inputs and outputs need to be populated for 'drop database ..'
          It should consist of all the tables/partitions in that database.
          It is fine if you want to do so in a follow-up, in which case, please file a dependent jira

          Show
          Namit Jain added a comment - Added comments at http://review.cloudera.org/r/508/diff Also, inputs and outputs need to be populated for 'drop database ..' It should consist of all the tables/partitions in that database. It is fine if you want to do so in a follow-up, in which case, please file a dependent jira
          Hide
          HBase Review Board added a comment -

          Message from: "Carl Steinbach" <carl@cloudera.com>

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          http://review.cloudera.org/r/508/
          -----------------------------------------------------------

          (Updated 2010-08-16 13:00:54.438322)

          Review request for Hive Developers.

          Changes
          -------

          • Fixed several eclipse launch configurations.
          • Refactored TestHiveMetaStore into TestEmbeddedHiveMetaStore and TestRemoteHiveMetaStore
          • Added a bunch of testcases.

          Summary
          -------

          Database/Scheme support for Hive.

          • Implemented 'SHOW DATABASES' command
          • Refactored TestHiveMetaStore and enabled tests for remote metastore client.
          • Added launch configurations for TestHiveMetaStore and TestHiveMetaStoreRemote

          This addresses bug HIVE-675.
          http://issues.apache.org/jira/browse/HIVE-675

          Diffs (updated)


          .gitignore 376cdc6
          build-common.xml d4ff895
          eclipse-templates/TestCliDriver.launchtemplate c304161
          eclipse-templates/TestEmbeddedHiveMetaStore.launchtemplate PRE-CREATION
          eclipse-templates/TestHive.launchtemplate 24efc12
          eclipse-templates/TestHiveMetaStoreChecker.launchtemplate PRE-CREATION
          eclipse-templates/TestRemoteHiveMetaStore.launchtemplate PRE-CREATION
          metastore/if/hive_metastore.thrift 478d0af
          metastore/src/gen-cpp/ThriftHiveMetastore.h e2538fb
          metastore/src/gen-cpp/ThriftHiveMetastore.cpp f945a3a
          metastore/src/gen-cpp/ThriftHiveMetastore_server.skeleton.cpp ed2bb99
          metastore/src/gen-cpp/hive_metastore_types.h 1b0c706
          metastore/src/gen-cpp/hive_metastore_types.cpp b5a403d
          metastore/src/gen-javabean/org/apache/hadoop/hive/metastore/api/Database.java 78c78d9
          metastore/src/gen-javabean/org/apache/hadoop/hive/metastore/api/ThriftHiveMetastore.java 25408d9
          metastore/src/gen-php/ThriftHiveMetastore.php ea4add5
          metastore/src/gen-php/hive_metastore_types.php 61872a0
          metastore/src/gen-py/hive_metastore/ThriftHiveMetastore-remote fc06cba
          metastore/src/gen-py/hive_metastore/ThriftHiveMetastore.py 4a0bc67
          metastore/src/gen-py/hive_metastore/ttypes.py ea7269e
          metastore/src/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java 39dbd52
          metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 4fb296a
          metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java c6541af
          metastore/src/java/org/apache/hadoop/hive/metastore/IMetaStoreClient.java 6013644
          metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java 0818689
          metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java a06384c
          metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java 4951bd6
          metastore/src/java/org/apache/hadoop/hive/metastore/Warehouse.java 4488f94
          metastore/src/model/org/apache/hadoop/hive/metastore/model/MDatabase.java b3e098d
          metastore/src/model/package.jdo 206ba75
          metastore/src/test/org/apache/hadoop/hive/metastore/TestEmbeddedHiveMetaStore.java PRE-CREATION
          metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java fff6aad
          metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaStoreRemote.java bc950b9
          metastore/src/test/org/apache/hadoop/hive/metastore/TestRemoteHiveMetaStore.java PRE-CREATION
          ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java ef1da6b
          ql/src/java/org/apache/hadoop/hive/ql/exec/MoveTask.java d59f48c
          ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java e0761fc
          ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java 3300e2a
          ql/src/java/org/apache/hadoop/hive/ql/parse/BaseSemanticAnalyzer.java eedf9e3
          ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java 1801c10
          ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g b080780
          ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 682a8b5
          ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzerFactory.java 6af3b17
          ql/src/java/org/apache/hadoop/hive/ql/plan/CreateDatabaseDesc.java PRE-CREATION
          ql/src/java/org/apache/hadoop/hive/ql/plan/DDLWork.java ed4ed22
          ql/src/java/org/apache/hadoop/hive/ql/plan/DropDatabaseDesc.java PRE-CREATION
          ql/src/java/org/apache/hadoop/hive/ql/plan/ShowDatabasesDesc.java PRE-CREATION
          ql/src/java/org/apache/hadoop/hive/ql/plan/SwitchDatabaseDesc.java PRE-CREATION
          ql/src/test/org/apache/hadoop/hive/ql/QTestUtil.java b4651a2
          ql/src/test/org/apache/hadoop/hive/ql/metadata/TestHive.java ab39ca4
          ql/src/test/org/apache/hadoop/hive/ql/metadata/TestHiveMetaStoreChecker.java 26cc71a
          ql/src/test/queries/clientnegative/database_create_already_exists.q PRE-CREATION
          ql/src/test/queries/clientnegative/database_create_invalid_name.q PRE-CREATION
          ql/src/test/queries/clientnegative/database_drop_does_not_exist.q PRE-CREATION
          ql/src/test/queries/clientnegative/database_drop_not_empty.q PRE-CREATION
          ql/src/test/queries/clientnegative/database_switch_does_not_exist.q PRE-CREATION
          ql/src/test/queries/clientpositive/add_part_exist.q 54d0096
          ql/src/test/queries/clientpositive/alter1.q 6f95685
          ql/src/test/queries/clientpositive/alter2.q 2388e21
          ql/src/test/queries/clientpositive/alter3.q d9d3b07
          ql/src/test/queries/clientpositive/alter4.q 35fa441
          ql/src/test/queries/clientpositive/database.q PRE-CREATION
          ql/src/test/queries/clientpositive/rename_column.q ce82ff7
          ql/src/test/results/clientnegative/database_already_exists.q.out PRE-CREATION
          ql/src/test/results/clientnegative/database_create_already_exists.q.out PRE-CREATION
          ql/src/test/results/clientnegative/database_create_invalid_name.q.out PRE-CREATION
          ql/src/test/results/clientnegative/database_drop_does_not_exist.q.out PRE-CREATION
          ql/src/test/results/clientnegative/database_drop_not_empty.q.out PRE-CREATION
          ql/src/test/results/clientnegative/database_switch_does_not_exist.q.out PRE-CREATION
          ql/src/test/results/clientpositive/add_part_exist.q.out 4bcdf8a
          ql/src/test/results/clientpositive/alter1.q.out ea143a6
          ql/src/test/results/clientpositive/alter2.q.out 6de5f19
          ql/src/test/results/clientpositive/alter3.q.out 2a1f4d5
          ql/src/test/results/clientpositive/alter4.q.out da7cd9a
          ql/src/test/results/clientpositive/database.q.out PRE-CREATION
          ql/src/test/results/clientpositive/rename_column.q.out 8d5da8d

          Diff: http://review.cloudera.org/r/508/diff

          Testing
          -------

          Thanks,

          Carl

          Show
          HBase Review Board added a comment - Message from: "Carl Steinbach" <carl@cloudera.com> ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: http://review.cloudera.org/r/508/ ----------------------------------------------------------- (Updated 2010-08-16 13:00:54.438322) Review request for Hive Developers. Changes ------- Fixed several eclipse launch configurations. Refactored TestHiveMetaStore into TestEmbeddedHiveMetaStore and TestRemoteHiveMetaStore Added a bunch of testcases. Summary ------- Database/Scheme support for Hive. Implemented 'SHOW DATABASES' command Refactored TestHiveMetaStore and enabled tests for remote metastore client. Added launch configurations for TestHiveMetaStore and TestHiveMetaStoreRemote This addresses bug HIVE-675 . http://issues.apache.org/jira/browse/HIVE-675 Diffs (updated) .gitignore 376cdc6 build-common.xml d4ff895 eclipse-templates/TestCliDriver.launchtemplate c304161 eclipse-templates/TestEmbeddedHiveMetaStore.launchtemplate PRE-CREATION eclipse-templates/TestHive.launchtemplate 24efc12 eclipse-templates/TestHiveMetaStoreChecker.launchtemplate PRE-CREATION eclipse-templates/TestRemoteHiveMetaStore.launchtemplate PRE-CREATION metastore/if/hive_metastore.thrift 478d0af metastore/src/gen-cpp/ThriftHiveMetastore.h e2538fb metastore/src/gen-cpp/ThriftHiveMetastore.cpp f945a3a metastore/src/gen-cpp/ThriftHiveMetastore_server.skeleton.cpp ed2bb99 metastore/src/gen-cpp/hive_metastore_types.h 1b0c706 metastore/src/gen-cpp/hive_metastore_types.cpp b5a403d metastore/src/gen-javabean/org/apache/hadoop/hive/metastore/api/Database.java 78c78d9 metastore/src/gen-javabean/org/apache/hadoop/hive/metastore/api/ThriftHiveMetastore.java 25408d9 metastore/src/gen-php/ThriftHiveMetastore.php ea4add5 metastore/src/gen-php/hive_metastore_types.php 61872a0 metastore/src/gen-py/hive_metastore/ThriftHiveMetastore-remote fc06cba metastore/src/gen-py/hive_metastore/ThriftHiveMetastore.py 4a0bc67 metastore/src/gen-py/hive_metastore/ttypes.py ea7269e metastore/src/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java 39dbd52 metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 4fb296a metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java c6541af metastore/src/java/org/apache/hadoop/hive/metastore/IMetaStoreClient.java 6013644 metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java 0818689 metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java a06384c metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java 4951bd6 metastore/src/java/org/apache/hadoop/hive/metastore/Warehouse.java 4488f94 metastore/src/model/org/apache/hadoop/hive/metastore/model/MDatabase.java b3e098d metastore/src/model/package.jdo 206ba75 metastore/src/test/org/apache/hadoop/hive/metastore/TestEmbeddedHiveMetaStore.java PRE-CREATION metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java fff6aad metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaStoreRemote.java bc950b9 metastore/src/test/org/apache/hadoop/hive/metastore/TestRemoteHiveMetaStore.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java ef1da6b ql/src/java/org/apache/hadoop/hive/ql/exec/MoveTask.java d59f48c ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java e0761fc ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java 3300e2a ql/src/java/org/apache/hadoop/hive/ql/parse/BaseSemanticAnalyzer.java eedf9e3 ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java 1801c10 ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g b080780 ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 682a8b5 ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzerFactory.java 6af3b17 ql/src/java/org/apache/hadoop/hive/ql/plan/CreateDatabaseDesc.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/plan/DDLWork.java ed4ed22 ql/src/java/org/apache/hadoop/hive/ql/plan/DropDatabaseDesc.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/plan/ShowDatabasesDesc.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/plan/SwitchDatabaseDesc.java PRE-CREATION ql/src/test/org/apache/hadoop/hive/ql/QTestUtil.java b4651a2 ql/src/test/org/apache/hadoop/hive/ql/metadata/TestHive.java ab39ca4 ql/src/test/org/apache/hadoop/hive/ql/metadata/TestHiveMetaStoreChecker.java 26cc71a ql/src/test/queries/clientnegative/database_create_already_exists.q PRE-CREATION ql/src/test/queries/clientnegative/database_create_invalid_name.q PRE-CREATION ql/src/test/queries/clientnegative/database_drop_does_not_exist.q PRE-CREATION ql/src/test/queries/clientnegative/database_drop_not_empty.q PRE-CREATION ql/src/test/queries/clientnegative/database_switch_does_not_exist.q PRE-CREATION ql/src/test/queries/clientpositive/add_part_exist.q 54d0096 ql/src/test/queries/clientpositive/alter1.q 6f95685 ql/src/test/queries/clientpositive/alter2.q 2388e21 ql/src/test/queries/clientpositive/alter3.q d9d3b07 ql/src/test/queries/clientpositive/alter4.q 35fa441 ql/src/test/queries/clientpositive/database.q PRE-CREATION ql/src/test/queries/clientpositive/rename_column.q ce82ff7 ql/src/test/results/clientnegative/database_already_exists.q.out PRE-CREATION ql/src/test/results/clientnegative/database_create_already_exists.q.out PRE-CREATION ql/src/test/results/clientnegative/database_create_invalid_name.q.out PRE-CREATION ql/src/test/results/clientnegative/database_drop_does_not_exist.q.out PRE-CREATION ql/src/test/results/clientnegative/database_drop_not_empty.q.out PRE-CREATION ql/src/test/results/clientnegative/database_switch_does_not_exist.q.out PRE-CREATION ql/src/test/results/clientpositive/add_part_exist.q.out 4bcdf8a ql/src/test/results/clientpositive/alter1.q.out ea143a6 ql/src/test/results/clientpositive/alter2.q.out 6de5f19 ql/src/test/results/clientpositive/alter3.q.out 2a1f4d5 ql/src/test/results/clientpositive/alter4.q.out da7cd9a ql/src/test/results/clientpositive/database.q.out PRE-CREATION ql/src/test/results/clientpositive/rename_column.q.out 8d5da8d Diff: http://review.cloudera.org/r/508/diff Testing ------- Thanks, Carl
          Hide
          HBase Review Board added a comment -

          Message from: "Carl Steinbach" <carl@cloudera.com>

          On 2010-08-09 15:51:55, John Sichi wrote:

          > metastore/src/model/package.jdo, line 11

          > <http://review.cloudera.org/r/508/diff/2/?file=5193#file5193line11>

          >

          > Do we really need to change this column name? I can see that it doesn't follow the conventions for other tables, but it might make upgrades a pain.

          >

          I reverted the change.

          On 2010-08-09 15:51:55, John Sichi wrote:

          > ql/src/java/org/apache/hadoop/hive/ql/plan/ShowDatabasesDesc.java, line 41

          > <http://review.cloudera.org/r/508/diff/2/?file=5209#file5209line41>

          >

          > Should this be db_name instead of tab_name?

          Changed.

          • Carl

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          http://review.cloudera.org/r/508/#review778
          -----------------------------------------------------------

          Show
          HBase Review Board added a comment - Message from: "Carl Steinbach" <carl@cloudera.com> On 2010-08-09 15:51:55, John Sichi wrote: > metastore/src/model/package.jdo, line 11 > < http://review.cloudera.org/r/508/diff/2/?file=5193#file5193line11 > > > Do we really need to change this column name? I can see that it doesn't follow the conventions for other tables, but it might make upgrades a pain. > I reverted the change. On 2010-08-09 15:51:55, John Sichi wrote: > ql/src/java/org/apache/hadoop/hive/ql/plan/ShowDatabasesDesc.java, line 41 > < http://review.cloudera.org/r/508/diff/2/?file=5209#file5209line41 > > > Should this be db_name instead of tab_name? Changed. Carl ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: http://review.cloudera.org/r/508/#review778 -----------------------------------------------------------
          Hide
          HBase Review Board added a comment -

          Message from: "Carl Steinbach" <carl@cloudera.com>

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          http://review.cloudera.org/r/508/
          -----------------------------------------------------------

          (Updated 2010-08-16 02:05:39.708868)

          Review request for Hive Developers.

          Changes
          -------

          Updated diff with https://issues.apache.org/jira/secure/attachment/12419726/hive-675-2009-9-16.patch

          Summary
          -------

          Database/Scheme support for Hive.

          • Implemented 'SHOW DATABASES' command
          • Refactored TestHiveMetaStore and enabled tests for remote metastore client.
          • Added launch configurations for TestHiveMetaStore and TestHiveMetaStoreRemote

          This addresses bug HIVE-675.
          http://issues.apache.org/jira/browse/HIVE-675

          Diffs (updated)


          .gitignore 376cdc6
          build-common.xml d4ff895
          eclipse-templates/TestEmbeddedHiveMetaStore.launchtemplate PRE-CREATION
          eclipse-templates/TestHive.launchtemplate 24efc12
          eclipse-templates/TestRemoteHiveMetaStore.launchtemplate PRE-CREATION
          metastore/if/hive_metastore.thrift 478d0af
          metastore/src/gen-cpp/ThriftHiveMetastore.h e2538fb
          metastore/src/gen-cpp/ThriftHiveMetastore.cpp f945a3a
          metastore/src/gen-cpp/ThriftHiveMetastore_server.skeleton.cpp ed2bb99
          metastore/src/gen-cpp/hive_metastore_types.h 1b0c706
          metastore/src/gen-cpp/hive_metastore_types.cpp b5a403d
          metastore/src/gen-javabean/org/apache/hadoop/hive/metastore/api/Database.java 78c78d9
          metastore/src/gen-javabean/org/apache/hadoop/hive/metastore/api/ThriftHiveMetastore.java 25408d9
          metastore/src/gen-php/ThriftHiveMetastore.php ea4add5
          metastore/src/gen-php/hive_metastore_types.php 61872a0
          metastore/src/gen-py/hive_metastore/ThriftHiveMetastore-remote fc06cba
          metastore/src/gen-py/hive_metastore/ThriftHiveMetastore.py 4a0bc67
          metastore/src/gen-py/hive_metastore/ttypes.py ea7269e
          metastore/src/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java 39dbd52
          metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 4fb296a
          metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java c6541af
          metastore/src/java/org/apache/hadoop/hive/metastore/IMetaStoreClient.java 6013644
          metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java 0818689
          metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java a06384c
          metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java 4951bd6
          metastore/src/java/org/apache/hadoop/hive/metastore/Warehouse.java 4488f94
          metastore/src/model/org/apache/hadoop/hive/metastore/model/MDatabase.java b3e098d
          metastore/src/model/package.jdo 206ba75
          metastore/src/test/org/apache/hadoop/hive/metastore/TestEmbeddedHiveMetaStore.java PRE-CREATION
          metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java fff6aad
          metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaStoreRemote.java bc950b9
          metastore/src/test/org/apache/hadoop/hive/metastore/TestRemoteHiveMetaStore.java PRE-CREATION
          ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java ef1da6b
          ql/src/java/org/apache/hadoop/hive/ql/exec/MoveTask.java d59f48c
          ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java e0761fc
          ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java 3300e2a
          ql/src/java/org/apache/hadoop/hive/ql/parse/BaseSemanticAnalyzer.java eedf9e3
          ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java 1801c10
          ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g b080780
          ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 682a8b5
          ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzerFactory.java 6af3b17
          ql/src/java/org/apache/hadoop/hive/ql/plan/CreateDatabaseDesc.java PRE-CREATION
          ql/src/java/org/apache/hadoop/hive/ql/plan/DDLWork.java ed4ed22
          ql/src/java/org/apache/hadoop/hive/ql/plan/DropDatabaseDesc.java PRE-CREATION
          ql/src/java/org/apache/hadoop/hive/ql/plan/ShowDatabasesDesc.java PRE-CREATION
          ql/src/java/org/apache/hadoop/hive/ql/plan/SwitchDatabaseDesc.java PRE-CREATION
          ql/src/test/org/apache/hadoop/hive/ql/QTestUtil.java b4651a2
          ql/src/test/org/apache/hadoop/hive/ql/metadata/TestHive.java ab39ca4
          ql/src/test/org/apache/hadoop/hive/ql/metadata/TestHiveMetaStoreChecker.java 26cc71a
          ql/src/test/queries/clientnegative/database_create_already_exists.q PRE-CREATION
          ql/src/test/queries/clientnegative/database_create_invalid_name.q PRE-CREATION
          ql/src/test/queries/clientnegative/database_drop_does_not_exist.q PRE-CREATION
          ql/src/test/queries/clientnegative/database_drop_not_empty.q PRE-CREATION
          ql/src/test/queries/clientnegative/database_switch_does_not_exist.q PRE-CREATION
          ql/src/test/queries/clientpositive/add_part_exist.q 54d0096
          ql/src/test/queries/clientpositive/alter1.q 6f95685
          ql/src/test/queries/clientpositive/alter2.q 2388e21
          ql/src/test/queries/clientpositive/alter3.q d9d3b07
          ql/src/test/queries/clientpositive/alter4.q 35fa441
          ql/src/test/queries/clientpositive/database.q PRE-CREATION
          ql/src/test/queries/clientpositive/rename_column.q ce82ff7
          ql/src/test/results/clientnegative/database_create_already_exists.q.out PRE-CREATION
          ql/src/test/results/clientnegative/database_create_invalid_name.q.out PRE-CREATION
          ql/src/test/results/clientnegative/database_drop_does_not_exist.q.out PRE-CREATION
          ql/src/test/results/clientnegative/database_drop_not_empty.q.out PRE-CREATION
          ql/src/test/results/clientnegative/database_switch_does_not_exist.q.out PRE-CREATION
          ql/src/test/results/clientpositive/add_part_exist.q.out 4bcdf8a
          ql/src/test/results/clientpositive/alter1.q.out ea143a6
          ql/src/test/results/clientpositive/alter2.q.out 6de5f19
          ql/src/test/results/clientpositive/alter3.q.out 2a1f4d5
          ql/src/test/results/clientpositive/alter4.q.out da7cd9a
          ql/src/test/results/clientpositive/database.q.out PRE-CREATION
          ql/src/test/results/clientpositive/rename_column.q.out 8d5da8d

          Diff: http://review.cloudera.org/r/508/diff

          Testing
          -------

          Thanks,

          Carl

          Show
          HBase Review Board added a comment - Message from: "Carl Steinbach" <carl@cloudera.com> ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: http://review.cloudera.org/r/508/ ----------------------------------------------------------- (Updated 2010-08-16 02:05:39.708868) Review request for Hive Developers. Changes ------- Updated diff with https://issues.apache.org/jira/secure/attachment/12419726/hive-675-2009-9-16.patch Summary ------- Database/Scheme support for Hive. Implemented 'SHOW DATABASES' command Refactored TestHiveMetaStore and enabled tests for remote metastore client. Added launch configurations for TestHiveMetaStore and TestHiveMetaStoreRemote This addresses bug HIVE-675 . http://issues.apache.org/jira/browse/HIVE-675 Diffs (updated) .gitignore 376cdc6 build-common.xml d4ff895 eclipse-templates/TestEmbeddedHiveMetaStore.launchtemplate PRE-CREATION eclipse-templates/TestHive.launchtemplate 24efc12 eclipse-templates/TestRemoteHiveMetaStore.launchtemplate PRE-CREATION metastore/if/hive_metastore.thrift 478d0af metastore/src/gen-cpp/ThriftHiveMetastore.h e2538fb metastore/src/gen-cpp/ThriftHiveMetastore.cpp f945a3a metastore/src/gen-cpp/ThriftHiveMetastore_server.skeleton.cpp ed2bb99 metastore/src/gen-cpp/hive_metastore_types.h 1b0c706 metastore/src/gen-cpp/hive_metastore_types.cpp b5a403d metastore/src/gen-javabean/org/apache/hadoop/hive/metastore/api/Database.java 78c78d9 metastore/src/gen-javabean/org/apache/hadoop/hive/metastore/api/ThriftHiveMetastore.java 25408d9 metastore/src/gen-php/ThriftHiveMetastore.php ea4add5 metastore/src/gen-php/hive_metastore_types.php 61872a0 metastore/src/gen-py/hive_metastore/ThriftHiveMetastore-remote fc06cba metastore/src/gen-py/hive_metastore/ThriftHiveMetastore.py 4a0bc67 metastore/src/gen-py/hive_metastore/ttypes.py ea7269e metastore/src/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java 39dbd52 metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 4fb296a metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java c6541af metastore/src/java/org/apache/hadoop/hive/metastore/IMetaStoreClient.java 6013644 metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java 0818689 metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java a06384c metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java 4951bd6 metastore/src/java/org/apache/hadoop/hive/metastore/Warehouse.java 4488f94 metastore/src/model/org/apache/hadoop/hive/metastore/model/MDatabase.java b3e098d metastore/src/model/package.jdo 206ba75 metastore/src/test/org/apache/hadoop/hive/metastore/TestEmbeddedHiveMetaStore.java PRE-CREATION metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java fff6aad metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaStoreRemote.java bc950b9 metastore/src/test/org/apache/hadoop/hive/metastore/TestRemoteHiveMetaStore.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java ef1da6b ql/src/java/org/apache/hadoop/hive/ql/exec/MoveTask.java d59f48c ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java e0761fc ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java 3300e2a ql/src/java/org/apache/hadoop/hive/ql/parse/BaseSemanticAnalyzer.java eedf9e3 ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java 1801c10 ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g b080780 ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 682a8b5 ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzerFactory.java 6af3b17 ql/src/java/org/apache/hadoop/hive/ql/plan/CreateDatabaseDesc.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/plan/DDLWork.java ed4ed22 ql/src/java/org/apache/hadoop/hive/ql/plan/DropDatabaseDesc.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/plan/ShowDatabasesDesc.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/plan/SwitchDatabaseDesc.java PRE-CREATION ql/src/test/org/apache/hadoop/hive/ql/QTestUtil.java b4651a2 ql/src/test/org/apache/hadoop/hive/ql/metadata/TestHive.java ab39ca4 ql/src/test/org/apache/hadoop/hive/ql/metadata/TestHiveMetaStoreChecker.java 26cc71a ql/src/test/queries/clientnegative/database_create_already_exists.q PRE-CREATION ql/src/test/queries/clientnegative/database_create_invalid_name.q PRE-CREATION ql/src/test/queries/clientnegative/database_drop_does_not_exist.q PRE-CREATION ql/src/test/queries/clientnegative/database_drop_not_empty.q PRE-CREATION ql/src/test/queries/clientnegative/database_switch_does_not_exist.q PRE-CREATION ql/src/test/queries/clientpositive/add_part_exist.q 54d0096 ql/src/test/queries/clientpositive/alter1.q 6f95685 ql/src/test/queries/clientpositive/alter2.q 2388e21 ql/src/test/queries/clientpositive/alter3.q d9d3b07 ql/src/test/queries/clientpositive/alter4.q 35fa441 ql/src/test/queries/clientpositive/database.q PRE-CREATION ql/src/test/queries/clientpositive/rename_column.q ce82ff7 ql/src/test/results/clientnegative/database_create_already_exists.q.out PRE-CREATION ql/src/test/results/clientnegative/database_create_invalid_name.q.out PRE-CREATION ql/src/test/results/clientnegative/database_drop_does_not_exist.q.out PRE-CREATION ql/src/test/results/clientnegative/database_drop_not_empty.q.out PRE-CREATION ql/src/test/results/clientnegative/database_switch_does_not_exist.q.out PRE-CREATION ql/src/test/results/clientpositive/add_part_exist.q.out 4bcdf8a ql/src/test/results/clientpositive/alter1.q.out ea143a6 ql/src/test/results/clientpositive/alter2.q.out 6de5f19 ql/src/test/results/clientpositive/alter3.q.out 2a1f4d5 ql/src/test/results/clientpositive/alter4.q.out da7cd9a ql/src/test/results/clientpositive/database.q.out PRE-CREATION ql/src/test/results/clientpositive/rename_column.q.out 8d5da8d Diff: http://review.cloudera.org/r/508/diff Testing ------- Thanks, Carl
          Hide
          HBase Review Board added a comment -

          Message from: "Carl Steinbach" <carl@cloudera.com>

          On 2010-08-11 06:57:04, namit jain wrote:

          > ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g, line 572

          > <http://review.cloudera.org/r/508/diff/2/?file=5203#file5203line572>

          >

          > remove KW_SCHEMAS

          In MySQL SHOW SCHEMAS is a synonym for SHOW DATABASES as of version 5.0.2. I think the general convention in Hive is to mimic the behavior of MySQL. Please let me know if there is a reason why we should not do that here.

          http://dev.mysql.com/doc/refman/5.0/en/show-databases.html

          • Carl

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          http://review.cloudera.org/r/508/#review838
          -----------------------------------------------------------

          Show
          HBase Review Board added a comment - Message from: "Carl Steinbach" <carl@cloudera.com> On 2010-08-11 06:57:04, namit jain wrote: > ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g, line 572 > < http://review.cloudera.org/r/508/diff/2/?file=5203#file5203line572 > > > remove KW_SCHEMAS In MySQL SHOW SCHEMAS is a synonym for SHOW DATABASES as of version 5.0.2. I think the general convention in Hive is to mimic the behavior of MySQL. Please let me know if there is a reason why we should not do that here. http://dev.mysql.com/doc/refman/5.0/en/show-databases.html Carl ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: http://review.cloudera.org/r/508/#review838 -----------------------------------------------------------
          Hide
          HBase Review Board added a comment -

          Message from: "namit jain" <njain@facebook.com>

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          http://review.cloudera.org/r/508/#review838
          -----------------------------------------------------------

          metastore/if/hive_metastore.thrift
          <http://review.cloudera.org/r/508/#comment2816>

          Same as John's earlier comment - do we need
          to change the name.

          No-one is using this table in the metastore currently,
          so it might be OK - but not sure, how JDO would
          handle renaming a column

          ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g
          <http://review.cloudera.org/r/508/#comment2817>

          remove KW_SCHEMAS

          ql/src/test/queries/clientpositive/database.q
          <http://review.cloudera.org/r/508/#comment2818>

          also add a test for 'show databases for a pattern'

          • namit
          Show
          HBase Review Board added a comment - Message from: "namit jain" <njain@facebook.com> ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: http://review.cloudera.org/r/508/#review838 ----------------------------------------------------------- metastore/if/hive_metastore.thrift < http://review.cloudera.org/r/508/#comment2816 > Same as John's earlier comment - do we need to change the name. No-one is using this table in the metastore currently, so it might be OK - but not sure, how JDO would handle renaming a column ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g < http://review.cloudera.org/r/508/#comment2817 > remove KW_SCHEMAS ql/src/test/queries/clientpositive/database.q < http://review.cloudera.org/r/508/#comment2818 > also add a test for 'show databases for a pattern' namit
          Hide
          Namit Jain added a comment -
          Show
          Namit Jain added a comment - comments posted on http://review.cloudera.org/r/508/diff
          Hide
          HBase Review Board added a comment -

          Message from: "John Sichi" <jsichi@facebook.com>

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          http://review.cloudera.org/r/508/#review778
          -----------------------------------------------------------

          metastore/src/model/package.jdo
          <http://review.cloudera.org/r/508/#comment2660>

          Do we really need to change this column name? I can see that it doesn't follow the conventions for other tables, but it might make upgrades a pain.

          ql/src/java/org/apache/hadoop/hive/ql/plan/ShowDatabasesDesc.java
          <http://review.cloudera.org/r/508/#comment2651>

          Should this be db_name instead of tab_name?

          • John
          Show
          HBase Review Board added a comment - Message from: "John Sichi" <jsichi@facebook.com> ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: http://review.cloudera.org/r/508/#review778 ----------------------------------------------------------- metastore/src/model/package.jdo < http://review.cloudera.org/r/508/#comment2660 > Do we really need to change this column name? I can see that it doesn't follow the conventions for other tables, but it might make upgrades a pain. ql/src/java/org/apache/hadoop/hive/ql/plan/ShowDatabasesDesc.java < http://review.cloudera.org/r/508/#comment2651 > Should this be db_name instead of tab_name? John
          Hide
          John Sichi added a comment -

          Any followup work needed here for HIVE-1126?

          Show
          John Sichi added a comment - Any followup work needed here for HIVE-1126 ?
          Hide
          John Sichi added a comment -

          I'm getting conflicts attempting to apply HIVE-675-2010-8-4.patch.txt on latest trunk.

          Show
          John Sichi added a comment - I'm getting conflicts attempting to apply HIVE-675 -2010-8-4.patch.txt on latest trunk.
          Hide
          Namit Jain added a comment -

          Thanks Carl, I will take a look

          Show
          Namit Jain added a comment - Thanks Carl, I will take a look
          Hide
          HBase Review Board added a comment -

          Message from: "Carl Steinbach" <carl@cloudera.com>

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          http://review.cloudera.org/r/508/
          -----------------------------------------------------------

          Review request for Hive Developers.

          Summary
          -------

          Database/Scheme support for Hive.

          • Implemented 'SHOW DATABASES' command
          • Refactored TestHiveMetaStore and enabled tests for remote metastore client.
          • Added launch configurations for TestHiveMetaStore and TestHiveMetaStoreRemote

          This addresses bug HIVE-675.
          http://issues.apache.org/jira/browse/HIVE-675

          Diffs


          build-common.xml d4ff895
          eclipse-templates/TestHive.launchtemplate 24efc12
          eclipse-templates/TestHiveMetaStore.launchtemplate PRE-CREATION
          eclipse-templates/TestHiveMetaStoreRemote.launchtemplate PRE-CREATION
          metastore/if/hive_metastore.thrift 478d0af
          metastore/src/gen-cpp/ThriftHiveMetastore.h e2538fb
          metastore/src/gen-cpp/ThriftHiveMetastore.cpp f945a3a
          metastore/src/gen-cpp/ThriftHiveMetastore_server.skeleton.cpp ed2bb99
          metastore/src/gen-cpp/hive_metastore_types.h 1b0c706
          metastore/src/gen-cpp/hive_metastore_types.cpp b5a403d
          metastore/src/gen-javabean/org/apache/hadoop/hive/metastore/api/Database.java 78c78d9
          metastore/src/gen-javabean/org/apache/hadoop/hive/metastore/api/ThriftHiveMetastore.java 25408d9
          metastore/src/gen-php/ThriftHiveMetastore.php ea4add5
          metastore/src/gen-php/hive_metastore_types.php 61872a0
          metastore/src/gen-py/hive_metastore/ThriftHiveMetastore-remote fc06cba
          metastore/src/gen-py/hive_metastore/ThriftHiveMetastore.py 4a0bc67
          metastore/src/gen-py/hive_metastore/ttypes.py ea7269e
          metastore/src/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java 39dbd52
          metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 4fb296a
          metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java c6541af
          metastore/src/java/org/apache/hadoop/hive/metastore/IMetaStoreClient.java 6013644
          metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java 0818689
          metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java a06384c
          metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java 4951bd6
          metastore/src/java/org/apache/hadoop/hive/metastore/Warehouse.java 4488f94
          metastore/src/model/org/apache/hadoop/hive/metastore/model/MDatabase.java b3e098d
          metastore/src/model/package.jdo 206ba75
          metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java fff6aad
          metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaStoreBase.java PRE-CREATION
          metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaStoreRemote.java bc950b9
          ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java bc268a4
          ql/src/java/org/apache/hadoop/hive/ql/exec/MoveTask.java d59f48c
          ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java 04dd9e3
          ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java 2ecda01
          ql/src/java/org/apache/hadoop/hive/ql/parse/BaseSemanticAnalyzer.java eedf9e3
          ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java 0484c91
          ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g 02bf926
          ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 70cd05f
          ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzerFactory.java eb079aa
          ql/src/java/org/apache/hadoop/hive/ql/plan/CreateDatabaseDesc.java PRE-CREATION
          ql/src/java/org/apache/hadoop/hive/ql/plan/DDLWork.java ed4ed22
          ql/src/java/org/apache/hadoop/hive/ql/plan/DropDatabaseDesc.java PRE-CREATION
          ql/src/java/org/apache/hadoop/hive/ql/plan/ShowDatabasesDesc.java PRE-CREATION
          ql/src/java/org/apache/hadoop/hive/ql/plan/SwitchDatabaseDesc.java PRE-CREATION
          ql/src/test/org/apache/hadoop/hive/ql/QTestUtil.java b4651a2
          ql/src/test/org/apache/hadoop/hive/ql/metadata/TestHive.java ab39ca4
          ql/src/test/org/apache/hadoop/hive/ql/metadata/TestHiveMetaStoreChecker.java 26cc71a
          ql/src/test/queries/clientnegative/create_database_bad_name.q PRE-CREATION
          ql/src/test/queries/clientnegative/database_switch_test.q PRE-CREATION
          ql/src/test/queries/clientpositive/database.q PRE-CREATION
          ql/src/test/results/clientnegative/create_database_bad_name.q.out PRE-CREATION
          ql/src/test/results/clientnegative/database_switch_test.q.out PRE-CREATION
          ql/src/test/results/clientpositive/database.q.out PRE-CREATION

          Diff: http://review.cloudera.org/r/508/diff

          Testing
          -------

          Thanks,

          Carl

          Show
          HBase Review Board added a comment - Message from: "Carl Steinbach" <carl@cloudera.com> ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: http://review.cloudera.org/r/508/ ----------------------------------------------------------- Review request for Hive Developers. Summary ------- Database/Scheme support for Hive. Implemented 'SHOW DATABASES' command Refactored TestHiveMetaStore and enabled tests for remote metastore client. Added launch configurations for TestHiveMetaStore and TestHiveMetaStoreRemote This addresses bug HIVE-675 . http://issues.apache.org/jira/browse/HIVE-675 Diffs build-common.xml d4ff895 eclipse-templates/TestHive.launchtemplate 24efc12 eclipse-templates/TestHiveMetaStore.launchtemplate PRE-CREATION eclipse-templates/TestHiveMetaStoreRemote.launchtemplate PRE-CREATION metastore/if/hive_metastore.thrift 478d0af metastore/src/gen-cpp/ThriftHiveMetastore.h e2538fb metastore/src/gen-cpp/ThriftHiveMetastore.cpp f945a3a metastore/src/gen-cpp/ThriftHiveMetastore_server.skeleton.cpp ed2bb99 metastore/src/gen-cpp/hive_metastore_types.h 1b0c706 metastore/src/gen-cpp/hive_metastore_types.cpp b5a403d metastore/src/gen-javabean/org/apache/hadoop/hive/metastore/api/Database.java 78c78d9 metastore/src/gen-javabean/org/apache/hadoop/hive/metastore/api/ThriftHiveMetastore.java 25408d9 metastore/src/gen-php/ThriftHiveMetastore.php ea4add5 metastore/src/gen-php/hive_metastore_types.php 61872a0 metastore/src/gen-py/hive_metastore/ThriftHiveMetastore-remote fc06cba metastore/src/gen-py/hive_metastore/ThriftHiveMetastore.py 4a0bc67 metastore/src/gen-py/hive_metastore/ttypes.py ea7269e metastore/src/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java 39dbd52 metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 4fb296a metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java c6541af metastore/src/java/org/apache/hadoop/hive/metastore/IMetaStoreClient.java 6013644 metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java 0818689 metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java a06384c metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java 4951bd6 metastore/src/java/org/apache/hadoop/hive/metastore/Warehouse.java 4488f94 metastore/src/model/org/apache/hadoop/hive/metastore/model/MDatabase.java b3e098d metastore/src/model/package.jdo 206ba75 metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java fff6aad metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaStoreBase.java PRE-CREATION metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaStoreRemote.java bc950b9 ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java bc268a4 ql/src/java/org/apache/hadoop/hive/ql/exec/MoveTask.java d59f48c ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java 04dd9e3 ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java 2ecda01 ql/src/java/org/apache/hadoop/hive/ql/parse/BaseSemanticAnalyzer.java eedf9e3 ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java 0484c91 ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g 02bf926 ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 70cd05f ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzerFactory.java eb079aa ql/src/java/org/apache/hadoop/hive/ql/plan/CreateDatabaseDesc.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/plan/DDLWork.java ed4ed22 ql/src/java/org/apache/hadoop/hive/ql/plan/DropDatabaseDesc.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/plan/ShowDatabasesDesc.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/plan/SwitchDatabaseDesc.java PRE-CREATION ql/src/test/org/apache/hadoop/hive/ql/QTestUtil.java b4651a2 ql/src/test/org/apache/hadoop/hive/ql/metadata/TestHive.java ab39ca4 ql/src/test/org/apache/hadoop/hive/ql/metadata/TestHiveMetaStoreChecker.java 26cc71a ql/src/test/queries/clientnegative/create_database_bad_name.q PRE-CREATION ql/src/test/queries/clientnegative/database_switch_test.q PRE-CREATION ql/src/test/queries/clientpositive/database.q PRE-CREATION ql/src/test/results/clientnegative/create_database_bad_name.q.out PRE-CREATION ql/src/test/results/clientnegative/database_switch_test.q.out PRE-CREATION ql/src/test/results/clientpositive/database.q.out PRE-CREATION Diff: http://review.cloudera.org/r/508/diff Testing ------- Thanks, Carl
          Hide
          Carl Steinbach added a comment -

          Posted on reviewboard: https://review.cloudera.org/r/508/

          Show
          Carl Steinbach added a comment - Posted on reviewboard: https://review.cloudera.org/r/508/
          Hide
          Carl Steinbach added a comment -

          HIVE-675-2010-8-4.patch.txt:

          • Implemented 'SHOW DATABASES' command
          • Refactored TestHiveMetaStore and enabled tests for remote metastore client.
          • Added launch configurations for TestHiveMetaStore and TestHiveMetaStoreRemote
          Show
          Carl Steinbach added a comment - HIVE-675 -2010-8-4.patch.txt: Implemented 'SHOW DATABASES' command Refactored TestHiveMetaStore and enabled tests for remote metastore client. Added launch configurations for TestHiveMetaStore and TestHiveMetaStoreRemote
          Hide
          Carl Steinbach added a comment -
          • Rebased Yongqiang's patch against trunk.
          • Added support for SHOW DATABASES
          • Added check to prevent DROP DATABASE on a non-empty DB

          Todo:

          • Update testcases
          Show
          Carl Steinbach added a comment - Rebased Yongqiang's patch against trunk. Added support for SHOW DATABASES Added check to prevent DROP DATABASE on a non-empty DB Todo: Update testcases
          Hide
          Carl Steinbach added a comment -

          @Pradeep: I'm almost done with this. Will post an updated patch soon.

          Show
          Carl Steinbach added a comment - @Pradeep: I'm almost done with this. Will post an updated patch soon.
          Hide
          Pradeep Kamath added a comment -

          Any updates on this patch? The last one was in May - any chance this will be committed soon or are there outstanding issues?

          Show
          Pradeep Kamath added a comment - Any updates on this patch? The last one was in May - any chance this will be committed soon or are there outstanding issues?
          Hide
          Carl Steinbach added a comment -

          I'm in the middle of updating this patch and plan to post the result in the next day.

          Show
          Carl Steinbach added a comment - I'm in the middle of updating this patch and plan to post the result in the next day.
          Hide
          Alex Loddengaard added a comment -

          Hi guys,

          Any updates here? Would be great to have this patch committed!

          Thanks,

          Alex

          Show
          Alex Loddengaard added a comment - Hi guys, Any updates here? Would be great to have this patch committed! Thanks, Alex
          Hide
          Alex Loddengaard added a comment -

          I will be out of the office Thursday, 1/7, through Wednesday, 1/13,
          back in the office Thursday, 1/14. I will be checking email fairly
          consistently in the evenings.

          Please contact Christophe Bisciglia (christophe@cloudera.com) with any
          support or training emergencies. Otherwise, you'll hear from me soon.

          Thanks,

          Alex

          Show
          Alex Loddengaard added a comment - I will be out of the office Thursday, 1/7, through Wednesday, 1/13, back in the office Thursday, 1/14. I will be checking email fairly consistently in the evenings. Please contact Christophe Bisciglia (christophe@cloudera.com) with any support or training emergencies. Otherwise, you'll hear from me soon. Thanks, Alex
          Hide
          Zheng Shao added a comment -

          Yongqiang, you can start adapt the patch to trunk right now.

          The plan is to branch 0.5 on 1/7/2010 (tomorrow). After that we can quickly review this diff and get it in.

          Show
          Zheng Shao added a comment - Yongqiang, you can start adapt the patch to trunk right now. The plan is to branch 0.5 on 1/7/2010 (tomorrow). After that we can quickly review this diff and get it in.
          Hide
          He Yongqiang added a comment -

          Hi Jeff,

          Sorry for the delay. Actually it is now holding for the release of 0.5 because it has many related jiras. Let's commit this after the release of 0.5.

          Show
          He Yongqiang added a comment - Hi Jeff, Sorry for the delay. Actually it is now holding for the release of 0.5 because it has many related jiras. Let's commit this after the release of 0.5.
          Hide
          Jeff Hammerbacher added a comment -

          Hey,

          Is this patch in a state where it could go into 0.5? If it's not going to be polished off, please let us know.

          Thanks,
          Jeff

          Show
          Jeff Hammerbacher added a comment - Hey, Is this patch in a state where it could go into 0.5? If it's not going to be polished off, please let us know. Thanks, Jeff
          Hide
          He Yongqiang added a comment -

          Hey Jeff,
          Sorry, i just saw your comments. And sorry for missing your earlier comment. The patch was ready to commit, only need a testcase. But i am sure it need to be adjust to trunk code. I will work out a new version soon.

          Show
          He Yongqiang added a comment - Hey Jeff, Sorry, i just saw your comments. And sorry for missing your earlier comment. The patch was ready to commit, only need a testcase. But i am sure it need to be adjust to trunk code. I will work out a new version soon.
          Hide
          Jeff Hammerbacher added a comment -

          Ping. Any chance this patch can be cleaned up and committed to trunk soon?

          Show
          Jeff Hammerbacher added a comment - Ping. Any chance this patch can be cleaned up and committed to trunk soon?
          Hide
          Jeff Hammerbacher added a comment -

          Hey Yongqiang,

          What's the status of this patch?

          Thanks,
          Jeff

          Show
          Jeff Hammerbacher added a comment - Hey Yongqiang, What's the status of this patch? Thanks, Jeff
          Hide
          He Yongqiang added a comment -

          Talked with Namit offline. The "create database" with an already existing directory as the db location will throw an exception, becaue databases will have tables as their sub-objects. the "drop database" command will also drop its corresponding data directory (as compatible with mysql and oracle's semantic).

          And need a test for the command "create database" with location, Namit will help writing the test code. Thanks Namit!

          Show
          He Yongqiang added a comment - Talked with Namit offline. The "create database" with an already existing directory as the db location will throw an exception, becaue databases will have tables as their sub-objects. the "drop database" command will also drop its corresponding data directory (as compatible with mysql and oracle's semantic). And need a test for the command "create database" with location, Namit will help writing the test code. Thanks Namit!
          Hide
          He Yongqiang added a comment -

          Sorry, the "FAILED: Unknown exception : null" will be adressed in a new diff.

          Show
          He Yongqiang added a comment - Sorry, the "FAILED: Unknown exception : null" will be adressed in a new diff.
          Hide
          He Yongqiang added a comment -

          >>Do you want to check for the existence of the location while creating a database - do we want to throw an error if it already exists.
          Currently, it will not throw any error if the database dir is already there. I think this is the same as creating table. And i always do "drop table" and "create table" stuff, and i can switch machines, and only need to redo the "create table ..." and the data is available. I think this is very useful.

          >>in create_db_bad_location, can you throw a more meaningful error - location is bad or something like that
          Actually i already throw an exception with msg "Database location must be an absolute URI.". But i think the error msg "FAILED: Unknown exception : null" is not unique to this, and i think it has been reported somewhere in mailist or a jira page.

          Show
          He Yongqiang added a comment - >>Do you want to check for the existence of the location while creating a database - do we want to throw an error if it already exists. Currently, it will not throw any error if the database dir is already there. I think this is the same as creating table. And i always do "drop table" and "create table" stuff, and i can switch machines, and only need to redo the "create table ..." and the data is available. I think this is very useful. >>in create_db_bad_location, can you throw a more meaningful error - location is bad or something like that Actually i already throw an exception with msg "Database location must be an absolute URI.". But i think the error msg "FAILED: Unknown exception : null" is not unique to this, and i think it has been reported somewhere in mailist or a jira page.
          Hide
          Namit Jain added a comment -

          Also, in create_db_bad_location, can you throw a more meaningful error – location is bad or something like that

          Show
          Namit Jain added a comment - Also, in create_db_bad_location, can you throw a more meaningful error – location is bad or something like that
          Hide
          Namit Jain added a comment -

          The changes look good - I am running tests right now.

          I just add 1 minor comment:
          Do you want to check for the existence of the location while creating a database - do we want to throw an error if it already exists.
          This applies to both default and user-specified location.

          Show
          Namit Jain added a comment - The changes look good - I am running tests right now. I just add 1 minor comment: Do you want to check for the existence of the location while creating a database - do we want to throw an error if it already exists. This applies to both default and user-specified location.
          Hide
          He Yongqiang added a comment -

          >>What is the reason for not using a default path (<hive.root>/dbname) if location argument is null? It is possible that tools other than Hive CLI can create databases and they don't necessarily know what is the default path should be.
          At the first try of implementing this feature, i thought it's not a good idea to host other database in the default db's directory (i did not know that the default db can not be drop at that time). So hive-675-2009-9-21.patch removed the constrain of must specifying a location.
          And also added "create database" testcase in positive cli test as Namit suggested in an earlier comment.
          Thanks Prasad and Namit!

          Show
          He Yongqiang added a comment - >>What is the reason for not using a default path (<hive.root>/dbname) if location argument is null? It is possible that tools other than Hive CLI can create databases and they don't necessarily know what is the default path should be. At the first try of implementing this feature, i thought it's not a good idea to host other database in the default db's directory (i did not know that the default db can not be drop at that time). So hive-675-2009-9-21.patch removed the constrain of must specifying a location. And also added "create database" testcase in positive cli test as Namit suggested in an earlier comment. Thanks Prasad and Namit!
          Hide
          Prasad Chakka added a comment -

          What is the reason for not using a default path (<hive.root>/dbname) if location argument is null? It is possible that tools other than Hive CLI can create databases and they don't necessarily know what is the default path should be.

          Show
          Prasad Chakka added a comment - What is the reason for not using a default path (<hive.root>/dbname) if location argument is null? It is possible that tools other than Hive CLI can create databases and they don't necessarily know what is the default path should be.
          Hide
          He Yongqiang added a comment -

          A new patch integrates Namit's comments (Thanks Namit!).
          >>1. No positive test for CREATE DATEBASE
          CREATE DATEBASE needs an absolute uri argument. So i guess hive cannot get such an uri at testing.
          >>3. More positive tests - the test that you mentioned in the beginning - insert into db2 while selecting from db1 -
          This also needs an file system location argument.
          >>4. database_test.q and database_switch_test.q seem to be the same.
          This is bad mistake i made. I intend to rename database_test.q to database_switch_test.q, but forget to delete the old one.
          >>2. Instead of having a separate processor, it might be a good idea to support 'use database' in Hive.g - in general, we are trying to add everything to Hive.g, since it will make adding different clients easy.
          moved it from processor to hive.g

          will open several follow up for this jira.
          1. move table from one database to another
          2. support ' show databases';
          3. support mutilpe databse support in HWI
          4.

          this current path does not support <DBNAME>.<TBLNAME>.<COLUMN>. If you are not planning to add that support then create a new JIRA.

          5. moving SessionState from ql to common (from Prasad)

          please add more.

          Show
          He Yongqiang added a comment - A new patch integrates Namit's comments (Thanks Namit!). >>1. No positive test for CREATE DATEBASE CREATE DATEBASE needs an absolute uri argument. So i guess hive cannot get such an uri at testing. >>3. More positive tests - the test that you mentioned in the beginning - insert into db2 while selecting from db1 - This also needs an file system location argument. >>4. database_test.q and database_switch_test.q seem to be the same. This is bad mistake i made. I intend to rename database_test.q to database_switch_test.q, but forget to delete the old one. >>2. Instead of having a separate processor, it might be a good idea to support 'use database' in Hive.g - in general, we are trying to add everything to Hive.g, since it will make adding different clients easy. moved it from processor to hive.g will open several follow up for this jira. 1. move table from one database to another 2. support ' show databases'; 3. support mutilpe databse support in HWI 4. this current path does not support <DBNAME>.<TBLNAME>.<COLUMN>. If you are not planning to add that support then create a new JIRA. 5. moving SessionState from ql to common (from Prasad) please add more.
          Hide
          Namit Jain added a comment -

          Mostly looks good, I had some minor comments:

          1. No positive test for CREATE DATEBASE
          2. Instead of having a separate processor, it might be a good idea to support 'use database' in Hive.g - in general, we are trying to add everything to Hive.g, since it will make adding different clients easy.
          3. More positive tests - the test that you mentioned in the beginning - insert into db2 while selecting from db1 -
          4. database_test.q and database_switch_test.q seem to be the same.

          Show
          Namit Jain added a comment - Mostly looks good, I had some minor comments: 1. No positive test for CREATE DATEBASE 2. Instead of having a separate processor, it might be a good idea to support 'use database' in Hive.g - in general, we are trying to add everything to Hive.g, since it will make adding different clients easy. 3. More positive tests - the test that you mentioned in the beginning - insert into db2 while selecting from db1 - 4. database_test.q and database_switch_test.q seem to be the same.
          Hide
          Prasad Chakka added a comment -

          Yeah, i never meant to fix this in this JIRA. just saying that we might need it in the future.

          Show
          Prasad Chakka added a comment - Yeah, i never meant to fix this in this JIRA. just saying that we might need it in the future.
          Hide
          He Yongqiang added a comment -

          hive-675-2009-9-16.patch does not move SessionState from ql to common. Let's finish that in another jira issue. (Thanks Prasad and Zheng!)

          Show
          He Yongqiang added a comment - hive-675-2009-9-16.patch does not move SessionState from ql to common. Let's finish that in another jira issue. (Thanks Prasad and Zheng!)
          Hide
          Zheng Shao added a comment -

          I think moving SessionState is a separate issue. The current database is part of Hive.java, so this issue does not depend on moving SessionState.
          We can discuss when and how to move SessionState in a separate jira.

          Show
          Zheng Shao added a comment - I think moving SessionState is a separate issue. The current database is part of Hive.java, so this issue does not depend on moving SessionState. We can discuss when and how to move SessionState in a separate jira.
          Hide
          He Yongqiang added a comment -

          There are two classes SessionState refers may block moving SessionState from ql to common, Utilities and HiveHistory.
          For Utilities, we can split it and add another Utilites to common package.
          But it is not easy for HiveHistory, it refers to QueryPlan and Task, which then refer to many others...
          So i think this would need a refactory of packing?

          Show
          He Yongqiang added a comment - There are two classes SessionState refers may block moving SessionState from ql to common, Utilities and HiveHistory. For Utilities, we can split it and add another Utilites to common package. But it is not easy for HiveHistory, it refers to QueryPlan and Task, which then refer to many others... So i think this would need a refactory of packing?
          Hide
          Prasad Chakka added a comment -

          There is no way to refer to SessionState from inside of hive.ql.metadata or hive.metadata.

          I think this problem is going to bite us. We may need to access Session context from all over the package so we need to put all this information in Common which is accessible to all.

          Show
          Prasad Chakka added a comment - There is no way to refer to SessionState from inside of hive.ql.metadata or hive.metadata. I think this problem is going to bite us. We may need to access Session context from all over the package so we need to put all this information in Common which is accessible to all.
          Hide
          He Yongqiang added a comment -

          >>if approach A for HIVE-584 is chosen then shouldn't the current patch work without any changes?
          yes, it will work without changes.
          But Hive is now a thread local obj itself, so just as Zheng said, we don't need to protect the current database with another level of ThreadLocal.

          Show
          He Yongqiang added a comment - >>if approach A for HIVE-584 is chosen then shouldn't the current patch work without any changes? yes, it will work without changes. But Hive is now a thread local obj itself, so just as Zheng said, we don't need to protect the current database with another level of ThreadLocal.
          Hide
          Prasad Chakka added a comment -

          wait, if approach A for HIVE-584 is chosen then shouldn't the current patch work without any changes?

          Show
          Prasad Chakka added a comment - wait, if approach A for HIVE-584 is chosen then shouldn't the current patch work without any changes?
          Hide
          He Yongqiang added a comment -

          Talked offline with Zheng, will upload a new fix when hive-584 committed.

          Show
          He Yongqiang added a comment - Talked offline with Zheng, will upload a new fix when hive-584 committed.
          Hide
          He Yongqiang added a comment -

          >>Assuming Hive is an inner field of SessionState, we don't need to protect the current database with another level of ThreadLocal.
          yes. hive is a singleton object.

           
          public class Hive {
          
            static final private Log LOG = LogFactory.getLog("hive.ql.metadata.Hive");
            static Hive db = null;
          .........
            public static Hive get(HiveConf c, boolean needsRefresh) throws HiveException {
              if(db == null || needsRefresh) {
                closeCurrent();
                c.set("fs.scheme.class","dfs");
                db = new Hive(c);
              }
              return db;
            }
          

          So even though Hive is an inner field of SessionState, but i think they all refer to the singleton instance at runtime. So we need to make the current database be a ThreadLocal variable.

          Show
          He Yongqiang added a comment - >>Assuming Hive is an inner field of SessionState, we don't need to protect the current database with another level of ThreadLocal. yes. hive is a singleton object. public class Hive { static final private Log LOG = LogFactory.getLog("hive.ql.metadata.Hive"); static Hive db = null; ......... public static Hive get(HiveConf c, boolean needsRefresh) throws HiveException { if(db == null || needsRefresh) { closeCurrent(); c.set("fs.scheme.class","dfs"); db = new Hive(c); } return db; } So even though Hive is an inner field of SessionState, but i think they all refer to the singleton instance at runtime. So we need to make the current database be a ThreadLocal variable.
          Hide
          Zheng Shao added a comment -

          I see. Assuming Hive is an inner field of SessionState, we don't need to protect the current database with another level of ThreadLocal.
          Thoughts?

          Show
          Zheng Shao added a comment - I see. Assuming Hive is an inner field of SessionState, we don't need to protect the current database with another level of ThreadLocal. Thoughts?
          Hide
          He Yongqiang added a comment -

          is hive always used with a SessionState?
          I think hive is an inner field of SessionState, so it maybe not good to call its container(parent session object) currentDatabase().

          Show
          He Yongqiang added a comment - is hive always used with a SessionState? I think hive is an inner field of SessionState, so it maybe not good to call its container(parent session object) currentDatabase().
          Hide
          Zheng Shao added a comment -

          Shall we put currentDatabase() into Session? We don't want another ThreadLocal variable.

          Show
          Zheng Shao added a comment - Shall we put currentDatabase() into Session? We don't want another ThreadLocal variable.
          Hide
          Prasad Chakka added a comment -

          Yes, by default currentDatabase() can be used and since only Hive.java holds the current database name, it should be fine.

          otherwise patch looks good.

          Show
          Prasad Chakka added a comment - Yes, by default currentDatabase() can be used and since only Hive.java holds the current database name, it should be fine. otherwise patch looks good.
          Hide
          He Yongqiang added a comment -

          Prasad, when i am looking into deprecating methods in Hive, i think we may need to open a new jira to track that. Hive.java has its currentDatabase tracking which database is currently in use, so it's reasonable to have methods without dbName argument. And most of methods in Hive.java do not have a dbName argument, they will use the current db by default. Do we really need to add new methods with the dbname argument now? What do you think?

          Show
          He Yongqiang added a comment - Prasad, when i am looking into deprecating methods in Hive, i think we may need to open a new jira to track that. Hive.java has its currentDatabase tracking which database is currently in use, so it's reasonable to have methods without dbName argument. And most of methods in Hive.java do not have a dbName argument, they will use the current db by default. Do we really need to add new methods with the dbname argument now? What do you think?
          Hide
          He Yongqiang added a comment -

          >>MetaStoreUtils.DEFAULT_DATABASE_NAME is sometimes replaced with currentDatabase and sometimes left empty (in DDLTask.java) why?
          When empty in DDLTask, it will use getCurrentDatabase() as the dbName inside Hive.

          Show
          He Yongqiang added a comment - >>MetaStoreUtils.DEFAULT_DATABASE_NAME is sometimes replaced with currentDatabase and sometimes left empty (in DDLTask.java) why? When empty in DDLTask, it will use getCurrentDatabase() as the dbName inside Hive.
          Hide
          He Yongqiang added a comment -

          >>i thought all Hive.java functions that do not take database name as argument are deprecated.
          yes. exactly. I was thinking to open a new jira to deprecate/remove them. I will make them deprecated in a new patch for this jira.

          Show
          He Yongqiang added a comment - >>i thought all Hive.java functions that do not take database name as argument are deprecated. yes. exactly. I was thinking to open a new jira to deprecate/remove them. I will make them deprecated in a new patch for this jira.
          Hide
          Prasad Chakka added a comment -

          MetaStoreUtils.DEFAULT_DATABASE_NAME is sometimes replaced with currentDatabase and sometimes left empty (in DDLTask.java) why?
          i thought all Hive.java functions that do not take database name as argument are deprecated. no?

          Show
          Prasad Chakka added a comment - MetaStoreUtils.DEFAULT_DATABASE_NAME is sometimes replaced with currentDatabase and sometimes left empty (in DDLTask.java) why? i thought all Hive.java functions that do not take database name as argument are deprecated. no?
          Hide
          He Yongqiang added a comment -

          Thanks for the detailed comments, Prasad!

          1) more test clientpositive & clientnegative tests

          added several tests.

          2) db location should take an URI rather than a path. since location can point to a different HDFS instance
          done.
          5) HiveMetaStore.java: if the given db location is not a proper URI then it is silently ignored and default path is being used. HiveMetastore should throw an error instead if the path is not absolute or a proper URI
          6) remove ql/junit233767264.properties from patch
          7) remove ql/junitvmwatcher1237844024.properties from patch
          8) there are lot of other locations where MetaStoreUtils.DEFAULT_DATABASE_NAME is used. It should be replaced by Hive.currentDatabase
          10) Hive.createDatabase() & dropDatabase() should throw AlreadyExistsException() instead of throwin HiveException
          11) rename isDatabaseExist() to databaseExists()

          done

          3) this path does not support <DBNAME>.<TBLNAME>.<COLUMN>. If you are not planning to add that support then create a new JIRA.
          4) there should be a way to easily move tbl from on db to another db. may be open a JIRA for that as well?

          Let's open new jiras. Once multiple databases is supported at client, lots of new issues will need to be created.

          9) is Hive.currentDatabase thread-safe. does it work in HiveServer where multiple threads can each have separate currentDatabases?
          12) currentDatabase should either be in SessionState or Hive. There is no need for it to be stored in both objects.

          currentDatabase is now removed from SessionState, and is stored as a threadlocal variable in Hive.

          Show
          He Yongqiang added a comment - Thanks for the detailed comments, Prasad! 1) more test clientpositive & clientnegative tests added several tests. 2) db location should take an URI rather than a path. since location can point to a different HDFS instance done. 5) HiveMetaStore.java: if the given db location is not a proper URI then it is silently ignored and default path is being used. HiveMetastore should throw an error instead if the path is not absolute or a proper URI 6) remove ql/junit233767264.properties from patch 7) remove ql/junitvmwatcher1237844024.properties from patch 8) there are lot of other locations where MetaStoreUtils.DEFAULT_DATABASE_NAME is used. It should be replaced by Hive.currentDatabase 10) Hive.createDatabase() & dropDatabase() should throw AlreadyExistsException() instead of throwin HiveException 11) rename isDatabaseExist() to databaseExists() done 3) this path does not support <DBNAME>.<TBLNAME>.<COLUMN>. If you are not planning to add that support then create a new JIRA. 4) there should be a way to easily move tbl from on db to another db. may be open a JIRA for that as well? Let's open new jiras. Once multiple databases is supported at client, lots of new issues will need to be created. 9) is Hive.currentDatabase thread-safe. does it work in HiveServer where multiple threads can each have separate currentDatabases? 12) currentDatabase should either be in SessionState or Hive. There is no need for it to be stored in both objects. currentDatabase is now removed from SessionState, and is stored as a threadlocal variable in Hive.
          Hide
          Prasad Chakka added a comment -

          1) more test clientpositive & clientnegative tests
          2) db location should take an URI rather than a path. since location can point to a different HDFS instance
          3) this path does not support <DBNAME>.<TBLNAME>.<COLUMN>. If you are not planning to add that support then create a new JIRA.
          4) there should be a way to easily move tbl from on db to another db. may be open a JIRA for that as well?
          5) HiveMetaStore.java: if the given db location is not a proper URI then it is silently ignored and default path is being used. HiveMetastore should throw an error instead if the path is not absolute or a proper URI
          6) remove ql/junit233767264.properties from patch
          7) remove ql/junitvmwatcher1237844024.properties from patch
          8) there are lot of other locations where MetaStoreUtils.DEFAULT_DATABASE_NAME is used. It should be replaced by Hive.currentDatabase
          9) is Hive.currentDatabase thread-safe. does it work in HiveServer where multiple threads can each have separate currentDatabases?
          10) Hive.createDatabase() & dropDatabase() should throw AlreadyExistsException() instead of throwin HiveException
          11) rename isDatabaseExist() to databaseExists()
          12) currentDatabase should either be in SessionState or Hive. There is no need for it to be stored in both objects.

          Show
          Prasad Chakka added a comment - 1) more test clientpositive & clientnegative tests 2) db location should take an URI rather than a path. since location can point to a different HDFS instance 3) this path does not support <DBNAME>.<TBLNAME>.<COLUMN>. If you are not planning to add that support then create a new JIRA. 4) there should be a way to easily move tbl from on db to another db. may be open a JIRA for that as well? 5) HiveMetaStore.java: if the given db location is not a proper URI then it is silently ignored and default path is being used. HiveMetastore should throw an error instead if the path is not absolute or a proper URI 6) remove ql/junit233767264.properties from patch 7) remove ql/junitvmwatcher1237844024.properties from patch 8) there are lot of other locations where MetaStoreUtils.DEFAULT_DATABASE_NAME is used. It should be replaced by Hive.currentDatabase 9) is Hive.currentDatabase thread-safe. does it work in HiveServer where multiple threads can each have separate currentDatabases? 10) Hive.createDatabase() & dropDatabase() should throw AlreadyExistsException() instead of throwin HiveException 11) rename isDatabaseExist() to databaseExists() 12) currentDatabase should either be in SessionState or Hive. There is no need for it to be stored in both objects.
          Hide
          He Yongqiang added a comment -

          Added three new commands:
          1) "USE db_name" to switch among existing databases
          2) "CREATE DATABASE db_name LOCATION 'absolute_path_of_db_location' "
          3) "DROP DATABASE db_name"

          By default, hive use the 'default' database.
          User can use the commands

          use db_1; 
          INSERT OVERWRITE DIRECTORY 'table_of_db2_hdfs_directory'  SELECT  ....  FROM table_of_db1 a ...";
          

          to query one table in db1 and insert results to table in another database (db2). This can achieve some basic security by the underlying file system used.

          Show
          He Yongqiang added a comment - Added three new commands: 1) "USE db_name" to switch among existing databases 2) "CREATE DATABASE db_name LOCATION 'absolute_path_of_db_location' " 3) "DROP DATABASE db_name" By default, hive use the 'default' database. User can use the commands use db_1; INSERT OVERWRITE DIRECTORY 'table_of_db2_hdfs_directory' SELECT .... FROM table_of_db1 a ..."; to query one table in db1 and insert results to table in another database (db2). This can achieve some basic security by the underlying file system used.

            People

            • Assignee:
              Carl Steinbach
              Reporter:
              Prasad Chakka
            • Votes:
              1 Vote for this issue
              Watchers:
              19 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development