Hive
  1. Hive
  2. HIVE-2612

support hive table/partitions exists in more than one region

    Details

    • Type: New Feature New Feature
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.9.0
    • Component/s: Metastore
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Requires a metastore upgrade.

      Description

      Note: The proposal for this feature was withdrawn, and this patch was subsequently reverted in HIVE-2965. This feature does not appear in any official release of Hive.

      1) add region object into hive metastore
      2) each partition/table has a primary region and a list of living regions, and also data location in each region

      1. HIVE-2612.8.patch.txt
        1.43 MB
        Kevin Wilfong
      2. ASF.LICENSE.NOT.GRANTED--HIVE-2612.D1707.1.patch
        1.43 MB
        Phabricator
      3. HIVE-2612.7.patch.txt
        1.43 MB
        Kevin Wilfong
      4. HIVE-2612.6.patch.txt
        1.56 MB
        Kevin Wilfong
      5. hive.2612.5.patch
        1.38 MB
        Namit Jain
      6. HIVE-2612.4.patch.txt
        1.14 MB
        Kevin Wilfong
      7. HIVE-2612.3.patch.txt
        1.13 MB
        Kevin Wilfong
      8. ASF.LICENSE.NOT.GRANTED--HIVE-2612.D1569.7.patch
        1.13 MB
        Phabricator
      9. ASF.LICENSE.NOT.GRANTED--HIVE-2612.D1569.6.patch
        1.13 MB
        Phabricator
      10. HIVE-2612.2.patch.txt
        1.13 MB
        Kevin Wilfong
      11. ASF.LICENSE.NOT.GRANTED--HIVE-2612.D1569.5.patch
        1.08 MB
        Phabricator
      12. ASF.LICENSE.NOT.GRANTED--HIVE-2612.D1569.4.patch
        1.08 MB
        Phabricator
      13. ASF.LICENSE.NOT.GRANTED--HIVE-2612.D1569.3.patch
        9 kB
        Phabricator
      14. ASF.LICENSE.NOT.GRANTED--HIVE-2612.D1569.2.patch
        10 kB
        Phabricator
      15. ASF.LICENSE.NOT.GRANTED--HIVE-2612.D1569.1.patch
        88 kB
        Phabricator
      16. HIVE-2612.1.patch
        1.03 MB
        He Yongqiang

        Issue Links

          Activity

          Hide
          Carl Steinbach added a comment -

          This is a big feature to implement. I really think it needs a design document before we start committing patches. Yongqiang, do you think you can write something up?

          Show
          Carl Steinbach added a comment - This is a big feature to implement. I really think it needs a design document before we start committing patches. Yongqiang, do you think you can write something up?
          Hide
          He Yongqiang added a comment -

          yeah. sure. internally we actually had some discussions around this. I can write something about it.

          Right now, our main concern is that the cluster feature may be not so useful for most people. So we want to know what other people think about the potential incompatibilities that this could introduce.

          actually sent out an discussion to dev@, copy it here:

          "
          We are planning to make hive run across multiple data centers
          (physical clusters). We prefer to use hive metastore to provide a
          unified namespace.
          Tables/partitions can exist in more than one cluster. And one cluster
          is defined as a primary cluster. A primary cluster is a table level
          property. A table T1's primary cluster is C1 meaning :1) C1 contains
          all data that is available in all other clusters. 2) write is only
          allowed in this cluster for table C1. but need to allow exceptions
          here 3) new partitions are only allowed to be created in C1. 4) all
          data changes to T1 happened in the primary cluster should be
          replicated to other clusters if there are any secondary clusters. but
          there should be a conf to disable it as there are some exception
          situations.

          The first thing that needs to be done is to make hive metastore have a
          concept of cluster. And that also means all thrift communication calls
          to metastore need to provide a cluster parameter. So we have there
          options here:
          1) add a cluster parameter to existing thrift interfaces
          or
          2) add new interfaces which do exactly the same set of functionalities
          as old ones but using a different name (use _on_cluster suffifx
          maybe?) and have a cluster parameter
          or
          3) overwrite database name for the purpose of cluster name. And allow
          a table co-exist in multiple databases. But that require to promote
          table to top level citizen, and degrade database. For example, "show
          tables" used to scan all tables in current db, but now need to scan
          all tables in all databases.

          We would like to get more ideas about which one to choose, and we are
          definitely open to other alternatives that we missed here.

          We are also looking for other systems that have solved similar
          problems. If anyone knows such a system, we would like to know.
          Appreciate that!
          "

          Show
          He Yongqiang added a comment - yeah. sure. internally we actually had some discussions around this. I can write something about it. Right now, our main concern is that the cluster feature may be not so useful for most people. So we want to know what other people think about the potential incompatibilities that this could introduce. actually sent out an discussion to dev@, copy it here: " We are planning to make hive run across multiple data centers (physical clusters). We prefer to use hive metastore to provide a unified namespace. Tables/partitions can exist in more than one cluster. And one cluster is defined as a primary cluster. A primary cluster is a table level property. A table T1's primary cluster is C1 meaning :1) C1 contains all data that is available in all other clusters. 2) write is only allowed in this cluster for table C1. but need to allow exceptions here 3) new partitions are only allowed to be created in C1. 4) all data changes to T1 happened in the primary cluster should be replicated to other clusters if there are any secondary clusters. but there should be a conf to disable it as there are some exception situations. The first thing that needs to be done is to make hive metastore have a concept of cluster. And that also means all thrift communication calls to metastore need to provide a cluster parameter. So we have there options here: 1) add a cluster parameter to existing thrift interfaces or 2) add new interfaces which do exactly the same set of functionalities as old ones but using a different name (use _on_cluster suffifx maybe?) and have a cluster parameter or 3) overwrite database name for the purpose of cluster name. And allow a table co-exist in multiple databases. But that require to promote table to top level citizen, and degrade database. For example, "show tables" used to scan all tables in current db, but now need to scan all tables in all databases. We would like to get more ideas about which one to choose, and we are definitely open to other alternatives that we missed here. We are also looking for other systems that have solved similar problems. If anyone knows such a system, we would like to know. Appreciate that! "
          Hide
          Steven Wong added a comment -

          Yongqiang, please clarify what you mean by option 3. Currently, databases contain tables (db1.foo is unrelated to db2.foo). Is option 3 saying make tables span databases (db1.foo is the same table as db2.foo) instead? That would be a radical change, so maybe I've misunderstood it.

          Show
          Steven Wong added a comment - Yongqiang, please clarify what you mean by option 3. Currently, databases contain tables (db1.foo is unrelated to db2.foo). Is option 3 saying make tables span databases (db1.foo is the same table as db2.foo) instead? That would be a radical change, so maybe I've misunderstood it.
          Hide
          He Yongqiang added a comment -

          yes. Your understanding is correct. db1.foo is the same table as db2.foo. It is listed as one potential option.

          Show
          He Yongqiang added a comment - yes. Your understanding is correct. db1.foo is the same table as db2.foo. It is listed as one potential option.
          Hide
          Steven Wong added a comment -

          Won't that conflict with authorization?

          Show
          Steven Wong added a comment - Won't that conflict with authorization?
          Hide
          He Yongqiang added a comment -

          Not just authorization. It requires a complete redesign of existing database feature. And the only benefit of doing that is the backward interface compatibility.

          @Carl, this should not be just a metastore change. We should also provide an utility to replicate data. but we can do it in a followup and focus on the metastore changes on this jira.

          Show
          He Yongqiang added a comment - Not just authorization. It requires a complete redesign of existing database feature. And the only benefit of doing that is the backward interface compatibility. @Carl, this should not be just a metastore change. We should also provide an utility to replicate data. but we can do it in a followup and focus on the metastore changes on this jira.
          Hide
          Carl Steinbach added a comment -

          A table T1's primary cluster is C1 meaning :1) C1 contains all data that is available in all other clusters.

          Does this mean that if T1's primary cluster is C1, then all of the partitions in T1 must also have have their primary partition set to C1? If that's the case then primary cluster should probably be a table level property, and the list of replica clusters can be a table/partition level property.

          2) write is only allowed in this cluster for table C1. but need to allow exceptions here

          What are the exceptions?

          4) all data changes to T1 happened in the primary cluster should be replicated to other clusters if there are any secondary clusters. but there should be a conf to disable it as there are some exception situations.

          What are the exceptions?

          How will dynamic partitions work? Where will new partitions get the list of replica clusters from? Will they inherit it from the table definition?

          Hive now supports insert-append into a partition (HIVE-306). Suppose that the metadata for a particular partition indicates that it is replicated to clusters C2 and C3. If I insert new data into the partition in the primary cluster C1, then the metadata is now invalid. How is this going to be handled?

          2) add new interfaces which do exactly the same set of functionalities as old ones but using a different name (use _on_cluster suffix maybe?) and have a cluster parameter

          This is going to introduce new codepaths that need to be tested separately, and also double the amount of work people need to do every time a new metastore API call is created. I don't think this is a good approach.

          3) overwrite database name for the purpose of cluster name. And allow a table co-exist in multiple databases. But that require to promote table to top level citizen, and degrade database. For example, "show tables" used to scan all tables in current db, but now need to scan all tables in all databases.

          I don't think this is an option since it breaks backwards compatibility and effectively changes the whole notion of what a db/schema is. A lot of people in the community already depend on this feature.

          1) add a cluster parameter to existing thrift interfaces

          This sounds like the best option to me. I think Thrift supports API evolution via default values for missing parameters, but setting a default value in this case may be a little tricky.

          Also, instead of modifying the Thrift interface, is it possible that you could instead leverage the work that's being done in HIVE-2720?

          Show
          Carl Steinbach added a comment - A table T1's primary cluster is C1 meaning :1) C1 contains all data that is available in all other clusters. Does this mean that if T1's primary cluster is C1, then all of the partitions in T1 must also have have their primary partition set to C1? If that's the case then primary cluster should probably be a table level property, and the list of replica clusters can be a table/partition level property. 2) write is only allowed in this cluster for table C1. but need to allow exceptions here What are the exceptions? 4) all data changes to T1 happened in the primary cluster should be replicated to other clusters if there are any secondary clusters. but there should be a conf to disable it as there are some exception situations. What are the exceptions? How will dynamic partitions work? Where will new partitions get the list of replica clusters from? Will they inherit it from the table definition? Hive now supports insert-append into a partition ( HIVE-306 ). Suppose that the metadata for a particular partition indicates that it is replicated to clusters C2 and C3. If I insert new data into the partition in the primary cluster C1, then the metadata is now invalid. How is this going to be handled? 2) add new interfaces which do exactly the same set of functionalities as old ones but using a different name (use _on_cluster suffix maybe?) and have a cluster parameter This is going to introduce new codepaths that need to be tested separately, and also double the amount of work people need to do every time a new metastore API call is created. I don't think this is a good approach. 3) overwrite database name for the purpose of cluster name. And allow a table co-exist in multiple databases. But that require to promote table to top level citizen, and degrade database. For example, "show tables" used to scan all tables in current db, but now need to scan all tables in all databases. I don't think this is an option since it breaks backwards compatibility and effectively changes the whole notion of what a db/schema is. A lot of people in the community already depend on this feature. 1) add a cluster parameter to existing thrift interfaces This sounds like the best option to me. I think Thrift supports API evolution via default values for missing parameters, but setting a default value in this case may be a little tricky. Also, instead of modifying the Thrift interface, is it possible that you could instead leverage the work that's being done in HIVE-2720 ?
          Hide
          Namit Jain added a comment -

          A table T1's primary cluster is C1 meaning :1) C1 contains all data that is available in all other clusters. Does this mean that if T1's primary cluster is C1, then all of the partitions in T1 must also have have their primary partition set to C1? If that's the case then primary cluster should probably be a table level property, and the list of replica clusters can be a table/partition level property.

          I agree

          Show
          Namit Jain added a comment - A table T1's primary cluster is C1 meaning :1) C1 contains all data that is available in all other clusters. Does this mean that if T1's primary cluster is C1, then all of the partitions in T1 must also have have their primary partition set to C1? If that's the case then primary cluster should probably be a table level property, and the list of replica clusters can be a table/partition level property. I agree
          Hide
          Namit Jain added a comment - - edited

          write is only allowed in this cluster for table C1. but need to allow exceptions here. What are the exceptions ?

          Currently, there should be no exceptions. Eventually, if we provide something in hive to do a cross-cluster write, that should be like an exception. There may be a hive command like, Replicate T@P from cluster1 to c1uster2.

          all data changes to T1 happened in the primary cluster should be replicated to other clusters if there are any secondary clusters. but there should be a conf to disable it as there are some exception situations.

          This question should not be relevant now. A much simpler to visualize this is: for every table, there is a primary cluster, and a list of secondary clusters. All the partitions belong to the primary cluster, and may belong to one or more secondary clusters. Every hive session has a current cluster, and the read happens from the current cluster. An error is thrown if the partition is missing from the current cluster, but is present in the primary cluster. I will write a new wiki, and attach it - it might be simpler to understand that way.

          Dynamic partitions should not require anything different.

          overwrite database name for the purpose of cluster name. And allow a table co-exist in multiple databases. But that require to promote table to top level citizen, and degrade database. For example, "show tables" used to scan all tables in current db, but now need to scan all tables in all databases. I don't think this is an option since it breaks backwards compatibility and effectively changes the whole notion of what a db/schema is. A lot of people in the community already depend on this feature.

          Agreed.

          add a cluster parameter to existing thrift interfaces. This sounds like the best option to me. I think Thrift supports API evolution via default values for missing parameters, but setting a default value in this case may be a little tricky.

          Agreed

          Also, instead of modifying the Thrift interface, is it possible that you could instead leverage the work that's being done in HIVE-2720?

          Will look into it

          Show
          Namit Jain added a comment - - edited write is only allowed in this cluster for table C1. but need to allow exceptions here. What are the exceptions ? Currently, there should be no exceptions. Eventually, if we provide something in hive to do a cross-cluster write, that should be like an exception. There may be a hive command like, Replicate T@P from cluster1 to c1uster2. all data changes to T1 happened in the primary cluster should be replicated to other clusters if there are any secondary clusters. but there should be a conf to disable it as there are some exception situations. This question should not be relevant now. A much simpler to visualize this is: for every table, there is a primary cluster, and a list of secondary clusters. All the partitions belong to the primary cluster, and may belong to one or more secondary clusters. Every hive session has a current cluster, and the read happens from the current cluster. An error is thrown if the partition is missing from the current cluster, but is present in the primary cluster. I will write a new wiki, and attach it - it might be simpler to understand that way. Dynamic partitions should not require anything different. overwrite database name for the purpose of cluster name. And allow a table co-exist in multiple databases. But that require to promote table to top level citizen, and degrade database. For example, "show tables" used to scan all tables in current db, but now need to scan all tables in all databases. I don't think this is an option since it breaks backwards compatibility and effectively changes the whole notion of what a db/schema is. A lot of people in the community already depend on this feature. Agreed. add a cluster parameter to existing thrift interfaces. This sounds like the best option to me. I think Thrift supports API evolution via default values for missing parameters, but setting a default value in this case may be a little tricky. Agreed Also, instead of modifying the Thrift interface, is it possible that you could instead leverage the work that's being done in HIVE-2720 ? Will look into it
          Hide
          He Yongqiang added a comment -

          a preliminary patch. Namit will take over the development.

          Things left:
          1) finish drop table command
          2) alter table add partition
          3) drop partition
          4) alter table add cluster
          5) insert overwrite...
          6) set table primary cluster

          Show
          He Yongqiang added a comment - a preliminary patch. Namit will take over the development. Things left: 1) finish drop table command 2) alter table add partition 3) drop partition 4) alter table add cluster 5) insert overwrite... 6) set table primary cluster
          Hide
          Namit Jain added a comment -

          https://cwiki.apache.org/confluence/display/Hive/Hive+across+Multiple+Data+Centers+(Physical+Clusters)

          Added a new document which explains some of the thinking and the design.
          Please comment

          Show
          Namit Jain added a comment - https://cwiki.apache.org/confluence/display/Hive/Hive+across+Multiple+Data+Centers+(Physical+Clusters ) Added a new document which explains some of the thinking and the design. Please comment
          Hide
          Phabricator added a comment -

          njain requested code review of "HIVE-2612 [jira] support hive table/partitions coexistes in more than one clusters".
          Reviewers: JIRA

          https://issues.apache.org/jira/browse/HIVE-2612

          HIVE-2612 Initial version - not for review

          1) add cluster object into hive metastore
          2) each partition/table has a creation cluster and a list of living clusters, and also data location in each cluster

          TEST PLAN
          EMPTY

          REVISION DETAIL
          https://reviews.facebook.net/D1569

          AFFECTED FILES
          metastore/src/model/package.jdo
          metastore/src/model/org/apache/hadoop/hive/metastore/model/MStorageDescriptor.java
          metastore/src/model/org/apache/hadoop/hive/metastore/model/MClusterStorageDescriptor.java
          metastore/src/model/org/apache/hadoop/hive/metastore/model/MCluster.java
          metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java
          metastore/src/java/org/apache/hadoop/hive/metastore/IMetaStoreClient.java
          metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java
          metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
          metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java
          metastore/if/hive_metastore.thrift
          common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
          ql/src/java/org/apache/hadoop/hive/ql/metadata/Partition.java
          ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
          ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java
          ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java
          ql/src/java/org/apache/hadoop/hive/ql/plan/DDLWork.java
          ql/src/java/org/apache/hadoop/hive/ql/plan/ClusterDDLDesc.java
          ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g
          ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java

          MANAGE HERALD DIFFERENTIAL RULES
          https://reviews.facebook.net/herald/view/differential/

          WHY DID I GET THIS EMAIL?
          https://reviews.facebook.net/herald/transcript/3291/

          Tip: use the X-Herald-Rules header to filter Herald messages in your client.

          Show
          Phabricator added a comment - njain requested code review of " HIVE-2612 [jira] support hive table/partitions coexistes in more than one clusters". Reviewers: JIRA https://issues.apache.org/jira/browse/HIVE-2612 HIVE-2612 Initial version - not for review 1) add cluster object into hive metastore 2) each partition/table has a creation cluster and a list of living clusters, and also data location in each cluster TEST PLAN EMPTY REVISION DETAIL https://reviews.facebook.net/D1569 AFFECTED FILES metastore/src/model/package.jdo metastore/src/model/org/apache/hadoop/hive/metastore/model/MStorageDescriptor.java metastore/src/model/org/apache/hadoop/hive/metastore/model/MClusterStorageDescriptor.java metastore/src/model/org/apache/hadoop/hive/metastore/model/MCluster.java metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java metastore/src/java/org/apache/hadoop/hive/metastore/IMetaStoreClient.java metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java metastore/if/hive_metastore.thrift common/src/java/org/apache/hadoop/hive/conf/HiveConf.java ql/src/java/org/apache/hadoop/hive/ql/metadata/Partition.java ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java ql/src/java/org/apache/hadoop/hive/ql/plan/DDLWork.java ql/src/java/org/apache/hadoop/hive/ql/plan/ClusterDDLDesc.java ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java MANAGE HERALD DIFFERENTIAL RULES https://reviews.facebook.net/herald/view/differential/ WHY DID I GET THIS EMAIL? https://reviews.facebook.net/herald/transcript/3291/ Tip: use the X-Herald-Rules header to filter Herald messages in your client.
          Show
          Namit Jain added a comment - https://cwiki.apache.org/confluence/display/Hive/Hive+across+Multiple+Data+Centers+%28Physical+Clusters%29 is the correct link to the wiki
          Hide
          Phabricator added a comment -

          njain updated the revision "HIVE-2612 [jira] support hive table/partitions coexistes in more than one clusters".
          Reviewers: JIRA

          REVISION DETAIL
          https://reviews.facebook.net/D1569

          AFFECTED FILES
          metastore/src/model/package.jdo
          metastore/src/model/org/apache/hadoop/hive/metastore/model/MStorageDescriptor.java
          metastore/src/model/org/apache/hadoop/hive/metastore/model/MClusterStorageDescriptor.java

          Show
          Phabricator added a comment - njain updated the revision " HIVE-2612 [jira] support hive table/partitions coexistes in more than one clusters". Reviewers: JIRA REVISION DETAIL https://reviews.facebook.net/D1569 AFFECTED FILES metastore/src/model/package.jdo metastore/src/model/org/apache/hadoop/hive/metastore/model/MStorageDescriptor.java metastore/src/model/org/apache/hadoop/hive/metastore/model/MClusterStorageDescriptor.java
          Hide
          Phabricator added a comment -

          njain updated the revision "HIVE-2612 [jira] support hive table/partitions coexistes in more than one clusters".
          Reviewers: JIRA

          REVISION DETAIL
          https://reviews.facebook.net/D1569

          AFFECTED FILES
          metastore/src/model/package.jdo
          metastore/src/model/org/apache/hadoop/hive/metastore/model/MStorageDescriptor.java
          metastore/src/model/org/apache/hadoop/hive/metastore/model/MClusterStorageDescriptor.java

          Show
          Phabricator added a comment - njain updated the revision " HIVE-2612 [jira] support hive table/partitions coexistes in more than one clusters". Reviewers: JIRA REVISION DETAIL https://reviews.facebook.net/D1569 AFFECTED FILES metastore/src/model/package.jdo metastore/src/model/org/apache/hadoop/hive/metastore/model/MStorageDescriptor.java metastore/src/model/org/apache/hadoop/hive/metastore/model/MClusterStorageDescriptor.java
          Hide
          Phabricator added a comment -

          njain updated the revision "HIVE-2612 [jira] support hive table/partitions coexistes in more than one clusters".
          Reviewers: JIRA

          REVISION DETAIL
          https://reviews.facebook.net/D1569

          AFFECTED FILES
          conf/hive-default.xml.template
          metastore/src/model/package.jdo
          metastore/src/model/org/apache/hadoop/hive/metastore/model/MStorageDescriptor.java
          metastore/src/model/org/apache/hadoop/hive/metastore/model/MClusterStorageDescriptor.java
          metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java
          metastore/src/gen/thrift/gen-py/hive_metastore/ttypes.py
          metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore.py
          metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp
          metastore/src/gen/thrift/gen-cpp/hive_metastore_types.cpp
          metastore/src/gen/thrift/gen-cpp/hive_metastore_types.h
          metastore/src/gen/thrift/gen-rb/hive_metastore_types.rb
          metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ThriftHiveMetastore.java
          metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Schema.java
          metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ClusterStorageDescriptor.java
          metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Partition.java
          metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Table.java
          metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/StorageDescriptor.java
          metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Index.java
          metastore/src/gen/thrift/gen-php/hive_metastore_constants.php
          metastore/src/gen/thrift/gen-php/ThriftHiveMetastore.php
          metastore/src/gen/thrift/gen-php/hive_metastore_types.php
          metastore/src/gen/thrift/gen-php/hive_metastore/ThriftHiveMetastore.php
          metastore/src/gen/thrift/gen-php/hive_metastore/hive_metastore_types.php
          metastore/if/hive_metastore.thrift
          common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
          ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
          ql/src/java/org/apache/hadoop/hive/ql/session/SessionState.java
          ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java
          ql/src/java/org/apache/hadoop/hive/ql/plan/CreateTableDesc.java
          ql/src/java/org/apache/hadoop/hive/ql/parse/ErrorMsg.java
          ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java

          Show
          Phabricator added a comment - njain updated the revision " HIVE-2612 [jira] support hive table/partitions coexistes in more than one clusters". Reviewers: JIRA REVISION DETAIL https://reviews.facebook.net/D1569 AFFECTED FILES conf/hive-default.xml.template metastore/src/model/package.jdo metastore/src/model/org/apache/hadoop/hive/metastore/model/MStorageDescriptor.java metastore/src/model/org/apache/hadoop/hive/metastore/model/MClusterStorageDescriptor.java metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java metastore/src/gen/thrift/gen-py/hive_metastore/ttypes.py metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore.py metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp metastore/src/gen/thrift/gen-cpp/hive_metastore_types.cpp metastore/src/gen/thrift/gen-cpp/hive_metastore_types.h metastore/src/gen/thrift/gen-rb/hive_metastore_types.rb metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ThriftHiveMetastore.java metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Schema.java metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ClusterStorageDescriptor.java metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Partition.java metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Table.java metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/StorageDescriptor.java metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Index.java metastore/src/gen/thrift/gen-php/hive_metastore_constants.php metastore/src/gen/thrift/gen-php/ThriftHiveMetastore.php metastore/src/gen/thrift/gen-php/hive_metastore_types.php metastore/src/gen/thrift/gen-php/hive_metastore/ThriftHiveMetastore.php metastore/src/gen/thrift/gen-php/hive_metastore/hive_metastore_types.php metastore/if/hive_metastore.thrift common/src/java/org/apache/hadoop/hive/conf/HiveConf.java ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java ql/src/java/org/apache/hadoop/hive/ql/session/SessionState.java ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java ql/src/java/org/apache/hadoop/hive/ql/plan/CreateTableDesc.java ql/src/java/org/apache/hadoop/hive/ql/parse/ErrorMsg.java ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java
          Hide
          Phabricator added a comment -

          njain updated the revision "HIVE-2612 [jira] support hive table/partitions coexistes in more than one clusters".
          Reviewers: JIRA

          REVISION DETAIL
          https://reviews.facebook.net/D1569

          AFFECTED FILES
          conf/hive-default.xml.template
          metastore/src/model/package.jdo
          metastore/src/model/org/apache/hadoop/hive/metastore/model/MStorageDescriptor.java
          metastore/src/model/org/apache/hadoop/hive/metastore/model/MClusterStorageDescriptor.java
          metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java
          metastore/src/gen/thrift/gen-py/hive_metastore/ttypes.py
          metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore.py
          metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp
          metastore/src/gen/thrift/gen-cpp/hive_metastore_types.cpp
          metastore/src/gen/thrift/gen-cpp/hive_metastore_types.h
          metastore/src/gen/thrift/gen-rb/hive_metastore_types.rb
          metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ThriftHiveMetastore.java
          metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Schema.java
          metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ClusterStorageDescriptor.java
          metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Partition.java
          metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Table.java
          metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/StorageDescriptor.java
          metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Index.java
          metastore/src/gen/thrift/gen-php/hive_metastore_constants.php
          metastore/src/gen/thrift/gen-php/ThriftHiveMetastore.php
          metastore/src/gen/thrift/gen-php/hive_metastore_types.php
          metastore/src/gen/thrift/gen-php/hive_metastore/ThriftHiveMetastore.php
          metastore/src/gen/thrift/gen-php/hive_metastore/hive_metastore_types.php
          metastore/if/hive_metastore.thrift
          common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
          ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
          ql/src/java/org/apache/hadoop/hive/ql/session/SessionState.java
          ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java
          ql/src/java/org/apache/hadoop/hive/ql/plan/CreateTableDesc.java
          ql/src/java/org/apache/hadoop/hive/ql/parse/ErrorMsg.java
          ql/src/java/org/apache/hadoop/hive/ql/parse/ImportSemanticAnalyzer.java
          ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java

          Show
          Phabricator added a comment - njain updated the revision " HIVE-2612 [jira] support hive table/partitions coexistes in more than one clusters". Reviewers: JIRA REVISION DETAIL https://reviews.facebook.net/D1569 AFFECTED FILES conf/hive-default.xml.template metastore/src/model/package.jdo metastore/src/model/org/apache/hadoop/hive/metastore/model/MStorageDescriptor.java metastore/src/model/org/apache/hadoop/hive/metastore/model/MClusterStorageDescriptor.java metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java metastore/src/gen/thrift/gen-py/hive_metastore/ttypes.py metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore.py metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp metastore/src/gen/thrift/gen-cpp/hive_metastore_types.cpp metastore/src/gen/thrift/gen-cpp/hive_metastore_types.h metastore/src/gen/thrift/gen-rb/hive_metastore_types.rb metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ThriftHiveMetastore.java metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Schema.java metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ClusterStorageDescriptor.java metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Partition.java metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Table.java metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/StorageDescriptor.java metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Index.java metastore/src/gen/thrift/gen-php/hive_metastore_constants.php metastore/src/gen/thrift/gen-php/ThriftHiveMetastore.php metastore/src/gen/thrift/gen-php/hive_metastore_types.php metastore/src/gen/thrift/gen-php/hive_metastore/ThriftHiveMetastore.php metastore/src/gen/thrift/gen-php/hive_metastore/hive_metastore_types.php metastore/if/hive_metastore.thrift common/src/java/org/apache/hadoop/hive/conf/HiveConf.java ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java ql/src/java/org/apache/hadoop/hive/ql/session/SessionState.java ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java ql/src/java/org/apache/hadoop/hive/ql/plan/CreateTableDesc.java ql/src/java/org/apache/hadoop/hive/ql/parse/ErrorMsg.java ql/src/java/org/apache/hadoop/hive/ql/parse/ImportSemanticAnalyzer.java ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java
          Hide
          Kevin Wilfong added a comment -

          I attached a patch to this JIRA which provides scripts to update the metastore for derby, MySQL and postgres.

          It also changes the default cluster name to '' (empty string) and fixes an inconsistency where the size of PRIMARY_CLUSTER_NAME in SDS had a different size than the CLUSTER_NAME column in CLUSTER_SDS.

          Show
          Kevin Wilfong added a comment - I attached a patch to this JIRA which provides scripts to update the metastore for derby, MySQL and postgres. It also changes the default cluster name to '' (empty string) and fixes an inconsistency where the size of PRIMARY_CLUSTER_NAME in SDS had a different size than the CLUSTER_NAME column in CLUSTER_SDS.
          Hide
          Namit Jain added a comment -

          Can everyone concerned please take a look ?

          For anyone not using clusters, they need to run the scripts provided in this patch to upgrade the metastore.
          The time taken for the upgrade depends on the size of the metastore (number of tables/partitions), but it should
          be fairly small - it is less than 10 minutes for facebook cluster.

          Show
          Namit Jain added a comment - Can everyone concerned please take a look ? For anyone not using clusters, they need to run the scripts provided in this patch to upgrade the metastore. The time taken for the upgrade depends on the size of the metastore (number of tables/partitions), but it should be fairly small - it is less than 10 minutes for facebook cluster.
          Hide
          Namit Jain added a comment -

          All the existing APIs will continue to work.

          Show
          Namit Jain added a comment - All the existing APIs will continue to work.
          Hide
          Phabricator added a comment -

          njain updated the revision "HIVE-2612 [jira] support hive table/partitions coexistes in more than one clusters".
          Reviewers: JIRA

          REVISION DETAIL
          https://reviews.facebook.net/D1569

          AFFECTED FILES
          conf/hive-default.xml.template
          metastore/scripts/upgrade/derby/010-HIVE-2612.derby.sql
          metastore/scripts/upgrade/derby/upgrade-0.9.0-to-0.10.0.derby.sql
          metastore/scripts/upgrade/derby/hive-schema-0.10.0.derby.sql
          metastore/scripts/upgrade/mysql/010-HIVE-2612.mysql.sql
          metastore/scripts/upgrade/mysql/upgrade-0.9.0-to-0.10.0.mysql.sql
          metastore/scripts/upgrade/mysql/hive-schema-0.10.0.mysql.sql
          metastore/scripts/upgrade/postgres/010-HIVE-2612.postgres.sql
          metastore/src/model/package.jdo
          metastore/src/model/org/apache/hadoop/hive/metastore/model/MStorageDescriptor.java
          metastore/src/model/org/apache/hadoop/hive/metastore/model/MClusterStorageDescriptor.java
          metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java
          metastore/src/gen/thrift/gen-py/hive_metastore/ttypes.py
          metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore.py
          metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp
          metastore/src/gen/thrift/gen-cpp/hive_metastore_types.cpp
          metastore/src/gen/thrift/gen-cpp/hive_metastore_types.h
          metastore/src/gen/thrift/gen-rb/hive_metastore_types.rb
          metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ThriftHiveMetastore.java
          metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Schema.java
          metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ClusterStorageDescriptor.java
          metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Partition.java
          metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Table.java
          metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/StorageDescriptor.java
          metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Index.java
          metastore/src/gen/thrift/gen-php/hive_metastore_constants.php
          metastore/src/gen/thrift/gen-php/ThriftHiveMetastore.php
          metastore/src/gen/thrift/gen-php/hive_metastore_types.php
          metastore/src/gen/thrift/gen-php/hive_metastore/ThriftHiveMetastore.php
          metastore/src/gen/thrift/gen-php/hive_metastore/hive_metastore_types.php
          metastore/if/hive_metastore.thrift
          common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
          ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
          ql/src/java/org/apache/hadoop/hive/ql/session/SessionState.java
          ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java
          ql/src/java/org/apache/hadoop/hive/ql/plan/CreateTableDesc.java
          ql/src/java/org/apache/hadoop/hive/ql/parse/ErrorMsg.java
          ql/src/java/org/apache/hadoop/hive/ql/parse/ImportSemanticAnalyzer.java
          ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java

          Show
          Phabricator added a comment - njain updated the revision " HIVE-2612 [jira] support hive table/partitions coexistes in more than one clusters". Reviewers: JIRA REVISION DETAIL https://reviews.facebook.net/D1569 AFFECTED FILES conf/hive-default.xml.template metastore/scripts/upgrade/derby/010- HIVE-2612 .derby.sql metastore/scripts/upgrade/derby/upgrade-0.9.0-to-0.10.0.derby.sql metastore/scripts/upgrade/derby/hive-schema-0.10.0.derby.sql metastore/scripts/upgrade/mysql/010- HIVE-2612 .mysql.sql metastore/scripts/upgrade/mysql/upgrade-0.9.0-to-0.10.0.mysql.sql metastore/scripts/upgrade/mysql/hive-schema-0.10.0.mysql.sql metastore/scripts/upgrade/postgres/010- HIVE-2612 .postgres.sql metastore/src/model/package.jdo metastore/src/model/org/apache/hadoop/hive/metastore/model/MStorageDescriptor.java metastore/src/model/org/apache/hadoop/hive/metastore/model/MClusterStorageDescriptor.java metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java metastore/src/gen/thrift/gen-py/hive_metastore/ttypes.py metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore.py metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp metastore/src/gen/thrift/gen-cpp/hive_metastore_types.cpp metastore/src/gen/thrift/gen-cpp/hive_metastore_types.h metastore/src/gen/thrift/gen-rb/hive_metastore_types.rb metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ThriftHiveMetastore.java metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Schema.java metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ClusterStorageDescriptor.java metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Partition.java metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Table.java metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/StorageDescriptor.java metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Index.java metastore/src/gen/thrift/gen-php/hive_metastore_constants.php metastore/src/gen/thrift/gen-php/ThriftHiveMetastore.php metastore/src/gen/thrift/gen-php/hive_metastore_types.php metastore/src/gen/thrift/gen-php/hive_metastore/ThriftHiveMetastore.php metastore/src/gen/thrift/gen-php/hive_metastore/hive_metastore_types.php metastore/if/hive_metastore.thrift common/src/java/org/apache/hadoop/hive/conf/HiveConf.java ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java ql/src/java/org/apache/hadoop/hive/ql/session/SessionState.java ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java ql/src/java/org/apache/hadoop/hive/ql/plan/CreateTableDesc.java ql/src/java/org/apache/hadoop/hive/ql/parse/ErrorMsg.java ql/src/java/org/apache/hadoop/hive/ql/parse/ImportSemanticAnalyzer.java ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java
          Hide
          Kevin Wilfong added a comment -

          This change will require the user to update their metastore schema. The scripts in the patch should be sufficient provided the schema is already up to date.

          The only schema changes needed are a new table and a new column is added to SDS, it should not take long, no more than five minutes depending on the size of the SDS table, to update.

          Show
          Kevin Wilfong added a comment - This change will require the user to update their metastore schema. The scripts in the patch should be sufficient provided the schema is already up to date. The only schema changes needed are a new table and a new column is added to SDS, it should not take long, no more than five minutes depending on the size of the SDS table, to update.
          Hide
          Phabricator added a comment -

          njain updated the revision "HIVE-2612 [jira] support hive table/partitions coexistes in more than one clusters".
          Reviewers: JIRA

          REVISION DETAIL
          https://reviews.facebook.net/D1569

          AFFECTED FILES
          conf/hive-default.xml.template
          metastore/scripts/upgrade/derby/010-HIVE-2612.derby.sql
          metastore/scripts/upgrade/derby/upgrade-0.9.0-to-0.10.0.derby.sql
          metastore/scripts/upgrade/derby/hive-schema-0.10.0.derby.sql
          metastore/scripts/upgrade/mysql/010-HIVE-2612.mysql.sql
          metastore/scripts/upgrade/mysql/upgrade-0.9.0-to-0.10.0.mysql.sql
          metastore/scripts/upgrade/mysql/hive-schema-0.10.0.mysql.sql
          metastore/scripts/upgrade/postgres/010-HIVE-2612.postgres.sql
          metastore/src/model/package.jdo
          metastore/src/model/org/apache/hadoop/hive/metastore/model/MStorageDescriptor.java
          metastore/src/model/org/apache/hadoop/hive/metastore/model/MClusterStorageDescriptor.java
          metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java
          metastore/src/gen/thrift/gen-py/hive_metastore/ttypes.py
          metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore.py
          metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp
          metastore/src/gen/thrift/gen-cpp/hive_metastore_types.cpp
          metastore/src/gen/thrift/gen-cpp/hive_metastore_types.h
          metastore/src/gen/thrift/gen-rb/hive_metastore_types.rb
          metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ThriftHiveMetastore.java
          metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Schema.java
          metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ClusterStorageDescriptor.java
          metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Partition.java
          metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Table.java
          metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/StorageDescriptor.java
          metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Index.java
          metastore/src/gen/thrift/gen-php/hive_metastore_constants.php
          metastore/src/gen/thrift/gen-php/ThriftHiveMetastore.php
          metastore/src/gen/thrift/gen-php/hive_metastore_types.php
          metastore/src/gen/thrift/gen-php/hive_metastore/ThriftHiveMetastore.php
          metastore/src/gen/thrift/gen-php/hive_metastore/hive_metastore_types.php
          metastore/if/hive_metastore.thrift
          common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
          ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
          ql/src/java/org/apache/hadoop/hive/ql/session/SessionState.java
          ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java
          ql/src/java/org/apache/hadoop/hive/ql/plan/CreateTableDesc.java
          ql/src/java/org/apache/hadoop/hive/ql/parse/ErrorMsg.java
          ql/src/java/org/apache/hadoop/hive/ql/parse/BaseSemanticAnalyzer.java
          ql/src/java/org/apache/hadoop/hive/ql/parse/ImportSemanticAnalyzer.java
          ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java

          Show
          Phabricator added a comment - njain updated the revision " HIVE-2612 [jira] support hive table/partitions coexistes in more than one clusters". Reviewers: JIRA REVISION DETAIL https://reviews.facebook.net/D1569 AFFECTED FILES conf/hive-default.xml.template metastore/scripts/upgrade/derby/010- HIVE-2612 .derby.sql metastore/scripts/upgrade/derby/upgrade-0.9.0-to-0.10.0.derby.sql metastore/scripts/upgrade/derby/hive-schema-0.10.0.derby.sql metastore/scripts/upgrade/mysql/010- HIVE-2612 .mysql.sql metastore/scripts/upgrade/mysql/upgrade-0.9.0-to-0.10.0.mysql.sql metastore/scripts/upgrade/mysql/hive-schema-0.10.0.mysql.sql metastore/scripts/upgrade/postgres/010- HIVE-2612 .postgres.sql metastore/src/model/package.jdo metastore/src/model/org/apache/hadoop/hive/metastore/model/MStorageDescriptor.java metastore/src/model/org/apache/hadoop/hive/metastore/model/MClusterStorageDescriptor.java metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java metastore/src/gen/thrift/gen-py/hive_metastore/ttypes.py metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore.py metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp metastore/src/gen/thrift/gen-cpp/hive_metastore_types.cpp metastore/src/gen/thrift/gen-cpp/hive_metastore_types.h metastore/src/gen/thrift/gen-rb/hive_metastore_types.rb metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ThriftHiveMetastore.java metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Schema.java metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ClusterStorageDescriptor.java metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Partition.java metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Table.java metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/StorageDescriptor.java metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Index.java metastore/src/gen/thrift/gen-php/hive_metastore_constants.php metastore/src/gen/thrift/gen-php/ThriftHiveMetastore.php metastore/src/gen/thrift/gen-php/hive_metastore_types.php metastore/src/gen/thrift/gen-php/hive_metastore/ThriftHiveMetastore.php metastore/src/gen/thrift/gen-php/hive_metastore/hive_metastore_types.php metastore/if/hive_metastore.thrift common/src/java/org/apache/hadoop/hive/conf/HiveConf.java ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java ql/src/java/org/apache/hadoop/hive/ql/session/SessionState.java ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java ql/src/java/org/apache/hadoop/hive/ql/plan/CreateTableDesc.java ql/src/java/org/apache/hadoop/hive/ql/parse/ErrorMsg.java ql/src/java/org/apache/hadoop/hive/ql/parse/BaseSemanticAnalyzer.java ql/src/java/org/apache/hadoop/hive/ql/parse/ImportSemanticAnalyzer.java ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java
          Hide
          Kevin Wilfong added a comment -

          I attached a patch which fixes an error seen where JDO was looking for a column which doesn't exist in the schema in the update scripts provided. The collection of MClusterStorageDescriptors was changed from a List to a Set, and a primary key was indicated in package.jdo. This fixes the error by removing the need to order the MClusterStorageDescriptors and providing a way to uniquely identify them. The primary key is already present in the upgrade scripts provided.

          Show
          Kevin Wilfong added a comment - I attached a patch which fixes an error seen where JDO was looking for a column which doesn't exist in the schema in the update scripts provided. The collection of MClusterStorageDescriptors was changed from a List to a Set, and a primary key was indicated in package.jdo. This fixes the error by removing the need to order the MClusterStorageDescriptors and providing a way to uniquely identify them. The primary key is already present in the upgrade scripts provided.
          Hide
          Kevin Wilfong added a comment -

          I attached a patch which adds checks in the metastore so that, for now, now table or partition can be created or altered to have a primary cluster name other than the default, as specified by the config variable hive.default.cluster.name If the primary cluster name for a table or partition is not set, it defaults to that value as well. I also changed the name of hive.use.default.cluster to hive.use.input.primary.cluster to better convert it's meaning.

          Show
          Kevin Wilfong added a comment - I attached a patch which adds checks in the metastore so that, for now, now table or partition can be created or altered to have a primary cluster name other than the default, as specified by the config variable hive.default.cluster.name If the primary cluster name for a table or partition is not set, it defaults to that value as well. I also changed the name of hive.use.default.cluster to hive.use.input.primary.cluster to better convert it's meaning.
          Hide
          Namit Jain added a comment -

          Since the word 'cluster' is used for different purposes - DFS cluster, MR cluster etc., changing to 'region'.
          Region and data center can be used in the same way for the purpose of this jira.

          Show
          Namit Jain added a comment - Since the word 'cluster' is used for different purposes - DFS cluster, MR cluster etc., changing to 'region'. Region and data center can be used in the same way for the purpose of this jira.
          Hide
          Namit Jain added a comment -

          Changed 'cluster' to 'region' and ran the testclidriver tests.
          Overwrote a bunch of test files.

          Kevin, can you change the upgrade files, since the schema has changed.

          Show
          Namit Jain added a comment - Changed 'cluster' to 'region' and ran the testclidriver tests. Overwrote a bunch of test files. Kevin, can you change the upgrade files, since the schema has changed.
          Hide
          Kevin Wilfong added a comment -

          Attached a patch containing the updated metastore upgrade scripts.

          Show
          Kevin Wilfong added a comment - Attached a patch containing the updated metastore upgrade scripts.
          Hide
          Kevin Wilfong added a comment -

          Updated the patch with some small changes:
          cleaned up a few remaining mentions of cluster instead of region
          always check/set the primary region name for a table in the metastore

          Show
          Kevin Wilfong added a comment - Updated the patch with some small changes: cleaned up a few remaining mentions of cluster instead of region always check/set the primary region name for a table in the metastore
          Hide
          Phabricator added a comment -

          kevinwilfong requested code review of "HIVE-2612 [jira] support hive table/partitions coexists in more than one region".
          Reviewers: JIRA, njain

          https://issues.apache.org/jira/browse/HIVE-2612

          1) add region object into hive metastore
          2) each partition/table has a primary region and a list of living regions, and also data location in each region

          This is just an initial patch, it sets up the metastore so that we can begin to support multiple regions, it simply adds a default primary region to each table/partition and adds an empty list of secondary regions.

          TEST PLAN
          Ran most of the unit tests.

          REVISION DETAIL
          https://reviews.facebook.net/D1707

          AFFECTED FILES
          conf/hive-default.xml.template
          metastore/scripts/upgrade/derby/010-HIVE-2612.derby.sql
          metastore/scripts/upgrade/derby/upgrade-0.9.0-to-0.10.0.derby.sql
          metastore/scripts/upgrade/derby/hive-schema-0.10.0.derby.sql
          metastore/scripts/upgrade/mysql/010-HIVE-2612.mysql.sql
          metastore/scripts/upgrade/mysql/upgrade-0.9.0-to-0.10.0.mysql.sql
          metastore/scripts/upgrade/mysql/hive-schema-0.10.0.mysql.sql
          metastore/scripts/upgrade/postgres/010-HIVE-2612.postgres.sql
          metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java
          metastore/src/model/package.jdo
          metastore/src/model/org/apache/hadoop/hive/metastore/model/MStorageDescriptor.java
          metastore/src/model/org/apache/hadoop/hive/metastore/model/MRegionStorageDescriptor.java
          metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
          metastore/src/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java
          metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java
          metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java
          metastore/src/gen/thrift/gen-py/hive_metastore/ttypes.py
          metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore.py
          metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp
          metastore/src/gen/thrift/gen-cpp/hive_metastore_types.cpp
          metastore/src/gen/thrift/gen-cpp/hive_metastore_types.h
          metastore/src/gen/thrift/gen-rb/hive_metastore_types.rb
          metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ThriftHiveMetastore.java
          metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Schema.java
          metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/RegionStorageDescriptor.java
          metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Partition.java
          metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Table.java
          metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/StorageDescriptor.java
          metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Index.java
          metastore/src/gen/thrift/gen-php/hive_metastore_constants.php
          metastore/src/gen/thrift/gen-php/ThriftHiveMetastore.php
          metastore/src/gen/thrift/gen-php/hive_metastore_types.php
          metastore/src/gen/thrift/gen-php/hive_metastore/ThriftHiveMetastore.php
          metastore/src/gen/thrift/gen-php/hive_metastore/hive_metastore_types.php
          metastore/if/hive_metastore.thrift
          common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
          contrib/src/test/results/clientnegative/serde_regex.q.out
          contrib/src/test/results/clientpositive/serde_regex.q.out
          contrib/src/test/results/clientpositive/fileformat_base64.q.out
          ql/src/test/results/clientpositive/inputddl3.q.out
          ql/src/test/results/clientpositive/rcfile_createas1.q.out
          ql/src/test/results/clientpositive/input15.q.out
          ql/src/test/results/clientpositive/union25.q.out
          ql/src/test/results/clientpositive/smb_mapjoin9.q.out
          ql/src/test/results/clientpositive/inputddl1.q.out
          ql/src/test/results/clientpositive/symlink_text_input_format.q.out
          ql/src/test/results/clientpositive/create_union_table.q.out
          ql/src/test/results/clientpositive/ctas.q.out
          ql/src/test/results/clientpositive/merge3.q.out
          ql/src/test/results/clientpositive/fileformat_text.q.out
          ql/src/test/results/clientpositive/inputddl2.q.out
          ql/src/test/results/clientpositive/fileformat_sequencefile.q.out
          ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
          ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java
          ql/src/java/org/apache/hadoop/hive/ql/plan/CreateTableDesc.java
          ql/src/java/org/apache/hadoop/hive/ql/parse/ErrorMsg.java
          ql/src/java/org/apache/hadoop/hive/ql/parse/ImportSemanticAnalyzer.java
          ql/src/java/org/apache/hadoop/hive/ql/parse/BaseSemanticAnalyzer.java
          ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java

          MANAGE HERALD DIFFERENTIAL RULES
          https://reviews.facebook.net/herald/view/differential/

          WHY DID I GET THIS EMAIL?
          https://reviews.facebook.net/herald/transcript/3639/

          Tip: use the X-Herald-Rules header to filter Herald messages in your client.

          Show
          Phabricator added a comment - kevinwilfong requested code review of " HIVE-2612 [jira] support hive table/partitions coexists in more than one region". Reviewers: JIRA, njain https://issues.apache.org/jira/browse/HIVE-2612 1) add region object into hive metastore 2) each partition/table has a primary region and a list of living regions, and also data location in each region This is just an initial patch, it sets up the metastore so that we can begin to support multiple regions, it simply adds a default primary region to each table/partition and adds an empty list of secondary regions. TEST PLAN Ran most of the unit tests. REVISION DETAIL https://reviews.facebook.net/D1707 AFFECTED FILES conf/hive-default.xml.template metastore/scripts/upgrade/derby/010- HIVE-2612 .derby.sql metastore/scripts/upgrade/derby/upgrade-0.9.0-to-0.10.0.derby.sql metastore/scripts/upgrade/derby/hive-schema-0.10.0.derby.sql metastore/scripts/upgrade/mysql/010- HIVE-2612 .mysql.sql metastore/scripts/upgrade/mysql/upgrade-0.9.0-to-0.10.0.mysql.sql metastore/scripts/upgrade/mysql/hive-schema-0.10.0.mysql.sql metastore/scripts/upgrade/postgres/010- HIVE-2612 .postgres.sql metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java metastore/src/model/package.jdo metastore/src/model/org/apache/hadoop/hive/metastore/model/MStorageDescriptor.java metastore/src/model/org/apache/hadoop/hive/metastore/model/MRegionStorageDescriptor.java metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java metastore/src/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java metastore/src/gen/thrift/gen-py/hive_metastore/ttypes.py metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore.py metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp metastore/src/gen/thrift/gen-cpp/hive_metastore_types.cpp metastore/src/gen/thrift/gen-cpp/hive_metastore_types.h metastore/src/gen/thrift/gen-rb/hive_metastore_types.rb metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ThriftHiveMetastore.java metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Schema.java metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/RegionStorageDescriptor.java metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Partition.java metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Table.java metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/StorageDescriptor.java metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Index.java metastore/src/gen/thrift/gen-php/hive_metastore_constants.php metastore/src/gen/thrift/gen-php/ThriftHiveMetastore.php metastore/src/gen/thrift/gen-php/hive_metastore_types.php metastore/src/gen/thrift/gen-php/hive_metastore/ThriftHiveMetastore.php metastore/src/gen/thrift/gen-php/hive_metastore/hive_metastore_types.php metastore/if/hive_metastore.thrift common/src/java/org/apache/hadoop/hive/conf/HiveConf.java contrib/src/test/results/clientnegative/serde_regex.q.out contrib/src/test/results/clientpositive/serde_regex.q.out contrib/src/test/results/clientpositive/fileformat_base64.q.out ql/src/test/results/clientpositive/inputddl3.q.out ql/src/test/results/clientpositive/rcfile_createas1.q.out ql/src/test/results/clientpositive/input15.q.out ql/src/test/results/clientpositive/union25.q.out ql/src/test/results/clientpositive/smb_mapjoin9.q.out ql/src/test/results/clientpositive/inputddl1.q.out ql/src/test/results/clientpositive/symlink_text_input_format.q.out ql/src/test/results/clientpositive/create_union_table.q.out ql/src/test/results/clientpositive/ctas.q.out ql/src/test/results/clientpositive/merge3.q.out ql/src/test/results/clientpositive/fileformat_text.q.out ql/src/test/results/clientpositive/inputddl2.q.out ql/src/test/results/clientpositive/fileformat_sequencefile.q.out ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java ql/src/java/org/apache/hadoop/hive/ql/plan/CreateTableDesc.java ql/src/java/org/apache/hadoop/hive/ql/parse/ErrorMsg.java ql/src/java/org/apache/hadoop/hive/ql/parse/ImportSemanticAnalyzer.java ql/src/java/org/apache/hadoop/hive/ql/parse/BaseSemanticAnalyzer.java ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java MANAGE HERALD DIFFERENTIAL RULES https://reviews.facebook.net/herald/view/differential/ WHY DID I GET THIS EMAIL? https://reviews.facebook.net/herald/transcript/3639/ Tip: use the X-Herald-Rules header to filter Herald messages in your client.
          Hide
          Namit Jain added a comment -

          Committed. Thanks Kevin and Yongqiang

          Show
          Namit Jain added a comment - Committed. Thanks Kevin and Yongqiang
          Hide
          Hudson added a comment -

          Integrated in Hive-trunk-h0.21 #1254 (See https://builds.apache.org/job/Hive-trunk-h0.21/1254/)
          HIVE-2612 support hive table/partitions exists in more than one region
          (Kevin Wilfong via namit)

          namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1243013
          Files :

          • /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
          • /hive/trunk/conf/hive-default.xml.template
          • /hive/trunk/contrib/src/test/results/clientnegative/serde_regex.q.out
          • /hive/trunk/contrib/src/test/results/clientpositive/fileformat_base64.q.out
          • /hive/trunk/contrib/src/test/results/clientpositive/serde_regex.q.out
          • /hive/trunk/metastore/if/hive_metastore.thrift
          • /hive/trunk/metastore/scripts/upgrade/derby/010-HIVE-2612.derby.sql
          • /hive/trunk/metastore/scripts/upgrade/derby/hive-schema-0.10.0.derby.sql
          • /hive/trunk/metastore/scripts/upgrade/derby/upgrade-0.9.0-to-0.10.0.derby.sql
          • /hive/trunk/metastore/scripts/upgrade/mysql/010-HIVE-2612.mysql.sql
          • /hive/trunk/metastore/scripts/upgrade/mysql/hive-schema-0.10.0.mysql.sql
          • /hive/trunk/metastore/scripts/upgrade/mysql/upgrade-0.9.0-to-0.10.0.mysql.sql
          • /hive/trunk/metastore/scripts/upgrade/postgres/010-HIVE-2612.postgres.sql
          • /hive/trunk/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp
          • /hive/trunk/metastore/src/gen/thrift/gen-cpp/hive_metastore_types.cpp
          • /hive/trunk/metastore/src/gen/thrift/gen-cpp/hive_metastore_types.h
          • /hive/trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Index.java
          • /hive/trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Partition.java
          • /hive/trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/RegionStorageDescriptor.java
          • /hive/trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Schema.java
          • /hive/trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/StorageDescriptor.java
          • /hive/trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Table.java
          • /hive/trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ThriftHiveMetastore.java
          • /hive/trunk/metastore/src/gen/thrift/gen-php/ThriftHiveMetastore.php
          • /hive/trunk/metastore/src/gen/thrift/gen-php/hive_metastore/ThriftHiveMetastore.php
          • /hive/trunk/metastore/src/gen/thrift/gen-php/hive_metastore/hive_metastore_types.php
          • /hive/trunk/metastore/src/gen/thrift/gen-php/hive_metastore_constants.php
          • /hive/trunk/metastore/src/gen/thrift/gen-php/hive_metastore_types.php
          • /hive/trunk/metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore.py
          • /hive/trunk/metastore/src/gen/thrift/gen-py/hive_metastore/ttypes.py
          • /hive/trunk/metastore/src/gen/thrift/gen-rb/hive_metastore_types.rb
          • /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java
          • /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
          • /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java
          • /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java
          • /hive/trunk/metastore/src/model/org/apache/hadoop/hive/metastore/model/MRegionStorageDescriptor.java
          • /hive/trunk/metastore/src/model/org/apache/hadoop/hive/metastore/model/MStorageDescriptor.java
          • /hive/trunk/metastore/src/model/package.jdo
          • /hive/trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/BaseSemanticAnalyzer.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/ErrorMsg.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/ImportSemanticAnalyzer.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/CreateTableDesc.java
          • /hive/trunk/ql/src/test/org/apache/hadoop/hive/ql/metadata/TestHive.java
          • /hive/trunk/ql/src/test/results/clientpositive/create_union_table.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/ctas.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/fileformat_sequencefile.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/fileformat_text.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/input15.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/inputddl1.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/inputddl2.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/inputddl3.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/merge3.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/rcfile_createas1.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/smb_mapjoin9.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/symlink_text_input_format.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/union25.q.out
          Show
          Hudson added a comment - Integrated in Hive-trunk-h0.21 #1254 (See https://builds.apache.org/job/Hive-trunk-h0.21/1254/ ) HIVE-2612 support hive table/partitions exists in more than one region (Kevin Wilfong via namit) namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1243013 Files : /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java /hive/trunk/conf/hive-default.xml.template /hive/trunk/contrib/src/test/results/clientnegative/serde_regex.q.out /hive/trunk/contrib/src/test/results/clientpositive/fileformat_base64.q.out /hive/trunk/contrib/src/test/results/clientpositive/serde_regex.q.out /hive/trunk/metastore/if/hive_metastore.thrift /hive/trunk/metastore/scripts/upgrade/derby/010- HIVE-2612 .derby.sql /hive/trunk/metastore/scripts/upgrade/derby/hive-schema-0.10.0.derby.sql /hive/trunk/metastore/scripts/upgrade/derby/upgrade-0.9.0-to-0.10.0.derby.sql /hive/trunk/metastore/scripts/upgrade/mysql/010- HIVE-2612 .mysql.sql /hive/trunk/metastore/scripts/upgrade/mysql/hive-schema-0.10.0.mysql.sql /hive/trunk/metastore/scripts/upgrade/mysql/upgrade-0.9.0-to-0.10.0.mysql.sql /hive/trunk/metastore/scripts/upgrade/postgres/010- HIVE-2612 .postgres.sql /hive/trunk/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp /hive/trunk/metastore/src/gen/thrift/gen-cpp/hive_metastore_types.cpp /hive/trunk/metastore/src/gen/thrift/gen-cpp/hive_metastore_types.h /hive/trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Index.java /hive/trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Partition.java /hive/trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/RegionStorageDescriptor.java /hive/trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Schema.java /hive/trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/StorageDescriptor.java /hive/trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Table.java /hive/trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ThriftHiveMetastore.java /hive/trunk/metastore/src/gen/thrift/gen-php/ThriftHiveMetastore.php /hive/trunk/metastore/src/gen/thrift/gen-php/hive_metastore/ThriftHiveMetastore.php /hive/trunk/metastore/src/gen/thrift/gen-php/hive_metastore/hive_metastore_types.php /hive/trunk/metastore/src/gen/thrift/gen-php/hive_metastore_constants.php /hive/trunk/metastore/src/gen/thrift/gen-php/hive_metastore_types.php /hive/trunk/metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore.py /hive/trunk/metastore/src/gen/thrift/gen-py/hive_metastore/ttypes.py /hive/trunk/metastore/src/gen/thrift/gen-rb/hive_metastore_types.rb /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java /hive/trunk/metastore/src/model/org/apache/hadoop/hive/metastore/model/MRegionStorageDescriptor.java /hive/trunk/metastore/src/model/org/apache/hadoop/hive/metastore/model/MStorageDescriptor.java /hive/trunk/metastore/src/model/package.jdo /hive/trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/BaseSemanticAnalyzer.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/ErrorMsg.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/ImportSemanticAnalyzer.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/CreateTableDesc.java /hive/trunk/ql/src/test/org/apache/hadoop/hive/ql/metadata/TestHive.java /hive/trunk/ql/src/test/results/clientpositive/create_union_table.q.out /hive/trunk/ql/src/test/results/clientpositive/ctas.q.out /hive/trunk/ql/src/test/results/clientpositive/fileformat_sequencefile.q.out /hive/trunk/ql/src/test/results/clientpositive/fileformat_text.q.out /hive/trunk/ql/src/test/results/clientpositive/input15.q.out /hive/trunk/ql/src/test/results/clientpositive/inputddl1.q.out /hive/trunk/ql/src/test/results/clientpositive/inputddl2.q.out /hive/trunk/ql/src/test/results/clientpositive/inputddl3.q.out /hive/trunk/ql/src/test/results/clientpositive/merge3.q.out /hive/trunk/ql/src/test/results/clientpositive/rcfile_createas1.q.out /hive/trunk/ql/src/test/results/clientpositive/smb_mapjoin9.q.out /hive/trunk/ql/src/test/results/clientpositive/symlink_text_input_format.q.out /hive/trunk/ql/src/test/results/clientpositive/union25.q.out
          Hide
          Ashutosh Chauhan added a comment -

          @Kevin,

          You named the script upgrade-0.9.0-to-0.10.0.mysql.sql, upgrade-0.9.0-to-0.10.0.derby.sql, hive-schema-0.10.0.derby.sql, hive-schema-0.10.0.mysql.sql But we have not released 0.9 yet. These should be named upgrade-0.8.0-to-0.9.0.mysql.sql, upgrade-0.8.0-to-0.9.0.derby.sql, hive-schema-0.9.0.derby.sql, hive-schema-0.9.0.mysql.sql respectively.

          Show
          Ashutosh Chauhan added a comment - @Kevin, You named the script upgrade-0.9.0-to-0.10.0.mysql.sql, upgrade-0.9.0-to-0.10.0.derby.sql, hive-schema-0.10.0.derby.sql, hive-schema-0.10.0.mysql.sql But we have not released 0.9 yet. These should be named upgrade-0.8.0-to-0.9.0.mysql.sql, upgrade-0.8.0-to-0.9.0.derby.sql, hive-schema-0.9.0.derby.sql, hive-schema-0.9.0.mysql.sql respectively.
          Hide
          Kevin Wilfong added a comment -

          @Ashutosh

          I'm sorry about that, there already were scripts with the name upgrade-0.8.0-to-0.9.0.mysql.sql, I didn't realize those were Hive version numbers, I thought this was a metastore versioning system.

          I can move the sql commands in those files into the 0.8.0-to-0.9.0 scripts and rename the schema-0.10.0 scripts to schema-0.9.0

          Show
          Kevin Wilfong added a comment - @Ashutosh I'm sorry about that, there already were scripts with the name upgrade-0.8.0-to-0.9.0.mysql.sql, I didn't realize those were Hive version numbers, I thought this was a metastore versioning system. I can move the sql commands in those files into the 0.8.0-to-0.9.0 scripts and rename the schema-0.10.0 scripts to schema-0.9.0
          Hide
          Hudson added a comment -

          Integrated in Hive-trunk-h0.21 #1386 (See https://builds.apache.org/job/Hive-trunk-h0.21/1386/)
          HIVE-2965 : Revert HIVE-2612 (hashutosh) (Revision 1328469)

          Result = SUCCESS
          hashutosh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1328469
          Files :

          • /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
          • /hive/trunk/conf/hive-default.xml.template
          • /hive/trunk/contrib/src/test/results/clientnegative/serde_regex.q.out
          • /hive/trunk/contrib/src/test/results/clientpositive/fileformat_base64.q.out
          • /hive/trunk/contrib/src/test/results/clientpositive/serde_regex.q.out
          • /hive/trunk/metastore/if/hive_metastore.thrift
          • /hive/trunk/metastore/scripts/upgrade/derby/009-HIVE-2612.derby.sql
          • /hive/trunk/metastore/scripts/upgrade/derby/hive-schema-0.9.0.derby.sql
          • /hive/trunk/metastore/scripts/upgrade/derby/upgrade-0.8.0-to-0.9.0.derby.sql
          • /hive/trunk/metastore/scripts/upgrade/mysql/009-HIVE-2612.mysql.sql
          • /hive/trunk/metastore/scripts/upgrade/mysql/hive-schema-0.9.0.mysql.sql
          • /hive/trunk/metastore/scripts/upgrade/mysql/upgrade-0.8.0-to-0.9.0.mysql.sql
          • /hive/trunk/metastore/scripts/upgrade/postgres/009-HIVE-2612.postgres.sql
          • /hive/trunk/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp
          • /hive/trunk/metastore/src/gen/thrift/gen-cpp/hive_metastore_types.cpp
          • /hive/trunk/metastore/src/gen/thrift/gen-cpp/hive_metastore_types.h
          • /hive/trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Index.java
          • /hive/trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Partition.java
          • /hive/trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/RegionStorageDescriptor.java
          • /hive/trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Schema.java
          • /hive/trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/StorageDescriptor.java
          • /hive/trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Table.java
          • /hive/trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ThriftHiveMetastore.java
          • /hive/trunk/metastore/src/gen/thrift/gen-php/ThriftHiveMetastore.php
          • /hive/trunk/metastore/src/gen/thrift/gen-php/hive_metastore/ThriftHiveMetastore.php
          • /hive/trunk/metastore/src/gen/thrift/gen-php/hive_metastore/hive_metastore_types.php
          • /hive/trunk/metastore/src/gen/thrift/gen-php/hive_metastore_constants.php
          • /hive/trunk/metastore/src/gen/thrift/gen-php/hive_metastore_types.php
          • /hive/trunk/metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore.py
          • /hive/trunk/metastore/src/gen/thrift/gen-py/hive_metastore/ttypes.py
          • /hive/trunk/metastore/src/gen/thrift/gen-rb/hive_metastore_types.rb
          • /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java
          • /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
          • /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java
          • /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java
          • /hive/trunk/metastore/src/model/org/apache/hadoop/hive/metastore/model/MRegionStorageDescriptor.java
          • /hive/trunk/metastore/src/model/org/apache/hadoop/hive/metastore/model/MStorageDescriptor.java
          • /hive/trunk/metastore/src/model/package.jdo
          • /hive/trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/BaseSemanticAnalyzer.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/ErrorMsg.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/ImportSemanticAnalyzer.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/CreateTableDesc.java
          • /hive/trunk/ql/src/test/org/apache/hadoop/hive/ql/metadata/TestHive.java
          • /hive/trunk/ql/src/test/results/clientpositive/create_union_table.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/ctas.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/fileformat_sequencefile.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/fileformat_text.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/input15.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/inputddl1.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/inputddl2.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/inputddl3.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/merge3.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/rcfile_createas1.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/smb_mapjoin9.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/symlink_text_input_format.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/union25.q.out
          Show
          Hudson added a comment - Integrated in Hive-trunk-h0.21 #1386 (See https://builds.apache.org/job/Hive-trunk-h0.21/1386/ ) HIVE-2965 : Revert HIVE-2612 (hashutosh) (Revision 1328469) Result = SUCCESS hashutosh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1328469 Files : /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java /hive/trunk/conf/hive-default.xml.template /hive/trunk/contrib/src/test/results/clientnegative/serde_regex.q.out /hive/trunk/contrib/src/test/results/clientpositive/fileformat_base64.q.out /hive/trunk/contrib/src/test/results/clientpositive/serde_regex.q.out /hive/trunk/metastore/if/hive_metastore.thrift /hive/trunk/metastore/scripts/upgrade/derby/009- HIVE-2612 .derby.sql /hive/trunk/metastore/scripts/upgrade/derby/hive-schema-0.9.0.derby.sql /hive/trunk/metastore/scripts/upgrade/derby/upgrade-0.8.0-to-0.9.0.derby.sql /hive/trunk/metastore/scripts/upgrade/mysql/009- HIVE-2612 .mysql.sql /hive/trunk/metastore/scripts/upgrade/mysql/hive-schema-0.9.0.mysql.sql /hive/trunk/metastore/scripts/upgrade/mysql/upgrade-0.8.0-to-0.9.0.mysql.sql /hive/trunk/metastore/scripts/upgrade/postgres/009- HIVE-2612 .postgres.sql /hive/trunk/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp /hive/trunk/metastore/src/gen/thrift/gen-cpp/hive_metastore_types.cpp /hive/trunk/metastore/src/gen/thrift/gen-cpp/hive_metastore_types.h /hive/trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Index.java /hive/trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Partition.java /hive/trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/RegionStorageDescriptor.java /hive/trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Schema.java /hive/trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/StorageDescriptor.java /hive/trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Table.java /hive/trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ThriftHiveMetastore.java /hive/trunk/metastore/src/gen/thrift/gen-php/ThriftHiveMetastore.php /hive/trunk/metastore/src/gen/thrift/gen-php/hive_metastore/ThriftHiveMetastore.php /hive/trunk/metastore/src/gen/thrift/gen-php/hive_metastore/hive_metastore_types.php /hive/trunk/metastore/src/gen/thrift/gen-php/hive_metastore_constants.php /hive/trunk/metastore/src/gen/thrift/gen-php/hive_metastore_types.php /hive/trunk/metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore.py /hive/trunk/metastore/src/gen/thrift/gen-py/hive_metastore/ttypes.py /hive/trunk/metastore/src/gen/thrift/gen-rb/hive_metastore_types.rb /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java /hive/trunk/metastore/src/model/org/apache/hadoop/hive/metastore/model/MRegionStorageDescriptor.java /hive/trunk/metastore/src/model/org/apache/hadoop/hive/metastore/model/MStorageDescriptor.java /hive/trunk/metastore/src/model/package.jdo /hive/trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/BaseSemanticAnalyzer.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/ErrorMsg.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/ImportSemanticAnalyzer.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/CreateTableDesc.java /hive/trunk/ql/src/test/org/apache/hadoop/hive/ql/metadata/TestHive.java /hive/trunk/ql/src/test/results/clientpositive/create_union_table.q.out /hive/trunk/ql/src/test/results/clientpositive/ctas.q.out /hive/trunk/ql/src/test/results/clientpositive/fileformat_sequencefile.q.out /hive/trunk/ql/src/test/results/clientpositive/fileformat_text.q.out /hive/trunk/ql/src/test/results/clientpositive/input15.q.out /hive/trunk/ql/src/test/results/clientpositive/inputddl1.q.out /hive/trunk/ql/src/test/results/clientpositive/inputddl2.q.out /hive/trunk/ql/src/test/results/clientpositive/inputddl3.q.out /hive/trunk/ql/src/test/results/clientpositive/merge3.q.out /hive/trunk/ql/src/test/results/clientpositive/rcfile_createas1.q.out /hive/trunk/ql/src/test/results/clientpositive/smb_mapjoin9.q.out /hive/trunk/ql/src/test/results/clientpositive/symlink_text_input_format.q.out /hive/trunk/ql/src/test/results/clientpositive/union25.q.out
          Hide
          Ashutosh Chauhan added a comment -

          This issue is closed now. It was released with the fix in 0.9.0. If there is a problem, please open a new jira and link this one with that.

          Show
          Ashutosh Chauhan added a comment - This issue is closed now. It was released with the fix in 0.9.0. If there is a problem, please open a new jira and link this one with that.
          Hide
          Phabricator added a comment -

          njain has abandoned the revision "HIVE-2612 [jira] support hive table/partitions coexistes in more than one clusters".

          REVISION DETAIL
          https://reviews.facebook.net/D1569

          Show
          Phabricator added a comment - njain has abandoned the revision " HIVE-2612 [jira] support hive table/partitions coexistes in more than one clusters". REVISION DETAIL https://reviews.facebook.net/D1569
          Hide
          Lars Francke added a comment -

          Can anyone explain to me what this means/how it works or point me to some documentation?

          Show
          Lars Francke added a comment - Can anyone explain to me what this means/how it works or point me to some documentation?
          Hide
          Kevin Wilfong added a comment -

          I believe this was reverted prior to the Version 0.9.0

          Show
          Kevin Wilfong added a comment - I believe this was reverted prior to the Version 0.9.0
          Show
          Kevin Wilfong added a comment - https://issues.apache.org/jira/browse/HIVE-2965
          Hide
          Lars Francke added a comment -

          Ah, okay, thank you.

          Show
          Lars Francke added a comment - Ah, okay, thank you.
          Hide
          Hudson added a comment -

          Integrated in Hive-trunk-hadoop2 #54 (See https://builds.apache.org/job/Hive-trunk-hadoop2/54/)
          HIVE-2965 : Revert HIVE-2612 (hashutosh) (Revision 1328469)
          HIVE-2612 support hive table/partitions exists in more than one region
          (Kevin Wilfong via namit) (Revision 1243013)

          Result = ABORTED
          hashutosh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1328469
          Files :

          • /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
          • /hive/trunk/conf/hive-default.xml.template
          • /hive/trunk/contrib/src/test/results/clientnegative/serde_regex.q.out
          • /hive/trunk/contrib/src/test/results/clientpositive/fileformat_base64.q.out
          • /hive/trunk/contrib/src/test/results/clientpositive/serde_regex.q.out
          • /hive/trunk/metastore/if/hive_metastore.thrift
          • /hive/trunk/metastore/scripts/upgrade/derby/009-HIVE-2612.derby.sql
          • /hive/trunk/metastore/scripts/upgrade/derby/hive-schema-0.9.0.derby.sql
          • /hive/trunk/metastore/scripts/upgrade/derby/upgrade-0.8.0-to-0.9.0.derby.sql
          • /hive/trunk/metastore/scripts/upgrade/mysql/009-HIVE-2612.mysql.sql
          • /hive/trunk/metastore/scripts/upgrade/mysql/hive-schema-0.9.0.mysql.sql
          • /hive/trunk/metastore/scripts/upgrade/mysql/upgrade-0.8.0-to-0.9.0.mysql.sql
          • /hive/trunk/metastore/scripts/upgrade/postgres/009-HIVE-2612.postgres.sql
          • /hive/trunk/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp
          • /hive/trunk/metastore/src/gen/thrift/gen-cpp/hive_metastore_types.cpp
          • /hive/trunk/metastore/src/gen/thrift/gen-cpp/hive_metastore_types.h
          • /hive/trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Index.java
          • /hive/trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Partition.java
          • /hive/trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/RegionStorageDescriptor.java
          • /hive/trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Schema.java
          • /hive/trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/StorageDescriptor.java
          • /hive/trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Table.java
          • /hive/trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ThriftHiveMetastore.java
          • /hive/trunk/metastore/src/gen/thrift/gen-php/ThriftHiveMetastore.php
          • /hive/trunk/metastore/src/gen/thrift/gen-php/hive_metastore/ThriftHiveMetastore.php
          • /hive/trunk/metastore/src/gen/thrift/gen-php/hive_metastore/hive_metastore_types.php
          • /hive/trunk/metastore/src/gen/thrift/gen-php/hive_metastore_constants.php
          • /hive/trunk/metastore/src/gen/thrift/gen-php/hive_metastore_types.php
          • /hive/trunk/metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore.py
          • /hive/trunk/metastore/src/gen/thrift/gen-py/hive_metastore/ttypes.py
          • /hive/trunk/metastore/src/gen/thrift/gen-rb/hive_metastore_types.rb
          • /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java
          • /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
          • /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java
          • /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java
          • /hive/trunk/metastore/src/model/org/apache/hadoop/hive/metastore/model/MRegionStorageDescriptor.java
          • /hive/trunk/metastore/src/model/org/apache/hadoop/hive/metastore/model/MStorageDescriptor.java
          • /hive/trunk/metastore/src/model/package.jdo
          • /hive/trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/BaseSemanticAnalyzer.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/ErrorMsg.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/ImportSemanticAnalyzer.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/CreateTableDesc.java
          • /hive/trunk/ql/src/test/org/apache/hadoop/hive/ql/metadata/TestHive.java
          • /hive/trunk/ql/src/test/results/clientpositive/create_union_table.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/ctas.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/fileformat_sequencefile.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/fileformat_text.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/input15.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/inputddl1.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/inputddl2.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/inputddl3.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/merge3.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/rcfile_createas1.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/smb_mapjoin9.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/symlink_text_input_format.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/union25.q.out

          namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1243013
          Files :

          • /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
          • /hive/trunk/conf/hive-default.xml.template
          • /hive/trunk/contrib/src/test/results/clientnegative/serde_regex.q.out
          • /hive/trunk/contrib/src/test/results/clientpositive/fileformat_base64.q.out
          • /hive/trunk/contrib/src/test/results/clientpositive/serde_regex.q.out
          • /hive/trunk/metastore/if/hive_metastore.thrift
          • /hive/trunk/metastore/scripts/upgrade/derby/010-HIVE-2612.derby.sql
          • /hive/trunk/metastore/scripts/upgrade/derby/hive-schema-0.10.0.derby.sql
          • /hive/trunk/metastore/scripts/upgrade/derby/upgrade-0.9.0-to-0.10.0.derby.sql
          • /hive/trunk/metastore/scripts/upgrade/mysql/010-HIVE-2612.mysql.sql
          • /hive/trunk/metastore/scripts/upgrade/mysql/hive-schema-0.10.0.mysql.sql
          • /hive/trunk/metastore/scripts/upgrade/mysql/upgrade-0.9.0-to-0.10.0.mysql.sql
          • /hive/trunk/metastore/scripts/upgrade/postgres/010-HIVE-2612.postgres.sql
          • /hive/trunk/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp
          • /hive/trunk/metastore/src/gen/thrift/gen-cpp/hive_metastore_types.cpp
          • /hive/trunk/metastore/src/gen/thrift/gen-cpp/hive_metastore_types.h
          • /hive/trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Index.java
          • /hive/trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Partition.java
          • /hive/trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/RegionStorageDescriptor.java
          • /hive/trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Schema.java
          • /hive/trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/StorageDescriptor.java
          • /hive/trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Table.java
          • /hive/trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ThriftHiveMetastore.java
          • /hive/trunk/metastore/src/gen/thrift/gen-php/ThriftHiveMetastore.php
          • /hive/trunk/metastore/src/gen/thrift/gen-php/hive_metastore/ThriftHiveMetastore.php
          • /hive/trunk/metastore/src/gen/thrift/gen-php/hive_metastore/hive_metastore_types.php
          • /hive/trunk/metastore/src/gen/thrift/gen-php/hive_metastore_constants.php
          • /hive/trunk/metastore/src/gen/thrift/gen-php/hive_metastore_types.php
          • /hive/trunk/metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore.py
          • /hive/trunk/metastore/src/gen/thrift/gen-py/hive_metastore/ttypes.py
          • /hive/trunk/metastore/src/gen/thrift/gen-rb/hive_metastore_types.rb
          • /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java
          • /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
          • /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java
          • /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java
          • /hive/trunk/metastore/src/model/org/apache/hadoop/hive/metastore/model/MRegionStorageDescriptor.java
          • /hive/trunk/metastore/src/model/org/apache/hadoop/hive/metastore/model/MStorageDescriptor.java
          • /hive/trunk/metastore/src/model/package.jdo
          • /hive/trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/BaseSemanticAnalyzer.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/ErrorMsg.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/ImportSemanticAnalyzer.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/CreateTableDesc.java
          • /hive/trunk/ql/src/test/org/apache/hadoop/hive/ql/metadata/TestHive.java
          • /hive/trunk/ql/src/test/results/clientpositive/create_union_table.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/ctas.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/fileformat_sequencefile.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/fileformat_text.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/input15.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/inputddl1.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/inputddl2.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/inputddl3.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/merge3.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/rcfile_createas1.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/smb_mapjoin9.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/symlink_text_input_format.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/union25.q.out
          Show
          Hudson added a comment - Integrated in Hive-trunk-hadoop2 #54 (See https://builds.apache.org/job/Hive-trunk-hadoop2/54/ ) HIVE-2965 : Revert HIVE-2612 (hashutosh) (Revision 1328469) HIVE-2612 support hive table/partitions exists in more than one region (Kevin Wilfong via namit) (Revision 1243013) Result = ABORTED hashutosh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1328469 Files : /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java /hive/trunk/conf/hive-default.xml.template /hive/trunk/contrib/src/test/results/clientnegative/serde_regex.q.out /hive/trunk/contrib/src/test/results/clientpositive/fileformat_base64.q.out /hive/trunk/contrib/src/test/results/clientpositive/serde_regex.q.out /hive/trunk/metastore/if/hive_metastore.thrift /hive/trunk/metastore/scripts/upgrade/derby/009- HIVE-2612 .derby.sql /hive/trunk/metastore/scripts/upgrade/derby/hive-schema-0.9.0.derby.sql /hive/trunk/metastore/scripts/upgrade/derby/upgrade-0.8.0-to-0.9.0.derby.sql /hive/trunk/metastore/scripts/upgrade/mysql/009- HIVE-2612 .mysql.sql /hive/trunk/metastore/scripts/upgrade/mysql/hive-schema-0.9.0.mysql.sql /hive/trunk/metastore/scripts/upgrade/mysql/upgrade-0.8.0-to-0.9.0.mysql.sql /hive/trunk/metastore/scripts/upgrade/postgres/009- HIVE-2612 .postgres.sql /hive/trunk/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp /hive/trunk/metastore/src/gen/thrift/gen-cpp/hive_metastore_types.cpp /hive/trunk/metastore/src/gen/thrift/gen-cpp/hive_metastore_types.h /hive/trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Index.java /hive/trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Partition.java /hive/trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/RegionStorageDescriptor.java /hive/trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Schema.java /hive/trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/StorageDescriptor.java /hive/trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Table.java /hive/trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ThriftHiveMetastore.java /hive/trunk/metastore/src/gen/thrift/gen-php/ThriftHiveMetastore.php /hive/trunk/metastore/src/gen/thrift/gen-php/hive_metastore/ThriftHiveMetastore.php /hive/trunk/metastore/src/gen/thrift/gen-php/hive_metastore/hive_metastore_types.php /hive/trunk/metastore/src/gen/thrift/gen-php/hive_metastore_constants.php /hive/trunk/metastore/src/gen/thrift/gen-php/hive_metastore_types.php /hive/trunk/metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore.py /hive/trunk/metastore/src/gen/thrift/gen-py/hive_metastore/ttypes.py /hive/trunk/metastore/src/gen/thrift/gen-rb/hive_metastore_types.rb /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java /hive/trunk/metastore/src/model/org/apache/hadoop/hive/metastore/model/MRegionStorageDescriptor.java /hive/trunk/metastore/src/model/org/apache/hadoop/hive/metastore/model/MStorageDescriptor.java /hive/trunk/metastore/src/model/package.jdo /hive/trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/BaseSemanticAnalyzer.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/ErrorMsg.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/ImportSemanticAnalyzer.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/CreateTableDesc.java /hive/trunk/ql/src/test/org/apache/hadoop/hive/ql/metadata/TestHive.java /hive/trunk/ql/src/test/results/clientpositive/create_union_table.q.out /hive/trunk/ql/src/test/results/clientpositive/ctas.q.out /hive/trunk/ql/src/test/results/clientpositive/fileformat_sequencefile.q.out /hive/trunk/ql/src/test/results/clientpositive/fileformat_text.q.out /hive/trunk/ql/src/test/results/clientpositive/input15.q.out /hive/trunk/ql/src/test/results/clientpositive/inputddl1.q.out /hive/trunk/ql/src/test/results/clientpositive/inputddl2.q.out /hive/trunk/ql/src/test/results/clientpositive/inputddl3.q.out /hive/trunk/ql/src/test/results/clientpositive/merge3.q.out /hive/trunk/ql/src/test/results/clientpositive/rcfile_createas1.q.out /hive/trunk/ql/src/test/results/clientpositive/smb_mapjoin9.q.out /hive/trunk/ql/src/test/results/clientpositive/symlink_text_input_format.q.out /hive/trunk/ql/src/test/results/clientpositive/union25.q.out namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1243013 Files : /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java /hive/trunk/conf/hive-default.xml.template /hive/trunk/contrib/src/test/results/clientnegative/serde_regex.q.out /hive/trunk/contrib/src/test/results/clientpositive/fileformat_base64.q.out /hive/trunk/contrib/src/test/results/clientpositive/serde_regex.q.out /hive/trunk/metastore/if/hive_metastore.thrift /hive/trunk/metastore/scripts/upgrade/derby/010- HIVE-2612 .derby.sql /hive/trunk/metastore/scripts/upgrade/derby/hive-schema-0.10.0.derby.sql /hive/trunk/metastore/scripts/upgrade/derby/upgrade-0.9.0-to-0.10.0.derby.sql /hive/trunk/metastore/scripts/upgrade/mysql/010- HIVE-2612 .mysql.sql /hive/trunk/metastore/scripts/upgrade/mysql/hive-schema-0.10.0.mysql.sql /hive/trunk/metastore/scripts/upgrade/mysql/upgrade-0.9.0-to-0.10.0.mysql.sql /hive/trunk/metastore/scripts/upgrade/postgres/010- HIVE-2612 .postgres.sql /hive/trunk/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp /hive/trunk/metastore/src/gen/thrift/gen-cpp/hive_metastore_types.cpp /hive/trunk/metastore/src/gen/thrift/gen-cpp/hive_metastore_types.h /hive/trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Index.java /hive/trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Partition.java /hive/trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/RegionStorageDescriptor.java /hive/trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Schema.java /hive/trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/StorageDescriptor.java /hive/trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Table.java /hive/trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ThriftHiveMetastore.java /hive/trunk/metastore/src/gen/thrift/gen-php/ThriftHiveMetastore.php /hive/trunk/metastore/src/gen/thrift/gen-php/hive_metastore/ThriftHiveMetastore.php /hive/trunk/metastore/src/gen/thrift/gen-php/hive_metastore/hive_metastore_types.php /hive/trunk/metastore/src/gen/thrift/gen-php/hive_metastore_constants.php /hive/trunk/metastore/src/gen/thrift/gen-php/hive_metastore_types.php /hive/trunk/metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore.py /hive/trunk/metastore/src/gen/thrift/gen-py/hive_metastore/ttypes.py /hive/trunk/metastore/src/gen/thrift/gen-rb/hive_metastore_types.rb /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java /hive/trunk/metastore/src/model/org/apache/hadoop/hive/metastore/model/MRegionStorageDescriptor.java /hive/trunk/metastore/src/model/org/apache/hadoop/hive/metastore/model/MStorageDescriptor.java /hive/trunk/metastore/src/model/package.jdo /hive/trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/BaseSemanticAnalyzer.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/ErrorMsg.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/ImportSemanticAnalyzer.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/CreateTableDesc.java /hive/trunk/ql/src/test/org/apache/hadoop/hive/ql/metadata/TestHive.java /hive/trunk/ql/src/test/results/clientpositive/create_union_table.q.out /hive/trunk/ql/src/test/results/clientpositive/ctas.q.out /hive/trunk/ql/src/test/results/clientpositive/fileformat_sequencefile.q.out /hive/trunk/ql/src/test/results/clientpositive/fileformat_text.q.out /hive/trunk/ql/src/test/results/clientpositive/input15.q.out /hive/trunk/ql/src/test/results/clientpositive/inputddl1.q.out /hive/trunk/ql/src/test/results/clientpositive/inputddl2.q.out /hive/trunk/ql/src/test/results/clientpositive/inputddl3.q.out /hive/trunk/ql/src/test/results/clientpositive/merge3.q.out /hive/trunk/ql/src/test/results/clientpositive/rcfile_createas1.q.out /hive/trunk/ql/src/test/results/clientpositive/smb_mapjoin9.q.out /hive/trunk/ql/src/test/results/clientpositive/symlink_text_input_format.q.out /hive/trunk/ql/src/test/results/clientpositive/union25.q.out

            People

            • Assignee:
              Kevin Wilfong
              Reporter:
              He Yongqiang
            • Votes:
              0 Vote for this issue
              Watchers:
              13 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development