Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-3680

Include Table information in Hive's AddPartitionEvent.

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 0.9.1
    • 0.9.1, 0.10.0
    • Metastore
    • None

    Description

      This has to do with a minor overhaul of the HCatalog notifications that we're attempting in HCATALOG-546.

      It is proposed that HCatalog's notifications (on Add/Drop of Partitions) provide details to identify the affected partitions.

      Using the Partition object in AddPartitionEvent, one is able to retrieve the values of the partition-keys and the name of the Table. However, the partition-keys themselves aren't available (since the Table instance isn't part of the AddPartitionEvent).

      Adding the table-reference to the AddPartitionEvent and DropPartitionEvent classes will expose all the info we need. (The alternative is to query the metastore for the table's schema and use the partition-keys from there. :/)

      I'll post a patch for this shortly.

      Attachments

        1. HIVE-3680.branch9.patch
          6 kB
          Mithun Radhakrishnan
        2. HIVE-3680.trunk.patch
          7 kB
          Mithun Radhakrishnan

        Issue Links

          Activity

            Proposed change.

            mithun Mithun Radhakrishnan added a comment - Proposed change.
            mgrover Mark Grover added a comment -

            For completeness, the review is at
            https://reviews.apache.org/r/7910/

            mgrover Mark Grover added a comment - For completeness, the review is at https://reviews.apache.org/r/7910/

            Patch looks good. Mithun, can you also add a test for completeness. TestMetaStoreEventListener.java seems to be appropriate to add this test.

            ashutoshc Ashutosh Chauhan added a comment - Patch looks good. Mithun, can you also add a test for completeness. TestMetaStoreEventListener.java seems to be appropriate to add this test.

            @ashutoshc: Quite right. Here's the updated diff(s), with tests modified. Does this look alright?

            mithun Mithun Radhakrishnan added a comment - @ ashutoshc : Quite right. Here's the updated diff(s), with tests modified. Does this look alright?

            Yup, it does. +1 Running tests now, will commit if tests pass.

            ashutoshc Ashutosh Chauhan added a comment - Yup, it does. +1 Running tests now, will commit if tests pass.

            Committed to trunk. Thanks, Mithun!

            ashutoshc Ashutosh Chauhan added a comment - Committed to trunk. Thanks, Mithun!

            Thank you, ashutoshc. :] Any chance at a backport to branch-9? The patch is included.

            mithun Mithun Radhakrishnan added a comment - Thank you, ashutoshc . :] Any chance at a backport to branch-9? The patch is included.
            hudson Hudson added a comment -

            Integrated in Hive-trunk-h0.21 #1799 (See https://builds.apache.org/job/Hive-trunk-h0.21/1799/)
            HIVE-3680 : Include Table information in Hive's AddPartitionEvent. (Mithun Radhakrishnan via Ashutosh Chauhan) (Revision 1409861)

            Result = FAILURE
            hashutosh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1409861
            Files :

            • /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
            • /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/events/AddPartitionEvent.java
            • /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/events/DropPartitionEvent.java
            • /hive/trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestMetaStoreEventListener.java
            hudson Hudson added a comment - Integrated in Hive-trunk-h0.21 #1799 (See https://builds.apache.org/job/Hive-trunk-h0.21/1799/ ) HIVE-3680 : Include Table information in Hive's AddPartitionEvent. (Mithun Radhakrishnan via Ashutosh Chauhan) (Revision 1409861) Result = FAILURE hashutosh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1409861 Files : /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/events/AddPartitionEvent.java /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/events/DropPartitionEvent.java /hive/trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestMetaStoreEventListener.java

            Committed to 0.9 branch as well.

            ashutoshc Ashutosh Chauhan added a comment - Committed to 0.9 branch as well.
            hudson Hudson added a comment -

            Integrated in Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false #207 (See https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/207/)
            HIVE-3680 : Include Table information in Hive's AddPartitionEvent. (Mithun RadhaKrishnan via Ashutosh Chauhan) (Revision 1412750)

            Result = FAILURE
            hashutosh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1412750
            Files :

            • /hive/branches/branch-0.9/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
            • /hive/branches/branch-0.9/metastore/src/java/org/apache/hadoop/hive/metastore/events/AddPartitionEvent.java
            • /hive/branches/branch-0.9/metastore/src/java/org/apache/hadoop/hive/metastore/events/DropPartitionEvent.java
            • /hive/branches/branch-0.9/metastore/src/test/org/apache/hadoop/hive/metastore/TestMetaStoreEventListener.java
            hudson Hudson added a comment - Integrated in Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false #207 (See https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/207/ ) HIVE-3680 : Include Table information in Hive's AddPartitionEvent. (Mithun RadhaKrishnan via Ashutosh Chauhan) (Revision 1412750) Result = FAILURE hashutosh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1412750 Files : /hive/branches/branch-0.9/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java /hive/branches/branch-0.9/metastore/src/java/org/apache/hadoop/hive/metastore/events/AddPartitionEvent.java /hive/branches/branch-0.9/metastore/src/java/org/apache/hadoop/hive/metastore/events/DropPartitionEvent.java /hive/branches/branch-0.9/metastore/src/test/org/apache/hadoop/hive/metastore/TestMetaStoreEventListener.java
            hudson Hudson added a comment -

            Integrated in Hive-0.9.1-SNAPSHOT-h0.21 #207 (See https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21/207/)
            HIVE-3680 : Include Table information in Hive's AddPartitionEvent. (Mithun RadhaKrishnan via Ashutosh Chauhan) (Revision 1412750)

            Result = SUCCESS
            hashutosh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1412750
            Files :

            • /hive/branches/branch-0.9/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
            • /hive/branches/branch-0.9/metastore/src/java/org/apache/hadoop/hive/metastore/events/AddPartitionEvent.java
            • /hive/branches/branch-0.9/metastore/src/java/org/apache/hadoop/hive/metastore/events/DropPartitionEvent.java
            • /hive/branches/branch-0.9/metastore/src/test/org/apache/hadoop/hive/metastore/TestMetaStoreEventListener.java
            hudson Hudson added a comment - Integrated in Hive-0.9.1-SNAPSHOT-h0.21 #207 (See https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21/207/ ) HIVE-3680 : Include Table information in Hive's AddPartitionEvent. (Mithun RadhaKrishnan via Ashutosh Chauhan) (Revision 1412750) Result = SUCCESS hashutosh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1412750 Files : /hive/branches/branch-0.9/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java /hive/branches/branch-0.9/metastore/src/java/org/apache/hadoop/hive/metastore/events/AddPartitionEvent.java /hive/branches/branch-0.9/metastore/src/java/org/apache/hadoop/hive/metastore/events/DropPartitionEvent.java /hive/branches/branch-0.9/metastore/src/test/org/apache/hadoop/hive/metastore/TestMetaStoreEventListener.java
            hudson Hudson added a comment -

            Integrated in Hive-trunk-hadoop2 #54 (See https://builds.apache.org/job/Hive-trunk-hadoop2/54/)
            HIVE-3680 : Include Table information in Hive's AddPartitionEvent. (Mithun Radhakrishnan via Ashutosh Chauhan) (Revision 1409861)

            Result = ABORTED
            hashutosh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1409861
            Files :

            • /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
            • /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/events/AddPartitionEvent.java
            • /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/events/DropPartitionEvent.java
            • /hive/trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestMetaStoreEventListener.java
            hudson Hudson added a comment - Integrated in Hive-trunk-hadoop2 #54 (See https://builds.apache.org/job/Hive-trunk-hadoop2/54/ ) HIVE-3680 : Include Table information in Hive's AddPartitionEvent. (Mithun Radhakrishnan via Ashutosh Chauhan) (Revision 1409861) Result = ABORTED hashutosh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1409861 Files : /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/events/AddPartitionEvent.java /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/events/DropPartitionEvent.java /hive/trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestMetaStoreEventListener.java

            This issue is fixed and released as part of 0.10.0 release. If you find an issue which seems to be related to this one, please create a new jira and link this one with new jira.

            ashutoshc Ashutosh Chauhan added a comment - This issue is fixed and released as part of 0.10.0 release. If you find an issue which seems to be related to this one, please create a new jira and link this one with new jira.

            People

              mithun Mithun Radhakrishnan
              mithun Mithun Radhakrishnan
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: