Hive
  1. Hive
  2. HIVE-4004

Incorrect status for AddPartition metastore event if RawStore commit fails

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 0.10.0
    • Fix Version/s: 0.11.0
    • Component/s: Metastore
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      For ADD PARTITION operations, the AddPartitionEvent does not care if the RawStore commit succeeded or not. This means that an AddPartitionEvent with status=true is fired even if the the actual ADD PARTITION operation failed. This will confuse any AddPartitionEvent listeners.

      Other MetastoreListenerEvents like CreateTableEvent correctly incorporate the status of the RawStore commit. Only AddPartitionEvent has this problem.

        Issue Links

          Activity

          Hide
          Dilip Joseph added a comment -
          Show
          Dilip Joseph added a comment - Code review at https://reviews.facebook.net/D8481
          Hide
          Dilip Joseph added a comment -

          HIVE-3938 touches the same part of the code, but does not fix the issue.

          Show
          Dilip Joseph added a comment - HIVE-3938 touches the same part of the code, but does not fix the issue.
          Hide
          Namit Jain added a comment -

          +1

          Show
          Namit Jain added a comment - +1
          Hide
          Namit Jain added a comment -

          Committed. Thanks Dilip

          Show
          Namit Jain added a comment - Committed. Thanks Dilip
          Hide
          Hudson added a comment -

          Integrated in hive-trunk-hadoop1 #93 (See https://builds.apache.org/job/hive-trunk-hadoop1/93/)
          HIVE-4004 Incorrect status for AddPartition metastore event if RawStore commit fails
          (Dilip Joseph via namit) (Revision 1448101)

          Result = ABORTED
          namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1448101
          Files :

          • /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
          • /hive/trunk/metastore/src/test/org/apache/hadoop/hive/metastore/DummyListener.java
          • /hive/trunk/metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreControlledCommit.java
          • /hive/trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestMetaStoreEventListenerOnlyOnCommit.java
          Show
          Hudson added a comment - Integrated in hive-trunk-hadoop1 #93 (See https://builds.apache.org/job/hive-trunk-hadoop1/93/ ) HIVE-4004 Incorrect status for AddPartition metastore event if RawStore commit fails (Dilip Joseph via namit) (Revision 1448101) Result = ABORTED namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1448101 Files : /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java /hive/trunk/metastore/src/test/org/apache/hadoop/hive/metastore/DummyListener.java /hive/trunk/metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreControlledCommit.java /hive/trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestMetaStoreEventListenerOnlyOnCommit.java
          Hide
          Hudson added a comment -

          Integrated in Hive-trunk-hadoop2 #130 (See https://builds.apache.org/job/Hive-trunk-hadoop2/130/)
          HIVE-4004 Incorrect status for AddPartition metastore event if RawStore commit fails
          (Dilip Joseph via namit) (Revision 1448101)

          Result = FAILURE
          namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1448101
          Files :

          • /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
          • /hive/trunk/metastore/src/test/org/apache/hadoop/hive/metastore/DummyListener.java
          • /hive/trunk/metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreControlledCommit.java
          • /hive/trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestMetaStoreEventListenerOnlyOnCommit.java
          Show
          Hudson added a comment - Integrated in Hive-trunk-hadoop2 #130 (See https://builds.apache.org/job/Hive-trunk-hadoop2/130/ ) HIVE-4004 Incorrect status for AddPartition metastore event if RawStore commit fails (Dilip Joseph via namit) (Revision 1448101) Result = FAILURE namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1448101 Files : /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java /hive/trunk/metastore/src/test/org/apache/hadoop/hive/metastore/DummyListener.java /hive/trunk/metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreControlledCommit.java /hive/trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestMetaStoreEventListenerOnlyOnCommit.java
          Hide
          Hudson added a comment -

          Integrated in Hive-trunk-h0.21 #1978 (See https://builds.apache.org/job/Hive-trunk-h0.21/1978/)
          HIVE-4004 Incorrect status for AddPartition metastore event if RawStore commit fails
          (Dilip Joseph via namit) (Revision 1448101)

          Result = SUCCESS
          namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1448101
          Files :

          • /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
          • /hive/trunk/metastore/src/test/org/apache/hadoop/hive/metastore/DummyListener.java
          • /hive/trunk/metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreControlledCommit.java
          • /hive/trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestMetaStoreEventListenerOnlyOnCommit.java
          Show
          Hudson added a comment - Integrated in Hive-trunk-h0.21 #1978 (See https://builds.apache.org/job/Hive-trunk-h0.21/1978/ ) HIVE-4004 Incorrect status for AddPartition metastore event if RawStore commit fails (Dilip Joseph via namit) (Revision 1448101) Result = SUCCESS namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1448101 Files : /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java /hive/trunk/metastore/src/test/org/apache/hadoop/hive/metastore/DummyListener.java /hive/trunk/metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreControlledCommit.java /hive/trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestMetaStoreEventListenerOnlyOnCommit.java
          Hide
          Mithun Radhakrishnan added a comment -

          This is a good fix. Could we please have this checked into branch-0.10/ as well?

          Show
          Mithun Radhakrishnan added a comment - This is a good fix. Could we please have this checked into branch-0.10/ as well?

            People

            • Assignee:
              Dilip Joseph
              Reporter:
              Dilip Joseph
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development