Hive
  1. Hive
  2. HIVE-2589

Newly created partition should inherit properties from table

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.8.1, 0.9.0
    • Component/s: Metastore
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      This will make all the info contained in table properties available to partitions.

      1. hive-2589.patch
        0.7 kB
        Ashutosh Chauhan
      2. hive-2589.patch
        12 kB
        Ashutosh Chauhan
      3. hive-2589_branch8.patch
        13 kB
        Ashutosh Chauhan
      4. hive-2589_4.patch
        13 kB
        Ashutosh Chauhan
      5. hive-2589_3.patch
        4 kB
        Ashutosh Chauhan
      6. hive-2589_2.patch
        11 kB
        Ashutosh Chauhan
      7. hive-2589_1.patch
        12 kB
        Ashutosh Chauhan
      8. ASF.LICENSE.NOT.GRANTED--HIVE-2589.D1335.2.patch
        12 kB
        Phabricator
      9. ASF.LICENSE.NOT.GRANTED--HIVE-2589.D1335.1.patch
        12 kB
        Phabricator

        Issue Links

          Activity

          Hide
          Hudson added a comment -

          Integrated in Hive-trunk-hadoop2 #54 (See https://builds.apache.org/job/Hive-trunk-hadoop2/54/)
          HIVE-2589 : Newly created partition should inherit properties from table (Ashutosh Chauhan) (Revision 1234235)
          HIVE-2719. Revert HIVE-2589 (He Yongqiang via cws) (Revision 1232766)
          HIVE-2589: Newly created partition should inherit properties from table (Ashutosh Chauhan) (Revision 1230390)

          Result = ABORTED
          hashutosh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1234235
          Files :

          • /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
          • /hive/trunk/conf/hive-default.xml.template
          • /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
          • /hive/trunk/ql/src/test/org/apache/hadoop/hive/ql/QTestUtil.java
          • /hive/trunk/ql/src/test/queries/clientpositive/part_inherit_tbl_props.q
          • /hive/trunk/ql/src/test/queries/clientpositive/part_inherit_tbl_props_empty.q
          • /hive/trunk/ql/src/test/queries/clientpositive/part_inherit_tbl_props_with_star.q
          • /hive/trunk/ql/src/test/results/clientpositive/part_inherit_tbl_props.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/part_inherit_tbl_props_empty.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/part_inherit_tbl_props_with_star.q.out

          cws : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1232766
          Files :

          • /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
          • /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
          • /hive/trunk/ql/src/test/queries/clientpositive/part_inherit_tbl_props.q
          • /hive/trunk/ql/src/test/queries/clientpositive/part_inherit_tbl_props_empty.q
          • /hive/trunk/ql/src/test/queries/clientpositive/part_inherit_tbl_props_with_star.q
          • /hive/trunk/ql/src/test/results/clientpositive/part_inherit_tbl_props.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/part_inherit_tbl_props_empty.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/part_inherit_tbl_props_with_star.q.out

          hashutosh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1230390
          Files :

          • /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
          • /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
          • /hive/trunk/ql/src/test/queries/clientpositive/part_inherit_tbl_props.q
          • /hive/trunk/ql/src/test/queries/clientpositive/part_inherit_tbl_props_empty.q
          • /hive/trunk/ql/src/test/queries/clientpositive/part_inherit_tbl_props_with_star.q
          • /hive/trunk/ql/src/test/results/clientpositive/part_inherit_tbl_props.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/part_inherit_tbl_props_empty.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/part_inherit_tbl_props_with_star.q.out
          Show
          Hudson added a comment - Integrated in Hive-trunk-hadoop2 #54 (See https://builds.apache.org/job/Hive-trunk-hadoop2/54/ ) HIVE-2589 : Newly created partition should inherit properties from table (Ashutosh Chauhan) (Revision 1234235) HIVE-2719 . Revert HIVE-2589 (He Yongqiang via cws) (Revision 1232766) HIVE-2589 : Newly created partition should inherit properties from table (Ashutosh Chauhan) (Revision 1230390) Result = ABORTED hashutosh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1234235 Files : /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java /hive/trunk/conf/hive-default.xml.template /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java /hive/trunk/ql/src/test/org/apache/hadoop/hive/ql/QTestUtil.java /hive/trunk/ql/src/test/queries/clientpositive/part_inherit_tbl_props.q /hive/trunk/ql/src/test/queries/clientpositive/part_inherit_tbl_props_empty.q /hive/trunk/ql/src/test/queries/clientpositive/part_inherit_tbl_props_with_star.q /hive/trunk/ql/src/test/results/clientpositive/part_inherit_tbl_props.q.out /hive/trunk/ql/src/test/results/clientpositive/part_inherit_tbl_props_empty.q.out /hive/trunk/ql/src/test/results/clientpositive/part_inherit_tbl_props_with_star.q.out cws : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1232766 Files : /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java /hive/trunk/ql/src/test/queries/clientpositive/part_inherit_tbl_props.q /hive/trunk/ql/src/test/queries/clientpositive/part_inherit_tbl_props_empty.q /hive/trunk/ql/src/test/queries/clientpositive/part_inherit_tbl_props_with_star.q /hive/trunk/ql/src/test/results/clientpositive/part_inherit_tbl_props.q.out /hive/trunk/ql/src/test/results/clientpositive/part_inherit_tbl_props_empty.q.out /hive/trunk/ql/src/test/results/clientpositive/part_inherit_tbl_props_with_star.q.out hashutosh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1230390 Files : /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java /hive/trunk/ql/src/test/queries/clientpositive/part_inherit_tbl_props.q /hive/trunk/ql/src/test/queries/clientpositive/part_inherit_tbl_props_empty.q /hive/trunk/ql/src/test/queries/clientpositive/part_inherit_tbl_props_with_star.q /hive/trunk/ql/src/test/results/clientpositive/part_inherit_tbl_props.q.out /hive/trunk/ql/src/test/results/clientpositive/part_inherit_tbl_props_empty.q.out /hive/trunk/ql/src/test/results/clientpositive/part_inherit_tbl_props_with_star.q.out
          Hide
          Ashutosh Chauhan added a comment -

          This issue is closed now. It was released with the fix in 0.9.0. If there is a problem, please open a new jira and link this one with that.

          Show
          Ashutosh Chauhan added a comment - This issue is closed now. It was released with the fix in 0.9.0. If there is a problem, please open a new jira and link this one with that.
          Hide
          Hudson added a comment -

          Integrated in Hive-0.8.1-SNAPSHOT-h0.21 #172 (See https://builds.apache.org/job/Hive-0.8.1-SNAPSHOT-h0.21/172/)
          HIVE-2589. Newly created partition should inherit properties from table (Ashutosh Chauhan via cws)

          cws : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1234958
          Files :

          • /hive/branches/branch-0.8-r2/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
          • /hive/branches/branch-0.8-r2/conf/hive-default.xml.template
          • /hive/branches/branch-0.8-r2/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
          • /hive/branches/branch-0.8-r2/ql/src/test/org/apache/hadoop/hive/ql/QTestUtil.java
          • /hive/branches/branch-0.8-r2/ql/src/test/queries/clientpositive/part_inherit_tbl_props.q
          • /hive/branches/branch-0.8-r2/ql/src/test/queries/clientpositive/part_inherit_tbl_props_empty.q
          • /hive/branches/branch-0.8-r2/ql/src/test/queries/clientpositive/part_inherit_tbl_props_with_star.q
          • /hive/branches/branch-0.8-r2/ql/src/test/results/clientpositive/part_inherit_tbl_props.q.out
          • /hive/branches/branch-0.8-r2/ql/src/test/results/clientpositive/part_inherit_tbl_props_empty.q.out
          • /hive/branches/branch-0.8-r2/ql/src/test/results/clientpositive/part_inherit_tbl_props_with_star.q.out
          Show
          Hudson added a comment - Integrated in Hive-0.8.1-SNAPSHOT-h0.21 #172 (See https://builds.apache.org/job/Hive-0.8.1-SNAPSHOT-h0.21/172/ ) HIVE-2589 . Newly created partition should inherit properties from table (Ashutosh Chauhan via cws) cws : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1234958 Files : /hive/branches/branch-0.8-r2/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java /hive/branches/branch-0.8-r2/conf/hive-default.xml.template /hive/branches/branch-0.8-r2/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java /hive/branches/branch-0.8-r2/ql/src/test/org/apache/hadoop/hive/ql/QTestUtil.java /hive/branches/branch-0.8-r2/ql/src/test/queries/clientpositive/part_inherit_tbl_props.q /hive/branches/branch-0.8-r2/ql/src/test/queries/clientpositive/part_inherit_tbl_props_empty.q /hive/branches/branch-0.8-r2/ql/src/test/queries/clientpositive/part_inherit_tbl_props_with_star.q /hive/branches/branch-0.8-r2/ql/src/test/results/clientpositive/part_inherit_tbl_props.q.out /hive/branches/branch-0.8-r2/ql/src/test/results/clientpositive/part_inherit_tbl_props_empty.q.out /hive/branches/branch-0.8-r2/ql/src/test/results/clientpositive/part_inherit_tbl_props_with_star.q.out
          Hide
          Carl Steinbach added a comment -

          Backported to branch-0.8-r2. Thanks Ashutosh!

          Show
          Carl Steinbach added a comment - Backported to branch-0.8-r2. Thanks Ashutosh!
          Hide
          Ashutosh Chauhan added a comment -

          Patch for 0.8 branch. This patch passes all the tests on 0.8 branch

          BUILD SUCCESSFUL
          Total time: 285 minutes 46 seconds
          
          Show
          Ashutosh Chauhan added a comment - Patch for 0.8 branch. This patch passes all the tests on 0.8 branch BUILD SUCCESSFUL Total time: 285 minutes 46 seconds
          Hide
          Hudson added a comment -

          Integrated in Hive-trunk-h0.21 #1212 (See https://builds.apache.org/job/Hive-trunk-h0.21/1212/)
          HIVE-2589 : Newly created partition should inherit properties from table (Ashutosh Chauhan)

          hashutosh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1234235
          Files :

          • /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
          • /hive/trunk/conf/hive-default.xml.template
          • /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
          • /hive/trunk/ql/src/test/org/apache/hadoop/hive/ql/QTestUtil.java
          • /hive/trunk/ql/src/test/queries/clientpositive/part_inherit_tbl_props.q
          • /hive/trunk/ql/src/test/queries/clientpositive/part_inherit_tbl_props_empty.q
          • /hive/trunk/ql/src/test/queries/clientpositive/part_inherit_tbl_props_with_star.q
          • /hive/trunk/ql/src/test/results/clientpositive/part_inherit_tbl_props.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/part_inherit_tbl_props_empty.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/part_inherit_tbl_props_with_star.q.out
          Show
          Hudson added a comment - Integrated in Hive-trunk-h0.21 #1212 (See https://builds.apache.org/job/Hive-trunk-h0.21/1212/ ) HIVE-2589 : Newly created partition should inherit properties from table (Ashutosh Chauhan) hashutosh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1234235 Files : /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java /hive/trunk/conf/hive-default.xml.template /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java /hive/trunk/ql/src/test/org/apache/hadoop/hive/ql/QTestUtil.java /hive/trunk/ql/src/test/queries/clientpositive/part_inherit_tbl_props.q /hive/trunk/ql/src/test/queries/clientpositive/part_inherit_tbl_props_empty.q /hive/trunk/ql/src/test/queries/clientpositive/part_inherit_tbl_props_with_star.q /hive/trunk/ql/src/test/results/clientpositive/part_inherit_tbl_props.q.out /hive/trunk/ql/src/test/results/clientpositive/part_inherit_tbl_props_empty.q.out /hive/trunk/ql/src/test/results/clientpositive/part_inherit_tbl_props_with_star.q.out
          Hide
          Ashutosh Chauhan added a comment -

          Committed to trunk.

          Show
          Ashutosh Chauhan added a comment - Committed to trunk.
          Hide
          Ashutosh Chauhan added a comment -

          Ran the tests on trunk. All passed except ppr_pushdown.q which always fails in my environment with or without this patch. It is also getting tracked at HIVE-2686. Will start test run on 0.8-branch now. Carl, can I commit there too ?

          Show
          Ashutosh Chauhan added a comment - Ran the tests on trunk. All passed except ppr_pushdown.q which always fails in my environment with or without this patch. It is also getting tracked at HIVE-2686 . Will start test run on 0.8-branch now. Carl, can I commit there too ?
          Hide
          Phabricator added a comment -

          cwsteinbach has accepted the revision "HIVE-2589 [jira] Newly created partition should inherit properties from table".

          +1

          @Ashutosh: Can you test and commit?

          REVISION DETAIL
          https://reviews.facebook.net/D1335

          Show
          Phabricator added a comment - cwsteinbach has accepted the revision " HIVE-2589 [jira] Newly created partition should inherit properties from table". +1 @Ashutosh: Can you test and commit? REVISION DETAIL https://reviews.facebook.net/D1335
          Hide
          Phabricator added a comment -

          ashutoshc updated the revision "HIVE-2589 [jira] Newly created partition should inherit properties from table".
          Reviewers: JIRA, cwsteinbach, heyongqiang, njain

          Addresses Carl's comments.

          REVISION DETAIL
          https://reviews.facebook.net/D1335

          AFFECTED FILES
          conf/hive-default.xml.template
          metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
          common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
          ql/src/test/results/clientpositive/part_inherit_tbl_props.q.out
          ql/src/test/results/clientpositive/part_inherit_tbl_props_empty.q.out
          ql/src/test/results/clientpositive/part_inherit_tbl_props_with_star.q.out
          ql/src/test/org/apache/hadoop/hive/ql/QTestUtil.java
          ql/src/test/queries/clientpositive/part_inherit_tbl_props_with_star.q
          ql/src/test/queries/clientpositive/part_inherit_tbl_props.q
          ql/src/test/queries/clientpositive/part_inherit_tbl_props_empty.q

          Show
          Phabricator added a comment - ashutoshc updated the revision " HIVE-2589 [jira] Newly created partition should inherit properties from table". Reviewers: JIRA, cwsteinbach, heyongqiang, njain Addresses Carl's comments. REVISION DETAIL https://reviews.facebook.net/D1335 AFFECTED FILES conf/hive-default.xml.template metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java common/src/java/org/apache/hadoop/hive/conf/HiveConf.java ql/src/test/results/clientpositive/part_inherit_tbl_props.q.out ql/src/test/results/clientpositive/part_inherit_tbl_props_empty.q.out ql/src/test/results/clientpositive/part_inherit_tbl_props_with_star.q.out ql/src/test/org/apache/hadoop/hive/ql/QTestUtil.java ql/src/test/queries/clientpositive/part_inherit_tbl_props_with_star.q ql/src/test/queries/clientpositive/part_inherit_tbl_props.q ql/src/test/queries/clientpositive/part_inherit_tbl_props_empty.q
          Hide
          Phabricator added a comment -

          ashutoshc has commented on the revision "HIVE-2589 [jira] Newly created partition should inherit properties from table".

          INLINE COMMENTS
          common/src/java/org/apache/hadoop/hive/conf/HiveConf.java:122 Yeah, if someone is changing this key, metastore should get reinitialized with new value for that session. Tests also fail if I take it out from here.
          metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java:1569 Will do so.
          metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java:1574 Will do so.
          ql/src/test/queries/clientpositive/part_inherit_tbl_props_empty.q:1 Will do so.

          REVISION DETAIL
          https://reviews.facebook.net/D1335

          Show
          Phabricator added a comment - ashutoshc has commented on the revision " HIVE-2589 [jira] Newly created partition should inherit properties from table". INLINE COMMENTS common/src/java/org/apache/hadoop/hive/conf/HiveConf.java:122 Yeah, if someone is changing this key, metastore should get reinitialized with new value for that session. Tests also fail if I take it out from here. metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java:1569 Will do so. metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java:1574 Will do so. ql/src/test/queries/clientpositive/part_inherit_tbl_props_empty.q:1 Will do so. REVISION DETAIL https://reviews.facebook.net/D1335
          Hide
          Phabricator added a comment -

          cwsteinbach has requested changes to the revision "HIVE-2589 [jira] Newly created partition should inherit properties from table".

          INLINE COMMENTS
          metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java:1574 The description in hive-default.xml.template should mention that '*' denotes that all table properties will be inherited by new partitions.
          metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java:1569 Please add a comment at the beginning of this block: "// Inherit table properties"
          ql/src/test/org/apache/hadoop/hive/ql/QTestUtil.java:363 Thanks for tracking this down!
          common/src/java/org/apache/hadoop/hive/conf/HiveConf.java:122 I'm not sure this belongs here. Do you really want the metastore to get reinitialized every time someone resets this configuration property?
          ql/src/test/queries/clientpositive/part_inherit_tbl_props_empty.q:1 Explicitly set the configuration property to "" just to make things clear.

          REVISION DETAIL
          https://reviews.facebook.net/D1335

          Show
          Phabricator added a comment - cwsteinbach has requested changes to the revision " HIVE-2589 [jira] Newly created partition should inherit properties from table". INLINE COMMENTS metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java:1574 The description in hive-default.xml.template should mention that '*' denotes that all table properties will be inherited by new partitions. metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java:1569 Please add a comment at the beginning of this block: "// Inherit table properties" ql/src/test/org/apache/hadoop/hive/ql/QTestUtil.java:363 Thanks for tracking this down! common/src/java/org/apache/hadoop/hive/conf/HiveConf.java:122 I'm not sure this belongs here. Do you really want the metastore to get reinitialized every time someone resets this configuration property? ql/src/test/queries/clientpositive/part_inherit_tbl_props_empty.q:1 Explicitly set the configuration property to "" just to make things clear. REVISION DETAIL https://reviews.facebook.net/D1335
          Hide
          Phabricator added a comment -

          ashutoshc requested code review of "HIVE-2589 [jira] Newly created partition should inherit properties from table".
          Reviewers: JIRA

          https://issues.apache.org/jira/browse/HIVE-2589

          Fixes an existing bug in test code which got exposed from HIVE-2589 patch. Fixes it.

          This will make all the info contained in table properties available to partitions.

          TEST PLAN
          EMPTY

          REVISION DETAIL
          https://reviews.facebook.net/D1335

          AFFECTED FILES
          conf/hive-default.xml.template
          metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
          common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
          ql/src/test/results/clientpositive/part_inherit_tbl_props.q.out
          ql/src/test/results/clientpositive/part_inherit_tbl_props_empty.q.out
          ql/src/test/results/clientpositive/part_inherit_tbl_props_with_star.q.out
          ql/src/test/org/apache/hadoop/hive/ql/QTestUtil.java
          ql/src/test/queries/clientpositive/part_inherit_tbl_props_with_star.q
          ql/src/test/queries/clientpositive/part_inherit_tbl_props.q
          ql/src/test/queries/clientpositive/part_inherit_tbl_props_empty.q

          MANAGE HERALD DIFFERENTIAL RULES
          https://reviews.facebook.net/herald/view/differential/

          WHY DID I GET THIS EMAIL?
          https://reviews.facebook.net/herald/transcript/2793/

          Tip: use the X-Herald-Rules header to filter Herald messages in your client.

          Show
          Phabricator added a comment - ashutoshc requested code review of " HIVE-2589 [jira] Newly created partition should inherit properties from table". Reviewers: JIRA https://issues.apache.org/jira/browse/HIVE-2589 Fixes an existing bug in test code which got exposed from HIVE-2589 patch. Fixes it. This will make all the info contained in table properties available to partitions. TEST PLAN EMPTY REVISION DETAIL https://reviews.facebook.net/D1335 AFFECTED FILES conf/hive-default.xml.template metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java common/src/java/org/apache/hadoop/hive/conf/HiveConf.java ql/src/test/results/clientpositive/part_inherit_tbl_props.q.out ql/src/test/results/clientpositive/part_inherit_tbl_props_empty.q.out ql/src/test/results/clientpositive/part_inherit_tbl_props_with_star.q.out ql/src/test/org/apache/hadoop/hive/ql/QTestUtil.java ql/src/test/queries/clientpositive/part_inherit_tbl_props_with_star.q ql/src/test/queries/clientpositive/part_inherit_tbl_props.q ql/src/test/queries/clientpositive/part_inherit_tbl_props_empty.q MANAGE HERALD DIFFERENTIAL RULES https://reviews.facebook.net/herald/view/differential/ WHY DID I GET THIS EMAIL? https://reviews.facebook.net/herald/transcript/2793/ Tip: use the X-Herald-Rules header to filter Herald messages in your client.
          Hide
          Ashutosh Chauhan added a comment -

          Ready for review. Tests passed on my local machine.

          Show
          Ashutosh Chauhan added a comment - Ready for review. Tests passed on my local machine.
          Hide
          Ashutosh Chauhan added a comment -

          Missed few files.

          Show
          Ashutosh Chauhan added a comment - Missed few files.
          Hide
          Ashutosh Chauhan added a comment -

          Same patch as HIVE-2589_2.patch with bug fix in QTestUtil.java which makes all the tests pass.

          Show
          Ashutosh Chauhan added a comment - Same patch as HIVE-2589 _2.patch with bug fix in QTestUtil.java which makes all the tests pass.
          Hide
          Thomas Weise added a comment -

          Need this for HCat 0.3

          Show
          Thomas Weise added a comment - Need this for HCat 0.3
          Hide
          Hudson added a comment -

          Integrated in Hive-trunk-h0.21 #1204 (See https://builds.apache.org/job/Hive-trunk-h0.21/1204/)
          HIVE-2719. Revert HIVE-2589 (He Yongqiang via cws)

          cws : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1232766
          Files :

          • /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
          • /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
          • /hive/trunk/ql/src/test/queries/clientpositive/part_inherit_tbl_props.q
          • /hive/trunk/ql/src/test/queries/clientpositive/part_inherit_tbl_props_empty.q
          • /hive/trunk/ql/src/test/queries/clientpositive/part_inherit_tbl_props_with_star.q
          • /hive/trunk/ql/src/test/results/clientpositive/part_inherit_tbl_props.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/part_inherit_tbl_props_empty.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/part_inherit_tbl_props_with_star.q.out
          Show
          Hudson added a comment - Integrated in Hive-trunk-h0.21 #1204 (See https://builds.apache.org/job/Hive-trunk-h0.21/1204/ ) HIVE-2719 . Revert HIVE-2589 (He Yongqiang via cws) cws : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1232766 Files : /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java /hive/trunk/ql/src/test/queries/clientpositive/part_inherit_tbl_props.q /hive/trunk/ql/src/test/queries/clientpositive/part_inherit_tbl_props_empty.q /hive/trunk/ql/src/test/queries/clientpositive/part_inherit_tbl_props_with_star.q /hive/trunk/ql/src/test/results/clientpositive/part_inherit_tbl_props.q.out /hive/trunk/ql/src/test/results/clientpositive/part_inherit_tbl_props_empty.q.out /hive/trunk/ql/src/test/results/clientpositive/part_inherit_tbl_props_with_star.q.out
          Hide
          Hudson added a comment -

          Integrated in Hive-trunk-h0.21-dbg #5 (See https://builds.apache.org/job/Hive-trunk-h0.21-dbg/5/)
          HIVE-2589: Newly created partition should inherit properties from table (Ashutosh Chauhan)

          hashutosh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1230390
          Files :

          • /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
          • /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
          • /hive/trunk/ql/src/test/queries/clientpositive/part_inherit_tbl_props.q
          • /hive/trunk/ql/src/test/queries/clientpositive/part_inherit_tbl_props_empty.q
          • /hive/trunk/ql/src/test/queries/clientpositive/part_inherit_tbl_props_with_star.q
          • /hive/trunk/ql/src/test/results/clientpositive/part_inherit_tbl_props.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/part_inherit_tbl_props_empty.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/part_inherit_tbl_props_with_star.q.out
          Show
          Hudson added a comment - Integrated in Hive-trunk-h0.21-dbg #5 (See https://builds.apache.org/job/Hive-trunk-h0.21-dbg/5/ ) HIVE-2589 : Newly created partition should inherit properties from table (Ashutosh Chauhan) hashutosh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1230390 Files : /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java /hive/trunk/ql/src/test/queries/clientpositive/part_inherit_tbl_props.q /hive/trunk/ql/src/test/queries/clientpositive/part_inherit_tbl_props_empty.q /hive/trunk/ql/src/test/queries/clientpositive/part_inherit_tbl_props_with_star.q /hive/trunk/ql/src/test/results/clientpositive/part_inherit_tbl_props.q.out /hive/trunk/ql/src/test/results/clientpositive/part_inherit_tbl_props_empty.q.out /hive/trunk/ql/src/test/results/clientpositive/part_inherit_tbl_props_with_star.q.out
          Hide
          Hudson added a comment -

          Integrated in Hive-trunk-h0.21-dbg #4 (See https://builds.apache.org/job/Hive-trunk-h0.21-dbg/4/)
          HIVE-2589: Newly created partition should inherit properties from table (Ashutosh Chauhan)

          hashutosh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1230390
          Files :

          • /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
          • /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
          • /hive/trunk/ql/src/test/queries/clientpositive/part_inherit_tbl_props.q
          • /hive/trunk/ql/src/test/queries/clientpositive/part_inherit_tbl_props_empty.q
          • /hive/trunk/ql/src/test/queries/clientpositive/part_inherit_tbl_props_with_star.q
          • /hive/trunk/ql/src/test/results/clientpositive/part_inherit_tbl_props.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/part_inherit_tbl_props_empty.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/part_inherit_tbl_props_with_star.q.out
          Show
          Hudson added a comment - Integrated in Hive-trunk-h0.21-dbg #4 (See https://builds.apache.org/job/Hive-trunk-h0.21-dbg/4/ ) HIVE-2589 : Newly created partition should inherit properties from table (Ashutosh Chauhan) hashutosh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1230390 Files : /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java /hive/trunk/ql/src/test/queries/clientpositive/part_inherit_tbl_props.q /hive/trunk/ql/src/test/queries/clientpositive/part_inherit_tbl_props_empty.q /hive/trunk/ql/src/test/queries/clientpositive/part_inherit_tbl_props_with_star.q /hive/trunk/ql/src/test/results/clientpositive/part_inherit_tbl_props.q.out /hive/trunk/ql/src/test/results/clientpositive/part_inherit_tbl_props_empty.q.out /hive/trunk/ql/src/test/results/clientpositive/part_inherit_tbl_props_with_star.q.out
          Hide
          Hudson added a comment -

          Integrated in Hive-trunk-h0.21-dbg #3 (See https://builds.apache.org/job/Hive-trunk-h0.21-dbg/3/)
          HIVE-2589: Newly created partition should inherit properties from table (Ashutosh Chauhan)

          hashutosh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1230390
          Files :

          • /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
          • /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
          • /hive/trunk/ql/src/test/queries/clientpositive/part_inherit_tbl_props.q
          • /hive/trunk/ql/src/test/queries/clientpositive/part_inherit_tbl_props_empty.q
          • /hive/trunk/ql/src/test/queries/clientpositive/part_inherit_tbl_props_with_star.q
          • /hive/trunk/ql/src/test/results/clientpositive/part_inherit_tbl_props.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/part_inherit_tbl_props_empty.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/part_inherit_tbl_props_with_star.q.out
          Show
          Hudson added a comment - Integrated in Hive-trunk-h0.21-dbg #3 (See https://builds.apache.org/job/Hive-trunk-h0.21-dbg/3/ ) HIVE-2589 : Newly created partition should inherit properties from table (Ashutosh Chauhan) hashutosh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1230390 Files : /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java /hive/trunk/ql/src/test/queries/clientpositive/part_inherit_tbl_props.q /hive/trunk/ql/src/test/queries/clientpositive/part_inherit_tbl_props_empty.q /hive/trunk/ql/src/test/queries/clientpositive/part_inherit_tbl_props_with_star.q /hive/trunk/ql/src/test/results/clientpositive/part_inherit_tbl_props.q.out /hive/trunk/ql/src/test/results/clientpositive/part_inherit_tbl_props_empty.q.out /hive/trunk/ql/src/test/results/clientpositive/part_inherit_tbl_props_with_star.q.out
          Hide
          Ashutosh Chauhan added a comment -

          This has come up before. Please, see the discussion on HIVE-2631

          Show
          Ashutosh Chauhan added a comment - This has come up before. Please, see the discussion on HIVE-2631
          Hide
          Namit Jain added a comment -

          @Ashutosh, usually we dont commit our patches ourselves. The chances of a mistake are high.

          Show
          Namit Jain added a comment - @Ashutosh, usually we dont commit our patches ourselves. The chances of a mistake are high.
          Hide
          Hudson added a comment -

          Integrated in Hive-trunk-h0.21 #1195 (See https://builds.apache.org/job/Hive-trunk-h0.21/1195/)
          HIVE-2589: Newly created partition should inherit properties from table (Ashutosh Chauhan)

          hashutosh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1230390
          Files :

          • /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
          • /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
          • /hive/trunk/ql/src/test/queries/clientpositive/part_inherit_tbl_props.q
          • /hive/trunk/ql/src/test/queries/clientpositive/part_inherit_tbl_props_empty.q
          • /hive/trunk/ql/src/test/queries/clientpositive/part_inherit_tbl_props_with_star.q
          • /hive/trunk/ql/src/test/results/clientpositive/part_inherit_tbl_props.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/part_inherit_tbl_props_empty.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/part_inherit_tbl_props_with_star.q.out
          Show
          Hudson added a comment - Integrated in Hive-trunk-h0.21 #1195 (See https://builds.apache.org/job/Hive-trunk-h0.21/1195/ ) HIVE-2589 : Newly created partition should inherit properties from table (Ashutosh Chauhan) hashutosh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1230390 Files : /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java /hive/trunk/ql/src/test/queries/clientpositive/part_inherit_tbl_props.q /hive/trunk/ql/src/test/queries/clientpositive/part_inherit_tbl_props_empty.q /hive/trunk/ql/src/test/queries/clientpositive/part_inherit_tbl_props_with_star.q /hive/trunk/ql/src/test/results/clientpositive/part_inherit_tbl_props.q.out /hive/trunk/ql/src/test/results/clientpositive/part_inherit_tbl_props_empty.q.out /hive/trunk/ql/src/test/results/clientpositive/part_inherit_tbl_props_with_star.q.out
          Hide
          Ashutosh Chauhan added a comment -

          Committed to trunk.

          Show
          Ashutosh Chauhan added a comment - Committed to trunk.
          Hide
          Namit Jain added a comment -

          Sorry for the delay, looks good

          +1

          Show
          Namit Jain added a comment - Sorry for the delay, looks good +1
          Hide
          Ashutosh Chauhan added a comment -

          Can anyone take a look at this one?

          Show
          Ashutosh Chauhan added a comment - Can anyone take a look at this one?
          Hide
          Ashutosh Chauhan added a comment -

          Ping.

          Show
          Ashutosh Chauhan added a comment - Ping.
          Hide
          Ashutosh Chauhan added a comment -

          @Namit,
          Can you take a look? Else I have to rebase it again : (

          Show
          Ashutosh Chauhan added a comment - @Namit, Can you take a look? Else I have to rebase it again : (
          Hide
          Ashutosh Chauhan added a comment -

          Patch Available

          Show
          Ashutosh Chauhan added a comment - Patch Available
          Hide
          Ashutosh Chauhan added a comment -

          Here you go. Rebased patch.

          Show
          Ashutosh Chauhan added a comment - Here you go. Rebased patch.
          Hide
          Namit Jain added a comment -

          @Ashutosh, sorry about the delay.
          Can you refresh ? I am getting some conflicts while applying the patch.
          I will take a look.

          Show
          Namit Jain added a comment - @Ashutosh, sorry about the delay. Can you refresh ? I am getting some conflicts while applying the patch. I will take a look.
          Hide
          Ashutosh Chauhan added a comment -

          Can any one take a look at this one?

          Show
          Ashutosh Chauhan added a comment - Can any one take a look at this one?
          Hide
          Anoop Sam John added a comment -

          Ashutosh...
          Sorry I misread the patch. Yes correct only... I have missed the part that in the
          if(inheritKeys.contains("*")) you will change the inheritKeys set itself...
          Yes perfect . No suggestion from my side...

          Thanks
          Anoop

          Show
          Anoop Sam John added a comment - Ashutosh... Sorry I misread the patch. Yes correct only... I have missed the part that in the if(inheritKeys.contains("*")) you will change the inheritKeys set itself... Yes perfect . No suggestion from my side... Thanks Anoop
          Hide
          Ashutosh Chauhan added a comment -

          @Anoop,
          If we have for loop in else statement, then no keys will get added in * case which is wrong. I don't understand why are you suggesting it that way.

          Show
          Ashutosh Chauhan added a comment - @Anoop, If we have for loop in else statement, then no keys will get added in * case which is wrong. I don't understand why are you suggesting it that way.
          Hide
          Anoop Sam John added a comment -

          if(inheritKeys.contains("*"))

          { inheritKeys = tblParams.keySet(); }
          for (String key : inheritKeys) {
          ...............

          Can we make the for loop in else.. when the keys contains *, then already all the table params are added... even if some other keys are added in teh comma seperated values also no need to consider thos...
          if(inheritKeys.contains("*")){ inheritKeys = tblParams.keySet(); }

          else{
          for (String key : inheritKeys)

          { ............... }
          Show
          Anoop Sam John added a comment - if(inheritKeys.contains("*")) { inheritKeys = tblParams.keySet(); } for (String key : inheritKeys) { ............... Can we make the for loop in else.. when the keys contains *, then already all the table params are added... even if some other keys are added in teh comma seperated values also no need to consider thos... if(inheritKeys.contains("*")){ inheritKeys = tblParams.keySet(); } else{ for (String key : inheritKeys) { ............... }
          Hide
          Ashutosh Chauhan added a comment -

          Thanks Anoop for taking a look. Updated patch incorporating Anoop's feedback.

          Show
          Ashutosh Chauhan added a comment - Thanks Anoop for taking a look. Updated patch incorporating Anoop's feedback.
          Hide
          Anoop Sam John added a comment -

          Hi Ashutosh
          I have gone through the patch and some thoughts came to my mind..
          1. What about the validation of the keys that user has provided in the xml for the key hive.metastore.partition.inherit.table.properties?
          Right now as per the code it will add an entry with that key and value as null in the parameters Map.
          2. When user has provided teh value for the key as *,key1 should not consider this *?

          Thanks
          Anoop

          Show
          Anoop Sam John added a comment - Hi Ashutosh I have gone through the patch and some thoughts came to my mind.. 1. What about the validation of the keys that user has provided in the xml for the key hive.metastore.partition.inherit.table.properties? Right now as per the code it will add an entry with that key and value as null in the parameters Map. 2. When user has provided teh value for the key as *,key1 should not consider this *? Thanks Anoop
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/2901/
          -----------------------------------------------------------

          Review request for hive.

          Summary
          -------

          Introduces a new config key to enable inheritance of table properties at the time of partition creation.

          This addresses bug HIVE-2589.
          https://issues.apache.org/jira/browse/HIVE-2589

          Diffs


          trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1204627
          trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 1204627
          trunk/ql/src/test/queries/clientpositive/part_inherit_tbl_props.q PRE-CREATION
          trunk/ql/src/test/queries/clientpositive/part_inherit_tbl_props_empty.q PRE-CREATION
          trunk/ql/src/test/queries/clientpositive/part_inherit_tbl_props_with_star.q PRE-CREATION
          trunk/ql/src/test/results/clientpositive/part_inherit_tbl_props.q.out PRE-CREATION
          trunk/ql/src/test/results/clientpositive/part_inherit_tbl_props_empty.q.out PRE-CREATION
          trunk/ql/src/test/results/clientpositive/part_inherit_tbl_props_with_star.q.out PRE-CREATION

          Diff: https://reviews.apache.org/r/2901/diff

          Testing
          -------

          Contains three new *.q tests to verify functionality for all three cases.

          Thanks,

          Ashutosh

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2901/ ----------------------------------------------------------- Review request for hive. Summary ------- Introduces a new config key to enable inheritance of table properties at the time of partition creation. This addresses bug HIVE-2589 . https://issues.apache.org/jira/browse/HIVE-2589 Diffs trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1204627 trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 1204627 trunk/ql/src/test/queries/clientpositive/part_inherit_tbl_props.q PRE-CREATION trunk/ql/src/test/queries/clientpositive/part_inherit_tbl_props_empty.q PRE-CREATION trunk/ql/src/test/queries/clientpositive/part_inherit_tbl_props_with_star.q PRE-CREATION trunk/ql/src/test/results/clientpositive/part_inherit_tbl_props.q.out PRE-CREATION trunk/ql/src/test/results/clientpositive/part_inherit_tbl_props_empty.q.out PRE-CREATION trunk/ql/src/test/results/clientpositive/part_inherit_tbl_props_with_star.q.out PRE-CREATION Diff: https://reviews.apache.org/r/2901/diff Testing ------- Contains three new *.q tests to verify functionality for all three cases. Thanks, Ashutosh
          Hide
          Ashutosh Chauhan added a comment -

          Patch ready for review.

          Show
          Ashutosh Chauhan added a comment - Patch ready for review.
          Hide
          Ashutosh Chauhan added a comment -

          Default behavior continues to remain as earlier wherein no properties are inherited. Value of config key is either * which implies all properties needs to be inherited or a comma separated list of keys which will be inherited.

          Show
          Ashutosh Chauhan added a comment - Default behavior continues to remain as earlier wherein no properties are inherited. Value of config key is either * which implies all properties needs to be inherited or a comma separated list of keys which will be inherited.
          Hide
          Ashutosh Chauhan added a comment -

          Illustrative patch.

          Show
          Ashutosh Chauhan added a comment - Illustrative patch.

            People

            • Assignee:
              Ashutosh Chauhan
              Reporter:
              Ashutosh Chauhan
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development