Hive
  1. Hive
  2. HIVE-2835

Change default configuration for hive.exec.dynamic.partition

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.9.0
    • Component/s: None
    • Labels:
      None
    • Release Note:
      Dynamic Partitioning is now on by default.

      Description

      I think we should enable dynamic partitions by default.

        Activity

        Hide
        Phabricator added a comment -

        omalley requested code review of "HIVE-2835 [jira] Change default configuration for hive.exec.dynamic.partition".
        Reviewers: JIRA

        <<Enter Revision Title>>

        Change default config for dynamic partitions

        I think we should enable dynamic partitions by default.

        TEST PLAN
        EMPTY

        REVISION DETAIL
        https://reviews.facebook.net/D2157

        AFFECTED FILES
        conf/hive-default.xml.template

        MANAGE HERALD DIFFERENTIAL RULES
        https://reviews.facebook.net/herald/view/differential/

        WHY DID I GET THIS EMAIL?
        https://reviews.facebook.net/herald/transcript/4719/

        Tip: use the X-Herald-Rules header to filter Herald messages in your client.

        Show
        Phabricator added a comment - omalley requested code review of " HIVE-2835 [jira] Change default configuration for hive.exec.dynamic.partition". Reviewers: JIRA <<Enter Revision Title>> Change default config for dynamic partitions I think we should enable dynamic partitions by default. TEST PLAN EMPTY REVISION DETAIL https://reviews.facebook.net/D2157 AFFECTED FILES conf/hive-default.xml.template MANAGE HERALD DIFFERENTIAL RULES https://reviews.facebook.net/herald/view/differential/ WHY DID I GET THIS EMAIL? https://reviews.facebook.net/herald/transcript/4719/ Tip: use the X-Herald-Rules header to filter Herald messages in your client.
        Hide
        Edward Capriolo added a comment -

        +1 will commit when tests pass.

        Show
        Edward Capriolo added a comment - +1 will commit when tests pass.
        Hide
        Edward Capriolo added a comment -

        Actually -1 changing the .xml.template does nothing. The change has to happen to HiveConf.java.

        Index: conf/hive-default.xml.template
        ===================================================================
        --- conf/hive-default.xml.template	(revision 1298087)
        +++ conf/hive-default.xml.template	(working copy)
        @@ -791,7 +791,7 @@
         
         <property>
           <name>hive.exec.dynamic.partition</name>
        -  <value>false</value>
        +  <value>true</value>
           <description>Whether or not to allow dynamic partitions in DML/DDL.</description>
         </property>
         
        Index: common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
        ===================================================================
        --- common/src/java/org/apache/hadoop/hive/conf/HiveConf.java	(revision 1298087)
        +++ common/src/java/org/apache/hadoop/hive/conf/HiveConf.java	(working copy)
        @@ -172,7 +172,7 @@
             EXECPARALLETHREADNUMBER("hive.exec.parallel.thread.number", 8),
             HIVESPECULATIVEEXECREDUCERS("hive.mapred.reduce.tasks.speculative.execution", true),
             HIVECOUNTERSPULLINTERVAL("hive.exec.counters.pull.interval", 1000L),
        -    DYNAMICPARTITIONING("hive.exec.dynamic.partition", false),
        +    DYNAMICPARTITIONING("hive.exec.dynamic.partition", true),
             DYNAMICPARTITIONINGMODE("hive.exec.dynamic.partition.mode", "strict"),
             DYNAMICPARTITIONMAXPARTS("hive.exec.max.dynamic.partitions", 1000),
             DYNAMICPARTITIONMAXPARTSPERNODE("hive.exec.max.dynamic.partitions.pernode", 100),
        

        We will see if the change effects the output of other tests.

        Show
        Edward Capriolo added a comment - Actually -1 changing the .xml.template does nothing. The change has to happen to HiveConf.java. Index: conf/hive-default.xml.template =================================================================== --- conf/hive-default.xml.template (revision 1298087) +++ conf/hive-default.xml.template (working copy) @@ -791,7 +791,7 @@ <property> <name>hive.exec.dynamic.partition</name> - <value>false</value> + <value>true</value> <description>Whether or not to allow dynamic partitions in DML/DDL.</description> </property> Index: common/src/java/org/apache/hadoop/hive/conf/HiveConf.java =================================================================== --- common/src/java/org/apache/hadoop/hive/conf/HiveConf.java (revision 1298087) +++ common/src/java/org/apache/hadoop/hive/conf/HiveConf.java (working copy) @@ -172,7 +172,7 @@ EXECPARALLETHREADNUMBER("hive.exec.parallel.thread.number", 8), HIVESPECULATIVEEXECREDUCERS("hive.mapred.reduce.tasks.speculative.execution", true), HIVECOUNTERSPULLINTERVAL("hive.exec.counters.pull.interval", 1000L), - DYNAMICPARTITIONING("hive.exec.dynamic.partition", false), + DYNAMICPARTITIONING("hive.exec.dynamic.partition", true), DYNAMICPARTITIONINGMODE("hive.exec.dynamic.partition.mode", "strict"), DYNAMICPARTITIONMAXPARTS("hive.exec.max.dynamic.partitions", 1000), DYNAMICPARTITIONMAXPARTSPERNODE("hive.exec.max.dynamic.partitions.pernode", 100), We will see if the change effects the output of other tests.
        Hide
        Owen O'Malley added a comment -

        Thanks for catching the problem, Ed. I uploaded a new patch although clearly you already have the fixed patch.

        Show
        Owen O'Malley added a comment - Thanks for catching the problem, Ed. I uploaded a new patch although clearly you already have the fixed patch.
        Hide
        Phabricator added a comment -

        omalley updated the revision "HIVE-2835 [jira] Change default configuration for hive.exec.dynamic.partition".
        Reviewers: JIRA

        Added change to HiveConf.java

        REVISION DETAIL
        https://reviews.facebook.net/D2157

        AFFECTED FILES
        common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
        conf/hive-default.xml.template

        Show
        Phabricator added a comment - omalley updated the revision " HIVE-2835 [jira] Change default configuration for hive.exec.dynamic.partition". Reviewers: JIRA Added change to HiveConf.java REVISION DETAIL https://reviews.facebook.net/D2157 AFFECTED FILES common/src/java/org/apache/hadoop/hive/conf/HiveConf.java conf/hive-default.xml.template
        Hide
        Phabricator added a comment -

        ashutoshc has accepted the revision "HIVE-2835 [jira] Change default configuration for hive.exec.dynamic.partition".

        +1 Will commit if tests pass

        REVISION DETAIL
        https://reviews.facebook.net/D2157

        BRANCH
        h-2835

        Show
        Phabricator added a comment - ashutoshc has accepted the revision " HIVE-2835 [jira] Change default configuration for hive.exec.dynamic.partition". +1 Will commit if tests pass REVISION DETAIL https://reviews.facebook.net/D2157 BRANCH h-2835
        Hide
        Ashutosh Chauhan added a comment -

        Committed to trunk. Thanks, Owen!

        Show
        Ashutosh Chauhan added a comment - Committed to trunk. Thanks, Owen!
        Hide
        Hudson added a comment -

        Integrated in Hive-trunk-h0.21 #1314 (See https://builds.apache.org/job/Hive-trunk-h0.21/1314/)
        HIVE-2835: Change default configuration for hive.exec.dynamic.partition (Owen Omalley via hashutosh) (Revision 1301348)

        Result = FAILURE
        hashutosh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1301348
        Files :

        • /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
        • /hive/trunk/conf/hive-default.xml.template
        Show
        Hudson added a comment - Integrated in Hive-trunk-h0.21 #1314 (See https://builds.apache.org/job/Hive-trunk-h0.21/1314/ ) HIVE-2835 : Change default configuration for hive.exec.dynamic.partition (Owen Omalley via hashutosh) (Revision 1301348) Result = FAILURE hashutosh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1301348 Files : /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java /hive/trunk/conf/hive-default.xml.template
        Hide
        Ashutosh Chauhan added a comment -

        This issue is closed now. It was released with the fix in 0.9.0. If there is a problem, please open a new jira and link this one with that.

        Show
        Ashutosh Chauhan added a comment - This issue is closed now. It was released with the fix in 0.9.0. If there is a problem, please open a new jira and link this one with that.
        Hide
        Hudson added a comment -

        Integrated in Hive-trunk-hadoop2 #54 (See https://builds.apache.org/job/Hive-trunk-hadoop2/54/)
        HIVE-2835: Change default configuration for hive.exec.dynamic.partition (Owen Omalley via hashutosh) (Revision 1301348)

        Result = ABORTED
        hashutosh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1301348
        Files :

        • /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
        • /hive/trunk/conf/hive-default.xml.template
        Show
        Hudson added a comment - Integrated in Hive-trunk-hadoop2 #54 (See https://builds.apache.org/job/Hive-trunk-hadoop2/54/ ) HIVE-2835 : Change default configuration for hive.exec.dynamic.partition (Owen Omalley via hashutosh) (Revision 1301348) Result = ABORTED hashutosh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1301348 Files : /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java /hive/trunk/conf/hive-default.xml.template

          People

          • Assignee:
            Owen O'Malley
            Reporter:
            Owen O'Malley
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development