Hive
  1. Hive
  2. HIVE-2109

No lock for some non-mapred tasks config variable hive.lock.mapred.only.operation added

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.8.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      do not do lock for some DDLTask or data move task.

      1. HIVE-2109.1.patch
        6 kB
        He Yongqiang
      2. HIVE-2109.2.patch
        8 kB
        He Yongqiang
      3. HIVE-2109.3.patch
        8 kB
        He Yongqiang
      4. HIVE-2109.4.patch
        29 kB
        He Yongqiang
      5. HIVE-2109.5.patch
        29 kB
        He Yongqiang

        Activity

        Hide
        Namit Jain added a comment -

        Add a new configuration parameter, where the DDLs do not need a lock.
        This parmeter's default value can be false, i.e the current behavior is
        not changed.

        Show
        Namit Jain added a comment - Add a new configuration parameter, where the DDLs do not need a lock. This parmeter's default value can be false, i.e the current behavior is not changed.
        Hide
        He Yongqiang added a comment -

        It is not easy to add a testcase for it. So i manually tested it.

        Show
        He Yongqiang added a comment - It is not easy to add a testcase for it. So i manually tested it.
        Hide
        Namit Jain added a comment -

        Change the name of the parameter:

        hive.lock.metadata.only.operation

        Add the new parameter in hive-default.xml

        Also, can you add a review board entry ?

        Show
        Namit Jain added a comment - Change the name of the parameter: hive.lock.metadata.only.operation Add the new parameter in hive-default.xml Also, can you add a review board entry ?
        Hide
        He Yongqiang added a comment -

        address namit's comments

        Show
        He Yongqiang added a comment - address namit's comments
        Hide
        He Yongqiang added a comment -
        Show
        He Yongqiang added a comment - review board: https://reviews.apache.org/r/617/
        Hide
        jiraposter@reviews.apache.org added a comment -

        -----------------------------------------------------------
        This is an automatically generated e-mail. To reply, visit:
        https://reviews.apache.org/r/617/
        -----------------------------------------------------------

        Review request for hive.

        Summary
        -------

        No lock for some non-mapred tasks.

        This addresses bug HIVE-2109.
        https://issues.apache.org/jira/browse/HIVE-2109

        Diffs


        trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1094342
        trunk/conf/hive-default.xml 1094342
        trunk/ql/src/java/org/apache/hadoop/hive/ql/Driver.java 1094342
        trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java 1094342
        trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/ExecDriver.java 1094342
        trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/MapredLocalTask.java 1094342
        trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Task.java 1094342
        trunk/ql/src/java/org/apache/hadoop/hive/ql/io/rcfile/merge/BlockMergeTask.java 1094342
        trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java 1094342
        trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/DDLWork.java 1094342

        Diff: https://reviews.apache.org/r/617/diff

        Testing
        -------

        Thanks,

        Yongqiang

        Show
        jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/617/ ----------------------------------------------------------- Review request for hive. Summary ------- No lock for some non-mapred tasks. This addresses bug HIVE-2109 . https://issues.apache.org/jira/browse/HIVE-2109 Diffs trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1094342 trunk/conf/hive-default.xml 1094342 trunk/ql/src/java/org/apache/hadoop/hive/ql/Driver.java 1094342 trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java 1094342 trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/ExecDriver.java 1094342 trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/MapredLocalTask.java 1094342 trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Task.java 1094342 trunk/ql/src/java/org/apache/hadoop/hive/ql/io/rcfile/merge/BlockMergeTask.java 1094342 trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java 1094342 trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/DDLWork.java 1094342 Diff: https://reviews.apache.org/r/617/diff Testing ------- Thanks, Yongqiang
        Hide
        Namit Jain added a comment -

        +1

        Show
        Namit Jain added a comment - +1
        Hide
        He Yongqiang added a comment -

        hope this time, it can pass all tests

        Show
        He Yongqiang added a comment - hope this time, it can pass all tests
        Hide
        Namit Jain added a comment - - edited

        Can you regenerate the patch - I am getting some conflicts

        Show
        Namit Jain added a comment - - edited Can you regenerate the patch - I am getting some conflicts
        Hide
        Namit Jain added a comment -

        Committed. Thanks Yongqiang

        Show
        Namit Jain added a comment - Committed. Thanks Yongqiang

          People

          • Assignee:
            He Yongqiang
            Reporter:
            He Yongqiang
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development