Hive
  1. Hive
  2. HIVE-4588

Support session level hooks for HiveServer2

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.11.0
    • Fix Version/s: 0.12.0
    • Component/s: HiveServer2
    • Labels:
      None

      Description

      Support session level hooks for HiveSrver2. The configured hooks will get executed at beginning of each new session.
      This is useful for auditing connections, possibly tuning the session level properties etc.

      1. HIVE-4588.3.patch
        11 kB
        Prasad Mujumdar
      2. HIVE-4588.5.patch
        18 kB
        Prasad Mujumdar
      3. HIVE-4588-1.patch
        10 kB
        Prasad Mujumdar

        Activity

        Transition Time In Source Status Execution Times Last Executer Last Execution Date
        Open Open Patch Available Patch Available
        5m 11s 1 Prasad Mujumdar 22/May/13 07:42
        Patch Available Patch Available Resolved Resolved
        93d 21h 52m 1 Navis 24/Aug/13 05:34
        Resolved Resolved Closed Closed
        52d 18h 54m 1 Ashutosh Chauhan 16/Oct/13 00:28
        Hide
        Lefty Leverenz added a comment -

        The patch adds hive.server2.session.hook to HiveConf.java. It's documented in the wiki here:

        Show
        Lefty Leverenz added a comment - The patch adds hive.server2.session.hook to HiveConf.java. It's documented in the wiki here: Configuration Properties – HiveServer2 – hive.server2.session.hook
        Ashutosh Chauhan made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Hide
        Ashutosh Chauhan added a comment -

        This issue has been fixed and released as part of 0.12 release. If you find further issues, please create a new jira and link it to this one.

        Show
        Ashutosh Chauhan added a comment - This issue has been fixed and released as part of 0.12 release. If you find further issues, please create a new jira and link it to this one.
        Hide
        Hudson added a comment -

        FAILURE: Integrated in Hive-trunk-hadoop2 #378 (See https://builds.apache.org/job/Hive-trunk-hadoop2/378/)
        HIVE-4588 : Support session level hooks for HiveServer2 (Prasad Mujumdar via Navis) (navis: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1517107)

        • /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
        • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/Driver.java
        • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/hooks/HookUtils.java
        • /hive/trunk/service/src/java/org/apache/hive/service/cli/session/HiveSessionHook.java
        • /hive/trunk/service/src/java/org/apache/hive/service/cli/session/HiveSessionHookContext.java
        • /hive/trunk/service/src/java/org/apache/hive/service/cli/session/HiveSessionHookContextImpl.java
        • /hive/trunk/service/src/java/org/apache/hive/service/cli/session/SessionManager.java
        • /hive/trunk/service/src/test/org/apache/hive/service/cli/session
        • /hive/trunk/service/src/test/org/apache/hive/service/cli/session/TestSessionHooks.java
        Show
        Hudson added a comment - FAILURE: Integrated in Hive-trunk-hadoop2 #378 (See https://builds.apache.org/job/Hive-trunk-hadoop2/378/ ) HIVE-4588 : Support session level hooks for HiveServer2 (Prasad Mujumdar via Navis) (navis: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1517107 ) /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/Driver.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/hooks/HookUtils.java /hive/trunk/service/src/java/org/apache/hive/service/cli/session/HiveSessionHook.java /hive/trunk/service/src/java/org/apache/hive/service/cli/session/HiveSessionHookContext.java /hive/trunk/service/src/java/org/apache/hive/service/cli/session/HiveSessionHookContextImpl.java /hive/trunk/service/src/java/org/apache/hive/service/cli/session/SessionManager.java /hive/trunk/service/src/test/org/apache/hive/service/cli/session /hive/trunk/service/src/test/org/apache/hive/service/cli/session/TestSessionHooks.java
        Hide
        Hudson added a comment -

        FAILURE: Integrated in Hive-trunk-h0.21 #2286 (See https://builds.apache.org/job/Hive-trunk-h0.21/2286/)
        HIVE-4588 : Support session level hooks for HiveServer2 (Prasad Mujumdar via Navis) (navis: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1517107)

        • /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
        • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/Driver.java
        • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/hooks/HookUtils.java
        • /hive/trunk/service/src/java/org/apache/hive/service/cli/session/HiveSessionHook.java
        • /hive/trunk/service/src/java/org/apache/hive/service/cli/session/HiveSessionHookContext.java
        • /hive/trunk/service/src/java/org/apache/hive/service/cli/session/HiveSessionHookContextImpl.java
        • /hive/trunk/service/src/java/org/apache/hive/service/cli/session/SessionManager.java
        • /hive/trunk/service/src/test/org/apache/hive/service/cli/session
        • /hive/trunk/service/src/test/org/apache/hive/service/cli/session/TestSessionHooks.java
        Show
        Hudson added a comment - FAILURE: Integrated in Hive-trunk-h0.21 #2286 (See https://builds.apache.org/job/Hive-trunk-h0.21/2286/ ) HIVE-4588 : Support session level hooks for HiveServer2 (Prasad Mujumdar via Navis) (navis: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1517107 ) /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/Driver.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/hooks/HookUtils.java /hive/trunk/service/src/java/org/apache/hive/service/cli/session/HiveSessionHook.java /hive/trunk/service/src/java/org/apache/hive/service/cli/session/HiveSessionHookContext.java /hive/trunk/service/src/java/org/apache/hive/service/cli/session/HiveSessionHookContextImpl.java /hive/trunk/service/src/java/org/apache/hive/service/cli/session/SessionManager.java /hive/trunk/service/src/test/org/apache/hive/service/cli/session /hive/trunk/service/src/test/org/apache/hive/service/cli/session/TestSessionHooks.java
        Hide
        Hudson added a comment -

        FAILURE: Integrated in Hive-trunk-hadoop1-ptest #136 (See https://builds.apache.org/job/Hive-trunk-hadoop1-ptest/136/)
        HIVE-4588 : Support session level hooks for HiveServer2 (Prasad Mujumdar via Navis) (navis: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1517107)

        • /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
        • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/Driver.java
        • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/hooks/HookUtils.java
        • /hive/trunk/service/src/java/org/apache/hive/service/cli/session/HiveSessionHook.java
        • /hive/trunk/service/src/java/org/apache/hive/service/cli/session/HiveSessionHookContext.java
        • /hive/trunk/service/src/java/org/apache/hive/service/cli/session/HiveSessionHookContextImpl.java
        • /hive/trunk/service/src/java/org/apache/hive/service/cli/session/SessionManager.java
        • /hive/trunk/service/src/test/org/apache/hive/service/cli/session
        • /hive/trunk/service/src/test/org/apache/hive/service/cli/session/TestSessionHooks.java
        Show
        Hudson added a comment - FAILURE: Integrated in Hive-trunk-hadoop1-ptest #136 (See https://builds.apache.org/job/Hive-trunk-hadoop1-ptest/136/ ) HIVE-4588 : Support session level hooks for HiveServer2 (Prasad Mujumdar via Navis) (navis: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1517107 ) /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/Driver.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/hooks/HookUtils.java /hive/trunk/service/src/java/org/apache/hive/service/cli/session/HiveSessionHook.java /hive/trunk/service/src/java/org/apache/hive/service/cli/session/HiveSessionHookContext.java /hive/trunk/service/src/java/org/apache/hive/service/cli/session/HiveSessionHookContextImpl.java /hive/trunk/service/src/java/org/apache/hive/service/cli/session/SessionManager.java /hive/trunk/service/src/test/org/apache/hive/service/cli/session /hive/trunk/service/src/test/org/apache/hive/service/cli/session/TestSessionHooks.java
        Hide
        Hudson added a comment -

        FAILURE: Integrated in Hive-trunk-hadoop2-ptest #68 (See https://builds.apache.org/job/Hive-trunk-hadoop2-ptest/68/)
        HIVE-4588 : Support session level hooks for HiveServer2 (Prasad Mujumdar via Navis) (navis: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1517107)

        • /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
        • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/Driver.java
        • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/hooks/HookUtils.java
        • /hive/trunk/service/src/java/org/apache/hive/service/cli/session/HiveSessionHook.java
        • /hive/trunk/service/src/java/org/apache/hive/service/cli/session/HiveSessionHookContext.java
        • /hive/trunk/service/src/java/org/apache/hive/service/cli/session/HiveSessionHookContextImpl.java
        • /hive/trunk/service/src/java/org/apache/hive/service/cli/session/SessionManager.java
        • /hive/trunk/service/src/test/org/apache/hive/service/cli/session
        • /hive/trunk/service/src/test/org/apache/hive/service/cli/session/TestSessionHooks.java
        Show
        Hudson added a comment - FAILURE: Integrated in Hive-trunk-hadoop2-ptest #68 (See https://builds.apache.org/job/Hive-trunk-hadoop2-ptest/68/ ) HIVE-4588 : Support session level hooks for HiveServer2 (Prasad Mujumdar via Navis) (navis: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1517107 ) /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/Driver.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/hooks/HookUtils.java /hive/trunk/service/src/java/org/apache/hive/service/cli/session/HiveSessionHook.java /hive/trunk/service/src/java/org/apache/hive/service/cli/session/HiveSessionHookContext.java /hive/trunk/service/src/java/org/apache/hive/service/cli/session/HiveSessionHookContextImpl.java /hive/trunk/service/src/java/org/apache/hive/service/cli/session/SessionManager.java /hive/trunk/service/src/test/org/apache/hive/service/cli/session /hive/trunk/service/src/test/org/apache/hive/service/cli/session/TestSessionHooks.java
        Hide
        Mikhail Antonov added a comment -

        Let me ask, if there're guys from Cloudera out here, any plans to backport it in current CDH 4.2.* / 4.3.* branches?

        Thanks!
        Mikhail

        Show
        Mikhail Antonov added a comment - Let me ask, if there're guys from Cloudera out here, any plans to backport it in current CDH 4.2.* / 4.3.* branches? Thanks! Mikhail
        Navis made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]
        Hide
        Navis added a comment -

        Committed to trunk. Thanks Prasad!

        Show
        Navis added a comment - Committed to trunk. Thanks Prasad!
        Hide
        Navis added a comment -

        +1

        Show
        Navis added a comment - +1
        Prasad Mujumdar made changes -
        Attachment HIVE-4588.4.patch [ 12599384 ]
        Prasad Mujumdar made changes -
        Attachment HIVE-4588.5.patch [ 12599469 ]
        Hide
        Navis added a comment -

        Ah, sorry. I've missed that. Left tiny comments.

        Show
        Navis added a comment - Ah, sorry. I've missed that. Left tiny comments.
        Prasad Mujumdar made changes -
        Attachment HIVE-4588.4.patch [ 12599384 ]
        Hide
        Prasad Mujumdar added a comment -

        Updated patch per review comments, rebased with latest on trunk

        Show
        Prasad Mujumdar added a comment - Updated patch per review comments, rebased with latest on trunk
        Hide
        Prasad Mujumdar added a comment -

        Navis Thanks for the feedback. There's already a RB request https://reviews.apache.org/r/11326/
        I have made most of the changes you suggested, details on the RB.

        Show
        Prasad Mujumdar added a comment - Navis Thanks for the feedback. There's already a RB request https://reviews.apache.org/r/11326/ I have made most of the changes you suggested, details on the RB.
        Hide
        Navis added a comment -

        Prasad Mujumdar plz make a RB or phabricator entry for a review.

        HiveSessionHook : extend org.apache.hadoop.hive.ql.hooks.Hook?
        HiveSessionHookContext.getSessionHandle() : would it be better to return SessionHandle rather than SessionHandle.toString()?
        SessionManager : how about extract method Driver.getHooks() into some utility class(JavaUtil?) and use that?

        Show
        Navis added a comment - Prasad Mujumdar plz make a RB or phabricator entry for a review. HiveSessionHook : extend org.apache.hadoop.hive.ql.hooks.Hook? HiveSessionHookContext.getSessionHandle() : would it be better to return SessionHandle rather than SessionHandle.toString()? SessionManager : how about extract method Driver.getHooks() into some utility class(JavaUtil?) and use that?
        Hide
        Prasad Mujumdar added a comment -

        Navis The patch is updated per your last suggestion. Would you like to take another look. Thanks!

        Show
        Prasad Mujumdar added a comment - Navis The patch is updated per your last suggestion. Would you like to take another look. Thanks!
        Hide
        Mikhail Antonov added a comment -

        Looks good! Any plans to backport on 0.11 anytime soon?

        Show
        Mikhail Antonov added a comment - Looks good! Any plans to backport on 0.11 anytime soon?
        Hide
        Hive QA added a comment -

        Overall: +1 all checks pass

        Here are the results of testing the latest attachment:
        https://issues.apache.org/jira/secure/attachment/12595363/HIVE-4588.3.patch

        SUCCESS: +1 2750 tests passed

        Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/275/testReport
        Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/275/console

        Messages:

        Executing org.apache.hive.ptest.execution.PrepPhase
        Executing org.apache.hive.ptest.execution.ExecutionPhase
        Executing org.apache.hive.ptest.execution.ReportingPhase
        

        This message is automatically generated.

        Show
        Hive QA added a comment - Overall : +1 all checks pass Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12595363/HIVE-4588.3.patch SUCCESS: +1 2750 tests passed Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/275/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/275/console Messages: Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase This message is automatically generated.
        Prasad Mujumdar made changes -
        Attachment HIVE-4588-2.patch [ 12589817 ]
        Prasad Mujumdar made changes -
        Attachment HIVE-4588.3.patch [ 12595363 ]
        Hide
        Prasad Mujumdar added a comment -

        The patch 2 was generated with incorrect version. Attached updated diff with rebase on latest trunk

        Show
        Prasad Mujumdar added a comment - The patch 2 was generated with incorrect version. Attached updated diff with rebase on latest trunk
        Prasad Mujumdar made changes -
        Attachment HIVE-4588-2.patch [ 12589817 ]
        Hide
        Prasad Mujumdar added a comment -

        Updated the patch with addional comments for new classes/interfaces

        Show
        Prasad Mujumdar added a comment - Updated the patch with addional comments for new classes/interfaces
        Hide
        Navis added a comment -

        Could you add some comments to newly added classes/interfaces? Thanks.

        Show
        Navis added a comment - Could you add some comments to newly added classes/interfaces? Thanks.
        Hide
        Shreepadma Venugopalan added a comment -

        +1 (non-binding), LGTM.

        Show
        Shreepadma Venugopalan added a comment - +1 (non-binding), LGTM.
        Prasad Mujumdar made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Hide
        Prasad Mujumdar added a comment -
        Show
        Prasad Mujumdar added a comment - Review request on https://reviews.apache.org/r/11326/
        Prasad Mujumdar made changes -
        Field Original Value New Value
        Attachment HIVE-4588-1.patch [ 12584254 ]
        Prasad Mujumdar created issue -

          People

          • Assignee:
            Prasad Mujumdar
            Reporter:
            Prasad Mujumdar
          • Votes:
            0 Vote for this issue
            Watchers:
            9 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development