Hive
  1. Hive
  2. HIVE-4588

Support session level hooks for HiveServer2

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.11.0
    • Fix Version/s: 0.12.0
    • Component/s: HiveServer2
    • Labels:
      None

      Description

      Support session level hooks for HiveSrver2. The configured hooks will get executed at beginning of each new session.
      This is useful for auditing connections, possibly tuning the session level properties etc.

      1. HIVE-4588-1.patch
        10 kB
        Prasad Mujumdar
      2. HIVE-4588.5.patch
        18 kB
        Prasad Mujumdar
      3. HIVE-4588.3.patch
        11 kB
        Prasad Mujumdar

        Activity

        Hide
        Prasad Mujumdar added a comment -
        Show
        Prasad Mujumdar added a comment - Review request on https://reviews.apache.org/r/11326/
        Hide
        Shreepadma Venugopalan added a comment -

        +1 (non-binding), LGTM.

        Show
        Shreepadma Venugopalan added a comment - +1 (non-binding), LGTM.
        Hide
        Navis added a comment -

        Could you add some comments to newly added classes/interfaces? Thanks.

        Show
        Navis added a comment - Could you add some comments to newly added classes/interfaces? Thanks.
        Hide
        Prasad Mujumdar added a comment -

        Updated the patch with addional comments for new classes/interfaces

        Show
        Prasad Mujumdar added a comment - Updated the patch with addional comments for new classes/interfaces
        Hide
        Prasad Mujumdar added a comment -

        The patch 2 was generated with incorrect version. Attached updated diff with rebase on latest trunk

        Show
        Prasad Mujumdar added a comment - The patch 2 was generated with incorrect version. Attached updated diff with rebase on latest trunk
        Hide
        Hive QA added a comment -

        Overall: +1 all checks pass

        Here are the results of testing the latest attachment:
        https://issues.apache.org/jira/secure/attachment/12595363/HIVE-4588.3.patch

        SUCCESS: +1 2750 tests passed

        Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/275/testReport
        Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/275/console

        Messages:

        Executing org.apache.hive.ptest.execution.PrepPhase
        Executing org.apache.hive.ptest.execution.ExecutionPhase
        Executing org.apache.hive.ptest.execution.ReportingPhase
        

        This message is automatically generated.

        Show
        Hive QA added a comment - Overall : +1 all checks pass Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12595363/HIVE-4588.3.patch SUCCESS: +1 2750 tests passed Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/275/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/275/console Messages: Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase This message is automatically generated.
        Hide
        Mikhail Antonov added a comment -

        Looks good! Any plans to backport on 0.11 anytime soon?

        Show
        Mikhail Antonov added a comment - Looks good! Any plans to backport on 0.11 anytime soon?
        Hide
        Prasad Mujumdar added a comment -

        Navis The patch is updated per your last suggestion. Would you like to take another look. Thanks!

        Show
        Prasad Mujumdar added a comment - Navis The patch is updated per your last suggestion. Would you like to take another look. Thanks!
        Hide
        Navis added a comment -

        Prasad Mujumdar plz make a RB or phabricator entry for a review.

        HiveSessionHook : extend org.apache.hadoop.hive.ql.hooks.Hook?
        HiveSessionHookContext.getSessionHandle() : would it be better to return SessionHandle rather than SessionHandle.toString()?
        SessionManager : how about extract method Driver.getHooks() into some utility class(JavaUtil?) and use that?

        Show
        Navis added a comment - Prasad Mujumdar plz make a RB or phabricator entry for a review. HiveSessionHook : extend org.apache.hadoop.hive.ql.hooks.Hook? HiveSessionHookContext.getSessionHandle() : would it be better to return SessionHandle rather than SessionHandle.toString()? SessionManager : how about extract method Driver.getHooks() into some utility class(JavaUtil?) and use that?
        Hide
        Prasad Mujumdar added a comment -

        Navis Thanks for the feedback. There's already a RB request https://reviews.apache.org/r/11326/
        I have made most of the changes you suggested, details on the RB.

        Show
        Prasad Mujumdar added a comment - Navis Thanks for the feedback. There's already a RB request https://reviews.apache.org/r/11326/ I have made most of the changes you suggested, details on the RB.
        Hide
        Prasad Mujumdar added a comment -

        Updated patch per review comments, rebased with latest on trunk

        Show
        Prasad Mujumdar added a comment - Updated patch per review comments, rebased with latest on trunk
        Hide
        Navis added a comment -

        Ah, sorry. I've missed that. Left tiny comments.

        Show
        Navis added a comment - Ah, sorry. I've missed that. Left tiny comments.
        Hide
        Navis added a comment -

        +1

        Show
        Navis added a comment - +1
        Hide
        Navis added a comment -

        Committed to trunk. Thanks Prasad!

        Show
        Navis added a comment - Committed to trunk. Thanks Prasad!
        Hide
        Mikhail Antonov added a comment -

        Let me ask, if there're guys from Cloudera out here, any plans to backport it in current CDH 4.2.* / 4.3.* branches?

        Thanks!
        Mikhail

        Show
        Mikhail Antonov added a comment - Let me ask, if there're guys from Cloudera out here, any plans to backport it in current CDH 4.2.* / 4.3.* branches? Thanks! Mikhail
        Hide
        Hudson added a comment -

        FAILURE: Integrated in Hive-trunk-hadoop2-ptest #68 (See https://builds.apache.org/job/Hive-trunk-hadoop2-ptest/68/)
        HIVE-4588 : Support session level hooks for HiveServer2 (Prasad Mujumdar via Navis) (navis: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1517107)

        • /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
        • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/Driver.java
        • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/hooks/HookUtils.java
        • /hive/trunk/service/src/java/org/apache/hive/service/cli/session/HiveSessionHook.java
        • /hive/trunk/service/src/java/org/apache/hive/service/cli/session/HiveSessionHookContext.java
        • /hive/trunk/service/src/java/org/apache/hive/service/cli/session/HiveSessionHookContextImpl.java
        • /hive/trunk/service/src/java/org/apache/hive/service/cli/session/SessionManager.java
        • /hive/trunk/service/src/test/org/apache/hive/service/cli/session
        • /hive/trunk/service/src/test/org/apache/hive/service/cli/session/TestSessionHooks.java
        Show
        Hudson added a comment - FAILURE: Integrated in Hive-trunk-hadoop2-ptest #68 (See https://builds.apache.org/job/Hive-trunk-hadoop2-ptest/68/ ) HIVE-4588 : Support session level hooks for HiveServer2 (Prasad Mujumdar via Navis) (navis: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1517107 ) /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/Driver.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/hooks/HookUtils.java /hive/trunk/service/src/java/org/apache/hive/service/cli/session/HiveSessionHook.java /hive/trunk/service/src/java/org/apache/hive/service/cli/session/HiveSessionHookContext.java /hive/trunk/service/src/java/org/apache/hive/service/cli/session/HiveSessionHookContextImpl.java /hive/trunk/service/src/java/org/apache/hive/service/cli/session/SessionManager.java /hive/trunk/service/src/test/org/apache/hive/service/cli/session /hive/trunk/service/src/test/org/apache/hive/service/cli/session/TestSessionHooks.java
        Hide
        Hudson added a comment -

        FAILURE: Integrated in Hive-trunk-hadoop1-ptest #136 (See https://builds.apache.org/job/Hive-trunk-hadoop1-ptest/136/)
        HIVE-4588 : Support session level hooks for HiveServer2 (Prasad Mujumdar via Navis) (navis: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1517107)

        • /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
        • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/Driver.java
        • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/hooks/HookUtils.java
        • /hive/trunk/service/src/java/org/apache/hive/service/cli/session/HiveSessionHook.java
        • /hive/trunk/service/src/java/org/apache/hive/service/cli/session/HiveSessionHookContext.java
        • /hive/trunk/service/src/java/org/apache/hive/service/cli/session/HiveSessionHookContextImpl.java
        • /hive/trunk/service/src/java/org/apache/hive/service/cli/session/SessionManager.java
        • /hive/trunk/service/src/test/org/apache/hive/service/cli/session
        • /hive/trunk/service/src/test/org/apache/hive/service/cli/session/TestSessionHooks.java
        Show
        Hudson added a comment - FAILURE: Integrated in Hive-trunk-hadoop1-ptest #136 (See https://builds.apache.org/job/Hive-trunk-hadoop1-ptest/136/ ) HIVE-4588 : Support session level hooks for HiveServer2 (Prasad Mujumdar via Navis) (navis: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1517107 ) /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/Driver.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/hooks/HookUtils.java /hive/trunk/service/src/java/org/apache/hive/service/cli/session/HiveSessionHook.java /hive/trunk/service/src/java/org/apache/hive/service/cli/session/HiveSessionHookContext.java /hive/trunk/service/src/java/org/apache/hive/service/cli/session/HiveSessionHookContextImpl.java /hive/trunk/service/src/java/org/apache/hive/service/cli/session/SessionManager.java /hive/trunk/service/src/test/org/apache/hive/service/cli/session /hive/trunk/service/src/test/org/apache/hive/service/cli/session/TestSessionHooks.java
        Hide
        Hudson added a comment -

        FAILURE: Integrated in Hive-trunk-h0.21 #2286 (See https://builds.apache.org/job/Hive-trunk-h0.21/2286/)
        HIVE-4588 : Support session level hooks for HiveServer2 (Prasad Mujumdar via Navis) (navis: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1517107)

        • /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
        • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/Driver.java
        • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/hooks/HookUtils.java
        • /hive/trunk/service/src/java/org/apache/hive/service/cli/session/HiveSessionHook.java
        • /hive/trunk/service/src/java/org/apache/hive/service/cli/session/HiveSessionHookContext.java
        • /hive/trunk/service/src/java/org/apache/hive/service/cli/session/HiveSessionHookContextImpl.java
        • /hive/trunk/service/src/java/org/apache/hive/service/cli/session/SessionManager.java
        • /hive/trunk/service/src/test/org/apache/hive/service/cli/session
        • /hive/trunk/service/src/test/org/apache/hive/service/cli/session/TestSessionHooks.java
        Show
        Hudson added a comment - FAILURE: Integrated in Hive-trunk-h0.21 #2286 (See https://builds.apache.org/job/Hive-trunk-h0.21/2286/ ) HIVE-4588 : Support session level hooks for HiveServer2 (Prasad Mujumdar via Navis) (navis: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1517107 ) /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/Driver.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/hooks/HookUtils.java /hive/trunk/service/src/java/org/apache/hive/service/cli/session/HiveSessionHook.java /hive/trunk/service/src/java/org/apache/hive/service/cli/session/HiveSessionHookContext.java /hive/trunk/service/src/java/org/apache/hive/service/cli/session/HiveSessionHookContextImpl.java /hive/trunk/service/src/java/org/apache/hive/service/cli/session/SessionManager.java /hive/trunk/service/src/test/org/apache/hive/service/cli/session /hive/trunk/service/src/test/org/apache/hive/service/cli/session/TestSessionHooks.java
        Hide
        Hudson added a comment -

        FAILURE: Integrated in Hive-trunk-hadoop2 #378 (See https://builds.apache.org/job/Hive-trunk-hadoop2/378/)
        HIVE-4588 : Support session level hooks for HiveServer2 (Prasad Mujumdar via Navis) (navis: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1517107)

        • /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
        • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/Driver.java
        • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/hooks/HookUtils.java
        • /hive/trunk/service/src/java/org/apache/hive/service/cli/session/HiveSessionHook.java
        • /hive/trunk/service/src/java/org/apache/hive/service/cli/session/HiveSessionHookContext.java
        • /hive/trunk/service/src/java/org/apache/hive/service/cli/session/HiveSessionHookContextImpl.java
        • /hive/trunk/service/src/java/org/apache/hive/service/cli/session/SessionManager.java
        • /hive/trunk/service/src/test/org/apache/hive/service/cli/session
        • /hive/trunk/service/src/test/org/apache/hive/service/cli/session/TestSessionHooks.java
        Show
        Hudson added a comment - FAILURE: Integrated in Hive-trunk-hadoop2 #378 (See https://builds.apache.org/job/Hive-trunk-hadoop2/378/ ) HIVE-4588 : Support session level hooks for HiveServer2 (Prasad Mujumdar via Navis) (navis: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1517107 ) /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/Driver.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/hooks/HookUtils.java /hive/trunk/service/src/java/org/apache/hive/service/cli/session/HiveSessionHook.java /hive/trunk/service/src/java/org/apache/hive/service/cli/session/HiveSessionHookContext.java /hive/trunk/service/src/java/org/apache/hive/service/cli/session/HiveSessionHookContextImpl.java /hive/trunk/service/src/java/org/apache/hive/service/cli/session/SessionManager.java /hive/trunk/service/src/test/org/apache/hive/service/cli/session /hive/trunk/service/src/test/org/apache/hive/service/cli/session/TestSessionHooks.java
        Hide
        Ashutosh Chauhan added a comment -

        This issue has been fixed and released as part of 0.12 release. If you find further issues, please create a new jira and link it to this one.

        Show
        Ashutosh Chauhan added a comment - This issue has been fixed and released as part of 0.12 release. If you find further issues, please create a new jira and link it to this one.
        Hide
        Lefty Leverenz added a comment -

        The patch adds hive.server2.session.hook to HiveConf.java. It's documented in the wiki here:

        Show
        Lefty Leverenz added a comment - The patch adds hive.server2.session.hook to HiveConf.java. It's documented in the wiki here: Configuration Properties – HiveServer2 – hive.server2.session.hook

          People

          • Assignee:
            Prasad Mujumdar
            Reporter:
            Prasad Mujumdar
          • Votes:
            0 Vote for this issue
            Watchers:
            9 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development