Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-4398

HS2 Resource leak: operation handles not cleaned when originating session is closed

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.10.0
    • Fix Version/s: 0.11.0
    • Component/s: HiveServer2
    • Labels:
      None

      Description


      In HS2 closing of sessions doesn't lead to closing of all the operation handles that the session had opened. This JIRA is meant to address this issue.

      1. HIVE-4398.branch-0.11.patch.3
        8 kB
        Ashish Vaidya
      2. HIVE-4398.branch-0.11.patch.2
        8 kB
        Ashish Vaidya
      3. HIVE-4398.branch-0.11.patch.1
        8 kB
        Ashish Vaidya
      4. HIVE-4398.branch-0.11.patch
        9 kB
        Ashish Vaidya

        Activity

        Hide
        hudson Hudson added a comment -

        Integrated in Hive-trunk-hadoop2 #179 (See https://builds.apache.org/job/Hive-trunk-hadoop2/179/)
        HIVE-4398 : HS2 Resource leak: operation handles not cleaned when originating session is closed (Ashish Vaidya via Ashutosh Chauhan) (Revision 1476647)

        Result = FAILURE
        hashutosh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1476647
        Files :

        • /hive/trunk/service/src/java/org/apache/hive/service/cli/session/HiveSessionImpl.java
        Show
        hudson Hudson added a comment - Integrated in Hive-trunk-hadoop2 #179 (See https://builds.apache.org/job/Hive-trunk-hadoop2/179/ ) HIVE-4398 : HS2 Resource leak: operation handles not cleaned when originating session is closed (Ashish Vaidya via Ashutosh Chauhan) (Revision 1476647) Result = FAILURE hashutosh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1476647 Files : /hive/trunk/service/src/java/org/apache/hive/service/cli/session/HiveSessionImpl.java
        Hide
        hudson Hudson added a comment -

        Integrated in Hive-trunk-h0.21 #2082 (See https://builds.apache.org/job/Hive-trunk-h0.21/2082/)
        HIVE-4398 : HS2 Resource leak: operation handles not cleaned when originating session is closed (Ashish Vaidya via Ashutosh Chauhan) (Revision 1476647)

        Result = FAILURE
        hashutosh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1476647
        Files :

        • /hive/trunk/service/src/java/org/apache/hive/service/cli/session/HiveSessionImpl.java
        Show
        hudson Hudson added a comment - Integrated in Hive-trunk-h0.21 #2082 (See https://builds.apache.org/job/Hive-trunk-h0.21/2082/ ) HIVE-4398 : HS2 Resource leak: operation handles not cleaned when originating session is closed (Ashish Vaidya via Ashutosh Chauhan) (Revision 1476647) Result = FAILURE hashutosh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1476647 Files : /hive/trunk/service/src/java/org/apache/hive/service/cli/session/HiveSessionImpl.java
        Hide
        ashutoshc Ashutosh Chauhan added a comment -

        Committed to trunk. Thanks, Ashish!

        Show
        ashutoshc Ashutosh Chauhan added a comment - Committed to trunk. Thanks, Ashish!
        Hide
        cwsteinbach Carl Steinbach added a comment -

        Thanks. +1

        Show
        cwsteinbach Carl Steinbach added a comment - Thanks. +1
        Hide
        vaidya Ashish Vaidya added a comment -


        Addresses latest comments.

        Show
        vaidya Ashish Vaidya added a comment - Addresses latest comments.
        Hide
        cwsteinbach Carl Steinbach added a comment -

        I left some more comments on RB. Thanks.

        Show
        cwsteinbach Carl Steinbach added a comment - I left some more comments on RB. Thanks.
        Hide
        thejas Thejas M Nair added a comment -

        Ashish Vaidya Can you please make the state of the jira 'patch available' using the 'submit patch' button ? That will make it more clear that it is ready for review and commit.

        Show
        thejas Thejas M Nair added a comment - Ashish Vaidya Can you please make the state of the jira 'patch available' using the 'submit patch' button ? That will make it more clear that it is ready for review and commit.
        Hide
        thejas Thejas M Nair added a comment -

        +1 (non binding)

        Show
        thejas Thejas M Nair added a comment - +1 (non binding)
        Hide
        vaidya Ashish Vaidya added a comment -


        New patch, addressing the comments, is up at <https://reviews.apache.org/r/10757/>.

        Show
        vaidya Ashish Vaidya added a comment - New patch, addressing the comments, is up at < https://reviews.apache.org/r/10757/ >.
        Hide
        cwsteinbach Carl Steinbach added a comment -

        I left some comments on RB.

        Show
        cwsteinbach Carl Steinbach added a comment - I left some comments on RB.
        Hide
        vaidya Ashish Vaidya added a comment -


        I think there's one big caveat with the new implementation – we shouldn't have anyone call OperationManager.closeOperation() directly; they should always go through a HiveSession object.

        Show
        vaidya Ashish Vaidya added a comment - I think there's one big caveat with the new implementation – we shouldn't have anyone call OperationManager.closeOperation() directly; they should always go through a HiveSession object.
        Hide
        vaidya Ashish Vaidya added a comment -


        Same as in the review board.

        Show
        vaidya Ashish Vaidya added a comment - Same as in the review board.
        Hide
        vaidya Ashish Vaidya added a comment -


        Created review request: <https://reviews.apache.org/r/10757/>

        Show
        vaidya Ashish Vaidya added a comment - Created review request: < https://reviews.apache.org/r/10757/ >
        Hide
        thejas Thejas M Nair added a comment -

        Ashish Vaidya Can you please upload this patch (or an updated patch) to reviewboard (https://reviews.apache.org) or phabricator ?
        Review comments -

        • operationToSession in OperationManager seems to be unnecessary. Operation.getParentSession() can be used to find the HiveSession associated with the operator.
        • I also think it would be better to store the list of Operations to be closed in the HiveSession itself (as a HashSet), instead of doing that in the OperationManager .
        Show
        thejas Thejas M Nair added a comment - Ashish Vaidya Can you please upload this patch (or an updated patch) to reviewboard ( https://reviews.apache.org ) or phabricator ? Review comments - operationToSession in OperationManager seems to be unnecessary. Operation.getParentSession() can be used to find the HiveSession associated with the operator. I also think it would be better to store the list of Operations to be closed in the HiveSession itself (as a HashSet), instead of doing that in the OperationManager .
        Hide
        vaidya Ashish Vaidya added a comment -


        Also works for trunk.

        Show
        vaidya Ashish Vaidya added a comment - Also works for trunk.
        Hide
        navis Navis added a comment -

        OP is not belong to a session. I also think it's strange but it seemed to be a by-design.

        Show
        navis Navis added a comment - OP is not belong to a session. I also think it's strange but it seemed to be a by-design.

          People

          • Assignee:
            vaidya Ashish Vaidya
            Reporter:
            vaidya Ashish Vaidya
          • Votes:
            0 Vote for this issue
            Watchers:
            9 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development