Hive
  1. Hive
  2. HIVE-4398

HS2 Resource leak: operation handles not cleaned when originating session is closed

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.10.0
    • Fix Version/s: 0.11.0
    • Component/s: HiveServer2
    • Labels:
      None

      Description


      In HS2 closing of sessions doesn't lead to closing of all the operation handles that the session had opened. This JIRA is meant to address this issue.

      1. HIVE-4398.branch-0.11.patch
        9 kB
        Ashish Vaidya
      2. HIVE-4398.branch-0.11.patch.1
        8 kB
        Ashish Vaidya
      3. HIVE-4398.branch-0.11.patch.2
        8 kB
        Ashish Vaidya
      4. HIVE-4398.branch-0.11.patch.3
        8 kB
        Ashish Vaidya

        Activity

        Hide
        Navis added a comment -

        OP is not belong to a session. I also think it's strange but it seemed to be a by-design.

        Show
        Navis added a comment - OP is not belong to a session. I also think it's strange but it seemed to be a by-design.
        Hide
        Ashish Vaidya added a comment -


        Also works for trunk.

        Show
        Ashish Vaidya added a comment - Also works for trunk.
        Hide
        Thejas M Nair added a comment -

        Ashish Vaidya Can you please upload this patch (or an updated patch) to reviewboard (https://reviews.apache.org) or phabricator ?
        Review comments -

        • operationToSession in OperationManager seems to be unnecessary. Operation.getParentSession() can be used to find the HiveSession associated with the operator.
        • I also think it would be better to store the list of Operations to be closed in the HiveSession itself (as a HashSet), instead of doing that in the OperationManager .
        Show
        Thejas M Nair added a comment - Ashish Vaidya Can you please upload this patch (or an updated patch) to reviewboard ( https://reviews.apache.org ) or phabricator ? Review comments - operationToSession in OperationManager seems to be unnecessary. Operation.getParentSession() can be used to find the HiveSession associated with the operator. I also think it would be better to store the list of Operations to be closed in the HiveSession itself (as a HashSet), instead of doing that in the OperationManager .
        Hide
        Ashish Vaidya added a comment -


        Created review request: <https://reviews.apache.org/r/10757/>

        Show
        Ashish Vaidya added a comment - Created review request: < https://reviews.apache.org/r/10757/ >
        Hide
        Ashish Vaidya added a comment -


        Same as in the review board.

        Show
        Ashish Vaidya added a comment - Same as in the review board.
        Hide
        Ashish Vaidya added a comment -


        I think there's one big caveat with the new implementation – we shouldn't have anyone call OperationManager.closeOperation() directly; they should always go through a HiveSession object.

        Show
        Ashish Vaidya added a comment - I think there's one big caveat with the new implementation – we shouldn't have anyone call OperationManager.closeOperation() directly; they should always go through a HiveSession object.
        Hide
        Carl Steinbach added a comment -

        I left some comments on RB.

        Show
        Carl Steinbach added a comment - I left some comments on RB.
        Hide
        Ashish Vaidya added a comment -


        New patch, addressing the comments, is up at <https://reviews.apache.org/r/10757/>.

        Show
        Ashish Vaidya added a comment - New patch, addressing the comments, is up at < https://reviews.apache.org/r/10757/ >.
        Hide
        Thejas M Nair added a comment -

        +1 (non binding)

        Show
        Thejas M Nair added a comment - +1 (non binding)
        Hide
        Thejas M Nair added a comment -

        Ashish Vaidya Can you please make the state of the jira 'patch available' using the 'submit patch' button ? That will make it more clear that it is ready for review and commit.

        Show
        Thejas M Nair added a comment - Ashish Vaidya Can you please make the state of the jira 'patch available' using the 'submit patch' button ? That will make it more clear that it is ready for review and commit.
        Hide
        Carl Steinbach added a comment -

        I left some more comments on RB. Thanks.

        Show
        Carl Steinbach added a comment - I left some more comments on RB. Thanks.
        Hide
        Ashish Vaidya added a comment -


        Addresses latest comments.

        Show
        Ashish Vaidya added a comment - Addresses latest comments.
        Hide
        Carl Steinbach added a comment -

        Thanks. +1

        Show
        Carl Steinbach added a comment - Thanks. +1
        Hide
        Ashutosh Chauhan added a comment -

        Committed to trunk. Thanks, Ashish!

        Show
        Ashutosh Chauhan added a comment - Committed to trunk. Thanks, Ashish!
        Hide
        Hudson added a comment -

        Integrated in Hive-trunk-h0.21 #2082 (See https://builds.apache.org/job/Hive-trunk-h0.21/2082/)
        HIVE-4398 : HS2 Resource leak: operation handles not cleaned when originating session is closed (Ashish Vaidya via Ashutosh Chauhan) (Revision 1476647)

        Result = FAILURE
        hashutosh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1476647
        Files :

        • /hive/trunk/service/src/java/org/apache/hive/service/cli/session/HiveSessionImpl.java
        Show
        Hudson added a comment - Integrated in Hive-trunk-h0.21 #2082 (See https://builds.apache.org/job/Hive-trunk-h0.21/2082/ ) HIVE-4398 : HS2 Resource leak: operation handles not cleaned when originating session is closed (Ashish Vaidya via Ashutosh Chauhan) (Revision 1476647) Result = FAILURE hashutosh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1476647 Files : /hive/trunk/service/src/java/org/apache/hive/service/cli/session/HiveSessionImpl.java
        Hide
        Hudson added a comment -

        Integrated in Hive-trunk-hadoop2 #179 (See https://builds.apache.org/job/Hive-trunk-hadoop2/179/)
        HIVE-4398 : HS2 Resource leak: operation handles not cleaned when originating session is closed (Ashish Vaidya via Ashutosh Chauhan) (Revision 1476647)

        Result = FAILURE
        hashutosh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1476647
        Files :

        • /hive/trunk/service/src/java/org/apache/hive/service/cli/session/HiveSessionImpl.java
        Show
        Hudson added a comment - Integrated in Hive-trunk-hadoop2 #179 (See https://builds.apache.org/job/Hive-trunk-hadoop2/179/ ) HIVE-4398 : HS2 Resource leak: operation handles not cleaned when originating session is closed (Ashish Vaidya via Ashutosh Chauhan) (Revision 1476647) Result = FAILURE hashutosh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1476647 Files : /hive/trunk/service/src/java/org/apache/hive/service/cli/session/HiveSessionImpl.java

          People

          • Assignee:
            Ashish Vaidya
            Reporter:
            Ashish Vaidya
          • Votes:
            0 Vote for this issue
            Watchers:
            10 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development