Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-20271 Improve HoS query cancellation handling
  3. HIVE-16961

Hive on Spark leaks spark application in case user cancels query and closes session

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.1.0
    • Fix Version/s: 3.0.0
    • Component/s: Spark
    • Labels:
      None

      Description

      It's found that a Spark application is leaked when user cancels query and closes the session while Hive is waiting for remote driver to connect back. This is found for asynchronous query execution, but seemingly equally applicable for synchronous submission when session is abruptly closed. The leaked Spark application that runs Spark driver connects back to Hive successfully and run for ever (until HS2 restarts), but receives no job submission because the session is already closed. Ideally, Hive should rejects the connection from the driver so the driver will exist.

        Attachments

        1. HIVE-16961.patch
          2 kB
          Xuefu Zhang
        2. HIVE-16961.patch
          2 kB
          Xuefu Zhang

          Activity

            People

            • Assignee:
              xuefuz Xuefu Zhang
              Reporter:
              xuefuz Xuefu Zhang
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: