Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-17718 Hive on Spark Debugging Improvements
  3. HIVE-14162

Allow disabling of long running job on Hive On Spark On YARN

Log workAgile BoardRank to TopRank to BottomBulk Copy AttachmentsBulk Move AttachmentsVotersWatch issueWatchersConvert to IssueMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 4.0.0-alpha-1
    • Spark
    • None

    Description

      Hive On Spark launches a long running process on the first query to handle all queries for that user session. In some use cases this is not desired, for instance when using Hue with large intervals between query executions.

      Could we have a property that would cause long running spark jobs to be terminated after each query execution and started again for the next one?

      Attachments

        1. HIVE-14162.9.patch
          28 kB
          Sahil Takiar
        2. HIVE-14162.8.patch
          28 kB
          Sahil Takiar
        3. HIVE-14162.7.patch
          24 kB
          Sahil Takiar
        4. HIVE-14162.6.patch
          24 kB
          Sahil Takiar
        5. HIVE-14162.5.patch
          24 kB
          Sahil Takiar
        6. HIVE-14162.4.patch
          24 kB
          Sahil Takiar
        7. HIVE-14162.3.patch
          18 kB
          Sahil Takiar
        8. HIVE-14162.2.patch
          14 kB
          Sahil Takiar
        9. HIVE-14162.1.patch
          5 kB
          Aihua Xu

        Issue Links

        is duplicated by

        New Feature - A new feature of the product, which has yet to be developed. HIVE-19878 Hive On Spark support AM shut down when there is no job submit

        • Minor - Minor loss of function, or other problem where easy workaround is present.
        • Patch Available
        is related to

        Sub-task - The sub-task of the issue HIVE-20790 SparkSession should be able to close a session while it is being opened

        • Major - Major loss of function.
        • Open

        Bug - A problem which impairs or prevents the functions of the product. HIVE-20742 SparkSessionManagerImpl maintenance thread only cleans up session once

        • Major - Major loss of function.
        • Closed
        relates to

        Sub-task - The sub-task of the issue HIVE-20519 Remove 30m min value for hive.spark.session.timeout

        • Major - Major loss of function.
        • Patch Available

        Improvement - An improvement or enhancement to an existing feature or task. HIVE-14124 Spark app name should be in line with MapReduce app name when using Hive On Spark

        • Minor - Minor loss of function, or other problem where easy workaround is present.
        • Open
        links to

        Web Link RB

        Activity
          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            stakiar Sahil Takiar Assign to me
            tscott@cloudera.com Thomas Scott
            Votes:
            2 Vote for this issue
            Watchers:
            12 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment