Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-17718 Hive on Spark Debugging Improvements
  3. HIVE-14162

Allow disabling of long running job on Hive On Spark On YARN

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 4.0.0-alpha-1
    • Spark
    • None

    Description

      Hive On Spark launches a long running process on the first query to handle all queries for that user session. In some use cases this is not desired, for instance when using Hue with large intervals between query executions.

      Could we have a property that would cause long running spark jobs to be terminated after each query execution and started again for the next one?

      Attachments

        1. HIVE-14162.9.patch
          28 kB
          Sahil Takiar
        2. HIVE-14162.8.patch
          28 kB
          Sahil Takiar
        3. HIVE-14162.7.patch
          24 kB
          Sahil Takiar
        4. HIVE-14162.6.patch
          24 kB
          Sahil Takiar
        5. HIVE-14162.5.patch
          24 kB
          Sahil Takiar
        6. HIVE-14162.4.patch
          24 kB
          Sahil Takiar
        7. HIVE-14162.3.patch
          18 kB
          Sahil Takiar
        8. HIVE-14162.2.patch
          14 kB
          Sahil Takiar
        9. HIVE-14162.1.patch
          5 kB
          Aihua Xu

        Issue Links

          is duplicated by

          New Feature - A new feature of the product, which has yet to be developed. HIVE-19878 Hive On Spark support AM shut down when there is no job submit

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Patch Available
          is related to

          Sub-task - The sub-task of the issue HIVE-20790 SparkSession should be able to close a session while it is being opened

          • Major - Major loss of function.
          • Open

          Bug - A problem which impairs or prevents the functions of the product. HIVE-20742 SparkSessionManagerImpl maintenance thread only cleans up session once

          • Major - Major loss of function.
          • Closed
          relates to

          Sub-task - The sub-task of the issue HIVE-20519 Remove 30m min value for hive.spark.session.timeout

          • Major - Major loss of function.
          • Patch Available

          Improvement - An improvement or enhancement to an existing feature or task. HIVE-14124 Spark app name should be in line with MapReduce app name when using Hive On Spark

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Open
          links to

          Web Link RB

          Activity

            People

              stakiar Sahil Takiar
              tscott@cloudera.com Thomas Scott
              Votes:
              2 Vote for this issue
              Watchers:
              12 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: