Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-18533

Add option to use InProcessLauncher to submit spark jobs

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 4.0.0-alpha-1
    • Spark
    • None

    Description

      See discussion in HIVE-16484 for details.

      I think this will help with reducing the amount of time it takes to open a HoS session + debuggability (no need launch a separate process to run a Spark app).

      Attachments

        1. HIVE-18533.1.patch
          40 kB
          Sahil Takiar
        2. HIVE-18533.2.patch
          40 kB
          Sahil Takiar
        3. HIVE-18533.3.patch
          46 kB
          Sahil Takiar
        4. HIVE-18533.4.patch
          47 kB
          Sahil Takiar
        5. HIVE-18533.5.patch
          52 kB
          Sahil Takiar
        6. HIVE-18533.6.patch
          54 kB
          Sahil Takiar
        7. HIVE-18533.7.patch
          56 kB
          Sahil Takiar
        8. HIVE-18533.8.patch
          56 kB
          Sahil Takiar
        9. HIVE-18533.9.patch
          56 kB
          Sahil Takiar
        10. HIVE-18533.91.patch
          55 kB
          Sahil Takiar
        11. HIVE-18533.94.patch
          55 kB
          Sahil Takiar
        12. HIVE-18533.95.patch
          55 kB
          Sahil Takiar
        13. HIVE-18533.96.patch
          55 kB
          Sahil Takiar
        14. HIVE-18533.97.patch
          54 kB
          Sahil Takiar
        15. HIVE-18533.98.patch
          54 kB
          Sahil Takiar
        16. HIVE-18831.93.patch
          43 kB
          Sahil Takiar

        Issue Links

          depends upon

          Task - A task that needs to be done. HIVE-18436 Upgrade to Spark 2.3.0

          • Major - Major loss of function.
          • Closed
          is related to

          Bug - A problem which impairs or prevents the functions of the product. SPARK-23785 LauncherBackend doesn't check state of connection before setting state

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. SPARK-23658 InProcessAppHandle uses the wrong class in getLogger

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. HIVE-19797 Enable Spark InProcessLauncher on secure clusters

          • Major - Major loss of function.
          • Open

          Improvement - An improvement or enhancement to an existing feature or task. SPARK-24243 Expose exceptions from InProcessAppHandle

          • Major - Major loss of function.
          • Resolved
          relates to

          Bug - A problem which impairs or prevents the functions of the product. HIVE-17718 Hive on Spark Debugging Improvements

          • Major - Major loss of function.
          • Open

          Bug - A problem which impairs or prevents the functions of the product. HIVE-16484 Investigate SparkLauncher for HoS as alternative to bin/spark-submit

          • Major - Major loss of function.
          • Resolved
          links to

          Web Link RB

          Activity

            People

              stakiar Sahil Takiar
              stakiar Sahil Takiar
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: