Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-18405

Add yarn-cluster mode support to Spark Thrift Server

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 1.6.0, 1.6.2, 2.0.0, 2.0.1
    • Fix Version/s: None
    • Component/s: Spark Core

      Description

      Currently, spark thrift server can run only on yarn-client mode.

      Can we add Yarn-Cluster mode support to spark thrift server?

      This will help us to launch multiple spark thrift server with different spark configurations and it really help in large distributed clusters where there is requirement to run complex sqls through STS. With client mode, there is a chance to overload local host with too much driver memory.

      Please let me know your thoughts.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              prabhu1984 Prabhu Kasinathan
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated: