Uploaded image for project: 'Livy'
  1. Livy
  2. LIVY-685

Running Livy on Openshift , with standalone Spark cluster

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 0.6.0
    • Fix Version/s: None
    • Component/s: Batch, Server
    • Environment:
      Openshift 3.11 on-premise, Spark 2.3.1 - cluster mode (standalone)

      Description

      Trying to submit a Spark batch job using Apache Livy. I am running my Spark cluster and Livy service on Openshift. the Spark cluster is in standalone cluster mode, Livy runs spark in cluster deploy mode. I've configured Livy to the Spark cluster master service:

      livy.spark.master = "spark://spark.spark.svc:7077

      when i try to run a scala spark job, using the POST /batches method and get the exception:

      Unable to connect to server spark://spark.spark.svc:7077

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              matarp13 Matar Peles
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated: