Uploaded image for project: 'Livy'
  1. Livy
  2. LIVY-616

Livy Server discovery

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: In Progress
    • Major
    • Resolution: Unresolved
    • None
    • 0.9.0
    • Server
    • None

    Description

      Currently, there isn't a way to get Livy Server URI by the client without setting Livy address explicitly to livy.conf. A client should set "livy.server.host" variable and then get it via LivyConf. The same behavior if you want to use Livy with Zeppelin, we need to set "zeppelin.livy.url". It very inconvenient when we install Livy packages on couple nodes and don't know where exactly Livy Server will be started e.g. by Ambari or Cloudera Manager. Also, in this case, we need to have Livy configuration files on a node where we want to get Livy address.
      It will be very helpful if we will add Livy Server address to Zookeeper and expose API for clients to get Livy URL to use it in client code for REST calls.
      Livy already supports state saving in Zookeeper but I don't see that we store Livy server address somewhere. Before starting investigating and implementation I want to ask here about this.

      Please, correct me if I missed something.
      Any comments will be highly appreciated!

      Attachments

        1. image-2019-08-28-17-08-21-590.png
          73 kB
          Oleksandr Shevchenko
        2. Livy Server discovery.pdf
          175 kB
          Oleksandr Shevchenko

        Issue Links

          Activity

            People

              jerryshao Saisai Shao
              oshevchenko Oleksandr Shevchenko
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 10m
                  10m