Apache Drill
  1. Apache Drill
  2. DRILL-58

Create a WEB UI war that can be deployed along with Drillbits

    Details

    • Type: Bug Bug
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: Future
    • Component/s: None
    • Labels:
      None

      Description

      A while ago Michael created a simple web UI to submit queries to Drill.
      It'd be nice if we could package it in war for and deploy it in Drillbits through jetty.
      Hive web ui (the way it's deployed) might be a good starting point.

        Activity

        Hide
        Michael Hausenblas added a comment -

        Beat me to it, David! Thanks for raising this issue and yes, as soon as I've got 57 covered I'll do this one.

        David: 'It'd be nice if we could package it in war for and deploy it in Drillbits through jetty. '

        In fact, it should be more straight-forward than that. Simple HTML5 app, no server dependencies.

        Show
        Michael Hausenblas added a comment - Beat me to it, David! Thanks for raising this issue and yes, as soon as I've got 57 covered I'll do this one. David: 'It'd be nice if we could package it in war for and deploy it in Drillbits through jetty. ' In fact, it should be more straight-forward than that. Simple HTML5 app, no server dependencies.
        Hide
        David Alves added a comment -

        Not sure how that'll work.
        I mean the point is to have drillbits expose a way to submit queries through a web ui right? It might even evolve to some kind of admin iface.
        Later on we could have add a REST iface to drill (which would allow a plain HTML 5 app) but right now it doesn't exist so you'll need a server component (jsp or the like) right?
        In any case even if there's no server side component the html still has to be served by some http server, I was suggesting embedded jetty and war packaging. What are your thoughts?

        Show
        David Alves added a comment - Not sure how that'll work. I mean the point is to have drillbits expose a way to submit queries through a web ui right? It might even evolve to some kind of admin iface. Later on we could have add a REST iface to drill (which would allow a plain HTML 5 app) but right now it doesn't exist so you'll need a server component (jsp or the like) right? In any case even if there's no server side component the html still has to be served by some http server, I was suggesting embedded jetty and war packaging. What are your thoughts?
        Hide
        Jacques Nadeau added a comment -

        My quick opinions on this:

        From experience, I want to avoid adding lots of interfaces directly to the Drillbit. For applications other than other Drillbits, I'd like to constrain that all to the UserAPI/RPC interface. The only other interface that I see being available is JMX for metrics. As such, my preference would be that what other non-query information that the web gui need be also sent through the Drillbit API. I think this will keep things simple and always ensure that cross version and cross language compatibility are managed. This also means that later managing the implementation of security is much simpler. Given that, if you need more functionality from the Drillbit it would be best to add additional request/response types and message types to the User API.

        As far as server infrastructures go, I've actually really liked some projects I've worked on recently where the server component only exposes a REST JSON API and static asset serving. I've used both Jersey and Resteasy in that way with much success. If someone wanted to do something else, I've actually found JSF+Primefaces reasonably competent albeit irritating and not very rest friendly (and slow/big).

        My other thought would be that if we want to have a server component, that we try to stick to servlet APIs for the functionality. Easy starting of Jetty is great if you don't already have infrastructure in place. However, productionizing many separate Jetty instances can be a real pain, especially if you have another web container that you utilize (like Weblogic or Glassfish). Since our requirements should be pretty sparse, it would be great if we didn't force people to use one container. (To me this just means that we separate out the WAR application from the running of it within embedded jetty or tomcat.)

        One other question here is whether the server component should even be in Java. I know a lot of people like Python & Django for these types of applications and if we commit to constraining interaction to the protobuf user api, the person that will actually spend a lot of time writing this component should feel comfortable using what makes the most sense given the requirements.

        Show
        Jacques Nadeau added a comment - My quick opinions on this: From experience, I want to avoid adding lots of interfaces directly to the Drillbit. For applications other than other Drillbits, I'd like to constrain that all to the UserAPI/RPC interface. The only other interface that I see being available is JMX for metrics. As such, my preference would be that what other non-query information that the web gui need be also sent through the Drillbit API. I think this will keep things simple and always ensure that cross version and cross language compatibility are managed. This also means that later managing the implementation of security is much simpler. Given that, if you need more functionality from the Drillbit it would be best to add additional request/response types and message types to the User API. As far as server infrastructures go, I've actually really liked some projects I've worked on recently where the server component only exposes a REST JSON API and static asset serving. I've used both Jersey and Resteasy in that way with much success. If someone wanted to do something else, I've actually found JSF+Primefaces reasonably competent albeit irritating and not very rest friendly (and slow/big). My other thought would be that if we want to have a server component, that we try to stick to servlet APIs for the functionality. Easy starting of Jetty is great if you don't already have infrastructure in place. However, productionizing many separate Jetty instances can be a real pain, especially if you have another web container that you utilize (like Weblogic or Glassfish). Since our requirements should be pretty sparse, it would be great if we didn't force people to use one container. (To me this just means that we separate out the WAR application from the running of it within embedded jetty or tomcat.) One other question here is whether the server component should even be in Java. I know a lot of people like Python & Django for these types of applications and if we commit to constraining interaction to the protobuf user api, the person that will actually spend a lot of time writing this component should feel comfortable using what makes the most sense given the requirements.
        Hide
        David Alves added a comment -

        +1 on most Jacques said, particularly on keeping it simple with servlets and using RPC as the sole comms mechanism (I just mentioned REST as one possibility done the line that would allow to have the app client side only, but REST might also make sense for other use cases, in any case it was not a suggestion that we try an go for it now).

        Just a few additional thoughs:

        I'm all for not making running jetty mandatory but giving the option to run a web ui embedded would ease a lot the deployment work for transient deployments and would provide a nice "aha" moment to whoever is trying drill for the first time.
        I've started making a drill service for apache whirr and I'm thinking that installing a completely different set of tools (as in ruby on rails for instance) will make the deployment times much steeper and the deployment harder to manage.

        The way I see it we could provide two ways (both explicitly configured): embedded in jetty and as a separate daemon on whatever container is preferred. In any case I'd prefer a jvm friendly container + framework.
        Also there's a lot of examples on this in the hadoop ecosystem.

        Show
        David Alves added a comment - +1 on most Jacques said, particularly on keeping it simple with servlets and using RPC as the sole comms mechanism (I just mentioned REST as one possibility done the line that would allow to have the app client side only, but REST might also make sense for other use cases, in any case it was not a suggestion that we try an go for it now). Just a few additional thoughs: I'm all for not making running jetty mandatory but giving the option to run a web ui embedded would ease a lot the deployment work for transient deployments and would provide a nice "aha" moment to whoever is trying drill for the first time. I've started making a drill service for apache whirr and I'm thinking that installing a completely different set of tools (as in ruby on rails for instance) will make the deployment times much steeper and the deployment harder to manage. The way I see it we could provide two ways (both explicitly configured): embedded in jetty and as a separate daemon on whatever container is preferred. In any case I'd prefer a jvm friendly container + framework. Also there's a lot of examples on this in the hadoop ecosystem.
        Hide
        Michael Hausenblas added a comment -

        Can we keep the interface and the UI separate, please? In the past 15y I've done Web apps that has turned out to be useful. So, for starters I will describe how the current app works and once we can agree on the interface (I'm strongly in favour for a REST interface for the job submission, BTW) we can discuss low-level details such as Jetty vs. Tomcat vs. whatever.

        To kick-off things: IMHO, the REST interface should be part of the foreman role of a Drillbit. This leads to the (at least for me) interesting question: how is the foreman determined, that is, how is a particular Drillbit assigned with this role?

        Show
        Michael Hausenblas added a comment - Can we keep the interface and the UI separate, please? In the past 15y I've done Web apps that has turned out to be useful. So, for starters I will describe how the current app works and once we can agree on the interface (I'm strongly in favour for a REST interface for the job submission, BTW) we can discuss low-level details such as Jetty vs. Tomcat vs. whatever. To kick-off things: IMHO, the REST interface should be part of the foreman role of a Drillbit. This leads to the (at least for me) interesting question: how is the foreman determined, that is, how is a particular Drillbit assigned with this role?
        Hide
        Ted Dunning added a comment -

        The way I see it we could provide two ways (both explicitly configured): embedded in jetty and as a separate daemon on whatever container is preferred. In any case I'd prefer a jvm friendly container + framework.
        Also there's a lot of examples on this in the hadoop ecosystem.

        The Hadoop examples are anti-patterns in my book.

        They are incredibly hard to integrate into other systems and often change form without changing substance (which breaks integration).

        Much better to do as Michael suggests.

        Show
        Ted Dunning added a comment - The way I see it we could provide two ways (both explicitly configured): embedded in jetty and as a separate daemon on whatever container is preferred. In any case I'd prefer a jvm friendly container + framework. Also there's a lot of examples on this in the hadoop ecosystem. The Hadoop examples are anti-patterns in my book. They are incredibly hard to integrate into other systems and often change form without changing substance (which breaks integration). Much better to do as Michael suggests.
        Hide
        Michael Hausenblas added a comment -

        Hari and I discussed this issue. We agreed on keeping this issue for the WebUI (for me) and Hari creates a new issue for the REST interface. These two issues will then be interlinked. In the first place, Hari will create a Google Docs-based design document for the REST interface covering query submission, status polling, data source configuration, result pagination, etc. and then start with the implementation. In parallel, I will port the old WebUI code over and make it work with the new REST interface.

        Show
        Michael Hausenblas added a comment - Hari and I discussed this issue. We agreed on keeping this issue for the WebUI (for me) and Hari creates a new issue for the REST interface. These two issues will then be interlinked. In the first place, Hari will create a Google Docs-based design document for the REST interface covering query submission, status polling, data source configuration, result pagination, etc. and then start with the implementation. In parallel, I will port the old WebUI code over and make it work with the new REST interface.
        Hide
        Michael Hausenblas added a comment -

        See DRILL-77 for the REST API dependencies of the Web UI.

        Show
        Michael Hausenblas added a comment - See DRILL-77 for the REST API dependencies of the Web UI.
        Hide
        Michael Hausenblas added a comment -

        Just sat in a session at LuceneRevolution [1] about Solr's new UI that follows the same design principles as we have, here.
        I shall follow up with Stefan Matheis, the developer who's taking care of the UI in Solr. I'm positive we can reuse stuff, at least experiences.

        [1] http://dublin.lucenerevolution.org/LSRDUB/#!/session/216806/

        Show
        Michael Hausenblas added a comment - Just sat in a session at LuceneRevolution [1] about Solr's new UI that follows the same design principles as we have, here. I shall follow up with Stefan Matheis, the developer who's taking care of the UI in Solr. I'm positive we can reuse stuff, at least experiences. [1] http://dublin.lucenerevolution.org/LSRDUB/#!/session/216806/

          People

          • Assignee:
            Michael Hausenblas
            Reporter:
            David Alves
          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:

              Development