Hadoop Common
  1. Hadoop Common
  2. HADOOP-353

Run datanode (or other hadoop servers) inside tomcat

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Won't Fix
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      Barry Kaplan is running hadoop data nodes inside tomcat and encountering some issues.

      http://issues.apache.org/jira/browse/HADOOP-211#action_12419360

      I'm filing this bug to capture discussion about the pros and cons of such an approach. I'd be curious to know what others on the list (who know more about java/tomcat) think about this proposal.

        Issue Links

          Activity

          Hide
          Barry Kaplan added a comment -

          Eric thanks for putting this in... I will help out as much as I can.

          To explain my setup we essentially have a web farm running just tomcat, it makes it simple for deployment and upgrades. Essentially all of our servers are used for utility purposes (indexing, statistic calulations, etc...) or as web servers. Ever web server is also a data node. One of our utility servers (we are working on using a poll method) is picked to be the namenode.

          The beauty of this design for us, is we can deploy a server and install java,tomcat,war file and have a server that is 100% utilized in our environment. Hadoop's DFS allows us to take advantage of the disk space available across our environment with 0 configuration.

          Show
          Barry Kaplan added a comment - Eric thanks for putting this in... I will help out as much as I can. To explain my setup we essentially have a web farm running just tomcat, it makes it simple for deployment and upgrades. Essentially all of our servers are used for utility purposes (indexing, statistic calulations, etc...) or as web servers. Ever web server is also a data node. One of our utility servers (we are working on using a poll method) is picked to be the namenode. The beauty of this design for us, is we can deploy a server and install java,tomcat,war file and have a server that is 100% utilized in our environment. Hadoop's DFS allows us to take advantage of the disk space available across our environment with 0 configuration.
          Hide
          eric baldeschwieler added a comment -

          A core issue is that we use JETTY for real work already in the tasktrackers and are actively adding more functionality to the it on the datanodes too. How does that interact with tomcat?

          Show
          eric baldeschwieler added a comment - A core issue is that we use JETTY for real work already in the tasktrackers and are actively adding more functionality to the it on the datanodes too. How does that interact with tomcat?
          Hide
          Harsh J added a comment -

          Is this still a problem today? If not, we may close this out.

          Show
          Harsh J added a comment - Is this still a problem today? If not, we may close this out.
          Hide
          Allen Wittenauer added a comment -

          I'm closing this out as Won't Fix. At this point, I think there is zero interesting in making Hadoop run inside another app server.

          Show
          Allen Wittenauer added a comment - I'm closing this out as Won't Fix. At this point, I think there is zero interesting in making Hadoop run inside another app server.

            People

            • Assignee:
              Unassigned
              Reporter:
              eric baldeschwieler
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development