HBase
  1. HBase
  2. HBASE-2811

Offer a fully integrated Hadoop+HBase+ZooKeeper solution

    Details

    • Type: New Feature New Feature
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      As dicussed at the Hadoop Summit and HUG11, we could improve usability a lot simply by shipping with all the required components and manage them all. I see two way of doing this:

      1. We keep all components separated and start/stop them in the right order. It would be much easier for us to develop but it would be much less friendly to use than the next solution.
      2. Integrate the DNs and TTs with the RSs, the NN, the JT and a ZK quorum member with the master. I'm not sure if it's easily doable but that's what we do for our unit tests so it must be possible.

      Then we need to make sure that we have options to disable that management like we currently do for ZK.

      This would also give us the chance to configure HDFS with the right amount of xcievers out of the box, among other settings.

        Activity

        Hide
        stack added a comment -

        Moving out of 0.92.0. Pull it back in if you think different.

        Show
        stack added a comment - Moving out of 0.92.0. Pull it back in if you think different.
        Hide
        Jonathan Gray added a comment -

        I'd really like to see #1 happen for 0.90. Someone did take ownership of this at some point, need to dig through my notes

        Show
        Jonathan Gray added a comment - I'd really like to see #1 happen for 0.90. Someone did take ownership of this at some point, need to dig through my notes
        Hide
        Jean-Daniel Cryans added a comment -

        Let's punt to 0.92.0

        Show
        Jean-Daniel Cryans added a comment - Let's punt to 0.92.0
        Hide
        Jonathan Gray added a comment -

        Sure, I do think it's an interesting idea. Not trying to just shoot it down, trying to think if there are benefits out there over approach #1.

        Show
        Jonathan Gray added a comment - Sure, I do think it's an interesting idea. Not trying to just shoot it down, trying to think if there are benefits out there over approach #1.
        Hide
        Jean-Daniel Cryans added a comment -

        About #2, it's just a proposal to counter the "too many components" argument.

        Show
        Jean-Daniel Cryans added a comment - About #2, it's just a proposal to counter the "too many components" argument.
        Hide
        Jonathan Gray added a comment -

        We should definitely do #1. +1 on configurable like ZK and also pre-configured HDFS.

        I think #2 could be interesting but I'm not sure in which contexts it makes sense. Single node, yeah. But if you have a cluster it seems like you benefit from multiple JVMs. Is it really that much more simple to launch a cluster of one process per server vs two or three?

        Are there some benefits I'm not thinking about?

        Show
        Jonathan Gray added a comment - We should definitely do #1. +1 on configurable like ZK and also pre-configured HDFS. I think #2 could be interesting but I'm not sure in which contexts it makes sense. Single node, yeah. But if you have a cluster it seems like you benefit from multiple JVMs. Is it really that much more simple to launch a cluster of one process per server vs two or three? Are there some benefits I'm not thinking about?

          People

          • Assignee:
            Unassigned
            Reporter:
            Jean-Daniel Cryans
          • Votes:
            1 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:

              Development