HBase
  1. HBase
  2. HBASE-2841

Support psuedo-distributed ZooKeeper in HBase

    Details

    • Type: Improvement Improvement
    • Status: Open
    • Priority: Trivial Trivial
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: scripts, Zookeeper
    • Labels:
      None
    • Tags:
      noob

      Description

      HBASE-1511 added psuedo-distributed support for RegionServers and backup Masters, however there were some obstacles preventing the last piece: psuedo-distributed ZooKeeper clients (HQuorumPeer). No major obstacles, just enough that I don't have time to work on them immediately. This would be a great noob task to get familiar with how our ZooKeeper code works and make minor changes that will greatly help us test this product. Note that if you're feeling extra ambitious, you could work on adding startZkCluster(int), killZkPeer(int), and restartZkPeer(int) to HBaseTestingUtility.java for JUnit test goodness.

        Issue Links

          Activity

          Hide
          Nicolas Spiegelberg added a comment -

          For developer...

          psuedo-distributed ZK is more work than the others based upon the way that ZooKeeperWrapper & HQuorumPeer was written. Should still be doable, but noting some of the problems for consideration:

          1. Need to add conf item (hbase.zookeeper.myid) to override ID lookup in HQuorumPeer.writeMyID()
          2. These config params should be different per HQuorumPeer: hbase.zookeeper.property.clientPort + hbase.zookeeper.property.dataDir
          3. hbase.zookeeper.property.server.<id> should be given to hbase as <addr>:<peer_port>:<leader_port>:<client_port>
          a. We currently don't allow a custom peer_port + leader_port even though ZK supports it
          b. Need extra client_port for ZooKeeperWrapper.setQuorumServers()
          4. GNU Parsing code needs to be added to HQuorumPeer to support overriding config values on the fly. See HMaster.doMain() for help

          Also, make sure that the master rewrite code does not affect this change, if you finish first.

          Show
          Nicolas Spiegelberg added a comment - For developer... psuedo-distributed ZK is more work than the others based upon the way that ZooKeeperWrapper & HQuorumPeer was written. Should still be doable, but noting some of the problems for consideration: 1. Need to add conf item (hbase.zookeeper.myid) to override ID lookup in HQuorumPeer.writeMyID() 2. These config params should be different per HQuorumPeer: hbase.zookeeper.property.clientPort + hbase.zookeeper.property.dataDir 3. hbase.zookeeper.property.server.<id> should be given to hbase as <addr>:<peer_port>:<leader_port>:<client_port> a. We currently don't allow a custom peer_port + leader_port even though ZK supports it b. Need extra client_port for ZooKeeperWrapper.setQuorumServers() 4. GNU Parsing code needs to be added to HQuorumPeer to support overriding config values on the fly. See HMaster.doMain() for help Also, make sure that the master rewrite code does not affect this change, if you finish first.
          Hide
          Nicolas Spiegelberg added a comment -

          Code might need slight tweeks if master rewrite changes HQuorumPeer or ZooKeeperWrapper

          Show
          Nicolas Spiegelberg added a comment - Code might need slight tweeks if master rewrite changes HQuorumPeer or ZooKeeperWrapper
          Show
          Nicolas Spiegelberg added a comment - Also see: http://hadoop.apache.org/zookeeper/docs/r3.3.0/zookeeperStarted.html#sc_RunningReplicatedZooKeeper

            People

            • Assignee:
              Unassigned
              Reporter:
              Nicolas Spiegelberg
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:

                Development