HBase
  1. HBase
  2. HBASE-2841

Support psuedo-distributed ZooKeeper in HBase

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Trivial Trivial
    • Resolution: Won't Fix
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: scripts, Zookeeper
    • Labels:
      None
    • Tags:
      noob

      Description

      HBASE-1511 added psuedo-distributed support for RegionServers and backup Masters, however there were some obstacles preventing the last piece: psuedo-distributed ZooKeeper clients (HQuorumPeer). No major obstacles, just enough that I don't have time to work on them immediately. This would be a great noob task to get familiar with how our ZooKeeper code works and make minor changes that will greatly help us test this product. Note that if you're feeling extra ambitious, you could work on adding startZkCluster(int), killZkPeer(int), and restartZkPeer(int) to HBaseTestingUtility.java for JUnit test goodness.

        Issue Links

          Activity

          Nicolas Spiegelberg created issue -
          Nicolas Spiegelberg made changes -
          Field Original Value New Value
          Link This issue relates to HBASE-1511 [ HBASE-1511 ]
          Hide
          Nicolas Spiegelberg added a comment -

          For developer...

          psuedo-distributed ZK is more work than the others based upon the way that ZooKeeperWrapper & HQuorumPeer was written. Should still be doable, but noting some of the problems for consideration:

          1. Need to add conf item (hbase.zookeeper.myid) to override ID lookup in HQuorumPeer.writeMyID()
          2. These config params should be different per HQuorumPeer: hbase.zookeeper.property.clientPort + hbase.zookeeper.property.dataDir
          3. hbase.zookeeper.property.server.<id> should be given to hbase as <addr>:<peer_port>:<leader_port>:<client_port>
          a. We currently don't allow a custom peer_port + leader_port even though ZK supports it
          b. Need extra client_port for ZooKeeperWrapper.setQuorumServers()
          4. GNU Parsing code needs to be added to HQuorumPeer to support overriding config values on the fly. See HMaster.doMain() for help

          Also, make sure that the master rewrite code does not affect this change, if you finish first.

          Show
          Nicolas Spiegelberg added a comment - For developer... psuedo-distributed ZK is more work than the others based upon the way that ZooKeeperWrapper & HQuorumPeer was written. Should still be doable, but noting some of the problems for consideration: 1. Need to add conf item (hbase.zookeeper.myid) to override ID lookup in HQuorumPeer.writeMyID() 2. These config params should be different per HQuorumPeer: hbase.zookeeper.property.clientPort + hbase.zookeeper.property.dataDir 3. hbase.zookeeper.property.server.<id> should be given to hbase as <addr>:<peer_port>:<leader_port>:<client_port> a. We currently don't allow a custom peer_port + leader_port even though ZK supports it b. Need extra client_port for ZooKeeperWrapper.setQuorumServers() 4. GNU Parsing code needs to be added to HQuorumPeer to support overriding config values on the fly. See HMaster.doMain() for help Also, make sure that the master rewrite code does not affect this change, if you finish first.
          Nicolas Spiegelberg made changes -
          Link This issue depends on HBASE-1816 [ HBASE-1816 ]
          Hide
          Nicolas Spiegelberg added a comment -

          Code might need slight tweeks if master rewrite changes HQuorumPeer or ZooKeeperWrapper

          Show
          Nicolas Spiegelberg added a comment - Code might need slight tweeks if master rewrite changes HQuorumPeer or ZooKeeperWrapper
          Show
          Nicolas Spiegelberg added a comment - Also see: http://hadoop.apache.org/zookeeper/docs/r3.3.0/zookeeperStarted.html#sc_RunningReplicatedZooKeeper
          Gavin made changes -
          Link This issue depends on HBASE-1816 [ HBASE-1816 ]
          Gavin made changes -
          Link This issue depends upon HBASE-1816 [ HBASE-1816 ]
          Hide
          Andrew Purtell added a comment -

          Stale issue. Reopen if still relevant.

          Lots of more current issues reworking our use of ZooKeeper, suggest moving relevant parts of this issue forward.

          Show
          Andrew Purtell added a comment - Stale issue. Reopen if still relevant. Lots of more current issues reworking our use of ZooKeeper, suggest moving relevant parts of this issue forward.
          Andrew Purtell made changes -
          Status Open [ 1 ] Resolved [ 5 ]
          Resolution Won't Fix [ 2 ]
          Transition Time In Source Status Execution Times Last Executer Last Execution Date
          Open Open Resolved Resolved
          1422d 21h 16m 1 Andrew Purtell 08/Jun/14 22:30

            People

            • Assignee:
              Unassigned
              Reporter:
              Nicolas Spiegelberg
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development