ZooKeeper
  1. ZooKeeper
  2. ZOOKEEPER-279

Allow specialization of quorum config parsing (e.g. variable expansion in zoo.cfg)

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 3.1.0
    • Fix Version/s: 3.1.1, 3.2.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      We would like to define certain parts of ZooKeeper's configuration using variables that get substituted. For example, we want the ZooKeeper quorum to be able to use a dataDir configured per user. In other words, something like:

      tickTime=2000
      dataDir=/tmp/zookeeper-$

      {user.name}

      clientPort=2181
      initLimit=5
      syncLimit=2
      server.1=zoo1:2888:3888
      server.2=zoo2:2888:3888
      server.3=zoo3:2888:3888

      I think Java already has a system for configuration that allows something like this using Properties?

      1. zookeeper-279.patch
        1 kB
        Jean-Daniel Cryans
      2. zookeeper-279.patch
        2 kB
        Patrick Hunt
      3. HBaseQPC.java
        3 kB
        Jean-Daniel Cryans

        Issue Links

          Activity

          Hide
          david babits added a comment -

          can somebody confirm whether this feature is available in zookeeper-3.4.5, I can't seem to see it implemented looking at source code in org.apache.zookeeper.server.quorum.QuorumPeerConfig

          Thanks.

          http://grepcode.com/file/repository.cloudera.com/content/repositories/releases/org.apache.zookeeper/zookeeper/3.4.5-cdh4.2.0/org/apache/zookeeper/server/quorum/QuorumPeerConfig.java#QuorumPeerConfig

          Show
          david babits added a comment - can somebody confirm whether this feature is available in zookeeper-3.4.5, I can't seem to see it implemented looking at source code in org.apache.zookeeper.server.quorum.QuorumPeerConfig Thanks. http://grepcode.com/file/repository.cloudera.com/content/repositories/releases/org.apache.zookeeper/zookeeper/3.4.5-cdh4.2.0/org/apache/zookeeper/server/quorum/QuorumPeerConfig.java#QuorumPeerConfig
          Hide
          Hudson added a comment -

          Integrated in ZooKeeper-trunk #251 (See http://hudson.zones.apache.org/hudson/job/ZooKeeper-trunk/251/)

          Show
          Hudson added a comment - Integrated in ZooKeeper-trunk #251 (See http://hudson.zones.apache.org/hudson/job/ZooKeeper-trunk/251/ )
          Hide
          Patrick Hunt added a comment -

          Committed revision 750597. (3.1 branch)
          Committed revision 750599. (mainline - 3.2)

          Thanks Jean-Daniel!

          Show
          Patrick Hunt added a comment - Committed revision 750597. (3.1 branch) Committed revision 750599. (mainline - 3.2) Thanks Jean-Daniel!
          Hide
          Patrick Hunt added a comment -

          Updated the patch to successfully apply against the latest codebase.

          +1 - will be committed for 3.1.1/3.2 releases.

          Show
          Patrick Hunt added a comment - Updated the patch to successfully apply against the latest codebase. +1 - will be committed for 3.1.1/3.2 releases.
          Hide
          Patrick Hunt added a comment -

          Yea, we are excited to see hbase using zk, please let us know if you have questions or would like to discuss recipes/approach/etc...

          Show
          Patrick Hunt added a comment - Yea, we are excited to see hbase using zk, please let us know if you have questions or would like to discuss recipes/approach/etc...
          Hide
          Jean-Daniel Cryans added a comment -

          Thanks a lot Patrick. We'll have other patches coming in in the following weeks so stay tunned

          Show
          Jean-Daniel Cryans added a comment - Thanks a lot Patrick. We'll have other patches coming in in the following weeks so stay tunned
          Hide
          Patrick Hunt added a comment -

          I'll take care of regenning the patch. Sorry for the delay, somehow we missed this one.

          Show
          Patrick Hunt added a comment - I'll take care of regenning the patch. Sorry for the delay, somehow we missed this one.
          Hide
          Mahadev konar added a comment -

          +1 for hte changes... the patch would need to be regenerated against the trunk.

          Show
          Mahadev konar added a comment - +1 for hte changes... the patch would need to be regenerated against the trunk.
          Hide
          Nitay Joffe added a comment -

          Can we get a review for this?

          Show
          Nitay Joffe added a comment - Can we get a review for this?
          Hide
          Jean-Daniel Cryans added a comment -

          Example of what we can do with previously posted patch.

          Show
          Jean-Daniel Cryans added a comment - Example of what we can do with previously posted patch.
          Hide
          Jean-Daniel Cryans added a comment -

          A patch against trunk that enables foreign configurations into QuorumPeerConfig.

          Show
          Jean-Daniel Cryans added a comment - A patch against trunk that enables foreign configurations into QuorumPeerConfig.
          Hide
          Nitay Joffe added a comment -

          More specifically, we are trying to make zoo.cfg play nicely with hadoop/hbase conf so that we can interpolate things like hadoop.tmp.dir as the default for the ZK dataDir.

          Show
          Nitay Joffe added a comment - More specifically, we are trying to make zoo.cfg play nicely with hadoop/hbase conf so that we can interpolate things like hadoop.tmp.dir as the default for the ZK dataDir.

            People

            • Assignee:
              Jean-Daniel Cryans
              Reporter:
              Nitay Joffe
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development