ZooKeeper
  1. ZooKeeper
  2. ZOOKEEPER-496

zookeeper-tree utility for export, import and incremental updates

    Details

    • Type: New Feature New Feature
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.3.0
    • Component/s: contrib
    • Labels:
      None
    • Environment:

      RHEL 4.6, libxml2

    • Hadoop Flags:
      Reviewed
    • Release Note:
      Hide
      ==========================================
      zktreeutil - Zookeeper Tree Data Utility
      Author: Anirban Roy
      Organization: Yahoo Inc.
      ==========================================

      zktreeutil program is intended to manage and manipulate zk-tree data quickly, effi-
      ciently and with ease. The utility operates on free-form ZK-tree and hence can be used
      for any cluster managed by Zookeeper. Here are the basic functionalities -

      EXPORT: The whole/partial ZK-tree is exported into a XML file. This helps in
      capturing a current snapshot of the data for backup/analysis. For a subtree
      export, one need to specify the path to the ZK-subtree with proper option.

      IMPORT: The ZK-tree can be imported from XML into ZK cluster. This helps in priming
      the new ZK cluster with static configuration. The import can be non-intrusive by
      making only the additions in the existing data. The import of subtree is also
      possible by optionally providing the path to the ZK-subtree.

      DIFF: Creates a diff between live ZK data vs data saved in XML file. Diff can ignore
      some ZK-tree branches (possibly dynamic data) on reading the optional ignore flag
      from XML file. Diffing on a ZK-subtree achieved by providing path to ZK-subtree with
      diff command.

      UPDATE: Make the incremental changes into the live ZK-tree from saved XML, essentia-
      lly after running the diff.

      DUMP: Dumps the ZK-tree on the standard output device reading either from live ZK
      server or XML file. Like export, ZK-subtree can be dumped with optionaly
      providing the path to the ZK-subtree, and till a certain depth of the (sub)tree.

      The exported ZK data into XML file can be shortened by only keeping the static ZK
      nodes which are required to prime a cluster. The dynamic zk nodes (created on-the-
      fly) can be ignored by setting a 'ignore' attribute at the root node of the dynamic
      subtree (see tests/zk_sample.xml), possibly deleting all inner ZK nodes under that.
      Once ignored, the whole subtree is ignored during DIFF, UPDATE and WRITE.

      Pre-requisites
      --------------
      1. Linux system with 2.6.X kernel.
      2. Zookeeper C client library (locally built at ../../c/.libs) >= 3.X.X
      3. Development build libraries (rpm packages):
        a. boost-devel >= 1.32.0
        b. libxml2-devel >= 2.7.3
        c. log4cxx0100-devel >= 0.10.0

      Build instructions
      ------------------
      1. cd into this directory
      2. autoreconf -if
      3. ./configure
      4. make
      5. 'zktreeutil' binary created under src directory

      Limitations
      -----------
      Current version works with text data only, binary data will be supported in future
      versions.

      Testing and usage of zktreeutil
      --------------------------------
      1. Run Zookeeper server locally on port 2181
      2. export LD_LIBRARY_PATH=../../c/.libs/:/usr/local/lib/
      3. ./src/zktreeutil --help # show help
      4. ./src/zktreeutil --zookeeper=localhost:2181 --import --xmlfile=tests/zk_sample.xml 2>/dev/null # import sample ZK tree
      5. ./src/zktreeutil --zookeeper=localhost:2181 --dump --path=/myapp/version-1.0 2>/dev/null # dump Zk subtree
      5. ./src/zktreeutil --zookeeper=localhost:2181 --dump --depth=3 2>/dev/null # dump Zk tree till certain depth
      6. ./src/zktreeutil --xmlfile=zk_sample.xml -D 2>/dev/null # dump the xml data
      7. Change zk_sample.xml with adding/deleting/chaging some nodes
      8. ./src/zktreeutil -z localhost:2181 -F -x zk_sample.xml -p /myapp/version-1.0/configuration 2>/dev/null # take a diff of changes
      9. ./src/zktreeutil -z localhost:2181 -E 2>/dev/null > zk_sample2.xml # export the mofied ZK tree
      10. ./src/zktreeutil -z localhost:2181 -U -x zk_sample.xml -p /myapp/version-1.0/distributions 2>/dev/null # update with incr. changes
      11. ./src/zktreeutil --zookeeper=localhost:2181 --import --force --xmlfile=zk_sample2.xml 2>/dev/null # re-prime the ZK tree
      Show
      ========================================== zktreeutil - Zookeeper Tree Data Utility Author: Anirban Roy Organization: Yahoo Inc. ========================================== zktreeutil program is intended to manage and manipulate zk-tree data quickly, effi- ciently and with ease. The utility operates on free-form ZK-tree and hence can be used for any cluster managed by Zookeeper. Here are the basic functionalities - EXPORT: The whole/partial ZK-tree is exported into a XML file. This helps in capturing a current snapshot of the data for backup/analysis. For a subtree export, one need to specify the path to the ZK-subtree with proper option. IMPORT: The ZK-tree can be imported from XML into ZK cluster. This helps in priming the new ZK cluster with static configuration. The import can be non-intrusive by making only the additions in the existing data. The import of subtree is also possible by optionally providing the path to the ZK-subtree. DIFF: Creates a diff between live ZK data vs data saved in XML file. Diff can ignore some ZK-tree branches (possibly dynamic data) on reading the optional ignore flag from XML file. Diffing on a ZK-subtree achieved by providing path to ZK-subtree with diff command. UPDATE: Make the incremental changes into the live ZK-tree from saved XML, essentia- lly after running the diff. DUMP: Dumps the ZK-tree on the standard output device reading either from live ZK server or XML file. Like export, ZK-subtree can be dumped with optionaly providing the path to the ZK-subtree, and till a certain depth of the (sub)tree. The exported ZK data into XML file can be shortened by only keeping the static ZK nodes which are required to prime a cluster. The dynamic zk nodes (created on-the- fly) can be ignored by setting a 'ignore' attribute at the root node of the dynamic subtree (see tests/zk_sample.xml), possibly deleting all inner ZK nodes under that. Once ignored, the whole subtree is ignored during DIFF, UPDATE and WRITE. Pre-requisites -------------- 1. Linux system with 2.6.X kernel. 2. Zookeeper C client library (locally built at ../../c/.libs) >= 3.X.X 3. Development build libraries (rpm packages):   a. boost-devel >= 1.32.0   b. libxml2-devel >= 2.7.3   c. log4cxx0100-devel >= 0.10.0 Build instructions ------------------ 1. cd into this directory 2. autoreconf -if 3. ./configure 4. make 5. 'zktreeutil' binary created under src directory Limitations ----------- Current version works with text data only, binary data will be supported in future versions. Testing and usage of zktreeutil -------------------------------- 1. Run Zookeeper server locally on port 2181 2. export LD_LIBRARY_PATH=../../c/.libs/:/usr/local/lib/ 3. ./src/zktreeutil --help # show help 4. ./src/zktreeutil --zookeeper=localhost:2181 --import --xmlfile=tests/zk_sample.xml 2>/dev/null # import sample ZK tree 5. ./src/zktreeutil --zookeeper=localhost:2181 --dump --path=/myapp/version-1.0 2>/dev/null # dump Zk subtree 5. ./src/zktreeutil --zookeeper=localhost:2181 --dump --depth=3 2>/dev/null # dump Zk tree till certain depth 6. ./src/zktreeutil --xmlfile=zk_sample.xml -D 2>/dev/null # dump the xml data 7. Change zk_sample.xml with adding/deleting/chaging some nodes 8. ./src/zktreeutil -z localhost:2181 -F -x zk_sample.xml -p /myapp/version-1.0/configuration 2>/dev/null # take a diff of changes 9. ./src/zktreeutil -z localhost:2181 -E 2>/dev/null > zk_sample2.xml # export the mofied ZK tree 10. ./src/zktreeutil -z localhost:2181 -U -x zk_sample.xml -p /myapp/version-1.0/distributions 2>/dev/null # update with incr. changes 11. ./src/zktreeutil --zookeeper=localhost:2181 --import --force --xmlfile=zk_sample2.xml 2>/dev/null # re-prime the ZK tree
    • Tags:
      zookeeper tree trree-data utility zktree

      Description

      ==========================================
      zktreeutil - Zookeeper Tree Data Utility
      Author: Anirban Roy
      Organization: Yahoo Inc.
      ==========================================

      zktreeutil program is intended to manage and manipulate zk-tree data quickly, effi-
      ciently and with ease. The utility operates on free-form ZK-tree and hence can be used
      for any cluster managed by Zookeeper. Here are the basic functionalities -

      EXPORT: The whole/partial ZK-tree is exported into a XML file. This helps in
      capturing a current snapshot of the data for backup/analysis. For a subtree
      export, one need to specify the path to the ZK-subtree with proper option.

      IMPORT: The ZK-tree can be imported from XML into ZK cluster. This helps in priming
      the new ZK cluster with static configuration. The import can be non-intrusive by
      making only the additions in the existing data. The import of subtree is also
      possible by optionally providing the path to the ZK-subtree.

      DIFF: Creates a diff between live ZK data vs data saved in XML file. Diff can ignore
      some ZK-tree branches (possibly dynamic data) on reading the optional ignore flag
      from XML file. Diffing on a ZK-subtree achieved by providing path to ZK-subtree with
      diff command.

      UPDATE: Make the incremental changes into the live ZK-tree from saved XML, essentia-
      lly after running the diff.

      DUMP: Dumps the ZK-tree on the standard output device reading either from live ZK
      server or XML file. Like export, ZK-subtree can be dumped with optionaly
      providing the path to the ZK-subtree, and till a certain depth of the (sub)tree.

      The exported ZK data into XML file can be shortened by only keeping the static ZK
      nodes which are required to prime a cluster. The dynamic zk nodes (created on-the-
      fly) can be ignored by setting a 'ignore' attribute at the root node of the dynamic
      subtree (see tests/zk_sample.xml), possibly deleting all inner ZK nodes under that.
      Once ignored, the whole subtree is ignored during DIFF, UPDATE and WRITE.

      1. zktreeutil_2.patch
        89 kB
        Benjamin Reed
      2. zktreeutil.patch
        73 kB
        Anirban Roy

        Activity

        Transition Time In Source Status Execution Times Last Executer Last Execution Date
        Open Open In Progress In Progress
        4d 1h 9m 1 Anirban Roy 11/Jan/10 18:50
        In Progress In Progress Open Open
        20s 1 Anirban Roy 11/Jan/10 18:51
        Patch Available Patch Available Open Open
        151d 19m 6 Benjamin Reed 12/Jan/10 19:39
        Open Open Patch Available Patch Available
        6d 23h 19m 7 Benjamin Reed 12/Jan/10 19:39
        Patch Available Patch Available Resolved Resolved
        2h 22m 1 Benjamin Reed 12/Jan/10 22:02
        Resolved Resolved Closed Closed
        72d 19h 22m 1 Patrick Hunt 26/Mar/10 17:24
        Patrick Hunt made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Hide
        Hudson added a comment -

        Integrated in ZooKeeper-trunk #665 (See http://hudson.zones.apache.org/hudson/job/ZooKeeper-trunk/665/)

        Show
        Hudson added a comment - Integrated in ZooKeeper-trunk #665 (See http://hudson.zones.apache.org/hudson/job/ZooKeeper-trunk/665/ )
        Benjamin Reed made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]
        Hide
        Benjamin Reed added a comment -

        Committed revision 898542.
        thanx anirban!

        Show
        Benjamin Reed added a comment - Committed revision 898542. thanx anirban!
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12430037/zktreeutil_2.patch
        against trunk revision 896800.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 3 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs warnings.

        -1 release audit. The applied patch generated 207 release audit warnings (more than the trunk's current 200 warnings).

        +1 core tests. The patch passed core unit tests.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h8.grid.sp2.yahoo.net/102/testReport/
        Release audit warnings: http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h8.grid.sp2.yahoo.net/102/artifact/trunk/patchprocess/releaseAuditDiffWarnings.txt
        Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h8.grid.sp2.yahoo.net/102/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Console output: http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h8.grid.sp2.yahoo.net/102/console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12430037/zktreeutil_2.patch against trunk revision 896800. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. -1 release audit. The applied patch generated 207 release audit warnings (more than the trunk's current 200 warnings). +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h8.grid.sp2.yahoo.net/102/testReport/ Release audit warnings: http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h8.grid.sp2.yahoo.net/102/artifact/trunk/patchprocess/releaseAuditDiffWarnings.txt Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h8.grid.sp2.yahoo.net/102/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h8.grid.sp2.yahoo.net/102/console This message is automatically generated.
        Benjamin Reed made changes -
        Hadoop Flags [Reviewed]
        Benjamin Reed made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Benjamin Reed made changes -
        Status Patch Available [ 10002 ] Open [ 1 ]
        Benjamin Reed made changes -
        Attachment zktreeutil_2.patch [ 12430037 ]
        Hide
        Benjamin Reed added a comment -

        i fixed the tabs using a simple sed script.

        Show
        Benjamin Reed added a comment - i fixed the tabs using a simple sed script.
        Hide
        Benjamin Reed added a comment -

        looks good anirban. sorry to be such a pain about this, but you still have tabs in the code. you need to be using spaces instead of tabs.

        Show
        Benjamin Reed added a comment - looks good anirban. sorry to be such a pain about this, but you still have tabs in the code. you need to be using spaces instead of tabs.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12429925/zktreeutil.patch
        against trunk revision 896800.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 3 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs warnings.

        -1 release audit. The applied patch generated 207 release audit warnings (more than the trunk's current 200 warnings).

        +1 core tests. The patch passed core unit tests.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h7.grid.sp2.yahoo.net/35/testReport/
        Release audit warnings: http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h7.grid.sp2.yahoo.net/35/artifact/trunk/patchprocess/releaseAuditDiffWarnings.txt
        Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h7.grid.sp2.yahoo.net/35/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Console output: http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h7.grid.sp2.yahoo.net/35/console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12429925/zktreeutil.patch against trunk revision 896800. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. -1 release audit. The applied patch generated 207 release audit warnings (more than the trunk's current 200 warnings). +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h7.grid.sp2.yahoo.net/35/testReport/ Release audit warnings: http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h7.grid.sp2.yahoo.net/35/artifact/trunk/patchprocess/releaseAuditDiffWarnings.txt Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h7.grid.sp2.yahoo.net/35/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h7.grid.sp2.yahoo.net/35/console This message is automatically generated.
        Anirban Roy made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Anirban Roy made changes -
        Status In Progress [ 3 ] Open [ 1 ]
        Anirban Roy made changes -
        Status Open [ 1 ] In Progress [ 3 ]
        Anirban Roy made changes -
        Attachment zktreeutil.patch [ 12429925 ]
        Hide
        Anirban Roy added a comment -

        Minor change: set tabwidth as 4.

        Show
        Anirban Roy added a comment - Minor change: set tabwidth as 4.
        Anirban Roy made changes -
        Attachment zktreeutil.patch [ 12429668 ]
        Anirban Roy made changes -
        Release Note ==========================================
        zktreeutil - Zookeeper Tree Data Utility
        Author: Anirban Roy
        Organization: Yahoo Inc.
        ==========================================

        zktreeutil program is intended to manage and manipulate zk-tree data quickly, effi-
        ciently and with ease. The utility operates on free-form ZK-tree and hence can be used
        for any cluster managed by Zookeeper. Here are the basic functionalities -

        EXPORT: The whole/partial ZK-tree is exported into a XML file. This helps in
        capturing a current snapshot of the data for backup/analysis. For a subtree
        export, one need to specify the path to the ZK-subtree with proper option.

        IMPORT: The ZK-tree can be imported from XML into ZK cluster. This helps in priming
        the new ZK cluster with static configuration. The import can be non-intrusive by
        making only the additions in the existing data. The import of subtree is also
        possible by optionally providing the path to the ZK-subtree.

        DIFF: Creates a diff between live ZK data vs data saved in XML file. Diff can ignore
        some ZK-tree branches (possibly dynamic data) on reading the optional ignore flag
        from XML file. Diffing on a ZK-subtree achieved by providing path to ZK-subtree with
        diff command.

        UPDATE: Make the incremental changes into the live ZK-tree from saved XML, essentia-
        lly after running the diff.

        DUMP: Dumps the ZK-tree on the standard output device reading either from live ZK
        server or XML file. Like export, ZK-subtree can be dumped with optionaly
        providing the path to the ZK-subtree, and till a certain depth of the (sub)tree.

        The exported ZK data into XML file can be shortened by only keeping the static ZK
        nodes which are required to prime a cluster. The dynamic zk nodes (created on-the-
        fly) can be ignored by setting a 'ignore' attribute at the root node of the dynamic
        subtree (see tests/zk_sample.xml), possibly deleting all inner ZK nodes under that.
        Once ignored, the whole subtree is ignored during DIFF, UPDATE and WRITE.

        Pre-requisites
        --------------
        1. Linux system with 2.6.X kernel.
        2. Zookeeper C client library >= 3.X.X
        3. Development build libraries:
          a. libxml2 >= 2.7.3
          b. log4cxx >= 0.10.0

        Build instructions
        ------------------
        1. cd into this directory
        2. autoreconf -if
        3. ./configure
        4. make
        5. 'zktreeutil' binary created under src directory

        Testing and usage of zktreeutil
        --------------------------------
        1. Run Zookeeper server locally on port 2181
        2. export LD_LIBRARY_PATH=../../c/.libs/:/usr/local/lib/
        3. ./src/zktreeutil --help # show help
        4. ./src/zktreeutil --zookeeper=localhost:2181 --import --xmlfile=tests/zk_sample.xml 2>/dev/null # import sample ZK tree
        5. ./src/zktreeutil --zookeeper=localhost:2181 --dump --path=/myapp/version-1.0 2>/dev/null # dump Zk subtree
        5. ./src/zktreeutil --zookeeper=localhost:2181 --dump --depth=3 2>/dev/null # dump Zk tree till certain depth
        6. ./src/zktreeutil --xmlfile=zk_sample.xml -D 2>/dev/null # dump the xml data
        7. Change zk_sample.xml with adding/deleting/chaging some nodes
        8. ./src/zktreeutil -z localhost:2181 -F -x zk_sample.xml -p /myapp/version-1.0/configuration 2>/dev/null # take a diff of changes
        9. ./src/zktreeutil -z localhost:2181 -E 2>/dev/null > zk_sample2.xml # export the mofied ZK tree
        10. ./src/zktreeutil -z localhost:2181 -U -x zk_sample.xml -p /myapp/version-1.0/distributions 2>/dev/null # update with incr. changes
        11. ./src/zktreeutil --zookeeper=localhost:2181 --import --force --xmlfile=zk_sample2.xml 2>/dev/null # re-prime the ZK tree
        ==========================================
        zktreeutil - Zookeeper Tree Data Utility
        Author: Anirban Roy
        Organization: Yahoo Inc.
        ==========================================

        zktreeutil program is intended to manage and manipulate zk-tree data quickly, effi-
        ciently and with ease. The utility operates on free-form ZK-tree and hence can be used
        for any cluster managed by Zookeeper. Here are the basic functionalities -

        EXPORT: The whole/partial ZK-tree is exported into a XML file. This helps in
        capturing a current snapshot of the data for backup/analysis. For a subtree
        export, one need to specify the path to the ZK-subtree with proper option.

        IMPORT: The ZK-tree can be imported from XML into ZK cluster. This helps in priming
        the new ZK cluster with static configuration. The import can be non-intrusive by
        making only the additions in the existing data. The import of subtree is also
        possible by optionally providing the path to the ZK-subtree.

        DIFF: Creates a diff between live ZK data vs data saved in XML file. Diff can ignore
        some ZK-tree branches (possibly dynamic data) on reading the optional ignore flag
        from XML file. Diffing on a ZK-subtree achieved by providing path to ZK-subtree with
        diff command.

        UPDATE: Make the incremental changes into the live ZK-tree from saved XML, essentia-
        lly after running the diff.

        DUMP: Dumps the ZK-tree on the standard output device reading either from live ZK
        server or XML file. Like export, ZK-subtree can be dumped with optionaly
        providing the path to the ZK-subtree, and till a certain depth of the (sub)tree.

        The exported ZK data into XML file can be shortened by only keeping the static ZK
        nodes which are required to prime a cluster. The dynamic zk nodes (created on-the-
        fly) can be ignored by setting a 'ignore' attribute at the root node of the dynamic
        subtree (see tests/zk_sample.xml), possibly deleting all inner ZK nodes under that.
        Once ignored, the whole subtree is ignored during DIFF, UPDATE and WRITE.

        Pre-requisites
        --------------
        1. Linux system with 2.6.X kernel.
        2. Zookeeper C client library (locally built at ../../c/.libs) >= 3.X.X
        3. Development build libraries (rpm packages):
          a. boost-devel >= 1.32.0
          b. libxml2-devel >= 2.7.3
          c. log4cxx0100-devel >= 0.10.0

        Build instructions
        ------------------
        1. cd into this directory
        2. autoreconf -if
        3. ./configure
        4. make
        5. 'zktreeutil' binary created under src directory

        Limitations
        -----------
        Current version works with text data only, binary data will be supported in future
        versions.

        Testing and usage of zktreeutil
        --------------------------------
        1. Run Zookeeper server locally on port 2181
        2. export LD_LIBRARY_PATH=../../c/.libs/:/usr/local/lib/
        3. ./src/zktreeutil --help # show help
        4. ./src/zktreeutil --zookeeper=localhost:2181 --import --xmlfile=tests/zk_sample.xml 2>/dev/null # import sample ZK tree
        5. ./src/zktreeutil --zookeeper=localhost:2181 --dump --path=/myapp/version-1.0 2>/dev/null # dump Zk subtree
        5. ./src/zktreeutil --zookeeper=localhost:2181 --dump --depth=3 2>/dev/null # dump Zk tree till certain depth
        6. ./src/zktreeutil --xmlfile=zk_sample.xml -D 2>/dev/null # dump the xml data
        7. Change zk_sample.xml with adding/deleting/chaging some nodes
        8. ./src/zktreeutil -z localhost:2181 -F -x zk_sample.xml -p /myapp/version-1.0/configuration 2>/dev/null # take a diff of changes
        9. ./src/zktreeutil -z localhost:2181 -E 2>/dev/null > zk_sample2.xml # export the mofied ZK tree
        10. ./src/zktreeutil -z localhost:2181 -U -x zk_sample.xml -p /myapp/version-1.0/distributions 2>/dev/null # update with incr. changes
        11. ./src/zktreeutil --zookeeper=localhost:2181 --import --force --xmlfile=zk_sample2.xml 2>/dev/null # re-prime the ZK tree
        Hide
        Anirban Roy added a comment -

        Minor code changes; added checks for prerequisite rpm packages and boost headers; updated README with 'limitations' of the current version.

        Show
        Anirban Roy added a comment - Minor code changes; added checks for prerequisite rpm packages and boost headers; updated README with 'limitations' of the current version.
        Anirban Roy made changes -
        Attachment zktreeutil.patch [ 12429668 ]
        Hide
        Anirban Roy added a comment -

        Ne patch addressing the issues found during building with zookeeper trunk.

        Show
        Anirban Roy added a comment - Ne patch addressing the issues found during building with zookeeper trunk.
        Anirban Roy made changes -
        Attachment zktreeutil.patch [ 12425366 ]
        Benjamin Reed made changes -
        Status Patch Available [ 10002 ] Open [ 1 ]
        Hide
        Benjamin Reed added a comment -

        this looks pretty cool anirban, unfortunately i cannot get it to compile: in one of the .h files you have a qualified member, which should not be qualified; you need to include string.h when you use memset; and the resulting makefile does not include zookeeper.so as a required library. the readme should also list libboost as a required library. this is minor, but could you also change the tabs to 4 spaces?

        i also notice that you use strings for values. what will happen in the values are binary?

        Show
        Benjamin Reed added a comment - this looks pretty cool anirban, unfortunately i cannot get it to compile: in one of the .h files you have a qualified member, which should not be qualified; you need to include string.h when you use memset; and the resulting makefile does not include zookeeper.so as a required library. the readme should also list libboost as a required library. this is minor, but could you also change the tabs to 4 spaces? i also notice that you use strings for values. what will happen in the values are binary?
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12425366/zktreeutil.patch
        against trunk revision 896800.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 3 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs warnings.

        -1 release audit. The applied patch generated 207 release audit warnings (more than the trunk's current 200 warnings).

        +1 core tests. The patch passed core unit tests.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h7.grid.sp2.yahoo.net/33/testReport/
        Release audit warnings: http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h7.grid.sp2.yahoo.net/33/artifact/trunk/patchprocess/releaseAuditDiffWarnings.txt
        Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h7.grid.sp2.yahoo.net/33/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Console output: http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h7.grid.sp2.yahoo.net/33/console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12425366/zktreeutil.patch against trunk revision 896800. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. -1 release audit. The applied patch generated 207 release audit warnings (more than the trunk's current 200 warnings). +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h7.grid.sp2.yahoo.net/33/testReport/ Release audit warnings: http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h7.grid.sp2.yahoo.net/33/artifact/trunk/patchprocess/releaseAuditDiffWarnings.txt Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h7.grid.sp2.yahoo.net/33/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h7.grid.sp2.yahoo.net/33/console This message is automatically generated.
        Benjamin Reed made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Benjamin Reed made changes -
        Status Patch Available [ 10002 ] Open [ 1 ]
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12425366/zktreeutil.patch
        against trunk revision 896551.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 3 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs warnings.

        -1 release audit. The applied patch generated 207 release audit warnings (more than the trunk's current 200 warnings).

        +1 core tests. The patch passed core unit tests.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h7.grid.sp2.yahoo.net/32/testReport/
        Release audit warnings: http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h7.grid.sp2.yahoo.net/32/artifact/trunk/patchprocess/releaseAuditDiffWarnings.txt
        Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h7.grid.sp2.yahoo.net/32/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Console output: http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h7.grid.sp2.yahoo.net/32/console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12425366/zktreeutil.patch against trunk revision 896551. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. -1 release audit. The applied patch generated 207 release audit warnings (more than the trunk's current 200 warnings). +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h7.grid.sp2.yahoo.net/32/testReport/ Release audit warnings: http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h7.grid.sp2.yahoo.net/32/artifact/trunk/patchprocess/releaseAuditDiffWarnings.txt Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h7.grid.sp2.yahoo.net/32/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h7.grid.sp2.yahoo.net/32/console This message is automatically generated.
        Benjamin Reed made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Benjamin Reed made changes -
        Status Patch Available [ 10002 ] Open [ 1 ]
        Benjamin Reed made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Benjamin Reed made changes -
        Status Patch Available [ 10002 ] Open [ 1 ]
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12425366/zktreeutil.patch
        against trunk revision 896144.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 3 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs warnings.

        -1 release audit. The applied patch generated 208 release audit warnings (more than the trunk's current 201 warnings).

        -1 core tests. The patch failed core unit tests.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h8.grid.sp2.yahoo.net/98/testReport/
        Release audit warnings: http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h8.grid.sp2.yahoo.net/98/artifact/trunk/patchprocess/releaseAuditDiffWarnings.txt
        Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h8.grid.sp2.yahoo.net/98/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Console output: http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h8.grid.sp2.yahoo.net/98/console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12425366/zktreeutil.patch against trunk revision 896144. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. -1 release audit. The applied patch generated 208 release audit warnings (more than the trunk's current 201 warnings). -1 core tests. The patch failed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h8.grid.sp2.yahoo.net/98/testReport/ Release audit warnings: http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h8.grid.sp2.yahoo.net/98/artifact/trunk/patchprocess/releaseAuditDiffWarnings.txt Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h8.grid.sp2.yahoo.net/98/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h8.grid.sp2.yahoo.net/98/console This message is automatically generated.
        Benjamin Reed made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Benjamin Reed made changes -
        Status Patch Available [ 10002 ] Open [ 1 ]
        Anirban Roy made changes -
        Attachment zktreeutil.diff [ 12416094 ]
        Anirban Roy made changes -
        Attachment zktreeutil.patch [ 12425366 ]
        Hide
        Anirban Roy added a comment -

        New patch added with features like operations on a subtree rather than the entire tree, dumping the tree up to a certain depth etc.

        Show
        Anirban Roy added a comment - New patch added with features like operations on a subtree rather than the entire tree, dumping the tree up to a certain depth etc.
        Anirban Roy made changes -
        Release Note ==========================================
        zktreeutil - Zookeeper Tree Data Utility
        Author: Anirban Roy
        Organization: Yahoo Inc.
        ==========================================

        zktreeutil program is intended to manage and manipulate zk-tree data quickly, effi-
        ciently and with ease.The utility operates on free-form ZK-tree and hence can be used
        for any cluster managed by Zookeeper. Here are the basic functionalities -

        EXPORT: The whole ZK-tree is exported into a XML file. This helps in capturing a cur-
        rent snapshot of the data for backup/analysis. In addition, the data can be dumped
        as tree-like text format which is easily readable.

        IMPORT: The ZK-tree can be imported from XML into ZK cluster. This helps in priming
        the new ZK cluster with static configuration. The import can be non-intrusive by
        making only the additions in the existing data.

        DIFF: Creates a diff between live ZK data vs data saved in XML file. Diff can ignore
        some ZK-tree branches (possibly dynamic data) on reading the flag from XML file.

        UPDATE: Make the incremental changes into the live ZK-tree from saved XML, essentia-
        lly after running the diff.

        DUMP: Dumps the ZK-tree on the standard output device reading either from live ZK
        server or XML file.

        The exported ZK data into XML file can be shortened by only keeping the static ZK
        nodes which are required to prime a cluster. The dynamic zk nodes (created on-the-
        fly) can be ignored by setting a 'ignore' attribute at the root node of the dynamic
        subtree (see tests/zk_sample.xml), possibly deleting all inner ZK nodes under that.
        Once ignored, the whole subtree is ignored during DIFF, UPDATE and WRITE.

        Pre-requisites
        --------------
        1. Linux system with 2.6.X kernel.
        2. Zookeeper C client library >= 3.X.X
        3. Development build libraries:
          a. libxml2 >= 2.7.3
          b. log4cxx >= 0.10.0

        Build instructions
        ------------------
        1. cd into this directory
        2. autoreconf -if
        3. ./configure
        4. make
        5. 'zktreeutil' binary created under src directory

        Testing zktreeutil
        ------------------
        1. Run Zookeeper server locally on port 2181
        2. export LD_LIBRARY_PATH=../../c/.libs/:/usr/local/lib/
        3. ./src/zktreeutil --help # show help
        4. ./src/zktreeutil --zookeeper=localhost:2181 --import --xmlfile=tests/zk_sample.xml 2>/dev/null # import sample ZK tree
        5. ./src/zktreeutil --zookeeper=localhost:2181 --dump 2>/dev/null # dump on the terminal
        6. ./src/zktreeutil --xmlfile=zk_sample.xml -D 2>/dev/null # dump the xml data
        7. Change zk_sample.xml with adding/deleting/chaging some nodes
        8. ./src/zktreeutil --zookeeper=localhost:2181 --diff --xmlfile=zk_sample.xml 2>/dev/null # take a diff of changes
        9. ./src/zktreeutil --zookeeper=localhost:2181 --export 2>/dev/null > zk_sample2.xml # export the mofied ZK tree
        10. ./src/zktreeutil --zookeeper=localhost:2181 --update --xmlfile=zk_sample.xml 2>/dev/null # update with incr. changes
        11. ./src/zktreeutil --zookeeper=localhost:2181 --import --force --xmlfile=zk_sample2.xml 2>/dev/null # re-prime the ZK tree
        ==========================================
        zktreeutil - Zookeeper Tree Data Utility
        Author: Anirban Roy
        Organization: Yahoo Inc.
        ==========================================

        zktreeutil program is intended to manage and manipulate zk-tree data quickly, effi-
        ciently and with ease. The utility operates on free-form ZK-tree and hence can be used
        for any cluster managed by Zookeeper. Here are the basic functionalities -

        EXPORT: The whole/partial ZK-tree is exported into a XML file. This helps in
        capturing a current snapshot of the data for backup/analysis. For a subtree
        export, one need to specify the path to the ZK-subtree with proper option.

        IMPORT: The ZK-tree can be imported from XML into ZK cluster. This helps in priming
        the new ZK cluster with static configuration. The import can be non-intrusive by
        making only the additions in the existing data. The import of subtree is also
        possible by optionally providing the path to the ZK-subtree.

        DIFF: Creates a diff between live ZK data vs data saved in XML file. Diff can ignore
        some ZK-tree branches (possibly dynamic data) on reading the optional ignore flag
        from XML file. Diffing on a ZK-subtree achieved by providing path to ZK-subtree with
        diff command.

        UPDATE: Make the incremental changes into the live ZK-tree from saved XML, essentia-
        lly after running the diff.

        DUMP: Dumps the ZK-tree on the standard output device reading either from live ZK
        server or XML file. Like export, ZK-subtree can be dumped with optionaly
        providing the path to the ZK-subtree, and till a certain depth of the (sub)tree.

        The exported ZK data into XML file can be shortened by only keeping the static ZK
        nodes which are required to prime a cluster. The dynamic zk nodes (created on-the-
        fly) can be ignored by setting a 'ignore' attribute at the root node of the dynamic
        subtree (see tests/zk_sample.xml), possibly deleting all inner ZK nodes under that.
        Once ignored, the whole subtree is ignored during DIFF, UPDATE and WRITE.

        Pre-requisites
        --------------
        1. Linux system with 2.6.X kernel.
        2. Zookeeper C client library >= 3.X.X
        3. Development build libraries:
          a. libxml2 >= 2.7.3
          b. log4cxx >= 0.10.0

        Build instructions
        ------------------
        1. cd into this directory
        2. autoreconf -if
        3. ./configure
        4. make
        5. 'zktreeutil' binary created under src directory

        Testing and usage of zktreeutil
        --------------------------------
        1. Run Zookeeper server locally on port 2181
        2. export LD_LIBRARY_PATH=../../c/.libs/:/usr/local/lib/
        3. ./src/zktreeutil --help # show help
        4. ./src/zktreeutil --zookeeper=localhost:2181 --import --xmlfile=tests/zk_sample.xml 2>/dev/null # import sample ZK tree
        5. ./src/zktreeutil --zookeeper=localhost:2181 --dump --path=/myapp/version-1.0 2>/dev/null # dump Zk subtree
        5. ./src/zktreeutil --zookeeper=localhost:2181 --dump --depth=3 2>/dev/null # dump Zk tree till certain depth
        6. ./src/zktreeutil --xmlfile=zk_sample.xml -D 2>/dev/null # dump the xml data
        7. Change zk_sample.xml with adding/deleting/chaging some nodes
        8. ./src/zktreeutil -z localhost:2181 -F -x zk_sample.xml -p /myapp/version-1.0/configuration 2>/dev/null # take a diff of changes
        9. ./src/zktreeutil -z localhost:2181 -E 2>/dev/null > zk_sample2.xml # export the mofied ZK tree
        10. ./src/zktreeutil -z localhost:2181 -U -x zk_sample.xml -p /myapp/version-1.0/distributions 2>/dev/null # update with incr. changes
        11. ./src/zktreeutil --zookeeper=localhost:2181 --import --force --xmlfile=zk_sample2.xml 2>/dev/null # re-prime the ZK tree
        Description I propose and plan to develop a ZK-tree utility for managing cluster data (ZK tree) quickly and efficiently. The utility operates on free-form ZK-tree and hence can be used for any cluster managed by Zookeeper. Here are the basic functionalities -

        Export ZK-tree: The whole ZK-tree is exported into a XML file. This helps in capturing a current snapshot of the data for backup/analysis. In addition, the data can be dumped as tree-like text format which is easily readable.

        Import ZK-tree: The ZK-tree can be imported from XML into ZK cluster. This helps in priming the new ZK cluster with static configuration. The import can be non-intrusive by making only the additions in the existing data.

        Diff ZK-tree: Creates a diff between live ZK data vs data saved in XML file. Diff can ignore some ZK-tree branches (possibly dynamic data) on reading the flag from XML file.

        Update ZK-tree: Make the incremental changes into the live ZK-tree from saved XML, essentially after running the diff.

        Thanks
        Anirban
        ==========================================
        zktreeutil - Zookeeper Tree Data Utility
        Author: Anirban Roy
        Organization: Yahoo Inc.
        ==========================================

        zktreeutil program is intended to manage and manipulate zk-tree data quickly, effi-
        ciently and with ease. The utility operates on free-form ZK-tree and hence can be used
        for any cluster managed by Zookeeper. Here are the basic functionalities -

        EXPORT: The whole/partial ZK-tree is exported into a XML file. This helps in
        capturing a current snapshot of the data for backup/analysis. For a subtree
        export, one need to specify the path to the ZK-subtree with proper option.

        IMPORT: The ZK-tree can be imported from XML into ZK cluster. This helps in priming
        the new ZK cluster with static configuration. The import can be non-intrusive by
        making only the additions in the existing data. The import of subtree is also
        possible by optionally providing the path to the ZK-subtree.

        DIFF: Creates a diff between live ZK data vs data saved in XML file. Diff can ignore
        some ZK-tree branches (possibly dynamic data) on reading the optional ignore flag
        from XML file. Diffing on a ZK-subtree achieved by providing path to ZK-subtree with
        diff command.

        UPDATE: Make the incremental changes into the live ZK-tree from saved XML, essentia-
        lly after running the diff.

        DUMP: Dumps the ZK-tree on the standard output device reading either from live ZK
        server or XML file. Like export, ZK-subtree can be dumped with optionaly
        providing the path to the ZK-subtree, and till a certain depth of the (sub)tree.

        The exported ZK data into XML file can be shortened by only keeping the static ZK
        nodes which are required to prime a cluster. The dynamic zk nodes (created on-the-
        fly) can be ignored by setting a 'ignore' attribute at the root node of the dynamic
        subtree (see tests/zk_sample.xml), possibly deleting all inner ZK nodes under that.
        Once ignored, the whole subtree is ignored during DIFF, UPDATE and WRITE.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12416094/zktreeutil.diff
        against trunk revision 802964.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 3 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs warnings.

        -1 release audit. The applied patch generated 183 release audit warnings (more than the trunk's current 176 warnings).

        +1 core tests. The patch passed core unit tests.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-vesta.apache.org/184/testReport/
        Release audit warnings: http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-vesta.apache.org/184/artifact/trunk/patchprocess/releaseAuditDiffWarnings.txt
        Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-vesta.apache.org/184/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Console output: http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-vesta.apache.org/184/console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12416094/zktreeutil.diff against trunk revision 802964. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. -1 release audit. The applied patch generated 183 release audit warnings (more than the trunk's current 176 warnings). +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-vesta.apache.org/184/testReport/ Release audit warnings: http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-vesta.apache.org/184/artifact/trunk/patchprocess/releaseAuditDiffWarnings.txt Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-vesta.apache.org/184/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-vesta.apache.org/184/console This message is automatically generated.
        Anirban Roy made changes -
        Release Note ==========================================
        zktreeutil - Zookeeper Tree Data Utility
        Author: Anirban Roy
        Organization: Yahoo Inc.
        ==========================================

        zktreeutil program is intendened to manage and manipulate zk-tree data quickly, effi-
        ciently and with ease.The utility operates on free-form ZK-tree and hence can be used
        for any cluster managed by Zookeeper. Here are the basic functionalities -

        EXPORT: The whole ZK-tree is exported into a XML file. This helps in capturing a cur-
        rent snapshot of the data for backup/analysis. In addition, the data can be dumped
        as tree-like text format which is easily readable.

        IMPORT: The ZK-tree can be imported from XML into ZK cluster. This helps in priming
        the new ZK cluster with static configuration. The import can be non-intrusive by
        making only the additions in the existing data.

        DIFF: Creates a diff between live ZK data vs data saved in XML file. Diff can ignore
        some ZK-tree branches (possibly dynamic data) on reading the flag from XML file.

        UPDATE: Make the incremental changes into the live ZK-tree from saved XML, essentia-
        illy after running the diff.

        DUMP: Dumps the ZK-tree on the standard output device reading either from live ZK
        server or XML file.

        The exported ZK data into XML file can be shortened by only keeping the static ZK
        nodes which are required to prime a cluster. The dynamic zk nodes (created on-the-
        fly) can be ignored by setting a 'ignore' attribute at the root node of the dynamic
        subtree (see tests/zk_sample.xml), possibly deleting all inner ZK nodes under that.
        Once ignored, the whole subtree is ignored during DIFF, UPDATE and WRITE.

        Pre-requisites
        --------------
        1. Linux system with 2.6.X kernel.
        2. Zookeeper C client library >= 3.X.X
        3. Development build libraries:
          a. libxml2 >= 2.7.3
          b. log4cxx >= 0.10.0

        Build instructions
        ------------------
        1. cd into this directory
        2. autoreconf -if
        3. ./configure
        4. make
        5. 'zktreeutil' binary created under src directory

        Testing zktreeutil
        ------------------
        1. Run Zookeeper server locally on port 2181
        2. export LD_LIBRARY_PATH=../../c/.libs/:/usr/local/lib/
        3. ./src/zktreeutil --help # show help
        4. ./src/zktreeutil --zookeeper=localhost:2181 --import --xmlfile=tests/zk_sample.xml 2>/dev/null # import sample ZK tree
        5. ./src/zktreeutil --zookeeper=localhost:2181 --dump 2>/dev/null # dump on the terminal
        6. ./src/zktreeutil --xmlfile=zk_sample.xml -D 2>/dev/null # dump the xml data
        7. Change zk_sample.xml with adding/deleting/chaging some nodes
        8. ./src/zktreeutil --zookeeper=localhost:2181 --diff --xmlfile=zk_sample.xml 2>/dev/null # take a diff of changes
        9. ./src/zktreeutil --zookeeper=localhost:2181 --export 2>/dev/null > zk_sample2.xml # export the mofied ZK tree
        10. ./src/zktreeutil --zookeeper=localhost:2181 --update --xmlfile=zk_sample.xml 2>/dev/null # update with incr. changes
        11. ./src/zktreeutil --zookeeper=localhost:2181 --import --force --xmlfile=zk_sample2.xml 2>/dev/null # re-prime the ZK tree
        ==========================================
        zktreeutil - Zookeeper Tree Data Utility
        Author: Anirban Roy
        Organization: Yahoo Inc.
        ==========================================

        zktreeutil program is intended to manage and manipulate zk-tree data quickly, effi-
        ciently and with ease.The utility operates on free-form ZK-tree and hence can be used
        for any cluster managed by Zookeeper. Here are the basic functionalities -

        EXPORT: The whole ZK-tree is exported into a XML file. This helps in capturing a cur-
        rent snapshot of the data for backup/analysis. In addition, the data can be dumped
        as tree-like text format which is easily readable.

        IMPORT: The ZK-tree can be imported from XML into ZK cluster. This helps in priming
        the new ZK cluster with static configuration. The import can be non-intrusive by
        making only the additions in the existing data.

        DIFF: Creates a diff between live ZK data vs data saved in XML file. Diff can ignore
        some ZK-tree branches (possibly dynamic data) on reading the flag from XML file.

        UPDATE: Make the incremental changes into the live ZK-tree from saved XML, essentia-
        lly after running the diff.

        DUMP: Dumps the ZK-tree on the standard output device reading either from live ZK
        server or XML file.

        The exported ZK data into XML file can be shortened by only keeping the static ZK
        nodes which are required to prime a cluster. The dynamic zk nodes (created on-the-
        fly) can be ignored by setting a 'ignore' attribute at the root node of the dynamic
        subtree (see tests/zk_sample.xml), possibly deleting all inner ZK nodes under that.
        Once ignored, the whole subtree is ignored during DIFF, UPDATE and WRITE.

        Pre-requisites
        --------------
        1. Linux system with 2.6.X kernel.
        2. Zookeeper C client library >= 3.X.X
        3. Development build libraries:
          a. libxml2 >= 2.7.3
          b. log4cxx >= 0.10.0

        Build instructions
        ------------------
        1. cd into this directory
        2. autoreconf -if
        3. ./configure
        4. make
        5. 'zktreeutil' binary created under src directory

        Testing zktreeutil
        ------------------
        1. Run Zookeeper server locally on port 2181
        2. export LD_LIBRARY_PATH=../../c/.libs/:/usr/local/lib/
        3. ./src/zktreeutil --help # show help
        4. ./src/zktreeutil --zookeeper=localhost:2181 --import --xmlfile=tests/zk_sample.xml 2>/dev/null # import sample ZK tree
        5. ./src/zktreeutil --zookeeper=localhost:2181 --dump 2>/dev/null # dump on the terminal
        6. ./src/zktreeutil --xmlfile=zk_sample.xml -D 2>/dev/null # dump the xml data
        7. Change zk_sample.xml with adding/deleting/chaging some nodes
        8. ./src/zktreeutil --zookeeper=localhost:2181 --diff --xmlfile=zk_sample.xml 2>/dev/null # take a diff of changes
        9. ./src/zktreeutil --zookeeper=localhost:2181 --export 2>/dev/null > zk_sample2.xml # export the mofied ZK tree
        10. ./src/zktreeutil --zookeeper=localhost:2181 --update --xmlfile=zk_sample.xml 2>/dev/null # update with incr. changes
        11. ./src/zktreeutil --zookeeper=localhost:2181 --import --force --xmlfile=zk_sample2.xml 2>/dev/null # re-prime the ZK tree
        Mahadev konar made changes -
        Assignee Anirban Roy [ r_anirban ]
        Affects Version/s 3.3.0 [ 12313976 ]
        Anirban Roy made changes -
        Attachment zktreeutil.diff [ 12416094 ]
        Hide
        Anirban Roy added a comment -

        zktreeutil diff with the latest code in svn trunk.

        Show
        Anirban Roy added a comment - zktreeutil diff with the latest code in svn trunk.
        Anirban Roy made changes -
        Field Original Value New Value
        Status Open [ 1 ] Patch Available [ 10002 ]
        Affects Version/s 3.3.0 [ 12313976 ]
        Affects Version/s 3.1.1 [ 12313649 ]
        Fix Version/s 3.3.0 [ 12313976 ]
        Tags zookeeper tree trree-data utility zktree
        Release Note ==========================================
        zktreeutil - Zookeeper Tree Data Utility
        Author: Anirban Roy
        Organization: Yahoo Inc.
        ==========================================

        zktreeutil program is intendened to manage and manipulate zk-tree data quickly, effi-
        ciently and with ease.The utility operates on free-form ZK-tree and hence can be used
        for any cluster managed by Zookeeper. Here are the basic functionalities -

        EXPORT: The whole ZK-tree is exported into a XML file. This helps in capturing a cur-
        rent snapshot of the data for backup/analysis. In addition, the data can be dumped
        as tree-like text format which is easily readable.

        IMPORT: The ZK-tree can be imported from XML into ZK cluster. This helps in priming
        the new ZK cluster with static configuration. The import can be non-intrusive by
        making only the additions in the existing data.

        DIFF: Creates a diff between live ZK data vs data saved in XML file. Diff can ignore
        some ZK-tree branches (possibly dynamic data) on reading the flag from XML file.

        UPDATE: Make the incremental changes into the live ZK-tree from saved XML, essentia-
        illy after running the diff.

        DUMP: Dumps the ZK-tree on the standard output device reading either from live ZK
        server or XML file.

        The exported ZK data into XML file can be shortened by only keeping the static ZK
        nodes which are required to prime a cluster. The dynamic zk nodes (created on-the-
        fly) can be ignored by setting a 'ignore' attribute at the root node of the dynamic
        subtree (see tests/zk_sample.xml), possibly deleting all inner ZK nodes under that.
        Once ignored, the whole subtree is ignored during DIFF, UPDATE and WRITE.

        Pre-requisites
        --------------
        1. Linux system with 2.6.X kernel.
        2. Zookeeper C client library >= 3.X.X
        3. Development build libraries:
          a. libxml2 >= 2.7.3
          b. log4cxx >= 0.10.0

        Build instructions
        ------------------
        1. cd into this directory
        2. autoreconf -if
        3. ./configure
        4. make
        5. 'zktreeutil' binary created under src directory

        Testing zktreeutil
        ------------------
        1. Run Zookeeper server locally on port 2181
        2. export LD_LIBRARY_PATH=../../c/.libs/:/usr/local/lib/
        3. ./src/zktreeutil --help # show help
        4. ./src/zktreeutil --zookeeper=localhost:2181 --import --xmlfile=tests/zk_sample.xml 2>/dev/null # import sample ZK tree
        5. ./src/zktreeutil --zookeeper=localhost:2181 --dump 2>/dev/null # dump on the terminal
        6. ./src/zktreeutil --xmlfile=zk_sample.xml -D 2>/dev/null # dump the xml data
        7. Change zk_sample.xml with adding/deleting/chaging some nodes
        8. ./src/zktreeutil --zookeeper=localhost:2181 --diff --xmlfile=zk_sample.xml 2>/dev/null # take a diff of changes
        9. ./src/zktreeutil --zookeeper=localhost:2181 --export 2>/dev/null > zk_sample2.xml # export the mofied ZK tree
        10. ./src/zktreeutil --zookeeper=localhost:2181 --update --xmlfile=zk_sample.xml 2>/dev/null # update with incr. changes
        11. ./src/zktreeutil --zookeeper=localhost:2181 --import --force --xmlfile=zk_sample2.xml 2>/dev/null # re-prime the ZK tree
        Hide
        Anirban Roy added a comment -

        New feature

        Show
        Anirban Roy added a comment - New feature
        Anirban Roy created issue -

          People

          • Assignee:
            Anirban Roy
            Reporter:
            Anirban Roy
          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Time Tracking

              Estimated:
              Original Estimate - 168h
              168h
              Remaining:
              Remaining Estimate - 168h
              168h
              Logged:
              Time Spent - Not Specified
              Not Specified

                Development