ZooKeeper
  1. ZooKeeper
  2. ZOOKEEPER-799

Add tools and recipes for monitoring as a contrib

    Details

    • Type: New Feature New Feature
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.4.0
    • Component/s: contrib
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Tools and Recipes for Monitoring ZooKeeper using Cacti, Nagios or Ganglia.
    • Tags:
      monitoring, cacti, nagios, ganglia, contrib

      Description

      Tools and Recipes for Monitoring ZooKeeper using Cacti, Nagios or Ganglia.

      1. ZOOKEEPER-799.patch
        40 kB
        Andrei Savu
      2. monitoring.tar.gz
        450 kB
        Andrei Savu

        Issue Links

          Activity

          Andrei Savu created issue -
          Andrei Savu made changes -
          Field Original Value New Value
          Link This issue is blocked by ZOOKEEPER-701 [ ZOOKEEPER-701 ]
          Hide
          Andrei Savu added a comment -

          I've attached a patch and an archive containing all the files in the patch + some screenshots. I'm waiting for your feedback.

          Show
          Andrei Savu added a comment - I've attached a patch and an archive containing all the files in the patch + some screenshots. I'm waiting for your feedback.
          Andrei Savu made changes -
          Attachment ZOOKEEPER-799.patch [ 12448356 ]
          Attachment monitoring.tar.gz [ 12448357 ]
          Hide
          Matt Massie added a comment -

          Andrei-

          Nice work on this patch!

          I've just looked over your work re: ganglia since it's what I'm familiar with.

          The GangliaHandler in check_zookeeper.py makes the assumption that all ZK variables are uint32. Looking at the list of exported variables it appears that assumption is safe but it's something to remember. In the future, you may want to export heterogeneous metrics with strings and floats for example.

          I think it might be possible to pass just the executable string 'gmetric' to subprocess.call instead of the full path '/usr/bin/gmetric'. This would allow for alternative locations of gmetric to be found using the PATH environment variable.

          Your python module looks correct to me but I didn't test it.

          You might want to drop a short sentence into the README that the python module only works with the 3.1.x versions of ganglia whereas the gmetric approach works for all versions.

          Show
          Matt Massie added a comment - Andrei- Nice work on this patch! I've just looked over your work re: ganglia since it's what I'm familiar with. The GangliaHandler in check_zookeeper.py makes the assumption that all ZK variables are uint32 . Looking at the list of exported variables it appears that assumption is safe but it's something to remember. In the future, you may want to export heterogeneous metrics with strings and floats for example. I think it might be possible to pass just the executable string 'gmetric' to subprocess.call instead of the full path '/usr/bin/gmetric' . This would allow for alternative locations of gmetric to be found using the PATH environment variable. Your python module looks correct to me but I didn't test it. You might want to drop a short sentence into the README that the python module only works with the 3.1.x versions of ganglia whereas the gmetric approach works for all versions.
          Hide
          Andrei Savu added a comment -

          Thanks for reviewing. I will check the data types again so that I'm
          sure I'm not making any wrong assumptions and I'm also going to let
          the user specify the path to gmetric as cli parameter.

          Show
          Andrei Savu added a comment - Thanks for reviewing. I will check the data types again so that I'm sure I'm not making any wrong assumptions and I'm also going to let the user specify the path to gmetric as cli parameter.
          Hide
          Andrei Savu added a comment -
          • fixed GangliaHandler in check_zookeeper.py to allow the user to specify the path to gmetric on the command line (default value: /usr/bin/gmetric)
          • it seems like it's safe to to use unit32, at least for now
          • added notice for the Ganglia Python module - it only works with 3.1.x

          Matt thanks again for reviewing this patch.

          Travis I would really like to know what you think about this because as I remember, at Twitter, you are using Ganglia for monitoring ZooKeeper.

          What else should I do to get this patch committed? Add more source code comments? More details in the docs?

          Show
          Andrei Savu added a comment - fixed GangliaHandler in check_zookeeper.py to allow the user to specify the path to gmetric on the command line (default value: /usr/bin/gmetric) it seems like it's safe to to use unit32 , at least for now added notice for the Ganglia Python module - it only works with 3.1.x Matt thanks again for reviewing this patch. Travis I would really like to know what you think about this because as I remember, at Twitter, you are using Ganglia for monitoring ZooKeeper. What else should I do to get this patch committed? Add more source code comments? More details in the docs?
          Andrei Savu made changes -
          Attachment ZOOKEEPER-799.patch [ 12449086 ]
          Attachment monitoring.tar.gz [ 12449087 ]
          Hide
          Andrei Savu added a comment -
          • removed two files: LICENCE and a copyright NOTICE

          I believe the patch is now ready to be committed.

          Show
          Andrei Savu added a comment - removed two files: LICENCE and a copyright NOTICE I believe the patch is now ready to be committed.
          Andrei Savu made changes -
          Attachment ZOOKEEPER-799.patch [ 12449150 ]
          Attachment monitoring.tar.gz [ 12449151 ]
          Andrei Savu made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Affects Version/s 3.3.0 [ 12313976 ]
          Affects Version/s 3.4.0 [ 12314469 ]
          Affects Version/s 3.3.1 [ 12314846 ]
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12449151/monitoring.tar.gz
          against trunk revision 962697.

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no tests are needed for this patch.

          -1 patch. The patch command could not apply the patch.

          Console output: http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h1.grid.sp2.yahoo.net/140/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12449151/monitoring.tar.gz against trunk revision 962697. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no tests are needed for this patch. -1 patch. The patch command could not apply the patch. Console output: http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h1.grid.sp2.yahoo.net/140/console This message is automatically generated.
          Andrei Savu made changes -
          Link This issue is required by ZOOKEEPER-701 [ ZOOKEEPER-701 ]
          Andrei Savu made changes -
          Link This issue is blocked by ZOOKEEPER-701 [ ZOOKEEPER-701 ]
          Andrei Savu made changes -
          Link This issue requires ZOOKEEPER-744 [ ZOOKEEPER-744 ]
          Hide
          Patrick Hunt added a comment -

          Andrei please fix the patch application issue and resubmit. thanks.

          Show
          Patrick Hunt added a comment - Andrei please fix the patch application issue and resubmit. thanks.
          Patrick Hunt made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Andrei Savu made changes -
          Attachment ZOOKEEPER-799.patch [ 12448356 ]
          Andrei Savu made changes -
          Attachment ZOOKEEPER-799.patch [ 12449086 ]
          Andrei Savu made changes -
          Attachment monitoring.tar.gz [ 12448357 ]
          Andrei Savu made changes -
          Attachment monitoring.tar.gz [ 12449087 ]
          Andrei Savu made changes -
          Attachment monitoring.tar.gz [ 12449351 ]
          Andrei Savu made changes -
          Attachment monitoring.tar.gz [ 12449151 ]
          Andrei Savu made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12449351/monitoring.tar.gz
          against trunk revision 962697.

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no tests are needed for this patch.

          -1 patch. The patch command could not apply the patch.

          Console output: http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h1.grid.sp2.yahoo.net/141/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12449351/monitoring.tar.gz against trunk revision 962697. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no tests are needed for this patch. -1 patch. The patch command could not apply the patch. Console output: http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h1.grid.sp2.yahoo.net/141/console This message is automatically generated.
          Hide
          Patrick Hunt added a comment -

          I see, both files are necessary to build. I'll take a look at this asap (don't worry about hudson).

          Show
          Patrick Hunt added a comment - I see, both files are necessary to build. I'll take a look at this asap (don't worry about hudson).
          Hide
          Andrei Savu added a comment -

          Not really. The archive only contains some extra files (screenshots). I don't understand why Hudson keeps trying to apply it as patch even of it's not marked for inclusion.

          original message
          Subject: [jira] Commented: (ZOOKEEPER-799) Add tools and recipes for monitoring as a contrib
          From: "Patrick Hunt (JIRA)" <jira@apache.org>
          Date: 13/07/2010 20:13

          [ https://issues.apache.org/jira/browse/ZOOKEEPER-799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12887852#action_12887852 ]

          Patrick Hunt commented on ZOOKEEPER-799:
          ----------------------------------------

          I see, both files are necessary to build. I'll take a look at this asap (don't worry about hudson).


          This message is automatically generated by JIRA.
          -
          You can reply to this email to add a comment to the issue online.

          Show
          Andrei Savu added a comment - Not really. The archive only contains some extra files (screenshots). I don't understand why Hudson keeps trying to apply it as patch even of it's not marked for inclusion. original message Subject: [jira] Commented: ( ZOOKEEPER-799 ) Add tools and recipes for monitoring as a contrib From: "Patrick Hunt (JIRA)" <jira@apache.org> Date: 13/07/2010 20:13 [ https://issues.apache.org/jira/browse/ZOOKEEPER-799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12887852#action_12887852 ] Patrick Hunt commented on ZOOKEEPER-799 : ---------------------------------------- I see, both files are necessary to build. I'll take a look at this asap (don't worry about hudson). – This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
          Patrick Hunt made changes -
          Fix Version/s 3.4.0 [ 12314469 ]
          Hide
          Patrick Hunt added a comment -

          probably because monitoring.tar is currently the most recent attachment.

          Show
          Patrick Hunt added a comment - probably because monitoring.tar is currently the most recent attachment.
          Hide
          Patrick Hunt added a comment -

          +1, great to see this patch go in!! thanks Andrei

          Show
          Patrick Hunt added a comment - +1, great to see this patch go in!! thanks Andrei
          Patrick Hunt made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Hadoop Flags [Reviewed]
          Resolution Fixed [ 1 ]
          Hide
          Andrei Savu added a comment -

          Awesome! Thanks for reviewing Patrick!

          Show
          Andrei Savu added a comment - Awesome! Thanks for reviewing Patrick!
          Hide
          Hudson added a comment -

          Integrated in ZooKeeper-trunk #881 (See http://hudson.zones.apache.org/hudson/job/ZooKeeper-trunk/881/)

          Show
          Hudson added a comment - Integrated in ZooKeeper-trunk #881 (See http://hudson.zones.apache.org/hudson/job/ZooKeeper-trunk/881/ )
          Mahadev konar made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Hide
          elasticdarwin added a comment -

          There is a bug in nagios module when using 'stat' keyword
          in _send_cmd method (check_zookeeper.py line 169)

          < data = s.recv(2048)

          > data = ''
          > while True:
          > pack = s.recv(2048)
          > if len(pack) is 0:
          > break
          > data += pack

          I'm not very familiar with python, you may rewrite this for better reading / performance.

          Thank you for your plugin~

          Show
          elasticdarwin added a comment - There is a bug in nagios module when using 'stat' keyword in _send_cmd method (check_zookeeper.py line 169) < data = s.recv(2048) — > data = '' > while True: > pack = s.recv(2048) > if len(pack) is 0: > break > data += pack I'm not very familiar with python, you may rewrite this for better reading / performance. Thank you for your plugin~
          Transition Time In Source Status Execution Times Last Executer Last Execution Date
          Patch Available Patch Available Open Open
          2d 4h 16m 1 Patrick Hunt 12/Jul/10 17:38
          Open Open Patch Available Patch Available
          11d 13h 1m 2 Andrei Savu 13/Jul/10 15:31
          Patch Available Patch Available Resolved Resolved
          16h 9m 1 Patrick Hunt 14/Jul/10 07:41
          Resolved Resolved Closed Closed
          497d 12h 41m 1 Mahadev konar 23/Nov/11 19:22

            People

            • Assignee:
              Andrei Savu
              Reporter:
              Andrei Savu
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development