ZooKeeper
  1. ZooKeeper
  2. ZOOKEEPER-1423

4lw and jmx should expose the size of the datadir/datalogdir

    Details

    • Type: Improvement Improvement
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: 3.5.0
    • Fix Version/s: None
    • Component/s: jmx
    • Labels:

      Description

      There are no metrics currently available on the size of the datadir/datalogdir. These grow w/o bound unless the cleanup script is run. It would be good to expose these metrics through jmx/4lw such that monitoring can be done on the size. Would key ppl in on whether cleanup was actually running. In particular this could be monitored/alerted on by third party systems (nagios, ganglia and the like).

      1. ZOOKEEPER-1423.patch
        18 kB
        Edward Ribeiro
      2. ZOOKEEPER-1423.2.patch
        18 kB
        Edward Ribeiro
      3. ZOOKEEPER-1423.3.patch
        18 kB
        Edward Ribeiro

        Activity

        Hide
        Edward Ribeiro added a comment -

        Hi folks,

        I am sending a first version of a patch to address this issue. I am aware that it should probably be rewritten to be in sync with ZOOKEEPER-1346, but I hope you have time to evaluate it and see if it's okay to include in 3.5.

        Any suggestion/correction is welcome.

        Cheers,
        E.

        Show
        Edward Ribeiro added a comment - Hi folks, I am sending a first version of a patch to address this issue. I am aware that it should probably be rewritten to be in sync with ZOOKEEPER-1346 , but I hope you have time to evaluate it and see if it's okay to include in 3.5. Any suggestion/correction is welcome. Cheers, E.
        Hide
        Edward Ribeiro added a comment -

        Adding a new version of the patch.

        Show
        Edward Ribeiro added a comment - Adding a new version of the patch.
        Hide
        Edward Ribeiro added a comment -

        This patch will need to be reworked so that it can be in sync with changes of ZOOKEEPER-1346, but I am uploading so that you see if it's alright.

        Show
        Edward Ribeiro added a comment - This patch will need to be reworked so that it can be in sync with changes of ZOOKEEPER-1346 , but I am uploading so that you see if it's alright.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12566395/ZOOKEEPER-1423.2.patch
        against trunk revision 1437257.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 6 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        -1 findbugs. The patch appears to introduce 2 new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed core unit tests.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/1361//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/1361//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/1361//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12566395/ZOOKEEPER-1423.2.patch against trunk revision 1437257. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 6 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 2 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/1361//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/1361//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/1361//console This message is automatically generated.
        Hide
        Edward Ribeiro added a comment -

        Excuse me, forgot two unused fields in ServerStats. Fixed.

        Show
        Edward Ribeiro added a comment - Excuse me, forgot two unused fields in ServerStats. Fixed.
        Hide
        Hadoop QA added a comment -

        +1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12566406/ZOOKEEPER-1423.3.patch
        against trunk revision 1437257.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 6 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed core unit tests.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/1362//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/1362//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/1362//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12566406/ZOOKEEPER-1423.3.patch against trunk revision 1437257. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 6 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/1362//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/1362//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/1362//console This message is automatically generated.
        Hide
        Rakesh R added a comment -

        Thanks Edward for the patch, lgtm mostly. I've few comments, please see:

        1. Format issues, please correct
          +	public long getDataDirSize();
          +        public long getLogDirSize();
          
          +
          +    public long getDataDirSize() {
          +	return provider.getDataDirSize();
          +    }
          +
          +    public long getLogDirSize() {
          +	return provider.getLogDirSize();
          +    }
          
        2. Can we print the o/p in a single line like:
          "datadir: " + dir + "size: " + dataSize
          "logdir: " + dir + "size: " + logSize
          
        3. Small correction,
          the total size of data and log (snapshot) directories' content in bytes.
          

          Actually it should be snapshot and log, can we rephrase like:

          the total size of snapshot and log files in bytes
          
        4. I feel, its good to expose the dirs path in MBean too.
        5. please provide javadocs for #getDataDirSize() and #getLogDirSize()
        Show
        Rakesh R added a comment - Thanks Edward for the patch, lgtm mostly. I've few comments, please see: Format issues, please correct + public long getDataDirSize(); + public long getLogDirSize(); + + public long getDataDirSize() { + return provider.getDataDirSize(); + } + + public long getLogDirSize() { + return provider.getLogDirSize(); + } Can we print the o/p in a single line like: "datadir: " + dir + "size: " + dataSize "logdir: " + dir + "size: " + logSize Small correction, the total size of data and log (snapshot) directories' content in bytes. Actually it should be snapshot and log, can we rephrase like: the total size of snapshot and log files in bytes I feel, its good to expose the dirs path in MBean too. please provide javadocs for #getDataDirSize() and #getLogDirSize()
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12566406/ZOOKEEPER-1423.3.patch
        against trunk revision 1582572.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 6 new or modified tests.

        -1 patch. The patch command could not apply the patch.

        Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/1992//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12566406/ZOOKEEPER-1423.3.patch against trunk revision 1582572. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 6 new or modified tests. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/1992//console This message is automatically generated.

          People

          • Assignee:
            Edward Ribeiro
            Reporter:
            Patrick Hunt
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:

              Development