ZooKeeper
  1. ZooKeeper
  2. ZOOKEEPER-1804

Stat the realtime tps of zookeepr server

    Details

    • Type: Improvement Improvement
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: 3.6.0
    • Component/s: server
    • Labels:
      None

      Description

      At this time, we assessed whether zookeeper supports some business scenarios, always use the number of subscribers, or to assess the number of clients。

      You konw, some times, many client connection with zookeeper, but do noting, and the onthers do complex business logic。

      So,we must stat the realtime tps of zookeepr。

      [-----------------Solution-------------------]

      Solution1:

      If you only want to know the real time transaction processed, you can use the patch "ZOOKEEPER-1804.patch".

      Solution2:

      If you also want to know how client use zookeeper, and the real time r/w ps of each zookeeper client, you can use the patch "ZOOKEEPER-1804-2.patch"
      use java properties: -Dserver_process_stats=true to open the function.

      Sample:
      $>echo rwps|nc localhost 2181
      RealTime R/W Statistics:
      getChildren2:   0.5994005994005994
      createSession:  1.6983016983016983
      closeSession:   0.999000999000999
      setData:     110.18981018981019
      setWatches:   129.17082917082917
      getChildren:    68.83116883116884
      delete:      19.980019980019982
      create:      22.27772227772228
      exists:      1806.2937062937062
      getDate:     729.5704295704296

      1. ZOOKEEPER-1804.patch
        2 kB
        Raul Gutierrez Segales
      2. ZOOKEEPER-1804.patch
        11 kB
        Leader Ni

        Activity

        Leader Ni created issue -
        Hide
        Raul Gutierrez Segales added a comment -

        we've been using this patch, so I guess something along this lines could work.

        Show
        Raul Gutierrez Segales added a comment - we've been using this patch, so I guess something along this lines could work.
        Raul Gutierrez Segales made changes -
        Field Original Value New Value
        Attachment ZOOKEEPER-1804.patch [ 12610864 ]
        Raul Gutierrez Segales made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12610864/ZOOKEEPER-1804.patch
        against trunk revision 1535491.

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed core unit tests.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/1727//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/1727//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/1727//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12610864/ZOOKEEPER-1804.patch against trunk revision 1535491. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/1727//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/1727//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/1727//console This message is automatically generated.
        Hide
        Leader Ni added a comment -

        patch2 sample

        $>echo rwps|nc localhost 2181
        RealTime R/W Statistics:
        getChildren2: 0.5994005994005994
        createSession: 1.6983016983016983
        closeSession: 0.999000999000999
        setData: 110.18981018981019
        setWatches: 129.17082917082917
        getChildren: 68.83116883116884
        delete: 19.980019980019982
        create: 22.27772227772228
        exists: 1806.2937062937062
        getDate: 729.5704295704296

        Show
        Leader Ni added a comment - patch2 sample $>echo rwps|nc localhost 2181 RealTime R/W Statistics: getChildren2: 0.5994005994005994 createSession: 1.6983016983016983 closeSession: 0.999000999000999 setData: 110.18981018981019 setWatches: 129.17082917082917 getChildren: 68.83116883116884 delete: 19.980019980019982 create: 22.27772227772228 exists: 1806.2937062937062 getDate: 729.5704295704296
        Leader Ni made changes -
        Description At this time, we assessed whether zookeeper supports some business scenarios, always use the number of subscribers, or to assess the number of clients。

        You konw, some times, many client connection with zookeeper, but do noting, and the onthers do complex business logic。

        So,we must stat the realtime tps of zookeepr。
        At this time, we assessed whether zookeeper supports some business scenarios, always use the number of subscribers, or to assess the number of clients。

        You konw, some times, many client connection with zookeeper, but do noting, and the onthers do complex business logic。

        So,we must stat the realtime tps of zookeepr。


        [-----------------Solution-------------------]

        Solution1:

        If you only want to know the real time transaction processed, you can use the patch "ZOOKEEPER-1804.patch".

        Solution2:

        If you also want to know how client use zookeeper, and the real time r/w ps of each zookeeper client, you can use the patch "ZOOKEEPER-1804-2.patch"

        Sample:
        $>echo rwps|nc localhost 2181
        RealTime R/W Statistics:
                getChildren2:   0.5994005994005994
                createSession:  1.6983016983016983
                closeSession:   0.999000999000999
                setData:     110.18981018981019
                setWatches:   129.17082917082917
                getChildren:    68.83116883116884
                delete:      19.980019980019982
                create:      22.27772227772228
                exists:      1806.2937062937062
                getDate:     729.5704295704296
        Leader Ni made changes -
        Description At this time, we assessed whether zookeeper supports some business scenarios, always use the number of subscribers, or to assess the number of clients。

        You konw, some times, many client connection with zookeeper, but do noting, and the onthers do complex business logic。

        So,we must stat the realtime tps of zookeepr。


        [-----------------Solution-------------------]

        Solution1:

        If you only want to know the real time transaction processed, you can use the patch "ZOOKEEPER-1804.patch".

        Solution2:

        If you also want to know how client use zookeeper, and the real time r/w ps of each zookeeper client, you can use the patch "ZOOKEEPER-1804-2.patch"

        Sample:
        $>echo rwps|nc localhost 2181
        RealTime R/W Statistics:
                getChildren2:   0.5994005994005994
                createSession:  1.6983016983016983
                closeSession:   0.999000999000999
                setData:     110.18981018981019
                setWatches:   129.17082917082917
                getChildren:    68.83116883116884
                delete:      19.980019980019982
                create:      22.27772227772228
                exists:      1806.2937062937062
                getDate:     729.5704295704296
        At this time, we assessed whether zookeeper supports some business scenarios, always use the number of subscribers, or to assess the number of clients。

        You konw, some times, many client connection with zookeeper, but do noting, and the onthers do complex business logic。

        So,we must stat the realtime tps of zookeepr。


        [-----------------Solution-------------------]

        Solution1:

        If you only want to know the real time transaction processed, you can use the patch "ZOOKEEPER-1804.patch".

        Solution2:

        If you also want to know how client use zookeeper, and the real time r/w ps of each zookeeper client, you can use the patch "ZOOKEEPER-1804-2.patch"
        use java properties: -Dserver_process_stats=true to open the function.

        Sample:
        $>echo rwps|nc localhost 2181
        RealTime R/W Statistics:
                getChildren2:   0.5994005994005994
                createSession:  1.6983016983016983
                closeSession:   0.999000999000999
                setData:     110.18981018981019
                setWatches:   129.17082917082917
                getChildren:    68.83116883116884
                delete:      19.980019980019982
                create:      22.27772227772228
                exists:      1806.2937062937062
                getDate:     729.5704295704296
        Leader Ni made changes -
        Attachment ZOOKEEPER-1804-2.patch [ 12611868 ]
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12611868/ZOOKEEPER-1804-2.patch
        against trunk revision 1535491.

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        -1 patch. The patch command could not apply the patch.

        Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/1738//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12611868/ZOOKEEPER-1804-2.patch against trunk revision 1535491. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/1738//console This message is automatically generated.
        Hide
        Patrick Hunt added a comment -

        Hi Leader Ni, in order for the patchbot to do it's work you'll need to attach a patch generated with the "--no-prefix" option in git, see the guide:
        https://cwiki.apache.org/confluence/display/ZOOKEEPER/HowToContribute

        Show
        Patrick Hunt added a comment - Hi Leader Ni , in order for the patchbot to do it's work you'll need to attach a patch generated with the "--no-prefix" option in git, see the guide: https://cwiki.apache.org/confluence/display/ZOOKEEPER/HowToContribute
        Leader Ni made changes -
        Attachment ZOOKEEPER-1804-2.patch [ 12611868 ]
        Hide
        Leader Ni added a comment -

        patch for trunk

        Show
        Leader Ni added a comment - patch for trunk
        Leader Ni made changes -
        Attachment ZOOKEEPER-1804.patch [ 12612555 ]
        Leader Ni made changes -
        Fix Version/s 3.5.0 [ 12316644 ]
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12612555/ZOOKEEPER-1804.patch
        against trunk revision 1539529.

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        -1 findbugs. The patch appears to introduce 4 new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed core unit tests.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/1746//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/1746//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/1746//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12612555/ZOOKEEPER-1804.patch against trunk revision 1539529. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 4 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/1746//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/1746//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/1746//console This message is automatically generated.
        Hide
        Raul Gutierrez Segales added a comment -

        Small styling nits, in things like:

        +			if ( zkServer == null ) {
        +				pw.println( ZK_NOT_SERVING );
        

        the spaces after ( and before ) aren't used in the rest of the code.

        Also - at the cost of introducing another dependency though - you might want to check Twitter's stats package which has convenience classes/methods for keeping stats (also useful for the case of write/read latency to keep p99, etc):

        http://twitter.github.io/commons/apidocs/#com.twitter.common.stats.Stats

        We use this in an internal branch atm.

        Show
        Raul Gutierrez Segales added a comment - Small styling nits, in things like: + if ( zkServer == null ) { + pw.println( ZK_NOT_SERVING ); the spaces after ( and before ) aren't used in the rest of the code. Also - at the cost of introducing another dependency though - you might want to check Twitter's stats package which has convenience classes/methods for keeping stats (also useful for the case of write/read latency to keep p99, etc): http://twitter.github.io/commons/apidocs/#com.twitter.common.stats.Stats We use this in an internal branch atm.
        Hide
        Patrick Hunt added a comment -

        I'd suggest Yammer metrics if we do go in that direction. I'd rather not see more dependencies but...

        Show
        Patrick Hunt added a comment - I'd suggest Yammer metrics if we do go in that direction. I'd rather not see more dependencies but...
        Hide
        Michi Mutsuzaki added a comment -

        I'm canceling the patch. Let's decide if we want to use an existing package like what Raul and Pat mentioned to keep track of statistics.

        Show
        Michi Mutsuzaki added a comment - I'm canceling the patch. Let's decide if we want to use an existing package like what Raul and Pat mentioned to keep track of statistics.
        Michi Mutsuzaki made changes -
        Status Patch Available [ 10002 ] Open [ 1 ]
        Michi Mutsuzaki made changes -
        Fix Version/s 3.6.0 [ 12326518 ]
        Fix Version/s 3.5.0 [ 12316644 ]
        Hide
        Flavio Junqueira added a comment -

        It'd be great to have agreement here, sounds like a good feature to have.

        Show
        Flavio Junqueira added a comment - It'd be great to have agreement here, sounds like a good feature to have.

          People

          • Assignee:
            Leader Ni
            Reporter:
            Leader Ni
          • Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

            • Due:
              Created:
              Updated:

              Development