ZooKeeper
  1. ZooKeeper
  2. ZOOKEEPER-1314

improve zkpython synchronous api implementation

    Details

    • Type: Improvement Improvement
    • Status: Open
    • Priority: Minor Minor
    • Resolution: Unresolved
    • Affects Version/s: 3.3.3
    • Fix Version/s: None
    • Component/s: contrib-bindings
    • Labels:
      None
    • Release Note:
      Hide
      Improves zkpython synchronous api; release GIL before synchronous calls, and do not limit returned znode name to 256 bytes for synchronous create call.
      Show
      Improves zkpython synchronous api; release GIL before synchronous calls, and do not limit returned znode name to 256 bytes for synchronous create call.

      Description

      Improves the following items in zkpython which are related to the Zookeeper synchronous API:

      1. For pyzoo_create, no longer limit the returned znode name to 256 bytes; dynamically allocate memory on the heap.
      2. For all the synchronous api calls, release the Python Global Interpreter Lock just before doing the synchronous call.

      I will attach the patch shortly.

      1. ZOOKEEPER-1314.patch
        6 kB
        Daniel Lescohier
      2. ZOOKEEPER-1314.patch
        8 kB
        Daniel Lescohier

        Activity

        Hide
        Thawan Kooburat added a comment -

        Henry Robinson Do you think this patch is good to go? I can help commit the patch

        Show
        Thawan Kooburat added a comment - Henry Robinson Do you think this patch is good to go? I can help commit the patch
        Hide
        Daniel Lescohier added a comment -

        Update to patch to resolve comment. Patch based on 3.3.6.

        Note: this should also resolve ZOOKEEPER-1023.

        Show
        Daniel Lescohier added a comment - Update to patch to resolve comment. Patch based on 3.3.6. Note: this should also resolve ZOOKEEPER-1023 .
        Hide
        Patrick Hunt added a comment -

        cancelling patch until Henry's comments are addressed.

        Show
        Patrick Hunt added a comment - cancelling patch until Henry's comments are addressed.
        Hide
        Henry Robinson added a comment -

        Hi Daniel -

        Looks good, I didn't know about Py_BEGIN_ALLOW_THREADS!

        Just one comment:

           if (err != ZOK) {
             PyErr_SetString(err_to_exception(err), zerror(err));
        -    return NULL;
        +    goto cleanup;
           }
         
        -  return Py_BuildValue("s", realbuf);
        +  returnval = Py_BuildValue("s", realbuf);
        +cleanup:
        +  free(realbuf);
        +  return returnval;
        

        I'd prefer:

           if (err != ZOK) {
             PyErr_SetString(err_to_exception(err), zerror(err));
           } else {
             returnval = Py_BuildValue("s", realbuf);
           }
          
        +  free(realbuf);
        +  return returnval;
        

        if that's functionally equivalent.

        Show
        Henry Robinson added a comment - Hi Daniel - Looks good, I didn't know about Py_BEGIN_ALLOW_THREADS! Just one comment: if (err != ZOK) { PyErr_SetString(err_to_exception(err), zerror(err)); - return NULL; + goto cleanup; } - return Py_BuildValue( "s" , realbuf); + returnval = Py_BuildValue( "s" , realbuf); +cleanup: + free(realbuf); + return returnval; I'd prefer: if (err != ZOK) { PyErr_SetString(err_to_exception(err), zerror(err)); } else { returnval = Py_BuildValue( "s" , realbuf); } + free(realbuf); + return returnval; if that's functionally equivalent.
        Hide
        Mahadev konar added a comment -

        @Henry,
        Can you please review this patch?

        Show
        Mahadev konar added a comment - @Henry, Can you please review this patch?
        Hide
        Hadoop QA added a comment -

        +1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12505816/ZOOKEEPER-1314.patch
        against trunk revision 1208979.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 3 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed core unit tests.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/802//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/802//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/802//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12505816/ZOOKEEPER-1314.patch against trunk revision 1208979. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/802//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/802//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/802//console This message is automatically generated.
        Hide
        Hadoop QA added a comment -

        +1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12505816/ZOOKEEPER-1314.patch
        against trunk revision 1208979.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 3 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed core unit tests.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/801//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/801//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/801//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12505816/ZOOKEEPER-1314.patch against trunk revision 1208979. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/801//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/801//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/801//console This message is automatically generated.
        Hide
        Daniel Lescohier added a comment -

        Notes on the implementation:

        1. int err is declared earlier in the function, because Py_BEGIN_ALLOW_THREADS opens a block, and we need access to the err variable outside that block.
        1. The zookeeper handle is copied to a local variable in the function so that we're not using a global variable while the GIL is released.
        Show
        Daniel Lescohier added a comment - Notes on the implementation: int err is declared earlier in the function, because Py_BEGIN_ALLOW_THREADS opens a block, and we need access to the err variable outside that block. The zookeeper handle is copied to a local variable in the function so that we're not using a global variable while the GIL is released.

          People

          • Assignee:
            Daniel Lescohier
            Reporter:
            Daniel Lescohier
          • Votes:
            1 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:

              Time Tracking

              Estimated:
              Original Estimate - 0.5h
              0.5h
              Remaining:
              Remaining Estimate - 0.5h
              0.5h
              Logged:
              Time Spent - Not Specified
              Not Specified

                Development