Uploaded image for project: 'ZooKeeper'
  1. ZooKeeper
  2. ZOOKEEPER-1560

Zookeeper client hangs on creation of large nodes

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 3.4.4, 3.5.0
    • 3.4.5, 3.5.0
    • java client
    • None
    • Reviewed

    Description

      To reproduce, try creating a node with 0.5M of data using java client. The test will hang waiting for a response from the server. See the attached patch for the test that reproduces the issue.

      It seems that ZOOKEEPER-1437 introduced a few issues to ClientCnxnSocketNIO.doIO that prevent ClientCnxnSocketNIO from sending large packets that require several invocations of SocketChannel.write to complete. The first issue is that the call to outgoingQueue.removeFirstOccurrence(p); removes the packet from the queue even if the packet wasn't completely sent yet. It looks to me that this call should be moved under if (!pbb.hasRemaining()) The second issue is that p.createBB() is reinitializing ByteBuffer on every iteration, which confuses SocketChannel.write. And the third issue is caused by extra calls to cnxn.getXid() that increment xid on every iteration and confuse the server.

      Attachments

        1. ZOOKEEPER-1560.patch
          0.9 kB
          Igor Motov
        2. zookeeper-1560-v1.txt
          2 kB
          Ted Yu
        3. zookeeper-1560-v2.txt
          3 kB
          Ted Yu
        4. zookeeper-1560-v3.txt
          3 kB
          Ted Yu
        5. zookeeper-1560-v4.txt
          4 kB
          Ted Yu
        6. zookeeper-1560-v5.txt
          4 kB
          Ted Yu
        7. zookeeper-1560-v6.txt
          5 kB
          Ted Yu
        8. zookeeper-1560-v7.txt
          5 kB
          Ted Yu
        9. ZOOKEEPER-1560-v8_r3.patch
          7 kB
          Skye Wanderman-Milne
        10. ZOOKEEPER-1560-v8_r4.patch
          8 kB
          Skye Wanderman-Milne
        11. ZOOKEEPER-1560-v8.patch
          6 kB
          Skye Wanderman-Milne

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            skye Skye Wanderman-Milne
            imotov Igor Motov
            Votes:
            0 Vote for this issue
            Watchers:
            12 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment