Uploaded image for project: 'ZooKeeper'
  1. ZooKeeper
  2. ZOOKEEPER-4052

Failed to read large znode that is written successfully

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 3.6.1, 3.6.2
    • Fix Version/s: None
    • Component/s: java client
    • Labels:
      None

      Description

      Description

      A client would fail to read a large znode that is successfully written. The reason is the lengths of the extra fields for CreateRequest and GetDataResponse are different: GetDataResponse's extra fields have slightly more length because of the stat field. The stat field occupies more bytes in a packet.

      How to Reproduce

      On my MacBook, the length of incoming buffer adds 84 more bytes on the original bytes data. So I try to write (1024 * 1024 - 1 - 84) bytes to ZK which is successful. And when I read it, it fails:

      Closing socket connection. Attempting reconnect except it is a SessionExpiredException.Closing socket connection. Attempting reconnect except it is a SessionExpiredException.java.io.IOException: Packet len 1048606 is out of range! at org.apache.zookeeper.ClientCnxnSocket.readLength(ClientCnxnSocket.java:121) at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:84) at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1275)2021-01-08 01:53:36,931 [main-EventThread] INFO  org.apache.helix.zookeeper.zkclient.ZkClient - zkclient 0, zookeeper state changed ( Disconnected )
      

      Solution

      We can add extra buffer size on both the client and the server for the sanity check based on the jute.maxbuffer, just like what has been done in BinaryInputArchive.java

      // Since this is a rough sanity check, add some padding to maxBuffer to
      // make up for extra fields, etc. (otherwise e.g. clients may be able to
      // write buffers larger than we can read from disk!)
      private void checkLength(int len) throws IOException {
          if (len < 0 || len > maxBufferSize + extraMaxBufferSize) {
              throw new IOException(UNREASONBLE_LENGTH + len);
          }
      }
      

       

        Attachments

          Activity

            People

            • Assignee:
              hzlu Huizhi Lu
              Reporter:
              hzlu Huizhi Lu
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated: