Uploaded image for project: 'TinkerPop'
  1. TinkerPop
  2. TINKERPOP-817

Gryo serialization of large responses fails and causes protocol desync

    Details

      Description

      Serialization of large responses in Gryo is not done correctly. It looks like there is some sort of framing or fragmentation issue. I haven't tracked it down the the root cause. It can be reproduced by sending the query "x" * 4066 whereas "x" * 4065 works.

      With debug logging turned on, the server indicates that it is writing the correct response message but when it is serialized it comes out only as the single byte 0x01.

      The error on the client is, in this case,

      io.netty.handler.codec.DecoderException: org.apache.tinkerpop.gremlin.driver.ser.SerializationException: org.apache.tinkerpop.shaded.kryo.KryoException: Buffer too small: capacity: 1, required: 8

      but different response sizes give different errors, for example, "x" * 10000 gives

      io.netty.handler.codec.DecoderException: org.apache.tinkerpop.gremlin.driver.ser.SerializationException: java.lang.IndexOutOfBoundsException: Index: 118, Size: 0

        Issue Links

          Activity

          Hide
          spmallette stephen mallette added a comment -

          Provided a bufferSize configuration option to the gryo serializer. The kryo buffer size was defaulted to 4096 and would those throw that "Buffer too small" exception. So....if you have large individual results you can bump this value to resolve this problem.

          https://github.com/apache/incubator-tinkerpop/commit/6a333d7f7c0e3d5e9c800cd4ae71a46771b8c296

          Show
          spmallette stephen mallette added a comment - Provided a bufferSize configuration option to the gryo serializer. The kryo buffer size was defaulted to 4096 and would those throw that "Buffer too small" exception. So....if you have large individual results you can bump this value to resolve this problem. https://github.com/apache/incubator-tinkerpop/commit/6a333d7f7c0e3d5e9c800cd4ae71a46771b8c296
          Hide
          spmallette stephen mallette added a comment -

          One of these days I'm going to learn when it is important to flush() stuff. The inconsistency between errors has been resolved here such that you should always get a "Buffer too small" KryoException....

          https://github.com/apache/incubator-tinkerpop/commit/af2ffa95fed6bf7f217bcd95eb88e9b4494dcd76

          Nice bug find - thanks.

          Show
          spmallette stephen mallette added a comment - One of these days I'm going to learn when it is important to flush() stuff. The inconsistency between errors has been resolved here such that you should always get a "Buffer too small" KryoException .... https://github.com/apache/incubator-tinkerpop/commit/af2ffa95fed6bf7f217bcd95eb88e9b4494dcd76 Nice bug find - thanks.

            People

            • Assignee:
              spmallette stephen mallette
              Reporter:
              gereedy Geoff Reedy
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development