Accumulo
  1. Accumulo
  2. ACCUMULO-2360

Need a way to configure TNonblockingServer.maxReadBufferBytes to prevent OOMs from network misbehavour

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: 1.5.0
    • Fix Version/s: 1.5.1, 1.6.0
    • Component/s: master, tserver
    • Labels:
      None

      Description

      1.5.0 introduced GENERAL_MAX_MESSAGE_SIZE (ACCUMULO-1141), a parameter to set the maximum frame size for the TFramedTransport. However, there is an underlying frame (I think this is a glossary conflict) read in TNonblockingServer that can still cause OOM errors if erroneously connected to (telnet, netcat, etc.), creating a stack trace as such

      2014-02-12 10:26:40,439 [util.TServerUtils$THsHaServer] ERROR: run() exiting due to uncaught error
      java.lang.OutOfMemoryError: Java heap space
              at java.nio.HeapByteBuffer.<init>(HeapByteBuffer.java:57)
              at java.nio.ByteBuffer.allocate(ByteBuffer.java:329)
              at org.apache.thrift.server.AbstractNonblockingServer$FrameBuffer.read(AbstractNonblockingServer.java:338)
              at org.apache.thrift.server.AbstractNonblockingServer$AbstractSelectThread.handleRead(AbstractNonblockingServer.java:202)
              at org.apache.thrift.server.TNonblockingServer$SelectAcceptThread.select(TNonblockingServer.java:198)
              at org.apache.thrift.server.TNonblockingServer$SelectAcceptThread.run(TNonblockingServer.java:154)

      I believe if we set maxReadBufferBytes to the server arguments, it will filter appropriately. The only decision I'm not sure about is if we should recycle the max message property or have a separate one.

        Issue Links

          Activity

          Hide
          ASF subversion and git services added a comment -

          Commit 1b41177b04428971bceb68774624ad4271e541c1 in branch refs/heads/master from John Vines
          [ https://git-wip-us.apache.org/repos/asf?p=accumulo.git;h=1b41177 ]

          ACCUMULO-2360 adding TNonblockingServer.maxReadBufferBytes setting

          Show
          ASF subversion and git services added a comment - Commit 1b41177b04428971bceb68774624ad4271e541c1 in branch refs/heads/master from John Vines [ https://git-wip-us.apache.org/repos/asf?p=accumulo.git;h=1b41177 ] ACCUMULO-2360 adding TNonblockingServer.maxReadBufferBytes setting
          Hide
          ASF subversion and git services added a comment -

          Commit 1b41177b04428971bceb68774624ad4271e541c1 in branch refs/heads/1.6.0-SNAPSHOT from John Vines
          [ https://git-wip-us.apache.org/repos/asf?p=accumulo.git;h=1b41177 ]

          ACCUMULO-2360 adding TNonblockingServer.maxReadBufferBytes setting

          Show
          ASF subversion and git services added a comment - Commit 1b41177b04428971bceb68774624ad4271e541c1 in branch refs/heads/1.6.0-SNAPSHOT from John Vines [ https://git-wip-us.apache.org/repos/asf?p=accumulo.git;h=1b41177 ] ACCUMULO-2360 adding TNonblockingServer.maxReadBufferBytes setting
          Hide
          ASF subversion and git services added a comment -

          Commit 1b41177b04428971bceb68774624ad4271e541c1 in branch refs/heads/1.5.1-SNAPSHOT from John Vines
          [ https://git-wip-us.apache.org/repos/asf?p=accumulo.git;h=1b41177 ]

          ACCUMULO-2360 adding TNonblockingServer.maxReadBufferBytes setting

          Show
          ASF subversion and git services added a comment - Commit 1b41177b04428971bceb68774624ad4271e541c1 in branch refs/heads/1.5.1-SNAPSHOT from John Vines [ https://git-wip-us.apache.org/repos/asf?p=accumulo.git;h=1b41177 ] ACCUMULO-2360 adding TNonblockingServer.maxReadBufferBytes setting
          Hide
          John Vines added a comment -

          I think reusing the same property makes the most sense here, since I think this property was the original hope of that property. We can revisit that property if we decide it doesn't make sense as at the transport frame level (as opposed to server socket frame).

          Show
          John Vines added a comment - I think reusing the same property makes the most sense here, since I think this property was the original hope of that property. We can revisit that property if we decide it doesn't make sense as at the transport frame level (as opposed to server socket frame).

            People

            • Assignee:
              John Vines
              Reporter:
              John Vines
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development