Hadoop Common
  1. Hadoop Common
  2. HADOOP-4802

RPC Server send buffer retains size of largest response ever sent

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Duplicate
    • Affects Version/s: 0.18.2, 0.19.0
    • Fix Version/s: 0.20.2
    • Component/s: ipc
    • Labels:
      None

      Description

      The stack-based ByteArrayOutputStream in Server.Hander is reset each time through the run loop. This will set the BAOS 'size' back to zero but the allocated backing buffer is unaltered. If during an Handlers' lifecycle, any particular RPC response was fat – Megabytes, even – the buffer expands during the write to accommodate the particular response but then never shrinks subsequently. If a hosting Server has had more than one 'fat payload' occurrence, the resultant occupied heap can provoke memory woes (See https://issues.apache.org/jira/browse/HBASE-900?focusedCommentId=12654009#action_12654009 for an extreme example; occasional payloads of 20-50MB with 30 handlers robbed the heap of 700MB).

        Issue Links

          Activity

          Tom White made changes -
          Fix Version/s 0.21.0 [ 12313563 ]
          Fix Version/s 0.22.0 [ 12314296 ]
          Suresh Srinivas made changes -
          Status Open [ 1 ] Resolved [ 5 ]
          Fix Version/s 0.20.2 [ 12314203 ]
          Fix Version/s 0.21.0 [ 12313563 ]
          Fix Version/s 0.22.0 [ 12314296 ]
          Resolution Duplicate [ 3 ]
          Hairong Kuang made changes -
          Link This issue duplicates HADOOP-6460 [ HADOOP-6460 ]
          stack made changes -
          Attachment 4802.patch [ 12395514 ]
          stack made changes -
          Attachment 4802-v4-TRUNK.patch [ 12395688 ]
          stack made changes -
          Attachment 4802-v3.patch [ 12395684 ]
          stack made changes -
          Attachment 4802-v2.patch [ 12395676 ]
          stack made changes -
          Attachment 4802-v4-TRUNK.patch [ 12395688 ]
          Raghu Angadi made changes -
          Link This issue is related to HADOOP-4813 [ HADOOP-4813 ]
          stack made changes -
          Attachment 4802-v3.patch [ 12395684 ]
          stack made changes -
          Attachment 4802-v2.patch [ 12395676 ]
          stack made changes -
          Field Original Value New Value
          Attachment 4802.patch [ 12395514 ]
          stack created issue -

            People

            • Assignee:
              Unassigned
              Reporter:
              stack
            • Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development