Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-11339

Reuse buffer for Hadoop RPC

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: ipc, performance
    • Labels:
      None

      Description

      For Hadoop RPCs, we will try to reuse the available connections.
      But when we process each rpc in the same connection, we will allocate a fresh heap byte buffer to store the rpc bytes data. The rpc message may be very large, i.e., datanode blocks report.

      There is chance to cause full gc as discussed in HDFS-7435

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                hitliuyi Yi Liu
                Reporter:
                hitliuyi Yi Liu
              • Votes:
                0 Vote for this issue
                Watchers:
                7 Start watching this issue

                Dates

                • Created:
                  Updated: