Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-4633

Potential memory leak in client RPC timeout mechanism

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Cannot Reproduce
    • Affects Version/s: 0.90.3
    • Fix Version/s: None
    • Component/s: Client
    • Labels:
      None
    • Environment:

      HBase version: 0.90.3 + Patches , Hadoop version: CDH3u0

      Description

      Relevant Jiras: https://issues.apache.org/jira/browse/HBASE-2937,
      https://issues.apache.org/jira/browse/HBASE-4003

      We have been using the 'hbase.client.operation.timeout' knob
      introduced in 2937 for quite some time now. It helps us enforce SLA.
      We have two HBase clusters and two HBase client clusters. One of them
      is much busier than the other.

      We have seen a deterministic behavior of clients running in busy
      cluster. Their (client's) memory footprint increases consistently
      after they have been up for roughly 24 hours.
      This memory footprint almost doubles from its usual value (usual case
      == RPC timeout disabled). After much investigation nothing concrete
      came out and we had to put a hack
      which keep heap size in control even when RPC timeout is enabled. Also
      note , the same behavior is not observed in 'not so busy
      cluster.

      The patch is here : https://gist.github.com/1288023

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                shrijeet Shrijeet Paliwal
              • Votes:
                0 Vote for this issue
                Watchers:
                11 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: