HBase
  1. HBase
  2. HBASE-4633

Potential memory leak in client RPC timeout mechanism

    Details

    • Type: Bug Bug
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: 0.90.3
    • Fix Version/s: None
    • Component/s: Client
    • Labels:
      None
    • Environment:

      HBase version: 0.90.3 + Patches , Hadoop version: CDH3u0

      Description

      Relevant Jiras: https://issues.apache.org/jira/browse/HBASE-2937,
      https://issues.apache.org/jira/browse/HBASE-4003

      We have been using the 'hbase.client.operation.timeout' knob
      introduced in 2937 for quite some time now. It helps us enforce SLA.
      We have two HBase clusters and two HBase client clusters. One of them
      is much busier than the other.

      We have seen a deterministic behavior of clients running in busy
      cluster. Their (client's) memory footprint increases consistently
      after they have been up for roughly 24 hours.
      This memory footprint almost doubles from its usual value (usual case
      == RPC timeout disabled). After much investigation nothing concrete
      came out and we had to put a hack
      which keep heap size in control even when RPC timeout is enabled. Also
      note , the same behavior is not observed in 'not so busy
      cluster.

      The patch is here : https://gist.github.com/1288023

        Issue Links

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              Shrijeet Paliwal
            • Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

              • Created:
                Updated:

                Development